How many concepts is too many

The cookbook and the first recipe#

Think of a cookbook that promises “just four techniques”. That’s the author’s count. A beginner’s count is different: open the first recipe and tally what you actually have to handle. The ingredients you have to buy. The steps where the recipe says “now you do this”. The amounts you have to set. The labels on jars you have to read correctly. And the rules nobody writes down, like “preheat the oven”, which you only learn when the cake doesn’t rise.

Frameworks are the same. Here’s LiveKit Agents, a Python framework for building voice agents, across two releases. Its README lists its core concepts, and its smallest complete example uses some number of the framework’s names:

The list grew from Agent, AgentSession and entrypoint to include AgentServer. The example shrank from 20 names to 17. The list describes how the framework is organised; the example is closer to what a newcomer has to hold in their head to get one thing working. They measure different things, so they can move in opposite directions, and an argument about “how many concepts” goes nowhere until you say which one you mean.

Counting what a user writes#

A count worth arguing about needs a rule you can rerun. Mine is a small script built on Python’s ast module, which parses source code into a tree the script can walk. It sorts what an example writes into four columns, the recipe’s four kinds of thing:

Two more columns come later, for frameworks that read things by name: settings, and import paths. The output always shows all six, and the file’s line count.

Each column is a different kind of thing to learn, so they aren’t added up. If you need one number to quote, quote names, because every example has some; but compare the columns side by side. And names are only a proxy for effort: app.post adds little once you’ve met app.get, while FastAPI’s Depends, its way of declaring what a handler needs, brings a whole idea with it.

The core of the script is one walk over the tree:

1
for node in ast.walk(tree):
2
    if isinstance(node, ast.ImportFrom) and from_framework(node.module):
3
        result.imports.update(a.name for a in node.names)
4
    elif isinstance(node, ast.Import):
5
        result.imports.update(a.asname or a.name for a in node.names if from_framework(a.name))
6
    elif isinstance(node, ast.Attribute) and is_fw(node.value):
7
        if isinstance(node.value, ast.Attribute) and node.value.attr in string_namespaces:
8
            result.strings.add(node.attr)  # e.g. pytest.mark.<anything>: a name the project makes up
9
        else:
10
            result.members.add(node.attr)
11
    ...  # hooks, options and strings from calls; import paths from every string

Abridged: hooks come from function and class definitions further down, and options and strings from calls.

ast.walk visits every node in the parsed file. An import or from ... import of the framework (from_framework checks the module name) adds to the names it imports. An attribute read on a framework object, like app.get, adds a member name. A variable counts as a framework object if it came from a framework import, was assigned from one, is annotated with a framework type, or is a parameter of a hook found by its prefix, which is the test is_fw applies. The one exception is a namespace like pytest.mark, which accepts any name you make up: that’s a string you have to spell right, not a name you learn.

Here’s what it makes of the example in FastAPI’s README, an app with two routes:

main.py: 2 names (1 imported, 1 members), 2 hooks, 0 options, 2 strings, 0 settings, 0 import paths, 8 lines
  imports: FastAPI
  members: get
  hooks: read_item, read_root
  options: -
  strings: /, /items/{item_id}
  settings: -
  import_paths: -

Two names, two hooks and two strings: a fair count of what the file says. But a newcomer needs four more things to write it, and none of them is in the file:

1. FastAPI matches the route’s {item_id} to the path parameter of the same name.
2. It converts and validates the value using the int annotation, and answers bad input with an HTTP 422 error.
3. It treats any other parameter of a simple type as a query parameter, from the URL’s ?q=.... A parameter typed as a Pydantic model becomes the request body instead.
4. Its command-line tool looks for a file called main.py (or app.py, api.py) and picks up the FastAPI app in it.

These are the recipe’s “preheat the oven”: implicit rules. Not a name, not a hook, not an option, not a string, and this counter can’t see them.

Where a concept lives decides when a mistake shows up#

Implicit rules are one of several places a concept can live outside the names a counter sees. Settings the framework reads by name are another, and so are strings it looks up later. Each place fails differently when you get it wrong, and the quickest way to find out how is to misspell one concept at a time and see when the framework notices. In the recipe, it’s the difference between a misspelled ingredient the shop can’t find and a mislabelled jar you only notice when you taste the cake.

A second script, typo_checks.py, does that against Django 6.1.1, pytest 9.1.1 and FastAPI 0.142.1, in fresh projects (the Django ones generated by Django’s own startproject):

Django, a typo inside a middleware path:
    manage.py check: System check identified no issues (0 silenced).
    loading the app, as the server does: ImportError: Module "django.middleware.common" does not define a "CommonMiddlewar" attribute/class
Django, a typo in a setting's name:
    manage.py check: System check identified no issues (0 silenced).
    TIME_ZONE = America/Chicago
pytest, a typo in a hook's name:
    pluggy._manager.PluginValidationError: unknown hook 'pytest_colection_modifyitems'; no tests ran in 0.00s
pytest, a typo in the hook prefix:
    1 passed in 0.00s
pytest, a typo in a marker's name:
    PytestUnknownMarkWarning: Unknown pytest.mark.slwo; 1 passed, 1 warning in 0.00s
FastAPI, a parameter name that doesn't match the path:
    app built; GET /items/5 -> 422 missing ['query', 'itemid']

The last line joins two of FastAPI’s implicit rules from above. The route says {item_id} but the function says itemid, so the path parameter goes unmatched, and itemid is treated as a required query parameter instead, which the request didn’t send: missing ['query', 'itemid'].

Here are all of them sorted from caught earliest to never caught, plus two LiveKit cases. Those two aren’t in the script, since they’d need LiveKit’s service; they follow from Python’s import rules, the argument’s type and the client’s source, as the LiveKit section below explains:

settings.py: 0 names (0 imported, 0 members), 0 hooks, 0 options, 0 strings, 17 settings, 24 import paths, 76 lines

1
# livekit-agents 1.0.13
2
from livekit.plugins import deepgram, openai, silero
3
...
4
stt=deepgram.STT(model="nova-3"),
5

6
# livekit-agents 1.8.3
7
stt=inference.STT("deepgram/nova-3", language="multi"),

Which reader pays#

The same framework can ask very different things of different people. pytest’s documentation has an example that adds a --runslow option and skips tests marked slow unless you pass it. A test author writes this:

1
import pytest
2

3

4
def test_func_fast():
5
    pass
6

7

8
@pytest.mark.slow
9
def test_func_slow():
10
    pass

The person extending pytest writes the conftest.py that makes the marker mean something:

1
import pytest
2

3

4
def pytest_addoption(parser):
5
    parser.addoption(
6
        "--runslow", action="store_true", default=False, help="run slow tests"
7
    )
8

9

10
def pytest_configure(config):
11
    config.addinivalue_line("markers", "slow: mark test as slow to run")
12

13

14
def pytest_collection_modifyitems(config, items):
15
    if config.getoption("--runslow"):
16
        # --runslow given in cli: do not skip slow tests
17
        return
18
    skip_slow = pytest.mark.skip(reason="need --runslow option to run")
19
    for item in items:
20
        if "slow" in item.keywords:
21
            item.add_marker(skip_slow)

The test author writes two test functions and a marker. The plugin author writes three hooks pytest calls, pytest_addoption, pytest_configure and pytest_collection_modifyitems, and uses the objects pytest hands them:

--hook-prefix pytest_ tells the counter that pytest calls functions with that prefix, so they’re hooks, and the arguments pytest passes them, parser, config and items, are framework objects. Without either flag, the script finds no hooks in conftest.py and only 3 names, pytest, mark and skip. In the cookbook’s terms, the test author is a cook following a recipe, and the plugin author is writing a new chapter of the book.

One number for “pytest” would mix those two jobs. So compare the same reader’s smallest real task in each framework: an app author’s quickstart with an app author’s quickstart, a plugin with a plugin.

Now you’ve met every flag, try the counter on each example. Start with the pytest conftest and switch between No flags and With the framework’s flags, then compare the two LiveKit releases one after the other:

And misspell things yourself. Try the Django setting’s name first, and see whether anything complains:

When more concepts are worth it#

A lower count isn’t automatically better. Extra concepts earn their place in two ways.

The first is an earlier check. Every name in LiveKit’s 1.0.13 example is something Python checks when the module loads (the imports) or when the line runs (the attributes), and a type checker catches all of them before anything runs. A count that rewards moving concepts into strings and settings rewards moving mistakes later.

The second is a seam you can see: a place to put your own code, with its position visible. Django’s MIDDLEWARE list is one. Each entry wraps every request. To add your own, you write one middleware and put its path in the list where you want it to run, and the list is the order: Django “applies middleware in the order it’s defined in MIDDLEWARE, top-down”, as its middleware docs put it. You can put a step anywhere, reorder steps, and read the whole order in one place.

LiveKit’s llm_node is the hook version: an agent overrides it to change how the language model is called. It’s cheaper, one hook and no settings, but you can’t add or move a step, and where it runs is in LiveKit’s docs, not your code. Both are seams; the list is the one you can see. And the list shows up in the settings and import-path columns and the hook in the hooks column, so a names-only count sees neither. Compare frameworks across all the columns, or the one whose pipeline is hidden will always look smaller.

Count your own quickstart#

So, is ten too many? Back to the cookbook: it depends which of the ten end up in the beginner’s first recipe, and which only the person writing a new chapter ever meets. A type that only people writing new steps see costs an app author nothing. Ten names in the first recipe would be a lot; ten types spread across columns and readers might not be. To find out for your framework, run the script on its smallest real example, adding --hook-prefix, --settings or --string-namespace wherever the framework finds things by name. Then:

1. Find the implicit rules. They aren’t a column. Read the docs for everything that happens “automatically”, then misspell each one and note when it’s caught: before it runs, when it loads, when it’s used, only with a warning, or never.
2. Fix the silent ones first. A rule that fails silently, like a misspelled setting name, needs a check or a reserved prefix more than it needs to be counted down.
3. Weigh each extra concept. Keep one that moves a check earlier or gives users a seam they can see; question one that does neither.
4. Split by reader. Count the app author’s quickstart and the extension author’s smallest plugin separately. If you’re choosing between frameworks, compare the same reader’s task in each, across all the columns.

The demo#

The scripts are in concept-count/. The counter needs only Python 3.9 or later; the typo checks need Python 3.12 or later, and install the pinned Django, pytest and FastAPI.

cd mainlog-code/concept-count
python3 count_concepts.py yourframework quickstart.py   # your own example
python3 run_comparison.py --detail                      # the six examples above, every counted item
python3 -m pip install -r requirements.txt
python3 typo_checks.py                                  # misspell one concept at a time

run_comparison.py downloads each example from GitHub at a pinned commit, so the numbers here don’t move when a README changes. The README lists the counter’s limits: it tracks framework objects by syntax, not by running the code, so objects that reach an example another way are missed.