What every team should know about LLMs

Teams skip the fundamentals because they look like trivia. They aren't. Almost every expensive mistake I'm called in to unpick — the 400-line CLAUDE.md, the twelve MCP servers nobody audits, "why did it forget what I just told it?", the bill that surprised finance — traces back to one of four things people were never told about how these models actually work.

Four mental models. Each one prevents a specific, recurring mistake. None of them requires maths.

1 · The model is frozen — everything else is you

Training and inference are two different machines wearing the same name. Training runs for months: millions of examples in parallel, each pass comparing output to expectation and nudging the numbers. Inference is what you use: one token at a time, sequentially, each new token re-reading everything before it. At the end of training the parameters are frozen. They do not move again.

The exercise I use to make that concrete is a handwritten-digit classifier — 784 inputs into 128, then 64, then 10 outputs. 109,386 learnable parameters, 82 seconds to train on a laptop, ~97.5% accurate. Small enough to print every number in it, which is the point: you can see that the neuron values lighting up during a run exist only during that run. They vanish when it ends. What was learned — the only thing that persists — is the weights. And after training, the weights are done. (That exercise is written up separately, with the figures from my own run, in train a network, then look inside it — the code is on GitHub if you'd rather run it than read about it.)

The mistake it prevents: expecting the model to learn. It doesn't remember yesterday's correction. It doesn't know your codebase, your conventions, or what shipped last week. Its knowledge stopped at a training cutoff that predates your problem. Everything after that line — your repo, your decisions, this morning's incident — is something you supply at inference time, every single time. That isn't a limitation to route around. It's the entire job description.

2 · The model never sees your text — only integers

Before a single word reaches the model, it's split into chunks and encoded as numbers. "Hello World!" becomes a handful of integers, and the model's reply comes back as integers that get decoded on the way out. Both steps happen outside the model. It has never seen a letter.

The chunks come from a vocabulary grown from data by repeatedly merging frequent pairs — around 50,000 entries in the GPT-3 era, 100k–260k in current models. Nothing is untokenizable; unusual text simply falls back to smaller pieces and costs more. The word understanding might be two tokens against a large vocabulary and five against a small one. Same word, different price.

The mistake it prevents: monitoring characters, lines, or "how long the file looks." Tokens are the unit of both billing and budget, and they don't track your intuition — a minified JSON blob and a paragraph of the same visual length are not the same purchase. Output tokens cost several times more than input tokens, for a reason worth internalising: producing each one means re-reading everything that came before it. Verbose output isn't just noise, it's the most expensive thing you can ask for.

3 · The context window is an attention budget, not a hard drive

This is the model that changes behaviour fastest. Every turn replays the whole conversation: system rules, your CLAUDE.md, every tool definition, every previous message, every tool result. The model is stateless. Nothing is "kept" between turns — it is re-sent, and the window only grows.

Two consequences that people find genuinely surprising.

Position matters. The beginning and the end of a long window get attended to; the middle sags — the same failure humans have with long documents. Something technically present in the context is not something reliably read. Which makes the largest silent tax the part you never chose: tool definitions loaded at startup, parked in that sagging middle, spending budget on every turn whether you use them or not. Twelve MCP servers isn't twelve capabilities. It's a standing levy on every request you make for the rest of the session.

Bigger windows are not the fix. Attention compares every token against every other token, so reading a prompt gets more expensive faster than the prompt gets longer, and every generated token re-reads the accumulated state behind it. That's real memory and real latency per request. The scarce resource is attention, not capacity — which is why a 10M-token window doesn't rescue a session you've filled with noise.

The mistake it prevents: stuffing the window and hoping. The discipline is the reverse. Measure first — /context will show you what's actually eating the budget, and it's rarely what you'd guess. Treat compaction as the lossy operation it is: an automatic summary replaces your history and the machine picks what survives. When it matters, choose what crosses instead — a written handoff, a file in the repo. Chosen context beats an auto-summary, and the file is the memory.

4 · An agent is a loop — the model never executes anything

"Agent" is the most over-mystified word in this field, and it comes apart in four steps.

• Declare — the available tools are described as text, in the system prompt. (That's why every tool you install is context you pay for, per turn, forever.)
• Choose — the model replies with a tool name, some arguments, and an id. That's all it does. It's a request, not an action.
• Execute — your code runs it. The runtime opens the file, runs the command, calls the API. The model is not in the room for this step.
• Feed back — the result re-enters the conversation as another message, and the loop repeats until the goal is met.

There is no autonomy inside the model. The autonomy is in the loop your runtime is willing to keep running.

The mistake it prevents: two of them, actually. First, looking for safety in the prompt. Every guardrail belongs at step three — the only step where anything real happens — which is why hooks and permissions live in the runtime and not in a politely-worded instruction. Second, reaching for an agent by default. If the steps are fixed and no judgment is required, write the script: it's faster, cheaper and it can't improvise. An agent earns its cost when the path genuinely can't be known in advance. Wrapping a cron job in a language model is how teams end up with something slower and less predictable than what they replaced.

Why this is the ground floor of everything else

Put the four together and the strategic picture falls out of them. The intelligence is rented — the same frozen weights, available to your competitor this afternoon at the same price, improving on a schedule neither of you controls. What isn't shared is the other half of the equation: the context you put in front of it, and the workflows your engineers build into the stack. Output is a function of both terms, and you only own one.

Which is why the work that compounds isn't wording the request — it's the harness around the model, built by the people who know the codebase, until the point where the work runs without you in the loop. The model is frozen. Your team isn't.

If you'd rather install these four models in your team's head than send them a link, that's precisely what a Foundations session is — same material, on your own repo, with the questions your engineers actually have.

← Back to the working notes