From token prediction to self-verifying loops
The engineering around the model made the difference. That's this whole talk.
Each upgrade fixes the last one's limit. One thread runs through all seven — the context window — and ● CONTEXT flags it. New to all this? Start with the beginner deck.
Where it started: a text box and a next-token predictor. Astonishing — and all it can do is guess.
A model is a probability distribution over the next token, sampled one token at a time.
Your keyboard sees ~one word; the old n-gram models saw two or three. An LLM with no useful context drifts exactly like this — the upgrade that sees everything is next.
● CONTEXT This is the thread: attention runs over the context window, so what you load into it is the program.
● CONTEXT Two of these let you watch attention actually run over the context — the thread, lit up.
Problem: hard answers don’t fit in one forward pass. Fix: let the model write itself a scratchpad — and dial how much.
Same model, same question. The only difference: it thought on paper before it answered.
● CONTEXT Reasoning is the model extending its own context — the scratchpad is program it writes for itself.
| Effort | Reach for it when… |
|---|---|
| low | cheapest & fastest — bulk, delegated grunt work. |
| medium | the cost-saving step-down for routine agentic work. |
| high | the API default (Opus 5.5: medium) — start here for most work. |
| xhigh | step up here for long coding & agentic runs — Anthropic's pick for 30-min+ tasks. |
| max | frontier problems only — past xhigh it mostly adds cost. |
Claude Code's "ultracode" = xhigh + standing permission to run multi-agent workflows (chapter 06) — a harness mode, not an API level.
Effort moves you along one model's curve; the model picks which curve — and a newer curve sits above and to the left. On Frontier‑Bench, Anthropic reports Opus 5 more than doubling Opus 4.8 at a lower cost per task; past xhigh, max buys ~nothing for roughly 33% more tokens. Its low/medium are unusually strong — re-sweep, don't inherit.
Problem: it can only talk — it can’t read your repo or run a test. Fix: tool calls, and a harness to execute them.
The brain — an API you rent. Fable · Opus · Sonnet (Anthropic) · GPT‑6 Astra (OpenAI) · Gemini 3.1 Pro (Google).
The loop around the model: assembles context, runs tools, manages permissions. Claude Code · Codex CLI · Antigravity CLI · Jules (async).
Where you meet the harness: terminal · IDE extension (VS Code, JetBrains, Antigravity) · desktop · web · CI.
The model only emits tokens — the harness is the program that gives them hands. Rent the model, run the harness, pick the client; most models work in most harnesses.
● CONTEXT A tool result is just more context. "Reading a file" = loading its tokens into the window.
Every arrow to the server carries the whole conversation again — the API keeps nothing between calls. Remember the growing bars; the why is next, the price is chapter 05.
Which tokens fill that context window is what you're really controlling.
Problem: one step per turn — you are the relay. Fix: the harness re-runs until a verifier says done.
Levels 1–3 are the deck so far: knowledge (ch 01), thinking (ch 02), tools (ch 03) — you mostly get them for free. Level 4 — closing the loop against a goal — is this chapter, and where the failure modes live.
That's the entire trick: a while-loop around a token predictor. Which raises the question the next slide answers — who decides when it's done?
"exit code 0" is. A verifier is an objective check the model can't argue with.
Never recognises completion — each lap piles new code on broken code. Burns tokens, gets worse.
Reward hacking: test.skip, hardcoded expected values, deleting tests, appending || true. The verifier goes green; the work is fake.
Both are spec bugs, not model bugs. The loop optimises exactly what you wrote down; if "pass" is gameable, it gets gamed.
Name the verifier command, forbid the shortcuts by name, and demand real evidence — test output, an exit code, a screenshot diff.
Problem: every lap re-sends the whole history — cost grows quadratically and quality rots. First, the diagnosis.
"fix it" is 3 tokens; the request that carries it is ~13,000. The other 12,997 are the memory — every reply, tool call, and file read gets re-sent, verbatim, forever.
Turn 20 drags all 19 previous turns with it — the longer the session, the heavier and slower every turn.
| Input tokens | Output tokens | |
|---|---|---|
| What | everything re-sent — history, files, tool results | generated text, incl. thinking tokens |
| Volume | huge, grows every turn | usually small |
| Price / token | baseline (cheap) | ~5× input |
| Cacheable? | yes — coming up | no |
| Model · September 2026 | Input / MTok | Output / MTok |
|---|---|---|
| Haiku 4.5 | $1 | $5 |
| Sonnet 5 | $2 | $10 |
| Opus 5.5 | $4 | $20 |
| Fable 5.1 | $10 | $50 |
Agentic sessions are input-dominated: the model reads far more than it writes.
Three patterns: the ~5× output premium holds across the lineup; each tier jump roughly doubles both prices; and Opus 5 landed at Opus 4.8's price — a generation of capability for free. That last one is the tell: this is a price per token, not a price per task. Next slide.
| Measured | Result |
|---|---|
| Opus 4.5 vs Sonnet 4.5 SWE-bench, medium effort | matched Sonnet's best score using 76% fewer output tokens |
| Opus 5 vs Opus 4.8 internal trading eval | best score on ~1⁄7 the reasoning tokens, under half the latency |
| Sonnet 5 vs Opus 4.8 BrowseComp | same accuracy at ~⅓ the per-task token cost |
| Opus 5 xhigh vs max GDPval-AA v2 | near-max score on 25% fewer output tokens |
A weaker model flails: it re-reads files, makes more tool calls, guesses wrong and retries — and every lap re-sends the whole history. You save 5× per token and spend 3× the tokens.
Why it happens: capability shows up as fewer laps, not just better answers. Fewer laps means less thinking, fewer tool calls, and — because every lap re-sends everything before it, so the session total grows quadratically — a much smaller bill.
The caveat that keeps this honest: it only holds where the task is hard enough for the shortcut to exist. Classification doesn't get cheaper on Opus. The rule stays cheapest model that clears the bar — measured on your work, not assumed.
The bill (Opus 5.5, $4 / $20 per MTok): input would be $0.09, but ~19k of it is cache-read at 0.05× → ~$0.02. Output: 1,600 × $20/M → ~$0.03.
The model reads 14× more than it writes — yet the tiny output side costs more: output is 5× the price and never cached. That's why thinking tokens are the expensive dial — billed as output here, then kept in context: they come back on the input side.
● CONTEXT Caching = paying less to re-send the same context.
● CONTEXT Cache is per-model — switch models mid-session and you pay to rebuild the cached context.
● CONTEXT Where agents go to get confused is the last stretch of the window — hygiene is just keeping it lean.
Problem: one window, one brain, carrying everything. Fix: route the cheapest adequate brain; delegate to fresh windows.
● CONTEXT Delegation is context engineering — spend the main window on decisions, not exhaust.
| Maker | Small | Medium | Large | Above (frontier) |
|---|---|---|---|---|
| Anthropic | Haiku | Sonnet | Opus | Fable |
| OpenAI · GPT‑6 | Luna | Sol | Astra | — |
| Google · Gemini 3.5/3.8 | Flash‑Lite | Flash | Pro | Deep Think |
S / M / L repeats at every lab; Anthropic adds Fable above the large model — priced above Opus (2.5× per token). Whether it still scores above it is the next slide. Snapshot, September 2026 — this reshuffles constantly.
| Benchmark · July 2026 | Sonnet 5 | Opus 4.8 | Opus 5 | Fable 5 |
|---|---|---|---|---|
| SWE-bench Pro · real repo fixes | 63.2 | 69.2 | 79.2 | 80.0 |
| OSWorld 2.0 · computer use | — | 55.7 | 70.6 | — |
| Frontier-Bench v0.1 · hard agentic | — | ~18 | ~43 | 33.7 |
| ARC-AGI-3 · novel problems | — | — | 30.2 | — |
Two shapes here. On ordinary coding the tiers are 10–15 points apart — real, not dramatic. On benchmarks built from problems the last generation couldn't touch, it's a 2×+ jump (Frontier-Bench) or 3× the July field (ARC-AGI-3, where the next-best model scored 7.8). SWE-bench Verified is saturating — Opus 5 sits at 96%. Caveat: Anthropic publishes most of these as claims rather than tables — exact figures come from third-party trackers and move a point or two.
Why it matters here: jumps this size make an old routing table wrong, not just suboptimal. Dashes are honest — not every model is run on every benchmark — and none of these is your workload. Re-measure.
● CONTEXT Thinking tokens are output that lands right back in the context window.
Before you reach for a bigger model, turn the effort up. opus·high is the workhorse; xhigh for long-horizon runs.
Route to the smallest model that clears the bar — measured. Escalate on failure, not vibes.
low effort on a new model often beats xhigh on the old one — a cheaper floor, not just a higher ceiling.
The escalation ladder: same model, more effort → bigger model → Opus 5.5 at max → Fable (for judgement work — two slides on). Move up only when something actually fails. (Switching models also busts your cache — one more reason not to hop on vibes.) And re-run the sweep after a model launch: Opus 5.5's low/medium are strong enough that inherited settings are usually wrong.
You hired a staff engineer. Use it for the hard call and the review — and only once Opus 5.5 at max effort has actually failed. Then hand the typing to something cheaper.
The API also has a built-in advisor tool (beta), consulted mid-generation. One rule: the advisor must be at least as capable as the executor — so sonnet·executor → opus/fable·advisor is fine, opus·executor → sonnet·advisor is a 400. Claude Code has /advisor and a --advisor flag; the rule above is the manual version.
Fable plans, delegates, and reviews what comes back; haiku / sonnet / opus do the typing.
The UI reports the wrapper's Claude model — the label is the only cue the real worker was GPT-6 Sol. Parallel cross-vendor workers need worktree isolation so edits don't collide.
| Tier · the lineup from before | Suggested usage |
|---|---|
| Small — Haiku · Luna · Flash-Lite | classification, log triage, autocomplete, high-volume pipelines — anything you'd regret paying Opus prices for. |
| Medium — Sonnet · Sol · Flash | the workhorse: implementation from a clear spec, tests, refactors, bulk subagents, data wrangling. |
| Large — Opus · Astra · Pro | the agentic default: long-horizon coding, debugging, multi-step tool work, user-facing writing & design. |
| Frontier — Fable · Deep Think | plan reviews, the hard architectural call, verify stages, final polish — never bulk implementation, mechanical edits, or first drafts. |
Same split every time: the top tier decides and checks; cheaper models produce the volume. Escalate on failure, not vibes — a cheaper model at higher effort often beats a pricier one at low. But price the task, not the token: on genuinely hard work the bigger model gets there in fewer laps, and fewer laps is where the money actually is.
● CONTEXT Each stage gets a fresh window — one stage's mess never carries into the next.
You never hand-write the script — you state the shape (stages, fan-out, the verify bar) and the model writes & runs the orchestration. Knowing the shape tells you what to ask for.
Reach for the simplest rung that does the job. A workflow for a one-line question is as wrong as eyeballing a thousand-file migration by hand.
Problem: everything you fixed dies at /clear — every session starts blank. Fix: durable context — CLAUDE.md · skills · plugins.
● CONTEXT CLAUDE.md is context you pay for every turn — the whole reason skills exist is to keep it thin.
The model knows the world, not your workflow. Release-notes format, review checklist, deploy steps — re-explained every single session.
Paste it all into CLAUDE.md and it rides along in every turn's context — paid every turn, relevant almost never, crowding out the actual work.
A one-line description that's always present; the full recipe loads only when the task matches. The cost of an index entry, the power of a manual.
● CONTEXT Skills answer one question: how do I teach the agent my job without paying for the lesson on every turn?
● CONTEXT Skills exist to be lazy-loaded into context only when it makes sense.
No registration, no restart — the harness matches your sentence to the description and loads the body. A skill is just context, injected at the right moment.
Matt Pocock's approach: dozens of tiny skills — conventions, testing style, PR etiquette — each triggering on its own cue.
A whole method as a skill system: brainstorm → plan → work → simplify → review → compound. Every solved problem becomes a new doc or skill.
A skill system is an engineering culture the agent can read — two worked examples on the next slides.
github.com/mattpocock/skills — alignment before code, always; read the files, copy the patterns, adapt the conventions.
The unit of progress isn't the merged PR — it's the knowledge that compounds. (Kieran Klaassen / Every.)
● CONTEXT A plugin is your team's context, made distributable.
Skills = your habits. Plugins = your team's habits — versioned, installable in one command.
A token predictor → that thinks → with hands → in a loop → whose context you engineer: route it, delegate it, cache it, write it down.
● CONTEXT Every marker in this deck pointed back here.
Thanks! Questions?
David Budáč · figures a September 2026 snapshot — models & prices move fast, so re-check the primary sources.