The AI Engineering Playbook¶
Vets Who Code · Hashflag Stack · Phase 3
A leveled, in-depth path for software engineers — from the first model call to spec-driven mastery. Eight rungs, four tiers, and the inversion at the top that turns a builder into an engineer.
| Rule | Principle |
|---|---|
| Rule 01 | Core before conditional. Build small. |
| Rule 02 | Termination is the boss at every altitude. |
| Rule 03 | Beginners climb up. Experts invert. |
Field Manual · V4.0 · The Complete Climb
How to read this
Architecture is a map. This is the path through it.¶
You already have the architecture — where each piece sits: spec, model, knowledge, harness, loop, and the agent-composition stack. This playbook is a different ordering: where each piece is learned. The two are not the same, and that difference is the point. The most notable reorder is deliberate — architecturally the knowledge layer sits before the harness, but here you build the loop first and add retrieval later, because retrieval bolted onto a loop you don't understand is how engineers ship RAG systems they can't debug. Dependency for understanding runs loop-first.
Rule 1 — Core before conditional.
Every rung is either core (present in almost every system) or conditional (only if your architecture calls for it — the knowledge layer is the clearest case; a pure tool-use or code-execution agent skips it entirely). This split is permission: one agent with a few tools is the floor, and a smaller first system is not a cut corner.
Rule 2 — Termination is the recurring boss.
"How do I call a tool" is easy. How does it know it's done is the actual job — and it returns at every altitude. The runtime loop, each subagent, the orchestrator, even the dev loop and the outer lifecycle loop all face the same problem raised one level. Where agents break is almost never the tool call; it's the stop condition.
Rule 3 — The through-line runs down to the token.
Tokenizer behavior and logprobs are not trivia. They become a confidence signal, which becomes a loop-control decision: low confidence → iterate; high confidence → exit. The deepest thread in the stack is token entropy → confidence → termination. Foundations are the control plane.
The inversion¶
The beginner — L0 upward: climbs bottom-up¶
Starts from a single model call and assembles primitives, because they don't yet know what's buildable. The ladder is how you learn what the pieces are.
The expert — L7 downward: starts at the spec¶
Works top-down from intent because the primitives are internalized. Writes the test of "done" before building, and lets the spec govern everything below it.
Climb the ladder once · Then invert it · That inversion is what "expert" means
The whole climb, one page
Map at a glance¶
Eight rungs in four tiers. Read the identity column as "what you can do when you're standing here." Tags mark whether the rung is core or conditional-by-architecture.
Tier I — Foundations · Can call a model reliably¶
| Rung | Name | What it means | Tag |
|---|---|---|---|
| L0 | The Call | Validated, structured data back from a single model call. | CORE |
| L1 | The Single Action | The model can do one thing — one tool, one hop, no loop. | CORE |
Tier II — The Agent · Can ship a loop that terminates¶
| Rung | Name | What it means | Tag |
|---|---|---|---|
| L2 | The Loop | Close the loop — your first real agent. Termination appears. | CORE |
| L3 | The Harness | The nouns around the loop. It survives contact with reality. | CORE |
Tier III — Making It Real · Can ground it and prove it¶
| Rung | Name | What it means | Tag |
|---|---|---|---|
| L4 | Knowledge & Confidence | Retrieval when needed; logprob gating as a loop-control signal. | COND+CORE |
| L5 | Proof | The eval harness plus observability. "It works" becomes numbers. | CORE |
Tier IV — Scaling Judgment · Knows what not to build¶
| Rung | Name | What it means | Tag |
|---|---|---|---|
| L6 | Composition | Skills, subagents, orchestrators — earned, never reflexive. | CORE |
| L7 | The Inversion | Spec-driven development and the outer loop. You start at the top. | CORE |
How to use this playbook¶
- Work it in order. Each rung assumes the one below it. The ordering is learning-dependency, not architecture.
- Do not skip L2. Termination is where most self-taught engineers stall; every rung above it inherits the problem.
- Conditional means optional-by-architecture, not optional-by-effort. Skip retrieval only if your system genuinely doesn't need it.
- Mastery of a rung = every checkbox hit and you can teach its failure modes to someone else. Anything less is "passed through," not "cleared."
The shape of the climb
The ladder, and the inversion¶
Eight rungs, bottom to top. You climb it once — assembling primitives from L0 upward. Then, at the top, you invert: the expert stops climbing and drives the whole stack downward from a spec. Learn upward; work downward.
| Rung | Name | Focus |
|---|---|---|
| L7 | The Inversion | SPEC-DRIVEN · OUTER LOOP |
| L6 | Composition | SKILLS · SUBAGENTS · ORCHESTRATORS |
| L5 | Proof | EVALS · OBSERVABILITY |
| L4 | Knowledge & Confidence | RAG · LOGPROB GATING |
| L3 | The Harness | CONTEXT · RECOVERY · STATE |
| L2 | The Loop | TERMINATION APPEARS |
| L1 | The Single Action | ONE TOOL · ONE HOP |
| L0 | The Call | STRUCTURED OUTPUT |
↑ Beginner climbs up from the atom · ↓ Expert drives down from the spec
Gold rung = the expert tier. Red left edge = conditional-by-architecture (L4 — skip if the system doesn't need retrieval). The two arrows are the two directions of travel.
Prefer print? Download the PDF edition — Field Manual, V4.0.