Pipelines

Real systems aren't one mechanism. They're pipelines.

Every production AI system is a composition of synthesizers. Cost is additive, verifiability is bottlenecked by the weakest stage, and the cache hits independently at every step. Three rules govern the whole arithmetic.

Part 01 · The composition operator

∘ — the operator the field treats as if it were invisible.

Every system is a sequence of stages, each with its own coordinate, its own cost, its own verifiability. The pipeline is the composition of stages, written left-to-right or top-to-bottom:

system = m_n ∘ … ∘ m₂ ∘ m₁

A system is a composition of mechanisms · each with its own coordinate

Each m_i has a coordinate c_i = ⟨Z, E, T, P, I, V, R⟩, an estimated cost from the cost function, and a receipt after it runs. The pipeline's receipt is the concatenation of stage receipts. There's no monolithic decision — only a chain of decisions, each individually optimal.

Part 02 · Three rules

What composition does to cost, verifiability, and cache.

Rule 1

Cost adds.

E(system) = Σ_i E(m_i)

The pipeline is not lower-cost than the sum of its parts. A five-stage RAG agent that costs 5 mJ + 50 mJ + 100 μJ + 5 mJ + 50 μJ costs 60.15 mJ end to end. The cost function must run per stage, not per system — routing one stage to a low-cost mechanism while another stage burns the savings is the most common failure mode in real pipelines.

Rule 2

Verifiability bottlenecks.

V(system) = min_i V(m_i)

A pipeline's output is only as checkable as its weakest stage. A formal verifier downstream of a hallucinating LLM cannot rescue what the LLM produced — it can only refuse to certify it. If any stage is V₅, the whole pipeline is V₅. To deploy in regulated markets, every stage must clear the V threshold the market demands.

Rule 3

Cache multiplies.

Each stage hits its cache independently. The cacheable fraction of stage 1 doesn't have to coincide with stage 2's. A query whose parse is cached but whose generation isn't still saves the parse's cost; a query whose retrieval is cached but whose post-processing isn't still saves the retrieval. A five-stage pipeline with 80% cache hit per stage runs only the missing fraction — not the missing-anywhere fraction. The cache becomes more powerful, not weaker, as pipelines get longer. → Cache

Part 03 · A worked pipeline

A RAG-assisted coding agent, decomposed.

One task: "Add a route to /api/users/:id/sessions following the existing pattern." Five stages, each routed per-stage. The pipeline receipt is the concatenation.

# Stage Mechanism Cost V Cacheable?

1 parse intent Deterministic ~10 μJ V₁ yes

2 retrieve route conventions Retrieval ~100 μJ V₂ partial

3 adapt template to new route Small model ~5 mJ V₂ no

4 type-check + verify imports Verifier ~50 mJ V₁ yes

5 format + emit Deterministic ~5 μJ V₁ yes

Pipeline cost (sum of stages) ~55 mJ Pipeline V (min of stages) V₂ · bottlenecked by stages 2, 3 Frontier-LLM-only equivalent ~3 J · V₄ Ratio ~55× lower-cost · two V notches better

The frontier model alone produces a fluent answer at V₄ for ~3 J. The five-stage pipeline produces the same answer at V₂ for ~55 mJ. The pipeline is auditable: every stage emits its own receipt, and stage 4 carries a formal certification that the imports resolved and the types check. Same output. Two notches more verifiable. Fifty-five times lower-cost. Because the composition is explicit.

Part 04 · What it enables

Pipelines unlock precision the monolith can't.

Precision V per stage

V₁ where it must be. V₂ where it's enough.

A monolithic system has one V. A pipeline can have V₁ where the law requires it (formal verification, type checks) and V₂ where retrieval is enough (drafting, summarization). The system enters regulated markets at the strict-stage V, not the loose-stage V.

Per-stage cache invalidation

Update one stage, keep the others cached.

When a regulation changes, only the retrieval stage's cache invalidates. The parse cache, the verifier cache, the format cache stay warm. Monolithic caches invalidate end-to-end on any change.

Stage-level routing

Each stage picks its own mechanism.

The router doesn't pick a "system" — it picks the right mechanism for each stage. When a lower-cost Z₂ retriever shows up, only that stage re-routes. The rest is untouched.

Audit trails span boundaries

Trace every decision to the stage that made it.

"Why did the pipeline return X?" decomposes into "Stage 2 retrieved Y, stage 3 adapted it to Z, stage 4 verified Z holds, stage 5 formatted." Each step signed. Each step replayable.

Part 05 · Where it fails

Composition breaks when stages couple.

The three rules — additive cost, min verifiability, multiplicative cache — assume stages are independent: the cost of stage 2 doesn't depend on what stage 1 produced; the cache of stage 4 doesn't change because stage 3's output changed. Most production pipelines are not fully independent.

Two patterns to watch for. Cyclic compositions — a generation stage that conditions on an authority that itself depends on the generation — have to break the cycle with a verifier or a fixed-point iteration with a budget. Conditional routing — stage k+1's mechanism depends on stage k's output — means the cost estimate has to integrate over the distribution of stage k's outputs, not just its expected cost.

Neither breaks the framework. They sharpen it. A pipeline is a directed graph of mechanisms with cost-and-V annotations; the three rules apply when the graph is a chain. When it isn't, you analyze the graph properly — but you're still adding, taking minima, and caching at the same granularity. The hard work is naming the stages; once they're named, the arithmetic is mechanical.

Continue

The arithmetic of deployed systems.

Composition is what receipts add up to. Verifiability is what the min-rule produces. Cache is what each stage hits independently.

The Receipt Verifiability Cache Back to Cost