Part 5 of 6

Scale: multi-agent systems and feedback control

Distributed-systems problems, evaluator loops as feedback controllers, persistent state, and routing work to the cheapest reliable executor.

Sections 15–18
15 · Distributed systems

Multiple agents create distributed-systems problems

The moment two workers run at once, forty years of distributed-systems engineering applies.

Duplicate work Conflicting changes Partial failure Stale state Race conditions Deadlocks Endless delegation Excessive resource use Incompatible outputs Lost ownership Repeated tool calls
Actor isolation

Each worker owns a clear task or resource.

Message passing

Structured messages, not uncontrolled shared context.

Correlation IDs

Every action belongs to a traceable task.

Distributed tracing

Follow a request across all workers, tools and state.

Timeouts

No worker may run indefinitely.

Circuit breakers

Stop repeatedly failing tools or services.

Bulkheads

A failing worker must not collapse the system.

Backpressure

Limit new work when the system is overloaded.

Consensus / quorum

Multiple evaluations when one judgment is unreliable.

Conflict resolution

Define how competing edits merge or get rejected.

Hard rule: do not allow several workers to edit the same files simultaneously without ownership, locking or merge rules.

What this means for the business
More agents usually buys coordination cost, not throughput

The moment work is split across several agents, you inherit the classic distributed-systems problems, including race conditions, conflicting edits, stale context and duplicated effort, but without the mature tooling that exists for conventional distributed systems. Throughput rarely scales with the number of agents, while the cost of merging and reconciling their work grows quickly. This is the pattern where impressive demos most often fail to survive real workloads.

Decision Default to one coordinated pipeline, and require evidence that parallel agents earn back their merge and reconciliation cost.
16 · Control theory

Evaluator-optimizer loops are feedback controllers

Control theory
Prompt loop
Setpoint
Target quality
Current output
Generated result
Sensor
Evaluator
Error signal
Target minus measured quality
Controller
Repair planner
Controlled system
Generator or agent
Disturbance
Model variability, missing context, tool failures
Target Generate Measure Gap Repair
generate improved result ↺
Continue only while
Quality below targetProgress is measurableBudget remainsA plausible repair existsIteration limit not reached
Stop when
Target reached · max iterationsBudget exhaustedNo progress detectedSame failure repeatsSafety boundary · human judgment needed
FAILURE MODE
Oscillation

Repeated switching between two solutions.

FAILURE MODE
Overcorrection

Fixing one criterion breaks another.

FAILURE MODE
Local optimum

Small repairs cannot reach the desired result.

FAILURE MODE
Reward hacking

Passes the grader without satisfying the real goal.

FAILURE MODE
Endless refinement

The loop never declares completion.

What this means for the business
An unbounded repair loop is an unbounded invoice

A generate, evaluate and repair cycle is a feedback controller, and it fails the way controllers fail: oscillating between two solutions, overcorrecting so one fix breaks another criterion, settling into a local optimum, satisfying the grader without meeting the real goal, or simply never declaring completion. Each iteration costs money and elapsed time, so these failure modes are budget events, not curiosities. The controls are ordinary: a target, a cap, a budget and explicit stop conditions.

Decision Define target quality, iteration cap, budget and stop conditions before any self-repair loop is switched on.
17 · Persistent state & checkpointing

Long-running agents need more than conversation history

A long-running agent should resume from explicit state, not reconstruct its progress from a long chat transcript.

The state store
Objective
current_objectiveoriginal_user_requeststructured_specificationaccepted_decisions
Progress
current_workflow_stagecompleted_tasksremaining_tasksfailed_approachesknown_blockers
Quality & artifacts
test_resultsevaluation_scoresrepository_revisionfiles_changedtools_used
Resources & governance
cost_consumed · tokensapproval_statusnext_permitted_action
Checkpoint after…
Classification
Specification
Planning
Before every irreversible action
Implementation
Verification · every repair iteration
Before deployment
What this means for the business
Conversation history is not a system of record

Long-running work must resume from explicit, stored state. When progress lives only in a growing transcript, cost rises with length, older decisions get crowded out and quietly forgotten, and an interruption loses work that has already been paid for. Checkpointed state makes a run resumable, auditable and cheap to continue, and it is the difference between a process that survives a deployment and one that has to start again.

Decision Require checkpointed, resumable state for anything long-running, and test recovery deliberately rather than discovering it during an incident.
18 · Model & resource routing

Use the cheapest reliable executor

Not every message should become an autonomous agent task. Route each decision to the lowest layer that can carry it reliably.

Human decision
Irreversible actions · ambiguous business trade-offs · legal or financial commitments · publishing · spending · deleting · external communication
Strong model
Ambiguous reasoning · architecture decisions · complex planning · creative direction · difficult repair · cross-domain synthesis
Small / inexpensive model
Classification · extraction · formatting · summarization · simple routing · low-risk transformations
Deterministic code
Validation · calculations · state transitions · permission checks · database operations · API contracts · exact transformations
escalate upward only when the layer below cannot decide reliably
What this means for the business
This is where the unit economics is decided

Every decision should be routed to the cheapest layer that can carry it reliably: deterministic code first, then a small model, then a strong model, and a human only where judgement or accountability genuinely requires it. Sending everything to the largest available model is the most common reason a promising feature has per-request costs that cannot scale, and it also adds latency users notice. Routing is a design decision with a direct margin impact.

Decision Require a routing table with the cost of each path, and review it again once real usage volumes are known.
Series

Continue the engineering guide

This is one part of a six-part guide on turning prompts into specification-driven probabilistic software.

Back to the series overview →

App & Cloud Architecture · Book a Product Discovery Call

More articles