Designing an Agent Harness
Short answer: The harness is everything that is not the model — the loop, the budget, the stop conditions, the guardrails, and the record of what happened. Model quality sets the ceiling. The harness decides whether you ever get near it.
Related guides: Building AI Skills covers the instructions a harness loads. Evaluating AI Systems covers proving the harness works. For a domain-specific worked example, see Building Agents for SEO Tasks.
The Loop
Every agent, regardless of framework, is this:
receive goal
↓
┌─→ decide next action ──→ call tool ──→ observe result ─┐
│ │
└──────────── still incomplete? ←─────────────────────────┘
↓
stop condition met
↓
produce output
Four things can go wrong, and each has a distinct fix:
| Failure | Looks like | Fix lives in |
|---|---|---|
| Bad decision | Calls the wrong tool repeatedly | Tool descriptions, instructions |
| Bad observation | Ignores what the tool returned | Result formatting, context budget |
| No stop | Loops until you kill it | Stop conditions |
| Bad output | Right work, unusable result | Output contract |
Debugging starts by identifying which of the four you have. Teams lose days rewriting prompts to fix what is actually a missing stop condition.
The Task Contract
Before the loop starts, write down what "done" means. If you cannot, the agent cannot either.
goal: Identify pages that lost organic clicks month over month and explain why
inputs:
- property: example.com
- window: last 60 days
tools_allowed: [get_search_analytics, inspect_url, fetch_page]
tools_forbidden: [publish, send_email, write_cms]
must_produce:
- table: url, clicks_delta, position_delta, hypothesis
- every hypothesis cites the tool output supporting it
stop_when:
- table has ≥ 1 row per qualifying URL, or
- 25 tool calls used, or
- 3 consecutive calls return no new information
on_uncertainty: state it in an "unclear" column, do not guess
Three parts do the heavy lifting:
tools_forbidden— an allowlist is not enough. Naming the dangerous things explicitly makes review easy.stop_whenwith an or — you need a success condition, a budget cap, and a no-progress detector. Any one alone fails.on_uncertainty— without it, uncertainty gets silently converted into confident prose.
Context Budget
Context is finite and degrades before it fills. Attention on any single instruction thins as the window grows, so a technically-fitting context can still produce worse results than a smaller one.
Allocate deliberately:
| Slice | Typical share | Rule |
|---|---|---|
| Instructions / skills | 10–20% | Load on trigger, not always |
| Tool definitions | 5–10% | Only tools relevant to this task |
| Working data | 40–60% | Summarize old observations aggressively |
| Output space | 20–30% | Reserve it; do not discover you are out |
Three techniques that buy the most room:
Summarize completed steps. Once a sub-task is done, replace its full transcript with a three-line result. The detail lives in the artifact store if anyone needs it.
Return references, not payloads. A tool that fetched a 40,000-token page should return a summary plus a path, not the page.
Give sub-tasks their own context. A sub-agent that reads twelve files and returns a paragraph keeps eleven-and-a-half files out of the main window. This is the single biggest lever on long-running agents.
Guardrails
Layer them. Any single guardrail will be worked around eventually.
| Layer | Mechanism | Stops |
|---|---|---|
| Capability | Tool allowlist; read-only credentials | Whole classes of damage |
| Budget | Max tool calls, max tokens, wall-clock timeout | Runaway cost |
| Validation | Schema-check every tool output before it enters context | Malformed data poisoning the run |
| Approval | Human gate before irreversible or external actions | The expensive mistake |
| Audit | Persist every call, input, and output | Everything else, after the fact |
A confirmation prompt on every step trains people to click through without reading. One gate immediately before "publish", "send", "delete", or "spend" gets actual attention.
Read-only by default
The strongest guardrail is architectural. Give the agent read credentials and have it propose writes as a structured artifact — a diff, a PR, a draft, a queued job. A human promotes the artifact. The agent never holds the ability to do the damaging thing.
Stop Conditions
Three kinds, and you need all three:
| Kind | Example | Catches |
|---|---|---|
| Success | Required output produced and validated | Normal completion |
| Budget | 25 tool calls / 10 minutes / N tokens | Runaway loops |
| No-progress | 3 consecutive calls added no new information | The subtle one |
The no-progress detector is the one most harnesses lack. An agent that re-queries the same endpoint with slightly different phrasing looks busy, burns budget, and never converges. Hash the tool results; if the last three are near-identical, stop and report what you have.
Single Agent or Many?
| Shape | Use when | Cost |
|---|---|---|
| Single agent, many tools | Most cases. Start here. | Context bloat on big tasks |
| Agent + sub-agents | Sub-tasks are read-heavy and independently summarizable | Coordination, partial failures |
| Orchestrator + workers | Genuinely parallel work over a known list | Highest build and debug cost |
| Fixed pipeline, models inside | The sequence never changes | None — this is just a program |
The honest default: if you do not have a concrete reason a single agent fails, use a single agent. Multi-agent systems fail in ways that are much harder to reproduce, because the failure depends on interleaving.
The clearest reason to fan out is context isolation — five agents each reading a different subsystem and returning a summary, so the parent never holds the raw material.
The Audit Trail
If you cannot answer "why did it conclude that?" a week later, you do not have a system you can operate.
Persist, per run: run ID and timestamp; goal and inputs; every tool call with arguments and result; the model and instruction version used; the final output; and the stop condition that fired.
That last field is diagnostic gold. A week of runs all ending on budget_exceeded tells you the task is under-scoped long before anyone complains about quality.
Failure Modes
| Symptom | Likely cause | Fix |
|---|---|---|
| Loops on the same tool | No no-progress detector | Hash results; stop on repeats |
| Ignores tool output | Result buried in a huge context | Summarize; put results last |
| Confidently wrong facts | No grounding tool for that data | Add the tool, or forbid the claim |
| Great work, wrong format | No output contract | Specify the schema in the task contract |
| Works alone, fails scheduled | Env, auth, or interactive assumptions | Run headless in CI before trusting it |
| Slow and expensive | Too many tools; too much always-on context | Trim the tool list per task |
| Non-reproducible | Nothing versioned | Persist instruction and model versions per run |
Production Checklist
- Task contract written, including
stop_whenandon_uncertainty - Tool allowlist set; destructive tools explicitly forbidden
- Credentials are read-only, or writes go through a proposal artifact
- Budget cap on tool calls, tokens, and wall-clock time
- No-progress detector implemented
- Every tool output schema-validated before entering context
- Human approval gate immediately before anything irreversible
- Full run log persisted, including which stop condition fired
- Instruction and model versions recorded per run
- Ran headless, on a schedule, against real data, at least once
- Someone other than the author has read the output and understood it
Next Steps
A harness is only as good as the tools it can reach.
Running agents in production and hitting something not covered here? Get in touch.