How to Build an Agent Harness That Doesn't Waste Your Model

Two papers from this past summer should change how you think about building agents.

In the first, researchers kept the model frozen — same weights, same tasks — and changed only the harness: how older tool results get trimmed as context fills, how stalled work gets handled. Fail-to-pass went from 28% to 49%. Complete solutions went from 43 to 72 (arXiv:2608.26218).

In the second, the same frontier model scored 45.9% on SWE-bench Pro under one scaffold and 55.4% under another — nine and a half points with zero model changes (arXiv:2605.23950).

Most teams shop for the model and inherit the harness. These numbers say that's backwards. The harness is half the solver, and it's the half you actually own.

Every benchmark score is a joint product. The number you read as "the model" is the model and the harness — and only one of them is printed on the box.

The design philosophy is already on this site — why the boring, explicit harness beats the clever one (The Best Agent Harness Is the One You Don't Notice). This is the build guide: what to construct, in what order.

One more reason this stopped being optional. Agents keep running longer — METR's time-horizon work put frontier task length at a seven-month doubling, and their 2026 update argues it's accelerating. A loop that mismanages context or double-fires a retry doesn't get caught at minute two. It compounds at hour three.

Step 1 — Build the interface before the loop

The Princeton SWE-agent team ran the cleanest experiment here: same model, same tasks, and the only variable was the agent-computer interface — how the model reads files, edits code, and sees errors. The custom interface solved 10.7 percentage points more issues than an identical agent dropped into a bare Linux shell (arXiv:2405.15793, NeurIPS 2024). The ablations also showed the sweet spot is narrow: a file viewer showing too few lines starves the model, showing the whole file loses it.

What that means when you're building:

• Budget every tool result. Design output for the reader it actually has — a model with a context budget, not a human with a screen. Window long files. Head-and-tail large logs. Give verbs to reach the rest.
• Match verbs to the task. Search, scroll, and range-read beat cat-everything, for the same reason your own grep beats opening every file.
• Make errors into recovery paths. An error message that names the failed field and the next legal move turns a dead end into a retry. The SWE-agent interface gates writes behind a linter for exactly this: the model learns the syntax from the rejection instead of dying to it.

Interface quality is the highest-leverage harness work, and it's measurable — same model, different interface, double-digit delta.

Step 2 — Put the control loop in code

The loop has five stages: propose, validate, execute, observe, decide. Every one of them should live in harness code that runs whether or not the model agrees with it.

• Retries get idempotency keys. Stamp each logical action with a fresh UUID v4 so a model-initiated retry can't double-fire a side effect. A retry is a distributed-systems event whether the caller is a load balancer or a probability distribution.
• Stop conditions are explicit. Token budget, wall clock, step count, spend cap — written down, in code, checked before each turn. "The model decided it was done" is not a stop condition; it's a hope.
• The decide stage is a policy, not a vibe. Retry on validation failure, escalate to a human on repeated failure, halt on budget exhaustion. If you can't write the policy as a switch statement, you haven't designed it yet.

Step 3 — Gate every tool call before the side effect

Your harness sees every tool call before it runs. That makes it the one place a check actually costs nothing and prevents everything.

The full walkthrough is a separate post, but the shape fits in a paragraph: validate the model's arguments against the tool's declared schema, reject by default on anything with side effects, and feed the violations back as a tool result — models are good at fixing their own calls when you hand them the exact path that failed. The schema already exists for any tool worth calling; if you're probing an unfamiliar server, the MCP Server Probe enumerates every exposed tool with its contract in one handshake.

What a validation gate buys you in the harness specifically: coercion stops. Frameworks love to "help" — fill defaults, cast types, warn and proceed. A harness-level gate rejects instead, because the harness doesn't answer to the model's convenience. It answers to the system the tool is about to mutate.

Step 4 — Treat the transcript as untrusted input

Here's the uncomfortable part. Everything that goes into context — tool results, web pages, log files, and eventually your own summaries of all of the above — is input from an untrusted source. The transcript gets re-read, re-summarized, and re-injected. That's an injection surface, and the harness owns it because the harness does the re-injecting.

But the same job pays twice. Look at where the biggest number in this post came from: the frozen-model study's treatment arm was context hygiene — mechanically shortening older tool results, responding to stalled work. That's what took fail-to-pass from 28% to 49%. The same curation that buys performance shrinks the injection surface, because less stale content re-enters the context at all.

Context curation is the one harness job that pays out twice — once in tokens, once in attack surface.

Practical rules: pin the system prompt where tool output can't overwrite it. Summarize by policy, not by the model's mood. Cap what gets re-injected from memory stores. Never let a tool result promote itself into instructions.

Step 5 — Make partial progress legible

Long-horizon runs will crash, halt on budget, or get interrupted. The harness should be able to answer, from its own records: what happened, what state got left behind, what's safe to resume.

• The journal is the source of truth. Append-only, one record per logical action: id, intent, arguments, outcome, artifacts. The transcript is a conversation; the journal is the ledger.
• Resume uses the same idempotency keys. A step that already committed its side effect is skipped, not re-run. This is why the keys in Step 2 aren't optional.
• State outlives the run. A checkpoint that only exists inside the model's context window isn't a checkpoint. It's a memory of a checkpoint.

If your harness can't produce a journal, it can't resume safely, and every long run is one crash away from starting over — or worse, from half-repeating itself.

Step 6 — Enforce permissions structurally, then test the whole thing

Permissions live in code, checked at the moment of execution: an allowlist the model can read but not edit, destructive operations defaulting to off, egress rules that don't bend when the model insists. Prompt-level permission language is a suggestion. The Agent Config Checker catches the config-layer version of this mistake; if you want a scored read on how bounded your agent actually is, the Agent Boundary Assessment takes minutes and tells you where the prompt-only enforcement lives.

Then treat the harness like the software it is:

• Replay transcripts against harness changes. You have journals from Step 5 — they're regression fixtures. A harness change that silently alters old runs is a bug you can catch before production.
• Record the harness version next to the model version. In every eval, every before/after comparison, every benchmark number you cite internally. The disclosure paper's whole point is that scores don't survive harness changes — including your scores.

What a good harness doesn't fix

Be honest about the limits, because the failure modes change shape:

• It can't make a weak model strong. A harness stops a strong model from being wasted. If the base capability isn't there, no scaffold saves you — and no amount of interface polish will.
• Every gate costs autonomy. Over-gate the loop and your agent becomes a ticket queue that needs a human per step. Count the gates; keep the ones that guard irreversible side effects, and be suspicious of the ones that just add friction.
• Your numbers stop being comparable. The moment you change the harness, your historical evals describe a different system. That's not a reason to avoid changing it — it's a reason to version it.
• Harness guardrails don't replace least privilege. A validation gate is not a substitute for a tool account that can only do what the tool needs. Defense in depth, or the gate becomes the single point that matters.

The loop you own

Model quality is rented. You pick the best you can afford, and the labs improve it out from under you every quarter. The harness is the part you own — and the evidence says it's worth half the outcome.

Run one check this week. Open the harness you actually run — homegrown or shipped — and answer three questions in writing. Where does the stop decision live? What happens to a failed tool call? What state survives a crash at step 3 of 7? If any answer is "the model decides" or "nothing," that's your first fix.

Build the loop like the score depends on it. The numbers say it does.