Harnesses
Harness Engineering is now a discipline. Here is what that means.
The model is becoming commoditized. The harness — everything around the model — is where the engineering value lives. Three data points prove it, and the implications are immediate.
- Published
- 27 Aug 2026
- Reading
- 7 min
- Class
- harnesses
half-life 90dfrom 27 Aug 2026
The question used to be "which model should we use?" It is still a valid question, but it is no longer the interesting one. The interesting question is: what happens around the model?
In February 2026, Mitchell Hashimoto — the person who built Terraform, Vagrant, and Packer — gave it a name. He called it Harness Engineering. Since then, OpenAI, Anthropic, LangChain, Google, and Martin Fowler have all converged on the same framing. When that many independent voices land on the same term within six months, it is not a trend. It is a recognition of something that was already true.
The core idea is this: an agent is a model plus a harness. The harness is everything that is not the model — the orchestration loop, tool calling, memory, context management, guardrails, error handling, permissions, observability, and state persistence. The model produces tokens. The harness decides what those tokens can touch, what happens when one of them is wrong, and what the model gets to see on the next turn.
If you have built an agent, you have built a harness. The question is whether you built it on purpose.
The evidence is not subtle
Three things happened in 2026 that made the discipline impossible to ignore.
The benchmark that changed the conversation
Artificial Analysis launched the Coding Agent Index in May 2026. It is the first independent benchmark that evaluates full agent stacks — model and harness together — rather than models in isolation. The methodology is straightforward: three benchmark suites (DeepSWE, Terminal-Bench v2.1, SWE-Atlas-QnA), pass@1 averaged over three attempts, with cost and token tracking.
The headline finding: the same model running on different harnesses produced a 32× cost difference with nearly identical code quality. Claude Opus 4.7 on one harness cost $0.07 per task. On another, $2.26. Same model. Same benchmark. Same quality threshold.
That number ended the debate about whether harness choice matters. It matters more than most model choices.
The research that proved harness-only gains
The HarnessX project (Darwin-Agent) took a different approach. They froze three models and rewrote only the scaffolding — the harness layer. The result: an average +14.5% pass@2 gain, with peaks of +44%. On a 9B parameter model, ALFWorld success went from 53% to 97%. No fine-tuning. No new weights. Just better engineering around the model.
This is the kind of result that changes how teams allocate resources. If you can get a 44% improvement by rewriting the harness, the marginal return on improving the model further drops significantly.
The open-source release that made it real
Y Combinator open-sourced QM (Quartermaster) in July 2026. It is the internal multi-agent harness YC uses across accounting, legal, events, and engineering — including building QM itself. The announcement hit 413,000 views and the repository gathered 14,000 GitHub stars in four weeks.
QM is significant not because it is the best harness (it may or may not be), but because it is the first major open-source harness designed for company-wide deployment. Every employee gets an isolated workspace with scoped memory, files, permissions, and sandboxed execution. It drives multiple coding agents interchangeably — Pi, OpenCode, Codex, Claude Code — with no vendor lock-in. It has admin controls, three security postures, and PostgreSQL persistence for everything.
The codebase is 74,814 lines of TypeScript with 90,196 lines of tests. More test code than product code. That ratio tells you something about what production-grade harness engineering looks like.
The three layers
The industry has settled on a useful hierarchy:
| Layer | What it controls | Scope |
|---|---|---|
| Prompt Engineering | Instructions the model receives | Single turn |
| Context Engineering | What the model sees and when | Session |
| Harness Engineering | Everything else | System |
Harness engineering encompasses the other two. A harness that manages context poorly will override the best prompt engineering. A harness with no observability makes it impossible to tell whether your prompts are working. The layers are useful for discussion, but in practice they are inseparable.
What this changes for developers
If you are building agents, the implication is direct: stop treating the harness as glue code. The loop, the tool definitions, the context management strategy, the error recovery, the permission model, the observability layer — these are not implementation details. They are the product.
A few concrete shifts:
Tool definitions are prompt. A parameter called q with no description will be misused. The same parameter called query with one sentence explaining it accepts a regular expression will not be. Every tool schema is documentation the model reads. Treat it with the same care.
Error messages are the recovery mechanism. A tool that returns Error: invalid input gives the model nothing to act on. One that returns path must be absolute; you passed "src/main.py", try "/app/src/main.py" gets corrected on the next turn without a human.
Context management is the survival mechanism. An agent that walks into the context limit dies at the least convenient moment. The harness has to decide what to keep, what to drop, and what to summarise. Get this wrong and your agent cannot work for an hour. Get it right and it can work for days.
What this changes for businesses
The business implications are less obvious but more significant.
The "which model" question is now secondary. The Artificial Analysis Index shows that harness choice can matter more than model choice for both cost and quality. A team spending weeks evaluating LLMs while using a default harness is optimising the wrong variable.
95% of enterprise AI agents die in prototype. The ones that survive to production have something in common: someone engineered the harness deliberately. The harness is what gets an agent from "works on my laptop" to "runs unattended in production." That transition is the hard part, and it is a harness engineering problem.
A new career track is emerging. "Harness Engineer" roles are appearing at AI-forward companies. The skill set is distinct from both ML engineering (training models) and traditional software engineering (building applications). It is closer to platform engineering — building the infrastructure that other people's code runs on — but with the added complexity that the "code" is partially non-deterministic.
VCs are funding the harness layer. The investment thesis is straightforward: models are commoditising, so the value is moving to the layer that makes them useful. Harness-layer startups are getting funded on the premise that the harness is the product, not the model.
The landscape right now
The harness ecosystem is fragmenting along use-case lines:
| Category | Examples | Focus |
|---|---|---|
| Coding Agent Harnesses | Claude Code, Cursor, Codex, OpenCode, Pi | Developer in a repo |
| Personal Assistants | Hermes, OpenClaw | Single power user |
| Company/Team Harnesses | QM | Whole organisation, multiplayer |
| Framework / Orchestration | LangGraph, CrewAI, AutoGen, Dify | Building custom agents |
| Eval / Observability | Arize Alyx, Artificial Analysis, BenchLM | Measuring harness quality |
The coding agent harnesses are the most mature. The company/team category is the newest and, arguably, the most important — it is where the multiplayer governance problem gets solved.
What to do about it
If you are a developer: build a harness on purpose. Write the forty-line loop once to understand it, then engineer the six pieces that make it production-ready. Read the Artificial Analysis Index. Compare your harness against the benchmarks.
If you are a business: stop evaluating models in isolation. Evaluate model-plus-harness pairs. Ask your AI vendor what happens when the context fills, what a tool error looks like from the model's side, and whether you can resume a run after a crash. Those questions tell you more about production readiness than any model benchmark.
The discipline is here. The evidence is public. The tools are open source. The question is whether you treat the harness as engineering or as an afterthought.