What is harness engineering? Making coding agents reliable
Harness engineering is the work of building the tools, checks and docs around a coding agent so it can't repeat a mistake. Origins, practices and a lint demo.

Harness engineering is the work of designing the environment around a coding agent (its instructions, tools, tests, linters, sandboxes and feedback loops) so that it produces correct work without a human fixing it by hand. The rule of thumb behind it: when an agent makes a mistake, don’t just correct the output, change the harness so the agent can’t make that mistake again.
The term took off in February 2026. Mitchell Hashimoto used it in My AI Adoption Journey on February 5, and six days later OpenAI published Harness engineering: leveraging Codex in an agent-first world, an account of a product built with zero lines of hand-written code. This post covers what both say, how it relates to context engineering, and a small example of the most useful technique in the OpenAI write-up.
Where does the term harness engineering come from?
A harness is the software that turns a model into an agent: the loop, the tools, the permissions, the context management. I explain that layer in what an agent harness is. Harness engineering is what you do to that layer for your own project.
Hashimoto’s definition is the clearest one I’ve read: “It is the idea that anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again.” He was modest about the name (“I don’t need to invent any new terms here; if another one exists, I’ll jump on the bandwagon”) and described two forms it takes:
- Better implicit prompting. For the agent running the wrong commands or finding the wrong APIs, update AGENTS.md. He links a Ghostty AGENTS.md where “each line in that file is based on a bad agent behavior”.
- Actual, programmed tools. Scripts to take screenshots, run filtered tests and so on, plus an AGENTS.md line so the agent knows they exist.
OpenAI’s post, by Ryan Lopopolo, gave the idea a team-sized case study and a slogan: “Humans steer. Agents execute.”
What did OpenAI’s harness engineering experiment show?
The setup: an internal product, first commit in late August 2025, and a rule that humans never write code directly. Five months later the repository held “on the order of a million lines of code”, about 1,500 merged PRs, “3.5 PRs per engineer per day” from a team that grew from three engineers to seven, and a product with “hundreds of users internally”. They estimate it took “about 1/10th the time it would have taken to write the code by hand”.
The post is worth reading in full, but the practices reduce to five.
A map, not a manual. They tried one big AGENTS.md and it failed: “Too much guidance becomes non-guidance.” The fix was a root AGENTS.md of “roughly 100 lines” that points into a structured docs/ directory treated as “the system of record”, with design docs, architecture docs, quality grades and execution plans checked in. I cover the file itself in what AGENTS.md is.
Make the app legible to the agent. The app boots per git worktree, Chrome DevTools Protocol is wired into the agent runtime, and each worktree gets its own short-lived logs and metrics stack the agent can query with LogQL and PromQL. With that in place, a prompt like “ensure service startup completes in under 800ms” becomes something the agent can check.
Enforce invariants mechanically. Each business domain follows fixed layers (Types, Config, Repo, Service, Runtime, UI) and code may only depend “forward”. Custom linters and structural tests enforce it. The detail I’d copy first: “Because the lints are custom, we write the error messages to inject remediation instructions into agent context.”
Agents review agents. Codex reviews its own change, requests more agent reviews, answers feedback and loops “until all agent reviewers are satisfied”. Humans “may review pull requests, but aren’t required to.”
Garbage collection. Codex copies whatever patterns exist, good or bad. The team used to spend every Friday cleaning up “AI slop”. They replaced that with “golden principles” in the repo and recurring background Codex tasks that open small refactoring PRs, most “reviewed in under a minute and automerged”. In their words: “This functions like garbage collection.”
What does a harness-engineered check look like?
Here is the lint idea in about 40 lines: an ast-based check for the forward-only layer rule, run against a toy repo, with error messages written for the agent that will read them.
# layer_lint.py: a custom architecture lint whose error messages are written for an agent.
# Rule (from OpenAI's harness engineering post): inside a domain, code may only depend
# "forward" through Types -> Config -> Repo -> Service -> Runtime -> UI.
import ast
LAYERS = ["types", "config", "repo", "service", "runtime", "ui"]
FILES = { # a toy repo: path -> source. Two of these break the rule.
"billing/types.py": "from dataclasses import dataclass",
"billing/repo.py": "from billing.types import Invoice\nimport sqlite3",
"billing/service.py": "from billing.repo import load\nfrom billing.ui import render_total",
"billing/ui.py": "from billing.service import total",
"billing/config.py": "from billing.runtime import ENV",
}
def layer_of(module):
name = module.split(".")[-1]
return LAYERS.index(name) if name in LAYERS else None
def lint(path, source):
here = layer_of(path.removesuffix(".py").replace("/", "."))
for node in ast.walk(ast.parse(source)):
if isinstance(node, ast.ImportFrom) and node.module:
there = layer_of(node.module)
if there is not None and there > here:
yield (f"{path}:{node.lineno}: {LAYERS[here]} imports {LAYERS[there]} ({node.module}).\n"
f" FIX: {LAYERS[here]} may only import from {', '.join(LAYERS[:here]) or 'nothing'}. "
f"Move the shared code down into a lower layer, or pass it in as an argument. "
f"Do not silence this check. See docs/ARCHITECTURE.md#layers.")
errors = [e for path, src in FILES.items() for e in lint(path, src)]
print("\n".join(errors) if errors else "ok")
print(f"\n{len(errors)} violation(s) in {len(FILES)} files")
assert len(errors) == 2
Output (Python 3.13.11):
billing/service.py:2: service imports ui (billing.ui).
FIX: service may only import from types, config, repo. Move the shared code down into a lower layer, or pass it in as an argument. Do not silence this check. See docs/ARCHITECTURE.md#layers.
billing/config.py:1: config imports runtime (billing.runtime).
FIX: config may only import from types. Move the shared code down into a lower layer, or pass it in as an argument. Do not silence this check. See docs/ARCHITECTURE.md#layers.
2 violation(s) in 5 files
A normal linter stops at the first line of each error. The second line is the harness engineering. It tells the agent the allowed direction, two acceptable fixes, the one fix it must not try (silencing the check), and where the longer explanation lives. That is a prompt, delivered at the exact moment it’s relevant, by a tool the agent already runs. It also never goes stale the way a paragraph in a long instruction file does, because CI fails if the rule stops matching the code.
How is harness engineering different from context engineering?
They overlap, and I don’t think the boundary is worth fighting over. Context engineering is about what goes into the model’s window on each call: which instructions, which files, which tool results, how history is compacted. Harness engineering is about the system the agent works inside: what it can run, what checks it, what fails the build, and what gets fixed automatically.
A practical test: if the change is a sentence the model reads, it’s context engineering. If the change is code that runs whether or not the model reads anything, it’s harness engineering. Hashimoto’s AGENTS.md lines sit on the first side, his screenshot scripts on the second, and OpenAI’s linters with remediation messages are both at once.
Where does harness engineering break?
It’s project-specific. OpenAI is explicit that their end-to-end autonomy “depends heavily on the specific structure and tooling of this repository and should not be assumed to generalize without similar investment”. A harness is an investment per codebase.
Loose merge gates are a trade-off. The same post says the repo runs with “minimal blocking merge gates” and handles test flakes with re-runs, and that this “would be irresponsible in a low-throughput environment”. I wouldn’t copy that part into a payments service.
Checks can be gamed. An agent under pressure to make a check pass may edit the check. Anthropic’s long-running harness post uses a JSON feature list and tells the agent “It is unacceptable to remove or edit tests”. I’d go further and make test and lint config files read-only to the agent, or flag any diff that touches them for human review.
Unknown long-term behavior. OpenAI says it doesn’t yet know “how architectural coherence evolves over years in a fully agent-generated system”. Nobody does.
What I would do
I’d start the way Hashimoto describes, because it costs nothing: keep a list of every agent mistake, and when one repeats, fix the harness instead of the output. Most of the early fixes are AGENTS.md lines. The second wave is scripts that let the agent see what it built: start the app, take a screenshot, tail the logs, run one test file.
Then I’d pick the two or three architecture rules that matter most and turn them into lints with remediation messages like the one above. From my years on data platforms, my first candidates would be “no raw SQL outside the repository layer”, “every new table has an owner and a retention rule”, and “no personal data in logs”. Those are rules people forget under deadline pressure, and agents forget them with no pressure at all.
Last, I’d schedule a weekly cleanup agent with a narrow brief (dead code, duplicated helpers, docs that no longer match the code), with small PRs that a human can approve in a minute. Pair it with agent sandboxing so none of this runs with more access than it needs.
FAQ
Who coined harness engineering?
Mitchell Hashimoto used the term on February 5, 2026, saying he had “grown to calling this ‘harness engineering’” and would switch if a better term existed. OpenAI’s February 11, 2026 post made it widely known.
Is harness engineering only for coding agents?
The term grew up around coding agents, but the idea applies to any agent: give it checks that tell it when it’s wrong, and fix the system rather than the output when it fails. Coding is just where checks (tests, types, lints) are cheapest to build.
Do I need OpenAI’s setup to do harness engineering?
No. A short AGENTS.md, one fast test command and a habit of turning repeated mistakes into checks is harness engineering. OpenAI’s observability stack and agent review loops are what it looks like after months of investment.
What is the difference between a harness and harness engineering?
The harness is the agent runtime itself, such as Claude Code or Codex. Harness engineering is the work you do around it for your project: docs, tools, lints, tests and cleanup jobs that shape what the agent can do and how it learns it was wrong.
Sources
- Mitchell Hashimoto, My AI Adoption Journey, February 5, 2026
- Ryan Lopopolo, OpenAI, Harness engineering: leveraging Codex in an agent-first world, February 11, 2026
- Anthropic, Effective harnesses for long-running agents, November 26, 2025
- Related on this site: what agentic engineering is and context inertia in long-running agents