What is agentic engineering? Shipping code with agents
Agentic engineering is building production software with coding agents that write and run code. How it differs from vibe coding, with a runnable test loop.

Agentic engineering is the practice of building software with coding agents (Claude Code, Codex, Gemini CLI and similar tools) that both write code and run it, while a human engineer stays responsible for what ships. The engineer’s job moves from typing code to setting the goal, giving the agent the right tools and context, and verifying the result to a production standard.
The term is Simon Willison’s working definition in his Agentic Engineering Patterns guide: “I use the term agentic engineering to describe the practice of developing software with the assistance of coding agents.” The rest of this post unpacks what that means in practice, and where it stops.
What is a coding agent, exactly?
Willison’s definition of an agent is short enough to memorize: “Agents run tools in a loop to achieve a goal.” The agent is ordinary software. It calls a model with your prompt and a list of tool definitions, runs whatever tool the model asks for, feeds the result back, and repeats. I walk through that loop in what an AI agent is, including a version built from scratch in Python.
A coding agent is the same loop with one extra tool: something that executes code. Willison calls code execution “the defining capability that makes agentic engineering possible”, because without it anything a model writes is text you still have to test by hand. With it, the agent can run the tests, read the failure, and try again until the software demonstrably works.
That one capability is why the field changed between 2023 and 2026. Autocomplete suggested lines. A coding agent runs pytest, reads the traceback, edits three files and runs it again.
How is agentic engineering different from vibe coding?
Andrej Karpathy coined “vibe coding” in February 2025 for prompting a model to write code while you “forget that the code even exists”. Willison argues the term is most useful in that narrow sense: unreviewed, prototype-quality code. Agentic engineering is the other end. You still use agents for most of the typing, but you hold the output to the standard you would hold your own code to.
The difference is accountability, and it shows up in three places:
- Who defines done. In vibe coding, done is “it seems to work”. In agentic engineering, done is a test suite, a type check, a lint run and a review that you defined before the agent started.
- Who reads the diff. A vibe coder may never open it. An agentic engineer either reads it or has built enough automated checks to know which parts don’t need reading.
- Who carries the risk. Willison puts it bluntly in a May 2026 post: for software other people use, “vibe coding is grossly irresponsible because it’s other people’s information.”
The same post admits the line is blurring for him: as agents get more reliable, he is “not reviewing every line of code that they write anymore, even for my production level stuff”, and he finds that uncomfortable. I think that discomfort is the right instinct. The answer isn’t to go back to reading every line; it’s to move the review into checks that run on every change, which is what harness engineering is about. I cover the vibe coding side in what vibe coding is.
What does the engineer actually do?
Willison’s list: “provide our coding agents with the tools they need to solve our problems, specify those problems in the right level of detail, and verify and iterate on the results.” OpenAI’s harness engineering post (February 11, 2026) describes the same shift from inside a team that wrote no code by hand for five months: “We prioritize work, translate user feedback into acceptance criteria, and validate outcomes.”
In concrete terms, the work splits into four jobs.
Specify. Write the goal as something checkable: acceptance criteria, a failing test, an example input and output. Vague goals produce plausible code that solves a different problem. This is where spec-driven development comes from.
Equip. Give the agent the commands, docs and access it needs and nothing more. That means a short AGENTS.md with the build and test commands, scripts that make the app observable, and a sandbox that limits what a bad command can break.
Verify. Run the tests, read the diff, try the feature. The agent can do the first part itself if the tests exist. The human part is checking that the tests test the right thing.
Feed back. Willison makes a point I keep returning to: “LLMs don’t learn from their past mistakes, but coding agents can, provided we deliberately update our instructions and tool harnesses.” Every repeated mistake is a missing line in AGENTS.md or a missing check in CI.
What does the verify loop look like in code?
Here is the part of agentic engineering that a script can show: a spec written as tests, drafts checked against it, and failures fed back as the next prompt. The model is scripted with three canned drafts so this runs offline; in a real agent each draft would come from a model call that receives the failure text.
# verify_loop.py: the loop that separates agentic engineering from vibe coding.
# The "model" is scripted (three canned drafts), so this runs offline with no key.
# What matters is the harness side: run real tests, feed failures back, stop on green.
import textwrap, traceback
TESTS = [ # the spec, written by the human before any code exists
("slugify('Hello World')", "hello-world"),
("slugify(' What is RAG? ')", "what-is-rag"),
("slugify('C++ & Rust')", "c-rust"),
("slugify('')", ""),
]
DRAFTS = [ # what a model might write on attempts 1, 2 and 3
"def slugify(s):\n return s.lower().replace(' ', '-')",
"import re\ndef slugify(s):\n return re.sub(r'[^a-z0-9]+', '-', s.lower())",
"import re\ndef slugify(s):\n return re.sub(r'[^a-z0-9]+', '-', s.lower()).strip('-')",
]
def run_tests(source):
ns, failures = {}, []
exec(source, ns) # ponytail: exec is fine for a demo; a real harness uses a sandboxed subprocess
for expr, want in TESTS:
try:
got = eval(expr, ns)
except Exception:
got = traceback.format_exc(limit=0).strip()
if got != want:
failures.append(f"{expr} returned {got!r}, expected {want!r}")
return failures
feedback = ""
for attempt, draft in enumerate(DRAFTS, 1):
# a real agent would send TESTS + feedback to the model here and get `draft` back
failures = run_tests(draft)
print(f"attempt {attempt}: {len(TESTS) - len(failures)}/{len(TESTS)} tests pass")
for f in failures:
print(" ", f)
if not failures:
print("green, hand to a human for review:\n" + textwrap.indent(draft, " "))
break
feedback = "\n".join(failures) # this text is the next prompt's most useful context
assert not failures, "loop ended without a passing draft"
Output (Python 3.13.11):
attempt 1: 2/4 tests pass
slugify(' What is RAG? ') returned '--what-is-rag?--', expected 'what-is-rag'
slugify('C++ & Rust') returned 'c++-&-rust', expected 'c-rust'
attempt 2: 3/4 tests pass
slugify(' What is RAG? ') returned '-what-is-rag-', expected 'what-is-rag'
attempt 3: 4/4 tests pass
green, hand to a human for review:
import re
def slugify(s):
return re.sub(r'[^a-z0-9]+', '-', s.lower()).strip('-')
Two things in that output are the whole discipline in miniature. First, the failure messages are precise (“returned ‘-what-is-rag-’, expected ‘what-is-rag’”), which is the most useful context a model can get on its next attempt. Second, the loop ends at “hand to a human for review”, not at “merge”. The tests were written by a person who knew that 'C++ & Rust' was an edge case worth pinning down. An agent would not have known that the product cares about it.
What does it look like at team scale?
The most detailed public account is OpenAI’s. Starting from an empty repository in late August 2025, a team that began with three engineers driving Codex produced “on the order of a million lines of code” and about 1,500 merged pull requests in five months, an average of 3.5 PRs per engineer per day, with “no manually-written code” as a rule. They say single Codex runs regularly work on one task “for upwards of six hours”.
The interesting part is what the engineers did instead. When something failed, “the fix was almost never ‘try harder.’” They asked “what capability is missing, and how do we make it both legible and enforceable for the agent?” and then had Codex build that capability. They also say plainly that the result “should not be assumed to generalize without similar investment”.
Mitchell Hashimoto describes the solo version in My AI Adoption Journey (February 5, 2026): whenever an agent makes a mistake, “you take the time to engineer a solution such that the agent never makes that mistake again”, either a line in AGENTS.md or a real tool such as a screenshot script or a filtered test runner.
Where does agentic engineering break?
Weak tests. An agent will satisfy the checks you have. If the tests are thin, it will produce code that passes thin tests. Anthropic’s long-running harness write-up (November 26, 2025) saw Claude mark features complete after unit tests and curl calls while the feature still didn’t work end to end, until it was told to test through a browser “as a human user would”.
Pattern copying. OpenAI found Codex “replicates patterns that already exist in the repository, even uneven or suboptimal ones”. Their team spent every Friday, 20% of the week, cleaning up “AI slop” until they automated the cleanup.
Review debt. Throughput goes up and human attention doesn’t. If nobody owns the review budget, the pile of unread diffs grows until something breaks in production.
Context drift on long runs. Agents that run for hours lock onto bad plans. I covered the evidence in context inertia.
What I would do
From a data-platform seat, the line I would circle is ownership: an agent can write the migration, but a person signs off on the data it touches. So I’d start with the checks, not the agent. Before handing a repo to Claude Code or Codex, I’d make sure one command runs the tests, one runs the linter and type checker, and both finish in minutes. Then a short AGENTS.md that names those commands. Then small tasks with written acceptance criteria, each in its own branch or git worktree.
After that, I’d keep a running list of agent mistakes and turn each repeat into a check or a doc line within the week. I’d keep human review mandatory for anything touching auth, money, personal data or schema changes, and let checks carry the rest. And I’d measure merged-and-not-reverted PRs, not lines generated.
FAQ
Who coined the term agentic engineering?
I can’t point to a single coiner. Simon Willison uses it as the name for responsible, professional use of coding agents in his Agentic Engineering Patterns guide, and contrasts it with Karpathy’s “vibe coding” from February 2025.
Is agentic engineering the same as vibe coding?
No. Vibe coding means accepting code you haven’t reviewed, which is fine for throwaway prototypes. Agentic engineering means the code meets a production bar: tested, reviewed or checked, and owned by an engineer who understands it.
Do I still need to know how to code?
Yes. Willison’s point is that writing code “has never been the sole activity of a software engineer”. Choosing the solution, specifying it, spotting a wrong test and judging a diff all need engineering skill, and agents raise how much of that judgment you need per hour.
Which tools count as coding agents?
Any agent that can edit files and run commands in a loop: Claude Code, OpenAI Codex, Gemini CLI, Cursor’s agent mode and GitHub Copilot’s agent are common examples. What matters is code execution, not the brand.
Sources
- Simon Willison, What is agentic engineering?, Agentic Engineering Patterns (read October 31, 2026)
- Simon Willison, Vibe coding and agentic engineering are getting closer than I’d like, May 6, 2026
- Ryan Lopopolo, OpenAI, Harness engineering: leveraging Codex in an agent-first world, February 11, 2026
- Mitchell Hashimoto, My AI Adoption Journey, February 5, 2026
- Anthropic, Effective harnesses for long-running agents, November 26, 2025
- Andrej Karpathy, the original “vibe coding” post, February 2025