What is human-in-the-loop AI? Approvals for AI agents
Human-in-the-loop AI pauses an agent so a person can approve, edit or reject an action. Where to put the checkpoints, how to avoid approval fatigue, with code.

Human-in-the-loop (HITL) AI is a design where an AI system pauses at defined points so a person can approve, edit or reject what it is about to do before it continues. For AI agents, that usually means a gate in front of risky tool calls (sending email, moving money, deleting data, pushing code) plus a way for the agent to hand control back when it is stuck. The hard part isn’t adding the pause; it’s choosing which actions get one, so people still read the prompts they’re shown.
Why do AI agents need a human in the loop?
Because an agent acts, and some actions can’t be undone. OpenAI’s December 2023 paper Practices for Governing Agentic AI Systems says some decisions are too important to delegate “if there is even a small chance that they’re done wrong”, with an irreversible large financial transaction as the example, and calls requiring a user to authorize such actions, “keeping a ‘human-in-the-loop’”, a standard way to limit the worst failures of agentic systems.
There are three distinct reasons to put a person in the loop, and they lead to different designs:
- Risk. The action is irreversible or expensive: payments, deletions, external messages, production deploys.
- Uncertainty. The agent is stuck or unsure: ambiguous intent, repeated failures, missing information.
- Accountability. Someone has to own the decision, by policy or by law, whatever the model’s accuracy.
Anthropic’s Building effective agents covers the second: agents “can then pause for human feedback at checkpoints or when encountering blockers.” OpenAI’s practical guide to building agents names two triggers, “exceeding failure thresholds” and “high-risk actions”, and gives canceling orders, authorizing large refunds and making payments as examples of the latter.
What are the levels of human involvement?
Engineers tend to use three terms, loosely:
- Human-in-the-loop: the system waits for a person before a step happens. Approval gates, review queues, “Do you want to proceed?”
- Human-on-the-loop: the system acts on its own while a person monitors and can interrupt. Dashboards, alerts, a stop button.
- Human-out-of-the-loop: the system acts with no review in the moment; people audit afterward, if at all.
Real products mix them per action. Claude Code is a clear example. Its permissions docs describe modes from default (“Prompts for permission on first use of each tool”) through acceptEdits (file edits approved automatically) and plan (explore read-only, edit nothing) to bypassPermissions, which skips prompts and which the docs say to use only “in isolated environments like containers or VMs.” Inside any mode, rules decide per tool call, and they are evaluated “in order: deny, then ask, then allow”, so a broad deny can’t be overridden by a narrower allow.
How do you implement human approval for agent tool calls?
The pattern is the same across stacks: classify each tool call, then allow it, deny it, or pause the run and ask. This script implements a small policy in the deny, ask, allow order, with a fallback by risk level using the low, medium and high ratings OpenAI’s guide suggests assigning to tools. The human is simulated; in a real app that function is a UI.
# hitl.py: a human-in-the-loop policy for agent tool calls. Python 3 stdlib only.
# Rules are checked deny -> ask -> allow (first match wins), then a risk default
# applies. Anything that pauses for a human is logged with who decided.
from fnmatch import fnmatch
RULES = {"deny": ["bash:rm -rf *", "payments.refund:*:amount>50000"],
"ask": ["bash:git push*", "email.send:*", "payments.refund:*"],
"allow": ["bash:git status*", "bash:pytest*", "files.read:*"]}
RISK = {"files.read": "low", "bash": "medium", "email.send": "high", "payments.refund": "high"}
def decide(tool, arg, amount=0):
key = f"{tool}:{arg}"
for verdict in ("deny", "ask", "allow"):
for rule in RULES[verdict]:
pattern, _, cond = rule.partition(":amount>")
if fnmatch(key, pattern) and (not cond or amount > int(cond)):
return verdict, rule
return ("ask" if RISK.get(tool, "high") == "high" else "allow"), "risk default"
def human(tool, arg): # stand-in for an approval UI
return "approve" if tool != "email.send" else "reject"
calls = [("files.read", "src/app.py", 0), ("bash", "pytest -q", 0), ("bash", "git push origin main", 0),
("payments.refund", "order-1182", 2_400), ("payments.refund", "order-1190", 80_000),
("email.send", "all-customers", 0), ("bash", "rm -rf build/", 0), ("bash", "ls", 0)]
asked = 0
for tool, arg, amount in calls:
verdict, why = decide(tool, arg, amount)
outcome = verdict
if verdict == "ask":
asked += 1
outcome = f"ask -> human {human(tool, arg)}s"
print(f"{tool + ' ' + arg:<36} {outcome:<24} ({why})")
print(f"\n{asked} of {len(calls)} calls needed a person")
Output (Python 3.14.5):
files.read src/app.py allow (files.read:*)
bash pytest -q allow (bash:pytest*)
bash git push origin main ask -> human approves (bash:git push*)
payments.refund order-1182 ask -> human approves (payments.refund:*)
payments.refund order-1190 deny (payments.refund:*:amount>50000)
email.send all-customers ask -> human rejects (email.send:*)
bash rm -rf build/ deny (bash:rm -rf *)
bash ls allow (risk default)
3 of 8 calls needed a person
Two details are worth copying. The refund over 50,000 is denied outright even though a broader rule says refunds should ask, because deny is checked first; no approval can override it, so a tired reviewer can’t click through it. And ls falls through to the risk default for bash, which is medium, so it runs; a stricter policy would make unknown shell commands ask.
Frameworks build this in. The OpenAI Agents SDK lets a tool declare needs_approval, either True or a function that decides per call. When approval is needed, the run pauses and returns interruptions; you call state.approve(...) or state.reject(...) and resume from a serialized RunState, which can wait hours for a person. Its approval callables “fail closed”: if the arguments are malformed or missing, the call requires manual approval instead of guessing. That’s the default I’d want in any home-grown version too.
What is approval fatigue, and how do you avoid it?
If every action needs approval, people stop reading and start clicking yes. The OpenAI governance paper names the problem: approval is harder to make meaningful when the user “must approve many decisions and thus must make each approval quickly.” The EU AI Act names the human side of it, automation bias, in Article 14, which requires that people overseeing high-risk systems are enabled “to remain aware of the possible tendency of automatically relying or over-relying on the output.”
Ways to keep approvals meaningful:
- Ask rarely. In the script, 3 of 8 calls needed a person. Read-only and reversible actions shouldn’t ask at all.
- Show the consequence, not the call. “Send this email to 4,812 customers” is reviewable;
email.send({"list": "all-customers"})is not. - Make approvals scoped. Claude Code’s “Yes, and don’t ask again” applies to a specific command pattern, not a whole tool, and the Agents SDK scopes approvals to a call ID unless you choose
always_approve. - Deny the unrecoverable. Some actions shouldn’t be approvable from the agent’s flow at all. Make a person do them through a separate path.
- Review afterward too. Log every approved action with who approved it, and sample them. The paper’s other practices, legible agent activity and automatic monitoring, are the human-on-the-loop half.
What does the law say about human oversight of AI?
For systems classed as high-risk under the EU AI Act, Article 14 requires that they “can be effectively overseen by natural persons during the period in which they are in use.” The people overseeing them must be able to understand the system’s limits, interpret its output, decide not to use it or override it, and “interrupt the system through a ‘stop’ button or a similar procedure.” For the identification systems listed in Annex III point 1(a), no action can be taken on an identification unless at least two people have separately verified it. The application dates are staggered; the text published on the Act’s reference site lists 2 December 2027 for Annex III high-risk systems. Most coding or support agents won’t be high-risk, but the requirements are a good checklist anyway. The EU AI Act for developers covers who it applies to.
What I would do
Start by listing every tool your agent has and rating each one: read-only, reversible write, or irreversible. Read-only runs without asking. Reversible writes run with logging. Irreversible actions ask, with a clear summary of the consequence, and the most dangerous ones are denied in code with no override from inside the agent.
Then add the uncertainty trigger: after two or three failed attempts at the same step, the agent stops and asks instead of trying a fourth time. Put the policy in the harness, not the prompt, because a prompt is a request and the model can be talked out of it by text in a tool result, which is how prompt injection works. From the privacy-compliance side of my Walmart years, the line I’d circle is the audit log: an approval nobody can trace afterward is worth very little. AI guardrails and agent sandboxing are the other two layers.
FAQ
What does human-in-the-loop mean in AI?
A system design where a person reviews and approves, edits or rejects some of the AI’s outputs or actions before they take effect, instead of the AI acting fully on its own.
What is the difference between human-in-the-loop and human-on-the-loop?
In the loop, the system waits for a person before acting. On the loop, the system acts on its own while a person monitors and can intervene or stop it.
Which agent actions should require human approval?
Irreversible or high-impact ones: payments and refunds, deleting data, sending messages outside the organization, deploying to production, and changing permissions. Also any step where the agent has failed repeatedly.
Does human-in-the-loop make AI agents safe?
It lowers risk for the actions it covers, but only if reviewers have enough context and few enough prompts to judge each one. It works best alongside deny rules, sandboxing and monitoring.
Sources
- Shavit, Agarwal, Brundage et al., OpenAI, Practices for Governing Agentic AI Systems, December 2023
- OpenAI, A practical guide to building agents, 2025
- OpenAI Agents SDK, Human-in-the-loop, as of October 31, 2026
- Anthropic, Building effective agents, December 19, 2024
- Anthropic, Claude Code: Configure permissions, as of October 31, 2026
- EU Artificial Intelligence Act, Article 14: Human Oversight, text as published on artificialintelligenceact.eu, accessed October 31, 2026