Approval Fatigue Is Why Human-in-the-Loop Fails

"Add a human approval step" is the default answer to almost every agent safety question. It sounds responsible, it is easy to put in a design review, and it is often the first thing a compliance team asks for. It also fails in a predictable way, and recent data from a vendor with every reason to measure it carefully shows how.

What the numbers say

In its announcement that auto mode becomes the default in Claude Code, Anthropic reports that users approve 97% of permission prompts. An earlier figure of 93% circulated in March coverage; the two numbers differ and Anthropic does not explain why, so treat the exact value as approximate. The direction is not in doubt: when a prompt almost always gets "yes", people stop reading it.

The same post describes a study with 1,053 paid testers. A dangerous command was swapped into one permission prompt per session, and human review caught just 13.6% of them, while Anthropic's auto mode classifier caught 89%. Anthropic ran this in a test environment rather than on real projects, and it is a vendor measuring its own product, so we would not quote the 89% as a general benchmark. The 13.6% is the number that matters for this argument: a human who is asked often enough becomes a poor safety control.

None of this means humans should be removed. It means the approval step has to be designed like any other control, with a failure mode in mind.

The failure mode: the gate that cries wolf

Arthur AI puts it bluntly: if a gate fires so often that reviewers stop reading, the threshold is wrong, and a gate that cries wolf trains people to ignore it. Rubber-stamping, they argue, creates the appearance of oversight without the substance.

That last point is the dangerous one for regulated teams. An audit log full of approvals looks like evidence of control. If every approval took one second and 97% of them were "yes", the log proves that a click happened, not that a decision did.

Gate the action, not the agent

The common mistake is to attach approval to the agent: "this agent needs sign-off". Real workloads mix actions of very different weight. The same support agent drafts replies, looks up orders, issues refunds and closes accounts. Treating these identically either drowns the reviewer or leaves the dangerous action under-protected.

A more useful model gates each action on a few properties:

• Reversibility. Can the effect be undone cheaply? A draft can be discarded. A wire transfer or a deleted record cannot.
• Blast radius. One customer, one team, or every tenant?
• Data sensitivity. Does the action read or write personal or regulated data?
• Confidence. Did the agent reach this step through a well-tested path or through an unusual chain of tool calls?

Low-risk, reversible actions run unattended. Anything irreversible waits for a person. Arthur AI makes the same split: approve before acting for irreversible, high-impact actions, and review after acting for lower-risk ones so speed is preserved. Anthropic's auto mode follows a similar logic by routing each tool call through a classifier aimed at irreversible actions, and falling back to manual approval after repeated blocks.

Make the approval worth reading

If a human is going to be the control, give them what they need to be one.

1. Show the consequence, not the call. "Run refund(order=…, amount=…)" is a function signature. "Refund 480 EUR to this customer, original payment 3 days ago, third refund this month" is a decision.
2. Show evidence. Attach the sources, the retrieved documents and the agent's stated reasoning, so a reviewer can disagree without redoing the work.
3. Deny by default. An approval that times out must not proceed. Silence is not consent.
4. Keep the queue small. If one person sees fifty requests an hour, the queue itself is the defect. Fix the thresholds, not the reviewer.

Measure the humans too

Most teams measure their agent and never measure the approval step. At minimum, log every approval, denial, override and timeout, and watch three numbers:

• Approval rate per action type. A type that is approved 99% of the time is a candidate for automation, or for a cheaper review-after-the-fact path.
• Time to decision. Very short response times are a signal of rubber-stamping.
• Post-approval failures. How often does an approved action still turn out wrong? If approvals do not predict outcomes, the gate is theatre.

Those are our opinions on what to track, not a standard. But they are cheap to implement and they turn "we have human oversight" from a claim into something you can test.

Our view

Human-in-the-loop is not a feature you switch on. It is a scarce resource, reviewer attention, that you spend on the few decisions where a person adds something a rule or a classifier cannot. Spend it everywhere and it is gone by lunchtime. The goal is not more approvals. It is fewer, better ones, with evidence attached and a record that shows whether they worked.

If you are designing approval flows for agents that touch money, customer data or production systems, talk to us.