Approve-every-output is not a control: what our human gate was actually shown

A human who approves every AI output feels like safety. It is not a control. In the largest field sample we know of on this question, it is the lower-yield design. And in our own system, the person at the gate could judge only what the screen rendered. Four times in one stretch, the screen showed the wrong thing.

Below: the number, the four failures, and what we changed.

The number

Stanford's Digital Economy Lab studied 51 production AI deployments across 41 organizations for The Enterprise AI Playbook. This is the line that matters to anyone designing a review step:

“Escalation-based models (AI handles 80%+ autonomously, humans review exceptions) delivered 71% median productivity gains versus 30% for approval models.”

The authors add a caveat, and we repeat it every time: “This may, in part, reflect different types of tasks addressed.” They also say their sample leans toward successful deployments. So this is an association, not a controlled result.

The same report has two more figures. In 46% of agentic implementations, a human reviews or approves each output before action. And the most frequent source of resistance was Legal, HR, Risk and Compliance, at 35%, ahead of internal end users at 23%.

Our read: nearly half of what gets called an agent is a person clearing a queue. The queue exists largely to reassure the functions that can stop the project. That is a real need. It is not the same thing as catching failures.

What our gate was actually shown

We run a generative video pipeline with a human approval step at the end. We built that step as the safety net. Here is what reached it.

1. An Approve button that was already disabled

An automated audio check failed any video under −30 LUFS as “noise pretending to be audio”. That threshold is right for narrated clips. But both videos waiting at the gate were generated footage with −33 to −34 LUFS of room noise and no voice, because the sound stage did not exist yet.

In our own incident note, that was “a 100% false block on the only class of video the pipeline currently produces, escalated to a human who had no button to act with.”

The human was in the loop. The check had already voted, and its vote outranked theirs.

# before
if loudness_lufs < -30: block("noise pretending to be audio")

# after (simplified sketch)
audio_planned = any(scene.has_audio
                    for scene in plan.scenes
                    if not scene.dropped and not scene.superseded)

if true_peak_dbtp > 0:      block("clipping")   # a defect whatever was planned
elif loudness_lufs < -30:
    if audio_planned:       block("too quiet")
    else:                   warn("under range, no audio planned")

A threshold is a claim about what the artifact was supposed to contain. Clipping stayed a hard block, because it is a defect of the track no matter what anyone planned.

2. A fix the generator never read

A reviewer rejected a video: the visuals had nothing to do with the text. That was accurate. The scene was then rewritten three times: stacks of paper, boxes, a rack of cables. Each time, the generator received a prompt pack written five days earlier. The code looked up a scene's prompt only through a reference into that pack and never read the scene's own text.

“Five generations at 9.6 credits each, byte-for-byte the same footage.”

Two fields in the same database had disagreed for five days, and nothing compared them. The step producing the scenes had flagged the mismatch in its notes, where it read as an opinion rather than a measurement. The rejection was correct but aimed at the wrong layer. The reviewer could see the output. They could not see which text had actually gone out.

# prompt resolution now (simplified sketch)
prompt = scene.inline_prompt or pack[scene.prompt_ref]   # the scene's own text wins

if scene.inline_prompt and scene.prompt_ref \
   and scene.inline_prompt != pack[scene.prompt_ref]:
    audit(scene.id, sent=prompt)   # one row: which text reached the generator

# plan schema: a scene carries a reference OR an inline prompt

3. A tool at its cap that went shallow instead of failing

Another step in the same system uses a metered search tool with a 1,000-call monthly cap. Usage stood at 1,005. Nothing errored. The next two runs returned four and five findings with no web search at all, falling back to videos with 48 to 167 views. After the plan was raised, the next run returned six findings in six minutes from noticeably deeper sources.

The counts looked normal the whole time. “The difference shows in the citations rather than in the counts, which is exactly why the counts warned nobody.” The step recorded its own degradation honestly, in a document nobody read.

alarm:   provider_usage >= monthly_cap         # watch the quota, not the symptom
bypass:  search -> lesser tool | other model   # written in advance: worse, but done

4. “Sent” shown as “published”

We added a watchdog for handovers to an external service. In its first minute, it found eight items handed over hours earlier with no verification job at all. Reading them back from the other system confirmed seven as genuinely live. The eighth had been rejected with “authentication failure”: a dead connector our system had never mentioned. In one sweep, the count of confirmed items went from 6 to 13.

“‘Sent’ is a fact about your request; ‘published’, ‘paid’ or ‘delivered’ is a fact about the other system, and only a read-back turns one into the other.”

This is a known category, not a private embarrassment. AWS Builder Center puts it as “you can't prompt your way around a tool that lies”. Telerik files it under silent failure, where “every span looks successful”.

on handover:  schedule verify(item)
verify:       read the item back from the other system
              retry every 30 min, five distinct outcomes
              no confirmation at 6 h -> explicit failure state
sweep:        any handover without a verify job gets one
watchdog:     raise anything unconfirmed past 6 h

If you want to know the size of your own gap, a generic version of the question looks like this:

SELECT count(*), min(handed_over_at)
FROM   handovers
WHERE  confirmed_at IS NULL;

The pattern, in our view

A human was near all four failures. None of the four fixes was more review. Each fix was a comparison the system could make on its own: the plan against the check, the intended prompt against the sent one, usage against the cap, our request against the other system's record.

The design we now prefer: the system handles routine cases and escalates exceptions to a person, with the evidence attached. That means which text went out, what the plan said, and what the other side confirmed. That is the escalation model from Stanford's sample. A person looking at an exception next to its evidence is making a judgment call. A person approving a rendered preview is performing a ritual.

We kept the human gate. We stopped treating it as the control.

What would change our mind

A study that holds task type constant and finds approval models matching escalation models. Stanford's authors say the gap may partly reflect different tasks. If that explains most of it, the productivity argument weakens. The visibility argument stands either way: a gate still sees only what the system renders.

FAQ

Is this an argument for removing humans from AI workflows? No. It is about where they sit. The escalation designs in Stanford's sample still have humans. They review exceptions rather than every output. Our own gate stays. What changed is what it gets shown.

Doesn't per-output approval keep Legal and Compliance happy? Often it does, and that matters: the same report names those functions as the most frequent source of resistance, at 35%. Our view is that they are better served by a record of what was sent and what was confirmed than by a queue of clicks.

How do I check whether my system has the “sent is not published” gap? Count everything handed to an external service that nothing has read back, and look at the age of the oldest item. Our first sweep found eight, and one of them was dead.

Why didn't anything warn us when the search tool hit its cap? Output counts stayed in range. The damage showed in source quality, and nothing measured that. Alarm on the provider's usage, not on the output.

Related case

RE Intelligence runs on the same principle, stated as “Postgres owns the truth, AI owns the story”. Every number comes from a deterministic query. Price forecasts are archived, graded against what actually happened, and the accuracy is shown to the user instead of hidden. The case is at iloblique.com/re-intelligence.

Failure #2 also exists as a short two-person sketch: {{anchor_url}}