Skip to content

Three subagents are not three researchers

Three copies of one model on one brief and one set of documents are one opinion sampled three times. That is a useful stability check, not three independent researchers. Independence comes from different tasks, sources and models, and it has a price you can measure.

subagents · evals · take

A tutorial pattern is going around. Hand one brief and one folder of documents to three parallel subagents, ask each for a launch strategy, and read the three reports as if three researchers had worked independently. We ran the strong version of that claim against the published evidence and against our own pipeline. Our fact-check of 30 September 2026 came back "partly". The parallel part holds. The independence part does not.

The claim, in its strongest form

The strongest version does not say "three copies are three people". It says three things. A language model is stochastic, so three runs take three paths. Each subagent starts with a clean context window, so none inherits another's drift. And the lead agent gets three drafts to reconcile instead of one. All three points are true.

There is published support for the first. Wang and colleagues (ICLR 2023) showed that sampling several reasoning paths from one model and voting on the answer gives a 17.9% gain on GSM8K. Anthropic's engineering post on its research system (June 2025) reports that its multi-agent setup beat the single agent on an internal research eval. The subagents in that setup had distinct tasks.

If the claim were "run it three times and you get a more stable answer", we would agree and stop here.

Where it breaks: one opinion, sampled three times

The claim is "three different researchers". Independence means errors that do not line up. Kim, Garg, Peng and Garg (ICML 2025) measured this across more than 350 language models. When two models both get an item wrong, they agree on the wrong answer 60% of the time. That holds even when the two models come from different providers.

That is between different models. Kim and colleagues measured different models, not three copies of one, so what follows is our inference, not theirs. We would expect three copies of the same model, with the same system prompt, reading the same documents, to sit at the far end of that curve.

Anthropic's own post shows the failure in practice. Before the team gave subagents distinct tasks, the subagents "performed the exact same searches". The gain Anthropic reports is for the setup with divided tasks, not for that early one.

The honest reading: three same-brief runs give you a stability check on one opinion. They do not give you a second opinion.

What we measured

Inside our own pipeline we ran the same planning job two ways. First as one agent with one brief and one document set. Then as a room of five roles, each with its own task.

Setup A: one agent, one brief, one document set
  time     7 min
  cost     $2.49
  output   3 proposals
  spread   all three anchored on the one platform the model knew best

Setup B: five roles, each with a different task
  time     9.5 min
  cost     $3.55
  output   plans covering six to seven platforms, plus a second audience

The second run took 2.5 minutes and $1.06 more. What it bought was spread. In our reading, that is what independence looks like when you can measure it: not three documents that agree with each other, but a plan that reaches places the single agent never looked. We bought the independence with different roles, and the price and the gain sit in the same table.

One caveat we owe the reader. This is one job, run once each way, in one pipeline. It is a measurement, not a study.

The counterargument we have to name

The vendor guides we read in that fact-check, Microsoft's agent guidance among them, say the same thing: default to one agent. The multipliers are the reason. By Anthropic's figures, an agent uses about 4 times the tokens of a chat, and a multi-agent system about 15 times. Multi-agent pays off when the task genuinely splits, or when it does not fit in one context window.

We agree, and we note what that statement is about. It is a statement about the task, not about the architecture. A task that does not split does not get better by being run three times under three names. A task that does split pays the multiplier for coverage, and the coverage is the thing you were buying.

Division of labor is not the point

A founder of an agent platform has argued that splitting work by role is not, by itself, a reason to run more than one agent. We agree with the core: copying an org chart into a prompt file buys nothing.

Our reasons for separate roles are different, and they are ours, not a published result. The first is context isolation. A role that reads only what it needs cannot be anchored by what it did not read, which is exactly what happened to our single agent and its one favorite platform. The second is a checkable handoff. A result that passes from one role to the next can be tested at the boundary. A single agent's internal reasoning cannot. Both are properties of where you draw the task boundary, not of how many agents you run.

When three copies are the right tool

Use three same-brief runs when:
  - you want to know whether an answer is stable
  - the output is short enough to vote on
  - a wrong answer costs more than 3x the tokens

Do not call them independent when:
  - same model, same prompt, same documents
  - the output is a long plan, not a votable answer
  - you need coverage, not agreement

To buy independence:
  - split the job into roles with different tasks
  - give roles different source sets
  - mix model families where you can
  - add a reconciliation step that compares, not merges

What would change our mind

A measurement. Take three same-brief subagents on one model, score the substantive disagreements between their reports, and show a rate near what Kim and colleagues found between different model families. If someone publishes that, we will rerun our job under their setup and print the correction here, at the same length.

FAQ

Is self-consistency the same thing as three subagents? Close in mechanism, different in use. Self-consistency votes on a short answer. Three report-writing subagents produce long documents nobody votes on. The gain Wang and colleagues measured came from the vote.

Does running three different providers fix the problem? Partly. Kim and colleagues found correlated errors across providers too. Different models help. Different tasks and different sources help more.

Is this an argument against multi-agent systems? No. Anthropic's gain over a single agent is a multi-agent result, measured with a distinct task per subagent. The argument is against calling three copies of one run "independent".

How do we decide between one agent and a room? Ask whether the task splits. If one role can read everything it needs and hold it, use one agent. If coverage or context size is the problem, split by job, give each role its own sources, and pay the multiplier knowing what it buys.

A related case

The same rule runs our products. In Holt, our language-learning product, the Arabic vocabulary bank went through 125 batches, each run as author, then skeptic, then fixer, then skeptic again, before a third full pass over the whole bank. At runtime a separate validator gates every generated card before a learner sees it. In neither step does a role grade its own output. The case is at iloblique.com/holt.

If you have something worth building, we'd like to hear about it.

Sources