When One AI Agent Stops Being Enough

“Our AI agent worked well at first and now its answers keep getting worse. Do we need a better model?” Usually not. Most organizations start with one agent that is expected to plan, execute, verify, secure and document, because such a setup is easy to launch and easy to explain to the board. At small scale it works. As more systems and decisions pass through the same agent, quality drops.

The fix is a change of architecture: a team of specialized agents coordinated by an orchestrator, often called an agent swarm. Anthropic’s research system, among others, is built this way, with a lead agent distributing work among subagents.¹

The limits of a do-everything agent

A general-purpose agent has to split its attention across every element of its environment with every task. A larger context window does not solve this: models retrieve information from the beginning and end of a text more reliably than from the middle, and the gap grows with length.²

The symptoms are similar in most organizations. Tasks take longer because the agent processes the whole context at every step, including the parts unrelated to the task. Similar requests produce different answers, so the team stops trusting the output and starts checking it by hand. The most expensive symptom shows up only at audit: security, GDPR compliance and documentation checks happen only when someone asks for them, not on a fixed schedule.

Dividing work into specialized roles

Work is split into a few roles with a tightly defined scope, a clear input and output and their own quality standard. A sample split has five roles: the Planner breaks a request into tasks, the Reviewer looks only for defects and standards violations, the Auditor assesses risk, the cleanup role removes dead code and the documentation role makes sure the description matches what was actually built.

Roles are chosen by risk, not by technology. The first candidate is the activity whose omission costs the most, such as a security or compliance review. The second is the activity a general-purpose agent performs least carefully because it always comes at the end of the run, such as documentation. A role makes sense when you can write one sentence describing a correct result and check it without reading the whole piece of work.

The same logic works outside engineering teams:

The principle is shared: narrow each role’s scope until its output becomes easy to check. In customer service this means, for example, that the role reviewing sensitive content does not write the reply but assesses a finished draft against a short list of prohibitions and policy requirements.

Orchestration instead of autonomy

An agent swarm does not mean several independent bots. Each specialist is a narrowed set of instructions with its own constraints and output format, and the orchestrator decides what happens next and who handles it. Nothing runs unsupervised just because it carries a role name. The orchestrator also enforces order: the Reviewer receives no task until the Planner has filled in every field of the handoff format.

The distinction matters most in regulated environments, where the AI Act requires human oversight of high-risk systems.³ A well-designed swarm resembles a structured process in which AI performs successive steps, not a group of independent decision-makers.

Where to place human approval points

A single agent that plans, builds and deploys in one run gives the reviewer one large, mixed output. A swarm produces a narrower, clearer output at each stage, so the approval point can be placed exactly where a decision carries consequences:

• before deploying a security-relevant change,
• before sending a message to a customer,
• before final approval of a document classification.

At the approval point the person should not receive the whole run to read. They receive the output of one role, the criterion to judge it by and any remarks from the Reviewer or Auditor. The decision comes down to whether the output meets the criterion, and its record, with the name of the approving person, goes to the log. That log is later the first document an auditor asks for.

For boards accountable to a supervisory board, a regulator or internal audit this is often a stronger argument than efficiency. AI management standards, led by ISO/IEC 42001, speak the same language: what counts is documented oversight and the ability to explain afterwards what the system did and why it was allowed to continue.

Cost and quality

Anthropic reports that a multi-agent system uses about 15 times more tokens than an ordinary chat with a model, and that on its own research evaluation it achieved a 90.2% advantage over a single agent.¹

• ~15× more tokens than an ordinary chat with a model
• 90.2% advantage over a single agent on Anthropic's research evaluation

A worked example: a contract review that a general-purpose agent performs in one pass goes, in a swarm, through clause extraction, comparison with company policy and a risk review. It costs several times more tokens, but each stage leaves a trace a lawyer can check in minutes instead of rereading the whole analysis. The numbers work out where the result is worth the difference and do not where one agent finishes the task on the first try. Part of the cost also moves to the start, into role design and orchestration logic. In return the system is easier to test and fix role by role, without rewriting the whole agent’s prompts every time part of the process changes.

What costs the most in the first swarm

The most expensive part turns out to be settling exactly what one role hands to the next. As long as the Planner’s output is a loose description, the Reviewer receives something that cannot be checked mechanically and the benefit of the split disappears.

1. Handoff format before the number of roles. Three fields are enough to start: task scope, acceptance criterion and a place for the verification result.
2. Roles added gradually. A new role is added once the handoff format works, because the format is the hardest thing to change.
3. Approval point before an irreversible decision. A swarm with five roles and a single approval point at the very end delivers the same as a general-purpose agent, only more expensively.

The question for an organization past its first pilot is whether its current architecture can carry what the business already expects of it. We run agent architecture reviews and role design as part of the AI Automations & Agents package.

Frequently asked questions