Playbooks · Coding tasks
METR: Anthropic's AI R&D sped up about 1.9 times
Written November 6, 2026
An independent audit says AI help accelerated Anthropic's R&D benchmarks 1.9x — heavily redacted, low confidence on the AI share. What it proves.
What METR measured
From 12 August to 29 October 2026, METR worked inside Anthropic with high access to internal data on the company's AI research and development process, including benchmark scores for internal models, training compute, and interviews with staff. The headline finding is about benchmark scores: measured by AECI, Anthropic's version of the Epoch Capabilities Index, benchmark scores for Anthropic's models have been growing since October 2025 at an estimated 1.9 times the pre-October 2025 rate, with a 95% confidence interval of 1.5 to 2.8 times. Put another way, METR estimates Anthropic made roughly two years of historical progress on benchmarks over the last year. The summary from METR is more cautious than the headline number: the team would not be surprised if the true acceleration was closer to 1.3 times or 3 times the historical rate.
What the audit found
METR is confident the acceleration was not caused by an acceleration in the scaling of training compute. Interviews and surveys with Anthropic staff are consistent with AI usage causing a doubling in the pace of capabilities progress, with staff self-reporting dramatic uplift from AI systems — though METR calls the survey evidence loose and hard to interpret. The acceleration came primarily through AI assistance rather than fully autonomous contributions, though METR notes some signs that fully autonomous contributions will play a larger role soon. METR also found no clear bottlenecks that would prevent further capabilities progress from limits on algorithmic progress, training compute, experiment compute, or data in the next 12 months — but says its understanding of the bottlenecks changed several times during the assessment, so it is not confident in that finding.
On attribution, The subjective estimate from METR is that a majority — perhaps around 80% — of the current acceleration is due to AI, with substantial uncertainty. The acceleration in late 2025 was likely influenced by non-AI factors too, and Anthropic's own staff did not agree on what explained it. The detailed estimates behind that split were redacted.
The redaction caveat, in full
Anthropic had the right to redact any non-public information in the report, and redacted a majority of the evidence METR found most informative for its conclusions. METR says plainly that the report does not achieve transparency about evidence: the public has to rely on the judgment of METR much more than would be ideal, given the weight of the questions. Some of the redacted evidence had multiple plausible interpretations, and both METR and Anthropic researchers have significant uncertainty about how to synthesize it. Robustness checks on the benchmark finding give acceleration rates between 1.1 times and 2.9 times depending on choices about benchmark inclusion, model dating, and saturation — METR would be surprised if the true figure showed no acceleration, or acceleration above 3 times. The audit is also non-causal in the strict sense: it estimates how much of the acceleration is attributable to AI from interviews, surveys, and internal analysis, not from a controlled comparison.
What it is not
It is not a general productivity multiplier. The 1.9 times figure describes the growth rate of benchmark scores on well-specified tasks for one company's internal models, not how much faster any particular team ships software with AI help. It is not fully autonomous R&D: the audit describes broad AI assistance, with researchers still in the loop. And it is not independently replicable — the underlying evidence is internal to Anthropic and largely redacted, so no outside party can re-run the analysis. METR also warns the benchmark measure itself could mislead: the acceleration could be explained by overfitting to the benchmark distribution or be an artifact of the AECI methodology, and the data points lag acceleration in research and are somewhat sparse.
What to do with it
Treat the finding as a signal about benchmark-improvement speed at one frontier lab, not as a promise for your own team. If you quote the number, carry the whole sentence with it: about 1.9 times, 95% confidence interval 1.5 to 2.8, heavily redacted, low confidence on how much of it AI caused. The practical takeaway for a builder is narrower than the headline: AI assistance is now a large part of how a frontier lab does R&D, and the lab's own staff report the uplift as large — which is a reason to measure AI's contribution to your own work, not to assume a doubling. For the discipline of reading claims against evidence, see The intelligence-explosion argument, and its evidence, and for measurement discipline in agent claims, see Fingerprinting AI coding agents from 33,580 PRs. The earlier METR audit of monitoring gaps is covered in What AI monitoring missed, per the METR audit.
Limits
- Anthropic redacted a majority of the most informative evidence. The public cannot judge the evidence itself, only the judgment of METR on it.
- The 1.9 times estimate carries wide uncertainty: a 95% confidence interval of 1.5 to 2.8, robustness checks spanning 1.1 to 2.9, and the view from METR that 1.3 or 3 times would not surprise them.
- The AI-attributed share — perhaps around 80% — is a subjective estimate given with substantial uncertainty, and non-AI factors likely played a large role in late 2025.
- The measure is benchmark scores on well-specified tasks (AECI), which may overfit or misrepresent broader capability growth, and the data points lag research acceleration.
- The assessment covers Anthropic's R&D in October 2026. It says nothing directly about other labs, other kinds of work, or what AI help does for a team that is not a frontier lab.
Related playbooks
- METR: Anthropic's 2x threshold may be crossed
- Anthropic's no-distillation rule lands November 12
- Use your own Anthropic API key with an agent: what it costs, and the billing trap to check first
- What AI monitoring missed, per the METR audit
Sources
- Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 — METR — the 1.9x estimate, the confidence interval, the attribution estimate, and the redaction caveats
Back to: Coding playbooks hub · Also: The intelligence-explosion argument, and its evidence · Fingerprinting AI coding agents from 33,580 PRs · What AI monitoring missed, per the METR audit