Playbooks · Coding tasks
METR: Anthropic's 2x threshold may be crossed
Written November 7, 2026
METR reviewed Anthropic's August Risk Report and says its "less than 2x" claim rests on contestable calls. The disputes, and what each side agrees on.
What METR reviewed
METR reviewed the "Risks from automated R&D" section of Anthropic's August 2026 Risk Report, alongside a private version of the report and non-public data Anthropic shared. The work ran from August 12 to October 29, 2026, as part of a high-access assessment of AI R&D acceleration at Anthropic. Anthropic's Responsible Scaling Policy sets two capability thresholds for automated R&D in key domains: a full-substitution threshold, where a model can fully substitute for the entire set of research scientists and engineers at competitive cost, and a 2x acceleration threshold, where the pace of AI progress doubles for reasons that likely relate to the automation of AI R&D. The 2x threshold is an early warning, not direct evidence of high risk.
On the points that matter most, the two sides agree. METR agrees with Anthropic that catastrophic risks from models as of July 15 were low, and that no model could fully substitute for research staff. Both METR's researchers and Anthropic's relevant staff lean toward the 2x threshold not having been crossed. The dispute is about confidence, not about the risk level — and about whether the public report says more than its evidence supports.
Where they disagree
The public report says leading indicators point to "meaningful acceleration starting in early-to-mid 2025, though by less than a factor of 2". METR calculated 1.97 from figures in the private report and says "around 2" is the fairer description: a reader would take "less than a factor of 2" to mean an estimate meaningfully below 2, and 1.97 is not that. METR's own analysis of the leading indicators finds a range of reasonable estimates between 1.3 and 2.8, depending on the definition and time period.
The second dispute is about attribution. The public report says Anthropic is "fairly confident" the 2025 acceleration came from factors other than its use of AI models. The private report says it is "plausible" the overall counterfactual acceleration attributable to AI exceeds a factor of 2 by July 15. METR broadly agrees with the private report's reading: directly estimating counterfactual acceleration is very hard, and the public report implies more confidence than the evidence carries.
The third dispute is about the capability trend. The public report reads the AECI scores — Anthropic's version of the Epoch Capabilities Index — as weak evidence against a sustained acceleration, a one-time jump instead. METR disagrees with the emphasis: reusing the report's own data, the estimated acceleration is around 1.5x, and with data added after publication it is 1.9 times. A continued acceleration, METR says, is more consistent with the acceleration in the leading indicators.
Do not conflate the 1.9x
The 1.9x figure comes from METR's companion assessment: benchmark scores for Anthropic's models have grown at an estimated 1.9 times their pre-October 2025 rate since October 2025, with a 95% confidence interval of 1.5 to 2.8. That is a different comparison, with a different baseline and a different date, from the threshold question about July 15. It is evidence that acceleration is real and continuing — it is not the verdict on whether the 2x threshold was crossed. The companion assessment is covered in METR: Anthropic's AI R&D sped up about 1.9 times; this page does not repeat its numbers beyond the distinction.
What a builder takes from it
A threshold call is a judgment with stated uncertainty, not a measurement. When a lab says a threshold was not crossed, ask what the underlying figures imply, how wide the range of reasonable estimates is, and how much of the evidence you are allowed to see. Here the most informative evidence was largely out of view: Anthropic redacted a majority of the evidence METR found most informative for its conclusions, so the public has to rely on METR's judgment more than would be ideal. The honest summary is the one both sides share — catastrophic risk from the July 15 models was low, and crossing the threshold was plausible but probably did not happen — with the public report's confidence on the second point overstated.
Limits
- METR does not say the threshold was crossed. It says crossing was plausible, and both sides lean toward it not having been crossed.
- The 1.9x acceleration estimate is a different comparison from the threshold question, with a different baseline and date. The two figures should not be swapped.
- Anthropic redacted a majority of the most informative evidence, so outside readers cannot check the disputed calculations themselves. The dispute rests on METR's judgment about evidence it cannot fully show.
- This page is a read of METR's review and assessment. No evaluation was repeated, and the assessment itself says external replication is not feasible.
Related playbooks
- METR: Anthropic's AI R&D sped up about 1.9 times
- Anthropic's no-distillation rule lands November 12
- Use your own Anthropic API key with an agent: what it costs, and the billing trap to check first
- What AI monitoring missed, per the METR audit
Sources
- Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026) — METR — the 1.97 calculation, the "around 2" wording, the 1.3–2.8 range, and the plausible-crossing statement
- Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 — METR — the 1.9x estimate with its 95% confidence interval, and the redaction of a majority of the most informative evidence
Back to: Coding playbooks hub · Also: METR: Anthropic's AI R&D sped up about 1.9 times · The intelligence-explosion argument, and its evidence · Astra did 40% less work on safety tasks