Anthropic is using AI to accelerate the development of its own models, according to a new report from AI safety research organization METR. The document suggests the company’s models are progressing roughly twice as fast as they were a year ago, with no clear sign of slowing down. The report also raises a question: how much does the public really know about this pace, and the risks that accompany it?
The report is the result of an eleven-week investigation within the company, as part of an assessment of its AI research and development activities. According to METR, benchmark scores suggest Anthropic has achieved “roughly two years of historical progress in the last year.” The researchers say they are “confident” this acceleration is not simply due to increased computing power used to train models, a factor long considered the primary driver of progress.
AI Accelerates Anthropic’s Research
According to METR, the majority of the acceleration observed in October “is due to AI,” and its contribution is increasing over time. This finding is based on informal surveys conducted on Slack and interviews with employees, which report significant productivity gains. On average, employees using the company’s most powerful internal model say they are about 12 times more productive than without AI assistance, roughly double what they reported a few months earlier with Mythos 5.
For now, this acceleration comes mainly from AI assisting human researchers, not from AI working alone. However, METR warns that “some signs indicate that fully autonomous contributions will soon play a more significant role.” The researchers found no clear signs of diminishing returns or bottlenecks likely to prevent even faster acceleration.
Furthermore, three new Anthropic models completed their training in the last six months without being published. They are designated by the code names Model 2, Model 3, and Model 4, whose training ended in July, between August and September, and between September and October, respectively. METR also mentions an unpublished model from 2022, called Unreleased 2022. The researchers specify that “these are the code names we were given; we understand they have different internal names.”
Redacted Information
Anthropic gave METR access to non-public information, including confidential research, model performance data, and interviews with employees. However, the report clarifies that access was not unlimited. Anthropic had the right to redact the final version, and METR states it submitted a report of about 100 pages, from which the public version had “the majority of the evidence” the organization considered most informative removed.
Anthropic justifies these redactions by the risk of benefiting its competitors. METR, for its part, believes “most of the redacted information was very high-level,” without commenting on the commercial nature of these choices. In a lengthy post on X, Anthropic co-founder Sam McCandlish acknowledges: “We redacted a lot, and I accept that this makes it harder to verify METR’s work.” He states the company made “the right decision,” and announces the upcoming publication of its own report on the speed of AI progress.
Growing Pressure for Transparency
The report has sparked sharp criticism among AI safety researchers. Daniel Kokotajlo, a former OpenAI researcher who leads the AI Futures Project, accuses Anthropic of “recklessly gambling with our lives.” Brianna Rosen, Director of Frontier Model Safety Research at the Institute for AI Policy and Strategy, believes that “the report was not sufficiently transparent about the evidence.” Anthropic declined to answer questions from The Verge and referred to Sam McCandlish’s post.
New York Assemblyman Alex Bores has called on Congress to “ban RSI, this year if possible.” RSI, or Recursive Self-Improvement, refers to AI systems capable of developing increasingly powerful successor systems without human intervention. Marius Hobbhahn, founder and CEO of Apollo Research, believes the results are “broadly in line” with what the field expected, “maybe a bit faster,” and he warns: “Brace for madness. This is moving very fast.”
