Leading AI labs, foremost among them OpenAI and Anthropic, have thousands of agents working simultaneously to improve their processes and output. In an analysis published in the Interconnects newsletter, the author questions how this pace is altering employee expectations and the idea that AI-related risks could lead to human extinction, which he deems “very religious.”
- The author doubts that true recursive self-improvement will arrive soon.
- He expects gains to come mainly from efficiency, not sudden breakthroughs.
- Experts give very different timelines for advanced AI capabilities.
- He keeps his own «lossy self-improvement» hypothesis as his reference.
In his view, the highly competitive culture of the San Francisco scene amplifies any AI-related concern. He recalls the numerous debates on AI safety and the viability of open-source AI that animated 2023 and 2024. The major risks predicted at the time did not materialize within the expected timeframes. He also cites incidents like “OpenAI-HuggingFace” and believes that rising anxiety, especially since agents gained efficiency in early 2026, could intensify further.
A Community Focused on the Near Future
The author revisits Richard Ngo’s summary, which observes that a large part of the AI safety community is oriented towards scenarios of intelligence explosion in the coming years. According to Ngo, these expectations will be “right in direction, but wrong on the facts.” He notably writes: “we won’t have superintelligence in the next 8 years, but things will move so fast that it will feel like those advocating for short timelines were right.”
The author relates this position to his own hypothesis, which he calls “lossy self-improvement”, meaning self-improvement “with loss,” as opposed to AI recursive self-improvement (RSI), where an AI contributes to improving its own design. He believes that automatable research remains too narrow to strongly accelerate progress, that diminishing returns from adding agents in parallel are real, and that resource constraints and political factors weigh heavily.
Vastly Different Timelines Among Experts
The author draws on Dwarkesh’s podcasts with Noam Brown, then with John Schulman, Beren Millidge, and Charlie O’Neill. In these discussions, the participants give timelines for several capabilities. For a “drop-in remote worker” capable of performing generalist office work over a month, John Schulman mentions about a year for a “decent” version, Charlie O’Neill about a year with programmatic access to tools, and two years if it has to go through a browser, while Beren Millidge talks about roughly three years for full generality.
For a tenfold increase in AI researcher productivity, John Schulman suggests about two years, Charlie O’Neill between five and ten years, and Beren Millidge finds the two-year estimate plausible without offering an alternative. For an AI surpassing the best human experts across all computer work, including multi-year projects (“ASI”), John Schulman mentions three to four years, Beren Millidge about five years for the domains targeted by labs, and Charlie O’Neill five to ten years.
Gains Mainly in Efficiency
The author emphasizes that progress does not cross sharp thresholds, as model intelligence remains uneven across tasks. He believes the experimental loop could become ten times faster, but not hypothesis formulation. In his view, self-improvement should primarily help make models less costly and improve multi-agent systems, as LLM service metrics are measurable.
He remains cautious about costs, however, as scaling laws show that exponential resources are needed for linear gains in intelligence. He cites John Schulman on post-training, which he considers a difficult task to automate: “it’s very easy to mess up post-training in a way that doesn’t show up in benchmarks.” He finally reports a passage from the Claude Fable 5.1 & Mythos 5.1 System Card: “We believe internal use of recent AI models has been a key factor in maintaining the current pace of progress, but we do not yet see clear signs of a dramatic acceleration beyond this pace.” For now, the author considers his own hypothesis to remain his reference.
