In a blog post, an AI researcher explains why model engineering should accelerate in the coming years, without fundamentally changing their nature. According to him, models will become within a few years superhuman GPU engineers, capable of making experimentation markedly easier. He also estimates that several projects, such as compute efficiency and the quality of reinforcement learning environments, offer progress within reach.
- The author expects rapid progress in model engineering within a few years.
- Models will become superhuman GPU engineers, making experiments much easier.
- Models will not fundamentally change their nature or reach general superintelligence.
- Efficiency gains will cut the cost of intelligence, but reinforcement learning data quality remains weak.
Research becoming increasingly easy
The author notes that industry researchers often tell him that AI will outperform them in their profession within a few years. He attributes this acceleration to the growing capabilities of models in infrastructure and engineering, a point on which he agrees. He adds, however, that these advances will not make the models radically different in their nature.
According to him, Ilya had, in some respects, announced the AI research era too early. Research is now much simpler thanks to agents de programmation, tools capable of writing and modifying code autonomously. The text sees this as the start of a period where good ideas could be worth more than good software execution.
The author recalls that the balance between research and engineering has already shifted in the past. Before the rise of deep learning, AI was mainly a research field. Today, top researchers are judged on their ability to implement and scale their ideas in complex infrastructures. He mentions an acceleration “of a few years” where the bottleneck will become less and less engineering.
Efficiency and cost of intelligence
A large part of the training and inference steps, i.e., using a model to respond, can be measured precisely. The text cites the number of tokens processed per second and per GPU, the number of tokens per request, FLOPs per token, and the cost per response. The author believes these metrics are highly optimizable and that AI agents could help improve the entire process within a few years.
Efficiency gains have already been significant, according to the text, with, for example, savings “on the order of 10 to 30 %” on a model’s service cost after its announcement at a certain price. The author expects the effective cost of model intelligence to drop in a near‑exponential manner, potentially faster than recent trends. He also forecasts that the joint design of accelerators and models will deliver additional gains over longer time scales.
The text also argues that a paradoxe de Jevons, whereby an efficiency gain can increase demand, should fully apply to agentic models. The author cites Meta’s agent Muse as an early sign of this trend, and anticipates more experiments of this kind for various audiences.
Data and learning environments
The text finally mentions the market for data used in reinforcement learning, those environments where a model learns through trial and reward. Several companies in this sector are said to have crossed the “$100 million” or “$1 billion” revenue threshold, a figure higher than the author had imagined. Yet he observes that the average quality of these productions is “remarkably low”, and that many researchers consider a portion of what they purchase to be of poor quality.
Large labs, however, derive a clear return on investment from these purchases, the text notes. For the author, the weakness of many environments is clearly fixable.
