Faced with rising AI costs, the most common response is to limit usage: consumption caps, seat limits, and approvals before launching agents. However, these undifferentiated restrictions can hinder useful applications and measure the wrong metric, according to the analysis.
The cost per token (a unit of text processed by a language model) measures consumption, not results. Some of the expense corresponds to waste, like re-running queries or uncontrolled loops, while another part represents the actual work of delivering software. A counter cannot distinguish between them, nor can a spending cap.
Measuring Delivered Software
A more relevant metric would be the cost per accepted merge request, i.e., the total cost of AI-assisted development divided by the number of changes that pass review and are integrated into the code, called “pull requests” in the jargon. It is necessary to track rework and defects in parallel, so that cheaper modifications do not mask a drop in quality.
If this cost decreases while usage increases, without a degradation in quality, developers are converting more consumption into useful software. The question, therefore, is not the number of tokens used, but the amount of completed work produced by the investment. To improve this balance, two levers are proposed: reserve expensive inference for tasks that need it, and reduce the cost of routine work.
Adapting the Model to the Task
Agentic development, where an agent writes and modifies code autonomously, does not require the most powerful model at every step. A cutting-edge model, or a flagship model hosted on-premises when company policy requires it, can be reserved for complex planning and novel reasoning. Simpler tasks, like code generation, test creation, or repeated refactoring, can be entrusted to open-weight models, operated directly by the company.
A gateway can route each request based on its complexity, data sensitivity, cost, and latency. The first model is thus used only when the situation justifies it. As the author summarizes, “Meter the exception, not the workload.”
Open-weight models make this allocation increasingly feasible. According to cited software engineering benchmarks, the best among them can approach cutting-edge performance on certain tasks. This observation does not guarantee equivalence across all code repositories or all workflows. It is therefore recommended to evaluate candidates on one’s own code, one’s own tests, and one’s own acceptance criteria before entrusting them with production work.
Changing the Economics of Repeated Use
The question remains of who owns the compute capacity. With rented inference, each call adds to the bill. With owned capacity, most of the cost is incurred upfront, so each additional call becomes cheaper as usage increases.
