Microsoft and Hugging Face present ThinkingBox-Training, a method that uses the final state of a database as a reward signal to train artificial intelligence agents. Applied to the Qwen3.8-27B model, it raises the average success rate from 51.70% to 60.78%, according to the authors, and the number of tasks succeeded on every attempt from 38 to 107. The code is available on GitHub.
ThinkingBox is first used to evaluate an agent’s work by comparing the final state of the database with the expected outcome. Each of the 507 tasks in ThinkingBox-Bench launches a real backend and a set of tools, lets the agent work, then checks what actually changed. Such a check returns a 1 or 0 from the final state, which can also serve as a reward for training.
The main challenge is selecting the training tasks. With the GRPO method, a group of eight attempts that all fail, or all succeed, provides no contrast to learn from. The authors therefore estimated, once, the probability of success for each task with the base model, then kept tasks that succeeded from time to time. They designate this notion by the term “offline policy relative difficulty,” where the policy here refers to the trained model. The final training set contains 187 tasks.
Gains in the Consistency of Successes
After 50 GRPO steps, the model was evaluated on the 507 tasks of ThinkingBox-Bench, which it had never seen during training. Each task is repeated 20 times, totaling 10,140 attempts. The number of tasks succeeded on every attempt rises from 38 (7.50%) to 107 (21.10%), while the number of tasks succeeded at least once slightly decreases, from 453 (89.35%) to 446 (87.97%).
The authors draw a precise interpretation from this: the model did not become broader, it became more consistent. They cite the example of a hotel room downgrade request, which requires prior manager approval. Before training, the agent modified the reservation and issued refunds without this approval in 15 out of 20 attempts. After training, it preserved the state of the reservation and payment in all 20 attempts.
An Estimated Cost Per Reliable Task
The authors also quantify the cost of this consistency. According to their estimate, the cost of the campaign of 20 passes drops from $925.80 to $855.58, a decrease of 7.6%. Relative to tasks succeeded on every attempt, the cost per reliable task falls from $24.36 to $8.00. These amounts are a comparative service indicator, not an actual bill, as they do not account for the cost of training.
The reference training was conducted on 24 H100 GPUs, with 18 groups of 8 attempts per update, i.e., 144 episodes. The authors emphasize that evaluation and training must interpret the model’s outputs in the same way; otherwise, the reward measures a different agent than the one being evaluated. The repository published on GitHub contains tools for data selection, trial generation, reward calculation, and integration with Verl.
