Hugging Face introduces decision models, a family of models that do not write a response, but evaluate predefined options and return a probability distribution over these options. The company seeks to define a common interface for these models, without imposing a single architecture.
Large Language Models (LLMs), which produce their response token by token, i.e., piece of text by piece of text, are very effective for writing, programming, or reasoning. But not all problems require generating text. Take a customer support system that receives the following message: ‘I was charged twice for my subscription. I need a refund today.’ It must determine which team should handle the request, its degree of urgency, and whether the customer is requesting a refund.
You can ask an LLM to return these responses as a JSON object. This approach is already widespread for classification and routing, but it has several limitations. The model is not guaranteed to respect the requested schema: it may produce invalid JSON, add unexpected keys, or respond outside the allowed choices. Constrained decoding libraries, like Outlines and XGrammar, reduce this risk by limiting the allowed tokens at each step, but they add extra work. Finally, when the application only needs to choose between known responses, the full text generation can be superfluous.
A Probability-Based Interface
A decision model receives a state, i.e., the information to evaluate, and a list of questions. It returns decisions accompanied by probabilities. The source distinguishes three types of questions. The choice type selects one option among several. The score type evaluates something on an ordered scale. The noul type answers a yes/no question in a probabilistic form.
In the example presented, the autotrust/GEV-26B-Decide model assigns the following probabilities to the service question: Billing 0.998, Technical support 0.001, and Sales 0.001. For urgency, the levels low, medium, and high receive 0.10, 0.26, and 0.72 respectively. The returned score corresponds to the expectation of this distribution, i.e., 1.7, and not to a level chosen halfway. For the question ‘Is the customer requesting a refund?’, the model directly returns the probability of a positive answer, here 0.92.
The notion of confidence, calculated for choice and score type questions, summarizes the concentration of these probabilities. The concept is not entirely new: the company TypeSafe AI had presented Jev, described as a ‘System One’ model. The term ‘decision model’ was ultimately chosen by the community, as it describes the computational role of the model without referring to a particular theory of cognition.
An Ecosystem Under Construction
The new format is already supported by several tools. The Transformers library offers a DecisionPipeline, an AutoModelForDecision class and a DecisionConfig class, as well as a /v1/systemone endpoint after launching a local server. The llama.cpp inference engine also accepts these models via the same API. The source cites the ggml-org/Kev-4B-GGUF model, served on port 9931. The Hub offers a dedicated tag, decision-model, as these models do not necessarily share the same architecture: some rely on encoders like ModernBERT, others on larger generative models.
Once inference is addressed, the source discusses training. A demonstration example fine-tunes the autotrust/JEV-9B model to route support requests, using the Trainer API. Native training support in TRL is the subject of a pull request, PR #8040, still open at the time of writing. Furthermore, Unsloth offers a recipe for transforming a generative LLM, like Qwen, into a decision model, using LoRA and a scoring head.
