StartLux-Decision: On-Demand Reasoning and Probability Calibration for Decision Models

Researchers from StartLux, Hong Kong University, and Southern University of Science and Technology present StartLux-Decision, a framework for decision models that unifies single-pass prediction (Instant) and on-demand reasoning (AutoThink) under a shared option-probability interface.

Unified Probability Interface and Attention Designs

Decision requests are formatted as "input state + question + candidate options," with the model outputting a probability distribution over options (single-choice, yes/no, or rating). Probabilities are read from the language model's existing output layer by mapping options to letters (A, B, C) and taking logits at the last prompt position, followed by temperature scaling and softmax — no extra classification head is added. Both Instant and AutoThink use this same readout.

The two versions differ in attention structure. Instant removes causal masking in Full Attention layers, allowing bidirectional token interaction within the prompt, while Linear Attention layers retain causal computation. AutoThink keeps full causal attention to support subsequent reasoning generation. This bidirectional change lets earlier state descriptions integrate later question and option information in middle layers, which helps tasks requiring re-interpretation of inputs in light of options. A controlled comparison across five model sizes (same data, same initial checkpoint) showed DI 0.3 public scores improved in four sizes, with a maximum gain of 2.4 points; the 2B model changed by −0.2 points. The paper notes 0.2–0.7 point variance across random seeds, concluding bidirectional attention helps in most but not all settings.

Three-Stage Training

Stage 1: Full-parameter supervised fine-tuning on ~2.7M decision samples (0.8B–9B) or ~800K (27B). Stage 2: Adapter training and merging on ~1.7M samples, simultaneously diverging attention designs. Instant is complete after stage 2. AutoThink proceeds to stage 3: reasoning supervised fine-tuning (4K–21K samples) followed by RLCD² reinforcement learning.

Read-or-Think: Confidence-Gated Reasoning

AutoThink first runs a causal forward pass to obtain initial option distribution p₀. If max probability ≥ 0.9 (default threshold), it returns immediately. Otherwise, it re-processes the prompt, generates a reasoning trace, and reads a second distribution p₁ from the trace end. The final output blends both distributions: p_final = 0.25·p₀ + 0.75·p₁. The 75% weight was chosen on a held-out training split to optimize Brier score (full-distribution quality), and is near-optimal for 4B, 9B, and 27B.

Two illustrative cases under the same export policy show the mechanism. Case 1: 3,332 records requested, 1 prior export in 30 days → initial p(refer to human) ≈ 0.96 → direct answer. Case 2: 8 records, 0 prior exports → initial p(refer) ≈ 0.57 → reasoning triggered. The model verifies the "<100 records" exception and approval conditions, uses 159 tokens, and shifts p(approve) to ≈0.995; blended probability ≈0.848, flipping the decision.

Across 7,800 DecisionPivot core questions, 27B AutoThink answered ~39% directly (single-question accuracy ~96%) and reasoned on ~61% (accuracy rising from ~57% to ~93% after blending). Reasoning corrected 1,783 previously wrong questions but introduced 94 new errors.

RLCD²: Reinforcement Learning for Calibrated Deliberative Decisions

Confidence gating decides when to reason, but not whether reasoning is effective. Stage 3 first restores reasoning generation and termination via supervised adapters, then applies RLCD². Unlike methods rewarding only final answer correctness, RLCD² explicitly optimizes the improvement in probability prediction quality from reasoning.

A frozen copy of the post-supervision model scores both pre- and post-reasoning option probabilities. The training model generates reasoning; the frozen model evaluates. Reward combines a probability-score increment and a length penalty: R = S(t, p₁) - S(t, p₀) - λ·max(0, L - B) where t is the target distribution, τ the reasoning trace, L its token length, S a decision-weighted strictly proper scoring rule (emphasizing high-confidence regions), B=3,072 token soft budget, λ=10⁻⁴ per token beyond B. Generation can continue to a hard limit of 4,096 tokens.

Reasoning Length and Termination Improvements

On 2,986 held-out samples (forced reasoning, 4,096 token cap), RLCD² markedly improved termination and efficiency. 9B: self-termination rose from 45.9% to 93.9%, mean length from 2,494 to 868 tokens (~65% reduction). 4B: 89.5% → 99.3%, 1,140 → 722 tokens. 27B: already 97.2% termination at 313 tokens, little room for improvement. Final decision accuracy on this set changed within intervals containing zero, so the primary supported conclusion is better termination and efficiency, not universal accuracy gains. Ablation showed length penalty drives termination; proper scoring reward drives calibration.

Shorter is not always better. On DecisionPivot, 4B pair accuracy dropped from 60.4% (no length penalty) to 52.9% when penalty started at 512 tokens; relaxing to 3,072 tokens recovered 59.9%. Removing penalty entirely gave 61.7% but left ~1/3 of reasoning traces non-terminating. The trade-off: suppress repetitive generation without compressing genuinely long computations. The 3,072 budget is a configuration for this trade-off, not a universal optimum.

DecisionPivot: Sensitivity to Decision-Critical Details

Average accuracy can mask failures on rule exceptions, threshold boundaries, or single-field changes. DecisionPivot contains 13 programmatically generated task categories (600 questions each, 7,800 total), covering filtering/matching, ledger reconciliation, scheduling conflicts, permission inheritance, target-state verification, tool-call validation, etc. Most questions form counterfactual pairs differing in one answer-changing detail; answers are programmatically verified.

The core metric is pair accuracy: both questions in a pair must be correct. This tests whether models distinguish superficially similar but decision-divergent cases. The benchmark was not used for training data, checkpoint selection, or inference config.

Results: 4B, 9B, 27B AutoThink pair accuracy = 59.9%, 59.9%, 86.9%; corresponding Instant = 41.5%, 47.5%, 75.4% (gaps +18.4, +12.4, +11.5 pp). Baselines: Decision 3.0 d3 71.3%, Perplexity Decider v1.1 70.1%, GLM-5.3 (low reasoning) 82.2%. Gains concentrate in tasks requiring computation or item-by-item checking (filtering, target-state verification, scheduling); policy interpretation, clause changes, and dependency handling favor strong single-pass models. Explicit reasoning's value depends on task computation needs.

Isolating Reasoning's Contribution Within a Fixed Model

Instant vs. AutoThink conflates attention direction and training differences. To isolate reasoning itself, the authors sweep confidence threshold 0→1 on a fixed AutoThink checkpoint. 27B: threshold 0 (no reasoning) → 48.0% pair accuracy; threshold 0.9 → 86.9%, avg 304 reasoning tokens/question (including zeros for non-reasoned). 9B: 22.3% → 59.9%, avg 1,365 tokens. Note: 48.0% is AutoThink without reasoning, not Instant's 75.4% — these are distinct comparisons.

On training-distribution held-out data, 27B default config gains ~2.5 pp over no-reasoning at 273 tokens/question; 9B gains ~1 pp at default threshold, and forcing reasoning on all questions does not further improve accuracy. Benefits and costs depend on task distribution; single cases or fixed reasoning ratios do not generalize.

Decision Index (DI 0.3) Scores

DI 0.3 public scores (chance-corrected, domain-aggregated, not raw accuracy): Instant 0.8B 53.8, 2B 59.1, 4B 67.9, 9B 70.0, 27B 72.8 — leading third-party decision models by 7.4–16.9 points. AutoThink adds modest overall gains: 4B 68.21, 9B 70.45, 27B 73.37. Domain breakdown reveals trade-offs: in "Knowledge & Reasoning," 4B and 9B gain ~7.0 and ~6.2 points over Instant; in some other domains Instant retains advantage. DI 0.3 public is only 20% of full leaderboard and was used in data recipe/model selection; DecisionPivot provides independent verification.

Policy-Aware Calibration

Single-prediction calibration does not transfer to downstream selection/routing strategies. With 9B as verifier, raw predictions at 80–90% confidence had 92% empirical accuracy; after best-of-32 selection, the same confidence bin dropped to 78%. Selection changes the conditional distribution of returned predictions: top scores reflect both quality and upward noise; other candidates' scores carry difficulty information not captured in the single chosen score. Standard calibration beforehand does not eliminate this shift as candidate pool grows.

The authors propose policy-aware calibration: run the actual selection/routing policy on labeled calibration data, then fit a calibration mapping using the returned predictions and policy-side information (candidate count, mean candidate score, trigger confidence for gating/cascading). This adjusts reliability estimates of final outputs without retraining the base model or correcting wrong answers. Across 184 settings (best-of-N, reasoning gating, cascading, filter-then-reason), signed confidence–accuracy gaps shrank from [−11.7, +20.5] pp to [−2.7, +2.1] pp. Calibration has distribution boundaries: transferring to unseen datasets leaves up to ~6.1 pp residual overconfidence. Reliability depends on candidate count, routing rules, and task distribution; policy changes require re-validation.

Deployment and Open Questions

Deployment experiments cover vLLM, llama.cpp, MLX, and quantization formats. On single H200 BF16 256-token prompt single-request: Instant median latency 24–52 ms; AutoThink adds trace generation and second readout. On Apple M5 Pro 64GB unified memory, 27B BF16 exceeds memory; 8-bit and 4-bit quantization enable local runs.

Open problems acknowledged: new decision data mainly improves covered task families, cross-family transfer evidence limited; fixed confidence threshold ≠ per-question optimal compute allocation; distribution shift degrades fitted calibration maps. StartLux-Decision maps each to a testable design: attention direction for single-pass representation, RLCD² for reasoning's probability gain vs. cost, DecisionPivot for detail sensitivity, policy-aware calibration for post-system probability behavior. Evaluation thus moves beyond "is the prediction correct" to "what did extra computation improve" and "what action does the returned confidence support."