Machine Heart

Oct 4, 2026 · Artificial Intelligence

ConfTuner: Tokenized Brier Score for Calibrated LLM Confidence Distributions

The article compares TypeSafe AI's Jev model with the NeurIPS 2025 ConfTuner paper, which introduces Tokenized Brier Score to train LLMs to output calibrated probability distributions over confidence tokens, achieving significant ECE reduction across multiple benchmarks with minimal training time, and extends the approach to decision tokens via JevTuner.