7B Model Outperforms GPT-5.6 and Opus 5 via Continual Self-Distillation

Researchers from the University of Illinois Urbana-Champaign (UIUC) and the Tsinghua-affiliated startup Astraculum introduced the Social World Model (SWM) framework (ICML 2026, arXiv:2606.11482). SWM treats prediction markets such as Polymarket and Kalshi as natural testbeds for modeling how collective human beliefs shift in response to breaking news. The core analogy: just as a physical world model predicts an object's trajectory from its current state and applied force, SWM predicts how the market's collective belief (state) transitions when a news event (action) arrives.

State Transition Formulation

The framework maps each prediction market contract to a state transition problem:

State (Belief) : The current market price, e.g., a contract trading at $0.62 implies a 62% crowd-sourced probability.

Event : A real-world news item that breaks during the prediction window.

Goal : Predict the next belief state — not the ground-truth outcome, but how the crowd will update its probability after seeing the news.

For example, for the question "Will the Fed cut rates in September?", the model ingests the current price trajectory and the past hour's news, then forecasts the price one hour later.

Deployment-Time Continual Learning via SDFT

Static LLMs freeze their weights after training, so experience gained during deployment cannot improve future predictions. To overcome this, the team adopted a Self-Distillation Fine-Tuning (SDFT) mechanism ( arXiv:2601.19897) that enables the model to learn from its own deployment feedback.

SDFT Teacher–Student Loop

Teacher : The same model, given privileged context — a post-hoc rationale written after the true market move is known. The teacher does not answer; it scores the student's reasoning token-by-token.

Student : Sees only the current news and price history, no privileged context, and must produce a direct prediction.

Distillation pulls the student's reasoning toward the teacher's hindsight-informed judgment, encoding the causal chain event → crowd sentiment → market action into the weights rather than memorizing price numbers.

390-Day Rolling Experiment

The team ran a strict chronological backtest over 390 days of real market data, split into monthly windows. Each month followed the cycle: Deploy → Collect Real Feedback → Train (SDFT) → Update Model → Continue Deploy .

Key Results

The continually updated 7B model was the only system to beat a simple price-only baseline (which ignores news).

Frontier static models — GPT-5.6, DeepSeek V4 Pro, Claude Opus 5 — all fell below the price-only baseline.

Monthly breakdown shows the 7B model's advantage stems from stability : static models suffered catastrophic drops in specific months (e.g., w01, w11), while the 7B model had no month with a large regression.

Probing on unrelated tasks after each of the 12 training rounds showed no degradation in reasoning length or quality, indicating no catastrophic forgetting.

TrajOps: Infrastructure for Continual Learning

To close the loop between data collection, inference, and training, the team built TrajOps , a minimal MLOps engine exposing three interfaces:

Collect : Precisely fetches news and delayed market outcomes without look-ahead leakage.

Inference : Manages the temporal horizon and serves the current model version.

Train : Invokes the built-in SDFT recipe to compile collected outcomes into weight updates.

The 7B model continues to run live on this infrastructure, posting daily prediction logs at trajops.astraculum.com.

Authors

Haofei Yu, Yining Zhao, Jiaxuan You (UIUC)

Chishang Yang, Junbo Yan, Senquan Gao (Astraculum)