Jul 9, 2026 · Artificial Intelligence

Token-Level Credit Assignment Outperforms Broadcast GRPO in LLM Math Reasoning

The paper identifies the broadcast‑style credit assignment of GRPO as a bottleneck for RL‑LLM math reasoning, proposes the Outcome‑Grounded Advantage Reshaping (OAR) framework with token‑importance estimation, and demonstrates that its two variants, OAR‑P and OAR‑G, consistently improve accuracy, training efficiency, and stability across multiple math benchmarks.

Credit AssignmentGRPOLLM

0 likes · 15 min read

Token-Level Credit Assignment Outperforms Broadcast GRPO in LLM Math Reasoning

Machine Learning Algorithms & Natural Language Processing

Feb 11, 2026 · Artificial Intelligence

Can TI‑DPO Fix DPO’s Blind Spot? Token‑Importance Guided Direct Preference Optimization for Better LLM Alignment

TI‑DPO introduces a hybrid weighting scheme and a triplet‑loss objective that weight tokens by gradient attribution and a Gaussian prior, enabling precise identification of critical tokens and yielding consistent performance gains over DPO, SimPO, and GRPO on Llama‑3, Mistral‑7B, and downstream benchmarks such as IFEval, TruthfulQA, and HumanEval.

Direct Preference OptimizationModel AlignmentRLHF

0 likes · 8 min read

Can TI‑DPO Fix DPO’s Blind Spot? Token‑Importance Guided Direct Preference Optimization for Better LLM Alignment

Token Importance

Token-Level Credit Assignment Outperforms Broadcast GRPO in LLM Math Reasoning

Can TI‑DPO Fix DPO’s Blind Spot? Token‑Importance Guided Direct Preference Optimization for Better LLM Alignment