Researchers from Stanford and Tsinghua identify sparse value and dopamine neurons in LLMs that encode state value and temporal difference errors, forming a causal reward subsystem enabling confidence estimation and process reward modeling for improved reasoning.
The article examines why DeepSeek’s large‑model training cannot yet leverage Monte‑Carlo Tree Search, detailing its reliance on SFT, GRPO‑driven CoT activation and rejection‑sampling, contrasting this with Google’s PRM‑based approaches, and proposing a MCTS‑powered data‑generation pipeline to overcome the “roast chicken and baijiu” training dilemma.
OpenClaw‑RL, a new reinforcement‑learning framework from Princeton, captures hidden evaluative and instructional signals in daily user interactions, converts them into real‑time training data, and uses a decoupled asynchronous architecture with binary RL and online policy distillation to achieve superior performance in both personal‑device and cloud‑scale scenarios.
Fin‑PRM, a domain‑specific process reward model for financial reasoning introduced by Alibaba’s Dianjin team, employs dual‑level step and trajectory rewards to provide fine‑grained supervision, achieving up to 12.9% accuracy gains in supervised fine‑tuning and 5.1% improvements in Best‑of‑N inference on benchmarks such as CFLUE and FinQA.
