Tagged articles

credit assignment

9 articles · Page 1 of 1

Oct 25, 2026 · Artificial Intelligence

Dust: Zeroth-Order Optimization Trains Transformers Without Backpropagation

Q Labs introduces Dust, a zeroth-order optimization method that trains Transformers by adding Gaussian noise to per-token activations instead of using backpropagation, achieving competitive results at 243M parameters and 20M tokens while challenging the necessity of differentiability in neural network training.

Q LabsTransformer pretrainingactivation perturbation

0 likes · 15 min read

Dust: Zeroth-Order Optimization Trains Transformers Without Backpropagation

Machine Learning Algorithms & Natural Language Processing

Oct 16, 2026 · Artificial Intelligence

A3 Framework: 14B CLI Agent Beats 235B Model via Structured Action Credit Assignment

Fudan and SJTU researchers open-source A3, a reinforcement learning framework that uses AST-based structured credit assignment to train a 14B CLI agent achieving 24.6% exact match on ShellOps, doubling baselines, and outperforming a 235B model on out-of-distribution tasks with minimal computational overhead.

Agentic RLCLI AgentsNeurIPS 2026

0 likes · 18 min read

A3 Framework: 14B CLI Agent Beats 235B Model via Structured Action Credit Assignment

Didi Tech

Oct 15, 2026 · Artificial Intelligence

Why Agents Need Lessons, Not Just Memory: Inside causal-memory's Causal Design

causal-memory replaces flat fact recall with a causal graph of decision→outcome edges, using a hippocampal-inspired engine, Pearl's causal ladder, credit assignment via Q-value backprop, and governance checks to cut repeated mistakes by half in real use.

Agent MemoryPearl's causal laddercausal reasoning

0 likes · 34 min read

Why Agents Need Lessons, Not Just Memory: Inside causal-memory's Causal Design

Machine Heart

Sep 25, 2026 · Artificial Intelligence

V-Rubrics: 350K Fine-Grained Rubrics Fix Credit Assignment in Multimodal RL

V-Rubrics decomposes 50,248 visual samples into 352,938 atomic rubrics across visual faithfulness, reasoning consistency, and instruction following, enabling fine-grained credit assignment in GRPO training that lifts Overall Avg. to 68.04 on general benchmarks and 62.45 on visual math and logic tasks.

EMNLP 2026GRPOQwen3-VL

0 likes · 13 min read

V-Rubrics: 350K Fine-Grained Rubrics Fix Credit Assignment in Multimodal RL

DataFunSummit

Jul 9, 2026 · Artificial Intelligence

Token-Level Credit Assignment Outperforms Broadcast GRPO in LLM Math Reasoning

The paper identifies the broadcast‑style credit assignment of GRPO as a bottleneck for RL‑LLM math reasoning, proposes the Outcome‑Grounded Advantage Reshaping (OAR) framework with token‑importance estimation, and demonstrates that its two variants, OAR‑P and OAR‑G, consistently improve accuracy, training efficiency, and stability across multiple math benchmarks.

GRPOLLMMathematical Reasoning

0 likes · 15 min read

Token-Level Credit Assignment Outperforms Broadcast GRPO in LLM Math Reasoning

Machine Learning Algorithms & Natural Language Processing

Jun 5, 2026 · Artificial Intelligence

StepOPSD: Precise Step‑Level Error Detection for Multi‑Turn Agent RL

StepOPSD adds a post‑hoc, step‑aware distillation stage to multi‑turn agent reinforcement learning, splitting rollouts into controllable steps, using successful trajectories as hindsight teachers to compute token‑level advantage adjustments, and demonstrating significant gains on ALFWorld and Search‑QA tasks where reward misalignment is most severe.

ALFWorldAdvantage WeightingSearch QA

0 likes · 13 min read

StepOPSD: Precise Step‑Level Error Detection for Multi‑Turn Agent RL

HyperAI Super Neural

May 28, 2026 · Artificial Intelligence

Large-Model RL Advances: Credit Allocation, Complex Reasoning, Agent Learning

HyperAI curates six cutting‑edge large‑model reinforcement‑learning papers—from ECHO’s free world‑model learning to DelTA’s discriminative token credit, GoLongRL’s capability‑oriented long‑context RL, Anti‑SD’s reverse distillation, RubricEM’s rubric‑guided policy decomposition, and Poly‑EPO’s diversity‑driven exploration—highlighting their methods, benchmarks, and performance gains.

Agent learningComplex ReasoningDiversity Exploration

0 likes · 10 min read

Large-Model RL Advances: Credit Allocation, Complex Reasoning, Agent Learning

Machine Heart

May 23, 2026 · Artificial Intelligence

Why Can’t LLMs Directly Copy AlphaGo’s MCTS Success?

The article analyzes why large language models cannot simply adopt AlphaGo’s Monte‑Carlo Tree Search, highlighting credit‑assignment difficulties, gradient‑variance explosion in multi‑step RL, and how AlphaGo’s tight integration of value and policy networks amortizes search in a way LLMs cannot replicate.

AlphaGoLLMMCTS

0 likes · 6 min read

Why Can’t LLMs Directly Copy AlphaGo’s MCTS Success?

Baobao Algorithm Notes

Feb 24, 2026 · Artificial Intelligence

The Bitter Lesson of Building Agentic RL in Terminal Environments

This article recounts the challenges of moving from single‑step RL with verifiable rewards to multi‑step agentic reinforcement learning in terminal environments, detailing infrastructure design, asynchronous pipelines, data quality checks, masking strategies, curriculum training, chunk‑based optimization, and practical lessons learned from large‑scale experiments.

Agentic RLAsynchronous TrainingEnvironment Augmentation

0 likes · 33 min read

The Bitter Lesson of Building Agentic RL in Terminal Environments