This survey examines how rubrics—structured, multi‑dimensional evaluation criteria—are defined, constructed, and applied to train and evaluate large language models, especially for open‑ended, high‑risk and agentic tasks, while highlighting current challenges such as reward hacking and bias.
The article analyzes the growing difficulty of evaluating large AI models, critiques traditional RLVR and RLHF approaches, introduces a Rubrics‑based evaluation paradigm, describes the design and three‑stage workflow of the CRAFT system, reports math‑domain experiments showing up to 6.2 percentage‑point gains, and outlines future extensions to other domains.
The survey from RUC‑Gaoling AI Institute reviews Rubrics for large language models, explaining why they are needed for open‑ended, high‑risk tasks, how they are constructed, and how they can be applied to policy and reward model training as well as multi‑dimensional evaluation across general and domain‑specific scenarios.
This survey examines how rubrics can decompose the vague notion of a "good answer" for large language models into concrete, multi‑dimensional evaluation criteria, detailing their definition, construction methods, applications in training and evaluation, and the open challenges they present.
