From One Test for All to Personalized Exams: USTC’s First Survey on Computerized Adaptive Testing (TPAMI 2026)

Traditional paper‑based exams give every examinee the same test, leading to mismatched difficulty and inaccurate ability estimates. Computerized Adaptive Testing (CAT) addresses this by dynamically selecting the most informative question after each response, aiming to measure true ability with as few items as possible.

What Is Adaptive Testing?

CAT functions like an intelligent interview‑er: based on psychometric and educational theory, the system updates an estimate of the examinee’s skill after each answer and then presents the next question that maximizes information gain.

Core Modules of a CAT System

Measurement Model

The measurement model evaluates the examinee’s cognitive state. Existing models include:

Item Response Theory (IRT) : treats ability as a single scalar.

Cognitive Diagnosis Models (e.g., DINA) : capture mastery of multiple knowledge points.

Deep Learning Models : use neural networks to uncover more complex cognitive structures.

Selection Algorithm

The selection algorithm decides the next item. Classical approaches rely on statistical criteria such as Fisher information or KL divergence, prioritizing items that best refine the ability boundary. For example, in GRE adaptive testing, a strong performance leads the system to present harder items, while weaker performance triggers easier items.

Recent work incorporates reinforcement learning, meta‑learning, and subset‑selection techniques, allowing the algorithm to learn optimal item‑selection policies directly from large historical response datasets. The survey compares five families of strategies—statistical learning, active learning, reinforcement learning, meta‑learning, and subset selection—highlighting trade‑offs among performance, efficiency, and complexity.

Question Bank

A high‑quality question bank is essential. Construction typically involves two steps: (1) analyzing each item’s difficulty, discrimination, and associated knowledge points; (2) assembling a balanced, diverse, and well‑covered pool. Both expert‑driven and automated statistical/deep‑learning methods are used.

Test Control

Beyond accuracy, practical CAT systems must manage exposure control, diversity, fairness, robustness, and retrieval efficiency. Ignoring these factors can lead to item leakage, bias, noise sensitivity, or slow response times—issues especially critical in high‑stakes examinations.

Future Directions

The survey emphasizes a pivotal insight: CAT is not limited to human education; it can also evaluate AI models. As AI agents such as OpenClaw become "digital employees" capable of task execution, adaptive testing offers a human‑like interview to assess reliability, stability, and task competence, moving beyond simple benchmark scores.

Overall, CAT—originating from psychometrics and cognitive science—is evolving into a universal intelligent assessment framework that benefits both human learners and AI systems.