Why Fixing Bad Cases Beats Adding More Data in RLHF

1. Model capability ceiling is the error‑feedback loop

Many assume that a stronger model comes from continuously adding data and training, but practitioners know that the real bottleneck is how quickly bad cases are identified and fixed. Without a negative alignment signal—examples of why an answer is wrong—the model only sees correct answers and cannot learn to avoid systematic errors.

2. What is a Bad Case?

A Bad Case is not a random mistake; it is a structurally biased sample that has high error frequency, high user cost, and clear deviation. For example: 我想办签证，需要什么材料？ Model answer: "护照，身份证" – this lacks critical information, steps, conditions, and risk warnings, illustrating a typical Bad Case.

3. Why Bad‑Case repair is more effective than adding data

Adding data quickly reaches diminishing returns: the first ten thousand samples improve coverage, but beyond that the marginal gain drops sharply. Bad Cases, on the other hand, act as explicit markers of capabilities the model lacks, providing a clear signal for correction.

Reason 1: New data mostly repeats patterns

Question conditions

Answer format

The model learns surface patterns but not reasoning ability.

Reason 2: Bad Cases expose missing capabilities

Repairing a Bad Case involves four actions:

Understand the task

Decompose the failure logic

Provide the correct reasoning structure

Show the model how to fix it

Reason 3: Bad‑Case repair creates a “reverse classroom”

The process—error detection, reconstruction of a counter‑example, demonstration of correct reasoning, and RLHF reinforcement—forms a loop that continuously upgrades model ability.

4. How to repair a Bad Case

The repair is not a simple rewrite; it follows a four‑step pipeline:

Step 1: Error localization

Identify why the answer is wrong (e.g., missing decision framework).

Step 2: Provide a reasoning structure

预算规划需要先评估收入
再划分固定支出、弹性支出与储蓄
再做比例
再做风险缓冲

Step 3: Demonstrate the correct answer

第一步：计算净收入
第二步：固定支出控制在百分比
第三步：弹性支出可浮动
第四步：留应急金
第五步：周期复盘

Step 4: Abstract the reasoning pattern

Teach the model to first abstract a framework, then instantiate steps, and finally output a structured prompt.

5. Bad‑Case repair as the RLHF life‑cycle backbone

模型输出错误 → Bad Case 标注 → 重写结构化示范 → RM‑R1 学它为什么错 → GRPO / PPO 改进策略模型 → 模型少犯错 → 更细的 Bad Case → 更高维能力

This loop continuously raises model capability, whereas without it the model’s output becomes more uniform, verbose, and improves accuracy very slowly.

6. Interview‑ready summary

Adding data mainly increases coverage, while Bad‑Case repair defines, explains, and corrects the model’s capability gaps, turning structural errors into alignment signals and transferable ability structures.

Thus, in real RLHF projects, model growth is driven by Bad‑Case repair rather than sheer sample quantity.