How Hierarchical Curriculum Learning Improves Dialogue Response Selection

Background

Existing dialogue response selection models typically treat all negative candidates as equally unhelpful, sampling random responses as negatives. This ignores the varying difficulty of negatives, making it hard for models to handle hard distractors in real conversations.

Core Motivation

Not all negatives are equally negative; some are easy to distinguish, while others are hard. For example, the sentence "The restaurant is expensive." is trivial, whereas a factual statement about a TV series is much harder to reject, demonstrating the need for differentiated negative sampling.

Method: Hierarchical Curriculum Learning (HCL)

The proposed HCL framework consists of two curricula:

Corpus‑Level Curriculum (CC) : gradually increases the difficulty of training instances based on a difficulty function pcc(t). Only samples with difficulty ≤ pcc(t) are used at step t.

Instance‑Level Curriculum (IC) : similarly controls the progression of negative difficulty with a function pic(t), where a smaller pic(t) indicates higher difficulty.

Difficulty is measured using a pretrained ranking model that computes a relevance score G(c, r) between context c and response r. The corpus‑level difficulty d_cc(c,r) normalizes G by the maximum score, while instance‑level difficulty is defined as the rank of G(c,r) in descending order.

Training starts with the easiest examples ( pcc(0) low) and progresses until pcc = 1, at which point any sample may be used.

Curriculum Details

Corpus‑Level Curriculum (CC)

The function pcc(t) controls the upper bound of difficulty; as t grows, the allowed difficulty increases, allowing the model to learn from progressively harder corpus examples.

Instance‑Level Curriculum (IC)

Initially negatives are randomly sampled (easy). As training proceeds, harder negatives are sampled from higher‑scoring responses, guided by pic(t). This gradually improves the model's ability to handle strong distractors.

Experiments

Datasets used: Douban (Chinese multi‑turn dialogues), Ubuntu (forum chat logs), and E‑Commerce (customer‑service chats). Each context provides ten candidate responses.

Evaluation metrics: MAP, MRR, P@1, and R_n@k.

Results show that both CC and IC individually improve performance; IC yields larger gains, indicating that recognizing mismatched information is crucial for optimal results.

Ablation studies on the Douban dataset confirm the contribution of each curriculum component.

Comparison with other learning strategies (Semi, CIR, Gray) demonstrates that HCL achieves the best scores across all metrics while remaining simpler—requiring no extra generative model or periodic re‑scoring of negatives.

Conclusion

The hierarchical curriculum learning framework effectively guides models from easy to hard examples at both corpus and instance levels, leading to superior dialogue response selection performance.