Wu Shixiong's Large Model Academy

Aug 7, 2026 · Artificial Intelligence

Why a Single -100 Line Determines Who the Multi‑Round SFT Learns to Speak

The article explains how using -100 as an ignored label in PyTorch cross‑entropy loss silently masks non‑assistant tokens, how to locate assistant spans via prefix‑difference, the trade‑offs between supervising only the final reply versus all assistant turns, and the essential pre‑training checks to avoid hidden masking errors in multi‑round SFT.