Can 10% of Instruction Data Match Full-Scale Fine-Tuning? The SPICE Solution
The SPICE method leverages Fisher Information Matrix submodularity and a novel gradient‑conflict penalty to select a small, high‑quality subset of instruction‑tuning data, achieving comparable or superior performance to full‑data fine‑tuning while dramatically reducing training cost.
