The SPICE method leverages Fisher Information Matrix submodularity and a novel gradient‑conflict penalty to select a small, high‑quality subset of instruction‑tuning data, achieving comparable or superior performance to full‑data fine‑tuning while dramatically reducing training cost.
The paper reveals that standard shared‑parameter Transformers suffer severe gradient conflict when jointly processing low‑entropy text and high‑entropy visual tokens, and proposes Uni‑X—a two‑end‑separated, middle‑shared autoregressive model that isolates modality‑specific layers, reduces conflict, improves efficiency, and achieves strong results on image generation and editing benchmarks.
The paper introduces ConFIG (Conflict‑Free Inverse Gradients), a mathematically proven method that resolves gradient conflicts among multiple loss terms in physics‑informed neural networks, multi‑task learning, and continual learning, and its momentum‑based variant M‑ConFIG that further accelerates training while maintaining accuracy.
