The SPICE method leverages Fisher Information Matrix submodularity and a novel gradient‑conflict penalty to select a small, high‑quality subset of instruction‑tuning data, achieving comparable or superior performance to full‑data fine‑tuning while dramatically reducing training cost.
This article provides a detailed technical analysis of the Olmo‑Thinking project, covering why a new open‑source LLM was built, the challenges of reinforcement learning at scale, data‑mix optimization, architectural bottlenecks such as missing GQA and QK‑Norm, and the post‑training techniques used to improve reasoning and long‑context capabilities.
Ant Insurance’s tech team leveraged reinforcement learning, focused data selection, and a multi‑dimensional reward system to dramatically reduce hallucinations in LLMs, achieving top‑rank performance on the HHEM leaderboard and robust improvements across instruction‑following and reasoning‑enhanced models.
This article examines the conditions under which graph neural network pre‑training is beneficial, proposes a data‑centric generator framework to assess transferability, introduces a data‑active pre‑training strategy that selects informative graphs, and presents experimental results showing that using less, well‑chosen data can outperform full‑scale pre‑training.
This article explains the Nuggets approach for selecting a high‑quality subset of annotated instructions to fine‑tune large language models, describing its three inputs, the gold‑score computation based on perplexity improvement, empirical results on Alpaca, and practical considerations such as task‑set design.
The article explains practical techniques for choosing and constructing fine‑tuning data for large language models, covering data diversity through similarity‑based clustering, semi‑supervised filtering with binary classifiers, and uncertainty‑driven sampling using perplexity or reward models to build an efficient, low‑cost pipeline.
This article presents eight practical pandas data‑selection techniques—including boolean indexing, loc/iloc, isin, str.contains, where/mask, query, filter, and any/all—illustrated with code examples and visual outputs to help Python users efficiently extract and analyze data.
