Can Small Language Models Match Big AI with the Skills Framework?

Progressive Disclosure and Decision Cost

The agent’s behavior is formalized as a partially observable Markov decision process (POMDP). Before each action the model decides whether to execute a Skill directly or to spend additional computation to reveal more details about that Skill, trading attention cost for clearer context.

Datasets and Evaluation

IMDB – basic sentiment classification.

FiNER – financial named‑entity‑recognition with complex domain tags.

InsurBench – real‑world insurance claim emails that are long, multilingual, and noisy.

These benchmarks test the model’s ability to extract precise actions from noisy, real‑world inputs.

Parameter Scale and Skills Routing

Open‑source models ranging from 200 M to 80 B parameters were compared against a high‑efficiency closed‑source baseline. Medium‑sized models (≈10 B–30 B) achieved the largest gains in Skills routing, especially on the financial and insurance datasets. Very small models (<4 B) struggled to identify relevant Skills. Model families were split into reasoning‑oriented and code‑generation‑oriented variants, showing that training objectives significantly affect Skills utilization.

Code‑Specialized Models Reduce Memory and Latency

Code‑focused models consistently outperformed instruction‑tuned counterparts on the lengthy insurance‑claim task, achieving higher accuracy while using less GPU memory and exhibiting lower end‑to‑end latency. This demonstrates a cost‑effective deployment path for budget‑constrained enterprises.

Impact of History and Prompt Keywords

Keeping dialogue history improves performance for medium‑sized models but nearly doubles memory consumption for very large models (≈80 B) with minimal accuracy gain. Replacing the prompt keyword “Skills” with synonyms such as “Expertise” or “Know‑how” alters both performance and memory footprint, indicating sensitivity to prompt phrasing.

Conclusions

The progressive, static‑Skills framework enables small and medium language models (≥4 B parameters) to approach the performance of much larger models on specialized industrial tasks, provided the Skills pool is carefully curated. Code‑specialized models further enhance efficiency, offering a practical solution for secure, low‑cost industrial AI deployment.

Reference: https://arxiv.org/pdf/2602.16653