Traditional NLP Returns: Specialized Small Models Beat LLMs on Cost & Speed

ScaleDown CDM Architecture and Benchmark Results

ScaleDown launched its Constrained Decoding Model (CDM) architecture with three task-specific models: Classify, Extract, and Summarize. Unlike general LLMs that generate tokens sequentially, CDM reads the input once and directly returns structured output, skipping the token-by-token generation loop.

In benchmarks published by ScaleDown and covered by Forbes (June 2026), CDM models were compared against Claude, GPT, and Gemini series on classification, extraction, and summarization tasks:

vs. Claude: +8% average accuracy, 1/161 cost, 3.8x speed

vs. OpenAI: +8.72% accuracy, 1/89 cost, 2.4x speed

vs. Gemini: +9% accuracy, 1/29 cost, 8.3x speed

For text classification, ScaleDown's cost was ~5,250x lower than Anthropic models and ~1,810x lower than OpenAI models. In a summarization scenario of 10,000 summaries/day, ScaleDown estimated $7.20/day vs. $58/day for GPT-4.1 Mini, with human evaluation showing no significant quality difference.

Model Capabilities and Pricing

Classify : Accepts predefined labels, returns calibrated probabilities and reasoning for each label. Use cases: customer service routing, intent detection, agent dispatch.

Extract : Takes natural-language descriptions of target entities, outputs structured JSON with confidence scores and citations. Use cases: extracting companies, names, dates from news; key clauses from contracts; PII detection.

Summarize : Handles long text, images, or PDFs up to 1 million tokens, returns summaries ≤1,000 tokens. Use cases: compressing agent context, generating document metadata, meeting/interview summarization.

Pricing: $0.05 per million input tokens, output free (including summaries). Integrated with OpenRouter, Vercel, LiteLLM, LangChain.

Jev Evolution and Multimodal Update

Jev (TypeSafe AI) focuses on structured decision-making: given context and candidate answers, it selects one and returns probabilities. On DecisionBench, ScaleDown's CDM scored 77.2% vs. Jev's 72%. ScaleDown also claims a 1M token context window, 31x Jev's.

Jev 1.14.0 added multimodal capabilities (images as input and as candidate examples) and a new "Box" output type that returns bounding-box coordinates for object localization in images (e.g., finding a coffee cup in a kitchen photo). These features are experimental and require manual enablement.

Industry Implications

The trend reflects a pragmatic reassessment: not every task needs a massive reasoning model. For high-volume, well-defined NLP tasks, specialized small models deliver better cost, speed, and output stability. As one founder put it, "You already gave the model the answer — why make it rewrite from scratch?" The "traditional NLP comeback" is less about old tech beating new tech, and more about right-sizing model capability to task requirements.

Small language models are now outperforming frontier AI on cost, speed, and accuracy for specific tasks. — Forbes, June 2026

References: Forbes article (https://www.forbes.com/sites/joetoscano1/2026/06/25/small-language-models-outperform-frontier-ai-on-cost-speed-and-accuracy/), Neal Patel tweet (https://x.com/neal_k_patel/status/2109978626124898587), EGafni tweet (https://x.com/EGafni/status/2110489489265312255), Jev 1.14 release notes (https://typesafe.ai/blog/jev-1-14).

Code example

[1]https://www.forbes.com/sites/joetoscano1/2026/06/25/small-language-models-outperform-frontier-ai-on-cost-speed-and-accuracy/
[2]https://x.com/neal_k_patel/status/2109978626124898587?s=46&t=iTysI4vQLQqCNJjSmBODPw
[3]https://x.com/EGafni/status/2110489489265312255?s=20
[4]https://typesafe.ai/blog/jev-1-14