Tagged articles

LLM routing

10 articles · Page 1 of 1

Oct 23, 2026 · Artificial Intelligence

Smart LLM Routing Still Saves 90%+ Despite Cache Loss on Model Switch

Analysis shows HiRoute's intelligent model routing saves over 90% costs despite cache loss when switching models, because switches are infrequent, one-time losses amortize over many requests, cheaper models often cost less even with full reprocessing, and cache discounts only affect input while savings come from output-heavy cheaper models.

AstraHiRouteKV cache

0 likes · 11 min read

Smart LLM Routing Still Saves 90%+ Despite Cache Loss on Model Switch

21CTO

Oct 18, 2026 · Industry Insights

5 Best AI Gateways Compared: Benchmarks, Routing & Enterprise Features

This article compares five leading AI gateways—Bifrost, Cloudflare AI Gateway, Vercel AI Gateway, LiteLLM, and OpenRouter—evaluating their performance benchmarks, model routing capabilities, deployment options, observability, cost control, and suitability for different production environments.

AI gatewayDeploymentLLM routing

0 likes · 24 min read

5 Best AI Gateways Compared: Benchmarks, Routing & Enterprise Features

Linyb Geek Road

Sep 24, 2026 · Artificial Intelligence

How Jev Cuts Agent Orchestration Costs 80% with Dynamic Harness Generation

Jev, a specialized classification model, reduces agent orchestration costs by 80% by handling routing, evaluation, intent decomposition, and dynamic harness generation at a fraction of GPT-4o's cost, enabling self-orchestrating multi-agent systems while maintaining quality, though plan-level error detection remains an open challenge.

JevLLM routingLLM-as-a-Judge

0 likes · 12 min read

How Jev Cuts Agent Orchestration Costs 80% with Dynamic Harness Generation

Architect

Sep 22, 2026 · Artificial Intelligence

Jev: Extracting Semantic Micro-Judgments from Agent Loops into Observable Decision Layers

TypeSafe's Jev isolates semantic micro-judgments — routing, risk scoring, boolean checks — from generative LLM calls using Choice, Score, and Noul primitives that return typed probabilities, enabling policy code to branch directly on observable, auditable signals instead of parsing free-text model outputs.

Choice Score NoulJevLLM routing

0 likes · 25 min read

Jev: Extracting Semantic Micro-Judgments from Agent Loops into Observable Decision Layers

Su San Talks Tech

Sep 22, 2026 · Artificial Intelligence

Jev: The 200x Faster AI That Replaces Text Generation with Structured Decisions

Jev, a non-autoregressive 'System One' model from TypeSafe AI, replaces token-by-token text generation with parallel hidden-state scoring to deliver 70–500 ms latency and 40–400× cost savings for classification, routing, and scoring tasks, while returning calibrated probabilities that enable confidence-threshold automation in Java, Python, and JavaScript ecosystems.

AI AgentsJava SDKJev

0 likes · 21 min read

Jev: The 200x Faster AI That Replaces Text Generation with Structured Decisions

Architecture Digest

Aug 12, 2026 · Artificial Intelligence

Practical Multi‑Model Routing with Embabel: Mixing DeepSeek and Claude

The article explains why a single LLM cannot satisfy all stages of an AI pipeline, introduces Embabel's declarative routing that separates concerns across four layers, shows how a four‑dimensional decision matrix assigns cheap or best models to each step, and presents benchmark results demonstrating up to 70% cost reduction while retaining 95% of the quality of an all‑Claude solution.

ClaudeDeepSeekEmbabel

0 likes · 16 min read

Practical Multi‑Model Routing with Embabel: Mixing DeepSeek and Claude

ThinkingAgent

Jul 5, 2026 · Artificial Intelligence

Building L1 AI Infra: Model Gateways, Smart Routing, and High‑Performance Inference Engines

The article presents a comprehensive, production‑ready guide for the L1 layer of AI infrastructure, detailing how model gateways unify calls, intelligent routing selects the optimal model, inference engines maximize GPU throughput, and quantization and KV‑Cache techniques dramatically cut costs while maintaining performance.

AI infrastructureKV cacheLLM routing

0 likes · 25 min read

Building L1 AI Infra: Model Gateways, Smart Routing, and High‑Performance Inference Engines

AI Engineering

Jun 29, 2026 · Artificial Intelligence

How Coinbase Halved AI Costs While Token Usage Continued to Surge

In June, Coinbase CEO Brian Armstrong revealed an internal AI cost‑optimization program that cut the company's AI dollar spend by almost 50% while token consumption kept growing exponentially, achieved through five concrete measures involving model defaults, intelligent routing, cache reuse, context trimming, and transparent usage monitoring.

AI cost optimizationAI infrastructureCoinbase

0 likes · 9 min read

How Coinbase Halved AI Costs While Token Usage Continued to Surge

James' Growth Diary

May 15, 2026 · Artificial Intelligence

Five Intent Recognition Designs: From Keyword Matching to Classifier to LLM Self‑Routing – A Decision Tree to Choose the Right One

The article breaks down five production‑grade intent‑recognition designs—keyword matching, regex‑rule engine, embedding classifier, fine‑tuned small model, and zero‑shot LLM routing—provides code snippets, latency and cost benchmarks, decision‑making rules, and shows how a layered architecture can cut API costs from ¥80,000 to ¥3,000 while keeping accuracy above 90%.

LLM routingLayered ArchitectureRule Engine

0 likes · 16 min read

Five Intent Recognition Designs: From Keyword Matching to Classifier to LLM Self‑Routing – A Decision Tree to Choose the Right One

Efficient Ops

Mar 23, 2026 · Artificial Intelligence

7 Multi‑Agent Design Patterns Every AI Engineer Should Know

This article explains the seven core multi‑agent design patterns—workflow, routing, parallel, loop, aggregation, network, and hierarchical—detailing their mechanics, use cases, implementation tips, and why modern agent frameworks are essential for dynamic, cross‑system AI applications.

LLM routingMulti-Agent Systemsagent frameworks

0 likes · 12 min read

7 Multi‑Agent Design Patterns Every AI Engineer Should Know