Jev: The Non-Generative AI Model Beating LLMs at Classification 200x Faster

Introduction: The Problem with LLMs for Classification

A team building an AI customer-service system needed to route tickets to the correct department (technical, billing, sales). Using a large language model, each classification took 2-3 seconds because the model first generated a reasoning trace before emitting the label, and sometimes invented new categories like "comprehensive support." The technical lead noted: "We spent most of our token budget on the AI's 'chatter.'"

Jev, released September 15, 2026 by TypeSafe AI (founded by former OpenAI researcher Diogo Almeida), solves this by not generating text at all. It is a "System One" model that takes a state (text or JSON) plus a set of typed questions and returns typed answers with calibrated confidence probabilities. Vercel's blog called it the fastest-adopted model in AI Gateway history: within 24 hours, nearly 13% of paid teams were using it — 2x adoption of GPT-5.6 series and 6x+ of Fable 5.1.

What Jev Does Not Do

Traditional classifiers (Naive Bayes, logistic regression, fine-tuned BERT) output a single label from a fixed set seen during training; they do not understand the semantics of the options. Jev, by contrast, receives a state and a set of typed questions (Choice, Score, Noul) and returns a typed answer with a calibrated probability. It has no messages, no generation, no streaming. Asked "urgent or not urgent," it returns a probability between 0 and 1, not an explanation.

Core Architecture: Three Primitives

Choice — select from a candidate set, returning a full probability distribution.

Score — rate against an ordered scale (2-10 levels), returning a weighted position.

Noul — give the probability that a judgment holds, 0-1.

Every answer includes a calibrated confidence. You can set a threshold: above it, auto-execute; below, escalate to human. "Your escalation strategy moves from a paragraph in a prompt to a single number in a config file."

Why Jev Is 200x Faster

3.1 Autoregressive Generation vs. Non-Autoregressive Scoring

Traditional LLMs generate token-by-token even for a single-letter answer ("The answer is A"), wasting time and tokens on intermediate tokens. Jev skips autoregressive decoding entirely, using a single parallel forward pass to score directly on hidden states. Business code receives a typed result, no parsing needed.

3.2 Parallel Multi-Question Evaluation

All questions in a single request are evaluated in one forward pass. Whether you ask 1 or 4 questions, latency stays 70-500 ms. Traditional LLMs serialize: 4 questions ≈ 4x latency.

3.3 Training: RLCD (Reinforcement Learning for Calibrated Decisions)

TypeSafe discloses three keywords: new architecture, parallel sampler, RLCD. Unlike RLHF which optimizes for "answers humans like," RLCD optimizes for "answers with honest probabilities." When Jev says 95%, statistically 95% are correct.

Benchmark Comparison

Output mode : Traditional LLM — token-by-token text generation; Jev — hidden-state direct scoring.

End-to-end latency : Traditional LLM — thousands of ms; Jev — 70-500 ms .

Speed vs. baseline : Traditional LLM — 1x; Jev — 20-200x faster .

Cost vs. baseline : Traditional LLM — 1x; Jev — 40-400x cheaper .

Output tokens : Traditional LLM — metered; Jev — Free forever .

Input price : Traditional LLM — baseline; Jev — $0.042 per million tokens .

Independent benchmarks: Jev median latency 105 ms vs. GPT-5.6 Luna 710 ms (reasoning off) / 808 ms (low reasoning). In extreme workflow tests, Jev up to 193.6x faster, 444.6x cheaper .

Java Ecosystem Integration

Official SDKs exist only for Python and JavaScript; TypeSafe recommends Java developers call the HTTP API directly. The community has filled the gap:

4.1 Community Java SDK (Recommended)

Pure Java 17+, only dependency Jackson. Does not depend on Spring AI's ChatModel because ChatModel assumes autoregressive models (messages in, generated text out, streaming). Jev has no messages, no generation, no streaming. Forcing it into ChatModel would lose typed questions and probability access — Jev's core value.

// Pure Java 17+, only dependency is Jackson
TypeSafeClient client = TypeSafeClient.fromEnv();
SystemOneResult result = client.evaluate(
    EvaluationRequest.of(
        "Help! My payouts have been failing for 3 days.")
    .noul("is_urgent", "Does this convey urgency?")
    .choice("department", "Which team should handle this?", Map.of(
        "billing", "Payments, invoicing, refunds",
        "technical", "Bugs, outages, integrations"))
    .score("frustration", "How frustrated is the customer?", 
        List.of("Calm", "Frustrated", "Very angry"))
    .build()
);

// Branch on probability
if (result.noul("is_urgent").isYes(0.7)) {
    // escalate
}

ChoiceAnswer dept = result.choice("department");
if (dept.confidenceOrZero() < 0.5) {
    // low confidence, hand off to human
}

Answers are not strings but type-safe values. Probabilities are first-class citizens — code branches on confidence. SDK also provides sync+async API (CompletableFuture), automatic retry with exponential backoff + jitter respecting Retry-After header, client-side validation throwing InvalidRequestException, and sealed type hierarchies for Question/Answer with UnknownAnswer fallback.

4.2 Spring Boot Starter

<dependency>
  <groupId>io.typesafe</groupId>
  <artifactId>typesafe-ai-java-spring-boot-starter</artifactId>
</dependency>

Auto-configures TypeSafeClient for injection into services.

4.3 Kotlin Client (kev)

Ktor Client + kotlinx.serialization, fully coroutine-based suspend functions.

4.4 MCP Server (jev-mcp-spring)

Spring AI-based MCP server exposing classify, score, check, health tools over HTTP and Streamable HTTP/SSE for agent workflows.

Use Cases with Real-World Numbers

5.1 Ticket Classification & Routing

Single request judges urgency, department, and sentiment in parallel — total latency <100 ms. Vercel case study: engineer Pranit Sharma's company replaced OpenAI ChatGPT Luna 5.6 with Jev for command safety review; speed improved 5-18x with higher accuracy.

5.2 Browser Agents: Choose Next Action, Don't Generate It

APUS's fast-browser-use extracts visible interactive elements into a numbered candidate set; a local Qwen3.5-9B model uses Jev's single forward pass to decide "click which, select which." On Apple M2 Pro, median Wikipedia retrieval task ~18 s, form fill/navigation ~3 s, only 4 model scoring calls per task, fully offline, zero cloud cost.

5.3 Model & Tool Selection

Agent picks a model or tool from candidates via Choice primitive. 1,000 email classification: Jev ~6 seconds, $0.09; GPT-5.6-class model ~5 minutes, $0.62.

5.4 Agent Execution Supervision

Jev monitors LLM agent trajectories to prevent jailbreaks/anomalies. Low cost and high speed make large-scale supervision feasible.

Jev's Role in Agent Workflows: Fast/Slow Division

Consensus emerging: expensive LLMs handle planning, reasoning, generation ("slow thinking"); high-frequency atomic classification, selection, scoring ("fast judgment") go to lightweight decision models like Jev. Diogo Almeida: "We optimized for human language for four years, but for automation that's useless. Computers speak a different language." Jev speaks the computer's language — types, probabilities, determinism.

Accuracy & Calibration (Independent Benchmarks)

49 tasks, 8,225 test items:

Jev ties or beats LLM baseline on 42 tasks.

Median latency 105 ms vs. baseline 710-808 ms.

Cost per 1,000 items $0.00016-$0.00019 vs. baseline $0.16-$0.19.

Strengths: Logical reasoning (LogiQA 0.77 vs 0.59), commonsense (WinoGrande 0.89 vs 0.66), science QA (ARC-Challenge 0.97 vs 0.87), knowledge (MMLU 0.94 vs 0.87). Calibration error (ECE) 0.07 vs. baseline 0.14-0.18. Keeping the most confident half of answers lifts accuracy by 7.6 percentage points — probabilities are actionable.

Weaknesses: Counting tasks (0.87 vs 0.99), very large option sets (77 options: 0.81 vs 0.87), negation inconsistency (average deviation 0.32 between a question and its negation).

"Zero Hallucination" — What It Really Means

Jev guarantees pattern matching: given options A, B, C, it will not invent D. Output space is strictly constrained to your candidate set. But it can still pick B when the correct answer is A. Type-correct, not fact-correct. "Zero hallucination" = it won't fabricate options, but it may misjudge. Calibrated probabilities let you manage risk via confidence thresholds. Pi framework CTO Armin Ronacher: "It shifts part of the hallucination problem to the user. If it returns 50%, treat it as a coin flip; if 95%, you can act on it."

Pros & Cons Summary

Pros

Extreme speed: 70-500 ms end-to-end, 20-200x faster; multi-question parallel, latency flat.

Extreme cost: output tokens free, input $0.042/M; 40-400x cheaper for decision tasks.

Type-safe: typed answers and distributions, no parsing; Java code branches directly on results.

Calibrated confidence: ECE 0.07 (half of baselines); top-half confidence boosts accuracy 7.6 pp.

No fabricated options: output space locked to your candidates.

Java ecosystem maturing: community SDK, Spring Boot starter, Kotlin client, MCP server with sync/async, retry, validation.

Clear agent role: fast/slow division — LLMs plan, Jev judges.

Cons

No official Java SDK (community-only or raw HTTP).

Fully closed-source: cloud API only, no weights, architecture/sampler/RLCD details unpublished.

Not a universal classifier: only "choose from given candidates." Cannot generate, reason, or plan.

Accuracy ceiling: underperforms LLMs on counting, huge option sets, negation consistency.

Confidence is statistical: reliable in aggregate, not a per-call guarantee.

Applicability Matrix

Ticket classification / intent recognition — ✅✅✅ Strongly recommended. Reason: multi-question parallel, sub-100 ms latency.

Browser agent action selection — ✅✅✅ Strongly recommended. Reason: candidate actions to Choice, eliminates format hallucination.

Model / tool routing — ✅✅✅ Strongly recommended. Reason: semantic choice among candidates, more reliable than LLM "thinking".

Content moderation / risk scoring — ✅✅✅ Strongly recommended. Reason: calibrated probabilities enable threshold-based auto-tiering.

Agent trajectory supervision — ✅✅✅ Strongly recommended. Reason: low cost enables large-scale monitoring.

Bulk data classification — ✅✅✅ Strongly recommended. Reason: 1,000 emails in 6 s for $0.09 vs. 5 min $0.62.

Tasks requiring text generation — ❌ Not recommended. Reason: Jev does not generate text.

Tasks requiring complex reasoning — ❌ Not recommended. Reason: Jev only judges, does not reason.

Very large option sets (>255) — ⚠️ Evaluate. Reason: requires two-stage mode, accuracy drops.

Conclusion

Jev wins because it cuts the most expensive, slowest part of AI — "talking." Using an autoregressive LLM for a classification is like hiring a novelist to press an elevator button: they draft, wordsmith, and type one character at a time, while you just need "3rd floor." Jev skips the writing and presses the button directly. It's not faster because it's smarter; it's faster because it doesn't do what isn't needed. 70 ms per judgment, zero output token cost — a cost structure autoregressive models can never match. The fast/slow agent division is becoming standard: LLMs for planning/reasoning/generation (slow), Jev for high-frequency atomic classification/selection/scoring (fast). Vercel's 24-hour adoption (13% of paid teams, 2x GPT-5.6, 6x+ Fable 5.1) confirms the shift.

Reference: Jev official docs — https://docs.typesafe.ai