Jev: The Non-Generative AI Model That's 200x Faster Than LLMs for Structured Decisions

Introduction: The Problem with LLMs for Classification

A team optimizing an AI customer-service system needed to classify tickets into departments (technical, billing, sales). Using a conventional LLM, each classification took 2-3 seconds because the model first generated a reasoning trace before emitting the label, and sometimes invented labels outside the provided set. The technical lead noted: "We spent most of our token budget on the AI's 'chatter'."

What Jev Is

Released on 15 September 2026 by TypeSafe AI (founded by former OpenAI researcher Diogo Almeida), Jev is a "System One" model that does not generate text. Instead, it accepts a state (arbitrary text or JSON) and a set of typed questions , then returns typed answers with calibrated confidence probabilities . It has no chat messages, no streaming, no free-form generation — it acts as a "semantic logic gate" that hands structured decisions back to code.

Core Capabilities

Choice — select from a candidate list, returning a full probability distribution.

Score — rate against an ordered scale (2-10 levels), returning a weighted position.

Noul — output the probability that a judgment holds, as a value between 0 and 1.

Every answer includes a calibrated confidence score, allowing threshold-based routing (e.g., auto-execute above 0.9, escalate to human below).

Why Jev Is 20-200x Faster and 40-400x Cheaper

Non-Autoregressive Single-Pass Inference

Traditional LLMs generate tokens sequentially — even to output "A" they first emit "The answer is" then "A". Jev skips autoregressive decoding entirely, performing a single parallel forward pass that scores directly on hidden states. One forward pass evaluates all questions simultaneously.

Parallel Multi-Question Evaluation

With LLMs, each additional question adds serial generation time. Jev evaluates all questions in the same forward pass, so latency stays flat at 70-500 ms regardless of question count.

Training: RLCD (Reinforcement Learning for Calibrated Decisions)

Unlike RLHF which optimizes for human-preferred answers, RLCD optimizes for honest probabilities . When Jev says 95%, statistically 95% of those answers are correct (ECE = 0.07 vs. 0.14-0.18 for baselines). Keeping the top-half most confident answers raises accuracy by 7.6 percentage points.

Comparison: Traditional LLM vs Jev

Output mode : Traditional LLM — token-by-token text generation; Jev — hidden-state direct scoring

End-to-end latency : Traditional LLM — thousands of ms; Jev — 70-500 ms

Speed vs. baseline : Traditional LLM — 1x; Jev — 20-200x faster

Cost vs. baseline : Traditional LLM — 1x; Jev — 40-400x cheaper

Output tokens : Traditional LLM — metered; Jev — free forever

Input price : Traditional LLM — baseline; Jev — $0.042 per million tokens

Independent benchmarks (49 tasks, 8,225 items): median latency 105 ms vs. 710-808 ms for GPT-5.6 Luna; cost per 1,000 items $0.00016-$0.00019 vs. $0.16-$0.19. Jev matches or beats LLM baselines on 42/49 tasks, excelling in logical reasoning (LogiQA 0.77 vs. 0.59), commonsense (WinoGrande 0.89 vs. 0.66), science QA (ARC-Challenge 0.97 vs. 0.87), and knowledge (MMLU 0.94 vs. 0.87). Weaknesses: counting tasks (0.87 vs. 0.99), very large option sets (77 options: 0.81 vs. 0.87), and inconsistent probabilities for a question and its negation (mean deviation 0.32).

Java Ecosystem Integration

Official SDKs exist only for Python and JavaScript, but the community has filled the gap:

1. Community Java SDK (recommended)

Pure Java 17+, single dependency (Jackson). It deliberately avoids Spring AI's ChatModel abstraction because that assumes autoregressive generation with streaming — Jev has none of those. The SDK exposes typed questions and answers as first-class citizens.

// Pure Java 17+, only dependency is Jackson
TypeSafeClient client = TypeSafeClient.fromEnv();
SystemOneResult result = client.evaluate(
    EvaluationRequest.of("Help! My payouts have been failing for 3 days.")
        .noul("is_urgent", "Does this convey urgency?")
        .choice("department", "Which team should handle this?", Map.of(
            "billing", "Payments, invoicing, refunds",
            "technical", "Bugs, outages, integrations"))
        .score("frustration", "How frustrated is the customer?", 
            List.of("Calm", "Frustrated", "Very angry"))
        .build());

// Branch on probability
if (result.noul("is_urgent").isYes(0.7)) {
    // escalate
}
ChoiceAnswer dept = result.choice("department");
if (dept.confidenceOrZero() < 0.5) {
    // low confidence → human review
}

Features: sync + async ( CompletableFuture), automatic retry with exponential backoff/jitter honoring Retry-After, client-side request validation throwing InvalidRequestException, sealed type hierarchies for Question / Answer with UnknownAnswer fallback.

2. Spring Boot Starter

<dependency>
  <groupId>io.typesafe</groupId>
  <artifactId>typesafe-ai-java-spring-boot-starter</artifactId>
</dependency>

Auto-configures TypeSafeClient for direct injection.

3. Kotlin Client ( kev )

Built on Ktor Client + kotlinx.serialization, fully coroutine-based suspend functions.

4. MCP Server ( jev-mcp-spring )

Spring AI-based MCP server exposing classify, score, check, health tools over HTTP and Streamable HTTP/SSE for agent workflows.

Key Use Cases

Ticket classification & routing — single request evaluates urgency, department, and sentiment in parallel (<100 ms). Vercel case study: 5-18x speedup, higher accuracy vs. ChatGPT Luna 5.6.

Browser agents — APUS's fast-browser-use feeds numbered interactive elements to Jev's Choice primitive; on an Apple M2 Pro, median Wikipedia retrieval task completes in ~18 s offline, form fills in ~3 s, only 4 model scoring calls, zero cloud cost.

Model/tool routing — 1,000 emails classified in ~6 s for $0.09 vs. 5 min and $0.62 with a frontier LLM.

Agent trajectory supervision — low-cost, high-speed monitoring of LLM agents for jailbreaks or anomalies.

Agent Architecture: Fast/Slow Division of Labor

Emerging consensus: expensive LLMs handle planning, reasoning, generation ("slow thinking"), while high-frequency atomic judgments (classification, selection, scoring) are delegated to lightweight decision models like Jev ("fast judgment"). Diogo Almeida: "We optimized human language for four years, but for automation that's useless. Computers speak a different language." Jev speaks the computer's language — types, probabilities, determinism.

"Zero Hallucination" Clarified

Jev guarantees pattern matching : given options A, B, C it will never invent D. However, it can still pick B when the correct answer is A. "Zero hallucination" means it won't fabricate options, but it can misjudge . Calibrated probabilities let you manage this risk via confidence thresholds. As Armin Ronacher (CTO of Pi) put it: "It pushes the hallucination problem partly to the user. If it returns 50%, treat it as a coin flip; if 95%, act on it."

Pros & Cons

Pros

Extreme speed (70-500 ms, 20-200x faster).

Extreme cost efficiency (output tokens free, input $0.042/M, 40-400x cheaper).

Type-safe structured outputs — no string parsing.

Calibrated confidence (ECE 0.07, top-half accuracy +7.6 pp).

Cannot invent options outside the candidate set.

Growing Java ecosystem (SDK, Spring starter, Kotlin client, MCP server).

Clear role in agent fast/slow architectures.

Cons

No official Java SDK (community-maintained only).

Fully closed-source, cloud-only API, no public weights or architecture details.

Not a universal classifier — only "choose from given candidates"; cannot generate, reason, or plan.

Accuracy ceiling: weaker on counting, huge option sets, and negation consistency.

Confidence is statistically calibrated, not a per-call correctness guarantee.

Applicability Matrix

Strongly Recommended (✅✅✅)

Ticket classification / intent recognition — multi-question parallel, sub-100 ms latency

Browser agent action selection — candidate actions fed to Choice, eliminates format hallucination

Model / tool routing — semantic selection among candidates, more reliable than LLM "reasoning"

Content moderation / risk scoring — calibrated probabilities enable threshold-based auto-tiering

Agent trajectory supervision — low cost enables large-scale monitoring

Bulk data classification — 1,000 emails in 6 s for $0.09 vs. 5 min / $0.62

Not Recommended (❌)

Text generation tasks — Jev does not generate text

Complex reasoning tasks — Jev only judges, does not reason

Evaluate (⚠️)

Very large option sets (>255) — requires two-stage pipeline, accuracy drops

Conclusion

Jev wins because it stops doing the unnecessary work . Asking an autoregressive LLM to classify is like hiring a novelist to press an elevator button — they draft prose before pressing "3". Jev skips the writing and just presses the button. 70 ms per judgment, zero output token cost — a cost structure autoregressive models can never match. The fast/split architecture is becoming standard: LLMs for slow thinking, Jev for fast judgment. Vercel's 24-hour adoption (13% of paid teams, 2x GPT-5.6, 6x+ Fable 5.1) confirms the shift.

Jev official docs : https://docs.typesafe.ai