JEV and System 1/2: Why AI Engineering Needs Fast/Slow Model Split

Introduction

After nearly two years of using Claude Code, GPT-5.x, Grok, and similar models for all tasks, the author questioned why simple problems are also sent to large models — a waste akin to asking a PhD to handle front-desk reception.

This changed upon discovering JEV, a project from TypeSafe AI. JEV is not a chat model but a "fast-thinking" model designed for software automation decisions, featuring a new architecture (non-standard Transformer), a parallel sampler, and a training algorithm called Reinforcement Learning for Calibrated Decisions (RLCD). While JEV itself is unproven, it signals a broader trend: System 1/2 layered models.

System 1/2: A Nobel-Winning Cognitive Framework

The concept originates from Daniel Kahneman's Thinking, Fast and Slow (2011). System 1 is fast, intuitive, automatic, and unconscious (e.g., instantly knowing 2+2=4). System 2 is slow, logical, effortful, and conscious (e.g., calculating 17×24). The key insight: humans rely on System 1 for ~95% of daily decisions, invoking System 2 only for complex problems.

Why AI Engineering Needs Layering

Current Problem

Today all AI applications use System 2 models (GPT-5, Claude, Gemini, Grok) for everything: chat replies, real-time risk control, routing decisions, anomaly detection, code writing. This creates three issues:

Too heavy — GPT-5 calls take ~1–3 seconds and cost high.

Too slow — unacceptable for real-time scenarios.

Overkill — simple decisions don't need deep reasoning.

Cost Gap

Cost differs by 200×. If 90% of decisions use a small model, total cost drops ~90%.

GPT-5: ~$30 per 1M tokens, ~2s latency

GPT-4o-mini: ~$0.15 per 1M tokens, ~300ms latency

Claude 3.5 Sonnet: ~$3 per 1M tokens, ~1s latency

Claude Haiku: ~$0.25 per 1M tokens, ~200ms latency

Latency-Sensitive Scenarios

Real-time game NPCs → must be <100ms

High-frequency trading → must be <10ms

Voice assistants → must be <500ms (otherwise users perceive lag)

GPT-5 cannot meet these; System 1 models can.

How the Layered Architecture Works

Key: System 1 is not just a filter; it makes decisions. ~80% of scenarios return results directly; only ~20% complex cases escalate to System 2.

System 1 = Front-desk receptionist (handles 80% of routine issues) System 2 = Expert consultant (only complex issues escalate)

The receptionist doesn't just screen who sees the expert; she solves problems herself.

Real-World Scenario Breakdown

Scenario 1: Customer Service

"How to reset password?" — 70% — System 1 (template reply)

"Why is my order delayed?" — 20% — System 1 + database query

"I want to complain about your service" — 10% — System 2 (deep understanding + sentiment + strategy)

Without layering: all queries hit GPT-4 → high cost, high latency.

With layering: 90% handled by System 1, only 10% escalate → cost down ~80%, latency from 3s to 200ms.

Scenario 2: Code Review

Format checks, obvious bugs — 80% — System 1 (rules + small model)

Logic errors, performance issues — 15% — System 2 (GPT-4 deep analysis)

Architecture advice, refactoring plans — 5% — System 2 + human expert

Scenario 3: Autonomous Driving (Extreme Example)

Lead car brakes → I brake — System 1 (reflex, <100ms)

Complex intersection navigation — System 2 (planning, ~1s)

If all decisions went through System 2, the car would crash.

Code Example: Implementing Layered Routing

The author provides a Python sketch illustrating the routing logic (confidence threshold 0.9):

def route_decision(user_input):
    # System 1: fast judgment
    fast_response = system1_model.predict(user_input, confidence=True)
    if fast_response.confidence > 0.9:
        return fast_response.result  # direct return
    else:
        # escalate to System 2
        return system2_model.generate(user_input)

# Usage
result = route_decision("How to reset password?")
# → System 1 handles, 200ms, $0.0001

result = route_decision("Analyze performance bottlenecks and suggest refactoring")
# → escalates to System 2, 2s, $0.01

The routing logic matters more than the model choice: how to decide "when to escalate", how to set confidence thresholds — these are engineering problems, not model problems.

JEV's Position in the Layered Architecture

Role: System 1 layer (fast decisions)

Competitors: GPT-4o-mini, Claude Haiku, local small models

Differentiation: Claims "natively built for machines", not for human chat

Risk: No benchmarks, no public API, appeared ~2 years ago, purely early-stage

Whether JEV succeeds is secondary. The direction it points to is critical:

AI systems are moving from "single super-model" to "layered collaboration".

Advice for Developers

What You Can Do Now

Learn layered architecture thinking — Not JEV, but the "layering" mindset. Read Kahneman's book (or at least a summary).

Experiment with layering in your projects — Use GPT-4o-mini for fast decisions, GPT-4 for complex reasoning; write routing logic (confidence thresholds).

Watch JEV but don't bet on it — Apply for early access, evaluate. Don't use in production until benchmarks exist.

Do You Need JEV? (Decision Checklist)

Are >80% of your AI calls simple decisions? → Need layering

Do users complain about latency? → Need layering

Is monthly AI cost >$1,000? → Need layering

Are you building real-time scenarios (gaming/voice/trading)? → Need layering

Does your agent system use GPT-5 for all decisions? → Need layering

5+ "Yes" answers → start layering this week.

Summary in Four Points

1. Layering Solves "Mismatch", Not "Not Smart Enough"

GPT-5/Claude excel at slow thinking: code writing, complaint handling, architecture advice. They are ill-suited for classification, routing, template replies, format checks, reflex braking. Mismatch consequences: 1–3s latency, 200× cost gap, real-time scenarios unusable. Layering gains are concrete: in the customer-service example, 90% via fast model → cost ~80% lower, latency from 3s to 200ms.

2. Each Layer Has a Distinct Role — Don't Reverse Them

System 1 — Decides autonomously: high confidence → return. Customer service 70% password reset, code review 80% format/obvious bugs, driving emergency braking. Does not act as pure filter; it solves problems itself.

System 2 — Handles low-confidence escalations: complaints, logic/performance, intersection planning. Does not serve as default entry; default entry is always System 1.

The 80/20, 90/10, Kahneman's 95/5 are different precisions of the same rule: the vast majority of decisions shouldn't enter the expert's room.

3. JEV Proves the Direction, Not the Product

JEV aims to be the "machine-native System 1": new architecture, parallel sampler, RLCD. It currently lacks benchmarks, public API, verifiable production data. It competes with GPT-4o-mini, Haiku, local small models for System 1, and has no System 2 counterpart. Betting on the product is premature; betting on the "single super-model → fast/slow collaboration" path is already actionable. Developers can implement layering today: mini/Haiku/1B models as fast layer, GPT-4o/Sonnet/70B as slow layer.

4. The Real Code to Write Is the Escalation Logic, Not Another Model Swap

Three things you must define yourself — model vendors won't:

What counts as a simple decision — formatting, classification, routing, templates, rule-covered checks.

When to escalate — the article uses confidence >0.9 for System 1, else System 2; threshold tuned by misclassification cost (finance/driving stricter than customer service).

What escalation failure looks like — fast decisions becoming slow (threshold too low), accurate decisions becoming wrong (threshold too high). Monitor these two failure modes before swapping models.

Self-test with the five questions above: simple-decision share, latency complaints, monthly cost, real-time needs, agent using GPT-5 for everything. If majority are "Yes", layering belongs in this week's sprint, not waiting for JEV's public release.

Bottom line: Fast models handle the vast majority of routine decisions; slow models only take complex ones. JEV is an unverified sample in this direction; what you can change today is the routing logic, not a bet on a new product.