Why LLMs Shouldn't Think Everything: Jev, Codex, and the Heterogeneous Agent Revolution

The Elephant in the Room: Agents Using a Sledgehammer to Crack Nuts

When discussing AI agents, the first instinct is to swap in a stronger foundation model (Frontier LLM) like GPT-6 or Codex. Yet engineers deploying coding agents in high-intensity industrial scenarios face a hidden pain point: large models are too expensive, too slow, and often over-exert on trivial details .

Consider a typical task: asking a coding agent to diagnose an iOS app's Bluetooth background disconnection and auto-reconnect failure. The traditional agent's actual execution chain often looks like this:

Read 50 tool definitions , autoregressively generating tokens to decide which tool to call first.

Full-text keyword search , returning 100 candidate file paths stuffed into context.

Read and analyze files one by one , discovering mismatches and re-searching, repeatedly burning expensive inference.

Run tests , feeding the model 2000 lines of redundant terminal logs for token-by-token parsing.

In this entire chain, the top-tier reasoning model with hundreds of billions of parameters and seconds-per-call latency wastes over 70% of its compute and tokens on ultra-low-entropy trivial decisions : tool selection, file picking, log stack filtering. This is akin to having a chief architect earning millions per year spend six hours a day manually comparing thousands of log lines in a console. Beyond the massive cost and latency waste, the fatal flaw is that large volumes of low-quality candidates severely pollute the attention window, causing degradation in subsequent core logic reasoning .

What Is Jev? Fundamental Difference from Traditional Transformers

In September 2026, TypeSafe AI, founded by former OpenAI researcher Diogo Almeida, released the Jev decision model, sparking immediate industry attention. It is not merely a "distilled small model" but a precise decoupling of compute morphology: separating 'semantic decisions' from expensive 'autoregressive text generation' .

1. Traditional Transformer (LLM): Expensive Autoregressive Unidirectional Long Chain

Generative models like GPT and Codex are autoregressive generation models : after prompt input, the model serially generates token sequences, finally parsed by the program as string JSON. This mechanism has inherent defects for "decision" tasks:

High sequential latency : each token depends serially, preventing deep parallelism.

Format collapse risk : even with JSON Schema constraints, occasional syntax parsing errors occur.

Confidence distortion : generative models are naturally overconfident, making it hard to quantify true certainty.

2. Jev: One-Shot Forward Decision Primitive for Structured Judgment

Jev abandons the obsession with long-text generation, focusing on structured semantic judgment . Its input is the current system state (State) and strongly typed questions (Typed Questions); a single forward pass (One-shot Forward Pass) directly outputs discrete choices and probability distributions. Three essential differences:

No autoregressive decoding : single forward pass completes high-dimensional semantic projection into discrete decision space, latency dropping from seconds to single-digit milliseconds.

Native type safety : output is natively enums or floating-point probabilities, completely avoiding JSON format parsing failures.

Statistical confidence calibration : Jev's output confidence has rigorous statistical meaning — when it gives 0.70 confidence, the true accuracy in statistical samples closely matches 70%, providing the system with a quantifiable fault-tolerance basis.

Traditional LLM : Understand → Reason → Generate text token-by-token → Parse result → Decide<br> Jev Decision Model : Understand ─────────────────────────── → Probability Decision Distribution

How Codex Integrates Jev: Open-Source Project jev-in-codex Analysis

In engineering practice, Jev does not replace Codex for writing code. Deep logic reasoning, state-machine deduction, cross-module architecture design still require strong reasoning models like Codex. Jev's landing point is embedding into the Codex Harness (runtime control layer) as a semantic decision middleware .

The open-source community has packaged this mechanism into a ready-to-use solution — GitHub open-source project teempai/jev-in-codex . Developers only need to provide an install command in a Codex session for one-click deployment.

Its core is a local MCP service written in Node.js, combined with local fast indexing tools (e.g., ripgrep) and the TypeSafe API, exposing three core decision primitives to Codex:

jev_select_capability : given a goal and available tool catalog, filters the most matching candidate subset in milliseconds — avoids stuffing all 50 tool schemas into context.

jev_search : local ripgrep coarse retrieval + Jev semantic reranking, returns precise code slices with line numbers — avoids Codex recursively grepping and reading irrelevant source code.

jev_triage : deduplicates, groups, and extracts key failure evidence from ultra-long test logs or output artifacts — avoids 2000 lines of irrelevant warnings blowing up context.

Collaboration Data Flow & Responsibility Separation

Local fast data fetching : local MCP uses C-written ripgrep to coarse-recall hundreds of candidates from the code repository in milliseconds.

Jev semantic denoising : local MCP feeds candidates to Jev in one batch. Jev does no long-text generation; single forward pass outputs scores and confidence, compressing to the 3 most precise code slices.

Codex final synthesis : Codex receives clean evidence, focuses on deep logic deduction and patch writing.

AGENTS.jev.md behavioral constraints : injected as persistent instruction set, forcing Codex to prioritize Jev pre-filtering when facing massive candidates, breaking the habit of brute-forcing everything.

Thinking, Fast and Slow: System 1 and System 2 Neural Architecture Concretized

Daniel Kahneman's Thinking, Fast and Slow proposes that the human brain is driven by intuitive, fast System 1 and deliberate, slow System 2 . Humans never engage System 2 to calculate which foot to step out with; only when System 1 detects a pit ahead does System 2 take over reasoning.

Our past mistake was wrapping the entire agent into a monolith purely composed of System 2. The mapping:

Cognitive traits : System 1 = intuitive, fast, automated, low energy; System 2 = deliberate, logical reasoning, high focus, high energy.

Corresponding components : System 1 = Jev (lightweight semantic decision model); System 2 = Codex / Frontier LLM (reasoning large model).

Compute morphology : System 1 = single forward parallel, low latency, probability distribution output; System 2 = autoregressive long-chain decoding, high latency, chain-of-thought (CoT) output.

Agent division of labor : System 1 = tool routing, candidate filtering, log denoising, safety gating; System 2 = business logic understanding, state-machine deduction, code writing, deep debugging.

The essence of this compute layering is not simply token saving, but rationalization of cognitive architecture : let Jev quickly judge what is worth thinking about, let Codex truly think deeply .

From Jev to Glean: Application AI's Underlying Logic Converges

This thinking is not isolated. Reviewing the $4B+ valued enterprise cognitive search and agent unicorn Glean reveals the exact same engineering DNA flowing underneath.

Facing tens of thousands of heterogeneous enterprise documents and conversations, Glean never uses large models for brute-force RAG. Instead, it built a massive lightweight model matrix (intent understanding, permission filtering, knowledge graph routing, multi-path retrieval reranking), with Frontier LLM only handling high-level fact synthesis and answer generation in the final step.

Comparison of Glean (enterprise AI search/agent) vs. Jev × Codex (general coding agent):

Core business scenario : Glean = enterprise-wide heterogeneous knowledge retrieval & intelligent collaboration; Jev×Codex = complex software engineering logic writing & autonomous debugging.

Lightweight decision layer tasks : Glean = intent recognition, permission judgment, graph routing, multi-source reranking; Jev×Codex = tool selection, code slice filtering, log triage, safety gating.

Core data faced : Glean = enterprise knowledge graph, multi-source docs, org structure; Jev×Codex = AST syntax trees, code snippets, terminal logs, agent state.

Frontier LLM tasks : Glean = final fact synthesis & long-form answers; Jev×Codex = final business architecture understanding, deduction & patch writing.

Unified underlying philosophy : front-load low-entropy semantic decisions, extremely compress noise fed into expensive main model .

The mature pattern Glean validated in enterprise knowledge, Jev is generalizing into the foundational paradigm for universal Agent Harness.

Endgame Evolution: Agents Moving Toward Heterogeneous Computing

Connecting Codex integration, Jev single-forward decisions, dual-system metaphor, and Glean's practice reveals a clear future agent evolution direction:

Past agents resembled early microcontroller systems: one prompt + giant autoregressive LLM + while-loop brute-forcing everything , severe compute waste and fragility.

Future Agent Harness is evolving into a modern heterogeneous computing operating system with extremely strict division of labor:

Frontier LLM (Codex/Claude) as the system's CPU : expensive compute, extreme generality & complex branch prediction, dedicated to chewing the hardest bones (complex reasoning & logic design).

Jev and similar decision models as dedicated co-processors (NPU/DPU) : specialized hardware-level acceleration for specific semantic scoring & candidate classification, high-frequency trigger, low latency, extremely cheap.

Agent Harness as the OS kernel : responsible for cross-component scheduling, memory management, and heterogeneous compute diversion.

Conclusion

From letting large models do everything, to learning exquisite division of labor; from brute-force parameter stacking, to system-level architecture compute layering — this is the inevitable path for any foundational technology maturing into industrialization.

Jev's greatest revelation to the industry: we have finally begun to liberate 'semantic decisions' from expensive 'natural language generation', turning them into infrastructure-grade compute primitives. Agents possessing this heterogeneous compute layering architecture are destined to establish overwhelming engineering moats in execution efficiency, stability, and cost.