Jev: Extracting Semantic Micro-Judgments from Agent Loops into Observable Decision Layers

Core Problem: LLMs Overused for Simple Semantic Decisions

Agent runtimes repeatedly face narrow semantic questions — which queue for a ticket, whether a command is risky, if retrieved context answers the query, whether tests pass — that require language understanding but not language generation. Historically these were handled by prompting a general LLM to produce text, then parsing a label, boolean, or score from the response. This incurs full generation latency, retry loops for format errors, and opaque reasoning that cannot be easily logged or evaluated.

Jev's Position in the Agent Loop

TypeSafe released Jev on 2026-09-15 as a System One Model . It sits between the general model (understanding, planning, generation) and policy code (threshold checks, branch selection, fallbacks). Jev receives a text or JSON state and a set of pre-declared questions with defined answer spaces; it returns typed results with probabilities. It does not run the agent loop, call tools, or produce final business content.

Current State
  ↓
General Model: understand, plan, generate
  ↓
Jev: choose, score, judge
  ↓
Policy Code: compare thresholds, combine conditions, pick branch
  ↓
Tools or Human: execute next step
  ↓
External results written back to state

Three Question Primitives

Developers declare the answer space upfront; Jev constrains its output to that space.

Choice — pick one from a predefined set (e.g., ticket routing, model selection, tool choice). Returns choice, per-option probabilities, and confidence.

Score — rate on a developer-defined ordinal scale (e.g., relevance, quality, risk as low/medium/high). Returns score, tier labels, and probabilities.

Noul — probability that a proposition holds (e.g., "needs human review", "command is destructive"). Output is a 0–1 probability; 0.5 means equal odds, not medium intensity — use Score for degree.

Multiple independent questions can be evaluated in parallel on the same state. The Python SDK ( typesafe_sdk, Python ≥3.10) calls POST https://api.typesafe.ai/v1/systemone with model jev-latest.

from typesafe_sdk import Choice, Noul, Score, TypeSafeClient

client = TypeSafeClient()
ticket = "我的 Stripe 已经连续 3 天连不上，订单都快丢了，请尽快处理。"

response = client.system_one(
    state=ticket,
    questions={
        "department": Choice(
            instructions="Which team should handle this ticket?",
            criteria={
                "billing": "Payment or subscription issues",
                "technical": "Bugs or integration problems",
                "sales": "Pricing or account questions",
                "other": "None of the above",
            },
        ),
        "frustration": Score(
            instructions="How frustrated does the customer appear?",
            criteria=[
                "Calm, just stating facts",
                "Frustrated but civil",
                "Very angry, strong language",
            ],
        ),
        "urgent": Noul(
            instructions="The message conveys urgency or time-sensitivity",
        ),
    },
)

Policy code then combines signals: department.confidence in a gray zone triggers human review; urgent.noul >= 0.8 escalates priority.

Key Distinction from Structured Output

Structured output asks a general model to format natural language into a schema. Jev bakes the answer type, candidate set, and probability fields into the question definition itself. The program receives a ready-to-branch result without guessing which option the model selected from an explanatory paragraph.

Case Study: Context Compaction without Summarization ( fast-jev-compaction )

Coding agents accumulate large histories of tool_use / tool_result (file listings, test logs, command output). Summarizing with a general LLM compresses text but may rewrite paths, error codes, and parameters into vague generalizations. fast-jev-compaction keeps user/assistant messages intact and only judges tool-call pairs via two Noul questions:

keepCall: knowing this tool call happened, does it still help future tasks?
keepResult: does the full tool result text still need to be retained?

A tiny decision tree maps the two probabilities to three actions:

keepResult >= threshold
  ├─ Yes: retain both call and result
  └─ No
       ├─ keepCall >= threshold: retain call, truncate result
       └─ No: delete both call and result

Jev supplies only the retention probabilities; code handles message-list reconstruction, budget accounting, and fallback (revert to summarization or keep raw history on Jev failure, format errors, gray-zone probabilities, or zero compression gain). Run logs must capture pre/post sizes, compaction stage, request count, and final action to diagnose whether issues stem from judgment, budget, or policy.

Where Jev Fits vs. Rules, Classifiers, and General LLMs

Deterministic rules (date math, field presence, permissions, string match) — write code; an if is faster and more testable.

Traditional classifiers — suit stable labels, ample training data, fixed input distributions. Jev is an on-demand semantic judgment interface: questions, candidates, and criteria can change with business state without retraining a dedicated classifier per micro-branch. Trade-off: Jev still requires calibration, evaluation, and fallbacks; type-safe outputs don't replace that engineering work.

General LLMs — remain best for open-ended tasks: writing replies, generating code, explaining reasoning, planning, or exploring when the answer space is unknown. Jev narrows the answer space first; runtime gets a directly readable result.

Three Natural Insertion Points in the Agent Loop

Model Routing — Jev selects fast vs. powerful models based on request state, moving routing logic out of implicit prompt heuristics into recorded inputs, options, and probabilities.

Tool Risk Judgment — Before execution, feed command, parameters, and repo state to Jev to classify as read-only, rollback-safe, or destructive (Git history modification, production resource touch, file deletion). Low-risk/high-confidence calls proceed; risky/uncertain ones route to confirmation, sandbox, or human. Permissions, sandboxes, and tests still enforce code-verifiable rules.

Result Verification — Agent claims "task complete" ≠ task actually complete. Jev judges whether tests passed, whether the same action repeats, whether output matches a policy, or whether review is needed. Hard tests stay as hard tests; semantic judgment fills gaps rules cannot express.

Commonality: Jev never writes final business content; it only supplies the signal the next branch needs.

Four Open-Source Projects Illustrating the Pattern

Each project delegates a narrow semantic decision to Jev while keeping execution and state management in code.

Browser Automation: jev-ultrafast — Jev picks the next DOM element (button, input, link) from currently available controls; a helper model generates text when needed; browser automation layer executes clicks/fills.

Feed Filtering: Your Signal — Plugin sends visible post text to Jev to judge relevance to user-defined topics, utility, or promo preferences; UI highlights, dims, folds, or hides. Jev does not verify facts in images or external links.

Search Filtering: Jev Search — Search service fetches links; Jev judges source, date range, and title/snippet relevance to the query; UI displays filtered results. Search handles connectivity; Jev handles selection. High relevance ≠ factual verification.

Context Compaction: fast-jev-compaction — As detailed above, Jev scores retention value of historical tool calls/results; code rebuilds the message list and measures compression gain.

Performance Claims and Caveats

TypeSafe's public figures: end-to-end latency ~70–500 ms, input cost $0.042/M tokens, output free; ~200× speed and ~400× cost advantage vs. a general-model workflow. These depend on comparison workflow, input size, and network; treat as positioning indicators, not capacity guarantees.

Speed comes from: no long-text generation, candidates fixed before call, parallel evaluation of independent questions, no generate-parse-retry loop. Meaningful comparison is end-to-end chain: which generation calls disappear, and what calibration/fallback/observability costs are added.

Probabilities Need Calibration Before Policy Use

A 0.52 vs. 0.48 split on ticket routing differs fundamentally from 0.98 confidence; they should not follow the same automation path. Typical policy layers:

High confidence + low consequence → auto-proceed

Mid confidence → stronger model or confirmation

Low confidence → human or info-gathering path

Thresholds must derive from near-production samples, not a universal 0.8. Two failure modes exist: (1) output shape violations — Jev guarantees schema conformance (no extra labels, no broken text); (2) semantic errors within valid options — Jev can confidently pick the wrong label. Type safety solves output shape, not judgment quality.

Recommended adoption: start with a low-risk, bounded-candidate, easily-audited branch (retrieval scope, ticket routing, search relevance, tool-result retention). Freeze question definitions, candidates, input state, and expected answers into an eval set. Shadow-run: log Jev probabilities without changing behavior. Measure actual error rates per confidence bucket to set automation thresholds. Record model version, question definitions, and thresholds in run logs for replay and comparison.

Harness Responsibilities for the Judgment Layer

Building on prior Harness discussions (context assembly, tool mounting, loop recovery, result verification), Jev adds two integration requirements:

Replayability — Capture triggering state, question definitions, candidates, model version, probability distribution, and chosen branch. Without this, a routing error leaves no trace of what Jev saw.

Explicit Policy — p > 0.8 is a condition, not a policy. Reversibility, privacy impact, confirmation requirements, timeout and gray-zone handling belong in config or code, not hidden in prompts or chat history.

Failure Loops — On tool failure, external state change, human rejection, or new evidence, the system updates state and re-judges: retry, swap tool, fall back, or terminate. Jev supplies the current semantic signal; runtime persists facts and orchestrates recovery.

Context determines what facts the judgment sees; Jev maps facts to constrained results; Harness closes the loop with replayable, verifiable execution.

Where to Start Integration

Pick a low-risk, limited-candidate, easily-verified judgment: retrieval scope, ticket routing, search relevance, tool-result retention. Even if it temporarily falls back to the old model or human path, the system retains control. Avoid questions needing long explanations, multi-step implicit reasoning, exact calculation, counting, date comparison, or unknown-value extraction. When the answer space shifts, let a general model or code propose candidates first, then use Jev to choose among them. Keep deterministic logic in code.

Architecturally, Jev is a new slot in the agent runtime: general model handles understanding/planning/generation; Jev handles bounded semantic judgments; policy code combines conditions and picks branches; Harness records state, invokes tools, and manages fallbacks/verification. This separation doesn't automatically yield reliability — its value is that formerly buried micro-judgments become independently observable, evaluable, and swappable (with rules, classifiers, or another model). For production systems, that boundary is worth studying more than "add a stronger chat model."