Open-Source Decision Model laya vs Jev: Speed Wins, Zero-Shot Fails

Background: Jev and the Open-Source Response

Jev is a closed-source, API-only model specialized for structured judgments (not text generation), claimed to be 20-200x faster than traditional LLMs. On September 18, 2025, NandhaKishor M released laya on GitHub ( https://github.com/NandhaKishorM/laya) as a fully open Apache 2.0 alternative. The repository gained over 13,000 stars in four days. The author states he published an arXiv paper, model weights, and dataset for sales-conversion prediction in March 2025, while TypeSafe AI later launched Jev with the same core idea but without releasing weights or data.

Architecture and Model Variants

laya totals 421M parameters: a ModernBERT-large encoder plus a ~25M decision head, plus a small head for "answer or escalate" routing. Because it does not generate tokens autoregressively, inference is millisecond-scale. The repo ships three checkpoints:

English – base encoder

Multilingual – covers 100+ languages, smaller and ~2x faster

typed-decisions – fine-tuned on the typed-decisions benchmark

A lightweight Router (sub-millisecond) detects input language and dispatches to the appropriate checkpoint. Built-in templates cover model routing (cheap vs flagship), prompt-injection/jailbreak detection, content moderation, and ticket triage. Installation is pip install laya (Python ≥3.10).

Decision Types

choice – pick one option, returns per-option probabilities (triage, intent classification)

score – rate on an ordered scale (e.g., urgency)

noul – true/false probability (phishing, jailbreak, churn risk)

Outputs are pure probabilities/numbers; no text parsing needed. However, the author notes that "no text generation ≠ no misjudgments" and encountered a concrete error during testing.

Benchmark Comparison: laya vs Jev

Numbers come from laya's official T4 self-tests and third-party Jev results (different tasks/samples, so only indicative).

Latency : laya ~6-7x faster than Jev. Independent benchmark by nibzard confirms Jev is fastest among tested APIs but measured p50 at 264-276 ms, not the advertised magnitude.

Accuracy (typed-decisions) : laya's fine-tuned checkpoint scores 0.766, beating Jev's 0.727 and the teacher model's 0.735 self-consistency ceiling. However, the two base checkpoints score only ~0.36 zero-shot — below random (0.318) and majority-class (0.461). The 0.766 figure comes entirely from the fine-tuned checkpoint trained on the benchmark's own training set.

Many-class weakness : On Banking77 (77 intents), laya drops to 0.425 vs Jev's 0.870 (on 72 labels). Cause: fixed token budget shared across options → 3-4 tokens per label, insufficient discrimination. Workarounds: increase head_max_len to 512, or pre-filter labels via embeddings to top-k.

Calibration (ECE) : laya 0.081 vs Jev 0.246. For confidence-based routing (auto-execute high confidence, escalate low), this gap directly determines whether thresholds are trustworthy.

Deployment model : Jev = closed API, per-token billing; laya = Apache 2.0 weights, self-hosted, data stays on-premise.

Hands-On Test on Apple Silicon MacBook (CPU Only)

Environment: Python 3.14, laya 0.3.5.

English email (repeated-billing complaint) : Four questions answered correctly in one forward pass — billing department (0.86), refund request (0.84), cancellation threat (0.82), urgency 1.44/2. Warm inference 63-68 ms total.

Chinese tickets : Router detected 76% Chinese characters, switched to multilingual checkpoint. Triage and refund correct, but "cancellation threat" returned 0.06 probability with 0.94 confidence (direction flipped; English checkpoint gave 0.82). Matches README admission: multilingual checkpoint ships without temperature fitting; confidence cannot be trusted until calibrated.

German single-sentence test : Router correctly identified Latin-script non-English, dispatched to multilingual model without running inference.

Cold-start penalty : Default Router keeps one checkpoint in memory; language switching triggers rebuild. Official CPU median 7.4 s per rebuild. First run (including weight download) took ~1 min each; subsequent cold builds 18-20 s (terminal log shows forward pass 19503 ms). Solution: Router(preload=True) — official figures: 32.8 ms (GPU) or 193-464 ms (CPU) per request after preload.

Runtime warning : English checkpoint temperature parameters out of range, auto-clamped; affected confidence bands marked uncalibrated.

Critical Limitation: Base Checkpoints Are Near-Random Without Fine-Tuning

The README's "Honest limits" section states zero-shot accuracy on typed-decisions is ~0.36 for both base checkpoints — worse than majority-class baseline (0.461). The 0.766 headline number comes exclusively from the fine-tuned third checkpoint. Official positioning: laya is a fast foundation for specialization , not a zero-shot decision engine.

The Router exists for the same reason: English checkpoint on Khmer scores 0.000 accuracy with 95.2% confidence — high confidence on wrong answers cannot be filtered post-hoc, so language routing must happen before inference.

Fine-Tuning Pipeline

Repo provides a Kaggle notebook (free 2×T4) covering the full loop: dataset construction → RLCD training (strict proper scoring rule rewards) → temperature calibration → evaluation → push to Hugging Face. 4 epochs, ~30k questions, reported 4-5 hours end-to-end. The value proposition: invest a few hours fine-tuning on your labeled data → self-hosted millisecond judgment model.

When to Choose Which

Choose laya if data cannot leave your environment, or judgment volume makes API costs significant, and you have labeled tickets/moderation/risk data for fine-tuning.

Choose Jev (or similar hosted API) if you need zero-shot out-of-the-box, handle dozens of classes, or want to avoid ops overhead.

Quick experiment : pip install laya or try the Hugging Face Space demo ( https://huggingface.co/spaces/convaiinnovations/laya-demo).

Summary

Jev validated demand for dedicated judgment models; laya delivers an open-source counterpart. Self-hosted deployment is straightforward. Official claims: 33 ms per question, 0.766 on typed-decisions after fine-tuning, full RLCD pipeline runnable on free Kaggle GPUs. Observed limitations confirm the README's caveats: base checkpoints near-random zero-shot, multilingual confidence requires user calibration, many-class classification degrades sharply. Direct drop-in replacement for Jev will likely disappoint; the viable path is using laya as a fast foundation and fine-tuning on proprietary labeled data.