Laya: Open-Source Decision Engine Beats Jev 7.8x Faster, 3.9% More Accurate

ChatGPT co-inventor Diogo Almeida's TypeSafe AI recently released Jev , a non-autoregressive decision model that outputs probabilities directly on a structured schema at $0.042/1M tokens. Shortly after, an open-source implementation named Laya appeared on GitHub with 4.6k stars under Apache 2.0.

Core Architecture: Single Forward Pass, No Text Generation

Laya takes an arbitrary state (email, ticket, JSON document) plus a set of typed questions and answers all of them in a single forward pass . Questions come in three primitives: choice – pick one option from a dictionary score – rate on an ordinal scale noul – answer a yes/no question

The output space contains only probabilities and numbers — no tokens are generated . Because no text is produced, hallucination is impossible; because no JSON is written, malformed JSON cannot occur. On a T4 GPU, latency is 33 ms per question, dropping to 7.2 ms per question in batch mode.

Model Backbone and Training

The backbone is a ModernBERT-large bidirectional encoder (421M parameters) . Each option occupies a [MASK] token position; after encoding, the model gathers scores at those positions — this is the entire meaning of "non-autoregressive": no token-by-token generation, all answers in one forward pass. Training uses RLCD (reinforcement learning with a strictly proper scoring rule as reward), so confidence scores are statistically trustworthy and can be used directly for gating: confidence ≥ 0.85 → auto-execute, otherwise escalate to human.

Router: Language-Aware Checkpoint Selection

Laya ships three checkpoints (English, 100+ languages, typed-decisions). A lightweight Router runs a pure-Python language detection (< 0.5 ms) before the forward pass to pick the best checkpoint. This pre-forward routing is critical: the English checkpoint on Khmer yields 0.000 accuracy with 0.952 confidence — confidently wrong, and the model's own confidence would not raise an alarm.

Benchmark Comparison vs. Jev

The README provides a detailed comparison table. Laya numbers are measured on T4; Jev numbers come from third-party publications (no API access, different methodology, noted by the author). Two standout details:

Laya's 0.766 accuracy exceeds the 0.735 teacher self-consistency ceiling — the fine-tuned student is more stable than its teacher.

On DAIR Emotion, Jev assigns zero probability to the correct label on 16% of samples — a hard failure for any confidence-based routing system, not just an error.

Multilingual Performance

The Router exists because multilingual coverage matters. The English checkpoint averages only 0.227 macro-accuracy across 51 languages, with only 23/51 languages usable. The multilingual checkpoint raises usable languages to 45/51 , sacrificing only a tiny amount on English tasks. The Router picks the best of both worlds.

Honest Limitations

The "Honest limits" section is notably transparent:

Base checkpoint zero-shot is near random — 0.362 and 0.342, below the 0.461 majority-class baseline. The 0.766 figure comes entirely from versions fine-tuned on the benchmark's training split. The README states plainly: treat Laya as a "fast base for specialization", not a zero-shot decision engine.

High-cardinality scenarios (50+ options) favor Jev (Banking77: 0.870 vs 0.425), limited by token budget.

Soft distribution matching also favors Jev (0.580 vs 0.471).

Laya — Multilingual, non-autoregressive System 1 decision engine
https://github.com/NandhaKishorM/laya
https://huggingface.co/convaiinnovations/laya