Skip to content
Jev in Practice: Lightweight Decision Layer for AI Workflows Delivers 10x Speedup

Jev in Practice: Lightweight Decision Layer for AI Workflows Delivers 10x Speedup

This article details practical integration of Jev, a structured decision model, into three AI testing workflows—test case labeling, UI automation, and knowledge base retrieval—showing 10–93× latency reduction and 31–93× cost savings versus general LLMs, with code examples and architectural insights.

DeWu Technology
DeWu Technology
DeWu Technology

Introduction

The author explores Jev, a tool that implements a lightweight decision layer in AI workflows. Unlike general LLMs that generate free-form text, Jev returns structured judgments from predefined answer sets, enabling lower latency, lower cost, and machine-readable output. The evaluation covers three testing scenarios on the AI Testing platform.

Jev vs. General LLMs: Output Form Difference

A simple question—"Is a bat a bird?"—illustrates the contrast. A general LLM produces a free-form explanation. Jev returns a fixed-structure judgment (e.g., "属于/不属于" with a confidence score). The author notes that Jev's "confidence 100%" is an output score, not a calibrated accuracy; adoption thresholds should combine task risk, human-labeled samples, and independent verification.

Scenario 1: Test Case Labeling — Batch High-Speed Execution

Setup

300 test cases labeled for priority (P1/P2/P3) and "financial risk" tags. Python script concurrently calls Jev API. The skill package ep-aitesting-jev from the AI Testing marketplace assists integration.

Results

DeepSeek (general LLM): 4 min 5 sec, ~0.31 CNY, 42 requests (batched multiple cases per request).

Jev: 5 sec, ~0.01 CNY, 300 requests (one per case).

Comparison: Jev ~49× faster, ~31× cheaper. Request patterns differ; Jev also supports multi-judgment per request via state and questions.

Results include a "suspected financial risk – pending review" tag for judgments below the adoption threshold; probabilities and confidence scores drive routing.

Scenario 2: UI Automation — Accelerating UI Test Execution

Approach

Interactive UI automation with Playwright: browser tool collects visible elements, Jev selects the next action from candidates based on the test step. Referenced the open-source npm package easy-jev-e2e.

Test Case

Login flow: enter username/password, verify result. Side-by-side recordings (real-time speed) show Jev-driven execution vs. general LLM-driven execution.

Results

General LLM: 90 sec, ~0.93 CNY.

Jev: 7 sec, ~0.01 CNY.

Comparison: Jev ~13× faster, ~93× cheaper. Cost figures rounded; ratios for reference.

Scenario 3: Knowledge Base Retrieval — Efficient Vector Re-ranking

Problem

Vector top-10 recall returns semantically similar but business-irrelevant candidates (e.g., domestic payment cases when targeting cross-border "XXX business"). Feeding noise to downstream Agent pollutes context.

Pipeline

Vector Top-10 → Jev three-dimensional judgment → Script filtering.

Each candidate gets one Jev request with three questions sharing the same state: relevance/noul: "Does the verification object directly belong to XXX business?" Returns probability (0–1). Criteria: core links, channels, transaction scenarios (cross-border, proxy, refund, reconciliation, compliance, switching). Keyword overlap alone insufficient. topic/choice: Pick from four mutually exclusive themes: target_business, other_crossborder, domestic_payment, non_payment. Returns probability distribution and confidence. quality/score: Ordered scale on three elements (scenario, operation object, expected result): "missing/vague", "basically complete but lacking detail", "complete and executable".

Python Implementation

import json
import os
from urllib.request import Request, urlopen

JEV_URL = "..."
QUESTIONS = {
    "relevance": {
        "type": "noul",
        "instructions": (
            "该用例的验证对象是否直接属于XXX业务?"
            "核心链路、通道或交易场景涉及XXX业务的收付汇、"
            "跨境通道(含代理行、业务通道、退款、对账、合规拦截、通道切换)才算相关;"
            "仅出现XXX业务、银行、支付等词,但验证其他业务,不算相关。"
        ),
    },
    "topic": {
        "type": "choice",
        "instructions": "该用例最准确归属的业务主题是哪个?",
        "criteria": {
            "target_business": "XXX业务(通道、收付汇、退款、对账、合规、切换)",
            "other_crossborder": "其他跨境支付(非目标业务的其他通道)",
            "domestic_payment": "境内支付收单",
            "non_payment": "非支付域(汇率展示、开户、物流、登录、营销等)"
        },
    },
    "quality": {
        "type": "score",
        "instructions": "作为基线测试用例,场景、操作对象、预期结果三要素是否完备清晰?",
        "criteria": ["要素缺失或表述含糊", "基本完整但细节不足", "三要素完备可直接执行"],
    },
}

def judge_case(case):
    payload = {"state": json.dumps(case, ensure_ascii=False), "questions": QUESTIONS}
    request = Request(
        JEV_URL,
        data=json.dumps(payload, ensure_ascii=False).encode("utf-8"),
        headers={
            "Content-Type": "application/json",
            "Authorization": "Bearer " + os.environ["AUTH_TOKEN"],
        },
        method="POST",
    )
    with urlopen(request, timeout=30) as response:
        body = json.load(response)
    if body.get("code") != 200:
        raise RuntimeError(f"Jev 调用失败:{body.get('code')}{body.get('message')}")
    return body["data"]

def refine(candidates):
    accepted, review, rejected = [], [], []
    total_cost = 0.0
    for case in candidates:
        data = judge_case(case)
        answers = data["answers"]
        relevance = answers["relevance"]["noul"]
        topic = answers["topic"]["choice"]
        quality = answers["quality"]["score"]
        row = {"case": case, "answers": answers, "trace_id": data.get("u_trace_id")}
        total_cost += data.get("usage", {}).get("cost", 0)
        if relevance < 0.8 or topic != "target_business":
            rejected.append(row)
        elif quality < 2:
            review.append(row)
        else:
            accepted.append(row)
    accepted.sort(key=lambda x: x["answers"]["relevance"]["noul"], reverse=True)
    return {"accepted": accepted, "review": review, "rejected": rejected,
            "jev_cost": total_cost}

if __name__ == "__main__":
    with open("vector_top10.json", encoding="utf-8") as f:
        candidates = json.load(f)
    print(json.dumps(refine(candidates), ensure_ascii=False, indent=2))

Outcome

Knowledge base: 16 baseline cases. Vector top-10 → Jev judgment → script split. Result: 6 accepted, 1 held for completion, 3 rejected . Script runtime ~4 sec, Jev total cost $0.000336 . Each judgment includes u_trace_id for traceability; HTTP 200 with non-200 envelope code treated as failure.

Conclusion & Integration Guidelines

All three scenarios share a pattern: define answer space first, then drive next steps via judgment. Integration checklist:

Three core judgment types: Noul (binary, 0–1 probability), Choice (pick from ≤255 options with probabilities), Score (custom ordered scale with meanings).

Key strengths: end-to-end 70–500 ms latency; input 0.3 CNY/M tokens, output free; strongly structured output (no JSON repair); built-in confidence for threshold-based fallback; suits high-frequency micro-decisions (ticket routing, content labeling, LLM output guardrails, Agent action routing, risk scoring).

Limitations: no open-ended generation, long reasoning, complex planning, memory; requires crisp option/scale definitions—ambiguity hurts accuracy; only for fixed answer spaces.

Agent role split: General LLM handles analysis, planning, tool calls; Jev handles explicit routing branches, result verification, risk scoring, input filtering. Below-threshold results escalate to main model or human.

Integration tips: Pass content as state; define questions and option descriptions unambiguously; set adopt/review/fallback thresholds; start with one high-frequency, well-bounded decision point, validate accuracy/latency/cost on samples, then expand.

Author recommends beginning with a single high-frequency, clearly bounded workflow step, defining answer space and criteria, measuring accuracy and failure paths alongside latency and cost, and expanding only after confirming benefits and fallback mechanisms.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

test automationcost optimizationlatency reductionLLM optimizationPython integrationAI workflowsJevstructured decision model
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.