How a QA Skill Cut Server‑Side Smoke Review from 2 Days to 3 Minutes

Why manual server‑side smoke pre‑review is slow

The time‑consuming part is not reading code but reconstructing dispersed context: mapping requirements to technical solutions, linking test cases to entry code, tracing state writes, verifying asynchronous tasks retain trigger‑time context, and confirming final outcomes. Legacy projects add historic issues that must be distinguished from the current change.

Separate new projects from legacy projects

Attempting a single generic Skill failed because the core questions differ.

Core question

New project: Is the entire requirement implemented correctly and completely?

Legacy project: Does the current change introduce a real issue?

Review starting point

New project: Full requirement and test cases.

Legacy project: Diff, PR, branch or commit.

Code scope

New project: Map each feature to entry code, core decision, final write‑back, notification and cleanup.

Legacy project: Trace from the change point to upstream callers and downstream results.

Focus

New project: Feature omissions, rule deviations, state and side‑effects.

Legacy project: Regression, compatibility, change omissions.

Problem attribution

New project: Determine if the issue belongs to the current requirement.

Legacy project: Distinguish current change from historic stock.

Complex iterations can combine both paths: first lock the current change with old-project-smoke-review, then, if needed, run new-project-smoke-review to verify the full functional loop.

Nine‑step review workflow

Read requirement, technical solution and test cases
→ Choose new‑project or old‑project path
→ Build complete feature mapping, or lock current diff
→ Follow real call chain to final state and side‑effects
→ Form candidate issue pool
→ Create evidence‑decision cards for each candidate
→ Exclude speculation, historical issues, and pure optimisations
→ Write only evidence‑supported issues into HTML report
→ QA and developers confirm final conclusions

Once the mapping and evidence thresholds are fixed, subsequent reviews reuse the same path, reducing a day‑long effort to a minute‑level process.

Evidence thresholds before reporting

Each candidate issue must satisfy seven questions:

What does the documentation or test case require?

How does the actual code behave?

Is there upstream safeguard?

Where does the call chain finally act?

What is the minimal supported impact?

Does it belong to the current requirement, the current change, or historic stock?

Is the collected evidence sufficient for a formal output?

Example: an asynchronous task that does not solidify the trigger context but reads the current state at execution may cause context drift. It enters the report because (1) the requirement and test case explicitly demand context preservation, (2) the code exhibits the loss‑and‑read pattern, and (3) the minimal impact is the context shift.

Content that fails the threshold includes pure speculation, optimisation‑only suggestions, unrelated historic issues, unproven assumptions, and partially‑supported candidates.

Separate semantic judgment from deterministic work

The runtime Agent performs material loading, code search, call‑chain tracing, and evidence judgement. The Skill organizes navigation, rules, and deterministic tools. Four layers cooperate: SKILL.md – navigation entry that lists which rule modules to load. references/ – stores feature mappings, evidence thresholds, problem‑attribution rules, exclusion conditions, and scenario‑specific rules. scripts/ – handles test‑case reading, XMind extraction, HTML report rendering, and other stable automation tasks. agents/ – keeps default display and invocation configurations for consistent Agent behaviour.

This division turns repetitive deterministic steps into reusable actions, leaving only business‑semantic decisions to the Agent.

Rule reuse for repeatable minutes‑level speed

When new miss‑review types appear, QA and developers review the root cause, augment scenario rules, evidence conditions, or exclusion criteria, version the rule module while preserving change reasons, and load the updated rules on demand. Subsequent similar cases automatically benefit from the updated rules.

Case study: 520 activity

The large‑scale 520 promotion, a new‑project review that normally required two days of manual effort, was completed end‑to‑end in 3 minutes 20 seconds . The generated structured HTML report contained more than ten evidence‑rich issues for QA and developers to confirm.

Why the speed gain is credible

The approach does not replace Code Review; it adds a layer that checks whether requirements and test scenarios truly flow into the code. By fixing reusable mappings, evidence thresholds, and modular rules, the process consistently achieves minute‑level execution without sacrificing credibility.

Engineering artifacts

Directory layout (illustrated in the original article):

skill/ – top‑level directory containing the Skill definition.

SKILL.md , references/, scripts/, agents/ – as described above.

Images in the source article illustrate the directory tree and workflow diagrams; they are retained for reference.