How I Turned a Generic LLM into a Precise E‑Commerce Risk Detector

1. Introduction: When an algorithm engineer meets an unpredictable AI

A risk‑control algorithm engineer faced massive user‑behavior clustering results and needed an efficient way to identify risky clusters. Initial prompts to a large language model (LLM) yielded inconsistent answers, prompting the realization that effective communication with the model was essential.

2. Stage One – From 0 to 1: Giving the AI an "operations manual"

The engineer introduced three key actions:

Role‑Playing: Starting the prompt with "You are a senior e‑commerce risk‑control expert..." to activate relevant knowledge.

Defining Analysis Dimensions: Explicitly listing focus areas such as recipient information, address analysis, and product‑value analysis.

Structured I/O: Using CSV input for multiple orders and requiring JSON output, enabling direct backend parsing.

This produced a V1 prompt that automated the workflow but still behaved like a junior analyst with high false‑positive rates.

3. Stage Two – Injecting Business Knowledge for Specific Analysis

The engineer added exemption rules and background knowledge to avoid common misclassifications:

High discounts do not equal risk when they stem from new‑user subsidies.

Random‑looking user IDs are not fraudulent; focus on the actual recipient name.

Zero‑price items are gifts, nicknames are normal, and benefit products are benign.

These rules reduced false positives and elevated the AI to a "mid‑level analyst".

4. Stage Three – Deepening Analysis: Teaching the AI to Think Like a Detective

To capture coordinated fraud, the engineer introduced two new capabilities:

Low‑value, high‑volume risk signals: Flagging bulk purchases of cheap consumables as potential arbitrage.

Consistency view: Detecting identical or highly similar shopping carts across different accounts as evidence of scripted behavior.

The AI progressed to a "senior analyst" capable of identifying group‑level threats.

5. Stage Four – Final Evolution: Enabling the AI to Make Judge‑like Decisions

The most challenging step was distinguishing genuine user clusters from coordinated fraud groups. The engineer introduced a dual‑hypothesis decision framework:

Hypothesis A: Coordinated risk gang.

Hypothesis B: Benign customer segment.

The prompt instructed the AI to first search for "hard links" such as identical non‑public shipping addresses; finding them leads to a risk verdict, otherwise consider benign explanations. Few‑shot examples for both cases were provided.

With this framework, the AI became a "risk‑control expert" capable of nuanced, evidence‑based judgments.

6. Summary and Reflections: Prompt Engineering Principles

Key takeaways include starting with imitation of expert thought, abstracting rules into reusable frameworks, enriching rules with business context, using negative examples as teaching tools, and moving from explicit instructions to a thinking model that empowers the AI to reason and decide.

Images