How I Turned a General LLM into a Precise E‑commerce Risk Detector

As a transaction‑risk algorithm engineer, I constantly deal with massive data and complex models, seeking hidden risk signals. I introduced a large language model (LLM) into my workflow and, after many trials, developed a step‑by‑step “Prompt engineering methodology” that turned a generic model into an AI expert capable of accurately identifying complex e‑commerce risk patterns.

Introduction: When an Algorithm Engineer Meets an Unpredictable AI

My recent project clustered user behavior embeddings to discover unknown risk patterns, producing hundreds of “suspected risk clusters.” Manual review was slow, inconsistent, and inefficient, prompting me to try a hot LLM to automate the assessment.

Stage 1 – From 0 to 1: Giving AI an Operations Manual

I realized the model needed clear guidance, so I created a structured prompt manual.

1. Role‑Playing : I began the prompt with a magic sentence such as “You are a senior e‑commerce risk‑control expert…”, which activates relevant knowledge in the model.

2. Defining Analysis Dimensions :

Recipient information analysis

Shipping address analysis

Product combination and value analysis

3. Structured I/O : Input data is organized in compact CSV format to maximize token efficiency; output is forced to strict JSON so my backend can parse it directly.

Result: The V1 prompt turned the LLM into a “junior analyst” that follows a clear workflow but still produces many false positives.

Stage 2 – Injecting Business Knowledge for Context‑Specific Analysis

I added exemption rules and background knowledge to reduce misclassifications.

Challenge 1: High discounts ≠ risk – I clarified that many orders are first‑time purchases with platform subsidies, so high discounts alone are not risky.

Challenge 2: Random strings ≠ fake names – I explained that system‑generated user IDs are random and not indicative of fraud; the model should focus on the actual recipient name.

Challenge 3: Zero‑price items, nicknames, benefit products ≠ risk – I added rules such as “Zero‑price items are usually gifts and not risky,” and “Nicknames are common and should not be flagged alone.”

Result: The AI’s “emotional intelligence” improved, and false‑positive rates dropped significantly.

Stage 3 – Deepening Analysis: Teaching AI to Think Like a Detective

I expanded the model’s perspective from individual orders to coordinated group behavior.

Bottleneck 1: Ignoring low‑value product risk – I instructed the model to treat large volumes of low‑value, fast‑moving goods (e.g., hundreds of beverage cases) as potential arbitrage signals.

Bottleneck 2: Lack of consistency view – I introduced the “shopping‑cart consistency” concept, telling the model that identical or highly similar product lists across different accounts indicate scripted or organized activity.

Result: The AI advanced to a “senior analyst” capable of detecting organized, large‑scale fraud.

Stage 4 – Ultimate Evolution: Enabling AI to Make Judge‑Like Decisions

The final challenge was distinguishing genuine fraud rings from benign clusters caused by marketing campaigns.

I implemented a “dual‑hypothesis decision framework”:

Hypothesis A: Coordinated risk ring

Hypothesis B: Benign customer segment

The model must first search for “hard links” – decisive evidence such as identical non‑public shipping addresses. If hard links exist, the cluster is deemed risky; otherwise, the model evaluates whether benign explanations suffice.

With few‑shot examples for both scenarios, the prompt became a complete expert system, allowing the AI to act as a risk‑control judge.

Summary and Reflections: My Prompt‑Engineering Principles

Start by mimicking expert thinking, then abstract rules into a reusable framework.

Rules form the skeleton; business context provides the flesh.

Negative examples are essential – teaching the AI what is not risk is as important as what is.

Move from “instructions” to a “thinking model” – the highest‑level prompt teaches methodology, not step‑by‑step commands.

This journey shows that prompt engineering is a creative cross‑discipline that bridges domain expertise and general AI, enabling engineers to empower powerful models with specialized knowledge.