Testing Process and Test Case Design for Activity Lottery Feature

When I first started an internship at Alibaba, I knew little about the responsibilities of a tester. My supervisor advised me to "work on projects and grow quickly through practice". This article summarizes the knowledge and insights I gained about testing during my probation period.

Requirement Design Phase – Test Planning and Requirement Evaluation : During requirement reviews, testers should understand the business goals, identify risks, define test scope, objectives, focus areas, and schedule. Consider hidden issues such as user experience, PR, financial risks, and address them early to avoid extra development and testing costs.

Development Design Phase – Analysis and Design : Testers need to be familiar with the technical solution, assess its suitability, and write test cases based on quality attributes. Conduct test‑case reviews with developers and product owners to align understanding and uncover hidden logic.

Testing Phase – Execution and Strategy Implementation : Perform smoke tests, verify defect fixes, run regression (manual or automated), and conduct non‑functional checks (performance, compatibility, monitoring). Organize business acceptance testing to ensure the product meets functional expectations.

Pre‑ and Post‑Release Activities :

Gray‑release verification – validate changes in a safe environment before full rollout.

Release coordination – ensure configuration items are correct and logs/monitoring meet expectations.

Online regression – monitor and re‑test in production under real network conditions.

Issue tracking – analyze post‑release problems to improve test coverage and prevent future loss.

Test Case Example – Lottery Feature :

Requirement: Users see a lottery entry on an activity page, a progress bar for prize pool consumption, and can draw prizes based on two rule sets (A and B). Unauthenticated users cannot draw.

Test Description

Precondition

Steps

Expected Result

User without real‑name authentication cannot draw

User not authenticated

1. User A opens activity page

2. User A attempts to draw

1. Prompt to complete real‑name authentication

2. No lottery API request is sent

Rule A user receives prize A

User matches Rule A

1. User A opens activity page

2. User A draws

1. User receives prize A

2. No prize B is awarded

Rule B user receives prize B

User matches Rule B

1. User B opens activity page

2. User B draws

1. User receives prize B

2. No prize A is awarded

Beyond functional checks, consider additional scenarios such as rule exclusivity, idempotency of draws, rule transitions, and API failures.

Analysis Steps :

Requirement analysis – identify scope, affected pages, API changes, data sources.

Technical solution review – evaluate rule exclusivity, idempotency, and exception handling.

Test Case Design Methods :

Equivalence class + boundary value partitioning.

Scenario‑based testing – cover normal, special, and abnormal scenarios.

Historical experience – incorporate known defects and operational risks.

Stability testing – performance, concurrency, availability, compatibility, and rate‑limit verification.

Stability Governance – Core Link Analysis : Map inter‑system calls to identify critical paths and focus testing on high‑risk links.

Dependency Analysis – Strong/Weak Dependencies : Use chaos engineering to inject failures and verify system behavior under downstream issues.

Rate‑Limit Safeguard : Ensure the lottery API returns a user‑friendly message when throttled and that non‑throttled users can still draw.

Test Description

Precondition

Steps

Expected Result

Lottery API rate‑limit prompt

QPS exceeds configured threshold

1. More than N users trigger the lottery simultaneously

1. Non‑throttled users draw normally

2. Throttled users see "The activity is very popular, please try again later"

Loss Prevention – Financial Flow Testing & Reconciliation : Verify prize limits, per‑user restrictions, and perform hourly SQL reconciliation and real‑time message checks to detect anomalies early.

Safety Production : Follow strict change‑window policies, monitor key business metrics after release, and ensure compliance with safety standards.

In summary, a thorough understanding of requirements, comprehensive test case design, and proactive stability and risk analysis are essential for delivering high‑quality, reliable features.