Why a Successful Feature Doesn’t Prove No Bugs: Planned Software Testing Strategies

Testing Is Not Just Clicking – It’s a Planned Search for Defects

Does a feature running successfully prove it has no problems? No. Correct order amount does not guarantee correct boundary amounts; a single service working does not guarantee multiple services cooperate; ten users placing orders does not mean ten thousand concurrent users won’t crash the system.

Testing aims to design test cases deliberately, uncover defects, verify that the system meets requirements, and reduce deployment risk.

Remember: testing can prove the existence of defects, but it cannot prove the software is absolutely defect‑free.

01 Four Levels of Software Testing

Software consists of small modules, so testing proceeds from small to large:

Unit Test → Integration Test → System Test → Acceptance Test

Think of software as a car:

Unit Test: inspect a single part</code><code>Integration Test: check if connected parts work together</code><code>System Test: verify the whole car runs</code><code>Acceptance Test: customer decides whether to take the car

02 Unit Testing – Inspecting Inside a Module

Unit tests target the smallest testable units such as functions, classes, or modules. For example, a "calculate discount" function in an order system would be tested for:

Correct discount for regular users</code><code>Correct discount for members</code><code>Handling of zero amount</code><code>Discount not exceeding order amount</code><code>Exception on null input

Unit testing focuses on:

Internal logic of the module</code><code>Local data structures</code><code>Boundary conditions</code><code>Exception handling

03 Integration Testing – Checking Inter‑Module Interactions

Each module running correctly alone does not guarantee correct interaction. An order system may include services such as Order, Inventory, Payment, Points, and SMS. Integration tests should verify:

Order service passes correct product ID to Inventory</code><code>Order service handles Inventory failure properly</code><code>Order status updates promptly after payment success</code><code>Points and SMS services receive messages after order creation

Integration testing concerns:

Module interfaces</code><code>Data transfer</code><code>Call order</code><code>Exception propagation</code><code>Module collaboration

Unit tests check inside a module; integration tests check between modules.

04 System Testing – Verifying the Complete System

System testing targets the fully integrated software system, checking not only business functions but also performance, stress, security, recovery, compatibility, and reliability. Example checks include:

Simulate 100 000 concurrent orders</code><code>Measure system response time</code><code>Simulate database failure and verify recovery</code><code>Check unauthorized users cannot view others' orders

05 Acceptance Testing – Does the System Meet Customer Needs?

Acceptance testing is performed from the user or customer perspective, confirming that the system satisfies requirements, contracts, and acceptance criteria. Example customer requirements:

Users must be able to query orders</code><code>Logistics info must be generated after payment</code><code>Refunds must complete within a specified time</code><code>System must support 100 000 orders per day

Common forms include Alpha testing (in‑house with user participation) and Beta testing (real users in production).

System testing asks “does the whole system work?”; acceptance testing asks “is this what the customer really wants?”

06 V‑Model – Mapping Development to Testing

Requirements Analysis ↔ Acceptance Test</code><code>System Design ↔ System Test</code><code>Architecture Design ↔ Integration Test</code><code>Detailed Design ↔ Unit Test</code><code>Implementation

Why this mapping?

Detailed design describes internal implementation → verified by unit tests</code><code>Architecture design describes module composition → verified by integration tests</code><code>System design describes overall capabilities → verified by system tests</code><code>Requirements describe what users want → verified by acceptance tests

07 Static vs. Dynamic Testing

Static testing does not run the program; it discovers problems through:

Requirement reviews</code><code>Design reviews</code><code>Code reviews</code><code>Walk‑throughs</code><code>Document checks

Examples of static findings:

Order amount may be null</code><code>Exception not caught</code><code>Variable not initialized</code><code>Interface documentation mismatches code

Dynamic testing runs the program, providing input data, executing, observing actual results, and comparing with expected outcomes.

Provide test data</code><code>Execute program</code><code>Observe results</code><code>Compare with expectations

Static testing: watch; dynamic testing: run.

08 Black‑Box vs. White‑Box Testing

Black‑box testing designs cases based on requirements, inputs, and outputs without looking at internal code. Example login tests:

Valid username/password can log in</code><code>Incorrect password shows error</code><code>Empty username handling</code><code>Account lock after repeated failures

White‑box testing uses knowledge of the code, designing cases around statements, conditions, branches, and paths. Example code:

if (isMember || amount >= 99) {</code><code>  free shipping;</code><code>} else {</code><code>  charge shipping;</code><code>}

White‑box considerations:

Has the true branch of the if executed?</code><code>Has the false branch executed?</code><code>Has the condition "isMember" been true and false?</code><code>Has the condition "amount >= 99" been true and false?

Test levels and test methods are different concepts.

Unit tests often use white‑box methods but can also be black‑box; system tests are usually black‑box.

09 Equivalence Partitioning – Grouping Large Input Sets

Assume registration age must be between 18 and 60. Input classes:

Valid class: 18‑60</code><code>Invalid: < 18</code><code>Invalid: > 60</code><code>Invalid: non‑numeric</code><code>Invalid: null

Select one representative from each class, e.g., 30, 10, 70, "abc", null.

Group similar inputs and test a few representatives.

10 Boundary Value Analysis – Errors Hide Near Thresholds

Using the same age example, test values around the limits: 17, 18, 19</code><code>59, 60, 61 If the code mistakenly uses age > 18 instead of age >= 18, the boundary test will reveal the off‑by‑one error.

Test just before, at, and just after each threshold.

11 Decision Table – Handling Multiple Condition Combinations

Free‑shipping rule: members get free shipping; non‑members get free shipping when order ≥ 99.

Member + amount ≥ 99 → free shipping</code><code>Member + amount < 99 → free shipping</code><code>Non‑member + amount ≥ 99 → free shipping</code><code>Non‑member + amount < 99 → charge shipping

Decision tables ensure all condition combinations are covered.

12 White‑Box Coverage Criteria

Statement Coverage

Every executable statement must run at least once. This is a weak criterion because it does not guarantee all decision outcomes are exercised.

Decision (Branch) Coverage

Each true and false branch of every decision must be executed.

Condition Coverage

Each basic condition within a decision must take both true and false values at least once.

Decision/Condition Coverage

Both the whole decision’s true/false outcomes and each basic condition’s true/false outcomes must appear.

Condition Combination Coverage

All possible combinations of basic conditions must be exercised (e.g., true‑true, true‑false, false‑true, false‑false for two conditions).

Path Coverage

Every possible execution path through the program must be traversed at least once. This is comprehensive but often impractical for large code bases.

Statement: walk every line; Decision: walk every fork; Condition: walk every branch condition; Combination: walk every condition combo; Path: walk every route.

13 Stub and Driver Modules

During integration testing, some modules may not be ready.

Top‑Down Integration

Start with the top‑level controller and gradually add lower modules. Missing lower modules are replaced by a fake module called a Stub.

Bottom‑Up Integration

Start with low‑level modules and move upward. If the upper module is missing, a fake program called a Driver invokes the lower module.

Top‑down uses stubs; bottom‑up uses drivers.

14 Complete E‑Commerce Order System Case

Requirements:

1. Order amount must be between 1 and 10 000.</code><code>2. Members or orders ≥ 99 get free shipping.</code><code>3. Order creation must call inventory service to deduct stock.</code><code>4. After successful payment, order status must be updated.</code><code>5. Simulate 100 000 concurrent orders before launch.</code><code>6. Customer acceptance based on requirement specification.

Test design steps:

Equivalence Partitioning for Order Amount

Valid: 1‑10 000</code><code>Invalid: < 1</code><code>Invalid: > 10 000</code><code>Invalid: non‑numeric</code><code>Invalid: null

Representative values: 5000, 0, 10001, "abc", null.

Boundary Value Analysis for Order Amount

0, 1, 2, 9999, 10 000, 10 001

Expected results: 0 invalid, 1‑valid, 2‑valid, 9999‑valid, 10 000‑valid, 10 001‑invalid.

Decision Table for Free‑Shipping Rule

Member + amount ≥ 99 → free shipping</code><code>Member + amount < 99 → free shipping</code><code>Non‑member + amount ≥ 99 → free shipping</code><code>Non‑member + amount < 99 → charge shipping

White‑Box Test of Shipping Code

if (isMember || amount >= 99) { free shipping; } else { charge shipping; }

Test data to cover true and false branches: "Member, 50" (true), "Non‑member, 50" (false). For condition‑combination coverage also test the other three combos.

Integration Test of Order ↔ Inventory Service

Correct product ID and quantity passed</code><code>Successful stock deduction when sufficient</code><code>Order creation fails when stock insufficient</code><code>Handle inventory service timeout</code><code>Avoid duplicate deductions on repeated requests

If inventory service is unavailable, use a Stub returning success, out‑of‑stock, or timeout.

Integration Test of Order ↔ Payment Service

Order becomes "Paid" after successful payment</code><code>Order stays "Pending" after payment failure</code><code>Correct handling of payment timeout</code><code>Idempotent handling of duplicate payment callbacks</code><code>Compensation when order update fails after payment success

Performance and Stress Test

Simulate 100 000 concurrent users placing orders. Monitor throughput, average response time, concurrent users, CPU/memory usage, DB connections, error rate, timeouts, and observe behavior under load, including rate‑limiting or degradation mechanisms.

Acceptance Test

Order amount range compliance</code><code>Member and amount‑≥ 99 free‑shipping rule</code><code>Correct inventory deduction</code><code>Correct order status after payment</code><code>100 k concurrent order performance meets contract</code><code>Proper error messages in exceptional cases

Acceptance confirms the system meets user needs and contractual obligations before delivery.

15 How to Write a Test Case Analysis

A good answer should not list only nouns. Use the unified sentence pattern:

For the "business problem", apply "test method", design or check "specific items", to discover "type of defect".

Example: "For the order amount upper‑limit issue, use boundary‑value analysis with values 0, 1, 2, 9999, 10 000, 10 001 to find off‑by‑one errors in comparison operators."

Another example: "For the order service calling the inventory service, use integration testing to verify product ID, quantity, stock deduction result, and exception handling, to uncover interface and error‑handling defects."

16 Knowledge Summary

Software Testing</code><code>├── Test Levels</code><code>│   ├── Unit: inside module</code><code>│   ├── Integration: module interfaces</code><code>│   ├── System: whole system</code><code>│   └── Acceptance: meets requirements</code><code>├── Black‑Box</code><code>│   ├── Equivalence Partitioning</code><code>│   ├── Boundary Value</code><code>│   └── Decision Table</code><code>├── White‑Box</code><code>│   ├── Statement Coverage</code><code>│   ├── Decision Coverage</code><code>│   ├── Condition Coverage</code><code>│   ├── Condition‑Decision Coverage</code><code>│   ├── Condition Combination Coverage</code><code>│   └── Path Coverage</code><code>└── Integration Strategies</code><code>    ├── Top‑Down: Stub</code><code>    └── Bottom‑Up: Driver

17 Closing Instructions

Close the article and, from memory, recite the complete e‑commerce case in Section 14.

Failing to answer at least eight questions prevents advancing to Day 20.

Remember: for test‑level questions answer "which level", for black‑box/white‑box answer "based on what", and for coverage answer "how much of the program was exercised".

Next section will cover CMMI and software process improvement.