When and How to Integrate Performance Testing into the Development Lifecycle

Current Situation

Test managers do not know what performance testers are doing.

They are unaware of test progress.

They cannot assess whether performance testing is effective.

They do not know how to assist performance testers.

Purpose of This Article

Understand performance test progress to better control the overall testing process.

Understand the quality of performance testing.

Ten Questions

When should performance testing be introduced?

What is the performance testing process?

Is it necessary to raise performance testing?

What types of performance testing exist?

How to analyze performance requirements?

How to measure performance?

What can (and cannot) performance testing achieve?

How to verify the quality of performance testing?

…

Q1: When Should Performance Testing Be Introduced

Performance testing in the development lifecycle

Unit testing – code‑level testing of execution efficiency, memory usage, and resource consumption, performed by developers.

Component/Service/API testing – testing of completed reusable functions, which may involve code‑level tests or external calls (e.g., webservice interfaces), performed by testers.

System testing – the whole system is implemented and tested by simulating user usage; this is the main focus of performance testing.

Production environment testing – after system testing passes, a more complete production‑like environment is built to test interactions among multiple deployed systems.

When to intervene in system testing?

Stable version – entering too late makes schedule uncertain and may affect functional testing; late discovery of performance issues leads to extensive rework.

Early – allows the process to run smoothly and data to be free of serious problems. Performance testing should start as early as possible, with attention to the user model in the performance plan.

Q2: What Is the Performance Testing Process

Agile testing emphasizes continuous verification, correction, and multiple iterations.

Traditional testing often lacks this verification step, causing the test direction to drift and reducing effectiveness.

Current progress stages

Documentation – steps 1‑3.

Execution – steps 4‑7.

In traditional methods, documentation is easy to inspect, but the execution phase may be invisible to the test manager, raising the question of whether an execution without reported issues is truly unknown before the performance report.

If the current workflow has this problem, it must be addressed through standardized execution processes and inspection mechanisms.

Q3: Is It Necessary to Raise Performance Testing?

New projects – generally require performance testing.

New versions – consider changes that may affect performance, such as increased user volume, data volume, or architectural changes (e.g., replacing a streaming server).

Test leads often wonder whether a minor feature addition warrants a new performance test. If the user model does not change, retesting may be unnecessary; otherwise, it is required.

Current challenges include low reusability of performance scripts (HTTP‑based scripts break easily) and the need to rebuild test environments, increasing cost. Improving reusability by moving testing closer to component/service/API level and advancing enterprise virtualization can mitigate these issues.

Q4: What Types of Performance Testing Exist

Benchmark testing – single‑user or no‑data tests that provide a baseline for future comparisons.

Load testing – apply moderate pressure (about 20% of maximum) to ensure normal operation.

Stress testing – apply expected maximum pressure to observe behavior under heavy load.

Capacity testing – continuously increase pressure until a bottleneck appears, providing growth insights.

Stability testing – run the system for an extended period to assess long‑term stability.

Except for capacity testing, the other types are generally required to obtain effective results; capacity testing adds valuable extra information.

Stability testing duration should be sufficient to produce convincing data (e.g., 12‑hour resource utilization metrics) rather than vague statements like “tested for 3 days”.

Q5: How to Analyze Performance Requirements

Performance requirements are usually provided by product owners; test leads must verify them from multiple angles:

User perspective – can the user accomplish the task? How fast?

Business perspective – throughput, TPS, hourly work volume, handling of pressure (e.g., partial service vs. complete slowdown).

Technical perspective – use of risky technologies, internal resource constraints.

Other perspectives – owner‑specified resource utilization targets (e.g., 60% server usage).

Requirements should also be evaluated for feasibility (e.g., instant SMS delivery may be impossible) and measurability (e.g., email delivery time).

Distinguish mandatory requirements from flexible expectations (e.g., 100% message delivery vs. acceptable page load time of 3 s).

Q6: How to Measure Performance

Evaluation criteria

User experience – the most authoritative metric.

Clear performance indicators – when direct user perception is unavailable, substitute with measurable data.

Common performance metrics

Response time – server‑side time for web services.

Page render time – especially first‑screen time for internet sites.

Throughput and TPS – business‑level processing capacity.

Specific criteria – e.g., message arrival rate within 1 s under load.

Metrics must be detectable; some specialized metrics (e.g., client‑side latency) may require extra preparation.

Q7: What Performance Testing Can (and Cannot) Do

Web system performance testing aims to simulate real user behavior and capture user perception.

Building a user model involves defining actions, paths, and frequencies for common, performance‑sensitive, and critical business flows.

Performance testing covers only a subset of system functionality; its primary goal is to uncover bottlenecks, not all defects.

Q8: How to Verify the Quality of Performance Testing

Execution process

Establish execution standards with defined deliverables at each checkpoint.

Assign reviewers to perform inspections according to the standards.

Record execution details, including parameter adjustments and script reruns.

Maintain continuous interaction and confirmation among stakeholders.

Performance report

Let data prove conclusions rather than stating conclusions without evidence.

Source: http://www.uml.org.cn/Test/201304085.asp