Mastering System Performance: Metrics, Strategies, and Real‑World Implementation

What Is Performance Optimization

As user volume and business iterations increase, systems can experience slowdowns, latency spikes, or crashes if they are not optimized. Performance optimization spans the entire software lifecycle.

Performance Metrics

Response Time (RT)

Measures the time required to complete a function. Common indicators are average response time (AVG) and percentile values (e.g., TP99).

AVG : total response time of all requests divided by request count.

Percentile (TPn) : the time within which a certain percentage of requests complete, expressed as TPn=m (e.g., TP99=5 ms means 99% of requests finish within 5 ms). Percentiles reflect overall latency better than averages, especially under long‑tail distributions.

Concurrency Capability

Usually measured by QPS (queries per second) or TPS (transactions per second), with TPS being more common in performance assessments.

The Essence of Performance Optimization

Treat response time as the time dimension.

Treat concurrency capability as the space dimension.

Optimization is essentially a trade‑off between time and space, or a “space‑time interchange”.

Example

Imagine a road limited to 50 km/h with only one lane, allowing at most 10 cars per hour. To increase throughput you can either add lanes (increase space) or raise the speed limit (increase time efficiency).

How to Perform Performance Optimization

Systematic Thinking of Optimization Points

People dimension : The technical team (development, testing, operations) collaborates—operations provide monitoring data, testing supplies load‑test results, and developers pinpoint concrete optimization targets.

Product dimension : Optimization is a business function. Consider the following steps:

Identify the business scenarios that need improvement.

Gather monitoring and load‑test data for those scenarios.

Locate system bottlenecks from the data and propose solutions.

Iterate steps 2‑3 until the performance goals are met.

Before investing, evaluate whether the scenario’s frequency justifies the effort.

Optimization Techniques

Requests traverse multiple hardware and software nodes; the total latency is the sum of each node’s processing time. Optimization methods fall into two categories:

Improve the efficiency of a single request.

Enable parallel handling of multiple requests.

Improving Single‑Request Efficiency

Speed up each node in the call chain

Database: add indexes, use read/write separation, sharding.

Application layer: introduce local or distributed caches, leverage Elasticsearch for complex queries.

Code: adopt more efficient algorithms and data structures (e.g., arrays for read‑heavy, linked lists for write‑heavy, bit‑wise operations for modulo).

Business‑level practices

Avoid duplicate queries.

Batch query operations whenever possible.

Choose the most appropriate downstream API (e.g., use a single “A+B” endpoint instead of calling “A” and “B” separately).

Parallelize internal processing Split a request into sub‑requests and process them concurrently, then merge results. A typical implementation uses CompletableFuture for parallel frameworks such as product‑detail page loading.

Asynchronous handling Offload non‑critical post‑processing (e.g., sending payment confirmations, awarding points) to a message queue (MQ). The main request returns quickly while a separate thread or scheduled job completes the background work.

Parallel Processing of Multiple Requests

When many external requests arrive, distribute them across multiple service instances via load balancers and use thread pools within each instance to handle them concurrently.

Real‑World Case Study: Live‑Streaming Product

The flow: a user visits the live‑stream product detail page, places an order, and then accesses the live‑stream room after a permission check.

Monitoring and log analysis revealed that the bottleneck lies in the “live‑product detail” service. The upstream service generated more RPC calls than the downstream service could handle, causing timeouts.

Missing cache and degradation for some non‑core interfaces caused request failures.

Core interfaces with poor performance blocked subsequent requests.

Downstream queries returned heavyweight payloads while upstream only needed a subset of fields.

Incorrect downstream API usage (calling two separate interfaces instead of a single comprehensive one).

Stateless query interfaces lacked caching, leading to frequent RPC calls.

Optimization Directions

First, map the entire call chain to understand interface dependencies and response times.

Weak‑Dependency Interfaces

RPC timeout strategy : Measure stable TP99 (or volatile TP95) response times and set timeout to 1.5 × TP99/TP95.

Cache strategy : Introduce a two‑level distributed cache (Cache A with TTL =m minutes, Cache B with TTL =n minutes, n > 2m). Read from Cache A first; if miss, read from Cache B and asynchronously refresh both caches.

Degradation strategy : Apply circuit‑breaker mechanisms and return default values on exceptions.

Strong‑Dependency Interfaces

RPC timeout : Set timeout to 1.5 × TP99 for strong‑dependency interfaces.

Retry : Based on response‑time variance, configure Dubbo retries to 2–3 attempts.

Cache :

Product basic info: use transparent multi‑level caching with pre‑warming and hotspot handling.

Other stateless query data: local cache.

Parallelizing Product‑Detail Aggregation

The product‑detail page aggregates independent sections (A, B, C, and basic info). The workload is split into four sub‑tasks and processed in parallel, then merged, dramatically reducing overall latency.

Consolidating Query Interfaces

Downstream services expose many overlapping query APIs. They are reorganized into three granularity levels—coarse (basic fields), medium (frequently used fields), and fine (detailed fields). Upstream services must select the appropriate level, eliminating redundant calls.

Conclusion

Performance optimization is an ongoing, iterative process. As features accumulate, the same node may require different tactics over time—initially an index addition may suffice, later batch queries or caching become necessary. The guiding principle is “specific problem, specific analysis”.