Performance Optimization Patterns for High‑Throughput Backend Systems

Abstract

Performance optimization covers reducing response time and increasing system throughput, while also improving service availability during traffic peaks. These goals can conflict; for example, caching reduces latency but may limit thread count and lower throughput, and retrying failed requests improves availability but increases latency.

Large‑scale companies like Meituan face three challenges: growing user base, increasingly complex business logic, and exploding data volume. The goal is to keep TP95 response time low while continuously raising throughput and availability during peaks. This article proposes optimization patterns to achieve that.

Introduction

Pattern‑Based Explanation

The article adopts a pattern‑based approach similar to the classic "Design Patterns" book: each performance optimization pattern is given a name, motivation, principle, a concrete Meituan case study, and a discussion of pros and cons. This helps readers quickly grasp the scenario, solution, and trade‑offs.

Case Study Background

All cases are derived from real Meituan projects. To keep the discussion focused, the author abstracts and simplifies problems, emphasizing that the presented solutions are feasible but not necessarily optimal for every situation. The patterns primarily apply to Java services, though many also apply to C++ services.

Design Principles

Key principles include the Minimum Viable Principle (quick entry and core functionality), Economic Principle (cost awareness across the software lifecycle), Code Reuse Principle (leveraging shared libraries or monolithic codebases), and Occam’s Razor (avoid unnecessary complexity).

Performance Degradation Patterns

Before presenting optimization patterns, the article defines several degradation (anti‑)patterns that optimization aims to avoid.

Long‑Request Congestion Anti‑Pattern

Extended request latency leads to thread pile‑up, increased memory usage, and possible swapping, causing system collapse.

Levered Multilayer Invocating Anti‑Pattern

One client action triggers multiple service calls across layers, creating exponential request growth and network pressure.

Recurrent Caching Anti‑Pattern

Unbounded caching fills memory, triggers frequent full GC, and leads to performance collapse during traffic spikes.

Performance Optimization Patterns

Horizontal Partitioning Pattern

Principle and Motivation

Split a request into multiple independent stages; stages run sequentially, but tasks within a stage run in parallel, reducing overall latency.

Case Study

Parallelizing multiple recommendation algorithms and their feature fetches cut TP95 latency by half.

Pros and Cons

Pros: lower average and TP95 latency, higher throughput and availability. Cons: significant refactoring, increased complexity, potential code duplication.

Vertical Partitioning Pattern

Principle and Motivation

Separate services by business function to avoid resource contention and mismatched availability requirements.

Case Study

Deploying the high‑availability "Guess You Like" recommendation separately from lower‑priority push recommendations reduced TP95 latency by 40%.

Pros and Cons

Pros: improved isolation and resource allocation. Cons: higher maintenance cost and duplicated code.

Runtime 3NF (Constant‑Variable Separation) Pattern

Principle and Motivation

Separate frequently changing data from immutable data to reduce memory, network, and processing overhead.

Case Study

Splitting volatile deal attributes into a separate table and class reduced network traffic and lowered client timeout rate from 9% to <0.01%.

Pros and Cons

Pros: performance gains when variable data is a small fraction of total data. Cons: breaks object‑oriented cohesion, adds maintenance complexity.

Locality Pattern

Principle and Motivation

Organize data services so that frequently co‑accessed data resides on the same server, reducing the number of remote calls.

Case Study

Using large‑memory servers to store all merchant and deal base info reduced per‑request service calls from thousands to a few.

Avoiding Over‑Generalized Solution Pattern

Principle and Motivation

Choose the most resource‑efficient solution for a problem rather than reusing a heavyweight generic system.

Case Study

Replacing a full‑text search service with a k‑d‑tree LBS solution cut average response time from 100 ms to 300 ns.

Sandbox (Real‑Time/Offline Separation) Pattern

Principle and Motivation

Enforce strict separation between online and offline services to avoid cross‑impact on availability and reliability.

Case Study

Isolating offline jobs from online services prevented cascading failures caused by offline writes to online clusters.

Degradation Pattern

Principle and Motivation

Design fallback mechanisms (traffic, quality, or functional degradation) to maintain partial service during severe performance issues.

Case Study

Smart degradation based on response‑time thresholds automatically reduced load without full service outage.

Other Optimization Suggestions

Remove dead code, avoid cross‑region calls, and other pragmatic tips.

Conclusion

Performance optimization patterns provide reusable solutions to recurring scalability problems. By understanding the problem, context, and trade‑offs, engineers can apply the appropriate pattern to improve latency, throughput, and availability.