Cutting 80% of Legacy Query Code: Refactoring Lessons for Faster, Safer Backend Services

Background

The team inherited a ten‑year‑old Query Optimizer module used in a search pipeline. The original codebase was large, hard to understand, and suffered from poor performance, stability issues, excessive memory usage (single process ~114 GB), long startup time (≈18 min), and lack of observability. It was built with GCC 4.8, preventing modern C++ features and could not be deployed on the self‑built cloud.

Motivation for Refactoring

Iteration efficiency was low – adding a simple operator required ~3 person‑days.

Stability problems manifested as occasional P99 latency spikes.

Startup time and memory consumption were unacceptable.

Insufficient monitoring and tracing tools.

Outdated build toolchain (GCC 4.8).

A three‑month refactoring effort was launched to address these issues.

Code Smells and Fixes

1. Duplicate Code

Two GBK↔UTF‑8 conversion functions differed only by parameter order.

Fix: Use static analysis (e.g., CodeCC) to detect duplication, extract the common logic into a shared utility function, and increase unit‑test coverage to enforce reuse.

2. Long Functions

One function spanned 1,380 lines, many of which were commented out.

Fix: Split the function into smaller, well‑named sub‑functions, delete dead code (version control preserves history), and document each extracted function.

3. Bloated Classes

A request‑handling class bundled HTTP service instances, cache instances, and dozens of strategy logics.

Fix: Define clear responsibilities, extract related fields and methods into separate classes, and document class purpose.

4. Long Parameter Lists

A method accepted 56 parameters, making calls error‑prone.

Fix: Group related parameters into a configuration struct or object, pass objects instead of many primitives, and increase unit‑test coverage.

5. Confusing Temporary Fields

Variables such as is_second encoded complex weight calculations, reducing readability.

Fix: Give variables meaningful names or eliminate them, apply the “least code” principle, and combine related logic into a single function when possible.

6. Overly Large Parameter Ranges

The worker object (containing proc_node) was passed to many functions that did not need the full data.

Fix: Apply the “least knowledge” principle – expose only required data and split large structs into smaller, purpose‑specific ones.

7. Unnecessary Serialization

Whole structures were serialized even when only a few fields were needed.

Fix: Define fine‑grained data contracts for each interface and pass only necessary fields.

8. Unnecessary Serial Execution

Two tokenization steps (with and without punctuation) were executed sequentially.

Fix: Identify independent tasks and schedule them with a DAG, measuring added latency and memory usage. This reduced main‑flow latency from 13.19 ms to 9.71 ms (≈26% improvement).

9. Ignored Compiler Warnings

After upgrading GCC from 4.8.5 to 8.3.1, a function lacking a return statement caused undefined behavior and crashes.

Fix: Enable -Wall -Werror to treat warnings as errors and address all warnings during code review.

10. Magic Numbers and Constants

Hard‑coded literals such as 43000 appeared without explanation.

Fix: Replace magic numbers with named constants and add clear comments, following coding standards.

11. Long If‑Chains

A hot‑path contained a chain of >100 conditional branches.

Fix: Replace the chain with a table‑driven lookup to improve branch prediction and maintainability.

Results

Codebase size reduced by ~80%.

Startup time decreased from 18 minutes to a few seconds.

Memory consumption dropped dramatically (single‑process usage no longer exceeds 114 GB).

Latency P99 spikes eliminated; overall latency improved.

Iteration efficiency increased – new operators can now be added in hours instead of days.

Observability improved with added monitoring and tracing.

Modern C++ features enabled by migrating to GCC 8.