Tencent News Bottom Page Service Upgrade: Refactoring, Configuration, and Performance Optimization

Tencent News’s bottom‑page service is a core scenario handling more than 35,000 QPS and over 1 billion requests per day across five major entry points (mobile client, WeChat/QQ plugins, web, share page, and mini‑program). The original implementation was spread across PHP and Go‑gin codebases, suffering from heavy technical debt, inconsistent frameworks, low stability, and poor development efficiency.

Key problems identified:

Historical code debt: ~100k lines of PHP with poorly designed, tangled logic; core functions exceed 2,000 lines; many legacy bugs.

Low development efficiency: the same feature had to be implemented separately for each of the five scenarios, consuming up to 5 person‑days per request.

Poor service stability: PHP service stability was low, with P99.9 latency > 3.8 s and availability around 99% (two‑nines). The PHP ecosystem no longer matched the company’s performance and monitoring requirements.

Refactoring effort: Approximately 100k lines of PHP were cleaned and re‑architected. The new design adopts a configuration‑driven approach to unify logic across scenarios, reducing code duplication and improving maintainability.

Configuration‑driven design: The service now uses the open‑source expr library to express business rules, mappings, and filters in JSON configuration. Example expressions include:

user.Age in 18..45 and user.Name not in ["admin", "root"]

foo matches "^[A-Z].*"

posts | filter(.CreatedAt >= now() - 7*duration("24h"))

Typical configuration snippets:

{
  "mapper": "ExprEngineMapper",
  "dst_path": "x",
  "source_path": "pathC",
  "desc": "描述信息",
  "ext": "news(src_path) == 1 || news('pathA') == 1 || news('pathB') == 1 ? 1 : nil",
  "data_source": "ResourceInfo"
}

{
  "mapper": "ExprEngineMapper",
  "dst_path": "x",
  "desc": "描述信息",
  "ext": "pre(news('pathA'), news('pathB'), '腾讯新闻')"
}

{
  "mapper": "ExprEngineMapper",
  "dst_path": "x",
  "desc": "下载链接",
  "ext": "'http://xxxx'"
}

{
  "mapper": "ExprEngineMapper",
  "dst_path": "x",
  "desc": "描述信息",
  "ext": "news('pathA') == 1 && req.Apptype == 'android' && req.Appver >= 7260"
}

Tooling support: To understand the legacy code, the team employed xhprof for call‑graph profiling and Xdebug together with PHP CodeCoverage to generate precise execution paths. These tools accelerated the refactor by revealing hot spots and untested branches.

Performance and stability improvements: After the migration, caching was introduced at both client‑side and upstream‑service levels, raising cache hit rates (up to 95% for hot articles) and reducing latency by roughly 10%. The new service now supports fine‑grained configuration layers (global, per‑article‑type, per‑scenario, sub‑scenario) and dynamic filters/Hooks to adapt to future requirements.

Configuration management: The “Rainbow” configuration‑as‑code platform was integrated to ensure configurations are versioned, tested, and automatically deployed via CI/CD pipelines, eliminating manual errors.

Design reflections: The service adopts a batch‑based data loading model that can be further evolved to a fully event‑driven, configuration‑only pipeline, removing batch constraints and enabling more flexible data acquisition.

In conclusion, the bottom‑page service overhaul demonstrates how systematic debt reduction, configuration‑driven architecture, and robust tooling can dramatically improve development efficiency, service stability, and performance for a high‑traffic backend system.