How Tencent Cut Hundreds of Gigabytes of Bandwidth with Advanced Image Compression

Images have become a visual language for human communication, consuming a large share of internet traffic; reducing data usage while preserving visual quality is a key research focus.

Tencent's TEG Architecture Platform's image storage system TPS handles trillions of images and petabytes of bandwidth, constantly evaluating new compression formats and optimizing legacy JPEG usage.

WebP and HEVC Emergence

Google introduced WebP in 2010, offering a predictive mode that reduces lossily compressed size by 25‑34% compared to JPEG at equal SSIM, and supports transparency with roughly one‑third the size of PNG. WebP’s open format and broad browser (Chrome, Opera) and Android support facilitated rapid adoption, saving over 500 GB of bandwidth in Tencent’s services.

HEVC/H.265 further reduces image size to about 46% of JPEG and animated GIFs to 20% of their original size, but its encoding/decoding speed is much slower. Tencent’s teams developed proprietary high‑efficiency kernels WXAM and SHARP, achieving multi‑fold speed gains over open‑source x265 while maintaining comparable compression.

JPEG Optimization Path

Despite new formats, JPEG still accounts for over one‑third of bandwidth due to compatibility constraints. Tencent applies quality‑based quantization, special handling for QR‑code‑centric images, and psychovisual compression using Google’s Guetzli, which reduces JPEG size by about 30% without the compatibility issues of WebP.

Psychovisual Method

Guetzli leverages three human‑vision insights:

Overlap of cone‑cell spectra allows lower‑precision encoding of blue near yellow regions.

Blue spatial resolution is lower than red/green, permitting coarse encoding of high‑frequency blue details.

Fine visual structures can be encoded with lower precision in areas with high visual noise.

These principles increase zero runs in the JPEG coefficient stream, improving entropy coding efficiency.

Encoding Principles

First iteration finds the optimal global quantization table.

Second iteration identifies coefficients eligible for zero‑out while preserving visual quality.

Third iteration aggressively zeroes selected coefficients, then fine‑tunes to meet size targets.

Performance Optimization

Extended libjpeg‑turbo to support all JPEG sampling formats, improving decode speed.

Ported the butteraugli visual metric to GPU, parallelizing per‑channel and per‑block calculations.

Early‑exit when further zero‑out would exceed global error bounds, cutting redundant work.

Switched from double‑precision to single‑precision arithmetic where acceptable.

Optimized memory and GPU buffer usage with tcmalloc and pooled GPU memory.

Reduced function call overhead by merging static parameter generators.

Balanced CPU/GPU workload based on image resolution to minimize PCIe transfer latency.

Persisted GPU contexts and reused streams to avoid costly re‑initialization.

Deployed multiple processes per GPU to increase utilization and throughput.

These changes reduced average processing latency to less than 10% of the original Guetzli runtime.

Business Deployment

After optimization, the image platform adopted asynchronous compression, persistent storage, and CDN cache‑time control, enabling large‑scale rollout across QQ Album, WeChat Moments, Tencent News, and video services, saving over 1 TB of bandwidth and cutting user download latency by more than 20%.

Future work includes adopting HEIF/HEVC on iOS devices and further psychovisual techniques to continue shrinking image sizes while preserving visual fidelity.