Bilibili Color Space Conversion Engine for Video Processing

Color space (Color Space) is a mathematical model used to describe and represent colors. Different color spaces serve various applications such as image processing, computer graphics, printing, and photography. Commonly used spaces include sRGB, DCI‑P3, BT.2020, etc.

Bilibili receives user‑uploaded videos with diverse color parameters (bit depth, chroma sampling, color space, transfer functions, etc.). Compatibility issues arise because end‑user devices and browsers may not fully support wide‑gamut or HDR content. To ensure consistent playback, Bilibili processes videos of different color spaces into a unified format, balancing accuracy and processing speed.

The conversion engine is built in three layers: filter layer, engine layer, and device layer. The filter layer provides various filters for handling different color parameters. The engine layer performs pre‑computations and path optimizations (e.g., pre‑computing matrix multiplications, pre‑copying GPU parameters) to reduce runtime work. The device layer implements each filter on different platforms, offering an X86 version with instruction‑set optimizations and a CUDA version.

2.1 Coordinate Transformation – Handles display matrices (rotation, flip, transpose) by applying the appropriate coordinate transform to both CPU‑decoded frames and CUDA frames.

2.2 Quantization / De‑quantization – Supports 8‑bit, 10‑bit, 12‑bit Y′CbCr ↔ YUV conversions, as well as PC‑range ↔ TV‑range transformations.

2.3 Chroma Subsampling – Uses bilinear interpolation for up‑sampling and down‑sampling of chroma (e.g., YUV420) to maintain smooth color transitions.

2.4 Color Matrix Conversion – Maps input signals to RGB using matrices such as BT.709. The engine can also invert the matrix to convert RGB′ to Y′U′V′ and handles special signals like ICtCp.

2.5 Transfer Function Conversion – Implements OETF/EOTF conversions (Gamma 2.4, PQ, HLG, etc.) to match human visual perception.

2.6 Gamut Mapping – Converts colors from a wide‑gamut space (e.g., BT.2020) to a narrow‑gamut space (e.g., sRGB) using either clipping or compression methods. The engine includes a proprietary gamut‑compression algorithm that preserves high‑saturation details.

2.7 Tone Mapping – Reduces HDR luminance range to SDR using a BT.2390‑based curve that combines linear‑to‑PQ conversion with Hermite spline compression, achieving good detail preservation in both shadows and highlights.

2.8 HDR Dynamic Metadata Rendering – Utilizes per‑frame dynamic metadata (e.g., Dolby Vision, HDR Vivid) to generate scene‑adaptive tone‑mapping curves, producing higher‑quality SDR outputs compared with static‑metadata approaches.

3 Engineering Acceleration – CPU side: pre‑computes matrix and interpolation constants, applies instruction‑set optimizations for matrix multiplication and power functions (vectorized pow via exp‑log), and uses lookup tables. GPU side: employs C++ templates to eliminate branching, merges kernels (e.g., range conversion + quantization) to reduce global memory I/O, and applies kernel fusion.

Performance tests show the CPU implementation reaches high FPS on an AMD Ryzen 9 5950X for 4K video, while the CUDA implementation achieves over 225 FPS on an RTX 2060 (excluding encode/decode time). The engine also supports HDR Vivid metadata rendering for upcoming Chinese HDR standards.

4 Summary and Outlook – The engine is now stable in production, serving millions of users. Ongoing work includes better objective evaluation of HDR‑to‑SDR quality (beyond HDR‑VDP and CIEDE2000) and integration of HDR Vivid support, aiming to deliver high‑quality HDR content on Bilibili clients.