AI‑Driven Video Coding: Expert Q&A on Intelligent Compression, Standards, and Future Directions

In the rapidly evolving field of video coding, traditional techniques such as transform‑quantization, prediction, entropy coding, and loop filtering are being replaced by more complex methods to improve compression performance. The discussion highlights the need for more efficient and intelligent coding approaches to drive fundamental changes in video compression.

The session features Wang Shenshe, a 2020 National Technology Invention Award winner and member of the National Engineering Laboratory for Digital Video Coding, and Chen Jing, former chief audio‑video scientist at 51Talk and engineer at Google Chrome Media, who discuss the challenges, innovations, and research trends in audio‑video coding.

Key Q&A topics:

1. Theoretical foundation of deep‑learning‑based coding: Wang notes that, unlike classic rate‑distortion theory, deep learning lacks a unified theoretical model; current solutions are often constrained approximations.

2. Current research status: Academic work has produced modest gains (e.g., >10% improvement in some cases) but hardware implementation remains difficult, limiting practical adoption.

3. Hybrid vs. end‑to‑end approaches: Combining deep learning with traditional codecs demands extensive hardware redesign, while pure neural‑network codecs simplify parameter computation but face data‑transfer bottlenecks, especially on FPGA.

4. Standardization prospects: No clear consensus on what to standardize (network architecture vs. coding parameters), though some domestic efforts aim to compress neural‑network models for industry use.

5. Probabilistic nature of deep learning: Performance depends heavily on training data coverage; achieving universal video understanding would require massive datasets.

6. H.266 vs. AV1 outlook: H.266 offers strong compression but suffers from unclear IP policies; AV1 may gain traction depending on ecosystem support.

7. Human‑centric vs. machine‑centric coding: Human perception focuses on visual enjoyment, whereas machine‑oriented coding targets task‑specific features such as object tracking.

8. Future 3‑D compatibility: Anticipated with the rise of the metaverse; early standards already hinted at 3‑D support.

9. Quality metrics: Traditional objective metrics (PSNR, SSIM) are supplemented by AI‑based assessments, but subjective human evaluation remains the ultimate benchmark.

10. Rate control in neural codecs: Early attempts linked QP to specific networks; newer methods treat QP or a lambda parameter as an input to the network to achieve bitrate control.

11. High‑frame‑rate demand: 60 fps (or higher) improves motion smoothness, especially for sports and immersive content.

12. Deployment considerations for platforms like Xiaohongshu: Encoding complexity must be balanced against compression gains; hardware acceleration (VPU, FPGA) is limited and often less efficient than software solutions.

13. Chip‑level AI acceleration: Existing mobile AI compute can aid encoding either by full‑pipeline neural codecs or by AI‑guided preprocessing to enhance traditional codecs.

Overall, the dialogue underscores both the promise and the open challenges of integrating deep learning into video compression, from theoretical foundations to practical hardware deployment.