How Youku Optimized Free‑Viewpoint Video Playback for a Seamless Mobile Experience

What Is Free Viewpoint

Free‑viewpoint video provides a novel viewing mode where each frame includes a depth map (Fig 1). The SDK processes the depth map to generate images from arbitrary angles.

Client Architecture Design

The client consists of five logical layers:

Playback Business Layer: Core user interactions such as angle‑rotation gear, free‑viewpoint prompts, and transition animations; feature toggles allow online enable/disable.

Player Middle Layer: Supports free‑viewpoint in the middle‑layer pipeline and downloads algorithm files, passing file paths to the algorithm layer.

Player Core Layer: Handles data exchange between the core and algorithm layer, composites processed texture data for on‑screen display.

Downloader: Manages VOD and live video file downloading using multi‑segment download to improve efficiency.

Algorithm Layer: Reconstructs frames from depth maps to the specified viewing angle.

Free Viewpoint Performance Optimization Scheme

Stalls were traced to insufficient buffered data. The optimization focuses on three levers: early data pre‑download, multi‑channel downloading, and lowering video bitrate.

Optimization Directions

Identify stall causes and address them via pre‑caching, intelligent bitrate adaptation, dynamic buffer sizing, and multi‑channel download.

Attempted Solutions

Nine approaches were evaluated; four proved viable and were adopted: pre‑caching, intelligent bitrate, dynamic buffer, and multi‑channel download.

Optimization Practices

Stall Rate Optimization

Intelligent Bitrate: Dynamically selects the next TS segment bitrate based on network conditions and playback state (Fig 2).

Dynamic Buffer: Adjusts the core buffer size via a strategy configuration platform to maximize download resource utilization.

Multi‑Channel Download: Splits each file into N chunks and downloads them in parallel, improving throughput (Fig 3).

Pre‑Caching: Caches playback control information and video streams; supports dynamic pre‑cache size; unifies free‑viewpoint configuration in the backend (Fig 4, Fig 5).

Scene Coverage

The SDK supports two rendering modes: a DIBR‑based normal mode and a degraded camera mode. Devices capable of DIBR use the full free‑viewpoint experience; others fall back to the degraded mode, expanding support from roughly 50% of devices to about 80%.

Data Comparison & Effect

After the upgrades, the free‑viewpoint version of “Street Dance 4” achieved nearly double the total playback volume compared with the previous season, and playback smoothness improved by about 70%.

Coverage benefits: low‑end devices that meet a minimum 4K input requirement can now use the degraded mode, raising overall device coverage from roughly 50% to about 80%.

Conclusion

Providing higher‑quality, richer viewing experiences drives continuous technical exploration; future work includes live free‑viewpoint streaming and further performance refinements.