How to Achieve Near‑Zero First‑Frame Delay in Video Playback

Background Introduction

First‑frame time is the interval from a user’s click to the display of the initial video frame. "Zero first frame" does not mean a literal 0 ms start, but rather a delay so short (< 100 ms) that users barely notice it.

Our player implements aggressive first‑frame optimizations that can compress this time to under 100 ms, delivering a perception of seamless playback. Certain scenarios (e.g., random playback, cases unsuitable for player reuse) may limit the applicability of some optimizations, but applying as many of the provided techniques as possible can approach a zero‑first‑frame experience for most users.

Composition of the First Frame

The first‑frame time is a core metric for video applications and a key factor in user experience. If loading the first frame takes several seconds, most users abandon playback, making first‑frame optimization critical.

The video playback flow includes obtaining the video URL, establishing network connections, downloading header data, and decoding/rendering. The following sections discuss generic optimization methods and scenario‑specific techniques.

General First‑Frame Optimization Methods

Fetching Playback URL

The first step is to retrieve the video resource URL. If the app server can generate the playback address via a VOD service and embed it in the feed, the client avoids an extra network request.

Network Connection

After obtaining the URL, the player connects to the CDN, starting with DNS resolution. Using HTTPDNS and pre‑resolving likely domains at app launch can reduce latency. Connection reuse (pre‑creating sockets) and TLS False Start with session reuse can eliminate additional RTTs.

Audio/Video Initial Packets

Reducing probe and moving the moov box to the file head shortens the time needed to fetch essential metadata. If the moov box resides at the file tail, extra requests are required; repositioning it to the head avoids this.

Audio/Video Decoding

Asynchronous decoder initialization and decoder reuse can cut the costly MediaCodec creation time on Android. Providing decoding information early allows the decoder to initialize while the network connection is being established, and reusing decoder instances eliminates repeated setup overhead.

Startup Watermark

Limiting immediate playback until a modest buffer is filled reduces stutter in the first 1‑3 seconds without significantly affecting first‑frame latency, improving overall viewing duration.

Preloading

Preloading part of the video data can accelerate start‑up, but the timing, amount, and parallelism must be balanced based on video length, current cache, network speed, and bitrate. For short videos (< 15 s) preloading can start after the current video finishes; for longer videos, decisions depend on predicted stall risk.

Pre‑rendering

Beyond preloading, pre‑rendering decodes and renders the first frame ahead of playback, omitting audio. This technique is especially effective in scrollable short‑video feeds, where the frame is ready when the user focuses on the card.

Scenario‑Specific Optimizations

Long‑Video Playback

Long videos have larger moov boxes (≈ 40 KB per minute). Using fragmented MP4 (fMP4) splits the video into small segments with indexes in the sidx box, drastically reducing the data needed for start‑up. Pre‑rendering during pre‑roll ads can also preload the main content’s first frame.

Playback with Historical Progress

When resuming from a saved position, seeking to the nearest keyframe and discarding frames until the target PTS can require downloading extra data (e.g., 20 Mb for a 5‑second GOP at 4 Mbps). Restricting start‑up to keyframe boundaries avoids this overhead, shortening first‑frame time.

Conclusion

The article presented optimization strategies for each stage of first‑frame processing, introduced preloading and pre‑rendering as powerful tools, and offered targeted solutions for long‑video and resume‑play scenarios.