How to Merge FLV.js and HLS.js into a Unified Web Video Player

Project Background

A PC‑based live‑stream variety show required a single player capable of playing HTTP‑FLV live streams and, on demand, replaying HLS (HTTP‑Live‑Streaming) recordings. The initial implementation combined flv.js and hls.js via video.js, resulting in a bundled size of ~377 KB. To reduce payload and simplify the API, the two libraries were merged into one unified player.

FLV.js Architecture

flv.js

downloads an FLV container, transmuxes it into ISO BMFF (MP4) fragments, and feeds those fragments to the HTML5 Video element through Media Source Extensions (MSE). The main components are:

NativePlayer : thin wrapper around the browser’s native video element.

FLVPlayer : public API that orchestrates playback.

MSEController : creates a SourceBuffer, receives InitSegment and MediaSegment data, and appends them in order.

Transmuxer : coordinates download, decode, and remux operations via TransmuxingWorker and TransmuxingController.

IOController : selects the appropriate loader for the current environment, buffers incoming bytes, and forwards them to the demuxer.

Demuxer : parses FLV payload into separate video and audio tracks.

Remuxer : converts the tracks into MP4 InitSegment and MediaSegment while synchronising audio‑video.

Data flow:

Loader → IOController → Demuxer → Remuxer → TransmuxingWorker/Controller → MSEController → SourceBuffer → Playback

.

HLS.js Architecture

hls.js

first fetches the index.m3u8 manifest, parses level and fragment (TS) URLs, downloads each TS segment, transmuxes it into MP4 fragments, and pushes the fragments to the video element via MSE. Its architecture is flatter and driven by an event‑based controller system.

Key modules:

buffer‑controller.js : reacts to MSE events (e.g., BUFFER_RESET, BUFFER_APPENDING) and appends MP4 fragments to the SourceBuffer.

playlist‑loader , XHRLoader , LevelController , StreamController : handle manifest loading, level selection, and fragment fetching.

Event chain:

MANIFEST_LOADING → LEVEL_LOADED → FRAG_LOADING → FRAG_LOADED → demux → remux → buffer‑controller → MSE

.

Integration Strategy

Both players share the high‑level pipeline download → demux → remux → MSE. The integration plan extends flv.js with HLS‑specific loader, demuxer, and remuxer while preserving a single IOController and MSEController.

Implementation Details

Loader Integration : Imported FragmentLoader, XHRLoader, M3U8Parser, LevelController, and StreamController from hls.js. The FragmentLoader now performs playlist parsing, level selection, and TS fetching, delegating HTTP requests to XHRLoader.

Codec Wrappers : Added FLVCodec (for FLV) and TSCodec (for HLS TS). Both are instantiated by a unified _mediaCodec inside TransmuxingController.

IOController Coordination : IOController calls bindDataSource on the selected codec. FLVCodec returns the number of bytes consumed so the controller can trim processed data; TSCodec always returns 0 because TS packets are processed as whole units.

Seek Support for HLS Replay : On a seek request, the player scans loaded level information, finds the fragment whose timestamp matches the target, and immediately loads that TS segment.

Error Handling & Logging : Integrated a logger that forwards errors into the existing flv.js error‑handling pipeline, ensuring consistent reporting.

TypeScript Definitions : Added TypeScript typings for all new classes and parameters to enable static type checking.

SEI (Supplemental Enhancement Information) Support : Listened for HJPlayer.Events.GET_SEI_INFO and exposed the decoded SEI payload as a Uint8Array for application‑level use.

Real‑Time Interactive Live Using SEI

The player can deliver interactive poll data embedded in the video stream via SEI NAL units, eliminating a separate WebSocket channel. Workflow:

Backend injects SEI containing poll JSON into the live stream.

During playback, the SDK decodes SEI, de‑duplicates it, and emits GET_SEI_INFO.

The UI component displays the options; user selections are sent back to the server for aggregation.

This approach guarantees perfect AV sync, reduces server cost, and provides acceptable latency when SEI is pre‑injected.

Technical Notes on SEI

In H.264, NAL unit type 6 denotes SEI. Custom SEI typically uses type 5 (user‑data‑unregistered). After the 16‑byte UUID, the remaining bytes constitute the payload; the payload length is derived from the preceding length field. Consumers must parse the payload according to the format defined by their video‑cloud provider.

Open‑Source Release

The combined player was deployed on the Huajiao PC site and open‑sourced at https://github.com/huajiaofrontend/HJPlayer. Developers can clone the repository, raise issues, or contribute enhancements.