Design and Implementation of Tubi Multimedia Processing Platform (TMPP)

RubyConf China introduced Tubi, a free high‑quality video streaming service, and its backend team presented the Tubi Multimedia Processing Platform (TMPP), which ingests raw media and transforms it into device‑compatible streams.

TMPP addresses three main challenges: the large number of content partners, the massive size of long‑form videos (often 40‑200 GB), and the diversity of legacy formats, requiring a flexible processing pipeline.

The platform decomposes processing into several stages—Parsing, Transcoding, Packaging, Breakfinding, and Checking—each handled by dedicated micro‑service worker clusters with distinct hardware requirements (e.g., high‑disk memory for Parsing, heavy CPU for Transcoding).

Multiple stages are grouped into a Processing Session, which contains one or more Processing Jobs; a session can be reused or extended to avoid re‑executing stable stages.

Communication between the central Controller and worker clusters uses gRPC with Dequeue‑Job and Report‑Job interfaces, allowing implementations in Elixir, Scala, or other languages.

Transcoding, the most resource‑intensive stage, employs parallel processing by slicing videos into thousands of fragments, enabling multi‑node, multi‑CPU execution and requiring a lead‑node/follower‑node resource‑registration scheme built on Erlang RPC.

Task scheduling distributes fragment‑level tasks across follower nodes, while the lead node orchestrates execution, achieving high CPU utilization (≈95 %) and low operational cost, as demonstrated by production metrics.