Twitter’s Media Platform: Scaling Image Uploads and Storage Optimizations

Today Twitter is creating and persisting 3,000 (200 GB) images per second, saving $6 million in 2015 thanks to improved media storage policies.

Originally a text‑only service in 2012, Twitter transitioned to a media‑rich platform by 2016, supporting photos with previews, multi‑photos, GIFs, vines, and inline video through a new Media Platform .

Henna Kermani, a Software Development Engineer at Twitter, presented the Media Platform at Mobile @Scale London, focusing on the image pipeline but noting that many lessons apply to other media types.

Key lessons from the talk include:

Doing the simplest thing that can possibly work can really screw you. An all‑or‑nothing image upload approach caused lock‑in and poor scalability on weak networks.

Decouple. Separating media upload from tweeting allowed independent optimization and greater operational flexibility.

Move handles not blobs. Storing data and referencing it with handles avoids moving large data chunks, reducing bandwidth and performance overhead.

Adopting segmented resumable uploads dramatically lowered media upload failure rates.

Experiment and research. A 20‑day TTL for image variants balanced storage and computation, saving nearly 4 TB of data per day, halving required compute servers, and saving millions of dollars annually.

On demand. Deleting old image variants and recreating them on‑the‑fly increased flexibility and centralized control.

Progressive JPEG proved to be an excellent image format with strong frontend and backend support, performing well on slower networks.