How Twitter Handles 3,000 Images Per Second: Architecture, Scaling, and Media Optimizations
The article explains how Twitter evolved from a text‑only service to a media‑rich platform capable of processing 3,000 images per second by decoupling upload and tweet creation, using handles instead of blobs, implementing resumable segmented uploads, applying TTL policies, and optimizing client‑side formats such as WebP and progressive JPEG.
Twitter now creates and stores up to 3,000 images (about 20 GB) each second, saving millions of dollars annually through storage and compute optimizations.
Early designs tightly coupled tweet creation with media upload, causing bandwidth waste, low scalability, and inefficient storage; large media remained in BlobStore for months without garbage collection.
Key architectural improvements include:
Separating tweet posting from media upload, allowing independent optimization of each path.
Storing media once and referencing it with a handle (mediaId) instead of moving large blobs.
Adopting segmented, resumable uploads that automatically recover from network interruptions, dramatically reducing failure rates in low‑bandwidth regions.
Setting a 20‑day TTL for image variants, deleting them after they become rarely accessed, which cuts storage growth by several terabytes and saves millions of dollars.
Introducing on‑demand variant generation via services like MinaBird, eliminating the need to pre‑store every size.
Client‑side optimizations for Android include a six‑month trial of WebP (25 % smaller than PNG/JPEG) and a switch to progressive JPEG, which offers better perceived performance on slow networks; Fresco library support further reduces load times and failure rates.
Overall, these changes make Twitter’s media pipeline more flexible, cost‑effective, and capable of handling massive image traffic at scale.
Architecture Digest
Focusing on Java backend development, covering application architecture from top-tier internet companies (high availability, high performance, high stability), big data, machine learning, Java architecture, and other popular fields.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.