Inside Twitter’s Scalable Architecture: Following, Caching, and Queues Explained

Twitter’s Core Business

Twitter’s fundamental operations revolve around two actions: following – users view posts from accounts they follow, and be followed – when a user posts a message, all of their followers see it.

Business Logic

The logic is straightforward. For the following side, the system retrieves the list of accounts a user follows and displays their recent posts. For the followed side, the front‑end JavaScript polls the back‑end to check whether any followed users have new messages and updates the UI accordingly.

Three‑Tier Architecture

The classic three‑tier design consists of:

Presentation tier : Apache web server parses HTTP requests and forwards them to the logic tier.

Logic tier : A Mongrel‑Rails server handles application logic, reducing development effort by reusing Rails modules.

Data tier : MySQL stores the core entities – users, messages, and user relationships.

When a user posts a message, the workflow is:

Store the message in the msg table.

Read the relation table to obtain the IDs of followers.

Fetch the latest status of those followers.

Push the new message into a queue for asynchronous processing.

The queue updates each follower’s homepage.

Key data schemas include:

User table: userid, name, pass, status, … Message table: msgid, author_id, msg, time, … Relation table:

relationid, following_ids, followed_ids

Cache = Cash: The Role of Caching

Cache dramatically reduces disk I/O, the biggest bottleneck for response time. Twitter targets an average response time of ~500 ms, ideally 200‑300 ms. The caching layer is introduced as follows:

Vector cache : Stores frequently accessed IDs such as newly posted msgids and author IDs. Hit rate ≈ 99%.

Row cache : Caches the full message content. Hit rate ≈ 95%.

Fragment cache : Caches page fragments for popular authors, improving read efficiency.

Page cache : Caches entire author homepages; lower hit rate (~40%) but reduces load on the database.

All caches are implemented with memcached. Additionally, a Varnish HTTP accelerator sits inside the Apache stack to cache search results, cutting overall load by roughly 50%.

Flood Control Through Isolation

Twitter uses a message‑queue system to isolate spikes in traffic. When a surge occurs (e.g., during a major event), Apache forwards requests to Mongrel, which quickly acknowledges the request and queues the work. This prevents the front‑end from returning HTTP 503 errors.

The queue is powered by kestrel, a custom message‑queue implementation. Each user has an associated queue; messages are placed into both the author’s and each follower’s queues.

Data Flow and Control Flow

Beyond simple request buffering, Twitter employs two flood‑mitigation strategies:

Massive memcached clusters act as a “reservoir” to absorb high I/O demand.

Kestrel queues serve as “release valves,” directing traffic to multiple machines to avoid overload.

Example flow for two authors and one reader:

User logs in via Apache; a worker process is allocated.

New message is posted; Apache forwards it to Mongrel.

Mongrel assigns a msgid, stores it in vector memcached, looks up followers (falling back to MySQL if needed), and caches the message body in row memcached.

Mongrel pushes the msgid into each follower’s kestrel queue and the author’s own queue.

Workers consume the queues, update the corresponding homepages, and return the updated page to Apache for the browser.

Push vs. Pull Trade‑off

Mongrel’s design separates concerns:

Storing IDs in memcached and queuing msgid in kestrel constitutes a successful “push” without directly writing to MySQL.

No explicit notification is sent to users; they pull new content when they poll.

This reflects Twitter’s event‑driven architecture: the logic tier is decoupled from the data tier, and upload/download paths are isolated, enabling independent scaling of each component.