How Uber Scales Its Real‑Time Ride‑Sharing Platform: Architecture & Lessons

According to reports, Uber's business has surged 38‑fold in the past four years. In a detailed interview titled “Scalable Uber Real‑Time Market Platform,” Uber’s chief system architect Matt Ranney explains how Uber’s software works.

The interview does not cover surge pricing, but it reveals Uber’s dispatch system, how it implements geospatial indexing, scales the system, improves availability, and handles failures—for example, using driver phones as an external distributed storage system during data‑center outages.

The overall impression is that Uber grew extremely fast, choosing architectures that support rapid growth and enable newly formed teams to move quickly. To maximize engineering speed, they employ a wide range of technologies on the backend.

Statistics

• Uber’s geospatial index targets one million writes per second, with read speed many times higher.

• The dispatch system runs on thousands of nodes.

Platform

• Node.js

• Python

• Java

• Go

• Native iOS and Android apps

• Microservices

• Redis

• Postgres

• MySQL

• Riak

• Twitter’s Redis‑based Twemproxy

• Google’s S2 geospatial library

• Ringpop – consistent hashing ring

• TChannel – network multiplexing and RPC framing protocol

• Thrift

Overview

• Uber connects passengers and drivers.

• Challenge: matching dynamic supply and demand in real time.

• The dispatch system is a real‑time market platform that matches drivers and riders via mobile phones.

• New Year’s Eve is Uber’s busiest time of the year.

• The industry has progressed rapidly; technologies like mobile phones, the Internet, and GPS, once sci‑fi concepts, are now commonplace.

Architecture Overview

• The driver and passenger native apps on phones drive the system.

• The backend primarily serves mobile traffic over the open Internet, without private networks or QoS.

• Clients connect to the dispatch system, which coordinates drivers and passengers.

• The dispatch system is largely written in Node.js.

• JavaScript can be used for interesting distributed‑system work, and passionate developers can ship features quickly.

• Mapping and ETA information are needed for smarter dispatch decisions.

• Language choices depend on integration: Python, C++, and Java are used.

• Services implement business logic, mostly as microservices written in Python.

• Databases include Postgres (the oldest), Redis (with Twemproxy and custom clusters), MySQL (including a distributed column store), and Riak.

• Post‑trip processing handles rating collection, email notifications, database updates, and payment scheduling, primarily in Python.

• Uber integrates many payment systems.

Old Dispatch System

• Limitations of the original system began to hinder growth, prompting a redesign.

• Although much of the system was rewritten, some services were retained.

• The old system was designed for dedicated passenger transport, assuming one passenger per vehicle and not supporting Uber Pool.

• It was city‑partitioned, which became hard to manage as more cities joined.

• Development was rapid, resulting in multiple points of failure rather than a single point.

New Dispatch System

• To support city sharding and new products, supply and demand concepts were generalized.

• Supply services track performance and state of all supply entities, with rich attribute models (seats, vehicle type, child seat, wheelchair access, etc.).

• Configuration tracking handles cases like partially occupied seats.

• Demand services track orders and all aspects of demand.

• Matching logic is handled by DISCO (dispatch optimization).

• The old system only matched currently available supply; DISCO supports future planning and route adjustments.

• Both supply and demand use geospatial indexes, requiring a routing engine.

Dispatch Flow

• Vehicle location updates are sent to a geo provider; DISCO queries the provider to match drivers and riders.

• The geo provider performs a coarse filter to produce nearby candidates.

• Candidates and demand are sent to a routing/ETA service to compute distances based on road networks.

• Results are sorted by ETA and returned to the driver.

• At airports, a virtual taxi queue is simulated, and providers are queued accordingly.

Geospatial Index

• Designed for high scalability: target one million writes per second, driven by driver updates every four seconds.

• Reads must be much faster than writes because every active app performs reads.

• The original index tracked only dispatchable supply, stored in memory on a few processes.

• The new world requires tracking all supply states and routes, resulting in massive data.

• The new service runs on several hundred processes.

• Uber uses Google’s S2 library to partition the earth into tiny cells, each with a unique 64‑bit ID.

• Cells at level 12 cover 3.31–6.38 km²; the cell ID serves as a partition key for updates and queries.

• DISCO computes a circular area around the passenger, uses cell IDs to query relevant partitions, and aggregates supply data.

• Scaling is achieved by adding more nodes for writes and increasing replication factors for reads.

• Fixed cell size at level 12 is a limitation; dynamic cell sizing could improve query fan‑out.

Routing

• Goals include reducing empty driving, minimizing passenger wait time, and minimizing overall ETA.

• The old system matched only currently available supply, which limited optimization.

• Selecting a driver already en route can reduce passenger wait and driver empty‑run time.

• Future models can handle dynamic conditions, such as sharing rides or delivering parcels.

Scalable Dispatch

• Dispatch is built with Node.js.

• It is a stateful service, so stateless scaling techniques do not apply.

• Node runs in a single process; to utilize multiple CPUs and machines, solutions like Ringpop are used.

• Ringpop implements a gossip‑based consistent hashing ring, providing an AP system in CAP terms.

• Ringpop is embedded in each Node process, enabling scalable membership and request routing.

• Gossip protocols (SWIM) provide weakly consistent group membership with fast convergence.

• Requests are forwarded to healthy nodes based on the hash ring.

• Ringpop relies on Uber’s own RPC mechanism, TChannel, which offers bidirectional request/response with performance comparable to Redis and 20× faster than HTTP.

• TChannel provides high‑performance forwarding paths, pipelining, load inspection, tracing, and a clean separation from HTTP.

• Uber is moving from HTTP/JSON to Thrift over TChannel.

Dispatch Availability

• High availability is critical; brief outages can cause revenue loss to competitors.

• All operations must be retryable and idempotent to avoid duplicate actions.

• Failures should be isolated; services are broken into small pieces so that a single instance failure only reduces capacity.

• Uber prefers terminating components (e.g., databases) to recover from failures, influencing technology choices such as using Riak over MySQL.

• Clients embed load‑balancer logic to bypass failed components, similar to Finagle’s approach.

Data‑Center Failure Handling

• Uber maintains a backup data center and can switch traffic via appropriate switches.

• Ongoing trips may not be replicated; driver phones act as the source of truth for trip state.

• The dispatch system periodically sends encrypted state summaries to driver phones; upon failover, the phone provides the missing state to resume the trip seamlessly.

Shortcomings

• Node’s handling of request fan‑out can introduce high latency.

• In a fan‑out system, small fluctuations can cause significant impact; higher fan‑out increases latency risk.

• A solution is to cancel in‑flight requests on backup servers; TChannel embeds this capability, allowing the faster response to win while canceling the slower duplicate.