Designing a Scalable Short URL Service: Key Decisions and Best Practices

Background

Short URL services convert a long URL into a short one; when a user accesses the short URL, the service looks up the original URL. Designing such a service requires consideration of several aspects.

Data Structure

The service can use a simple key‑value store where the key is the generated short URL (unique) and the value is the original long URL.

Algorithm

A straightforward algorithm maps each long URL to a unique short key. One common approach is to start from 1 and increment for each new URL, encoding the integer in base‑36 (26 letters + 10 digits). Before generating a new key, the service must check whether the long URL already exists; if it does, the existing short URL is returned. Duplicate detection can be done with a hash set, though more efficient methods may be considered.

Key and Value Length

The value (original URL) can be limited to 500 characters, which covers most URLs. The key format is t.cn/** . The length of the key determines the capacity: a 5‑character key supports over 60 million URLs, while a 6‑character key supports about 2.1 billion.

Data Capacity

Estimate the storage requirements. For high efficiency the service is usually kept in memory. If a single machine can hold the data, a single‑node deployment is sufficient; otherwise the data must be sharded.

Sharding Strategy

Range‑based sharding

Advantages: simple scaling—add a new server when the current capacity is exceeded. Disadvantages: load may be uneven because newer short URLs receive more traffic.

Modulo‑based sharding

Advantages: more balanced load across servers. Disadvantages: scaling is harder.

In practice one can start with a capacity estimate, use modulo sharding initially, and switch to range‑based sharding for overflow keys.

Interface Design

Define the request and response protocols for creating and retrieving short URLs.

Concurrent Read/Write and Storage

The key‑value data can be stored in an STL hash map, but it is not thread‑safe, so locking or a concurrent container is required. For higher performance, an in‑memory store such as Redis or Memcached can be used, with asynchronous reads/writes to improve concurrency.

Network

If the request volume fits within a gigabit network, a single‑threaded non‑blocking event loop (IO multiplexing) may suffice. For larger traffic, multiple reactor loops can be employed, with a front‑end reactor handling connections and delegating request processing to worker reactors. Simple read/write logic can be handled directly in the event loop, while complex updates may be offloaded to a thread pool.

Security

Optional defenses against malicious users who generate many short URLs to exhaust the key space include validating URL format and rate‑limiting requests per source.

Example

The service at http://t.im/ uses a base‑36 incremental scheme. Sample mappings:

http://t.im/vgu8

http://t.im/vgu9

http://t.im/vgu0

http://t.im/vgua

These examples show modest concurrency, with requests spaced a few seconds apart, and no special URL validation rules.