How Short URLs Work: Design, Storage, and High‑Concurrency Implementation

Background

People often receive spam SMS that contain short links, like the example shown below.

Benefits of Short URLs

SMS and many platforms have character limits; long links cannot fit in the message body.

They look cleaner and more user‑friendly than long, parameter‑filled URLs.

They enable tracking and analytics when users click the link.

They hide underlying parameters, improving security.

These reasons explain why most spam messages contain short URLs.

Short URL Basic Principle

The short‑URL workflow consists of four steps:

A service maps a long URL to a short URL (e.g., www.baidu.com → www.t.cn/1).

The short URL is embedded in the SMS or other content.

The user clicks the short URL; the browser follows a 301/302 redirect to the long URL.

The target content is displayed.

This article focuses on the first step: converting a long URL to a short one.

Service Design

How to store the mapping?

The mapping must be persisted; otherwise it would be lost on service restart. A relational database such as MySQL can store the mapping using an auto‑increment primary key.

How to guarantee one‑to‑one mapping?

A simple auto‑increment scheme does not guarantee that the same long URL always receives the same short URL; repeated requests produce different short URLs. Achieving strict one‑to‑one mapping would require large storage overhead and possibly an in‑memory cache.

Partial one‑to‑one mapping can be achieved by caching recent or popular mappings in a key‑value store.

Short URL storage format

Short URLs are often generated by converting an integer to a higher base (e.g., base‑32) to reduce length. Storing the integer directly (decimal) is space‑efficient and simplifies range queries, while conversion to other bases can be performed on demand.

High concurrency

Storing counters in MySQL can become a bottleneck under high QPS. Optimizations include:

Caching popular long URLs and recent URLs (e.g., using Redis).

Batch issuing IDs: fetch a block of IDs (e.g., 10,000) from MySQL, allocate them in memory, and replenish when the remaining pool falls below a threshold.

Cache

Cache hot long URLs and recent URLs (e.g., last hour) in memory or Redis. If a request hits the cache, the short URL can be returned immediately without generating a new one.

Batch issuance

Instead of incrementing the counter for every request, retrieve a large batch of IDs from the database, serve them from memory, and write back the updated counter asynchronously.

Distributed design

A single ID generator is a single point of failure. One approach is to run multiple generators, each responsible for a distinct range of IDs (e.g., one generator issues numbers ending with 0, another with 1, etc.). This eliminates inter‑generator communication and reduces contention.

Implementation

For simplicity, the example uses Redis instead of MySQL for storage.

package util;

import redis.clients.jedis.Jedis;

public class ShortURLUtil {
    private static final String SHORT_URL_KEY = "SHORT_URL_KEY";
    private static final String LOCALHOST = "http://localhost:4444/";
    private static final String SHORT_LONG_PREFIX = "short_long_prefix_";
    private static final String CACHE_KEY_PREFIX = "cache_key_prefix_";
    private static final int CACHE_SECONDS = 1 * 60 * 60;

    private final String redisConfig;
    private final Jedis jedis;

    public ShortURLUtil(String redisConfig) {
        this.redisConfig = redisConfig;
        this.jedis = new Jedis(this.redisConfig);
    }

    public String getShortURL(String longURL, Decimal decimal) {
        // Check cache
        String cache = jedis.get(CACHE_KEY_PREFIX + longURL);
        if (cache != null) {
            return LOCALHOST + toOtherBaseString(Long.valueOf(cache), decimal.x);
        }
        // Increment counter
        long num = jedis.incr(SHORT_URL_KEY);
        // Store mapping (could be MySQL)
        jedis.set(SHORT_LONG_PREFIX + num, longURL);
        // Write cache
        jedis.setex(CACHE_KEY_PREFIX + longURL, CACHE_SECONDS, String.valueOf(num));
        return LOCALHOST + toOtherBaseString(num, decimal.x);
    }

    private static final char[] digits = {
        '0','1','2','3','4','5','6','7','8','9',
        'A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z',
        'a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z'
    };

    private String toOtherBaseString(long n, int base) {
        long num = n < 0 ? ((long)2 * 0x7fffffff) + n + 2 : n;
        char[] buf = new char[32];
        int charPos = 32;
        while ((num / base) > 0) {
            buf[--charPos] = digits[(int)(num % base)];
            num /= base;
        }
        buf[--charPos] = digits[(int)(num % base)];
        return new String(buf, charPos, 32 - charPos);
    }

    enum Decimal {
        D32(32), D64(64);
        int x;
        Decimal(int x) { this.x = x; }
    }

    public static void main(String[] args) {
        for (int i = 0; i < 100; i++) {
            System.out.println(new ShortURLUtil("localhost").getShortURL("www.baidudu.com", Decimal.D32));
            System.out.println(new ShortURLUtil("localhost").getShortURL("www.baidu.com", Decimal.D64));
        }
    }
}