Scaling Username Uniqueness: DB, Redis Cache & Bloom Filter

Introduction

When registering an application, checking whether a username is already taken is trivial for a small user base, but becomes challenging when the number of users reaches hundreds of millions or billions.

Database Solution

The most straightforward method is to query the database directly:

<code>public class UsernameUniquenessChecker {
    private static final String DB_URL = "jdbc:mysql://localhost:3306/your_database";
    private static final String DB_USER = "your_username";
    private static final String DB_PASSWORD = "your_password";

    public static boolean isUsernameUnique(String username) {
        try (Connection conn = DriverManager.getConnection(DB_URL, DB_USER, DB_PASSWORD)) {
            String sql = "SELECT COUNT(*) FROM users WHERE username = ?";
            try (PreparedStatement stmt = conn.prepareStatement(sql)) {
                stmt.setString(1, username);
                try (ResultSet rs = stmt.executeQuery()) {
                    if (rs.next()) {
                        int count = rs.getInt(1);
                        return count == 0; // If count is 0, username is unique
                    }
                }
            }
        } catch (SQLException e) {
            e.printStackTrace();
        }
        return false; // In case of an error, consider the username as non‑unique
    }

    public static void main(String[] args) {
        String desiredUsername = "new_user";
        boolean isUnique = isUsernameUnique(desiredUsername);
        if (isUnique) {
            System.out.println("Username '" + desiredUsername + "' is unique. Proceed with registration.");
        } else {
            System.out.println("Username '" + desiredUsername + "' is already in use. Choose a different one.");
        }
    }
}
</code>

This approach suffers from several problems:

High latency and poor performance, especially when the data set is large.

Significant load on the database due to frequent SELECT queries.

Poor scalability; vertical scaling of the database can be costly and limited.

Cache Solution

To reduce database load, a Redis cache can be introduced for fast uniqueness checks:

<code>public class UsernameCache {
    private static final String REDIS_HOST = "localhost";
    private static final int REDIS_PORT = 6379;
    private static final int CACHE_EXPIRATION_SECONDS = 3600;
    private static JedisPool jedisPool;

    static {
        JedisPoolConfig poolConfig = new JedisPoolConfig();
        jedisPool = new JedisPool(poolConfig, REDIS_HOST, REDIS_PORT);
    }

    public static boolean isUsernameUnique(String username) {
        try (Jedis jedis = jedisPool.getResource()) {
            if (jedis.sismember("usernames", username)) {
                return false; // Username is not unique
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        return true; // Username is unique (not found in cache)
    }

    public static void addToCache(String username) {
        try (Jedis jedis = jedisPool.getResource()) {
            jedis.sadd("usernames", username);
            jedis.expire("usernames", CACHE_EXPIRATION_SECONDS);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void close() {
        jedisPool.close();
    }
}
</code>

The main drawback is memory consumption: storing 1 billion usernames at roughly 20 bytes each would require about 20 GB of RAM.

Bloom Filter Solution

A Bloom filter offers a memory‑efficient probabilistic data structure for membership testing, ideal for large‑scale uniqueness verification.

Core components:

Bit array : a large array of bits, initially all 0, used to represent the presence of elements.

Hash functions : multiple independent hash functions map an element to several positions in the bit array.

Operations:

Add element : hash the element with each function and set the corresponding bits to 1.

Query element : hash the element and check the bits; if any bit is 0, the element is definitely absent; if all are 1, the element is probably present (with a configurable false‑positive rate).

Redis provides a Bloom filter module, allowing direct use of these operations. Example implementation:

<code>import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

public class BloomFilterExample {
    public static void main(String[] args) {
        JedisPoolConfig poolConfig = new JedisPoolConfig();
        JedisPool jedisPool = new JedisPool(poolConfig, "localhost", 6379);
        try (Jedis jedis = jedisPool.getResource()) {
            // Create a Bloom filter named "usernameFilter" for 10 million elements with 1% false‑positive rate
            jedis.bfCreate("usernameFilter", 10000000, 0.01);
            // Add a username
            jedis.bfAdd("usernameFilter", "alvin");
            // Check existence
            boolean exists = jedis.bfExists("usernameFilter", "alvin");
            System.out.println("Username exists: " + exists);
        }
    }
}
</code>

Advantages:

Significant memory savings: storing 10 billion usernames with a 0.1 % false‑positive rate requires only about 1.67 GB, far less than the 20 GB needed for a raw cache.

Constant‑time (O(1)) lookups.

Disadvantages:

False positives are possible; the filter may report an element as present when it is not.

Elements cannot be deleted without affecting the false‑positive rate.

Conclusion

Using a Redis‑backed Bloom filter provides an efficient in‑memory solution for large‑scale username uniqueness checks, balancing memory consumption against an acceptable error rate, and is also applicable to scenarios such as cache‑penetration protection and malicious‑access mitigation.