Can Redis Power Simple Service Discovery? A Practical Guide

1. What Is Service Discovery?

Service discovery is a straightforward concept: a service provider (typically an HTTP server offering an API) registers its address, and a service consumer (a process needing that API) looks up the address from a service registry. The registry acts as a dictionary mapping service names to lists of provider addresses.

When a provider fails, it should deregister promptly so consumers can obtain fresh addresses; when a new provider appears, the registry should notify consumers.

2. Using Redis as a Service Registry

Redis’s rich data structures make it suitable for storing the service dictionary. Each service name can be represented by a SET containing IP:Port strings. Providers add their address with SADD and remove it with SREM. Consumers retrieve all addresses via SMEMBERS or a random one via SRANDMEMBER.

However, this simple approach has three main issues:

Provider crash handling: If a provider is killed (e.g., kill -9) it cannot deregister, leaving stale entries. The solution is to implement a keep‑alive heartbeat where providers report their health every few seconds. The registry stores the heartbeat timestamp in a ZSET (score = timestamp) and periodically removes entries that have not reported recently.

Consumer notification of changes: Two strategies are possible. (1) Polling: consumers periodically check for changes, optionally using a version key that increments on each registry update. (2) Pub/Sub: the registry publishes a global version change; consumers listen on a single channel and react when the version increments.

Redis single‑point failure: Redis is not distributed by default. Production‑grade discovery systems use distributed stores such as Zookeeper, etcd, or Consul. Redis can be made more reliable with Sentinel, which promotes a replica to master if the primary fails.

Even if the registry cluster goes down, consumers retain the last known list in memory and can continue operating.

3. Service Providers Beyond HTTP

Providers are not limited to HTTP servers; they can be databases, RPC services, UDP services, etc. For example, a MySQL instance can be registered via an agent that also monitors its health and updates the registry accordingly.

4. Service Configuration Reload

Modern discovery systems often store per‑service configuration alongside the address. When a configuration entry changes, the registry pushes updates to interested services, enabling real‑time configuration reload (e.g., changing database URLs or business parameters).

5. Service Management UI

A management dashboard can visualize the registry state, show service dependencies as a tree, and provide detailed information for each registered instance.

6. A Simple Implementation – Captain

The author built a lightweight discovery system called Captain on top of Redis. It wraps Redis operations behind an HTTP API for registration and lookup, without using Pub/Sub. The source code is available on GitHub, including a client SDK, though it is intended for learning rather than production use.