Extending Redis with Custom Commands and Lua Scripts for Safety and Speed

Redis Command Issues

Thread safety issue

Although Redis runs single‑threaded, thread‑safety problems arise when multiple clients act like concurrent threads sharing the same memory without proper synchronization, leading to race conditions such as assigning the same user two tasks.

Redis stores a user state, e.g., user5277=idle.

Client A reads the state ( status = get("user5277")).

Client B reads the same state.

Client A sets the user to busy ( set("user5277", "busy")) and assigns a task.

Client B also sets the user to busy.

Both clients end up assigning the user two tasks.

The root cause is that, despite command serialization, the high execution speed of Redis means that unsynchronized client requests can be reordered, producing inconsistent results. Adding locks solves the problem but introduces complexity such as lock granularity and dead‑lock handling.

Efficiency issue

Redis can achieve up to 100 k write QPS and 600 k read QPS, but real‑world performance is often limited by network latency: a memory access costs about 100 ns, while a round‑trip between data centers can take around 500 µs, a huge disparity.

When a client reads a key, uses its value to form another key, and reads again, two network round‑trips occur. Because ordinary Redis commands lack server‑side computation, complex operations must be performed on the client, increasing latency. Although Redis provides pipeline, it only helps when commands are independent.

Embedded Lua Execution

Redis includes a Lua interpreter, allowing multiple commands to be combined into a single atomic script that runs on the server.

Lua

Lightweight: core library only, small compiled size.

Efficient: written in ANSI C, fast startup and execution.

Embeddable: can be integrated into other languages or systems (e.g., OpenResty).

Execution steps

When Redis starts (>=2.6), it creates a Lua environment, loads Lua libraries, defines the global redis table, and registers commands such as redis.pcall.

Check if the script has been executed; if not, compile it into a Lua function using its SHA1.

Bind timeout and error‑handling hooks to the function.

Create a pseudo‑client that executes Redis commands from within Lua.

Process the pseudo‑client's return values and send the final result back to the real client.

Usage

Lua scripts are invoked with EVAL (one‑off) or EVALSHA (cached). To use EVALSHA, first load the script with SCRIPT LOAD, which returns the script's SHA1.

127.0.0.1:6379> EVAL "return 'hello'" 0
"hello"

127.0.0.1:6379> SCRIPT LOAD "return redis.pcall('GET', ARGV[1])"
"20b602dcc1bb4ba8fca6b74ab364c05c58161a0a"

127.0.0.1:6379> EVALSHA 20b602dcc1bb4ba8fca6b74ab364c05c58161a0a 0 test
"zbs"

The EVAL syntax is EVAL script numkeys key [key ...] arg [arg ...]. Inside a Lua script, KEYS[N] and ARGV[N] refer to keys and arguments, indexed from 1.

When a Redis command returns no value, Lua receives false, not nil.

Lua Script Examples

Example 1: Retrieve a hash field value and then get the key stored in that field.

// Use: EVAL script 2 A B
local tmpKey = redis.call('HGET', KEYS[1], KEYS[2]);
return redis.call('GET', tmpKey);

Example 2: Pop multiple values from a list until a count is reached or the list is empty.

// Use: EVAL script 2 list count
local list = {};
local item = false;
local num = tonumber(KEYS[2]);
while (num > 0) do
    item = redis.call('LPOP', KEYS[1]);
    if item == false then break; end;
    table.insert(list, item);
    num = num - 1;
end;
return list;

Example 3: Get the top‑N elements of a sorted set and fetch detailed hash information for each.

local elements = redis.call('ZRANK', KEYS[1], 0, KEY[2]);
local detail = {};
for index, ele in elements do
    local info = redis.call('HGETALL', ele);
    table.insert(detail, info);
end;
return detail;

Some Thoughts

Use cases

Atomic operations to avoid data conflicts between clients.

Combine multiple dependent requests into a single round‑trip.

Precautions

Declare variables as local to avoid polluting the global Lua environment.

Be aware of the script's time complexity; a long‑running script blocks the single Redis thread.

If a Lua script errors, previous commands cannot be rolled back.

When commands are independent, pipeline is simpler than a Lua script.

Conclusion

Switching to a new tech stack can feel overwhelming, but mastering Redis custom commands and Lua scripting provides powerful tools for building efficient, safe, and maintainable data‑layer solutions.