Scaling Homepage Carousel to Millions of QPS with OpenResty, Lua, Redis, and MQ

High‑traffic e‑commerce homepages like JD.com and Taobao display carousel images that become hotspot data; without proper handling, high QPS can slow page loads or crash back‑ends.

To address this, a solution using OpenResty, Lua, Redis and a message queue (e.g., Kafka or RocketMQ) is presented, capable of supporting tens of thousands to millions of concurrent connections.

Technology stack – OpenResty

OpenResty is a high‑performance web platform built on Nginx and Lua, integrating many Lua libraries and third‑party modules, suitable for handling 10K‑1M+ concurrent connections.

Hot data publishing

Operations staff publish hotspot data (e.g., carousel configuration) to MySQL, then send an add/modify message to an MQ, which synchronizes the data to Redis for fast reads. The slight delay from MQ is acceptable for display‑only data; lower latency can be achieved with binlog listeners such as Canal.

Hot data retrieval

When a client requests the data, the request first reaches OpenResty. Lua scripts check Redis; if the data exists, it is returned immediately. If not, the script fetches the data from MySQL, writes it to Redis, and returns it to the client.

Overall workflow diagram

Sample Lua script implementing the above logic:

<code>ngx.header.content_type="application/json;charset=utf8"
local uri_args = ngx.req.get_uri_args();
local id = uri_args["id"];
local cache_ngx = ngx.shared.dis_cache;
local hotCache = cache_ngx:get('hot_cache_'..id);
ngx.say(hotCache)
if hotCache == "" or hotCache == nil then
    local redis = require("resty.redis");
    local red = redis:new()
    red:set_timeout(2000)
    red:connect("192.168.223.134", 6379)   
    local rescontent=red:get("content_"..id);
    if ngx.null == rescontent then
        local cjson = require("cjson");
        local mysql = require("resty.mysql");
        local db = mysql:new();
        db:set_timeout(2000)
        local props = {
            host = "192.168.223.132",
            port = 3306,
            database = "test",
            user = "root",
            password = "123456"
        }
        local res = db:connect(props);
        local select_sql = "select * from tb_hot where id="..id.." order by created";
        res = db:query(select_sql);
        local responsejson = cjson.encode(res);
        red:set("content_"..id,responsejson);
        ngx.say(responsejson);
        db:close()
    else
        cache_ngx:set('hot_cache_'..id, rescontent, 10*60);
        ngx.say(rescontent)
    end
    red:close()
else
    ngx.say(contentCache)
end</code>

Conclusion: The OpenResty + Lua + Redis approach provides a simple, highly usable solution that can handle hundreds of thousands to millions of QPS for hotspot data.