How We Unified API Gateways with Apache APISIX: Architecture, Plugins, and Deployment

Background

API gateways serve as the entry point for external traffic to internal services, shielding differences between microservices and providing dynamic routing, authentication, traffic control, protocol conversion, load balancing, etc., playing a crucial role in service governance.

During the growth of ShouQianBa, multiple teams built their own API gateways with different languages and stacks, resulting in a rudimentary management platform, missing features, security risks, maintenance difficulty, duplicated effort, and performance issues.

To address these problems, ShouQianBa standardized the technology stack and built a new generation gateway based on Apache APISIX .

Why Apache APISIX

High‑performance open‑source project with an active community.

Built on OpenResty (an Nginx extension), so network handling is delegated to Nginx while developers focus on functional components.

Uses etcd to store services and routes, enabling real‑time metadata updates.

Provides rich plugins and an extensible plugin mechanism.

Plugins are written in Lua, facilitating custom development.

These advantages motivated the choice of APISIX.

Secondary Development

ShouQianBa’s APISIX customization focuses on plugins and the management platform.

Plugins

Official APISIX plugins are extensive but do not satisfy all requirements; some configurations are overly complex, so plugins are trimmed and simplified.

Custom plugins use five extension points: check_schema, init, access, header_filter, and body_filter.

Reuse

Plugins are divided into generic and business‑specific ones. To reduce redundancy, inheritance and reuse are applied.

The response format is standardized as HTTP 200 with a JSON body {code, data, msg} , where code=10000 indicates success, requiring downstream response parsing and repackaging.

Because Nginx streams responses, body_filter_by_lua may be invoked multiple times. To avoid duplicated logic, a base plugin wosai_plugin handles status code setting in header_filter and aggregates the full response in body_filter, exposing a resolve_response callback for specific plugins.

-- Base plugin wosai-plugin example code --
local _M = {}

function _M.header_filter(conf, ctx)
    if ngx.ctx.access_exit then
        return
    end
    ngx.ctx.origin_http_status = ngx.status
    ngx.ctx.origin_content_type = ngx.header.content_type or ''
    ngx.status = 200
    core.response.clear_header_as_body_modified()
    ngx.header.content_type = "application/json;charset=utf-8"
end

function _M.body_filter(conf, ctx, resolve_response)
    if ngx.ctx.access_exit then
        return
    end
    local chunk, eof = ngx.arg[1], ngx.arg[2]
    if ngx.ctx.buffered == nil then
        ngx.ctx.buffered = {}
    end
    if chunk ~= "" and not ngx.is_subrequest then
        table.insert(ngx.ctx.buffered, chunk)
        ngx.arg[1] = nil
    end
    if eof then
        local resp_str = table.concat(ngx.ctx.buffered)
        ngx.ctx.buffered = nil
        local resp = resolve_response(conf, ctx, resp_str)
        ngx.arg[1] = cjson.encode(resp)
        ngx.arg[2] = true
    end
end

return _M

-- Plugin wosai-jsonrpc example code --
local _M = {}

function _M.resolve_response(conf, ctx, resp_str)
    -- parse and wrap response
end

function _M.header_filter(conf, ctx)
    wosai_plugin.header_filter(conf, ctx)
end

function _M.body_filter(conf, ctx, resolve_response)
    if not resolve_response then
        resolve_response = _M.resolve_response
    end
    wosai_plugin.body_filter(conf, ctx, resolve_response)
end

return _M

Logging

Standard APISIX logging plugins could not capture the original request after protocol conversion. Custom logging uses access_by_lua_block, body_filter_by_lua_block, and log_by_lua_block to record the raw request, response, and generate logs.

access_by_lua_block {
    wosai_logger.log_request()   -- record original request
    apisix.http_access_phase()
}

body_filter_by_lua_block {
    apisix.http_body_filter_phase()
    wosai_logger.log_response() -- record response
}

log_by_lua_block {
    wosai_logger.log()          -- generate log
    apisix.http_log_phase()
}

Modifications to ngx_tpl.lua during initialization embed these hooks with minimal intrusion.

Management Platform

APISIX’s built‑in dashboard lacks user permission control and multi‑gateway management, so ShouQianBa built a custom platform using the APISIX Admin API, supporting multi‑gateway management, strict permissions, operation logs, dynamic plugin configuration, and separate test/production environments.

The front‑end renders plugin UIs dynamically based on schemas using XRender , greatly simplifying development.

Version Upgrade

The project’s directory structure separates custom plugins from APISIX core, facilitating clean upgrades. The Dockerfile copies configuration, plugins, and modified source files into the official APISIX image; upgrading only requires syncing the modified files and updating the base image tag.

FROM apache/apisix:2.14.0-alpine

WORKDIR /usr/local/apisix
ENV TZ Asia/Shanghai

COPY conf           conf
COPY lua/plugins    apisix/plugins
COPY lua/cli/ngx_tpl.lua    apisix/cli
COPY lua/constants.lua      apisix

EXPOSE 9080 9443

CMD ["sh","-c","/usr/bin/apisix init && /usr/bin/apisix init_etcd && /usr/local/openresty/bin/openresty -p /usr/local/apisix -c /usr/local/apisix/conf/nginx.conf -g 'daemon off;'" ]
STOPSIGNAL SIGQUIT

Deployment

ShouQianBa deploys APISIX with shared code and resource isolation: the same codebase and management platform are used across business lines, while each line runs an independent APISIX instance with its own domain, ensuring reliability.

Conclusion

Since May of the previous year, more than half of ShouQianBa’s business lines have migrated to the APISIX gateway, with no major incidents observed. The gateway’s performance and reliability have been validated in production, freeing teams from low‑level gateway maintenance and allowing them to focus on business development.

APISIX’s design and architecture are excellent, and its plugin system makes custom high‑performance API gateways straightforward to build.

References

Apache APISIX: https://github.com/apache/apisix

lua‑nginx‑module: https://github.com/openresty/lua-nginx-module#body_filter_by_lua_block

XRender: https://xrender.fun