Design and Implementation of an API Gateway for High-Concurrency Scenarios

Preface – The API gateway serves as a unified entry point for services, handling protocol translation, upstream protection, lifecycle management, and traffic governance at both cluster and service levels.

Stages – The development journey is divided into six phases: requirements analysis, research and selection (choosing KONG over APISIX), platform positioning, version 1.0 (plug‑and‑play business integration), version 2.0 (full API control for the 818 global auto show), and continuous improvement.

Platform Architecture – The gateway manages environments and domains, offering registration, publishing, authorization, and decommissioning of APIs, with monitoring, alerting, logging, debugging, and policy plugins such as MOCK, CORS, rate limiting, caching, black‑/white‑listing, and concurrency control.

Core Technologies

Routes – Define matching rules (hosts, paths, methods) that bind client requests to Services.

Services – Abstract upstream services, specifying URL, protocol, host, port, and path, and can be associated with multiple Routes.

Plugins – Extend functionality (e.g., authentication, rate limiting) by attaching configurations to Services or Consumers.

Authentication

BasicAuth – Uses Base64‑encoded username:password in the Authorization header.

HMAC – Generates a signature with a secret key and includes X‑Date and Authorization headers.

OAuth2 – Supports authorization‑code, implicit, password, and client‑credentials flows.

Code Snippet

public class Main {
    public static void main(String[] arg) throws Exception {
        String url = "http://dev.test.com";
        String username = "dev_test";
        String secret = "dev_test";
        String auth = "hmac username=\"{1}\", algorithm=\"hmac-sha256\", headers=\"X-Date\", signature=\"{2}\"";
        // Get current GMT time (X-Date)
        DateFormat dateFormat = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss 'GMT'", Locale.US);
        dateFormat.setTimeZone(TimeZone.getTimeZone("GMT"));
        String now = dateFormat.format(Calendar.getInstance().getTime());
        System.out.println(now);
        // Compute signature string
        String hash_string = "X-Date: " + now;
        String signature = Base64.getEncoder().encodeToString(sha256_HMAC(hash_string, secret));
        System.out.println(signature);
        // Assemble request headers
        auth = auth.replace("{1}", username).replace("{2}", signature);
        Map<String, String> headers = new HashMap<>();
        headers.put("X-Date", now);
        headers.put("Authorization", auth);
        String ret = sendGet(url, "", headers); // HTTP GET request
        System.out.println(ret);
    }
    /**
     * sha256_HMAC encryption
     */
    private static byte[] sha256_HMAC(String message, String secret) {
        try {
            Mac sha256_HMAC = Mac.getInstance("HmacSHA256");
            SecretKeySpec secret_key = new SecretKeySpec(secret.getBytes(), "HmacSHA256");
            sha256_HMAC.init(secret_key);
            return sha256_HMAC.doFinal(message.getBytes());
        } catch (Exception e) {
            System.out.println("Error HmacSHA256 ==" + e.getMessage());
        }
        return new byte[0];
    }
}

Performance Testing – Ten rounds of internal and external load tests were conducted, reaching up to 1 million QPS on the gateway; during the actual 818 event, the gateway sustained ~110 k QPS with no performance degradation.

Challenges & Solutions – Addressed large request bodies, log collection latency, and traffic spikes by increasing BODY limits, partitioning log collection, adding backup nodes, enabling rate limiting, caching, and dynamic scaling.

Future Direction – The gateway is evolving toward a service‑mesh model (e.g., Istio, Kong Mesh) with service discovery capabilities, aiming to meet growing micro‑service integration demands.