Inside LApiGateway: Building a Scalable Microservice Gateway for High Availability

1.1 What is a Microservice Gateway

With the rise of microservice architecture, monolithic applications are split into many independent services, causing API scale to grow dramatically. A microservice gateway sits between external requests and internal services, providing protocol conversion, authentication, rate limiting, parameter validation, monitoring, and other common functions.

1.2 Development Background of LApiGateway

Huolala, a fast‑growing logistics tech company, initially deployed independent service gateways for each business domain, which worked well when the company was small. As the company expanded, the number of domain gateways increased, leading to higher maintenance costs, security and stability risks, and low server utilization.

Rising cost of developing and maintaining common features

Increasing security and stability risks due to divergent implementations

Low resource utilization from isolated deployments

Therefore, Huolala decided to develop its own microservice gateway, LApiGateway, aiming to provide developers with an efficient, stable, and excellent gateway experience.

2.1 Language Choice

Considering the company's Java‑centric ecosystem, Java was chosen as the primary language. The open‑source Spring Cloud Gateway (SCG) was used as the foundation, offering reduced development risk and a non‑blocking asynchronous framework that minimizes thread consumption. Additionally, Reactor’s stream‑based coding is more elegant than imperative approaches.

2.2 LApiGateway Architecture Design

LApiGateway consists of four components:

Service Registry – uses Consul for service discovery.

Configuration Center – uses Huolala’s LConfig service to store and manage routing configurations, with a notification mechanism that enables second‑level configuration updates.

LApiGateway OPS – a management platform for developers to maintain service configurations.

LApiGateway Core – processes requests according to service configurations and supports HTTP, JsonRpc, and gRpc protocols.

2.2.1 Service Discovery

LApiGateway uses Consul as the service registry to obtain downstream service node information.

2.2.2 Configuration Center

The gateway configuration is stored in Huolala’s LConfig service. Its change‑notification mechanism allows configuration updates to take effect within seconds.

2.2.3 LApiGateway OPS

The OPS platform provides developers with a protected configuration space, CRUD operations for service routing, change history and rollback, automatic SOA interface generation, and integration with the MOC platform for operation logging.

Protected independent service configuration space

Service routing CRUD

Configuration change history and rollback

Automatic SOA routing generation

Integration with MOC for operation records

2.2.4 Routing and Plugins

Routing is the core concept; each route is defined by matching rules and plugins that forward matching requests to the appropriate backend service.

Plugins are modular, plug‑and‑play components. LApiGateway currently offers over 30 plugins covering data transformation, protocol conversion, authentication, rate limiting, traffic shifting, API aggregation, etc.

2.2.4.3 Gateway DSL

The DSL enables complex data construction, request data access, helper function calls, and conditional expressions, allowing developers to implement advanced scenarios such as extracting authenticated user IDs and injecting them into request headers.

gateway:
  routes:
    - id: test_account_center_auth
      uri: lb://orchestrator
      predicates:
        - name: Query
          args:
            param: _m
            regexp: account_center_auth
      plugins:
        - name: AddRequestHeader
          args:
            name: userID
            value: isNull(accountcenterauth.data.user_id) ? "guest" : accountcenterauth.data.user_id
        - name: AccountCenterAuth

2.3 High‑Availability Design

2.3.1 Physical Isolation

Different gateway clusters are allocated per domain, with further separation for high‑core or high‑traffic services, preventing a single cluster failure from affecting all business.

2.3.2 Service Isolation

Each service has an independent configuration space in LApiGateway OPS, ensuring that configurations do not interfere across services.

2.3.3 Gray‑Scale Migration

A gray‑scale migration plugin on the KONG traffic gateway enables safe, incremental migration of large traffic volumes.

2.3.4 Traffic Pre‑warming

During peak periods, newly launched pods are gradually warmed up based on Consul registration timestamps to avoid request spikes.

2.3.5 Adaptive Load Balancing

Unlike simple Round‑Robin, the adaptive load‑balancing algorithm detects unhealthy nodes and intelligently redistributes traffic, improving response times and resilience.

3 Future Outlook

3.1 Ultimate Stability

LApiGateway already handles core external traffic and supports multi‑lane architecture. Future work includes one‑click fault‑cluster migration to quickly switch traffic from a failed cluster to a healthy one.

3.2 Ultimate User Experience

Planned improvements:

Plugin marketplace for one‑click discovery and installation.

Route inference to maintain a single source of truth across environments with automated validation.

Intelligent operations to auto‑scale clusters, auto‑register services, and automate migrations.