Designing a High‑Availability Microservice Gateway with Nacos and Higress

Why Re‑architect the Gateway

In modern internet or enterprise digital scenarios, a gateway must provide a unified entry point, hide backend topology, handle routing, protocol conversion, authentication, rate limiting, gray releases, observability, and fault isolation. When the number of services grows from dozens to hundreds and instances reach thousands, static reverse‑proxy solutions quickly expose problems such as manual upstream updates, lack of real‑time health checks, and risky hot‑reloading.

Desired Production‑Grade Capabilities

Dynamic service discovery and instant instance change awareness

Hot configuration updates without restarts

High‑performance request forwarding with low latency

Stateless, horizontally scalable instances

Built‑in traffic governance (rate limiting, circuit breaking, gray release)

Comprehensive metrics, logs, and trace collection

Fault‑tolerant design that protects against failures, not just normal traffic

Nacos + Higress Collaboration

Role of Nacos

Nacos acts as the control‑plane, providing two core services:

Service registry: manages service instances, metadata, weights, and health status

Configuration center: stores gateway policies, business configs, and environment‑specific parameters

Role of Higress

Higress, built on Envoy, serves as the data‑plane and handles north‑south traffic. Its strengths include:

High‑performance data processing

Strong dynamic configuration propagation

Easy integration with service discovery, gray release, authentication, and security policies

Full compatibility with modern cloud‑native governance models

Interaction Model

The system works as follows:

Nacos knows which services exist, their health, and associated policies.

Higress consumes this information to route traffic, enforce governance, and update its runtime state.

A simplified call flow:

Client → SLB / Ingress / LVS → Higress Gateway Cluster → Route matching → Service discovery result → Forward to backend microservice
Nacos continuously provides service list, metadata, and config → Higress dynamically updates routing and cluster state

Core Architectural Design

The recommended production topology includes DNS/GSLB, a four‑layer load balancer (L4), multiple Higress pods (stateless), a Nacos cluster (≥3 nodes), and a MySQL HA database for persistence. The architecture is divided into four layers:

Entry layer : Provides public or internal unified access, handles L4 forwarding and basic health checks, masks individual gateway failures.

Gateway layer : Implements L7 traffic governance, authentication, rate limiting, and observability while remaining stateless for easy scaling.

Control layer : Nacos supplies service catalogs and policy configurations, ensuring consistency across the fleet.

Service layer : Business services register themselves to Nacos via SDK or sidecar and expose health endpoints for graceful traffic handover.

Control‑Plane vs Data‑Plane Separation

Separating control and data planes brings several benefits:

Configuration changes do not interrupt request forwarding.

Scaling the data plane does not require copying control‑plane state.

Control and forwarding layers can be independently scaled, improving fault isolation.

Service Registration Process

When a service instance starts, it:

Collects its IP, port, protocol, version, weight, etc.

Registers itself to Nacos.

Periodically sends heartbeats.

On shutdown, deregisters or marks itself unavailable.

Example Java registration code:

Properties properties = new Properties();
properties.put("serverAddr", "127.0.0.1:8848");
NamingService namingService = NamingFactory.createNamingService(properties);
Instance instance = new Instance();
instance.setIp("10.20.3.15");
instance.setPort(8080);
instance.setWeight(1.0);
instance.setHealthy(true);
instance.setEnabled(true);
instance.setMetadata(Map.of(
    "version", "v1",
    "zone", "hangzhou-a",
    "protocol", "http"
));
namingService.registerInstance("order-service", "prod", instance);

Dynamic Instance Change Flow in Higress

When Nacos updates the service list, Higress automatically:

Detects the change event.

Updates local routing and upstream cluster status.

Routes new traffic to healthy instances.

Removes or de‑weights faulty instances.

Key advantages: immediate scaling effect, graceful draining during shrink, and weight‑based routing based on region or tags.

Hot Configuration Propagation

Gateway configuration (routing rules, auth policies, rate‑limit thresholds, black/white lists, gray‑release conditions) is stored in Nacos. A typical update flow:

Config change → Write to Nacos Config Center → Higress subscribes and detects change → Updates runtime routing or policy → New requests follow the new strategy

This enables minute‑level or even second‑level effect for adding routes, adjusting limits, disabling risky APIs, or temporarily diverting traffic.

Production‑Ready Deployment

Deployment Goals

Nacos cluster with at least 3 nodes

MySQL HA (master‑slave or cluster)

Higress with ≥2 replicas

Four‑layer load balancer in front of the gateway

Separate storage for metrics, logs, and traces

Docker‑Compose Validation Stack

version: "3.9"
services:
  mysql:
    image: mysql:8.0
    container_name: nacos-mysql
    environment:
      MYSQL_ROOT_PASSWORD: nacos123
      MYSQL_DATABASE: nacos
    command: ["mysqld", "--character-set-server=utf8mb4", "--collation-server=utf8mb4_unicode_ci"]
    ports:
      - "3306:3306"
    volumes:
      - ./data/mysql:/var/lib/mysql

  nacos:
    image: nacos/nacos-server:v2.3.2
    container_name: nacos
    environment:
      MODE: standalone
      SPRING_DATASOURCE_PLATFORM: mysql
      MYSQL_SERVICE_HOST: mysql
      MYSQL_SERVICE_PORT: 8848
      MYSQL_SERVICE_DB_NAME: nacos
      MYSQL_SERVICE_USER: root
      MYSQL_SERVICE_PASSWORD: nacos123
    ports:
      - "8848:8848"
      - "9848:9848"
    depends_on:
      - mysql

  higress:
    image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:latest
    container_name: higress
    ports:
      - "80:80"
      - "443:443"
      - "15021:15021"
    depends_on:
      - nacos

Kubernetes Production Recommendation

Deploy Higress as a Deployment with resource requests and limits, readiness/liveness probes, and HPA.

Deploy Nacos as a StatefulSet with three replicas and persistent storage.

Use an external MySQL HA instance for persistence.

Leverage HPA/VPA for elastic scaling.

Higress Deployment example (excerpt):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: higress-gateway
  namespace: gateway-system
spec:
  replicas: 3
  selector:
    matchLabels:
      app: higress-gateway
  template:
    metadata:
      labels:
        app: higress-gateway
    spec:
      containers:
        - name: higress
          image: higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/gateway:latest
          ports:
            - containerPort: 80
            - containerPort: 443
            - containerPort: 15021
          resources:
            requests:
              cpu: "500m"
              memory: "512Mi"
            limits:
              cpu: "2"
              memory: "2Gi"
          readinessProbe:
            httpGet:
              path: /ready
              port: 15021
            initialDelaySeconds: 5
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /healthz/ready
              port: 15021
            initialDelaySeconds: 10
            periodSeconds: 10

HPA example for automatic scaling based on CPU utilization:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: higress-gateway-hpa
  namespace: gateway-system
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: higress-gateway
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65

Nacos Cluster Recommendations

At least three nodes to avoid single points of failure.

Separate from business databases to improve control‑plane availability.

Monitor Nacos metrics and logs independently.

High‑Concurrency & High‑Availability Engineering

Capacity Estimation

Estimate required instances using:

required_instances = peak_QPS / safe_QPS_per_instance * redundancy_factor

Example: peak 80,000 QPS, safe 12,000 QPS per instance, redundancy 1.5 → ~10 instances.

Connection & Thread Model

Keep‑Alive connection reuse

Proper downstream connection pool sizing

Idle connection reclamation

TLS session reuse and HTTP/2 multiplexing

Too small a pool causes request queuing and latency spikes; too large a pool exhausts file descriptors and overloads backend instances.

Timeout, Retry, and Circuit‑Breaker Coordination

Timeout must be shorter than the overall SLA.

Retries only for idempotent APIs (usually 1‑2 attempts).

Circuit‑breaker should trigger before a full service collapse.

Non‑idempotent operations (e.g., order creation) should not be retried at the gateway.

Observability & Fault Diagnosis

Metric Suite

Total requests and per‑route request counts

2xx/4xx/5xx distribution

Upstream timeout and circuit‑breaker trigger counts

Rate‑limit hit counts

P50/P95/P99 latency

Active connections and healthy upstream instance counts

Prometheus scrape configuration (excerpt):

scrape_configs:
  - job_name: higress
    static_configs:
      - targets: ["higress-gateway-1:15020", "higress-gateway-2:15020"]
  - job_name: nacos
    static_configs:
      - targets: ["nacos-1:8848", "nacos-2:8848", "nacos-3:8848"]

Log Design

Access logs (JSON) with timestamp, traceId, client IP, host, path, method, response code, upstream service, upstream instance, latency, user/tenant ID.

Error logs for gateway failures.

Audit logs for configuration changes.

Tracing

Gateway generates or propagates a traceId, enabling end‑to‑end visibility across services, databases, and message queues. Sampling rates can be tied to traffic tiers.

Typical Failure Scenarios

502 surge: Check Higress error logs, upstream health, protocol mismatches, timeout/retry settings.

Latency spikes: Examine TLS handshake volume, downstream connection pool saturation, GC or CPU spikes on instances, recent config changes.

Post‑release errors for specific users: Verify gray‑release rules, required headers/cookies, version metadata, and dirty instances in the registry.

Business Service Integration Example

An order service built with Spring Boot registers to Nacos and is exposed via the gateway.

Maven Dependencies

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    <dependency>
        <groupId>com.alibaba.cloud</groupId>
        <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
</dependencies>

application.yml

server:
  port: 8080
spring:
  application:
    name: order-service
  cloud:
    nacos:
      discovery:
        server-addr: 127.0.0.1:8848
        namespace: prod
        group: DEFAULT_GROUP
        metadata:
          version: v1
          zone: hz-a
          protocol: http
management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus
  endpoint:
    health:
      show-details: always

Controller Code (excerpt)

@RestController
@RequestMapping("/api/orders")
public class OrderController {
    @Value("${spring.application.name}")
    private String appName;
    @Value("${spring.cloud.nacos.discovery.metadata.version:v1}")
    private String version;

    @GetMapping("/{orderId}")
    public Map<String, Object> queryOrder(@PathVariable String orderId,
                                          @RequestHeader(value = "x-request-id", required = false) String requestId) {
        Map<String, Object> result = new HashMap<>();
        result.put("service", appName);
        result.put("version", version);
        result.put("orderId", orderId);
        result.put("status", "PAID");
        result.put("amount", 299.00);
        result.put("requestId", requestId);
        result.put("timestamp", Instant.now().toString());
        return result;
    }

    @GetMapping("/healthz")
    public Map<String, Object> health() {
        return Map.of("status", "UP");
    }
}

Graceful Shutdown Hook

@Component
public class GracefulShutdown {
    private final NamingService namingService;
    @Value("${spring.application.name}")
    private String serviceName;
    @Value("${server.port}")
    private int port;
    public GracefulShutdown(NamingService namingService) { this.namingService = namingService; }
    @PreDestroy
    public void offline() throws Exception {
        String ip = java.net.InetAddress.getLocalHost().getHostAddress();
        namingService.deregisterInstance(serviceName, ip, port);
        Thread.sleep(5000L);
    }
}

These steps prevent the gateway from routing traffic to instances that are about to shut down, avoiding 502/504 spikes during deployments.

Core Gateway Capabilities

Routing Design

Routing matches not only path but also domain, headers, method, query parameters, source IP, and user/tenant tags. Example rules:

Path /api/orders/** → order-service Header x-canary: true → order-service-v2 Requests from overseas IP ranges → overseas data‑center instances

Gray Release Strategy

Prefer explicit label‑based routing over random traffic splitting. Dimensions include header, cookie, user‑ID hash, region/tenant, and version.

Rate Limiting

Global limit to protect gateway capacity.

Per‑route limit for hot endpoints.

Tenant or user‑level limit to prevent abuse.

Recommended keys: IP, userId, appId, tenantId, URI template.

Circuit Breaking & Timeouts

Connection timeout: 50‑200 ms

Request timeout: 1‑3 s (adjust per API tier)

Retry: 1‑2 attempts for idempotent calls only

Circuit‑breaker triggers on error rate, slow‑request ratio, or consecutive failures.

Authentication & Security

JWT/OAuth2 token validation

API‑Key verification

HMAC signature checks

IP black/white lists

Referer/Origin validation

Basic WAF rules

Unified Request Header Enrichment

x-request-id

, x-trace-id, x-forwarded-for, x-real-ip, x-env, x-user-id (or masked identifier)

Production Case: E‑Commerce Flash Sale

During a flash‑sale window, traffic spikes include an 8× increase on the homepage, 10× on product‑detail APIs, and 5× on order APIs. The recommended mitigation:

Offload static assets to CDN; let the gateway handle only dynamic requests.

Cache product‑detail data at the gateway for short periods; keep inventory checks uncached.

Apply dedicated rate limits and user‑level throttling on the order API.

Set conservative timeouts for order and inventory services.

Gracefully degrade non‑critical APIs (e.g., recommendations, comments) during peaks.

Gray Release Example for Order Service v2

Register v2 instance with metadata version=v2.

Route 1 % of traffic via header x-canary: true or a whitelist.

Monitor 5xx, latency, and inventory consistency before scaling to 10 %, 30 %, then 100 %.

Architecture Evolution Roadmap

Stage 1 – Single‑environment entry: ≤20 services, basic unified entry, Nacos registration, baseline monitoring.

Stage 2 – High‑availability cluster: Stateless gateway, Nacos HA, automated config rollout.

Stage 3 – Fine‑grained traffic governance: Tag‑based routing, multi‑level rate limiting, standardized circuit breaking.

Stage 4 – Multi‑region active‑active: Cross‑AZ traffic dispatch, data‑consistency strategies, disaster‑recovery drills.

Conclusion

Nacos + Higress is more than a simple “registry + gateway” combo; it delivers a control‑plane/data‑plane separation, dynamic governance, and high‑throughput capabilities required for production‑grade microservice entry points. To adopt it successfully, teams must implement unified service discovery, hot configuration, stateless scaling, coordinated rate limiting, circuit breaking, graceful deployment, and comprehensive observability.