Scalable Architecture: DNS, Load Balancing, API Gateways & Microservices

Overall Architecture

Clients such as mobile apps, PCs, and third‑party services first resolve the service domain via a traditional Local DNS or an HttpDNS service to obtain the IP of a load balancer. The load balancer forwards traffic to a unified access layer that maintains long‑lived connections. An API gateway sits at the entry point of the microservice ecosystem, handling protocol conversion, routing, authentication, traffic control, and caching. Business servers use a push system for real‑time notifications (e.g., IM, alerts) and communicate with each other via a proprietary RPC protocol, while external third‑party services are accessed through a NAT gateway.

DNS Resolution

Traditional DNS

DNS translates domain names to IP addresses using a distributed directory service. The resolution process typically involves the client querying a local DNS (often provided by the ISP), which then iteratively contacts higher‑level DNS servers until the authoritative answer is found.

Client performs a recursive query to the local DNS (edge DNS server).

Local DNS iteratively queries upstream DNS servers to retrieve the IP.

HttpDNS

HttpDNS sends DNS queries over HTTP to a DNS server, bypassing the ISP’s local DNS. This avoids DNS hijacking and cross‑network access issues, improving reliability for mobile internet services.

Compared with traditional Local DNS, HttpDNS offers:

Speed: Nodes cover top ISPs, Southeast Asia, and North America, providing precise and fast resolution.

Security: Bypasses ISP DNS to prevent hijacking and pollution.

Intelligence: Accurately identifies request sources and directs traffic to optimal nodes.

Reliability: Multi‑region IP clusters with automatic failover ensure >99% SLA.

Load Balancing

To overcome single‑machine performance limits and single‑point failures, load balancers distribute traffic across multiple servers using scheduling algorithms and perform health checks to remove faulty nodes.

Implementations include hardware solutions (e.g., F5) and software solutions such as LVS, NGINX, and HAProxy. Load balancing operates at L4 (transport layer) or L7 (application layer).

L4 vs L7

L4 forwards packets based on transport‑layer information (e.g., TCP SYN) and keeps the connection bound to a specific backend server.

L7 terminates the client connection, parses the HTTP request, selects a backend server, and establishes a separate connection to that server.

LVS Forwarding Modes

LVS (IP load balancing) works below L4 and supports several forwarding modes:

DR (Direct Routing): The load balancer rewrites the MAC address and forwards the packet to the real server, which replies directly to the client. Both the balancer and real server must share a network segment and the real server must configure a virtual IP.

NAT: The balancer rewrites the destination IP to the real server’s IP; the response passes back through the balancer, which rewrites the source IP before sending to the client.

TUNNEL: The balancer encapsulates the packet in an IP tunnel to the real server; the real server replies directly to the client.

FULL NAT: Similar to NAT but with source NAT (SNAT) applied, allowing the response to route back through the balancer, eliminating the need for the balancer to act as a gateway.

Scheduling Algorithms

Round Robin: Distributes requests sequentially across servers without considering load.

Weighted Round Robin: Assigns higher probability to servers with larger weights, useful when server capacities differ.

Least Connections: Sends traffic to the server with the fewest active connections.

Hash: Maps a request key’s hash to a server index.

Consistent Hashing: Minimizes redistribution when nodes are added or removed, improving stability in distributed systems.

API Gateway

The API gateway is a clustered front‑end that provides a single entry point for external clients. It abstracts internal services and offers REST/HTTP APIs while handling authentication, monitoring, load balancing, caching, and traffic control.

Core functions include full lifecycle management of APIs (creation, publishing, versioning, rollback) and environment‑specific testing.

Configuration is split into front‑end (HTTP method, URL, parameters) and back‑end (service name, parameters) settings, enabling seamless translation from client requests to microservice calls.

Asynchronous Processing

By leveraging non‑blocking I/O and multiplexing (e.g., Netty + NIO, Tomcat/Jetty + NIO, or Spring 5 WebFlux), the gateway can handle massive concurrent traffic with few threads, reducing context switches and increasing throughput.

Filter Chain

The gateway uses a responsibility‑chain pattern where filters implement routing, protocol conversion, caching, rate limiting, monitoring, and logging. Heavy operations should be avoided within filters.

Request Rate Limiting

Rate limiting protects the system from traffic spikes. Implementations can be cluster‑wide (using Redis) or single‑node (in‑memory). Common algorithms include counters, leaky bucket, and the recommended token‑bucket approach.

Circuit Breaker & Degradation

When downstream services become unavailable or slow, the upstream service can short‑circuit calls to preserve overall availability. A circuit‑breaker follows the classic state diagram (closed, open, half‑open). Service degradation reduces or disables non‑critical APIs, returning cached or static responses to keep core functionality alive.

PUSH Notification System

The push system supports multiple channels (Apple, Huawei, Xiaomi, FCM) and provides both console‑driven and server‑side integration. The workflow includes device connection, registration, user binding, message persistence, and delivery via TCP or third‑party channels. If a device is offline, messages are stored and retried upon reconnection, with deduplication logic on the client side.

Microservice Ecosystem

Microservices communicate via proprietary RPC protocols and are orchestrated behind the API gateway. The architecture emphasizes isolation (thread‑pool or cluster isolation) and recommends cluster isolation for Java services due to thread overhead.

Key Takeaways

The presented architecture demonstrates how to combine DNS strategies, load‑balancing techniques, API gateway capabilities, rate‑limiting, circuit‑breaker patterns, and push notification mechanisms to build a resilient, high‑performance backend system suitable for modern internet companies.