How Modern Internet Companies Design Scalable Backend Architecture

Overall Architecture

APP, PC and third‑party callers obtain the load‑balancer IP via traditional DNS (LocalDNS) or HttpDNS for more real‑time resolution.

Requests go through the load balancer to a unified access layer that maintains long connections.

The API gateway serves as the entry point for microservices, handling protocol conversion, routing, authentication, traffic control and caching.

Business servers use a PUSH system for real‑time notifications such as IM and alerts.

Business servers communicate via a proprietary RPC protocol and access external third‑party services through a NAT gateway.

Domain Name Resolution

Traditional DNS

DNS is a distributed directory service that maps domain names to IP addresses.

The resolution process involves the client recursively querying a LocalDNS (usually the ISP’s edge DNS) to obtain the IP, and the LocalDNS performing iterative queries to reach authoritative name servers.

Client recursively queries LocalDNS (edge DNS) to get the IP.

LocalDNS iteratively queries upstream DNS servers until the IP is resolved.

HttpDNS

HttpDNS sends DNS queries over HTTP, bypassing the ISP’s LocalDNS to avoid hijacking and cross‑network issues, improving reliability for mobile services.

Compared with LocalDNS, Tencent Cloud HttpDNS offers higher speed, security, intelligence and reliability.

Load Balancing

Load balancing distributes traffic across multiple servers to solve single‑machine performance and single‑point‑failure problems.

Clients first reach a load‑balancer, which uses scheduling algorithms to forward traffic to application servers and performs health checks to remove faulty nodes.

Hardware solutions (e.g., F5) and software solutions (LVS, NGINX, HAProxy) exist, with L4 (transport‑layer) and L7 (application‑layer) modes.

L4 vs L7

L4 load balancers forward packets based on transport‑layer information, while L7 load balancers act as proxies, terminating client connections and forwarding application‑layer requests.

LVS Forwarding Modes

LVS operates below L4 and supports DR (direct routing), NAT, TUNNEL and FULL NAT modes, each with different packet‑rewriting and routing characteristics.

Scheduling Algorithms

Common algorithms include round‑robin, weighted round‑robin, least connections, hash, and consistent hash for distributed systems.

API Gateway

The API gateway is a cluster that provides a single external entry point, handling REST/HTTP APIs, authentication, monitoring, load balancing, caching and traffic control.

API Management

Core functions include full lifecycle management, testing, versioning and rollback.

Front‑end configuration covers HTTP methods, URLs and parameters; back‑end configuration defines microservice names and parameters.

Full Asynchronous Processing

Using non‑blocking I/O and multiplexing (e.g., Netty+NIO or Spring WebFlux) enables high concurrency with few threads.

Chain Processing

Implemented via a filter chain (responsibility‑chain pattern) for routing, protocol conversion, caching, rate limiting, monitoring and logging.

Request Rate Limiting

Rate limiting protects the system from overload; implementations can be cluster‑wide (using Redis) or single‑node (in‑memory), with algorithms such as counter, leaky bucket and token bucket (recommended).

Circuit Breaker and Degradation

When downstream services become unavailable or slow, circuit breaking stops calls to protect upstream stability; degradation can return cached or simplified responses.

Business Isolation

To prevent cross‑impact between services, isolation can be achieved via thread‑pool or cluster isolation, with cluster isolation preferred for Java.

PUSH Notification

The push system supports multiple channels (Apple, Huawei, Xiaomi, FCM) and provides both console‑driven and server‑side integration for real‑time user interaction.

Microservice System

Illustrates the overall microservice architecture.