Huolala’s Journey to a Scalable Service Mesh: Key Lessons from Their Self‑Built Framework

Background

Founded in 2013, Huolala grew into a large internet logistics platform with over 1,000 internal microservices. Early on, services communicated via simple PHP (and a few Java) HTTP calls, which supported fast iteration but later caused stability, governance, and scalability issues as the system expanded.

Why Build Our Own Solution

Existing open‑source options such as Dubbo and Istio did not meet Huolala’s needs, leading the team to develop a custom framework.

Why Not Dubbo

No native PHP support and limited unified service discovery for PHP services.

Overly flexible with >100 extension points, making standardization and management difficult.

Requires substantial customization to fit Huolala’s observability, registry, and deployment practices.

Lacks built‑in governance features and has a weak console.

Consequently, Dubbo was not adopted, though many of its design ideas were referenced.

Why Not Istio

Limited built‑in governance and high cost of extending Envoy filters.

Transparent traffic hijacking introduces compatibility problems with legacy protocols.

Huolala’s environment includes many VMs, not just Kubernetes.

Large clusters cause configuration bloat and MCP scaling issues.

Additionally, Istio’s complexity, longer call chains, and the team’s lack of operational experience prevented its use.

Other Considerations

Diverse and special business scenarios required full control over architecture and code.

Open‑source control planes were too minimal to satisfy Huolala’s reporting and management needs.

Core Design

Architecture

The overall design separates the control plane from the data plane, borrowing concepts from service mesh and leveraging open‑source components.

Data Plane : Java SDK and PHP Proxy provide APIs, RPC protocols, and governance capabilities, running as sidecars or within business processes.

Control Plane : Handles service governance, lifecycle management, and acts as a tool and data‑operation center.

Key Components

Apollo for configuration, decoupling data and control planes.

Consul cluster as the service registry.

PHP and “Generic Call”

To integrate legacy PHP services, Huolala introduced a sidecar (GoProxy) for registration and dynamic configuration, enabling Java services to invoke PHP services via a generic RPC client that follows standard RPC semantics while providing unified discovery, routing, and governance.

RPC Protocol

Huolala offers two RPC protocols: JSON‑RPC over HTTP 1.1 (default, easy to adopt, compatible with legacy HTTP services) and gRPC over HTTP 2.0 (strong contract, high performance, streaming, cross‑language).

SDK Overall Design

The Java SDK mirrors Dubbo’s successful patterns while adding optimizations:

Introduced an internal SPI for framework‑only extensions, balancing extensibility and simplicity.

Enhanced @Wrapper annotation with ignore capability.

Improved automatic injection via @Inject.

Service introspection decouples from traditional registries, using Consul for discovery, Apollo for dynamic config, and an admin control plane for metadata and event reporting.

The core call chain consists of Cluster (fault tolerance), Directory (service discovery), Router (routing), LoadBalance, Filter (interceptor for both client and server), and ClusterFilter (pre‑directory interceptor).

Service Governance and CommandKey

Beyond basic RPC, Huolala emphasizes governance: fault tolerance, routing (version, gray, AZ), and traffic control (rate limiting, circuit breaking, degradation, dynamic timeout, logging, authentication, verification). Inspired by Sentinel’s resource model, a CommandKey mechanism maps each request to a structured key, simplifying rule configuration and hierarchical coverage.

Achievements

By mid‑2022, Huolala’s service‑mesh framework had achieved:

Over 700 integrated services.

More than 5,000 service instances.

Daily call volume in the hundred‑billion range.

Future Evolution

Huolala plans to continue improving the framework by enhancing onboarding efficiency, expanding governance coverage across development, testing, and production, building a proxy‑less Java SDK using JavaAgent, and supporting multi‑language (Golang) runtimes to unify governance across languages.