How Meituan’s Shepherd API Gateway Boosts Efficiency and Reliability

Background

In a micro‑service architecture, service splitting multiplies API scale, making API gateways essential. Meituan built Shepherd, a fully self‑developed unified API gateway to replace traditional web‑layer gateways, allowing developers to expose functions and data via configuration.

What is an API gateway?

An API gateway sits between external requests and internal services, handling protocol conversion, authentication, rate limiting, parameter validation, monitoring, and more.

Why Shepherd?

Before Shepherd, developers had to build separate web applications for each API, managing code, components, machines, and databases, resulting in low efficiency and poor resource utilization.

Shepherd provides a high‑performance, highly‑available, and extensible solution, enabling developers to open APIs through simple configuration.

Technical Design and Implementation

Overall Architecture

The gateway consists of a control plane (Shepherd management platform and monitoring center), a configuration center (using Meituan’s Lion service), and a data plane (the Shepherd server).

Control Plane

Developers create APIs, input parameters, generate DSL scripts, test via documentation and mock, then publish with approval, gray release, and rollback mechanisms. Monitoring tracks failures and triggers alerts. The entire lifecycle is configuration‑driven and can be completed in about ten minutes.

Configuration Center

API configurations are described using a custom DSL and stored in Lion with local caching for dynamic, zero‑downtime updates.

Name, Group : API name and group.

Request : Domain, path, parameters.

Response : Result assembly, error handling, headers, cookies.

Filters, FilterConfigs : Functional components and their settings.

Invokers : Backend service (RPC/HTTP/Function) invocation rules.

Data Plane

The data plane builds in‑memory routing tables for API paths, using a MAP for static routes and a prefix‑tree for variable paths. Functional components handle tracing, monitoring, logging, validation, authentication, rate limiting, circuit breaking, and gray release.

High‑Availability Design

Shepherd processes requests asynchronously using Jetty (later Netty) I/O threads, offloading work to business thread pools and supporting async RPC/HTTP calls. Performance optimizations include enabling Nginx keep‑alive, request pre‑warming, asynchronous logging, and reducing CPU hotspots, achieving QPS >15,000.

Service Isolation

Cluster isolation per business line and request isolation via fast/slow thread pools prevent long‑running APIs from blocking shared resources.

Stability Measures

Traffic Control : Multi‑dimensional rate limiting.

Request Caching : For idempotent, frequent queries.

Timeout Management : Fast failure on overruns.

Circuit Breaking : Automatic fallback on failure thresholds.

Security

Integrated signing, SSO, UAC/UPM, Passport, merchant authentication, anti‑scraping, etc., configurable via the platform.

Gray Release

Supports API‑level and downstream service gray releases with various strategies and dynamic adjustments.

Monitoring & Alerting

Provides 360° monitoring of business, machine, and JVM metrics with multi‑dimensional alerts.

Self‑Healing

Elastic scaling based on CPU, rapid node removal, and component‑level isolation.

Migratability

Non‑core APIs can be migrated to Shepherd via a gray‑release SDK, ensuring stable transition.

Usability Design

Automatic DSL Generation

Developers input API documentation, app key, service name, and method; Shepherd fetches JSON Schema from the service framework and auto‑generates mock data and parameter mappings, dramatically improving efficiency.

API Operation Efficiency

Features fast API creation (four types), batch operations for common configurations, and import/export between environments to avoid repetitive setup.

Extensibility Design

Custom Components

Supports loading custom components for special business logic such as custom signatures or result handling.

Service Orchestration

Integrates with Meituan’s internal “Pirate” middleware via RPC to enable multi‑service composition without burdening the client.

Future Plans

Shepherd now serves over 18,000 APIs with daily calls in the billions. Upcoming focus areas include cloud‑native architecture evolution, static website hosting, and a component marketplace.

Cloud‑Native Evolution

Plans to migrate the gateway to Meituan’s Serverless platform Nest, extract core functions into SDKs, and allow developers to select only needed components, reducing WAR size and improving security.

Static Website Hosting

Will provide a general solution for hosting static sites with storage, custom domains, authentication, and CI/CD integration.

Component Marketplace

Encourages developers to share custom components, fostering an ecosystem and avoiding duplicate effort.