Effective Service Governance for Serverless: Challenges and Solutions

Serverless is a double‑edged sword: it boosts development efficiency but also raises operational complexity, especially when functions are deployed at massive scale. This article, based on Chen Hao’s talk at ServerlessDays China 2021, discusses how to achieve more effective service governance for Serverless.

1. What is Serverless?

Serverless originated in 2006 with Zimiki’s “Pay as you go” model, which failed commercially. Subsequent attempts such as Google App Engine (2008), PiCloud, dotCloud, and various Chinese APP Engine services also collapsed. The real breakthrough came with AWS Lambda (2014) and the introduction of API Gateway (2015), followed by a wave of open‑source projects (Knative, OpenFaaS, Kubeless, Fn, OpenLambda, IronFunctions, Fission, Apache OpenWhisk) that revived Serverless.

Serverless is often equated with FaaS, but the distinction lies in the supporting infrastructure that makes resources “invisible” to developers.

2. Serverless‑related problems

Key operational questions include service discovery, health checking, gray‑release/A‑B testing, metric monitoring, workflow vs. event‑driven orchestration, call‑chain tracing, dependency management, fault tolerance, and SLA guarantees. While a simple “Hello World” function is easy, building a production‑grade, scalable system is far more complex.

Serverless can reduce operational overhead and cost, and enable developers to act as SREs, but it does not automatically simplify business logic.

3. Required supporting facilities for Serverless

The essential infrastructure includes:

Scalable underlying resources and orchestration

Full‑stack observability

Service governance (registration, discovery, configuration, fault tolerance, traffic management)

Observability : Collecting massive metrics is useless without correlation. Correlated data yields information, which can reveal causal relationships and enable knowledge‑driven decisions.

Incident response and health checks : Fast fault localisation, SLA reporting, capacity analysis, and unified health‑checking (beyond simple liveness) are critical. Organizations often have fragmented monitoring stacks; Serverless demands a holistic view.

Traffic management : Traffic shading (gray release), traffic filtering, degradation, and circuit‑breaking are needed to protect services and enable safe rollouts.

Service registration & discovery : How do Serverless functions register themselves? Should they rely on Kubernetes DNS, classic Java service‑registry, or a hybrid approach? Configuration management must shift from a resource‑centric CMDB to a service‑centric view.

4. Overall solution

Two main implementation styles are discussed:

Mesh‑based : Use a sidecar (e.g., Istio) for traffic routing and observability, combined with a Java Agent for in‑process metrics.

SDK‑based : Embed a lightweight SDK into functions to capture internal behavior; more intrusive but works for static languages.

Both approaches rely on Kubernetes as the foundation; the mesh model is preferred for its language‑agnostic nature.

The envisioned architecture places a sidecar and Java Agent alongside each service, handling traffic management, observability, and service registration transparently, allowing developers to focus on business logic while the platform ensures reliability and scalability.

Speaker bio

Chen Hao – Founder of MegaEase, Tencent Cloud TVP, former senior engineering manager at Amazon and technology director at Alibaba. He focuses on cloud‑native and micro‑service scheduling solutions.

For further reading, see the linked articles and the full conference video.