Why SkyWalking Beats Zipkin and Pinpoint: A Deep Dive into APM Tools

0. Introduction to APM

In micro‑service architectures a single request often touches many services, possibly written in different languages and deployed across thousands of servers in multiple data centers. This complexity requires tools that can monitor system behavior and help quickly locate performance problems; such tools are known as Application Performance Monitoring (APM) systems.

1. Google Dapper Overview

1.1 Dapper Challenges

A typical Google search request may involve hundreds of query servers and multiple subsystems (ads, spell‑check, image/video/news). The overall latency is highly sensitive to any inefficient subsystem, and engineers need a way to pinpoint which service caused a slowdown.

1.2 Design Goals

Ubiquitous deployment – every component should be traceable.

Low overhead – tracing must add minimal performance cost.

Application transparency – instrumentation should be minimally invasive.

Scalability – the system must support distributed deployment and extensibility.

Fast, comprehensive data analysis.

1.3 Distributed Tracing Principles

1.3.1 Trace Tree and Span

A span is Dapper’s basic work unit. Each span represents a single logical operation (e.g., an RPC or DB call) and is identified by a 64‑bit ID. Spans are linked together to form a trace tree.

type Span struct {
    TraceID int64 // identifier for the whole request
    Name string
    ID int64 // span ID
    ParentID int64 // parent span ID, null for root
    Annotation []Annotation // timestamps and metadata
    Debug bool
}

1.3.2 TraceID

The TraceID uniquely identifies an entire request flow from client to server, allowing reconstruction of the full call chain.

1.3.3 Annotation

Annotations record specific events within a span (e.g., client start, server receive). Four standard annotations are used:

(1) cs : Client Start (2) sr : Server Receive (3) ss : Server Send (4) cr : Client Received

type Annotation struct {
    Timestamp int64
    Value string
    Host Endpoint
    Duration int32
}

1.3.4 Sampling Rate

To keep overhead low, Dapper supports configurable sampling rates and variable sampling, allowing only a subset of requests to be traced.

2. APM Component Selection

Most modern APM solutions are inspired by Google Dapper. This section compares three open‑source tools: Zipkin, Pinpoint, and SkyWalking.

2.1 Comparison Items

Probe performance – impact on throughput, CPU, and memory.

Collector scalability – ability to scale horizontally.

Comprehensive trace data analysis – code‑level visibility.

Developer transparency – ease of enabling/disabling without code changes.

Full topology visualization – automatic detection of service topology.

2.2 Probe Performance

Benchmarks on a Spring‑based application (Spring Boot, MVC, Redis, MySQL) show that SkyWalking’s probe has the smallest impact on throughput, Zipkin is moderate, and Pinpoint’s probe reduces throughput significantly under load.

2.3 Collector Scalability

Zipkin : Server and agents communicate via HTTP or MQ; MQ‑based async consumption allows horizontal scaling.

SkyWalking : Collector supports standalone and cluster modes; communication uses gRPC.

Pinpoint : Supports both single‑node and cluster deployments; agents use Thrift to send data.

2.4 Trace Data Analysis

SkyWalking and Pinpoint provide richer, code‑level analysis than Zipkin. Pinpoint records SQL statements and supports extensive alert rules, while SkyWalking supports over 20 middleware integrations.

2.5 Developer Transparency

Zipkin requires code modifications to enable tracing, whereas SkyWalking and Pinpoint rely on bytecode instrumentation, making them invisible to developers.

2.6 Topology Visualization

All three tools can display full service topology. Pinpoint offers the most detailed view (including DB names), Zipkin shows service‑to‑service links, and SkyWalking provides a balanced view.

2.7 Community Support

Zipkin is backed by Twitter, SkyWalking is an Apache incubating project with strong community activity, while Pinpoint has a smaller team.

2.8 Summary

Considering probe performance, collector scalability, analysis depth, developer transparency, topology, and community support, SkyWalking emerges as the most advantageous choice, and the team adopts SkyWalking as the APM solution.