Design and Implementation of Pitcher: A Go‑Based Reverse Proxy Middleware

Pitcher is a custom reverse‑proxy middleware written in Go, created to replace generic solutions like Nginx or Tengine for higher development efficiency, customizability, and lower maintenance cost.

The system follows a "one‑core‑multiple‑routes" design: a stable core server handles basic functions while optional side‑modules provide extended features, allowing selective disabling of modules during overload.

The core server uses a master goroutine with multiple worker goroutines, creating a new goroutine for each incoming HTTP request, similar to Nginx’s master‑worker model.

Request processing is divided into eleven phases, with six key callback registration points (Accept, BEFORE_CLUSTER, AFTER_CLUSTER, BACKEND_RESPONSE, REQUEST_FINISH, FINISH) where handlers can be attached, enabling flexible extension.

Modules group related handlers and can be dynamically enabled or disabled, supporting service degradation by shutting down non‑essential modules when load spikes.

Configuration hot‑reload is achieved via pointer switching: a new configuration is loaded into a separate memory region and atomically swapped, relying on Go’s garbage collector to keep the old configuration alive until all in‑flight requests finish.

Graceful restart options evaluated include fork‑process, pointer switching, reuse‑port, and health‑check + supervisord; the latter is currently used in production.

GC performance is critical; optimizations such as object merging, stack allocation, sync.Pool, cgo, memory pools, and upgrading the Go runtime (e.g., from 1.4.2 to 1.7) significantly reduce pause times.

Service splitting reduces request residence time in Pitcher, mitigating GC pressure during long‑running backend calls.

Transport choices include TCP with protobuf for structured messages, UDP for high‑throughput low‑latency internal communication, and Unix Domain Sockets for fast intra‑machine IPC.

Service degradation mechanisms cover global traffic throttling, business‑level quota isolation, and fault‑aware scaling, using either timer‑based or request‑count‑based detection to avoid false positives.

Rapid scaling is supported via a SaltStack‑driven one‑click expansion, achieving node addition within ten seconds.

Typical production issues such as SYN retransmission during backend restarts are addressed by increasing the accept queue size and implementing a slow‑start mechanism that gradually ramps up traffic to newly restarted services.