Traffic Management System Based on Service Isolation

This article describes how to implement a traffic management system using service isolation to address challenges in full-link pressure testing, parallel feature testing, and online gray release.

With the development of business, we encountered common pain points in microservice advancement, such as difficulty in full-link pressure testing, inability to test different versions of the same service simultaneously in test environments, and challenges in gray traffic processing when multiple services need to be updated together.

Specifically, full-link pressure testing previously required building a complete new environment, which was time-consuming and costly. We wanted to isolate part of the traffic from online machines for pressure testing to save environment setup costs. For parallel feature testing, fast product iteration often requires testing different versions of the same service simultaneously, causing resource contention in test environments. For online gray release, we needed to isolate part of the online traffic for gray verification to reduce risks when new versions go online.

To solve these problems, we abstracted a logical concept called "link" - the path from a business request entering the system to completion, including all microservices and data nodes it passes through. We divided links into two types: "strong links" (disallow traffic to other links) and "weak links" (allow traffic to be redirected to other links).

Based on Consul's tag functionality, we implemented link functionality by setting different tags for different nodes. We developed a middleware system to support all desired calling logic and a management console (Director) for convenient governance of all nodes and links.

We divided the online environment into several links: basic link for daily business processing, gray link for new feature verification, pressure test link for isolated testing, and parallel feature test link for independent testing by different teams. The link isolation system has become a powerful tool in our internal service governance, efficiently solving many difficult scenarios and has been widely used across all R&D departments.