Visualization in Operations: Service Delivery and Service Metrics

First Part: Visualized Service Delivery

There is no better word than “visualization” to capture the essence of operations, and visualization should be divided into two aspects: visualized service delivery and visualized service metrics.

Early operations began with ITIL, when many did not even know what operations meant; ITIL provided the best practice for IT services and started the exploration of CMDB, service desk, incident, change, availability, and capacity management, building corresponding platforms. However, when faced with large‑scale operations, the heavy focus on processes and standards made it difficult to improve agility and precision, and the boundaries of a complete IT service remained unclear.

ITIL introduced the concept of an IT service: providing an efficient, consistent, transparent, user‑oriented service while hiding all implementation details. The challenge is to package individual tasks into a complete IT operations service, which became the focus of automation. Traditional operations are siloed—network, data‑center, servers, application deployment—each handled by separate teams. Agile and lean operations require an integrated platform to orchestrate these tasks, turning delivery functions into a delivery service.

DevOps’ “automate everything” offers an answer: a user‑oriented, agile continuous delivery pipeline. This pipeline can be split into two scenarios: continuous delivery of infrastructure and continuous delivery of applications, analogous to IaaS and PaaS.

Continuous delivery of infrastructure is well supported by public‑cloud IaaS platforms using software‑defined compute, storage, and networking. In private environments, many customers adopt virtual machines or private‑cloud solutions, and lightweight virtualization such as Docker enables image‑level rapid application delivery.

Continuous delivery of software starts the moment code is generated and proceeds through compilation, testing, gray‑environment acceptance, and production deployment, aiming for full automation. Package‑based CI is straightforward with open‑source tools like Jenkins or Go, but configuration management of those packages is a critical factor. Large‑scale deployments therefore rely on dedicated continuous‑deployment platforms that seamlessly integrate with CI tools.

After solving package delivery, the goal shifts to delivering whole applications—from front‑end access to back‑end storage. Two approaches exist: PaaS platforms and visual deployment services based on application architecture. PaaS offers unified APIs for underlying services such as databases, storage, and caches; Cloud Foundry is a classic example, and Alibaba UC provides a similar platform.

In practice, few companies can encapsulate services like MySQL, Memcached, or FastDFS as public resources, which imposes constraints on development. A lighter, unconstrained automation approach treats each component of an application architecture as an object (with attributes and state) and orchestrates them like a visual IDE programming environment.

In summary, the ultimate goal of operations automation is visualization: complex workflows must be expressed visually so that everyone can understand, execute, and obtain consistent results.

Second Part: Visualized Service Metrics

“Except for God, everyone must speak with data.” This principle underlies data‑driven operations, as described in a previous article on the topic.

Currently, an end‑to‑end data analysis system is being built, standardizing data into layered categories—from infrastructure, upper‑layer components, application services, interfaces, to the user side—based on application topology, collecting various metrics into a unified analysis platform.

The system implementation of this layered data framework is shown below.

Once a standardized data collection, analysis, and visualization pipeline is established, it can be replicated across other applications; data gathering, analysis, display, and alerting become standardized, supporting fault localization, service optimization, architectural improvement, and operations planning.

To integrate these data per application, both a static view derived from configuration files and a dynamic view generated from interface calls are used. The dynamic view relies on a name‑service center that schedules calls and colors them, producing a runtime access graph.

While dynamic views quickly locate large‑scale failures, they are less effective for single‑user issues. Distributed tracing (e.g., Zipkin, Google Dapper) addresses this gap. By implementing a unified service scheduling framework and name‑service center, business code can report colored trace data without intrusion, enabling rapid pinpointing of individual problems.

Data visualization is crucial both at the overall level and for specific business flows. First, dashboards reflect one’s understanding of operations; second, shared visual data aligns team interpretations; finally, consistent visual data drives DevOps, revealing the core value of data.

Therefore, the ability to visualize represents the capability of operations—higher visualization equals higher operational competence, prompting the question of which operational services have been visualized and measured.