Observability and Quality Assurance: Strategies for Test Teams

Background

Quality teams are actively building application monitoring to ensure online service stability and are exploring how test teams can contribute to observability under the growing observability paradigm.

Understanding Observability

What is Observability

Observability, originally from control theory, refers to the degree to which a system’s internal state can be inferred from its external outputs; in software it means using metrics, logs, and traces to build a complete view for fault diagnosis and rapid recovery.

Observability vs. Monitoring

Monitoring focuses on collecting and analyzing specific metrics, while observability infers internal states and supports data‑driven decision making; both aim to improve system control and fault handling.

Quality Assurance Goals

Objectives include comprehensive system monitoring, proactive health detection, rapid anomaly localization before user impact, and providing real‑time and historical comparative data to support technical decisions.

Quality Assurance Approach

The approach builds a monitoring foundation and extends it with data observability capabilities, covering resource‑level, service stability, business‑function, business‑data, and log‑clustering monitoring.

Test‑Team Focus

Test teams should prioritize business‑function monitoring (e.g., read/write interface checks), business‑data validation (core data volume, correctness, trend thresholds), and log‑clustering monitoring with short‑term threshold alerts and long‑term availability calculations such as Application Availability = (Total Traffic - Error Traffic) / Total Traffic .

Observability Dimensions

Module‑level observability provides trend analysis and alerts for individual components, while system‑level observability aggregates logs across modules, enables linked alerts, fault localization, and cross‑system data analysis.

Presentation

Plans include integrating alerts into monitoring dashboards and offering multi‑channel notification services (email, messaging, voice) for timely incident response.