This article outlines a systematic approach to evaluating test automation effectiveness by detailing twelve quantifiable metrics—from bug detection rates and execution stability to platform usage and ROI—while emphasizing the critical mindset of automation champions for successful adoption.
This article presents a ready‑to‑use ELK‑based solution for private‑deployment environments, detailing fast one‑click deployment, log and metric collection with Filebeat and Metricbeat, centralized viewing in Kibana, alerting via ElastAlert, and extensible monitoring dashboards.
This article explains the evolution, responsibilities, and evaluation methods of the product‑effect assessment team at Amap, covering offline testing, AB experiments, metric analysis, automated scoring models, and the tools that support a comprehensive product‑performance framework.
This guide explains why monitoring is essential for the entire operations lifecycle, outlines key monitoring objectives, describes practical methods and workflows, reviews a range of open‑source tools (including Zabbix, MRTG, Ganglia, Nagios, Smokeping, OpenTSDB), and details metric categories such as hardware, system, application, network, log, security, API, performance and business monitoring.
This article examines the frequent pitfalls of recommendation systems—misleading metrics, over‑optimizing precision, data leakage, feature inconsistencies, and distribution bias—that cause offline AUC improvements to translate into lower online CTR and CPM, and offers practical mitigation strategies.
This comprehensive 2020 front‑end performance checklist guides teams through planning, metric selection, environment setup, static asset optimization, build and transport strategies, testing, monitoring, and realistic goal setting, offering tools, budgets, and cultural practices to achieve faster, more responsive web experiences.
Designers often overlook data analysis, yet mastering it helps identify problems, set measurable goals, generate hypotheses, collect and interpret metrics, and draw actionable conclusions, ultimately guiding product decisions and improving conversion rates across various scenarios such as page performance, financial calculations, and insurance plan selection.
This article walks through constructing a comprehensive MySQL monitoring solution using Prometheus and Grafana, covering exporter installation, configuration, key performance metrics, alert rule definitions, and practical tips for visualizing and troubleshooting MySQL instances in production environments.
This article explores how large wealth‑management platforms can model product recommendation as a mapping between customers and financial products, defines various evaluation goals such as transaction volume, revenue and user satisfaction, and outlines a systematic A/B‑testing workflow for comparing and optimizing recommendation algorithms.
This article walks through setting up a production‑grade MySQL monitoring solution using Prometheus and Grafana, covering exporter installation, MySQL user configuration, systemd service setup, Prometheus job definition, key MySQL performance metrics, and basic alerting rules.
This tutorial walks through Micrometer's metric types, how to register them with MeterRegistry, apply tags and naming conventions, and integrate the framework into Spring Boot applications with Actuator, Prometheus scraping, and Grafana visualization for comprehensive backend monitoring.
This article introduces the core concepts and architecture of the open‑source Prometheus monitoring system, explains its data model and metric types, and provides a step‑by‑step guide to install a Prometheus server, collect host metrics with Node Exporter, and visualize them using Grafana.
This article explains how to choose and combine appropriate evaluation metrics for recommendation systems by considering the specific scenario, business model, offline versus online testing, ecosystem balance, and user behavior, providing practical methods and a concise summary of common metric types.
This article outlines a DevOps‑inspired metric system for front‑end projects, describing the five development stages—design, development, personal verification, version verification, and online verification—and proposing concrete, automatable indicators to help leaders assess and improve delivery quality and speed.
The article reveals that JMeter often reports inflated throughput because it includes client‑side processing time—such as regex extraction and post‑processors—in its calculations, leading to misleading performance metrics unless corrected with micro‑benchmark adjustments.
The article explains the nature of raw data, defines metrics as data combined with business scenarios, introduces five‑question criteria for good indicators, and compares two popular metric‑system frameworks—the Pirate Metrics (AARRR) model and the First‑Key‑Metric method, illustrating their application in business decision‑making.
This article provides a comprehensive overview of Prometheus, covering its open‑source monitoring architecture, multi‑dimensional data model, query language, storage mechanisms, service discovery, alerting workflow with Alertmanager, and visualization using Grafana, all illustrated with key diagrams and configuration examples.
This article introduces OpenTSDB, a highly scalable time‑series monitoring system built on HBase, explains its architecture, demonstrates how it solves common monitoring challenges, and shows practical usage examples including data modeling, collector integration, and real‑world deployment insights.
This tutorial walks you through using Spring Boot Actuator to monitor microservice applications, covering quick setup, essential endpoints such as health, metrics, loggers, and shutdown, customizing health indicators, dynamically changing log levels at runtime, and securing actuator endpoints with Spring Security.
This article explains how to gather build metrics from Jenkins pipelines, store them in InfluxDB deployed on Kubernetes, and display the data using Grafana dashboards, providing a practical DevOps solution for continuous delivery measurement and feedback.
This article provides a comprehensive overview of using Prometheus and Grafana for metric collection, storage, querying with PromQL, visualization, and alerting, including exporter integration, metric types, high‑availability setups, and practical examples for modern microservice architectures.
This article introduces Prometheus and Grafana as a powerful monitoring stack, explains their architecture, metric collection, storage options, query language, integration with Grafana for dashboards and alerts, and shares practical deployment patterns and high‑availability solutions.
This guide introduces the open‑source GitStats tool, explains how to install its Python and Gnuplot dependencies on Linux, demonstrates step‑by‑step commands to generate a comprehensive HTML report for a Git repository, and interprets the resulting statistics on commits, contributors, file types, and tags.
This article examines common mistakes in measuring R&D efficiency, outlines seven guiding principles, presents case studies from leading tech companies, and shares practical steps and lessons for implementing a comprehensive R&D metrics system.
This article explains why many system alerts are poorly designed, describes the true purpose of alerts as actionable notifications, distinguishes business rule monitoring from reliability monitoring, and presents practical metrics, strategies, and simple anomaly‑detection algorithms to create high‑quality, actionable alerts for reliable operations.
This article explains the background, source‑code analysis, and practical implementation of Flink's LatencyMarker feature for measuring end‑to‑end job latency, including metric exposure, configuration options, and code snippets illustrating how latency markers are emitted and processed within the streaming pipeline.
The talk explains how Elastic Stack can be used to achieve comprehensive observability for cloud‑native applications through a four‑step methodology—health checks, metrics, logging, and tracing—detailing the challenges, implementation details, and best practices for monitoring and debugging modern microservice systems.
This guide explains how to add Spring Boot Actuator to a project, configure security, enable and customize monitoring endpoints, and use the built‑in REST APIs such as health, info, beans, metrics, and shutdown for production‑grade application observability.
This article compares Meituan’s in‑house EMonitor with the open‑source CAT platform, outlines their core monitoring models, sampling pipelines, custom metrics and integration capabilities, and traces the evolution of monitoring stages from log‑based to intelligent root‑cause analysis.
Gaode’s continuous delivery journey transformed its multi‑stage release process into a standardized, automated pipeline that delivers backend services within an hour, cuts manual effort, boosts release efficiency by up to 90%, and embeds early quality checks and data‑driven insights for faster, higher‑quality business value.
The article details how Yu'ebao, a financial platform with over five hundred million users and assets exceeding one trillion yuan, designs and operates a comprehensive service‑governance framework—including metrics collection, bi‑weekly governance committee reviews, dynamic and static call‑chain analysis, and layered service architecture—to balance security, compliance, and rapid service delivery.
This article explains the importance of code review for improving software quality, outlines the benefits of using Gerrit as a CR tool, describes the detailed workflow and promotion strategies, and discusses future extensions such as CI integration, issue tracking, and metric‑driven improvements.
This article provides a comprehensive guide to Elasticsearch’s architecture, node roles, data organization, and the most important performance metrics—including search, indexing, memory, JVM garbage collection, host‑level system metrics, cluster health, and resource saturation—offering practical advice on monitoring and tuning the cluster for reliability and efficiency.
The article argues that diversity should be treated as a tool rather than a final objective in recommendation systems, explains why it is hard to quantify, discusses appropriate metrics such as user feedback and engagement, and presents practical strategies—including expert rules, richer recall pipelines, and list‑wise modeling—to improve diversity while optimizing true business goals.
On September 18, Kubernetes v1.16 was released with 31 enhancements—including GA for custom resources, a revamped metrics system, volume resizing, Windows workload improvements, kubeadm alpha support, CSI integration, and the new Endpoint Slices feature that dramatically improves scalability and network topology handling.
This article explains the three main monitoring approaches—log collection, distributed tracing, and metric gathering—in microservice architectures, outlines the layered monitoring model, lists key system, application, and user metrics, and reviews popular open‑source time‑series monitoring tools such as Prometheus, OpenTSDB, and InfluxDB.
Prometheus, an open‑source monitoring and alerting toolkit originally developed by SoundCloud and now a CNCF project, offers multidimensional data models, flexible queries, pull‑based data collection, various metric types (counter, gauge, summary, histogram), local and remote storage, service discovery, and integrates with Grafana for visualization.
This article reviews Ant Financial's cloud‑native observability journey, covering its origins, the three pillars of tracing, metrics and logging, community projects like OpenTelemetry, practical implementations, sampling strategies, and future directions for unified microservice, mesh, and serverless monitoring.
This article explains how to correctly use Service Level Agreements (SLA) to define performance testing goals, align business and service metrics, set thresholds in the PTS platform, and automate test termination, thereby improving testing efficiency and reliability in cloud environments.
This roundup covers major Kubernetes ecosystem news, including the CNCF archiving rkt, GitHub Actions support for Azure CI/CD, Intel's GPU plugin, security CVE details, upstream feature enhancements, open‑source tool recommendations, and curated reading on policy enforcement and virtual clusters.
This guide explains how to enable Spring Boot's default metrics via Actuator, view them through the /actuator/metrics endpoint, and enrich them with Micrometer MeterBinders for JVM, cache, Hibernate, and Tomcat, including code samples and logging output.
This article explains the fundamentals of Prometheus histogram metrics, illustrates why they are cumulative, shows how to drop unwanted buckets with relabeling, and demonstrates quantile calculations using the histogram_quantile function, providing practical examples and code snippets for effective monitoring.
This article explains why comprehensive, multi‑layer monitoring is essential for distributed systems, outlines environment, program, and business metrics, recommends practical tools such as Zabbix, open‑falcon, Prometheus and Grafana, and provides a step‑by‑step evolution plan and alerting strategy.
This article provides a comprehensive overview of Prometheus, an open‑source monitoring and alerting system, covering its origins, key features, architecture, core components, metric types, data collection methods, service discovery, storage options, and query capabilities.
This article explains how A/B testing serves as a vital measurement and optimization tool for internet products, covering metric definition, experiment management platforms, traffic splitting strategies, orthogonal and exclusive rules, and essential statistical concepts such as hypothesis testing, t‑test, z‑test, and p‑value analysis.
This article explains Prometheus metric naming conventions, label usage, metric types such as Counter, Gauge, Summary, and Histogram, and describes the structure of data samples, providing examples and best‑practice guidelines for defining and classifying metrics in monitoring systems.
The article records Google VP Michael Bachman's 2018 talk on engineering productivity, covering the origins of EP, Google’s rebranding of QA to an Engineering Productivity team, and a detailed Q&A on rollbacks, data models, metric visualisation, tool adoption, testing strategies, static scanning, and experimentation practices.
The article explains how Google introduced the Engineering Productivity (EP) team, its evolution from QA, the organizational model across product areas, and the various metrics used to assess developer productivity, providing insight into large‑scale software engineering management.
This case study details how a user product center team applied the APD agile methodology and stream development practices—such as visual value streams, explicit process rules, WIP limits, bug‑board visualization, workflow management, and an evolved TDD approach—to double throughput, cut delivery cycles by 60%, and reduce monthly bugs from 50 to 2, demonstrating measurable R&D efficiency gains.
Senior Director Wang Xiaoxiang of Qunar shares how a data‑centric, metrics‑driven DevOps approach transformed the company's engineering efficiency, detailing Qunar’s measurement model, indicator definitions, technology choices, results, and strategies for leveraging metrics to continuously improve the software delivery lifecycle.
This article explains how to build a comprehensive monitoring system for distributed applications by classifying monitoring functions, describing data quadrants, outlining core modules such as collection, processing, feature extraction, and visualization, and reviewing typical implementations for metrics, logs, tracing, alerting, and the key open‑source components involved.
This article walks through a complete data‑analysis workflow—scraping Zhihu dating‑preference answers, cleaning and filtering the data, deriving gender and activity metrics, designing a four‑step screening process, and finally ranking candidates with a custom like‑to‑comment index—to help a single programmer create a concise, high‑quality list of potential partners.
The article explains the importance of bug analysis for improving product quality, outlines the information that can be analyzed, describes common bug resolution categories, discusses bug severity levels, and shows how various stakeholders can use bug metrics to drive process optimization and predict future issues.
This article explains why most alerts are poorly designed, defines actionable alerts, outlines monitoring objectives, discusses metric selection, and presents simple yet powerful algorithms for anomaly detection to improve system reliability and operational efficiency.
This article presents nine practical logging best‑practice recommendations—from understanding human and machine audiences and standardizing log formats to leveraging metrics, proper alerting, severity levels, contextual information, and advanced framework features—helping operations teams improve system performance and troubleshooting efficiency.
This article presents a comprehensive approach to testing, automating, and monitoring AI prediction models in a high‑traffic environment, covering background, challenges, evaluation metrics, data sampling methods, automated test scripts, and online monitoring to ensure model accuracy, performance, and reliability.
To meet Xianyu’s rapid‑release targets, the team built a CI‑integrated automation suite that gathers Flutter FPS, CPU, memory and network data via extended Android tools, drives mixed Flutter/Native UI tests with Appium and integration‑test, and generates lane‑by‑lane performance reports that quickly flag regressions across dozens of feature branches.
This article explains how to configure Spring Boot Actuator to export metrics to InfluxDB and Prometheus, covering Docker setup, Micrometer dependencies, application properties, sample controller code, test data generation, and visualisation with Grafana.
This article defines R&D efficiency as the ability to continuously deliver value quickly, explains the problem of efficiency silos, introduces five comprehensive measurement groups, showcases a defect‑trend chart example, and presents the practical 2‑1‑1 target framework for improving development speed and quality.
This article explains how to approach monitoring for a newly introduced system by focusing on white‑box metric collection, distinguishing basic and business metrics, outlining common collection methods, and detailing Google SRE's four golden indicators—error, latency, traffic, and saturation—to guide effective observability.
This article explains the differences between white‑box and black‑box monitoring, outlines common data‑collection techniques for both basic and business metrics, and details Google SRE’s four golden indicators—error, latency, traffic, and saturation—to help engineers design effective monitoring solutions.
The article presents seven tactical principles for agile coaching, explains their system‑dynamic foundations, and offers practical guidance on building leadership groups, clarifying roles, aligning management, measuring improvement, strengthening execution, optimizing communication, and fostering deliberate practice to drive sustainable agile transformation.
This article details Meituan's systematic approach to reducing iOS app cold‑start time, covering the definition of startup phases, identification of performance bottlenecks, phased launch, self‑registration of startup tasks, code slimming, profiling tools, and continuous monitoring to achieve smoother user experience.
The article analyzes why many organizations stall on the DevOps adoption curve, identifies the root causes of the "gap", proposes two critical thinking areas—measurement and structural change—and outlines practical steps and future directions for sustainable DevOps transformation.
This article explains agile development’s core business goals of faster value delivery and effective learning, defines agile as early value delivery and adaptive response, and presents a comprehensive five‑group metric system—including release capability, demand response cycle, throughput, built‑in quality, and external delivery quality—to assess and improve continuous delivery performance.
This article outlines common challenges managers face with performance testing, explains when testing should be introduced, describes its process, types, metrics, and quality checks, and provides practical guidance for analyzing requirements and ensuring effective, repeatable performance test execution.
This talk explains how visualizing end‑to‑end value flow, focusing on demand bottlenecks, defining proper metrics, and aligning features with key business goals can dramatically speed up product innovation cycles and deliver higher‑value software.
This article outlines Baidu's comprehensive product evaluation framework, describing its multi‑layer assessment system, the combination of subjective and objective metrics, and a suite of common evaluation methods such as indicator analysis, AB testing, user feedback, behavior analysis, big‑data profiling, and competitor comparison.
This article outlines the challenges of algorithm testing in real‑world engineering, proposes a step‑by‑step testing framework—from understanding business context and verifying data exchanges to evaluating performance metrics and iterating improvements—while offering practical advice and examples.
This article introduces the recently open‑sourced SOFA middleware components—including SOFATracer, SOFALookout, SOFA‑Boot, SOFA‑RPC (Node and Java), Eggjs integration, and JarsLink—explaining their features, usage scenarios, and providing GitHub links for developers to adopt these financial‑grade cloud‑native solutions.
This article describes Qunar's approach to quantifying, automatically detecting, and optimizing performance issues in React Native mobile apps by defining measurable metrics, real‑time data sampling, automated analysis rules, and visual feedback to enable proactive APM during development and testing.
This article summarizes Wang Qing’s DevOps talk, outlining the current state of DevOps, showcasing concrete case studies from ING, CapitalOne and a major Chinese bank, and providing a step‑by‑step guide to building a practical DevOps toolchain for traditional enterprises.
By applying a PDCA‑based quality‑operation framework that aligns QA KPIs, drills defect data across dimensions, automates test‑gate checks, and drives continuous improvement actions, Meituan Dianping’s Intelligent Payment team reduced severe defect ratios, met defined metric targets, and boosted iteration efficiency while supporting rapid business growth.
This article explains how to assess customer acquisition by defining key metrics such as new users, retained users, ROI, and LTV, detailing their calculations, assumptions, and practical steps for product teams to measure and improve acquisition efficiency throughout a product’s lifecycle.
This guide explains how to collect and visualize Kubernetes performance metrics—CPU, memory, network, and filesystem—by deploying Heapster with a StatsD sink that forwards data to Splunk's Metrics Store, then using Splunk SPL commands and dashboards for real‑time monitoring.
This article presents a comprehensive 2018 frontend performance checklist that guides teams from establishing a performance culture and selecting metrics to optimizing build processes, static assets, HTTP/2, security, and monitoring, while offering quick‑win actions for immediate impact.
The author shares personal insights from a recent discussion with Microsoft Seattle on DevOps transformation, highlighting the lack of a standard definition, various performance metrics, the role of story points, and considerations for UI automation testing within agile teams.
The article outlines Baidu's comprehensive continuous integration (CI) capability model, detailing its guiding principles, standardized metrics, tiered evaluation system, and practical implementation steps to improve software quality and development efficiency across numerous product lines.
This article outlines the real‑world demand for MySQL performance management, critiques traditional monitoring approaches, and presents a product‑focused design that emphasizes lightweight data collection, incremental recording, metric‑driven drill‑down, and comprehensive analysis techniques such as AS/PS and QRTi.
This article explains how to anticipate and monitor Elasticsearch issues such as node unavailability, OOM errors, and long garbage‑collection pauses by tracking key performance metrics across query, indexing, memory, and system levels, helping prevent service disruptions.
Pandora.js, the Alibaba Midway team's first open-source project, consolidates years of production-grade Node.js operations by offering three core capabilities—manageable application and process control, comprehensive metrics, and Open-Tracing-based request tracing—exposed via RESTful APIs and logs for easy integration.
Metrics act as health checks for applications, enabling developers to monitor performance, track changes, and assess stability; this article explains Pandora.js’s metric naming conventions, types like Gauge, Counter, Histogram, and Meter, and provides practical Node.js code examples for implementing these measurements.
This article explains how to assess unsupervised clustering algorithms by describing realistic user‑watching scenarios, outlining common cluster and algorithm types, presenting five key evaluation criteria, and introducing practical metrics such as RMSSTD, R‑Square, and the improved Hubert‑Gamma statistic.
The USE Method provides a concise, structured framework for quickly locating resource bottlenecks and errors in complex systems by examining utilization, saturation, and error metrics across hardware, software, and cloud environments, enabling practitioners to prioritize and resolve performance issues efficiently.
This article explains Kafka end‑to‑end auditing, compares three products (Chaperone, Confluent Control Center, Kafka Monitor), describes timestamp and index embedding techniques, and outlines their architectures, metrics, and implementation details for detecting data loss, duplication, and latency.
This article explains essential iOS performance metrics, compares Xcode Instruments, third‑party SDKs, and custom instrumentation, and provides code examples for measuring startup time, memory, CPU, frame rate, network latency, UI blocking, and power consumption.
This article describes LinkedIn's network‑level MySQL query analyzer, explaining why it was needed, its three‑component architecture (agent, central server, UI), fingerprinting and hash‑map techniques, performance benchmarks, metric collection, security considerations, and the benefits it brings to database operations.
This guide explains how to choose between push and pull monitoring models, introduces Prometheus architecture and metric syntax, shows Node.js client integration with code examples, and covers Alertmanager features and Grafana visualization for effective application monitoring.
This article outlines the fundamentals, objectives, methods, core processes, tool selection, metrics, alerting, and interview tips for constructing a robust monitoring ecosystem, emphasizing Zabbix while comparing various open‑source solutions.
This article recounts how the Taobao Live team applied agile practices over three months, tracking capacity, lead time, quality, and predictability, and demonstrates measurable improvements in delivery speed, defect reduction, and business metric alignment through focused goals, rapid hypothesis testing, and continuous delivery.
The article explores how artificial intelligence can transform DevOps by introducing predictive metrics, machine‑learning models, and intelligent automation to improve code review, defect forecasting, and overall software lifecycle management, while discussing current research, challenges, and future directions.
This article explains the fundamentals of monitoring, distinguishes traditional OPS from SRE perspectives, defines monitoring objects and metrics, introduces quantitative thinking with SLI/SLO, and presents the USE method with a MySQL example to help engineers detect and prevent failures efficiently.
This article describes the background, architecture, encountered performance bottlenecks, and a three‑level caching strategy (Redis‑based) implemented in Python to accelerate a measurement platform that aggregates bug, issue, and code metrics for a rapidly growing development team.
This article presents a comprehensive server‑side monitoring solution covering functional and performance requirements, monitoring objects, design choices between self‑monitoring and centralized reporting, system architecture, API definitions, key challenges such as key collisions, data formats, storage options, and operational considerations.
Metrics‑Driven Development (MDD) extends test‑driven principles by embedding real‑time monitoring into design, enabling rapid, precise, and granular software iterations, improving early problem detection, decision support, and aligning development with DevOps culture.
This article explains Ctrip's flight ticket tracking framework (UBT), detailing client‑side and server‑side event collection methods, the purpose and trade‑offs of each tracking type, metric definitions, data association challenges, common pitfalls, and best practices for reliable data‑driven analysis.
This guide explains how to integrate Spring Boot Actuator into a traditional Spring application by manually adding required Maven dependencies, importing core Actuator configuration classes, and defining beans that expose the /health and /metrics monitoring endpoints.
This article outlines mobile performance testing methods—including startup time, page response time, basic metrics, Monkey testing, and image‑recognition techniques—reviews third‑party tools, and introduces the in‑house ETest solution with its features and future development plans.
This article presents a detailed overview of mobile app performance testing, covering the importance of performance metrics for video applications, the prioritized dimensions such as smoothness, crash, memory and power consumption, and practical methods and tools for acquiring Android and iOS system metrics.
This article explains what OpenStack is, outlines its core modules, and details the most important monitoring metrics for Nova, Neutron, Keystone, hypervisors, tenants, and RabbitMQ, helping engineers build a robust, scalable OpenStack monitoring solution.
