This article explains how Tubi monitors the quality of its recommendation system by identifying potential failure points, tracking key data streams such as model input, final recommendation output, and training data, and designing a scalable, real‑time monitoring solution with clear protocols and extensible metrics.
This multi‑chapter guide provides in‑depth, hands‑on instruction for configuring and optimizing all Prometheus components, exploring Kubernetes monitoring, source‑code analysis, custom exporter development, high‑availability setups, service discovery, resource‑efficient scraping, and integrating Thanos for long‑term storage.
This article analyzes why a single Prometheus instance repeatedly runs out of memory and crashes, explains the underlying storage mechanisms, and presents practical solutions such as metric reduction, retention tuning, federation architecture, and remote storage integration to improve stability and scalability.
This article introduces ten indispensable tools for operations engineers, detailing each tool's functionality, typical use cases, key advantages, and real‑world examples, helping professionals streamline automation, monitoring, configuration, and deployment tasks and improve overall system reliability.
This article reviews the ten most important open‑source monitoring and observability tools for modern DevOps teams in 2024, outlining each tool's key features, advantages, disadvantages, and how they compare for performance, scalability, cost and ease of use.
This article explains how AIOps leverages AI, machine learning, and dynamic threshold techniques to handle massive, multimodal monitoring data, improve anomaly detection, and enhance IT operation reliability through metric classification, baseline prediction, and automated fault remediation.
A port alarm revealed a missing Java process, which was later traced to an OOM kill triggered by vim loading a 37 GB nginx log into an 8 GB container, illustrating how editor behavior and Linux's OOM killer can unexpectedly terminate critical services.
This article examines how database middleware tackles the demanding needs of large‑scale internet services by providing centralized connection‑pool management, transparent read‑write splitting, diverse load‑balancing algorithms, sharding support, automatic failover, security controls, comprehensive monitoring, and flexible backup‑recovery mechanisms.
This guide walks you through a rapid 60‑second Linux performance analysis, detailing key commands like uptime, dmesg, vmstat, mpstat, pidstat, iostat, free, sar and top, and explains how to interpret their most important metrics for troubleshooting system issues.
This article recounts how a production database deadlock was uncovered during testing, explains the use of the EasyBI monitoring tool to collect and visualize error and claim statistics, and shares the step‑by‑step configuration, analysis, and lessons learned for preventing similar issues in future systems.
Facing rapid business expansion, Huolala’s quality assurance team tackled redundant code, high regression costs, and lack of coverage metrics by designing an intelligent automated testing framework that analyzes effective code, provides smart test case recommendations, visualizes progress, and integrates monitoring, resulting in significant coverage improvements and efficiency gains across services.
This article compiles a comprehensive set of interview-style questions covering Linux file handling, CPU metrics, link types, TCP handshakes, process vs thread, TCP/UDP differences, DDoS mitigation, Keepalived operation, TIME_WAIT optimization, container networking, Kubernetes components, deployment strategies, monitoring concepts, Prometheus architecture, and common web‑site operational issues.
This article examines the various hardware and OS resources that affect Linux performance—including CPU, memory, disk I/O, and network bandwidth—then details practical optimization techniques and essential monitoring tools such as vmstat, iostat, free, sar, and netstat to diagnose and improve system efficiency.
This guide walks through setting up a Dameng data‑guard service using a primary, standby, and monitor server, covering data preparation, configuration of dm.ini, dmmal.ini, dmarch.ini, dmwatcher.ini, starting services, OGUID setup, mode switching, and monitoring to achieve high‑availability replication.
This tutorial demonstrates how to monitor Nginx using Prometheus by enabling the stub_status module, installing and running the Nginx Exporter, configuring Prometheus scrape jobs, and optionally visualizing the collected metrics in Grafana, providing a complete end‑to‑end monitoring solution.
This guide explains the background, architecture, common CRDs, installation steps, and practical usage of Prometheus‑Operator on Kubernetes, including how to create Deployments, Services, ServiceMonitors, and visualize collected metrics with Prometheus UI.
This article provides a comprehensive comparison between Zabbix and Prometheus, covering their functional architecture, metric collection methods, data storage, query capabilities, visualization options, and alerting mechanisms, helping readers decide which monitoring system best fits their enterprise needs.
An extensive overview of essential DevOps tools for 2024, covering categories such as version control, CI/CD, container orchestration, configuration management, infrastructure as code, monitoring, collaboration, artifact repositories, testing, security, deployment automation, serverless platforms, and database management to guide effective tool selection.
This guide walks ops teams through real‑world incident scenarios, from quick symptom identification and emergency recovery to improving monitoring, crafting concise emergency plans, and leveraging automation for smarter fault handling, helping organizations restore services faster and reduce downtime.
This article explains why message queues can become backlogged, identifies producer and consumer causes, and provides practical strategies—including adding consumers, increasing queue capacity, optimizing consumption logic, implementing failure handling, and rapid remediation steps—to quickly resolve backlog in production environments.
This article provides a curated set of Bash scripts covering MySQL replication monitoring, directory change detection, bulk user creation, website health checks, remote command execution, LNMP stack deployment, server resource reporting, high‑resource process identification, and automated deployment of Java and PHP projects, offering practical automation tools for system administrators.
This article explains the design and implementation of a robust payment‑channel monitoring system that uses Redis for time‑series storage, Prometheus for metrics, and custom algorithms to achieve fast fault detection, low false‑alarm rates, and automatic channel switching.
This article explains how to use Prometheus to collect, define, and alert on business‑level metrics in a Kubernetes environment, covering observability concepts, metric types, Go code examples, and scraping configurations for reliable monitoring.
This guide explains monitoring fundamentals, clears common misconceptions, compares black‑box and white‑box approaches, outlines key metrics such as latency, traffic, errors and saturation, and provides a deep dive into Prometheus architecture, data model, query language, and practical examples for CPU, memory, and disk monitoring.
At the 21st GOPS Global Operations Conference, New Oriental's senior operations manager Qi Chen detailed the demand, technical, and focus pressures that drove a phased, full‑process observability standardization, leveraging OpenTelemetry, Telegraf, Loki and CMDB tagging to achieve cost reduction and higher stability.
This article compiles a set of practical Bash scripts that cover MySQL replication monitoring, directory change detection with real‑time sync, bulk user creation, website health checks, remote command execution, one‑click LNMP deployment, resource usage reporting, high‑CPU process identification, and automated Java/Tomcat and PHP project deployments.
This article explains how to build a log platform by integrating native Kibana with ClickHouse using an open‑source proxy (CKibana), covering migration motivations, architecture, configuration steps, advanced features like sampling and caching, and the resulting cost and stability benefits.
This article systematically outlines common front‑end performance optimization techniques, explains key web performance metrics such as Speed Index, FCP, CLS, LCP and TBT, and provides practical strategies for resource compression, network and code optimization, as well as monitoring and measurement best practices.
This article outlines a comprehensive, step‑by‑step approach for SRE incident commanders, covering fault perception, grading, team organization, remediation tactics, transparent communication, and post‑mortem practices to efficiently resolve service disruptions.
This tutorial walks you through installing Helm, using Helm commands to find and install the Prometheus Helm chart, exposing the Prometheus service, and accessing the monitoring UI on a Minikube Kubernetes cluster, complete with code snippets and screenshots.
To handle growing payment traffic and unreliable third‑party gateways, the team at Zhuanzhuan designed an automated payment‑channel management platform that uses a custom Redis‑based time‑series store, Prometheus monitoring, and a sliding‑window failure‑rate algorithm to detect, alert, and eventually auto‑switch faulty channels.
This guide shows how to set up a MySQL server and a mysqld_exporter using Docker Compose, configure Prometheus to scrape the exporter, and create alert rules for MySQL downtime and slow queries, providing a complete monitoring solution.
The article explains how combining traditional monitoring with modern observability, supported by data quality practices and unified workflows, enables more reliable, scalable, and insightful application performance management in agile and cloud‑native environments.
This article presents a detailed case study of how ZhaiZhai designed and implemented a unified monitoring platform—combining business services, middleware, and operations resources—by selecting Prometheus and M3DB, automating Grafana dashboards, creating a low‑noise alerting system, and achieving large‑scale observability with significant cost and efficiency gains.
This article describes how a QA team collaborated with developers to create an automated error‑log cleanup and monitoring system, detailing the background, offline follow‑up process, identified pain points, the design of a scheduled statistics solution, platform capabilities, observed benefits, and future improvement plans.
This article presents a detailed three‑step plan used by Zhezhuan to detect and eliminate zombie services, unused code methods, and obsolete component dependencies through monitoring, static analysis with Spoon, and Java‑agent based runtime tracing, achieving significant resource savings and improved code health.
This article compares two popular open‑source Kubernetes monitoring and alerting solutions—kube‑prometheus and Nightingale—detailing their features, deployment steps, advantages, drawbacks, and providing guidance on choosing or combining them based on specific operational needs.
The article outlines backend exception‑handling best practices, detailing business requirements such as memory‑safe multithreaded throws, clear separation of concerns, framework fallback strategies, simple macro‑based APIs, unified error‑code monitoring, rich debugging information, extensible type‑erased models, and appropriate handling of critical, recoverable, and checked exceptions across development and production environments.
This guide explains how to track Linux user activity and system events using native commands such as who, w, last, ps, ss, journalctl, and the auditd framework, providing step‑by‑step examples and advanced auditing techniques for security and compliance.
Guming’s Front‑End Data Center integrates monitoring, performance, logging, and analytics for web, mini‑programs, Flutter clients, and Node.js services, offering real‑time alerts, high availability, sampling, multi‑channel data pipelines, custom charting, and detailed CPU/GC profiling to streamline issue diagnosis and business insights.
This article explains what Redis hot keys are, the scenarios that generate them, their risks, and provides practical monitoring methods and mitigation strategies—including cache pre‑warming, distributed caching, rate limiting, and secondary caches—to keep production systems stable.
This guide explains how to install, configure, and run Prometheus Alertmanager with Docker, set up routing and receivers, integrate it with Prometheus alert rules, test alerts, customize email templates, and optimize notification settings for reliable monitoring and alerting.
The Weimob Technology Salon session on "Elasticsearch in Weimob's Practice" shares practical usage recommendations, monitoring setups with Prometheus and Grafana, field‑type guidance, and solutions to common operational challenges, offering developers actionable insights for high‑performance search deployments.
This article details how Zyb’s multi‑cloud database platform integrates diverse database types, a unified proxy layer, intelligent lifecycle management, automated task orchestration, monitoring, resource allocation, backup, and fault‑handling to achieve efficient, reliable, and secure database operations across cloud environments.
This article outlines how a large‑scale Kafka deployment of over a thousand machines across dozens of clusters was engineered for stability and efficiency through a custom Guardian controller that adds partition‑level throttling, automatic balancing, multi‑tenant isolation, cross‑IDC management, tiered storage, audit capabilities, and fully automated operational workflows.
This article details Bilibili's step‑by‑step transformation of a fragmented, high‑coupling live‑commerce system into a modular, platform‑centric architecture, covering product middle‑platform construction, unified standards, storage migration, monitoring with Prometheus/Grafana, and performance gains such as a three‑fold query speedup and a reduction of development cycles from 46 to 5 person‑days.
The article explains the importance of CI/CD observability, outlines common pipeline problems, introduces Grafana's GraCIe plugin built on OpenTelemetry, and discusses how enhanced visibility can improve reliability, decision‑making, and future standardization across CI/CD platforms.
This article explores how to build a robust engineering architecture by prioritizing product value, defining clear layered and DDD structures, selecting appropriate technologies, and establishing standards for exception, logging, monitoring, and team collaboration to achieve scalability, maintainability, reliability, security, and high performance.
Bilibili transformed its fast‑growing live‑streaming e‑commerce operation by constructing a modular platform that separates product, user, and application layers, introduces a unified product middle‑platform, standardized capabilities, real‑time attribute handling, and robust monitoring and governance, thereby reducing technical debt, improving stability, and preparing for hundred‑billion‑level GMV scaling.
This article explains how to use Filebeat together with Graylog to collect, ship, store, and analyze logs from multiple environments, covering tool introductions, configuration files, Docker deployment, Spring Boot integration, and practical search syntax for effective log monitoring.
This article presents Bilibili's big‑data metadata warehouse, covering its background, technology selection between data‑lake and data‑warehouse solutions, the architecture built on Prometheus, StarRocks, Flink and Routine Load, performance comparisons, diagnostic system design, and future development plans.
This guide walks through a full Kubernetes monitoring solution using cAdvisor, node_exporter, Prometheus, and Grafana, covering architecture, data collection, service discovery, deployment steps with DaemonSets, and detailed YAML configurations for a production‑ready observability stack.
This guide explains what Elastic Real User Monitoring (RUM) is, outlines its key benefits, and provides step‑by‑step instructions with code snippets for integrating the Elastic RUM JavaScript agent into a Hugo static site, including configuration parameters and how to view the collected data in Kibana.
2023 saw rapid evolution in the front‑end ecosystem, highlighted by major events, a controversial Gemini AI demo, SkyWalking‑based performance and error monitoring, innovative text‑overflow handling, CSS techniques that boost long‑list rendering by up to seven times, and an automatic, non‑intrusive skeleton‑screen generation solution.
This article reviews five leading log management solutions—Graylog, LogDNA, ELK Stack, Grafana Loki, and Splunk—detailing their core components, key features, and why they are valuable for monitoring, troubleshooting, and securing modern IT environments.
This article introduces the architecture and core functions of Automotive Home Push Platform, covering its development history, technical implementation, monitoring system, and future plans for intelligent message distribution.
This article outlines nine practical best‑practice steps for designing a low‑complexity, high‑efficiency microservices ecosystem, covering principles such as the Single Responsibility Principle, cross‑functional team organization, appropriate tooling, asynchronous communication, DevSecOps security, independent data stores, isolated deployment, orchestration, and effective monitoring, each illustrated with concrete examples.
This guide walks you through why GPT‑4 outperforms GPT‑3.5, shows step‑by‑step how to create a Zabbix expert consultant using the new GPTs feature, and explains advanced configuration, knowledge‑base feeding, testing, and future possibilities for AI‑enhanced monitoring.
This article introduces ten indispensable tools for operations engineers, detailing each tool's functionality, suitable scenarios, advantages, and real‑world examples, and includes practical code snippets to help automate, monitor, and manage infrastructure efficiently.
The article walks through a real‑world online supermarket project, showing how a simple monolithic system evolves into a fully‑featured microservice architecture, detailing each refactoring stage, the problems encountered, and the concrete solutions such as service extraction, database sharding, monitoring, tracing, gateways, service discovery, reliability patterns, testing, and service‑mesh adoption.
This article walks through the journey of transforming a simple online supermarket from a monolithic application to a fully fledged microservice architecture, highlighting the motivations, design decisions, component breakdown, operational challenges, monitoring, tracing, resilience patterns, testing strategies, and the role of service meshes.
The article outlines common logging problems and presents a comprehensive six‑step strategy—including clear logging standards, systematic standard management, a unified SDK, centralized log management systems, regular standard reviews, and lifecycle deprecation—to transform chaotic logs into a reliable tool that boosts development efficiency.
This article chronicles the year‑long evolution of the voice‑chat room system, detailing how product‑driven requirements forced successive redesigns of both the live‑streaming and RTC subsystems, the introduction of session‑and‑channel abstractions, migration of mic‑seat management to the backend, and the implementation of monitoring, testing, and deployment practices that keep the architecture stable and extensible.
This article explains why monitoring alert noise harms efficiency, presents metrics such as recall and accuracy, details rule‑based, blacklist/whitelist, ratio‑based, and intelligent noise‑reduction techniques, shares Java code examples, and shows measurable results after applying the governance process.
This article walks through the evolution of an online supermarket from a simple monolithic website to a fully decoupled microservice architecture, covering initial requirements, common pitfalls, service decomposition, database splitting, monitoring, tracing, logging, gateways, service discovery, circuit breaking, testing, frameworks, and service mesh, while illustrating each step with diagrams and practical advice.
Baidu’s Aladdin search service safeguards stability during massive traffic spikes—such as Gaokao, the Tokyo and Beijing Olympics—by mapping dependencies, deploying multi‑dimensional monitoring, adding scaling layers like multi‑region Redis, and establishing rapid‑response on‑call teams, achieving over 99.99 % uptime and near‑real‑time data updates.
This article details Qunar's comprehensive overhaul of its monitoring platform—introducing second‑level metrics, redesigning storage with VictoriaMetrics, optimizing client and server data collection, and building a root‑cause analysis tool—to dramatically reduce order‑related fault discovery time from minutes to under one minute.
This talk explains the three major observability challenges in Kubernetes, demonstrates how eBPF enables comprehensive, low‑overhead data collection across all stack layers, and outlines a practical workflow that combines architecture awareness, application‑level metrics, and fault‑tree analysis to achieve automated root‑cause diagnosis.
SCADA systems are essential industrial automation platforms that monitor, collect, and control data across sectors such as energy, manufacturing, and water treatment, using sensors, PLCs, and HMI software, and modern implementations integrate SQL databases and web technologies for enhanced efficiency.
The article explains what constitutes a big key in Redis, why they arise, the performance and blocking issues they cause, and provides multiple practical methods—including built‑in commands, SCAN, MEMORY USAGE, RDB analysis tools, and cloud services—to detect and mitigate big keys effectively.
This article explains what Redis hotkeys are, the performance and replication problems they cause, various techniques for detecting them—including client statistics, MONITOR, the HOTKEYS command, and TCP packet capture—and practical mitigation approaches such as sharding, multi‑level caching, and monitoring optimization.
The article examines a real‑world Kafka record error that surfaced after 8 am, outlines how to quickly locate and correct the issue by 10 am while minimizing impact, and presents comprehensive strategies for timeliness, data quality, and stability in real‑time data pipelines.
The article analyzes the importance of system stability during major sales promotions, presents data‑driven insights on traffic and revenue, identifies key challenges such as massive traffic, data volume, and complex workflows, and offers comprehensive operational, application, storage, and monitoring measures to guarantee reliable performance under extreme load.
This article details Huolala's user event tracking (埋点) system, covering its background, challenges, the construction of a four‑module management platform, backend SDK design, monitoring and quality assurance mechanisms, and future plans for service integration, data lineage, and governance optimization.
This article compares the open‑source monitoring solutions Prometheus and Zabbix, outlining their histories, architectures, data collection methods, scalability, storage models, configuration complexity, community activity, and suitability for different environments such as traditional servers versus cloud‑native container platforms.
The article outlines how game QA and third‑party providers can improve cooperation by aligning basic performance concepts such as TPS, QPS and concurrency, selecting appropriate rate‑limiting strategies, establishing precise monitoring and alerting, and preparing clear incident‑response and delivery standards.
This article details Bilibili's data quality assurance framework, covering its evolution across four data platform stages, the architecture of its quality data warehouse, core capabilities such as a complete assurance system, digital‑driven continuous optimization, and efficient incident handling, plus case studies, future plans, and a Q&A session.
The Industrial and Commercial Bank of China software development center created an SRE panoramic monitoring view system that unifies data channels, standardizes metrics, offers multi‑dimensional dashboards, and introduces an intelligent Ops Assistant, dramatically improving fault detection, response speed, and cross‑team operational efficiency.
This article explains how to design and deploy a complete monitoring solution using Prometheus, various exporters, Grafana, and Alertmanager on Docker and Kubernetes, covering installation, configuration, visualization, and best‑practice tips for reliable operations monitoring.
This article explains how Docker integrates with DevOps practices to enhance automation, streamline continuous integration and deployment, enable comprehensive container, application, and infrastructure monitoring, and centralize log collection and analysis, providing practical code examples for building, testing, deploying, and managing services efficiently.
This article shares a Java developer’s six‑year experience with Arthas, detailing how vmtool, hot‑swap, OGNL filtering and various Arthas commands can dramatically improve problem‑diagnosis, reduce log noise, and accelerate debugging in production environments.
This guide walks you through preparing a Kubernetes cluster, creating deployment manifests, configuring Grafana and Prometheus, and verifying the monitoring setup, including code snippets and step‑by‑step commands for a seamless installation on a lightweight cloud server.
The article explains how Corona leverages three time‑series databases—InfluxDB for storing pre‑aggregated user metrics and platform health data, ClickHouse for real‑time multidimensional log analysis with aggregations, and ElasticSearch for full‑text searchable log monitoring—detailing their schema designs and query examples.
This article walks through Yuque’s October 23 service disruption, detailing each timeline milestone, analyzing the root causes, highlighting the importance of monitoring and data integrity checks, and offering concrete post‑mortem recommendations to improve future incident handling.
This article explains how to design SLA metrics for object storage services, describes the S3 protocol, proposes availability calculations, outlines monitoring and alerting rules, and provides practical implementation examples using s3cmd, Python boto, and Java SDK to ensure reliable cloud storage operations.
This article examines how a metrics‑driven development approach, combined with observability tools like Prometheus, helped a large online used‑car marketplace improve system insight, accelerate business processes, and deliver measurable performance and efficiency gains across both customer‑facing and dealer‑facing operations.
This article explains how to use Prometheus to monitor business‑level metrics in a Kubernetes environment, covering observability fundamentals, metric definitions, metric types, exposing metrics via a /metrics endpoint, and practical Go code examples for defining, recording, and scraping custom metrics.
This article introduces DynamicTp, a Java framework that extends ThreadPoolExecutor with dynamic configuration, real‑time monitoring, and alerting, enabling developers to adjust thread‑pool parameters on the fly, integrate with popular configuration centers, and achieve high‑availability and scalability in microservice environments.
This article visually dissects Prometheus, explaining its architecture, core features, data collection methods, exporter role, PromQL query language, and alerting workflow, while contrasting it with ELK and highlighting practical configuration examples for real‑world monitoring.
This article provides a detailed overview of the most widely used DevOps tools across categories such as version control, CI/CD, container orchestration, configuration management, infrastructure as code, monitoring, collaboration, artifact repositories, testing, security, deployment automation, serverless, and database management, helping practitioners choose the right solutions for their pipelines.
This comprehensive guide introduces Prometheus as an open‑source monitoring solution, covering metric exposition, scraping, storage, PromQL queries, custom exporters in Go, dynamic configuration reloads, Grafana dashboards, and Alertmanager alerting with practical code examples.
This article explains why consistent API return‑code design matters, shows how to model error codes after HTTP status segmentation, adds structured messages for both developers and end‑users, and demonstrates how unified codes enable effective monitoring and alerting.
This article walks through practical methods for troubleshooting slow service alerts, diagnosing Redis performance bottlenecks, and reproducing issues with local demos and load simulations, offering concrete metrics, command‑line checks, and mitigation strategies such as scaling, rate‑limiting, and pipeline optimization.
This article presents a practical pre‑alert monitoring solution for a high‑volume fund trading system, detailing how simple time‑based key‑point checks and targeted alerts reduce instant and end‑of‑day alarms, improve issue detection within 15 minutes, and enhance overall system stability and reconciliation efficiency.
This article explains what logs are, why they are crucial for debugging, monitoring and auditing, when to record them, the fundamental principles and level conventions, and provides concrete best‑practice recommendations and code examples to help Java developers implement robust, secure, and efficient logging.
This article comprehensively discusses the essential aspects of constructing a system engineering architecture, emphasizing value‑first decision making, layered and DDD architectural patterns, technology selection criteria, exception handling, logging, monitoring, and the importance of establishing shared consensus among teams.
This guide walks you through installing Kibana, configuring its connection to Elasticsearch, creating index patterns, using Discover for searches, mastering Lucene‑based query syntax, building visualizations, assembling dashboards, and monitoring logs, all illustrated with clear screenshots and code examples.
This article explains Memcached’s architecture, key characteristics, suitable and unsuitable scenarios, memory management and LRU mechanisms, version details, and provides a comprehensive guide to monitoring its performance and health using Prometheus and Alibaba Cloud ARMS dashboards.
This article details how JDV, JD.com’s internal visual dashboard platform, tackled the massive data‑intensive 618 promotion by implementing real‑time updates, cross‑midnight count stops, request‑state control, heartbeat monitoring, proxy data sources, and a suite of developer tools to ensure stability, performance, and rapid feature delivery.
This guide explains why monitoring systems need self‑monitoring, how to leverage their own /metrics endpoints for internal health checks, and how to supplement them with a lightweight external monitor using catpaw plugins and FlashDuty for robust alerting.
