The automated service fault localization system ingests massive real‑time instrumentation data, builds call‑chain graphs, and instantly pinpoints the exact component causing timeouts or other errors, achieving developer‑level accuracy within seconds instead of minutes while remaining simple, fast, and fully automated.
This article explains the fundamentals of forward and reverse proxies, compares their characteristics and differences, and provides practical Nginx configuration examples for implementing reverse proxy, load balancing, and cross‑origin handling in web applications.
This article explains the various DevOps team topologies—including anti‑patterns A‑G and nine positive types—detailing their characteristics, applicability, and potential effectiveness so organizations can select the most suitable structure for their value‑stream delivery goals.
This article compares three container alerting frameworks—Alertmanager, Kapacitor, and Bosun—explains why Bosun was chosen for its flexible HTTP API rule deployment and low learning curve, and provides step‑by‑step configuration, rule definition, notification, and templating examples for integrating Bosun with Prometheus.
At the 2019 DevOps International Summit in Beijing, Guangdong Mobile and Huawei unveiled their first carrier‑level DevOps deployment for 5G network elements, detailing the challenges of traditional rollout, the shift to X86‑based cloud infrastructure, and a step‑by‑step pipeline that cut deployment time by over 90%.
This guide compiles the most indispensable Linux commands—from directory and file manipulation, navigation, and text processing to compression, daily system administration, status monitoring, networking, and database access—providing concise examples and practical tips for both beginners and seasoned users.
This guide explains how to install Netdata, a fast web‑based Linux performance monitor, and apply a Chinese localization by using a forked repository, running the provided installer script, and configuring the service to view detailed system metrics through a clean UI.
This article explains China’s inaugural DevOps Capability Maturity Model, its continuous delivery requirements, the assessment process, participating enterprises, and how organizations can apply for certification in the upcoming evaluation round.
This article describes Ctrip’s large‑scale continuous delivery system, its benefits for efficiency, quality, reliability and team collaboration, the evolution of its deployment models, the design of a unified Jenkins‑based build platform, and practical experiences running Jenkins on Kubernetes with elastic scheduling and workspace management.
This guide explains ETCD’s high‑availability architecture and provides detailed step‑by‑step recovery procedures for single‑node failures, majority‑node outages, and database‑space‑exceeded errors, including status checks, member removal and addition, snapshot restoration, compaction, defragmentation, and alarm clearing.
This article details how MaFengWo’s payment center evolved from a simple 1.0 implementation to a robust 2.0 architecture, covering core capabilities, modular design, monitoring, and the operational lessons learned for building a high‑availability, scalable payment platform.
This article presents a case study of an overseas user experiencing video upload failures due to international roaming, explains how Qunar's network detection tool (QunarNDT) was used to diagnose the issue, outlines the findings about domestic routing and timeout, and offers recommendations for improving user experience.
This article explains DNS fundamentals, the step‑by‑step resolution process, TTL caching, and how DNS‑based Global Server Load Balancing (GSLB) can direct traffic to the nearest active data‑center, providing a practical guide for building multi‑active, high‑availability infrastructures.
This article introduces server clustering and load‑balancing concepts, compares popular software such as LVS, Nginx, and HAProxy, explains their architectures, NAT and DR modes, and outlines each solution's strengths and weaknesses for building high‑performance web services.
This article provides a comprehensive overview of Prometheus, an open‑source monitoring and alerting system, covering its origins, key features, architecture, core components, metric types, data collection methods, service discovery, storage options, and query capabilities.
This article provides an in‑depth overview of Puppet, covering its background, C/S architecture, installation steps for master, CA and agent nodes, detailed configuration options, resource definitions, and common package and file resources, illustrating how to automate large‑scale server cluster management.
This article recounts AdMaster's five‑year journey from a few dozen servers to thousands, detailing the evolution of their monitoring infrastructure, the challenges faced at each scale stage, and the design of a self‑built, distributed monitoring platform that delivers real‑time alerts, visualized data, and business‑level insights.
This article explains common scenarios where empty variables, spaces, special characters, or failed directory changes cause accidental deletions in Linux, and provides practical shell techniques—such as quoting, parameter expansion, set -u, and logical checks—to safeguard against disastrous rm -rf commands.
This article shares AdMaster's five‑year operations journey, detailing how the team scaled monitoring from under 200 machines to over a thousand, the evolution of their monitoring stack, the design of a custom distributed system, and practical Q&A insights for large‑scale infrastructure management.
This guide explains how to identify and resolve common Java production problems such as sudden CPU spikes, excessive Full GC, thread blocking, waiting states, and deadlocks by using tools like top, jstack, jstat, and memory‑dump analysis to pinpoint the root cause and apply appropriate fixes.
This guide walks through downloading, extracting, and setting up Node Exporter, Alertmanager, Prometheus, and Grafana on a Linux server, configuring their systemd services, customizing alert rules, and verifying the monitoring and alerting pipeline with screenshots of each verification step.
This article shares hard‑earned operational guidelines for Linux servers, covering safe testing, cautious use of rm ‑rf, the importance of backups, strict access control, SSH hardening, firewall rules, intrusion detection, systematic monitoring, performance tuning, and maintaining a calm mindset to prevent costly incidents.
WeChat, China’s top social app with over a billion users, has applied DevOps practices to dramatically improve development efficiency, code quality, and accelerate the feedback cycle from requirements to delivery, while confronting real‑world challenges in tooling, processes, reliability, and automation costs.
This article explains Prometheus metric naming conventions, label usage, metric types such as Counter, Gauge, Summary, and Histogram, and describes the structure of data samples, providing examples and best‑practice guidelines for defining and classifying metrics in monitoring systems.
This article explains the fundamental flaws of typical alert systems, distinguishes between business rule and reliability monitoring, outlines essential metrics and strategies for effective alerts, and presents simple yet powerful anomaly‑detection algorithms to ensure alerts are actionable and reduce noise.
This article presents an in‑depth interview with Alibaba Cloud senior expert Sun Tingtao, detailing the architecture, core features, design challenges, and operational strategies of the Alibaba Cloud Log Service that handles over 10 PB of daily log data for massive, diverse production workloads.
This article explains common scenarios that lead to accidental directory or file deletions in Linux shell scripts—such as empty variables, spaces in paths, special characters, and failed cd commands—and provides practical Bash techniques like variable expansion checks, quoting, set -u, logical short‑circuiting, and safer prompts to avoid catastrophic rm -rf mistakes.
This article explains how the team designed and implemented a unified monitoring and alert platform for a multi‑service transportation business, covering architecture, data collection, storage, rule engine, alert delivery, troubleshooting aids, encountered pitfalls, and future enhancements.
The 2019 Enterprise‑Level DevOps Empowerment Forum in Chengdu gathered experts from major Chinese banks and telecoms to share practical experiences, including China Merchants Bank’s K8s‑based pipeline, measurement challenges, and collaborative Q&A, illustrating how organizations can accelerate DevOps adoption and improve delivery efficiency.
This guide introduces a collection of practical Linux operations tools—including Nethogs, IOZone, IOTop, IPtraf, IFTop, HTop, NMON, MultiTail, Fail2ban, Tmux, Agedu, NMap and Httperf—providing concise installation commands, basic usage examples, and key options to help system administrators monitor performance, security and resources efficiently.
This article details Meituan's R&D team's systematic PDCA‑based approach to resource cost optimization, covering methodology definition, planning, execution, checking, and iterative improvement across infrastructure, big‑data, and shared services, ultimately saving tens of millions of yuan.
This article analyses why seemingly normal PHP‑FPM requests can cause Nginx to return 502 errors, revealing a FastCGI fastcgi_buffer_size bug triggered by warning output, describing the reproduction steps, detailed packet analysis, the underlying protocol mechanics, and practical recommendations for developers and operators.
This guide introduces a dozen practical Linux operation tools—including Nethogs, IOZone, IOTop, IPtraf, IFTop, Fail2ban, Tmux, and others—providing concise descriptions, download links, and ready‑to‑run installation commands to help system administrators boost monitoring, performance testing, and security on their servers.
This article explores the evolution of operations roles, presents detailed skill maps for system, web, big‑data, and container operations, explains essential log types, and outlines ELK‑based architectures and big‑data‑driven monitoring practices for building a robust, future‑proof operations platform.
This article examines Alibaba Cloud’s intelligent data center ecosystem, covering market share, global distribution, operational challenges, AIOps evolution, multi‑layered infrastructure platforms, demand forecasting, fault prediction, and future smart‑automation prospects for large‑scale cloud operations.
This guide reviews four practical backup solutions—Git for versioned file control, Rsync for command‑line incremental syncing, Dropbox for cloud‑based GUI storage, and macOS Time Machine for full system snapshots—explaining their key features, typical use cases, and basic setup steps.
This talk shares a senior director’s experience building a hybrid multi‑cloud infrastructure for a game company, covering stability, efficiency, cost challenges, design‑for‑failure principles, standardization, resource automation, and the cultural and organizational factors that affect successful cloud operations.
This article explains the fundamentals of Linux control groups (cgroups), their components and relationships, and provides step‑by‑step guidance on creating hierarchies, mounting, configuring subsystems, and applying cgroup limits to Docker and Kubernetes containers.
This tutorial explains how to install the Python Paramiko library, create an SSH connection to a remote server using password or key authentication, and then elevate to root privileges with sudo, while highlighting additional capabilities of the module.
This guide explains how to select and grade server racks, outlines essential criteria such as load capacity, ventilation, power distribution, and cable management, and compares three cable‑routing techniques to help data‑center operators make reliable, future‑proof decisions.
Zhang Guanshi, the operations director at Huya Live, shares how his team designs a hybrid‑cloud architecture, implements a six‑pillar reliability framework, and leverages real‑time monitoring, AIOps, and rapid‑recovery tools to maintain stable, low‑latency live video streams for millions of viewers.
This article outlines Qunar's operational strategy for reducing failures and extending uptime through precise fault detection, rapid recovery, and AI-powered predictive health management, detailing the evolution of their OPS processes, practical implementations, and future challenges in applying PHM to internet services.
iQIYI built a three‑component CDN Inspection System that automatically generates tasks, centrally processes and analyzes results, and runs edge measurements to monitor millions of hybrid CDN servers in real time, detecting configuration errors, file mismatches and traffic anomalies, enabling proactive remediation and 100 % local coverage.
This article introduces the 2019 Accelerate State of DevOps survey, explains how to join the questionnaire, provides background on previous reports, shares a detailed interview with Nicole Forsgren about research design and key findings such as architecture, cloud adoption, and outsourcing, and encourages community participation.
This article reviews common open‑source monitoring tools, shares the evolution of China Unicom's big‑data platform monitoring, and provides practical guidance on selecting collectors, databases, and visualization components, with detailed configurations for Prometheus, Alertmanager, Grafana, and automation recovery techniques.
This article shares practical lessons from a real‑world ops incident, emphasizing the need for clear change background, optimal timing, project‑style management, and strict process adherence to reduce risk and improve production reliability.
The article explains the launch of the world’s first DevOps capability maturity model, the collaborative effort behind it, Didi’s role as a standards workgroup member, and how its OE (OneExperience) platform embodies the new guidelines to streamline the entire software delivery lifecycle.
This guide walks you through the entire career ladder of a senior operations engineer, covering essential Linux, networking, monitoring, container, automation, and security skills, while offering practical advice on job roles, learning paths, and professional growth.
This guide outlines the career stages, skill sets, and practical tasks for DevOps engineers—from entry‑level troubleshooting to senior‑level architecture, automation, and performance optimization—providing concrete learning paths, tools, and personal development advice to help engineers advance their operations careers.
This article reviews common open‑source monitoring combinations, compares their strengths and weaknesses, and shares practical guidance on selecting collectors, storage back‑ends, and visualization tools such as Telegraf, InfluxDB, Prometheus, Grafana, and alertmanager for large‑scale data platform operations.
This article dissects the 2019 Spring Festival (春运) campaign by 58.com, revealing how a win‑win design mindset, data‑driven insights, and integrated business collaboration transformed user experience, increased traffic, and delivered measurable results across multiple channels and game‑based interactions.
This article presents a comprehensive set of MySQL high‑performance optimization guidelines, covering naming conventions, table design, data types, index strategies, SQL coding standards, replication, backup, and operational best practices to improve efficiency, reliability, and scalability of database systems.
An in‑depth guide walks through the essential layers of modern website architecture—including front‑end optimization, application frameworks, service distribution, storage solutions, backend processing, monitoring, security, and data‑center design—offering practical strategies for building high‑performance, scalable web systems.
This guide outlines six practical strategies—rollback testing, cautious destructive actions, clear command prompts, verified backups, careful handovers, and proactive monitoring—to help operations teams minimize outages and maintain high system availability.
This article explains how modern internet systems use server clusters and load balancers, compares the three most popular software solutions—LVS, Nginx, and HAProxy—covers their architectures, NAT and DR modes, advantages, disadvantages, and provides guidance on selecting the right tool for different scale scenarios.
This article explains how to transform a traditional monolithic deployment into a fully automated micro‑services environment by containerizing services, using Consul for dynamic service discovery and configuration, and configuring HAProxy with DNS resolvers to achieve seamless load balancing and zero‑downtime updates.
This guide explains Jenkins project management, covering naming conventions, creating new projects, configuring build history, parameterized builds, triggers, Jenkinsfile setup, as well as building, viewing logs, and debugging pipelines with illustrative screenshots.
This article outlines the three main load‑balancing methods—DNS, hardware, and software—detailing their advantages and drawbacks, then explains common algorithms such as round‑robin, weighted round‑robin, least‑connections, performance‑based, and hash, and provides guidance on combining them for optimal architecture.
This article explains the meaning of the Linux uptime/top output, defines system load average as the average number of runnable and uninterruptible processes, distinguishes it from CPU utilization, and provides guidance on interpreting load values for single‑core and multi‑core systems.
This article explains the shortcomings of the legacy signal design in a scheduling platform, outlines four major dependency problems, and presents a newly engineered signal system with modular functions, instance ID generation, competitive priority rules, and state management to reliably support complex cross‑period and parallel job dependencies.
The article examines real‑world DevOps transformation cases, illustrating how shifting operations staff into development teams can create both integration challenges and opportunities, and proposes a framework for distinguishing repeatable versus unique work to guide effective consolidation, standardization, and automation in software delivery.
This article explores the full spectrum of modern operations, covering environment deployment, troubleshooting, backup, high availability, monitoring, security, automation, virtualization, and cloud services, while highlighting essential tools and best practices for both Linux and Windows environments.
This article explains why most alerts are poorly designed, defines actionable alerts, outlines monitoring objectives, discusses metric selection, and presents simple yet powerful algorithms for anomaly detection to improve system reliability and operational efficiency.
Reflecting on a decade of AWS, this article shares ten hard‑earned lessons—from building evolvable systems and anticipating failures to prioritizing security, automation, and open platforms—that guide the design, operation, and scaling of cloud services for today’s enterprises.
This article explains the evolution of Linux device management, introduces udev’s architecture and key features, details rule file syntax and operators, and provides practical examples—including disabling VXLAN offload, setting I/O scheduler, and enabling SR‑IOV—complete with command‑line snippets.
This article explains how large enterprises such as BAT and banks implement strict auditing and supervision of operational privileges, using personal accounts, command logging, OSSEC monitoring, firewall limits, and cross‑team oversight to enforce the principle of least privilege.
This article presents a detailed, practice‑driven guide for enterprise WiFi, covering network planning, full‑coverage design, channel optimization, security, KPI‑based monitoring, probe‑based measurement, troubleshooting techniques, and real‑world case studies from Ctrip, highlighting how systematic operations can ensure high‑quality wireless service.
This guide details the prerequisites, multiple deployment methods (WAR, macOS, Windows, Linux), and post‑installation configuration steps for Jenkins, including unlocking the instance, installing plugins, creating an admin user, setting an update site, and configuring a slave node.
This article presents a collection of practical Linux command‑line shortcuts and techniques—including cursor navigation, history execution, disk and memory inspection, process management, multi‑command chaining, and file handling—that can significantly improve efficiency for developers and system administrators.
This article describes a monitoring system that leverages machine learning to perform threshold‑free, real‑time anomaly detection on macro business indicators such as network traffic and access volume, detailing its architecture, sample labeling, model training, and multi‑level alarm strategies.
The article describes how the 58 monitoring platform reduces alarm storms through alarm convergence, intelligent merging using Gini‑based decision trees, and automated self‑healing, thereby improving alert quality, cutting noise by about 70%, and helping engineers resolve incidents faster.
A collection of vivid DBA anecdotes reveals common pitfalls—from missed alerts and accidental production restarts to unsafe terminal habits—and distills practical safeguards that any database operator can adopt to avoid costly mishaps.
Ops engineers who embrace the cloud—leveraging serverless, Kubernetes, AI, edge and elastic resources—gain cost‑efficient scalability, avoid on‑premise limitations, and open career paths such as cloud reliability engineer, solution architect, integration specialist or technical operations manager, ensuring relevance in the dominant, irreversible cloud‑first future.
The article examines the difficulties of quantifying software development efficiency, critiques common metric approaches, and proposes a principled framework that emphasizes global, outcome‑oriented indicators across delivery efficiency, quality, and capability to guide systematic R&D performance improvement.
This article explains how software teams can select, combine, and interpret nine essential metrics—including lead time, cycle time, team velocity, defect rates, MTBF, MTTR, and security incident counts—to drive continuous improvement, align with business goals, and ultimately achieve successful outcomes.
The article outlines Alibaba’s Hema delivery platform’s end‑to‑end stability strategy, detailing a 7‑layer funnel review process, three core norms (development, architecture, stability), and 23 practical tactics—including core‑noncore isolation, proactive monitoring, fault prevention, rapid recovery, and service‑level controls—to ensure reliable 30‑minute deliveries despite complex logistics and external disruptions.
This article explains the concepts of forward and reverse proxy servers, why organizations use them, and provides step‑by‑step configuration examples for both Nginx and Apache to help engineers enforce network policies and secure service access.
This article examines how data center operators and the supply chain are advancing toward 400 Gbps Ethernet, detailing the technical innovations, market forces, and future challenges that shape high‑speed networking, optical modules, and ASIC development for ultra‑large scale data centers.
This article presents nine practical logging best‑practice recommendations—from understanding human and machine audiences and standardizing log formats to leveraging metrics, proper alerting, severity levels, contextual information, and advanced framework features—helping operations teams improve system performance and troubleshooting efficiency.
The article examines how rapid cloud adoption, AI‑ops, and DevOps blur traditional IT operations roles, arguing that ops must shift from a low‑value cost center to a profit‑generating, efficiency‑driving function through mindset change, institutional innovation, expanded responsibilities, modern tools, and continuous skill upgrades.
This article explains server clustering and load‑balancing concepts, detailing the architecture and operation of LVS, Nginx, and HAProxy, and compares their advantages, disadvantages, and typical deployment scenarios; it also discusses NAT and DR modes, load‑balancer placement, and best‑practice recommendations for different traffic volumes.
The article chronicles the design, evolution, and lessons learned of JD Digital Technology’s self‑built host monitoring platform “DiTing”, detailing its initial requirements, V1 architecture, subsequent V2 and V3 redesigns, encountered challenges, and future directions toward intelligent operations.
The article introduces Octopus, the core automation service of 58 Group, detailing its overall architecture, the Octopus Agent lifecycle, communication mechanisms, management center capabilities, and key functional modules such as server information collection, command execution, deployment, permission control, and file transfer.
The article dissects the Aegis combat system’s dual‑screen UI, explaining why a simple 2D top‑down map paired with a side view outperforms flashy 3D graphics, how multi‑target blocks replace tables for faster decision‑making, and how human‑factor testing, eye‑tracking and standardized symbols dramatically improve combat efficiency.
The article chronicles the design, implementation, and iterative evolution of JD Digital Technology’s in‑house host monitoring platform Diting, detailing its V1, V2, and V3 architectures, the challenges encountered at each stage, and future directions toward intelligent, automated operations.
A seasoned operations professional argues that despite sensational claims, the ops function remains essential—driven by its core responsibilities of quality, cost, efficiency, and security, evolving with cloud computing, DevOps, and emerging IoT demands.
This article explains the principles and components of high‑availability (HA) clusters, covering active/standby nodes, resource stickiness and constraints, heartbeat and quorum mechanisms, split‑brain avoidance, failure detection methods, and the minimal setup required for a reliable web‑service HA deployment.
This article examines how DevOps has been reduced to a collection of automation tools, critiques the resulting "same‑bed‑different‑dreams" separation of development and operations, and outlines the cultural principles—shared responsibility, trust, autonomy, built‑in quality, feedback, and automation—necessary for a genuine DevOps transformation.
The article argues that operations will not disappear, explaining its essential functions—quality, cost, efficiency, and security—how cloud computing reshapes the role, the evolution toward DevOps, and why both cloud outages and industry trends actually underscore ops’ enduring importance.
A personal account of switching jobs and the first day on an operations team, detailing a critical menu‑click error, the rapid response, and the deeper reflections on risk awareness, teamwork, and professional growth in high‑stakes production environments.
This article details NetEase's journey in designing and implementing a large‑scale database automation platform, covering its requirements, tool‑based operations, architecture, AIOps integration, and the practical lessons learned for managing thousands of database clusters efficiently.
This guide presents practical Bash scripting techniques—including precautions, random string generation, color output functions, bulk user creation, package checks, service status verification, host liveness testing, resource monitoring, disk usage audits, and website availability checks—to help you automate Linux system administration tasks effectively.
This article reflects on the misconceptions of IT operations roles, examines why many view ops as a low‑value job, and offers a practical roadmap—automation, DevOps, cloud services, and Python—to help ops engineers future‑proof their careers amid evolving industry trends.
This article traces the historical development of DevOps from the early days of self‑developed and self‑maintained computer programs, through the rise of professional developers and operations engineers, to the modern agile era where development and operations must collaborate to meet rapid market changes.
This article traces the evolution of DevOps from Patrick Debois' 2007 frustrations as a Belgian IT consultant through key conferences, blogs, and publications that shaped the DevOps movement, highlighting its roots in Agile practices and the convergence of development and operations.
This guide walks you through Linux command‑line shortcuts, the five key system‑operation metrics, and powerful tools like smem, ps, and sort to efficiently monitor CPU, memory, processes, disks, and network while also handling zombie processes.
This article analyzes the essential capabilities required for building a DevOps infrastructure in traditional enterprises across foundation, development, testing, operations, and project management, mapping each capability to representative tools and offering guidance on flexible, evolving architecture design.
The article outlines Ant Financial's technical risk prevention platform, describing the challenges of financial‑grade distributed architectures, the multi‑layer risk assurance system, the TRaaS platform's risk baseline, handling, and change‑control mechanisms, and how these practices empower partners to achieve high‑availability and secure financial services.
This article presents thirty practical recommendations for deploying, configuring, and maintaining Pika—a high‑capacity, RocksDB‑backed Redis‑compatible storage system—covering version selection, thread settings, hardware choices, key design, memory management, replication, backup, compaction, security, and monitoring to achieve reliable and high‑performance operation.
This talk reveals why middleware operations in financial institutions feel especially painful, examines the specific cost, autonomy, and reliability challenges, and outlines a step‑by‑step evolution toward tool‑driven platforms, hybrid‑cloud deployment, and AIOps that reduce manual toil and improve system resilience.
