This article describes how a rapidly growing e‑commerce business used RPA combined with an intelligent Q&A system to automate 24/7 customer service on Enterprise WeChat, addressing resource limits, repeated queries, and large‑scale promotion spikes while preserving merchants' existing communication habits.
This article explains how a large internal network suffered severe slowdown and packet loss due to a routing loop, how traffic analysis revealed massive UDP2425 usage consuming 99% of bandwidth, and the step‑by‑step method used to identify and eliminate the loop.
This article compiles insights from technical leaders, developers, product managers, and operations staff on building effective CMDBs, highlighting common pitfalls, essential success factors, Huawei’s experience, and practical lessons for improving data modeling, automation, and cross‑team collaboration.
This guide presents a concise reference of common Linux command‑line utilities—including uname, hostname, dmesg, stat, du, date, echo, watch, which, whereis, locate and updatedb—showing their typical options, usage examples and visual output to help users quickly retrieve system information, manage files and monitor processes.
This article explains three common software release strategies—blue‑green deployment, rolling deployment, and canary (gray) deployment—detailing their principles, advantages, potential pitfalls, and practical considerations, while also contrasting them with A/B testing and noting related operational concerns.
This guide explains how to analyze Linux load situations—whether CPU and load are both high or CPU is low while load remains high—by using commands like top, vmstat, iostat, sar, and jstack, and provides practical troubleshooting steps for common I/O‑related issues.
This guide explains why log analysis is essential for Linux system health, walks through installing Logwatch on CentOS, configuring its core settings, automating daily runs via cron, and interpreting sample output for connections, SSH activity, package installs, and disk usage.
This article details Huolala's one‑stop monitoring platform called Monitor, covering its multi‑cloud architecture, data collection pipelines, real‑time business monitoring, unified alarm handling, and future AI‑driven enhancements, while sharing concrete metrics, incident case studies, and practical implementation steps for large‑scale observability.
This guide summarizes JD.com’s extensive JDK8 garbage‑collector optimization experience, covering version requirements, how to select the right GC, core and collector‑specific parameter settings, logging configuration, and concrete methods to assess GC health for latency‑sensitive and throughput‑oriented workloads.
Facing growing microservice complexity, HuoLala designed a comprehensive fault‑drill system—covering management, tooling, and operations—to simulate failures, control blast radius, automate scenarios, and continuously improve resilience, ultimately reducing downtime and enhancing system stability across more than ten business units.
Facing unpredictable software behavior, the article explains why traditional system‑level monitoring often misses critical business issues, especially in complex pricing services, and presents a comprehensive approach that combines result (black‑box) and process (white‑box) monitoring, practical metrics, and actionable recommendations to improve observability and reduce operational risk.
The article explains what high availability really means, why it’s essential, outlines six core design principles, compares cloud and on‑premise HA, highlights common pitfalls, and provides a practical multi‑AZ architecture example to help engineers build more reliable systems.
This article explains how Ansible, a Python‑based automation tool, can be used to create, modify, and delete Unix crontab entries, covering why automation is needed, installation steps, and practical playbook examples for managing scheduled tasks across multiple hosts.
This guide introduces two lightweight Kubernetes log‑tailing tools—Kubetail and Stern—explaining their installation on various platforms, core command‑line options, and practical usage examples for aggregating and filtering pod logs in real time.
This article explores the emerging role of technical operations for developer‑focused companies, sharing practical frameworks for organizing tech communities, driving content engagement, building technical brands, and defining the core abilities of effective tech ops, all illustrated with real examples from Alibaba.
The article outlines a practical framework for business‑level monitoring in banking systems, describing three evolution stages, key metrics such as transaction success rates and volume spikes, concrete alert rules, and the technical challenges of data collection, standardization, and massive parameter management.
This article examines how 58租房 introduced the “Housekeeper Talk” video‑tour product, detailing the advantages of video viewings, the end‑to‑end workflow for creators, the UI enhancements for renters, and the measurable impact on user engagement and connection rates.
Performance testing is crucial for ensuring software reliability, and this guide outlines essential best practices—including setting clear goals, selecting appropriate tools, crafting maintainable scripts, using realistic data, running long‑duration loads, and scheduling regular tests—to help engineers achieve stable, high‑performing applications.
This article explains how the CAICT‑led DevOps Capability Maturity Model guides major Chinese telecom operators—China Mobile, China Telecom, China Unicom, ZTE, Huawei and China Tower—through 31 assessments, showcasing project‑level improvements, integration of resources, and measurable gains in delivery speed, quality and operational efficiency across the industry.
Amid a nationwide digital transformation push, leading Chinese banks adopt the CAICT‑led DevOps Capability Maturity Model, using standardized assessments to improve IT efficiency, integrate resources, and support business systems, with detailed case studies from nine city‑commercial and other financial institutions illustrating measurable gains.
The article explains how capacity management balances cost control and service quality through defined goals, three development stages, detailed resource optimization methods, stress‑testing metrics and standards, and automated scaling to achieve significant cost reductions while maintaining system stability.
By integrating Kubernetes with DevOps practices, teams can automate deployment pipelines, achieve dynamic resource allocation, centralize monitoring with tools like Prometheus and Grafana, and treat infrastructure as code, resulting in faster, higher-quality software delivery and improved collaboration between development and operations.
This article examines how major Chinese state‑owned banks adopted the CAICT‑led DevOps Capability Maturity Model, detailing assessment data, case studies of continuous delivery, security, and system‑tool standards, and the measurable improvements in efficiency, risk reduction, and business agility achieved across dozens of projects.
This article reviews how seven major Chinese joint‑stock banks adopted the CAICT DevOps Capability Maturity Model, detailing their evaluation results, project implementations, and the operational improvements achieved across continuous delivery, technical operations, security, and BizDevOps practices.
At the 2023 DOIS DevOps International Summit, China Pacific Insurance leveraged a one‑stop R&D efficiency platform to successfully achieve Level‑3 continuous delivery certification for four projects, illustrating how standardized processes, integrated toolchains, and end‑to‑end metrics can accelerate digital transformation in the insurance industry.
This guide presents a collection of advanced kubectl techniques—including printing API details, filtering and deleting pods by status, counting pods per node, analyzing pod distribution across machines, and leveraging kubectl proxy—providing practical command examples and explanations for experienced Kubernetes users.
This article reviews common deployment techniques—blue‑green, rolling, and canary (gray) releases—explaining their workflows, advantages, limitations, and best‑practice precautions to help teams choose the right strategy for reliable production updates.
This article explores how the rise of cloud‑native technologies forces traditional operations to transform into service‑oriented platforms, detailing new organizational structures, the OPaS model, onion‑layered migration, practical steps, and key lessons for successful ops modernization.
Shopify’s rapid adoption of ChatGPT‑powered assistants and the later launch of the Sidekick AI tool led to aggressive cost‑cutting, a 20% workforce reduction, deteriorating customer support, rising fraud risks, and widespread criticism from merchants and insiders.
This guide provides Bash scripts to filter process details by PID or name, retrieve comprehensive user information, and apply a series of system hardening configurations—including password policies, login restrictions, and file attribute locks—to improve Linux server security and manageability.
This article examines the challenges of traditional on‑call duty systems for operations teams, proposes an SRE‑inspired rotation model that involves developers, defines concrete KPI targets, and describes how automation and chat‑bot tools can streamline incident response and reduce internal friction.
The article recommends two essential Google SRE books for backend developers, explains what SRE is, how it differs from traditional operations, and shows how the concepts like SLI/SLO, incident postmortems, and reliability engineering can be applied to improve system availability and stability.
This article analyzes high‑profile service outages, defines the production environment and its components, categorizes five types of production changes, and presents a comprehensive change‑management framework—including organizational roles, step‑by‑step procedures, and best‑practice tips—to help teams reduce risk and maintain system stability.
This article examines the trade‑offs between running Kubernetes on bare metal versus virtual machines, covering performance differences, orchestration complexity, hardware visibility, capacity planning, network impact, and when each approach makes sense for modern cloud‑native workloads.
This article introduces common Linux command‑line tools such as top, vmstat, pidstat, iostat, sar, netstat, and tcpdump, explaining the meaning of their output fields and how to use them for diagnosing CPU, memory, disk I/O, and network performance issues on servers.
The article details the Bank of Communications' participation in the 2023 XOps Forum, its successful DevOps continuous‑delivery level‑3 assessment for two projects, the resulting efficiency gains, metrics, and future plans for expanding DevOps practices across the organization.
This article explains performance monitoring concepts, lists essential metrics such as response time and CPU utilization, introduces popular monitoring tools like Prometheus and New Relic, and outlines a step‑by‑step process for selecting, configuring, visualizing, alerting, and continuously improving system performance.
This guide explains essential Linux monitoring tools—top, vmstat, pidstat, iostat, netstat, sar, and tcpdump—detailing the metrics they expose, how to interpret CPU, memory, disk, and network statistics, and practical command examples for effective server performance troubleshooting.
The article examines recent Stack Overflow research and academic studies revealing that a large share of developers consider many meetings unnecessary, explains the clash between Maker's and Manager's schedules, and offers practical ways to reduce meeting waste and boost productivity.
This article details how a rapidly expanding B2B fresh‑food company restructured its R&D organization, adopted a matrix management model, and built a comprehensive distributed infrastructure—including task scheduling, service discovery, messaging, logging, file storage, CDN, configuration, sharding, search, caching, and monitoring—to support nationwide warehouse operations and future growth.
When a pod crashes and continuously restarts, using kubelet logs may fail to capture output, but the kubectl --previous flag lets you access the logs of the previous container instance by reading the linked log files stored under /var/log/pods.
This guide explains the three main ELK deployment architectures, compares Logstash and Filebeat collectors, introduces a cache‑queue option for high‑volume logs, and provides practical solutions for multiline log merging, timestamp correction, and module‑level filtering in Kibana, helping operations teams build efficient log pipelines.
This article explains why website performance directly impacts business goals, outlines a four‑scenario testing framework (baseline, capacity, stability, and exception), and provides practical steps for environment setup, data preparation, parameterization, and execution using JMeter.
This article explains how HuoLaLa’s stability team tackled the challenge of proving their work’s value by designing and implementing a comprehensive stability metric system from scratch, detailing the motivations, principles, step‑by‑step construction, data platform, cultural adoption, measurable results, and future plans.
In this detailed case study, HuoLala's stability leader shares how a two‑year, zero‑to‑one stability metric framework was designed, implemented, and iterated—covering the why, the pain points, the metric definition process, data collection platform, cultural adoption, and the resulting 78% fault reduction and SLA improvement from three to four nines.
This article introduces a variety of Linux command‑line tools used for stress testing and performance monitoring, covering CPU, memory, disk, network, processes, load, temperature and other utilities with brief usage examples and practical tips.
This article describes Qunar's comprehensive root cause analysis platform, detailing its background, data-driven fault categorization, architecture—including trace, runtime, middleware, and event analysis modules—and demonstrates its high accuracy and practical impact on reducing incident resolution times across microservice services.
This guide explains how to set up a gray‑release (canary) deployment using Nginx as a reverse‑proxy gateway, Docker containers for isolation, and two versions of a NestJS service, with traffic split controlled by cookies and configurable percentages.
This article explores common pain points in test development, demonstrates how Shell scripting can automate repetitive Linux tasks, introduces basic commands like copy and concatenate, and presents real‑world case studies such as auto‑comment generation, memory‑usage monitoring, service management, and function encapsulation to boost productivity.
The article introduces common release techniques—blue‑green, rolling, and canary (gray) deployments—explaining their principles, advantages, drawbacks, and practical considerations, and distinguishes them from A/B testing in modern software delivery pipelines.
This guide walks through essential Redis operations, covering real‑time monitoring with the INFO command and Prometheus‑compatible exporters, data migration using Redis‑shake, consistency verification via Redis‑full‑check, and comprehensive cluster management with CacheCloud, providing practical tools for reliable Redis administration.
This article examines the shortcomings of traditional host‑based deployments, explains the GitOps methodology with declarative configuration and automation tools, and details a real‑world implementation at NetEase Cloud Music using Horizon CD, Argo CD, and Helm to achieve scalable, reliable, and version‑controlled cloud‑native releases.
This article explains the origins, evolution, and current architecture of WeChat’s security data warehouse, detailing its unified feature storage, data quality guarantees, multi‑IDC synchronization, and operational system that streamlines feature management, analysis, and deployment to support the platform’s massive security strategy.
The article examines the March 29 Vipshop data‑center outage that caused over a billion‑yuan loss, explains the cooling‑system failure that triggered a 12‑hour P0 incident, discusses its impact on Tencent services, and analyzes why high‑concurrency crashes remain common, offering availability tier insights and mitigation strategies.
China Life (Overseas) detailed how its OnePartner insurance marketing platform achieved advanced DevOps continuous‑delivery maturity through CAICT assessment, highlighting the benefits, challenges, and future plans of standardised, tool‑enabled digital transformation for the insurance industry.
ICBC Trust Fund Management’s Transaction Management Platform passed the CAICT DevOps Continuous Delivery Level 3 assessment, showcasing how standardized DevOps practices, tool empowerment, and cultural change dramatically cut build times, accelerate releases, and boost overall digital transformation efficiency.
China Information Communication Research Institute announced that Shanghai Financial Futures Information Technology Co., the tech subsidiary of the China Financial Futures Exchange, passed the DevOps Continuous Delivery Level 3 assessment, marking the first such achievement among domestic securities and futures exchanges and showcasing how standardized DevOps practices can boost digital transformation, quality, and efficiency.
The China Academy of Information and Communications Technology released the newest DevOps Capability Maturity Model evaluation results, showing that 78 leading firms across banking, finance, internet, and telecom sectors have collectively completed 224 projects, highlighting the impact of standardization and tool empowerment on enterprise competitiveness.
The June 2023 release by China’s Academy of Information and Communications Technology details how 78 leading firms across banking, finance, telecom and internet sectors have passed the DevOps Capability Maturity Model assessments, highlighting the impact of standardized pipelines, tool empowerment and industry‑wide adoption on quality, efficiency and competitiveness.
China Life (Overseas) passed the CAICT DevOps continuous delivery Level 2 assessment, showcasing how standardized DevOps practices and a one‑stop insurance marketing platform dramatically improved development efficiency, quality, and market competitiveness while highlighting challenges, outcomes, and future plans.
This article explains the structure of Linux Ext (2/3/4) filesystems, detailing superblocks, inode tables and data blocks, then describes block groups, the role of superblocks, and outlines the differences between hardware and software RAID as well as the principles and risks of using LVM for flexible storage management.
Grafana 10 introduces a suite of enhancements—including Correlations for cross‑data‑source linking, the Scenes front‑end library for building stunning dashboards, new Canvas, Trends, and Datagrid panels, CSV drag‑and‑drop support, sub‑folder organization, and improved data‑source selection—aimed at boosting analysis, collaboration, and efficiency for monitoring teams.
This article outlines the evolution from manual operations to DevOps and SRE‑driven “big operations,” detailing system reliability and continuity practices, observability concepts, and the development of AIOps maturity standards, offering a comprehensive guide for building stable, efficient, and secure operational frameworks.
This tutorial explains how to develop a lightweight Python network monitoring utility using the ping3 library to send ICMP echo requests, measure latency, handle errors, and extend the script for batch pinging, timeout configuration, result logging, and packet‑loss calculation.
The article examines three common resume pitfalls for operations candidates—unclear focus, breadth without depth, and vague personal planning—and offers concrete strategies to highlight strengths, showcase impactful projects, and present a clear career trajectory to attract interview opportunities.
This article explains the fundamentals of Site Reliability Engineering, detailing how SRE combines development and operations to improve stability through metrics like MTBF and MTTR, the roles of SLI/SLO, the VALET selection method, and the practical use of error budgets for quantifying work and guiding alerts.
This article examines the technical and operational challenges of the 618 shopping festival, presenting data‑driven insights and detailed strategies—including modular deployment, monitoring, logging, fast‑failure, rate limiting, database and cache optimizations, and emergency response plans—to help teams maintain system stability under massive traffic spikes.
This guide walks you through practical Linux commands—tail, head, cat, grep, sed, wc, and pagination tools—to quickly locate, filter, and examine specific error entries within large log files, boosting troubleshooting efficiency for system administrators.
The article argues that DevOps is still alive, explains the shortcomings of isolated operational practices, introduces platform engineering as the next evolution, and discusses practical considerations such as third‑party software selection, cloud‑native adoption, and the role of internal developer platforms in improving organizational efficiency.
The article explains the concept of systemic risk in both economics and technology, compares it with non‑systemic risk, describes how it propagates, lists common sources, outlines its impact on technical teams and business value, and provides a step‑by‑step framework for modeling, identifying, and governing such risks.
This article introduces a collection of essential Linux command‑line tools for stress testing and monitoring system resources such as CPU, memory, disk, network, processes, load, temperature, and other aspects, helping administrators assess stability and performance.
This article examines why message queues become backlogged—covering producer overload, broker persistence failures, and consumer bottlenecks—and outlines a step‑by‑step scaling and remediation strategy to restore smooth processing, including temporary queue expansion, load‑balanced forwarding, and post‑recovery cleanup.
This article describes how Qunar's technology center defined, measured, and managed system complexity through a custom modeling framework, implemented a dashboard for continuous monitoring, and established an anti‑corruption governance process that limits complexity growth to maintain low maintenance costs across hundreds of applications and systems.
The second openKylin community board meeting in Beijing detailed governance rules, controlled open‑source initiatives, open‑build infrastructure, innovation projects, ecosystem expansion, and the nomination of new board members, highlighting the community’s rapid growth, extensive SIG groups, and strategic plans for future development.
This article details a comprehensive investigation of an out‑of‑memory crash in a critical Elasticsearch cluster, explains how GC logs and heap dumps revealed a to‑space‑exhausted condition, and describes the G1GC tuning parameters that eliminated the nightly spikes and stabilized performance.
The article examines Vipshop's massive March 2023 outage caused by an IDC cooling failure, critiques superficial PPT‑driven reliability claims, and offers practical SRE insights on fault drills, true multi‑active architectures, and how ops teams can gain influence despite budget constraints.
This guide outlines the most common production failures—including network, server, database, software, security, storage, configuration, and third‑party service issues—and provides step‑by‑step methods to detect, troubleshoot, and resolve each problem, helping maintain system stability and reliability.
This article explores how Tencent builds and practices its SRE quality operation system, focusing on On‑Call incident management, standardized channels, alert handling, data quality models, and the resulting improvements in reliability, MTTR reduction, and data‑driven decision making.
A comprehensive collection of Linux operations interview questions covering topics such as system administration, RAID configurations, load balancing, middleware, MySQL troubleshooting, network monitoring, security, scripting, and best practices for optimizing and maintaining Linux servers.
The article explains why software performance testing is essential, outlines a comprehensive testing workflow, reviews popular load‑testing tools, offers guidance on selecting the right tool, and lists the technical and analytical skills needed to become an effective performance testing engineer.
This article details Bilibili's SRE team's design and implementation of a high‑availability multi‑active architecture, covering zone types, same‑city and cross‑region deployments, traffic routing, cache consistency, message handling, governance, and practical lessons learned from real‑world incidents.
This analysis examines why Chinese state‑owned enterprises face unclear digital‑transformation goals, weak strategic positioning, fragmented data, talent shortages, and inadequate technology ecosystems, and it outlines the root causes, typical case studies, and recommended actions to achieve effective digital change.
A concise guide for Linux administrators that introduces thirteen practical monitoring and security tools—ranging from network bandwidth trackers like Nethogs to vulnerability scanners like NMap—complete with installation steps, usage examples, and key configuration tips.
This article compares four popular open‑source log collection tools—Logstash, Fluentd, Fluent Bit, and Vector—examining their key features, performance, resource usage, scalability, security, and ecosystem to help enterprises select the most suitable solution for their specific logging needs.
The article analyzes a serverless scaling failure where newly added instances suffered high CPU and frequent Full GC leading to JVM crashes, reproduces the issue under load, and demonstrates how Sentinel's CPU‑based circuit‑breaker rule mitigates the problem across cold and hot start scenarios.
The article explains how data‑driven thinking replaces traditional money‑burning growth tactics by establishing logical loops, experimental validation, and concrete case studies in acquisition, activation, and targeting, while outlining the essential collaborative factors needed for successful data‑powered operations.
This article details the end‑to‑end process of migrating an anti‑cheat service to a new data center, verifying strategy effectiveness, building a real‑sample regression pipeline, and automating integration steps using GoAPI and traffic‑comparison platforms to ensure functional consistency and security.
The article shows how AI large‑language models such as ChatGPT can support every stage of the software development lifecycle—from extracting requirements and designing solutions to generating code, tests, deployment scripts, and operational diagnostics—while warning about model inaccuracies, hallucinations, intellectual‑property and privacy risks.
This article traces the origins of Site Reliability Engineering from Margaret Hamilton’s pioneering work on the Apollo program, through Google’s formal SRE team creation, and highlights the key differences between SRE and traditional operations practices.
This article details JD's end‑to‑end H5 observability solution, covering the challenges of hybrid app development, the design of a three‑stage UEM platform, deep active and passive monitoring, automated quality gates, and real‑world case studies that demonstrate cost savings and performance improvements.
This guide outlines practical methods to avoid split‑brain incidents in production HA clusters, including dual‑cable heartbeats, arbitration mechanisms, monitoring alerts, disk‑locking techniques, and a detailed keepalived‑based script for automated detection and recovery.
This guide outlines the most common production failures—including network, server, database, software, security, storage, configuration, and third‑party service issues—and provides detailed steps for detecting, diagnosing, and resolving each type to maintain system stability and reliability.
This article explains RunnerGo's scene management module, covering the interface between left‑hand navigation and the main scene area, how to create and link interfaces and controllers into executable business scenarios, configure scene settings, debug scenes, and manage test case sets, with links to the project's repositories.
This article presents a comprehensive reference of common Linux/Unix command-line utilities covering system information, date handling, shutdown/reboot, file and directory management, searching, mounting, disk usage, user/group administration, permissions, special attributes, compression, package management, backup, and networking, providing a handy guide for system administrators and developers.
This article presents a comprehensive overview of 17 top Chinese enterprises that have passed the DevOps Capability Maturity Model assessment across 20 evaluation items, detailing their project implementations, performance improvements, and the broader significance of the model for digital transformation.
Amid a nationwide digital transformation, leading Chinese telecom operators have leveraged the CAICT‑backed DevOps Capability Maturity Model to evaluate and improve their IT performance, integrating team resources and talent to better support business systems, with detailed case studies and measurable outcomes across dozens of projects.
This article outlines Huolala's comprehensive approach to event‑tracking data quality, covering goal definition, industry research, a six‑step transformation plan, real‑time monitoring, tool development, and future outlook for their in‑house tracking platform.
Amid digital transformation, nine Chinese city‑commercial banks and financial institutions adopted the CAICT‑led DevOps Capability Maturity Model, achieving significant IT efficiency gains, integrating resources, and enhancing business support across continuous delivery, technical operation, security, and system tooling, with detailed project case studies and a comprehensive overview of the standard.
This article presents a systematic overview of the emerging XOps ecosystem—including DevOps, BizDevOps, AIOps, FinOps, and SRE—detailing their relationships, maturity models, standards, and practical guidance for enterprises seeking to achieve efficient, secure, and data‑driven digital transformation.
The article outlines how ChatGPT 4.0 can significantly improve product managers' workflow across research, planning, design, project execution, and iteration by providing prompt engineering techniques, practical examples, and actionable recommendations while emphasizing security and information‑risk considerations.
