This article explains the Linux kernel parameters that limit open files and TCP connections, shows how to increase those limits with sysctl and limits.conf, and estimates the maximum number of concurrent connections a server or client can support based on memory and port constraints.
The article explains how qualitative analysis serves as a measurable metric throughout the software lifecycle, outlines five key qualitative methods—interviews, root‑cause analysis, maturity assessment, reviews, and post‑mortems—and demonstrates their practical application for continuous quality improvement.
This article explains why oversized logs hurt system performance, then presents a step‑by‑step methodology—including printing only necessary logs, merging duplicate entries, and simplifying payloads—illustrated with real Java code and a concrete case study that reduces daily log volume from 5 GB to under 1 GB.
This article breaks down the essential design and operational considerations for achieving high availability across six layers—development standards, application services, storage, product strategy, operations deployment, and incident response—providing concrete practices, metrics, and safeguards to reach four‑nine (99.99%) uptime.
Guangdong Mobile’s CRM system, supporting over 130 million users, passed the China Information & Communication Research Institute’s DevOps technical‑operation 2+ level assessment, showcasing a landmark achievement in standardized, tool‑enabled DevOps practices that boost quality, safety, and market competitiveness.
The article details how China’s CITIC Securities leveraged the national DevOps and DevSecOps maturity models, passed Level 2 security assessments, and integrated cultural, procedural, and technical practices to enhance its institutional business service platform, improve security, and accelerate its digital transformation.
This article explains how to maintain a DevOps environment by implementing comprehensive monitoring, handling fault detection and performance metrics, automating alerts in a continuously changing cloud landscape, and integrating NOC and MSP practices for 24/7 reliability and efficient incident response.
In a recent interview, the head of the System Operations Department at China Merchants Securities explains how their next‑generation core trading system, built on a distributed micro‑service architecture with open‑source components and cloud‑native tools, achieved Level 2 technical‑operation DevOps certification, detailing the challenges, improvements, and future plans for digital transformation.
This article explains what stability assurance is, outlines a systematic workflow—including anomaly identification, monitoring configuration, impact assessment, and solution planning—and provides practical methods such as capacity estimation, traffic limiting, load testing, scaling, and pre‑heating to ensure services remain stable during both daily operations and high‑traffic events.
Loggie is a cloud-native, Go-based log-collection agent that replaces Filebeat and Flume by using a micro-kernel producer-consumer architecture with hot-swappable pipelines, achieving 2 GB/s read speeds, 1.6‑2.6× higher throughput while using only a quarter of the CPU, and providing built-in observability, reliability, and latency monitoring for large-scale enterprise deployments.
The iDesk service platform is a comprehensive internal tool that evolved through three development phases, adopts a BS+Service architecture with modular local services, offers extensive software management and self‑service utilities, integrates tightly with TripPal and service accounts, and implements robust operational monitoring to achieve high availability and user satisfaction.
This guide explains why MySQL monitoring is critical, defines five key metric dimensions, shows how to collect them with Prometheus and the MySQL Exporter, provides ready‑to‑use alert rules, and walks through the full setup and dashboard creation on Alibaba Cloud.
This article describes how Zhaozhuan’s testing environment evolved through three versions—physical isolation, automatic‑IP‑tag routing, and manual‑tag routing—detailing the architectural background, implementation principles, advantages, drawbacks, and supporting tools that dramatically reduced deployment time and resource consumption while introducing new operational challenges.
This guide walks you through installing the required build tools, configuring go-bin-rpm with a concise rpm.json file, generating RPM packages, and automating the process with a Makefile for seamless Go binary deployment on RPM‑based systems.
In this interview, Liu Lei, General Manager of the Bank of Communications Software Development Center, explains how three flagship projects passed the DevOps Continuous Delivery Level‑3 assessment, detailing the standards, metrics, tooling improvements and the broader impact on the bank’s digital transformation.
This article explains how a fintech payment platform achieves high availability and optimal channel selection by using decision‑tree routing, sliding‑window negative‑feedback, pressure‑detection services, and component fallback strategies such as RabbitMQ with Redis, supporting millions of daily transactions.
This article defines enterprise‑grade SaaS, contrasts it with consumer products, and presents a comprehensive framework for product, data, and system stability—including isolation requirements, SLA metrics, risk modeling, mitigation plans, and continuous review—to help SaaS teams deliver dependable services.
The article illustrates how to improve the efficiency of an image‑recognition service by measuring performance, redesigning architecture with parallel processing and message queues, and then extends the analogy to enterprise workflow optimization, emphasizing the need to quantify, visualize, and continuously refine DevOps processes.
This article details Nanjing Bank's successful passage of the CAICT DevOps Continuous Delivery Level 3 assessment for its digital credit card project, highlighting the bank's DevOps practices, challenges, benefits, and future plans within a broader digital transformation context.
This article explains why excessive INFO‑level logs can cause performance problems, presents three practical strategies—printing only necessary logs, merging log entries, and simplifying log content with code examples—and demonstrates their impact through a real‑world Java bean pipeline case that cuts daily log volume from about 5 GB to under 1 GB.
This article introduces seven popular Kubernetes management solutions—including K9s, Rancher, the native Dashboard with Kubectl and Kubeadm, Helm, KubeSpray, Kontena Lens, and WKSctl—detailing their key features, usage scenarios, and how they help streamline cluster monitoring, deployment, scaling, and security across cloud‑native environments.
China Agricultural Bank’s distributed core customer information project passed the Level‑3 DevOps Application Design assessment, showcasing a cloud‑native micro‑service architecture, comprehensive DevOps practices, and measurable improvements in scalability, observability, and security that set a new industry benchmark.
This comprehensive guide covers kubectl autocomplete, context configuration, object creation, resource viewing, updating, patching, editing, scaling, deletion, pod and node interaction, as well as the versatile kubectl set commands, formatted output options, and visual references for effective Kubernetes cluster management.
The article details how Tencent’s health‑code platform leveraged a cloud‑native, serverless architecture, extensive observability (Prometheus, Grafana, RUM), rigorous capacity testing, chaos engineering, and ITIL‑based change management to sustain billions of page views, support massive concurrency, and ensure reliable, scalable epidemic‑control services.
Guojin Securities' Commission Treasure App achieved Level‑3 continuous delivery in the CAICT DevOps assessment, showcasing how standardized DevOps practices, tool integration, and a unified platform boosted development efficiency, security, and digital transformation across the financial services sector.
Ansible, a rapidly popular open‑source automation tool built on Python, enables batch system configuration, program deployment, and command execution through thousands of built‑in modules, offering a simple yet powerful solution for operations engineers, illustrated here with 16 comprehensive images.
This article shares practical Linux command techniques—including xargs, background execution, process monitoring, multitail, continuous ping logging, TCP state inspection, and SSH port forwarding—to help system administrators streamline tasks, improve script efficiency, and troubleshoot performance issues.
An in‑depth interview with Liu Lei, General Manager of Bank of Communications' Software Development Center, reveals how three flagship projects passed the Level‑3 Continuous Delivery assessment, illustrating the bank's DevOps transformation, metric improvements, and future roadmap within China's digital banking landscape.
An in‑depth interview with Liu Lei, GM of Bank of Communications' Software Development Center, reveals how the bank’s three flagship projects passed the DevOps Continuous Delivery Level‑3 assessment, boosting automation, efficiency, and digital transformation across its financial services.
The China Academy of Information and Communications Technology released the latest results of its DevOps Capability Maturity Model assessments, showing how standardization, tool empowerment and continuous delivery pipelines boost quality, efficiency, security and competitiveness across banks, telecom, finance and internet enterprises.
The article introduces the AIOps (Artificial Intelligence for IT Operations) capability maturity model developed by China’s Information and Communication Research Institute, explains its two parts—general capabilities and system/tool technical requirements—lists the evaluated modules, and announces the upcoming certification ceremony and contact details for participation.
China Agricultural Bank’s recent DevOps and AIOps assessments, covering 17 projects across continuous delivery, security, application design, and intelligent operations, showcase how standardized processes, tool empowerment, and rigorous evaluation boosted efficiency, safety, and digital transformation, offering actionable insights for large enterprises seeking similar maturity.
On December 18, 2022, Alibaba Cloud's Hong Kong Region Zone C suffered a massive service interruption—the longest in its operational history—prompting a detailed incident response, extensive service impact across compute, storage, and networking, and a thorough analysis that led to concrete infrastructure and communication improvements.
This case study examines a data‑service company's transition to a microservice architecture, detailing the initial benefits such as improved visibility and reduced deployment cost, the subsequent explosion of complexity, queue‑head blocking, shared‑library versioning issues, and the trade‑offs that led the team to partially revert to a monolithic design.
This article systematically explores high‑availability system design from development standards, capacity planning, application services, storage, product strategies, operations deployment, to incident response, presenting key concepts, architectural patterns, and practical guidelines for building resilient services.
This article explores Baidu's intelligent testing practices, covering spectrum‑based root‑cause localization, error‑code driven build‑system diagnostics, revenue‑change stop‑loss decision workflows, and search UI case‑level tracing, illustrating how data, algorithms, and engineering combine to reduce manual effort and accelerate issue resolution.
This article analyzes common problems in a Prometheus‑Alertmanager monitoring setup—such as alert noise, lack of escalation, suppression and silence management—and presents a comprehensive refactor that introduces per‑cluster Alertmanager instances, custom escalation logic, suppression tables, and Python scripts to handle alert routing, silencing, and recovery.
The article explains how platform engineering, emerging from DevOps fatigue, unifies development and operations by providing internal developer platforms that reduce cognitive load, improve self‑service, and enable teams to focus on core product work, especially as organizations grow beyond twenty developers.
This article details Tencent CDN's challenges and solutions for business continuity, covering bandwidth and device resource constraints, massive request handling, fault‑management lifecycle, automation bottlenecks, and the implementation of AIOps, intelligent alerts, capacity planning, and root‑cause analysis to ensure reliable service.
This article introduces the evolution of operations automation, highlights the power of concise shell scripts, and presents a 70‑page handbook covering fundamentals to advanced topics together with 100 ready‑to‑run script examples for Linux system administration and DevOps tasks.
This article explains how to design effective Prometheus metrics, choose appropriate vectors, labels, buckets, and naming conventions, and offers Grafana usage tricks to help engineers monitor online services, batch jobs, and offline processing systems with clear, actionable insights.
This article explains why ZooKeeper can run out of disk space due to excessive snapshots and transaction logs, describes the underlying file‑generation mechanism, and provides concrete configuration parameters and best‑practice recommendations to control file growth and keep the cluster stable.
This article chronicles Bilibili's five‑year evolution of Site Reliability Engineering, detailing the introduction of SRE culture, the construction of high‑availability and multi‑active architectures, capacity management with Kubernetes, VPA/HPA, incident case studies, and the ongoing transformation of SRE practices across the organization.
This article details Alibaba Cloud's Flink Cluster Inspector, explaining the business challenges of hotspot machines, the analysis of resource over‑use, and the four‑stage solution—pre‑profiling, in‑process self‑healing, post‑recovery, and observability—that reduces latency, cuts costs, and improves operational efficiency.
This guide details how to configure Alibaba Cloud Log Service alarms—setting one‑day tokens, handling 1024‑byte truncation, removing record limits with analysis statements, adding a 10‑second query offset for timeliness—and shows how to visualize the data in Grafana using SQL queries for multi‑line and pie charts with timestamp conversion and time‑series filling.
Vivo’s Database‑as‑a‑Service platform tackles cloud‑native challenges by automating massive instance management with self‑service work orders and self‑healing, enabling elastic scaling through mixed‑deployment and multi‑threaded Redis tools, optimizing costs via automatic package shrinkage, and safeguarding personal data with full‑chain encryption, while outlining a roadmap toward AI‑driven fault handling, container‑based resources, and advanced privacy governance.
This guide explains how to push a Kubernetes cluster toward its theoretical limit of 5,000 nodes by detailing official limits, master node sizing for GCE and AWS, kube‑apiserver high‑availability and connection‑count tuning, scheduler and controller‑manager leader election settings, kubelet optimizations, and DNS anti‑affinity configuration.
This guide explains how to collect essential Windows metrics with Prometheus, set up Grafana dashboards for CPU, memory, disk, network, and process monitoring, and configure alert rules, while also comparing self‑hosted and Alibaba Cloud Prometheus solutions for seamless Windows observability.
Learn essential techniques for designing fault‑tolerant microservices, including graceful degradation, change management, health checks, self‑healing, failover caching, retry strategies, rate limiting, circuit breakers, and testing failures, to ensure high availability and reliability in distributed cloud‑native systems.
The article outlines a complete data‑quality‑management framework that first captures business expectations, then translates them into basic and personalized measurement rules, defines four assurance approaches for handling violations, and scales operation with indicators, tooling, and metrics to continuously improve data quality across the lifecycle.
The article details Qunar's user‑experience digital platform, explaining its background, measurement model, metric scoring, data‑collection mechanisms for Android and iOS, and the operational plan that drives continuous improvement across teams and products.
This article explains the motivations for selecting Grafana Loki instead of ELK/EFK, introduces its core concepts and features, provides step‑by‑step deployment instructions for Promtail and Loki, and demonstrates how to configure Grafana, query logs, and handle label indexing, dynamic tags, and high‑cardinality challenges.
The article explains the 10×‑better theory, introduces Dapr as a cloud‑native sidecar framework, and details how it improves productivity for developers, enhances security, resilience and observability for operators, and offers multi‑language, multi‑environment flexibility for architects, while also acknowledging its drawbacks.
This guide shares practical operations best practices, covering safe online procedures, data protection, security hardening, daily monitoring, performance tuning, and the right mindset to avoid costly mistakes and keep production environments stable and secure.
This article explains how RocketMQ's message lifecycle can be fully observed using OpenTelemetry‑based metrics, covering producer, broker, and consumer stages, and shows practical monitoring, alerting, and troubleshooting practices for cloud‑native deployments.
This comprehensive guide explains how metadata connects source data, warehouses, and applications, outlines its technical and business classifications, demonstrates its value for data management, profiling, portals, and ETL development, and details optimization, storage, lifecycle, and quality practices essential for robust big‑data operations.
This article presents a comprehensive, three‑part guide on the current state of full‑stack operations monitoring, common challenges and solutions, and a real‑world use case, illustrating how Alibaba Cloud Elasticsearch can improve observability, boost performance, and cut costs for complex distributed systems.
This article presents a comprehensive overview of Volcano Engine's A/B testing platform, detailing its four core stages—reliable experiment system, efficient data construction, scientific statistical analysis, and fine-grained governance—while explaining execution components, data pipelines, statistical methods, and operational best practices for large‑scale experimentation.
This article explores the unique challenges of front‑end operations, outlines typical build failures caused by dependency, environment, and permission mismatches, and proposes Docker‑based DevOps solutions to streamline development, testing, and production workflows for full‑stack teams.
The incident postmortem guideline advocates a dialectical view of failures, rapid low‑severity recovery, and a structured process—covering background, impact scope, timeline replay, deep root‑cause analysis, SMART improvement actions, responsibility assignment, and PDCA‑validated closure—to enhance system resilience, team anti‑fragility, and knowledge sharing.
This guide explains how to use the powerful Linux tar command for full and incremental backups, including common options, exclusion patterns, splitting large archives, automated scheduling with cron, and reliable restoration of files and entire filesystems.
An in‑depth case study of a data‑service company's transition to microservices details the initial benefits such as visibility and reduced deployment cost, the subsequent problems of queue head blocking, shared‑library versioning, scaling complexity, and the eventual trade‑offs that led to a partial monolith rollback.
Guoxin Securities’ QianKun centralized operation platform showcases a cloud‑native, micro‑service architecture that achieved 99.95% availability through containerization, multi‑region deployment, AI‑driven capacity forecasting, and comprehensive DevOps practices, offering a 24/7 seamless account‑opening experience and setting industry benchmarks.
This article explains what maturity models are, why they matter for evaluating and improving organizational processes, reviews common business process maturity models (BPMM) and their limitations, introduces the Capability Maturity Model (CMM) and the Agile ISO Maturity Model (AIMM), and offers guidance on selecting and applying a suitable model.
Effective incident management begins with clear classification; this guide explains how technical leaders can categorize online failures by nature, severity, and source—distinguishing usability versus financial loss incidents, ranking P0‑P3 levels, and identifying external, operational, product, and system‑quality fault types—to improve stability and learning.
This case study examines how Google decommissioned its legacy file‑server backend, focusing on the design, management, and migration of the main directory, and highlights the operational lessons and SRE practices that ensured a smooth transition without service disruption.
This article details Zuoyebang's journey from a single‑cloud setup to a multi‑cloud, multi‑active architecture, covering business drivers, design principles, network planning, compute and storage strategies, traffic scheduling, container migration, operational management, and the measurable cost, stability, and efficiency benefits achieved.
This article explains why modern organizations replace manual cron jobs with automated digital workflow management using Apache Airflow, detailing its troubleshooting, flexibility, monitoring, rich web UI, CLI/API, complex dependency handling, scalability, containerization, and extensibility through plugins and integrations.
The article details Liaoning Mobile's award-winning DevOps transformation, describing the team's background, agile implementation, toolchain construction, challenges faced, system highlights, and measurable results that earned them the prestigious Communication Industry DevOps Team Award at the 2022 GOITI ceremony.
This article explains software delivery pipelines, covering their components, deployment stages, continuous integration and delivery practices, and release strategies such as blue/green and canary deployments, highlighting how automation improves reliability and reduces risk.
This article outlines the Kubernetes pod lifecycle, describes the five pod phases, enumerates 13 typical failure scenarios—including scheduling, image pull, dependency, init container, probe, and OOM issues—provides error states, root causes, and step‑by‑step kubectl commands for diagnosis and remediation.
This article explains the TiDB rolling upgrade method, describing TiDB's architecture, key components, relevant terminology, the TiUP tool, and a step‑by‑step core upgrade workflow with code snippets and configuration adjustments to minimize service interruption.
This guide explains how to analyze and troubleshoot situations where Linux systems show high load averages despite low CPU usage, covering common load analysis methods, key commands like top, vmstat, iostat, sar, and ps, and practical solutions for I/O bottlenecks and D‑state processes.
In this interview, Novant Health’s Executive Vice President and Chief Transformation & Digital Officer Angela Yochem explains how CIOs must shift the narrative from viewing IT as a cost center to recognizing its essential role in digital health operations, new revenue streams, and strategic business growth.
This article details how a large‑scale membership platform achieves high performance and near‑zero downtime by employing dual‑center Elasticsearch clusters, traffic‑isolated ES architectures, deep ES optimizations, Redis caching with distributed locks, and a seamless MySQL migration with partitioned, dual‑center databases.
This article provides a systematic overview of monitoring fundamentals, compares three popular open‑source monitoring solutions—Zabbix, Open‑Falcon, and Prometheus—and offers practical guidance for selecting the most suitable system based on scale, features, and operational needs.
This article explains why disk‑space alerts demand automated handling, introduces the concept of fault self‑healing, outlines required process standards, describes monitoring platform dimensions, details a multi‑source self‑healing platform architecture, and offers practical steps for integration, notification, and continuous improvement.
This guide details key Redis monitoring metrics—including performance, memory, activity, persistence, and error indicators—along with practical commands, configuration settings, and code snippets to help operators efficiently track and troubleshoot Redis instances.
This article examines the evolution of a monitoring platform, identifies key challenges such as alarm overload and fragmented data, and presents a solution that combines Grafana with Diagram and FlowCharting plugins to create business‑process‑oriented, data‑driven visualizations for faster issue resolution.
This guide walks you through the essential steps, commands, and precautions for safely draining nodes, backing up applications, CRDs, and etcd, then shutting down and later restarting a Kubernetes cluster while avoiding common pitfalls.
This article outlines eBay's key scalability best practices—including functional decomposition, horizontal sharding, avoiding distributed transactions, asynchronous decoupling, virtualization, and intelligent caching—to demonstrate how large‑scale web systems can achieve linear resource growth and high availability.
This guide examines the challenges and benefits of competing alone in mathematical modeling contests and offers practical pre‑competition and in‑competition preparation tips to help solo participants manage workload, maintain motivation, and produce a complete, well‑written solution.
This article explains how to achieve load balancing using Nginx, covering the concepts of hardware and software load balancers, various Nginx balancing algorithms with configuration examples, and a step‑by‑step guide to integrate Nginx with a SpringBoot application, test it, and handle common pitfalls.
The two‑day 2022 GOPS Global Operations Conference in Shanghai featured 16 tracks, over 80 speakers, new DevOps standards, extensive assessment results, and a wealth of sessions on DevOps, AIOps, cloud‑native practices, security, and industry case studies, offering a comprehensive snapshot of modern operations engineering.
The rolling deployment strategy incrementally replaces old application instances with new ones, allowing users to encounter both versions during rollout, and is praised for ease of implementation, low risk, and default support in platforms like Kubernetes, though it can be slow, costly for large infrastructures, and may affect user experience.
China Minsheng Bank’s centralized operation business processing system passed the Level 2 technical operation assessment of the national DevOps maturity model, showcasing how standardized DevOps practices, continuous delivery pipelines, and cross‑team collaboration can boost efficiency, safety, and competitiveness in the banking sector.
This article shares practical security best practices for operations teams, covering why security is often overlooked, real incident examples, and detailed guidelines on port hardening, system hardening (login management, vulnerability scanning, baseline checks), application, network, and data protection, emphasizing continuous investment and simple safeguards.
This guide explains what Kubernetes events are, how to list and filter them, categorizes common event types, and shows practical ways to collect, store, and alert on events using native commands and open‑source tools, helping teams reduce alert fatigue and improve cluster observability.
Liaoning Mobile’s Channel Management System project passed the CAICT DevOps Capability Maturity Model Level‑3 continuous delivery assessment, showcasing how standardized DevOps practices, toolchains, and agile transformation boosted delivery speed, team capability, and operational efficiency, positioning the carrier at the forefront of China’s digital transformation.
This article analyzes the difficulties of initializing stable environments, the poor usability and low reliability of feature environments, proposes concrete solutions such as unified test environments, streamlined creation workflows, middleware adjustments, testing and documentation, and shares practical reflections from real deployments.
A government information system suffered a week of instability, including service deadlocks, Tomcat memory overflows, and load‑balancing failures, prompting a deep forensic analysis that uncovered database lock‑ups, faulty front‑end loops, inadequate monitoring, and misconfigured logging, leading to concrete remediation steps and lessons for future reliability.
This article details the step‑by‑step evolution of a multi‑cloud network at Zuoyebang, covering three construction phases, quality‑improvement measures such as fault prevention and rapid recovery, and ongoing operational governance that together deliver a flexible, high‑availability cloud infrastructure.
The article examines the inherent and accidental complexities of Kubernetes as a distributed cluster manager, discusses challenges in resource scheduling, infrastructure diversity, and operational overhead, and explores how cloud‑native solutions such as managed services, nodeless and serverless Kubernetes architectures attempt to reduce these complexities while introducing new trade‑offs.
The article clarifies the distinctions between platform engineering, DevOps, and SRE, explaining their origins, common misconceptions, challenges such as shadow operations and developer cognitive load, and how platform engineering builds on these practices to deliver self‑service internal developer platforms that improve productivity and reliability.
This comprehensive guide explains how to configure Nginx as an HTTP server, static file server, reverse proxy, and load balancer, covering directory setup, location matching rules, priority order, upstream strategies, and useful directives such as return, rewrite, error_page, logging and access control.
The article examines platform engineering as a discipline that unifies developers' desire to avoid infrastructure work with enterprises' need for control, critiques the hype around DevOps, and argues that effective internal developer platforms require solid fundamentals, IaC practices, and cultural change.
This article presents a practical three‑step methodology—understanding, managing, and preventing—to troubleshoot Kubernetes deployments, explains how to leverage monitoring, observability, and incident‑response tools, and offers guidance on fostering team collaboration and building resilient, self‑healing cloud‑native systems.
This article reports how 21 Chinese banking institutions evaluated 82 projects using the CAICT-led DevOps Capability Maturity Model, detailing the breakdown across state‑owned, joint‑stock, and city commercial banks, and explains the model’s standards and industry impact.
The article examines growing developer frustration with operational responsibilities, the perceived decline of DevOps, and how platform engineering and Site Reliability Engineering are emerging as new approaches to balance development speed with reliable operations in cloud‑native environments.
The article explains how microservice architectures can achieve high availability by isolating failures, employing graceful degradation, change‑management strategies, health checks, fallback caching, retry logic, rate limiting, circuit breakers, and chaos testing, while acknowledging the added complexity and cost of such reliability engineering.
