The article examines the challenges of moving large, multi‑team software organizations from waterfall to lean‑agile practices, offering concrete tactics for product planning, cross‑team coordination, integration, testing, and release, and concludes with a note on an upcoming DevOps hackathon.
This article examines Meituan's five‑year evolution of instant logistics, detailing the distributed, high‑concurrency architecture, AI‑driven optimization, scalability techniques, fault‑tolerance mechanisms, and future challenges faced by its real‑time delivery platform.
This article presents a comprehensive, data‑driven case study of how Tencent’s internal Kubernetes/TKE platform reduced monthly CPU usage by up to 70% and memory usage by 50% through systematic cost data collection, VPA/HPA enhancements, custom scheduling, node‑level over‑commit, and safe node decommissioning, while maintaining zero‑incident reliability.
In a Stack Overflow Podcast interview, former GitHub CTO Jason Warner recounts his unconventional path from a Connecticut farm to leading massive platform scaling, shares insights on engineering leadership, product strategy, venture investing, and the future of cloud, data, and blockchain technologies.
This article translates and expands on a HikariCP wiki post, explaining why a small database connection pool often yields better performance than a large one, presenting benchmark data, a practical sizing formula, and guidance for tuning pools in various environments.
This article shares G7 Tech’s practical experience of exposing Kafka to the public internet, covering encryption, AAA, three authentication schemes, listener configuration, scaling for massive topics with Kubernetes, storage optimization, and integration with the gmq management platform and Kafka‑REST.
The article explains how to build highly available stateless services by using redundant deployment, vertical and horizontal scaling, appropriate load‑balancing algorithms, monitoring, and automated recovery, and also discusses high‑concurrency identification, CDN/OSS usage, and practical recommendations for cloud‑native environments.
PayPal’s engineering team detailed their journey of scaling Kubernetes from a few hundred nodes to over 4,100 nodes and 200,000 Pods, describing cluster topology, workload generation, API server bottlenecks, controller manager and scheduler tuning, extensive etcd optimizations, and the resulting performance gains that met Kubernetes SLOs.
This article provides a comprehensive technical guide on Redis performance, covering single‑threaded characteristics, master‑slave replication, Sentinel high‑availability mechanisms, Redis Cluster architecture, configuration steps, code examples, and alternative middleware solutions for scaling and fault tolerance.
This article details Car Home's AutoStream platform evolution from Storm to Flink‑based versions, covering real‑time application scenarios, strict budget‑controlled resource management, automatic scaling, lake‑house architecture with Iceberg, PyFlink integration, and future plans for resource optimisation and batch‑stream unification.
This article details how a membership system serving billions of users achieves high performance and high availability through a dual‑center Elasticsearch cluster, traffic‑isolated ES clusters, Redis cache with distributed locks, MySQL dual‑center partitioning, and fine‑grained flow‑control and degradation mechanisms, all while ensuring zero‑downtime migrations and consistent data.
This article explains how to design data architecture for microservice systems, covering the advantages of microservices, decoupling principles, lightweight APIs, DevOps integration, database per service versus shared databases, polyglot persistence, and why MongoDB is a suitable choice for scalable, dynamic, and sharded data storage.
PayPal’s engineering team detailed their journey of scaling Kubernetes from a few hundred nodes to over 4,000 nodes and 200,000 pods, describing the cluster topology, workload generation, bottlenecks in the API server, controller manager, scheduler, and etcd, and the optimizations that enabled stable performance at massive scale.
Meituan’s instant logistics platform evolved from vertical services to a micro‑service, distributed architecture that handles massive order‑rider matching, ultra‑low latency, and high availability, leveraging AI for pricing, ETA, scheduling, and employing robust scaling, consistency, and disaster‑recovery techniques.
This guide walks through setting up an Undermoon Redis cluster using Docker Compose, covering prerequisites, building images, registering proxies, creating and scaling the cluster, and demonstrating automatic failover with concrete command‑line examples.
This article explains how to horizontally scale RabbitMQ clusters to handle millions of messages per second by leveraging cluster modes, mirror queues, sharding plugins, consistent‑hash exchanges, federation, and high‑availability configurations, while also covering practical scenarios such as retries, delayed tasks, and Spring AMQP integration.
On December 20 the Xi'an health‑code app "One‑Code Pass" suffered a massive outage as a sudden traffic surge overwhelmed its query‑heavy backend, exposing network bottlenecks and a lack of scaling mechanisms, prompting a detailed technical analysis and proposed architectural remedies.
This article explains the fundamentals of load balancing, covering its purpose, vertical and horizontal scaling, various classifications such as DNS, IP, link‑layer and hybrid methods, common algorithms like round‑robin and weighted, as well as hardware solutions, providing a comprehensive guide for building scalable, high‑availability systems.
This article explains the design of an ECS‑based Function‑as‑a‑Service platform, covering multi‑tenant deployment, rapid horizontal scaling, resource‑utilization optimization, avalanche‑prevention strategies, and high‑density deployment techniques that together enable fast, cost‑effective cloud‑native serverless workloads.
This guide explains how to scale a Redis cluster by deploying new nodes, using redis‑trib to add them, redistributing hash slots through resharding, and setting up a four‑master four‑slave cross‑replication topology while verifying cluster health with CLI commands.
This article explains how to expand a ClickHouse cluster by adding new replica nodes, detailing the use of the internal_replication parameter, configuration steps for ReplicatedMergeTree and MergeTree engines, and verification of data synchronization across replicas.
The article explains the reasons for scaling, outlines hardware and component expansion strategies, introduces the AKF splitting principle for distributed systems, discusses database clustering and distributed ID generation methods such as UUID and Snowflake, and describes elastic scaling challenges and solutions.
This article explains why capacity expansion is necessary, outlines hardware and component scaling strategies, introduces the AKF splitting principle for Redis clusters, discusses challenges of distributed scaling such as data consistency and high concurrency, and reviews database clustering and distributed ID generation methods like UUID and Snowflake.
The article explains why performance optimization has limits and outlines practical scaling methods—including whole‑machine and component upgrades, AKF splitting, database clustering, distributed ID generation (UUID and Snowflake), and elastic scaling—while also discussing the challenges each approach introduces.
This article explains why capacity expansion is necessary, outlines hardware and component scaling strategies, introduces AKF splitting principles, discusses database clustering and distributed ID generation methods such as UUID and Snowflake, and highlights the benefits and challenges of elastic scaling.
This article explains why database capacity planning is challenging and presents a systematic approach—including workload analysis, performance modeling, consistency considerations, and node scaling decisions—using the open‑source NoSQL database ScyllaDB to guide accurate capacity estimation.
This article explains why sizing a database cluster is challenging, outlines a step‑by‑step methodology for estimating workload, configuration and performance, discusses the impact of consistency levels, secondary indexes, materialized views and maintenance, and shows how ScyllaDB can be used to model and simplify capacity planning.
This article introduces various Redis scaling solutions—including basic partitioning, master‑slave replication, Sentinel high‑availability, and Redis Cluster—explaining their concepts, typical usage patterns, configuration commands, advantages, and drawbacks to help developers choose the right approach for high‑traffic environments.
This article explains the classification of relational and NoSQL databases, analyzes common performance bottlenecks such as query latency, large fields, and write overhead, and presents practical optimization methods including caching, proper indexing, transaction handling, read‑write separation, and sharding for large‑scale systems.
This article explains how to extend Redis beyond a single instance by covering partitioning, master‑slave replication, Sentinel automatic failover, and Redis Cluster, describing their usage methods, advantages, and drawbacks for high‑traffic, high‑availability scenarios.
This article explains how Facebook engineered a massive‑scale continuous delivery pipeline—evolving from manual cherry‑pick releases to a near‑continuous deployment system that safely pushes hundreds of code changes per hour to both web and mobile users worldwide.
This article explains how agile architecture differs from traditional approaches, outlines the full lifecycle of agile architecture, defines responsibilities, introduces the role of an architecture owner, and provides practical guidance for modeling, scaling, communicating, and evolving architecture in large‑scale agile projects while avoiding over‑engineering.
This guide walks through essential Kubernetes operations such as isolating and recovering nodes, expanding clusters with new nodes, dynamically scaling Pods, managing Labels, scheduling Pods to specific Nodes, performing rolling updates, and configuring high‑availability for etcd and Master components, all with concrete command‑line examples and YAML snippets.
This article explains how to design highly available stateless services by covering redundancy, vertical and horizontal scaling, load‑balancing algorithms, high‑concurrency identification, and the use of CDN/OSS, offering practical guidance for robust backend architecture.
This article provides a comprehensive analysis of Redis Cluster, covering node and slot assignment, command execution, resharding, redirection, fault‑tolerance, gossip communication, scaling strategies, configuration limits, and practical code examples for building and operating a high‑availability sharded Redis deployment.
This article explains how to build highly available stateless services by using redundant deployment, vertical and horizontal scaling, various load‑balancing algorithms, and automatic recovery mechanisms, while also covering monitoring, high‑concurrency identification, and the role of CDN and OSS in resilient architecture.
This article provides a concise yet comprehensive overview of Kubernetes, covering its core architecture, the workflow of creating deployments, pod fundamentals, scaling and rolling updates, networking basics, service discovery, and external access methods such as NodePort, LoadBalancer, and Ingress.
This article explains how to build highly available stateless services by covering redundant deployment, load‑balancing algorithms, vertical and horizontal scaling, CDN/OSS usage, and practical recommendations for monitoring, automated recovery, and minimizing failure impact.
This article explains how Alibaba Cloud Function Compute (FC) was integrated into internal services to support the Double‑12 promotion, detailing the business requirements, network challenges, hybrid‑cloud solutions, container types, auto‑scaling, rate‑limiting, and the serverless platform’s deployment and gray‑release mechanisms.
This tutorial walks through building a Redis Cluster on Redis 6.0+, covering node startup, handshaking, slot assignment, master‑slave replication, command routing, failover handling, and practical scaling operations such as adding, rebalancing, and removing nodes using redis‑cli commands.
This article provides a step‑by‑step tutorial on building a Redis Cluster on a single server, covering node configuration, cluster handshaking, slot assignment, master‑slave replication, command routing, failover handling, and practical scaling operations such as adding and removing nodes using redis‑cli.
This article explains how feature engineering transforms raw data into numerical representations for machine‑learning models, covering mapping of numeric and categorical values, one‑hot and multi‑hot encoding, sparse representations, scaling, handling outliers, binning, data quality checks, and feature interactions to capture non‑linear relationships.
This tutorial walks through building a Redis 6+ cluster on a single server, covering node startup, cluster handshake, slot assignment, master‑slave replication, command routing, failover handling, and practical scaling operations such as adding and removing nodes.
This tutorial walks through setting up a Redis Cluster on a single server with six nodes, covering node startup, handshake, slot assignment, master‑slave replication, command routing, fault‑tolerance, and practical scaling operations such as adding, rebalancing, and removing nodes.
This tutorial walks through setting up a Redis Cluster on a single server with six nodes, covering node startup, handshake, slot assignment, master‑slave replication, command routing, failover handling, and practical scaling operations such as adding, rebalancing, and removing nodes.
The article humorously maps various relationship scenarios to core IT concepts such as backup strategies, high‑availability mechanisms, scaling methods, security measures, cloud services, and big‑data techniques, providing an engaging overview of fundamental system design principles.
This article explains how JD Cloud’s JCHDB database handled PB‑level data growth during the 11.11 shopping festival, detailing the high‑availability architecture, performance optimizations, scaling techniques, and the eight‑step preparation process that enabled millions of queries per second and terabit‑level traffic.
This article explores the challenges and solutions for implementing double-tap zoom functionality in touch-enabled devices, addressing common problems with existing methods and proposing a mathematical approach to achieve accurate scaling and panning.
This article introduces Kubernetes fundamentals, explains its production‑grade nature, container concepts, orchestration features, core architecture, and provides step‑by‑step commands for installing Minikube, creating a cluster, deploying an Nginx application, exposing it as a service, scaling, updating, and deleting the deployment.
During the pandemic, Alibaba accelerated YuEBao from a single user to over a hundred million in a year, built the health code system in days, increased R&D efficiency by 20‑30%, and outlined its cloud‑native, AI‑driven technology roadmap and the engineer qualities it values most.
This article provides a comprehensive, top‑down and bottom‑up explanation of Elasticsearch’s internal architecture, covering clusters, nodes, shards, Lucene segments, inverted indexes, stored fields, document values, caching, merging, routing, scaling, and query processing, while addressing common performance questions.
This guide details the end‑to‑end process for performing a 50,000‑user concurrent load test with JMeter and BlazeMeter, including script creation, local validation, sandbox execution, engine scaling, cluster testing, and Master/Slave configuration to achieve the required concurrency.
This article explains common I/O and CPU bottlenecks in databases, compares horizontal and vertical sharding techniques, outlines practical partitioning strategies, introduces popular sharding tools, and provides step‑by‑step guidance for implementing and scaling sharded architectures.
This article explains common I/O and CPU bottlenecks in databases, introduces horizontal and vertical sharding concepts, compares sharding tools, outlines practical sharding steps, discusses typical sharding issues such as non‑partition queries and expansion, and provides a concise summary and example implementation.
The article details how New Oriental's Cloud Classroom backend, built with Java, Spring, MySQL, Redis, Kafka, Sentinel, and other modern technologies, scaled to support millions of users and a hundred‑fold surge in demand during the pandemic through architectural optimizations, distributed caching, traffic control, and rapid performance improvements.
This article outlines the design, functional requirements, CRD definition, architecture, deployment, scaling, monitoring, fault‑tolerance, and upgrade strategies of a Zookeeper operator on Kubernetes, including code examples, service configurations, and integration with Prometheus and OAM standards.
This article explains how to extend Redis beyond single‑node limits by using partitioning, master‑slave replication, Sentinel for automatic failover, and Redis Cluster with hash slots, detailing their usage, advantages, drawbacks, and configuration examples for building high‑availability and scalable in‑memory data stores.
This article explains how Tencent Cloud’s Serverless Cloud Function (SCF) leverages Cloud Disk Snapshot technology to achieve the concurrent creation of 6000 virtual machines within a minute, detailing the snapshot‑based creation method, system architecture, performance challenges, and the engineering solutions that dramatically improve latency and bandwidth usage.
This article explains how to design and scale a RabbitMQ cluster capable of handling millions of messages per second, covering core concepts, Google’s large‑scale test setup, sharding and federation plugins, mirror queues, reliability mechanisms, and practical tips for high‑availability and performance optimization.
This article explains common database performance bottlenecks such as IO and CPU limits, introduces horizontal and vertical sharding concepts, compares sharding tools, outlines practical implementation steps, discusses common pitfalls, and provides scaling strategies to keep your data layer responsive under heavy load.
This article explains why Redis performance matters for high‑traffic services and provides a comprehensive set of practical optimizations—including network latency reduction, command pipelining, avoiding slow commands, tuning persistence, OS/hardware settings, and scaling with sharding—to help you keep Redis fast and reliable.
This article explains how to design and implement a cloud‑native, large‑scale distributed monitoring system using Prometheus, covering its limitations, service‑level sharding, centralized storage, federation, and high‑availability strategies to overcome scaling challenges in Kubernetes environments.
This article explains how Tencent Meeting leveraged Tencent Cloud’s Kubernetes Engine (TKE) and a suite of cloud‑native extensions—dynamic routing, fixed networking, parallel scaling, and controlled batch upgrades—to achieve rapid, reliable version iteration and massive capacity growth during the COVID‑19 surge.
Segment’s engineering team recounts their evolution from a simple monolith to a sprawling micro‑service ecosystem and back again, detailing queue bottlenecks, repo fragmentation, shared‑library chaos, and how consolidating everything into a single codebase restored performance, scalability, and developer productivity.
During the COVID‑19 pandemic, Tencent Classroom’s front‑end team faced unprecedented traffic spikes, forcing rapid decisions on domain stability, video streaming, data platforms, messaging, monitoring, and deployment pipelines, while sharing lessons on scaling, resilience, and collaborative development under extreme pressure.
This guide explains how Kubernetes StatefulSet solves the challenges of deploying stateful applications by providing stable network identities, persistent storage, ordered scaling, and flexible update strategies, and walks through example manifests, creation commands, status inspection, upgrade procedures, and scaling policies.
This article provides a comprehensive guide on designing Elasticsearch indices for massive data volumes, covering shard and replica sizing, mapping strategies, rollover templates, curator cleanup, tokenization choices, query type selection, and multi‑table association techniques to achieve efficient, reliable search at PB‑scale.
This article explains how e‑commerce promotions differ from offline sales by offering lower participation thresholds and flexible discount tactics, outlines methods for estimating and handling traffic spikes, and provides detailed strategies for system capacity planning, load testing, monitoring, and incident response to ensure stable large‑scale promotional events.
This article explains how to evolve a simple single‑node backend by adding a reverse proxy, introducing load balancers, scaling databases, adopting micro‑services, leveraging caches and CDNs, using message queues, and applying partitioning techniques to handle massive traffic while maintaining consistency and reliability.
This article explains what Kubernetes Operators are, how they extend the controller pattern to encode software‑specific knowledge, illustrates practical examples, examines resource consumption and scaling of operators, and discusses their suitability for serverless models within cloud‑native environments.
Youzan scales demand backlog management from a single product to multiple lines by aligning OKR‑driven strategic goals with stakeholder inputs, centralizing ownership in a product‑owner‑led backlog that integrates information, uses user‑story, impact‑mapping and MoSCoW prioritization, employs fixed Scrum/Kanban cycles, splits large backlogs by domain, and leverages electronic kanban tools while continuously refining granularity and value‑loop closure.
The article presents a systematic capacity‑estimation methodology that links service traffic to order volume, uses CPU‑Idle as a primary metric, predicts traffic growth and upper‑bound limits, validates predictions with load‑testing, and provides scaling recommendations while noting limitations of the CPU‑Idle baseline.
This comprehensive guide details how Codis extends Redis with a proxy‑based architecture to achieve transparent horizontal scaling, smooth data migration, high availability, fault tolerance, and operational best‑practices, while also covering common Redis pitfalls and performance tuning.
This article explains how Redis Cluster distributes data across 16,384 hash slots, outlines its high performance, availability, and scaling advantages, and details key limitations such as client support, single‑database restriction, and multi‑key operation constraints with practical hash‑tag solutions.
This article explains how IO and CPU bottlenecks increase active database connections, describes horizontal and vertical sharding techniques, outlines practical steps and tools for implementing sharding, and discusses common challenges such as non‑partition key queries and expansion strategies.
This article chronicles Xianyu’s journey from a modest tea‑room startup to a platform with tens of millions of daily active users, detailing each architectural phase—trial, development, platform, and cloud‑native integration—and the technical decisions that enabled rapid scaling, cross‑platform development, and operational efficiency.
This comprehensive guide covers Kafka's core concepts, design principles, installation steps, configuration tweaks, performance optimizations, permission management, common operational commands, cluster scaling, log retention settings, and monitoring scripts to help you build and maintain a robust Kafka ecosystem.
This article presents a distributed system that efficiently supports large‑scale n‑gram language models for automatic speech recognition by introducing caching, a two‑level distributed index, batch processing, and a cascading fault‑tolerance mechanism, demonstrating robust scalability and low communication overhead in Tencent's WeChat ASR service.
This article explains how the WTable distributed key‑value store leverages RocksDB’s LSM‑tree architecture and slot‑based data distribution to redesign its scaling workflow, separating full and incremental data migration to reduce compaction overhead and achieve high‑speed, low‑impact cluster expansion.
The article recounts Segment's journey from a monolithic system to a sprawling micro‑service architecture, the operational pain points that emerged, and how consolidating over 140 services into a single codebase improved testing speed, deployment simplicity, and overall developer productivity while revealing new trade‑offs.
This article recounts AdMaster's five‑year journey from a few dozen servers to thousands, detailing the evolution of their monitoring infrastructure, the challenges faced at each scale stage, and the design of a self‑built, distributed monitoring platform that delivers real‑time alerts, visualized data, and business‑level insights.
This article shares AdMaster's five‑year operations journey, detailing how the team scaled monitoring from under 200 machines to over a thousand, the evolution of their monitoring stack, the design of a custom distributed system, and practical Q&A insights for large‑scale infrastructure management.
This article explores how enterprises can effectively use Redis by partitioning clusters for independent or shared use, addressing key naming conflicts, implementing graceful scaling with Zookeeper, monitoring performance via Open-Falcon, and quickly isolating problematic business traffic to maintain system stability.
This guide walks you through creating a Kubernetes cluster on Alibaba Cloud, installing Istio and Knative components, deploying a Go Hello World service, and using scripts and load‑testing tools to observe Knative’s zero‑scale‑to‑one and automatic scaling behavior.
This article explains how Apache Flink uses state for incremental stream processing, describes the different state backends, details the persistence mechanism, and shows how both OperatorState and KeyedState are redistributed during scaling using partition and key‑group algorithms.
The article recounts the Forbidden City’s first night‑time Lantern Festival event, the overwhelming demand that caused the museum’s ticketing website to crash, and includes an interview with a senior operations engineer who explains the causes of such overloads and outlines rapid mitigation and scaling strategies.
Database bottlenecks often require sharding; this guide explains core concepts of database partitioning, global ID generation methods, shard allocation strategies, smooth scaling techniques, and popular solutions, helping engineers decide when and how to implement effective sharding without premature over‑design.
This article details CTrip’s experience of containerizing a 200 TB+ Redis deployment with millions of queries per second, covering the motivations, architecture, Kubernetes strategies, performance testing, operational challenges, and the practical solutions they devised to achieve high scalability and resource efficiency.
This article presents a comprehensive knowledge map of MySQL deployment scenarios—including single‑master, master‑slave, master‑multiple‑slaves, horizontal and vertical clustering, and mixed modes—detailing backup methods, performance tuning, scaling strategies, and high‑availability considerations.
Microsoft's Core Services Engineering (CSE) team transformed from a waterfall development model to an agile, DevOps‑driven process using Visual Studio Team Services, introducing engineering fundamentals, a four‑level maturity model, a scaled agile framework, and a rotating Directly Responsible Individual role to accelerate delivery, improve quality, and enhance customer satisfaction.
Since 2017, Coinbase has faced rapid cryptocurrency‑driven traffic growth, prompting a series of backend engineering improvements—including database upgrades, monitoring enhancements, relationship refactoring, caching, and a custom traffic capture‑replay system—to ensure reliability and scalability during demand spikes.
To address growing business complexity and throughput demands, this article outlines systematic approaches to decompose monolithic systems—covering horizontal and vertical scaling, application and database sharding, service governance, caching, and the evolution toward microservices—highlighting practical techniques and real-world experiences.
The article critiques how large‑scale agile frameworks like SAFe undermine traditional PMOs, arguing that their top‑down, process‑heavy approach erodes product‑team autonomy, stifles innovation, and often serves more as a marketing gimmick than a genuine path to agile transformation.
The article summarizes the GIAC Shenzhen conference, highlighting front‑end, mobile, and backend optimization techniques, large‑scale architecture designs, cloud‑native solutions, big‑data testing strategies, and quality assurance practices shared by top Chinese internet firms.
Tencent Cloud’s senior engineer Yang Yuan explains how their managed Kafka service tackles version diversity, resource allocation, dynamic scaling, broker addition/removal, and partition migration using versioned clusters, bin‑packing algorithms, penalty weighting, and predictive scheduling to sustain trillions of messages and billions of messages per minute.
This article explains how to perform essential front‑end image processing with the HTML5 canvas—handling cross‑origin loading, ensuring images are fully loaded, then scaling, cropping, and rotating them while preserving aspect ratio and exporting the results as base64 strings, laying groundwork for later composition techniques.
This article outlines Didi's MySQL database architecture, the challenges of managing thousands of instances, and the step‑by‑step automation framework—including dbproxy, high‑availability, backup, monitoring, and deployment modules—that reduces manual DBA work by over 70%.
This article details how Tongcheng Network migrated massive ticketing and flight‑booking workloads to TiDB, combining sharding and a distributed SQL database to handle terabytes of data, real‑time monitoring, and high QPS traffic while simplifying operations and reducing costs.
The article describes how a Go‑based backend, using a two‑layer job/worker pattern with buffered channels and configurable worker pools, can reliably ingest millions of POST requests per minute, serialize payloads to Amazon S3, and dramatically reduce server count through Elastic Beanstalk auto‑scaling.
This article explains why MySQL can become a bottleneck as load grows and presents seven practical techniques—using EXPLAIN, building proper indexes, tweaking defaults, caching data in memory, adopting SSDs, scaling horizontally, and improving visibility—to keep MySQL fast and reliable.
