0 views collected around this technical thread.
The article reviews Didi’s 12‑hour P0 outage caused by a Kubernetes upgrade failure in a massive cluster, discusses the root causes, and proposes general solutions such as federation, careful upgrade planning, and multi‑master designs to avoid similar incidents.
This article provides an in-depth overview of Kafka, covering its core value as a message queue, fundamental concepts, cluster architecture, producer and consumer configurations, scaling strategies, monitoring tools, and practical operational commands for building and maintaining high‑throughput, highly available streaming systems.
This article provides a step‑by‑step guide for scaling an OceanBase cluster, covering both white‑screen and black‑screen methods to add zones (replicas) and resources (OBServers), including configuration file preparation, deployment commands, zone addition, verification queries, and procedures for both expansion and contraction.
This guide explains how to adjust node quotas, kernel parameters, and etcd configurations for large Kubernetes clusters, covering cloud provider limits, GCE and Alibaba Cloud settings, API server tuning, and pod resource best practices to ensure reliable scaling and performance.
The article presents the architecture of WLock, a Paxos‑based distributed lock service, analyzes key isolation schemes, evaluates cluster expansion and splitting, and details a multi‑step key migration process—including forward and reverse migration, node scaling, and consistency safeguards—to achieve high‑availability and isolated lock handling in multi‑tenant environments.
Bilibili’s Presto on Hadoop combines a multi‑engine offline platform with Kubernetes‑managed YARN scheduling, Ranger security, and a custom dispatcher, scaling to over 400 nodes handling 160 k daily queries on 10 PB, while adding coordinator HA, resource‑group punishment, query limits, Alluxio caching, dynamic filtering, and numerous SQL‑level enhancements, with future auto‑scaling and materialized‑view automation.
This guide explains how to adjust node quotas, tune kernel parameters, configure high‑availability etcd clusters, and set optimal Kube‑APIServer and Pod settings for large‑scale Kubernetes deployments, ensuring stability and performance as the cluster grows.
This article examines how LinkedIn tackled severe scheduling slowdowns when its Hadoop YARN cluster grew to nearly 10,000 nodes, analyzes the root causes of resource‑manager bottlenecks, and describes the fairness‑redefinition and scheduling‑logic patches that restored throughput and scalability.
To sustain Vivo Push Platform’s massive real‑time traffic, engineers re‑architected two Redis clusters, trimmed capacity by 58 %, split clusters, randomized hotspot‑prone keys, and introduced three‑level caching, cutting peak CPU load by 15 %, halving response time and improving overall Redis efficiency during peak loads.
This article explains how to scale, back up, and upgrade Kubernetes clusters with Sealer by modifying the Clusterfile, using join/delete commands for both ALI_CLOUD and BAREMETAL providers, and configuring backup plugins and upgrade workflows.
This guide outlines how to prepare large‑scale Kubernetes clusters on public clouds by increasing node quotas, adjusting kernel parameters, configuring high‑availability etcd with the etcd‑operator, tuning kube‑apiserver settings, and applying pod‑level best practices for resource limits and affinity.
This article walks through a mid‑size gaming company's journey of deploying, tuning, and scaling an Elasticsearch cluster for massive log volumes, covering hot‑cold node architecture, ILM policies, shard management, Logstash‑Kafka optimization, emergency expansions, and the promise of searchable snapshots to achieve petabyte‑scale storage with cost efficiency.
To handle a gaming company's million‑QPS log stream, the team built a hot‑cold Tencent Cloud Elasticsearch cluster with ILM‑driven tiering, scaled CPU/heap, reduced shard count via shrink and replica tweaks, tuned Logstash‑Kafka pipelines, and employed COS snapshots and searchable snapshots, achieving stable performance and lower cost.
This article presents a comprehensive case study of 58.com’s massive Hadoop‑based offline computing platform, detailing its architecture, scaling challenges, performance‑tuning measures, YARN and SparkSQL upgrades, and the systematic cross‑data‑center migration of thousands of nodes and petabytes of data.
This article details how 58.com built a massive Hadoop‑based offline computing platform with over 4,000 servers and hundreds of petabytes of storage, addressing scaling, stability, GC, YARN scheduling, SparkSQL migration, storage operations, and a large‑scale cross‑datacenter migration.
This article details a real‑world RocketMQ case where expanding a topic's queue count caused two consumer groups to miss messages on one broker, explains the root cause of missing subscription metadata after cluster scaling, and outlines the manual steps taken to restore full consumption.
Meltwater details how it processes millions of daily media posts using a custom‑tuned Elasticsearch 1.7.6 cluster of over 400 nodes on AWS, covering data volume, query complexity, node configuration, indexing strategy, performance optimizations, and lessons learned for large‑scale search deployments.