Tagged articles
Cluster Optimization
12 articles · Page 1 of 1
This article details a systematic, end‑to‑end Kubernetes performance tuning plan—from kernel and container‑runtime tweaks on the node level to resource limits, scheduler policies, and pod‑level configurations—that can triple cluster throughput, cut latency by up to 80%, and dramatically improve stability.
Cluster OptimizationKubernetesPerformance Tuning
0 likes · 13 min read

This article explains Kubernetes resource management, covering compute and non‑compute resources, key mechanisms such as quotas and autoscalers, monitoring tools, and optimization strategies to improve cluster efficiency, scalability, and cost effectiveness.
Cluster OptimizationKubernetescloud native
0 likes · 19 min read

NetEase’s senior big‑data engineer shares how the company’s large‑scale data platform leverages Hadoop, HDFS, YARN and related technologies, detailing multi‑layer architecture, cross‑cloud deployment, storage optimizations, NameNode performance enhancements, RPC prioritization, and practical lessons from operating petabyte‑scale clusters.
Cluster OptimizationHDFSPerformance Tuning
0 likes · 23 min read

This article presents NetEase's big data platform architecture, detailing multi‑layer storage and compute design, HDFS deployment challenges, NameNode and NameSpace performance optimizations, cluster scaling strategies, data tiering, hardware upgrades, and real‑world business use cases, illustrating practical large‑scale big data engineering.
Cluster OptimizationData ManagementHDFS
0 likes · 23 min read

This article explains how Kuaishou’s massive offline compute clusters—tens of thousands of machines processing hundreds of petabytes daily—are managed by a heavily customized YARN stack and the home‑grown Kwai Scheduler, detailing architecture, scheduler evolution, multi‑scenario optimizations, and future scaling plans.
Cluster OptimizationKwai SchedulerLarge-Scale Scheduling
0 likes · 14 min read

This article presents Kuaishou's large‑scale offline computing challenges and describes how the team customized YARN and built the Kwai scheduler to achieve multi‑threaded, pluggable resource scheduling for clusters of tens of thousands of nodes, supporting diverse workloads such as ETL, ad‑hoc queries, machine‑learning training, and real‑time Flink jobs.
Cluster OptimizationKwai SchedulerYARN
0 likes · 15 min read

This guide consolidates everyday Elasticsearch tuning techniques—covering configuration file tweaks, system‑level settings, and usage‑level optimizations such as memory locking, discovery settings, fault detection, queue sizing, translog handling, bulk indexing, shard management, and disk I/O—to help you build a stable, high‑throughput search cluster.
Cluster OptimizationElasticsearchindexing
0 likes · 18 min read

This article details the end‑to‑end process of connecting an OLAP data source to SparkSQL and presents a comprehensive performance‑tuning guide covering cluster‑level resource allocation, single‑node On‑CPU/Off‑CPU analysis, flame‑graph profiling, Java Flight Recorder usage, and garbage‑collection optimization.
Cluster OptimizationGCOLAP
0 likes · 16 min read

This article describes ByteDance’s four‑year series of customizations to Hadoop YARN—covering utilization improvements, multi‑load scenario optimizations, stability enhancements, and multi‑region active‑active deployment—along with practical production experiences, architectural details, and future work directions.
ByteDanceCluster OptimizationHadoop
0 likes · 12 min read

The article explains how maintaining an optimal number of HBase regions (typically 20‑200 per RegionServer) and appropriate region size, along with careful MemStore and compaction settings, can prevent memory pressure, reduce GC pauses, and enhance overall cluster stability and throughput.
Cluster OptimizationHBasePerformance Tuning
0 likes · 5 min read

Jiaoxiao Fang’s talk details Kuaishou’s YARN deployment, covering its architecture, support for offline, real‑time and ML workloads, and recent enhancements such as event‑handling stability, refined preemption, high‑throughput parallel scheduling, shuffle‑caching for small I/O, plus plans for job protection and multi‑cluster resource utilization.
Cluster OptimizationHadoopKuaishou
0 likes · 16 min read

This article walks through a detailed Hadoop performance tuning case using the TPCx-HS benchmark, explaining the bottlenecks in TeraGen and TeraSort, the optimization strategies applied, hardware considerations, and the resulting improvements in CPU and network utilization.
Cluster OptimizationHadoopMapReduce
0 likes · 9 min read
