Kafka on PaaSTA: Running Kafka on Kubernetes at Yelp (Part 1 – Architecture)

Yelp migrated its high‑throughput Kafka clusters from dedicated EC2 instances to a Kubernetes‑based PaaSTA platform, introducing a custom Kafka operator and Cruise Control to streamline provisioning, lifecycle management, upgrades, and scaling, thereby reducing deployment time and improving maintainability.

Cloud NativeCruise ControlKubernetes

0 likes · 11 min read

Kafka on PaaSTA: Running Kafka on Kubernetes at Yelp (Part 1 – Architecture)

Cloud Native Technology Community

Jun 28, 2022 · Cloud Native

How Yelp Re‑engineered Kafka on Kubernetes with PaaSTA: Architecture Deep Dive

Yelp migrated its massive Kafka deployment from EC2 to a Kubernetes‑based PaaSTA platform, introducing a custom Kafka operator and Cruise Control to cut provisioning time, simplify upgrades, improve lifecycle management, and reduce reliance on Puppet, while maintaining high‑throughput data pipelines.

Cloud NativeCruise ControlKafka

0 likes · 11 min read

How Yelp Re‑engineered Kafka on Kubernetes with PaaSTA: Architecture Deep Dive

ITPUB

Jun 4, 2022 · Operations

Mastering Kafka Load Balancing with Cruise Control: From Manual Migration to Automated Optimization

This article explains why Kafka suffers from broker‑side load imbalance, walks through manual replica migration examples, and then details how Cruise Control automates load balancing, supports resource‑group targeting, leader‑replica dispersion, and provides step‑by‑step deployment instructions.

Cluster ManagementCruise ControlKafka

0 likes · 21 min read

Mastering Kafka Load Balancing with Cruise Control: From Manual Migration to Automated Optimization

vivo Internet Technology

May 31, 2022 · Big Data

Kafka Load Balancing and Cruise Control: Concepts, Manual Migration, and Deployment

Kafka’s server‑side load imbalance, caused by static replica placement on broker disks, makes manual replica migration infeasible at scale, but Cruise Control automates metric collection, analysis, and execution of fine‑grained rebalance plans—including broker de‑commissioning and leader dispersion—allowing large clusters to expand and operate efficiently.

Big DataCluster ManagementCruise Control

0 likes · 21 min read

Kafka Load Balancing and Cruise Control: Concepts, Manual Migration, and Deployment