How to Quickly Scale Kafka Topic Replicas with Know Streaming

Feature Overview

Replica scaling is not a native Kafka feature. When a topic is created with a replication factor of 1 and later needs higher availability, Kafka offers no direct way to change the factor, creating operational difficulty. Know Streaming supplies a plug‑in‑based mechanism to perform replica expansion or reduction.

Procedure

1. Topic → Batch Change → Migrate Replicas

2. Select the topics whose replica count you want to adjust (multiple selection supported).

Highlights

Batch scaling of multiple topics

You can select several topics at once and set different replica counts for each.

Custom target nodes

You may specify the broker nodes where the new replicas should be placed. The number of target nodes must be at least as large as the maximum replica count among the selected topics. For example, if one topic is expanded to 2 replicas and another to 3, you need at least three target nodes.

Preview and manual adjustment of the replica plan

The “Preview Task Plan” button shows the proposed partition‑to‑broker assignment after scaling, allowing you to edit individual assignments if needed (e.g., to make a particular replica the leader).

This feature lets you verify the final distribution and optionally reorder brokers or set leader preferences.

Throttled scaling

Replica expansion or reduction involves copying and deleting replica data. To limit impact on normal workloads, you can set a throttling threshold for the whole operation, reducing pressure on the cluster.

Implementation Principle

Replica scaling is essentially a partition‑replica reassignment. Refer to the standard partition‑replica reassignment process for details.

Know Streaming

Know Streaming originates from years of internal Kafka operation experience at an internet company. It provides a zero‑intrusion, plug‑in‑based enterprise‑grade Kafka service that works with 0.10.x‑3.x clusters without invasive changes, lowering the barrier for real‑time stream data management.

GitHub: https://github.com/didi/KnowStreaming