How Cloud Disk Types Affect Kafka Instance Performance: A Hands‑On Test

Performance Impact of Cloud Disk Types on Alibaba Cloud Kafka Instances

Kafka instances on Alibaba Cloud exhibit performance variations not only due to their compute specifications but also because of the cloud disk type selected at creation. For the Serverless Kafka offering, no disk type can be chosen; its throughput is limited by the reserved write and read capacities, with peak values up to twice the reserved limits.

Experiment Overview

The experiment uses Cloud Architect Design Tools (CADT) to provision Kafka instances of different types, set up a load‑testing environment, and measure how disk type and instance configuration affect throughput and latency. The results help users select appropriate specifications for their workloads.

Deployment Architecture

One Elastic Compute Service (ECS) instance with an Elastic IP (EIP) serves as the test client host. The EIP enables SSH login and downloading of the benchmark client.

The official Kafka load‑testing tool (Kafka performance test utilities) runs on the ECS.

Three Kafka instances are created in the same VPC, each with a distinct cloud disk type (e.g., standard SSD, high‑performance SSD) for comparative testing.

Key Steps

Environment provisioning : Use CADT to create the ECS host, bind the EIP, and deploy three Kafka instances with the desired disk types.

Load‑test configuration :

Obtain the Kafka endpoint (bootstrap servers) from the deployed instances.

SSH into the ECS, install the Kafka benchmark client, and configure test variables such as message size, batch size, and target throughput.

Execute performance test : Run the benchmark tool against each Kafka instance, record metrics (throughput, latency, error rate), and compare results across disk types.

Resource cleanup : After testing, release all resources (ECS, EIP, Kafka instances) with a single operation in CADT.

Prerequisites

An Alibaba Cloud account with real‑name verification completed.

Pay‑as‑you‑go resources purchased; account balance should exceed ¥100 (¥200 recommended) to cover hourly charges during the experiment.

Notes on Serverless Kafka

Serverless Kafka does not expose a disk type option. Its performance ceiling is directly tied to the reserved write and read capacities defined at purchase. The observed peak write and read throughputs can reach up to 2× the reserved values.

Relevant Resources

Alibaba Cloud VPC documentation for private network isolation.

CADT documentation for template‑based deployment.

Kafka official performance testing utilities (e.g., kafka-producer-perf-test, kafka-consumer-perf-test).