Mastering High‑Concurrency Account Balance Deductions with Kafka, Sharding, and Parallel Processing

High‑concurrency scenarios are divided into read‑heavy and write‑heavy workloads; account balance deduction belongs to the latter and is more difficult to handle.

In ad‑billing platforms, each click or view triggers an immediate balance deduction, requiring real‑time processing to avoid platform loss.

Original State

The system initially performed sequential steps: record transaction, anti‑fraud verification, charge the account, update the transaction record, and execute downstream processing, without any high‑concurrency optimizations.

Asynchronous Peak Shaving with Kafka

To absorb burst traffic, the design introduces Kafka. After a user clicks an ad, the request is persisted as a transaction record, then the data is published to Kafka for asynchronous processing. This decouples the front‑end request from the heavy downstream logic.

Consumers pull batches of messages (default poll() size 500, configurable via max.poll.records) and process them in parallel using a thread pool, improving throughput. However, excessive message backlog can delay critical actions such as disabling ads when an account balance reaches zero.

Parallel Processing

When sustained high traffic occurs, merely using Kafka is insufficient; parallelizing consumer processing further raises throughput. The approach sacrifices strict ordering, which is acceptable for ad‑billing.

Database Sharding

The billing system relies on two main tables: a massive transaction (deduction) log and an advertiser account balance table. Frequent updates to the balance table cause row‑level lock contention.

By using the advertiser ID as a sharding key, both tables are split across multiple databases and tables, and cold data in the balance table is archived to prevent table‑size bloat.

Hotspot Dispersion

When a single large advertiser generates tens of thousands of deductions per second, the sharding scheme alone may not suffice. The industry solution is to split the main account into multiple sub‑accounts distributed across shards, thereby spreading the write hotspot.

For example, a 100 million‑unit balance can be divided into ten sub‑accounts of 10 million each; any deduction can target any sub‑account, and the transaction is recorded accordingly.

Batch Deduction

Another method to reduce write hotspots is to aggregate multiple deductions for the same balance record into a single batch update. Consumers can poll a batch of messages, sum the amounts, and apply a single update, or a scheduled task can periodically aggregate deductions over a short interval (e.g., 10 seconds).

Control Measures

Keep the scheduled aggregation interval short (around 10 seconds) to limit latency.

When an account balance becomes insufficient (but not exactly zero), immediately notify upstream systems to stop ad delivery, preventing overdraft losses.

These combined techniques—Kafka‑based asynchronous processing, parallel consumer execution, sharding with hotspot dispersion, and batch updates—enable the system to handle extreme write concurrency while maintaining real‑time accuracy and preventing platform losses.