How to Fix Uneven Data Distribution in Multi‑Threaded Backend Systems

Panic

On a Monday the author noticed a flood of messages in the company chat, suspecting a production line issue.

Root Cause

The system periodically assigns data to employees based on a set of prioritized rules; if none match, a fallback list of employee IDs is used and data is distributed evenly among them.

To improve efficiency, the system processes data in parallel, splitting every n items into a batch, creating k batches that are handled concurrently.

If the employee list length is m, each employee is expected to receive at least ⌊n/m⌋ items, with n % m employees receiving one extra item; with k batches this imbalance can be amplified by k.

When k = 1 and n < m, only a subset of employees handle data.

When k is large, some employees consistently handle more data than others.

In practice, m often exceeds n, leaving m − n employees without any data. A quick fix was to increase n so that n > m, assuming this would guarantee coverage.

However, when daily data volume is low, increasing n does not help because the total data may still be less than n, and the same uneven distribution persists.

Root Problem

The core issue is that the parallel processing logic assigns data based on per‑thread averages without considering overall fairness.

Before deployment the team recognized this but dismissed it, assuming high data volume and avoiding locking mechanisms that could hurt performance.

Current Solution

The implemented solution uses Redis as a counter. Each time a piece of data is assigned, the counter is incremented, and the modulo of the counter with the employee list length determines the target employee index, achieving both efficiency and average fairness.

Conclusion

While the solution appears straightforward, it highlights a common cognitive blind spot: seemingly simple fixes can hide deeper architectural trade‑offs.