Why a Simple HashMap Bug Crashed Our High‑Concurrency Service and How to Fix It

A senior architect with extensive high‑concurrency experience joined the team and proposed a monitoring requirement: collect the average response time and total request count for every API endpoint. The implementation used a ConcurrentHashMap where each request retrieved a key, incremented a value, and stored the data.

After the code went live, the service quickly ran out of memory. Using jmap and Eclipse MAT, the team discovered that the map contained millions of distinct MonitorKey objects because the key class did not override equals and hashCode. Consequently, identical requests generated separate keys, causing severe memory bloat.

class MonitorKey {
    private String url;
    private String desc;
    // missing equals() and hashCode()
}

The fix was to implement proper equals and hashCode methods for MonitorKey (and similarly for MonitorValue). After the change, the map correctly collapsed duplicate entries and the memory leak disappeared.

Another problem surfaced in the visit method. The original code performed a non‑atomic check‑then‑act sequence:

Thread 1 reads the value for key a (null) and creates object b.

Thread 2 reads the same key (still null) and creates object c.

Both threads store their objects, overwriting each other.

This race condition caused lost updates.

Two remediation approaches were debated:

Adding the synchronized keyword to the entire method, guaranteeing exclusive access but incurring coarse‑grained locking.

Using the atomic putIfAbsent method of ConcurrentHashMap combined with thread‑safe counters.

public void visit(String url, String desc, long timeCost) {
    MonitorKey key = new MonitorKey(url, desc);
    MonitorValue value = monitors.putIfAbsent(key, new MonitorValue());
    value.count.getAndIncrement();
    value.totalTime.getAndAdd(timeCost);
    value.avgTime = value.totalTime.get() / value.count.get();
}

Team leadership favored the minimal‑change synchronized solution, citing lower risk, while the author advocated the more scalable putIfAbsent approach.

The incident taught several lessons: always override equals and hashCode for objects used as map keys, prefer fine‑grained atomic operations over broad synchronization in high‑concurrency code, and conduct thorough post‑mortems to share findings and prevent recurrence.