This article presents a comprehensive design and implementation of a high‑concurrency memory pool in C++, covering concepts such as fixed‑size block allocation, thread‑local caches, central and page caches, lock‑free techniques, span management, and performance benchmarking against standard malloc.
Long Young GC pauses in gRPC/Netty services were traced to Netty’s default thread‑local cache creating many MpscArrayQueue objects, and disabling the cache with the JVM options ‑Dio.netty.allocator.useCacheForAllThreads=false and ‑Dio.grpc.netty.shaded.io.netty.allocator.useCacheForAllThreads=false reduced GC time from up to 900 ms to about 100 ms, stabilizing the service.
MySQL uses a Thread Cache pool to store idle connection threads, dramatically speeding up client connection creation; understanding parameters like thread_cache_size and thread_stack, monitoring cache hit rates, and proper tuning can keep the hit rate around 90% for optimal performance.
