This article breaks down Java's three memory regions—Heap, Off‑Heap, and No‑Heap—detailing their definitions, management mechanisms, performance impacts, typical use cases, and provides concrete code samples for allocating each type of memory.
This article explains why moving large local caches to off‑heap memory using the open‑source OHC library can reduce GC pressure, provides a step‑by‑step demo with custom serialization, compares it with a HashMap OOM example, and dives into the library's core allocation mechanisms and configuration options.
This article explains the concepts of virtual memory, kernel and user modes, DMA, and why data must move between kernel and user spaces, then compares standard I/O with zero‑copy techniques, MMAP, and off‑heap buffers, showing how they reduce copy operations and improve Java server performance.
This article explains Spark’s unified memory management model, detailing the division between on‑heap and off‑heap memory, the roles of execution, storage, user, and reserved memory, configuration parameters, dynamic allocation, and how these concepts affect performance and resource utilization.
This article explains Flink's memory architecture, covering on‑heap and off‑heap memory concepts, garbage collection, allocation strategies, memory segments, buffers, the memory manager, and how network transmission and back‑pressure are handled to achieve efficient streaming processing.
The article introduces Oak, an off‑heap concurrent key‑value map, explains its design and performance benefits over ConcurrentSkipListMap, and details extensive offline and real‑time ingestion experiments in Apache Druid that demonstrate reduced memory usage, lower CPU consumption, and faster data loading.
This article analyzes Apache Spark's executor-side memory management model, focusing on the UnifiedMemoryManager in Spark 2.2.1, detailing on‑heap and off‑heap memory regions, dynamic execution/storage memory sharing, task memory allocation, and practical configuration examples.
This article compiles a series of HBase questions and answers covering write performance, bulk loading, single‑node configuration, column scalability, transaction isolation, fast deletion methods, off‑heap optimizations, bulkload modes, Hive integration, direct HFile reads, and region planning.
When the recommendation service of the Mafengwo app experiences database disconnections, third‑party timeouts, or network jitter, a locally‑deployed off‑heap cache built with OHC and SpringBoot can return pre‑computed results, isolating business logic, reducing latency, and improving user experience during failures.
This article explains the fundamentals of JVM garbage collection, its generational algorithms and associated performance issues, and then details Apache Flink's memory management architecture, including MemorySegment, off‑heap buffers, serialization mechanisms, and type information for efficient big‑data processing.
This article provides a detailed analysis of DBLE's memory management module, explaining its structure, configuration parameters, allocation and recycling logic, and includes annotated source code snippets for classes such as DirectByteBufferPool and ByteBufferPage, illustrating how off‑heap and on‑heap memory are handled.
This article explains why a Java process consumes far more virtual memory than the configured heap size by detailing JVM subsystems, off‑heap allocations, native libraries, thread stacks, and other factors that contribute to overall memory usage.
This article explains Apache Spark's memory architecture, covering the shift from static to unified memory management, the roles of on‑heap and off‑heap memory, configurable parameters, dynamic memory sharing between execution and storage, and the legacy mode introduced in Spark 1.6.
This article explains Spark's executor memory architecture, covering on‑heap and off‑heap memory planning, static and unified memory managers, storage and execution memory allocation, RDD persistence, eviction policies, and shuffle memory usage, providing practical guidance for performance tuning.
When the recommendation service experiences database disconnections, third‑party timeouts, or network jitter, we designed an off‑heap local disaster‑recovery cache using OHC and SpringBoot that isolates cache logic, writes asynchronously, backs up to disk, and restores availability, keeping latency under 10 ms and improving user experience.
This article explains Spark's executor memory architecture, covering on‑heap and off‑heap allocation, static versus unified memory managers, storage and execution memory handling, RDD persistence levels, eviction policies, and shuffle memory usage, providing practical formulas and configuration tips for optimal performance.
This article examines Spark's unified memory manager in version 2.2.1, detailing on‑heap and off‑heap memory regions, the four on‑heap memory pools, dynamic execution‑storage memory sharing, task memory accounting, and provides concrete calculation examples to explain UI discrepancies and runtime memory limits.
This article explains how a Janus service gateway ran out of memory due to unreleased Netty off‑heap ByteBufs, how enabling Netty's leak detection and adjusting JVM parameters helped reproduce the issue, and how to fix the leak by explicitly releasing the buffers.
This article explains Spark's executor memory architecture, covering on‑heap and off‑heap allocation, static versus unified memory managers, storage and execution memory handling, RDD persistence, eviction policies, and the role of Tungsten's page‑based management in optimizing performance.
The article explains how Flink tackles JVM memory challenges by using proactive memory management, a custom serialization framework, cache‑friendly binary operations, and off‑heap memory techniques to reduce GC pressure, avoid OOM, and improve performance in big‑data workloads.
