This article explains how Streaming KV uses 4 attention sink tokens and a sliding window to convert linear KV cache growth into fixed memory usage, enabling million-token conversations without OOM, while clarifying it preserves generation stability not long-term recall, with Mac deployment options.
This article dissects MHA, MQA, GQA, and FlashAttention across five analytical layers, revealing how structural optimizations (MQA/GQA) reduce KV Cache memory by sharing keys/values, while FlashAttention tackles memory bandwidth via tiled computation and online softmax, and why modern LLMs combine GQA with FlashAttention for 2-4x speedups without accuracy loss.
Discord is rewriting its desktop client from Electron to Rust with the GPUI framework and shifting mobile away from React Native to native UI while sharing a Rust business-logic core, achieving 290 MB memory usage in tests versus multi-gigabyte Electron footprints.
This comprehensive guide walks through efficient LoRA/QLoRA fine-tuning of 7B and 13B language models on a single 24GB GPU, covering memory budgeting, environment setup, data preparation, training configuration, evaluation strategies, model merging, deployment verification, and production best practices with concrete code examples and benchmarks.
Discord is rewriting its desktop client in Rust using the GPUI framework, replacing Electron to drastically reduce memory usage from 1–2 GB to around 290 MB, while sharing core logic with mobile apps via Rust.
Slotstream streams the 105GB Qwen3.8-Flash-Next MoE model from SSD on Apple Silicon Macs with as little as 16GB RAM, achieving 20–23 tok/s on a 48GB M5 Pro, though prefill latency and limited features remain trade-offs.
This guide details running Qwen3.8-Flash-Next 125B MoE on Apple M5 Max 128GB, covering hardware bandwidth limits, memory constraints, framework selection (MLX vs llama.cpp), command-level optimizations including MTP speculative decoding and GPU memory limit tuning, and realistic performance expectations of 45–60 tok/s with MTP enabled.
FastH3 Omni, a distilled version of MiniMax H3, enables local video-with-audio generation on a 36GB M4 Max Mac by using MLX and INT6 quantization to reduce peak memory from 123GB to 19.5GB, though benchmarks are limited to a single device and quality trade-offs remain unverified.
Cloudflare's VoidZero team develops tsrs, a Rust port of the TypeScript compiler that reduces memory usage by 70% and achieves up to 8x faster type checking than the Go-based version, while adding WebAssembly support for browser-based type checking.
This article presents a systematic approach to memory optimization across three dimensions—reducing allocation frequency, accelerating allocation speed, and improving memory usage efficiency—with practical C++ techniques including stack allocation, object pools, size-class allocators, huge pages, data structure packing, and false sharing prevention.
Tencent and HKU propose AdaBool, an adaptive GPU framework for sparse boolean matrix multiplication that dynamically selects offset arrays or bitmaps per block, achieving up to 264x throughput over cuSPARSE/cuBool and 2-7.9x memory reduction, deployed for million-ad real-time filtering.
Adaptive Data RepresentationGPU accelerationReal-time Ad Filtering
This article presents a systematic, evidence-driven framework for Go memory optimization, covering allocation profiling, GC tuning, container memory accounting, runtime metrics observability, and validation methodologies to prevent Kubernetes OOM kills while meeting SLOs.
This article details real-world pitfalls when deploying large language models locally on Mac using Ollama, covering memory overflow causes, key parameter tuning (num_ctx, num_parallel, num_gpu), quantization level selection, model recommendations for coding vs chat, and monitoring strategies to avoid swap-induced performance degradation.
TypeScript 7.0 rewrites its compiler in Go, enabling multi-threaded type checking and shared memory, cutting VS Code codebase type-check time from 125.7s to 10.6s (12x faster) and reducing memory usage by half, eliminating editor lag in large projects.
The author describes how exporting 500k order records caused a 2GB heap OOM, then fixed it by replacing XSSFWorkbook with streaming SXSSFWorkbook, switching from full entity loading to cursor-based batch queries with DTOs, reusing cell styles, and finally decoupling large exports from HTTP requests via async task processing.
DeepSeek's new paper introduces DSec, a production sandbox infrastructure that manages 300 million daily sandbox instances with 380,000 concurrent at peak, using on-demand image loading, composable layers, memory sharing, and CPU QoS to support large-scale agentic reinforcement learning training.
The article shows how to handle 10GB file downloads in Spring Boot by replacing in-memory byte[] loading with streaming via FileSystemResource and leveraging Spring MVC's built-in Range request support for resumable downloads, while covering security, immutability, and object storage offloading.
This article presents 18 practical coding techniques derived from million-QPS production systems to reduce CPU consumption and improve performance, covering type unification, string handling, loop optimization, caching strategies, and data structure selection with concrete before/after code examples.
A pure C inference engine runs the 2.78 trillion parameter Kimi K3 model on just 8GB RAM by streaming weights from disk, leveraging MoE sparsity (only 3.7% active per token) and computing directly on compressed formats, achieving correct output at 32.69 seconds per token on consumer hardware.
This guide details a production-grade architecture for million-row Excel import/export using EasyExcel streaming reads, SXSSF streaming writes, JDBC batch inserts with rewriteBatchedStatements, async MQ processing, and resource isolation — achieving 10-100x speedups and sub-100MB memory.
The article explains how to solve the classic massive‑data deduplication problem of 4 billion QQ numbers within a 1 GB memory limit by analyzing memory requirements, comparing BitMap and Bloom filter approaches, and detailing a step‑by‑step BitMap implementation with its pros and cons.
An analysis of Xiaomi's HyperOS 4 adopting Flutter for UI and Rust for system services, weighing performance benefits against Dart VM maintenance burdens and divergence from Android's mainline optimization.
This article walks through the redesign of a high‑traffic, multi‑dimensional DAU counting system that grew from 8 GB of Redis Set memory to a 24 MB HyperLogLog‑based solution supporting 200 million daily active users, 40 k QPS, and flexible time‑window queries while keeping latency under 20 ms.
Microsoft admits Windows 11’s memory bloat and, driven by low‑end PC demand and a global RAM shortage, unveils four improvement initiatives—including a dedicated 8 GB‑plus memory optimization, a new allocator, WinUI 3 refactoring, and streamlined WebView2—while still lacking a concrete rollout timeline.
Microsoft’s latest blog confirms the company will specifically optimize Windows 11 to run smoothly on devices with only 8 GB of RAM, citing rising memory costs, OEM pricing pressures, and the need to improve responsiveness for low‑memory PCs.
Next.js 16.3 dramatically reduces development server memory consumption—up to 90% in large projects—adds Turbopack disk caching and memory reclamation, speeds repeated builds up to 5.5×, improves SSR throughput by 22%, and ships native Agent DX with optional Instant Navigations.
By replacing the naive List‑based deserialization with Jackson’s JsonParser streaming in Spring Boot 3.5.0, the article shows how processing ~300 k Student records can cut peak heap from 220 MB to 22 MB, avoiding OOM while filtering records on the fly.
This article explores floating-point precision issues in HarmonyOS development, demonstrating how float's 7-digit limit corrupts sensor data, why double increases memory 30% and fails in Native calls, and why BigDecimal kills real-time performance, concluding with a scenario-based selection strategy and validation mechanism.
This article introduces the Flyweight design pattern, explains its intrinsic and extrinsic state concepts, presents real-world use cases such as board games and text editors, and provides a complete Java implementation with an abstract piece, concrete piece, factory pool, and test demonstration, highlighting memory savings and potential pitfalls.
Redis sustains millions of concurrent operations not merely because it is single‑threaded, but thanks to its non‑blocking event‑loop I/O, compact in‑memory data structures, serialized command execution that eliminates lock contention, and robust production features such as replication, sharding, persistence, observability and governance.
Project Valhalla's JEP 401, introducing value classes and objects, merges into the OpenJDK mainline as a preview feature in JDK 28, involving 1,816 files and over 197,000 new lines, with a cautious rollout timeline and notable design trade‑offs for Java performance.
This article explains how to prevent out‑of‑memory errors when importing huge Excel files in Java by using the open‑source excel‑streaming‑reader library, showing Maven setup, streaming code examples, memory‑saving configurations for shared strings and comments, and tips for handling extremely large workbooks.
Douyin's HarmonyOS team collaborated with DevEco Studio to optimize large-scale monorepo development, cutting code indexing time 80% (10→2 min), reducing IDE memory usage 65% (20→7 GB), and accelerating builds 30% (4.5→3 min), delivering a 50% overall efficiency gain now integrated into DevEco Studio 26.
An industrial study by STMicroelectronics and Inria shows that Rust firmware on an STM32U585 MCU uses 45% less RAM than equivalent C code while matching performance, incurs about 10% larger ROM, offers superior portability, and highlights safety benefits and remaining limitations.
The article explains string interning as a memory‑saving technique, shows how to implement it manually in Go, compares the go4.org/intern library with the standard‑library unique package, and presents benchmark results that reveal memory savings but a modest speed trade‑off.
A ten‑week side‑by‑side study by ST, Inria and university researchers shows that Rust firmware for an STM32U585 microcontroller uses 45 % less RAM than an equivalent C implementation while matching performance, highlighting Rust's zero‑heap model, portability advantages and the remaining limitations for low‑end MCUs.
fastutil dramatically outperforms Java’s standard collections by eliminating boxing, cutting memory usage and GC pressure, offering primitive‑specific maps and lists, providing 64‑bit indexed BigArrays for massive data, and delivering faster I/O utilities, all while remaining API‑compatible.
Despite being limited to just 128 KB of ROM, the 1987 NES game Contra achieved a surprisingly long storyline by exploiting tile‑based graphics, tightly packed audio data generated by the Ricoh 2A03 chip, and minimalist code that leveraged the console’s built‑in hardware functions.
jcode, a Rust‑based AI agent framework, uses only 27.8 MB per agent and 14 ms startup time, enabling 10‑20 concurrent agents on an 8 GB laptop, outperforming Claude Code, GitHub Copilot CLI and other Python‑based solutions in memory, speed, and scalability.
Linux’s default 4 KB pages cause massive page tables and TLB misses in high‑memory workloads; this article explains the HugePage mechanism, its types, how it reduces page‑table entries, improves TLB hit rates, lowers fragmentation, and provides step‑by‑step configuration for static and transparent huge pages in production.
MemPO introduces a self‑memory policy optimization framework that lets long‑horizon LLM agents autonomously manage and refine their memory via reinforcement learning, using global‑trajectory and informative‑memory advantage estimates, achieving up to 25.98% F1 gain and 73% token reduction on benchmark tasks.
This article explains the fork system call in the Linux kernel, details the copy‑on‑write (COW) mechanism that underpins its efficiency, provides code examples, and explores practical scenarios and performance implications for process creation, memory usage, and concurrent server programming.
The paper introduces MemPO, a self‑memory strategy optimization algorithm that lets large language model agents actively manage their memory, dramatically improving accuracy on complex multi‑step tasks while reducing token consumption by up to 73%, and validates the approach with extensive experiments and analysis.
The article explains the KV Cache mechanism that stores previously computed key/value vectors to avoid redundant Transformer calculations, delivering roughly a 5× speedup, while also detailing why generating output tokens is far more expensive than processing input tokens due to serial generation and memory trade‑offs.
Google’s newly released TurboQuant algorithm compresses KV‑Cache from 16‑bit to 3‑bit, slashing memory usage to one‑sixth while preserving zero accuracy loss, dramatically accelerating large‑language‑model inference on GPUs and reshaping the memory market.
The article shows how to diagnose and dramatically cut Python’s memory usage by using built‑in tools such as sys.getsizeof, psutil, __slots__, generator expressions, memory‑mapped files (mmap) and string interning, providing concrete code examples, benchmarks and practical tips to avoid common pitfalls.
The article explains MIT’s Attention Matching technique that reformulates large‑model context compression as a linear regression problem, detailing its theoretical foundations, three‑step gradient‑free implementation, architectural adaptations, non‑uniform budgeting, and extensive evaluations showing orders‑of‑magnitude speed gains with minimal accuracy loss.
Learn five practical Python techniques—streaming large files, using generator pipelines, leveraging __slots__, avoiding temporary objects in loops, and reusing buffers—that together can reduce memory consumption by up to 70% and dramatically improve performance when processing gigabyte‑scale datasets.
Researchers at Oak Ridge National Laboratory introduced D‑CHAG, a distributed cross‑channel hierarchical aggregation method that cuts memory consumption by up to 75% and more than doubles throughput when training massive multi‑channel foundation models on up to 1,024 AMD GPUs, as demonstrated on hyperspectral imaging and weather‑forecasting workloads.
The article walks through five concrete Python techniques—streaming file reads, generator expressions, __slots__, avoiding temporary objects in loops, and reusing buffers—showing code examples and measured memory reductions that together lowered overall RAM consumption by about 70%.
This article walks through why using a Redis Set for 200 million daily active users consumes 6‑8 GB of memory, how switching to a sharded Bitmap reduces usage to about 24 MB, and how Roaring Bitmap and ClickHouse further optimize sparse data and analytical workloads.
This article presents a step‑by‑step experiment that uses JMeter to generate 20,000 HTTP calls against a SpringBoot 2.5.4 service, captures detailed GC logs, calculates the heap memory allocated per request, analyses the impact of request payload and logging, and offers practical tuning recommendations.
This article explains how the rapid evolution of AI agents caused context explosion, why the original fixed‑window memory failed, and how CodeGenius introduced a layered memory system that unloads stale data, deduplicates files, generates structural summaries, and dynamically compresses dialogue to keep prompts stable, reduce token cost, and improve task continuity.
This article explains JEP 519’s Compact Object Headers in Java 25, detailing how reducing the object header from up to 128 bits to 64 bits cuts memory overhead, improves cache locality, lowers garbage‑collection pressure, and delivers measurable performance gains for JVM workloads.
This article explains the fundamentals of Linux thread stack memory, identifies why default stack sizes can cause waste or overflow in high‑concurrency scenarios, and provides practical techniques—including stack‑size tuning, code refactoring, and memory‑mapping—to reduce memory usage and improve service stability.
The article breaks down the principles of KV caching in large language models, explaining how Q/K/V behavior differs between training and inference, the role of prompts, cache size trade‑offs, and the complexities of concurrent inference, all backed by concrete examples and references.
The article introduces a fine‑grained activation offloading technique implemented in Megatron‑Core that offloads module‑level activations to CPU, overlaps transfer with computation, and remains compatible with pipeline and virtual pipeline parallelism, dramatically reducing peak GPU memory for large language models while incurring minimal throughput loss.
This article explains how to measure the exact heap memory consumption of individual SpringBoot HTTP and RPC requests by calculating required memory from request size, configuring JVM GC parameters, creating a JMeter test plan, collecting GC logs, and analyzing results to identify optimization opportunities.
This article presents a collection of practical embedded‑system optimization techniques, covering time‑efficiency improvements such as avoiding floating‑point arithmetic and inlining functions, space‑efficiency strategies like choosing appropriate data types, using unions and flexible arrays, as well as loop unrolling, bit‑field usage, and data‑type selection to maximize performance on resource‑constrained devices.
This guide walks you through why Tomcat's default JVM settings are insufficient, common pitfalls when manually editing catalina.sh, and the recommended, maintainable approach using setenv.sh to configure heap, Metaspace, and GC options for reliable, high‑performance server operation.
This article explains how Python's __slots__ attribute can dramatically reduce memory usage and speed up attribute access when creating large numbers of objects, providing detailed examples, performance measurements, best‑practice guidelines, and practical use‑case scenarios.
This comprehensive guide shows how a seasoned operations engineer can dramatically improve Tomcat stability and throughput by understanding JVM memory structures, configuring optimal JVM and connector parameters, selecting the right garbage collector, and implementing robust monitoring and diagnostics to prevent crashes and latency.
This article explains how using a bitmap can reduce the memory needed to store billions of integers from about 15 GB to under 500 MB, describes the bitmap concept, introduces Bloom filters, outlines their principles, advantages, common use cases, and provides Java and Redis code examples for implementation.
This article explains how Python's default class memory overhead can be dramatically reduced using the __slots__ magic attribute, providing detailed code examples, memory and speed benchmarks, practical use cases, limitations, and best‑practice recommendations for developers who need high‑performance, memory‑efficient objects.
This article explains how Redis achieves ultra‑high concurrency by combining a single‑threaded event‑driven core, highly optimized in‑memory data structures, efficient I/O multiplexing, smart memory management, asynchronous persistence, and scalable clustering, allowing it to comfortably serve hundreds of thousands of simultaneous requests.
This article analyzes the memory bottleneck of massive language models, explains the mathematical modeling of memory requirements, evaluates traditional sharding limits, and details how GPT‑OSS’s MXFP4 quantization combined with Mixture‑of‑Experts reduces memory, bandwidth, and compute demands enough to fit a 1200‑billion‑parameter model onto an 80 GB GPU with minimal accuracy loss.
This article explains how PHP lazy evaluation using generators and the Iterator API can dramatically reduce memory consumption when loading huge CSV files, provides side‑by‑side code examples and benchmarks, and offers guidance on when to choose each approach for real‑world data processing tasks.
Learn how to efficiently remove duplicates from 4 billion QQ numbers using a memory‑friendly Bitmap approach that fits within a 1 GB limit, including calculations, step‑by‑step implementation, Java code, and a discussion of its advantages and drawbacks.
This article examines why std::vector can become a bottleneck when handling millions of elements, analyzes memory consumption, insertion/deletion costs, and cache behavior, and presents practical optimizations such as pre‑allocation, move semantics, SIMD vectorization, and cache‑friendly designs illustrated with real‑world case studies and code examples.
This article explains the challenges of high‑dimensional vector retrieval, introduces quantization techniques—especially the binary RaBitQ method and its MRQ extension—detailing their compression ratios, speed gains, compatibility with search indexes, and real‑world performance results in the VSAG system.
This article explores why early 1980s games such as Contra could deliver rich graphics, music, and gameplay within just 128KB of memory, explaining tile‑based rendering, limited audio chip capabilities, and ultra‑compact code techniques that made such feats possible.
This article explains why memory optimization is critical for Unity mobile games, outlines a three‑step workflow of testing, analysis, and fixing, details practical testing methods, data‑analysis toolchains, and concrete optimization techniques such as resource LOD, array pre‑allocation, string deduplication, compression, and atlas management, and concludes with a concise checklist for effective memory management.
This tutorial walks through LLaMA‑Factory fine‑tuning by explaining how to choose learning rate, epochs, batch size, cutoff length, LoRA rank, and validation split, and shows how to estimate and reduce GPU memory usage with techniques like gradient accumulation, liger_kernel, and DeepSpeed.
This article details the memory‑bloat problem of the Electron‑based Tencent Docs desktop client, outlines several optimization ideas, explains the implemented tombstone mechanism and Windows‑level memory cleaning plus Chromium page‑freezing, and presents benchmark results showing a roughly 60% reduction in RAM consumption.
This article breaks down GPU memory consumption for large Transformer models, explains how to estimate each component—parameters, optimizer state, activations, gradients—and shows how parallelism, mixed precision, and recomputation strategies can dramatically reduce the footprint.
Learn how Python's eager import mechanism can cause slow startup in large projects and discover practical lazy import techniques—including function-level imports, custom LazyLoader classes, context manager approaches, and standard library solutions—that dramatically reduce launch time and memory usage while maintaining IDE support.
This article explains MySQL's memory architecture, why memory usage often stays high after spikes, and provides practical steps—including connection checks, slow query analysis, workload scaling, and switching to jemalloc—to diagnose and reduce memory consumption on 360's internal cloud platform.
This article explains how to efficiently remove duplicates from 4 billion QQ numbers within a 1 GB memory limit by replacing the naïve HashSet approach with a memory‑saving Bitmap data structure, complete with calculations, algorithm steps, Java code, and a discussion of its pros and cons.
This article explores Python's list comprehensions and generator expressions, detailing their syntax, performance characteristics, memory usage, multi‑level nesting, and practical tips such as dictionary/set comprehensions and integration with functional programming, helping developers choose the appropriate tool for efficient data processing.
This article explains three Redis-based methods—Hash, Bitmap, and HyperLogLog—for tracking website user visits, detailing how each structure works, their implementation steps, memory and accuracy trade‑offs, and guidance on choosing the best approach for different traffic scenarios.
The article explains what constitutes a Redis large key, outlines its performance and stability risks, describes common scenarios and root causes, and provides practical detection commands, mitigation techniques such as splitting, compression, proper data modeling, and monitoring strategies to prevent future issues.
The article introduces BladeDISC++, a dynamic‑shape AI compiler from Alibaba Cloud PAI, explains the memory‑peak challenges of dynamic‑shape deep‑learning workloads, describes its symbolic‑shape graph, joint compile‑time/runtime optimizations such as operation fusion, scheduling and just‑in‑time rematerialization, and presents Llama2 experiments showing significant GPU memory savings and throughput gains.
This article explains PHP generator functions introduced in PHP 5.5 and enhanced in PHP 7, demonstrating how to implement infinite iterable objects such as a Fibonacci sequence, and discusses their advantages for processing large data sets, memory efficiency, and typical use cases like log handling and pagination.
This article details the challenges of video corruption detection on a creator platform, analyzes existing server‑side and client‑side approaches, and presents a WebAssembly‑based solution using ffmpeg, WorkerFS, and memory‑growth tuning that reduces memory consumption by up to 78% while speeding up large‑file processing.
Go’s empty struct occupies zero bytes thanks to a special zerobase address, and understanding this behavior reveals how to use empty structs for memory‑efficient patterns such as sets, signal channels, and struct alignment tricks.
Java’s String.intern() method adds a string to the constant pool, enabling memory sharing and faster reference comparisons; this article explains its behavior, demonstrates usage with code examples, and discusses performance benefits and cautions for optimal use in real‑world applications.
This article explains the challenges of dynamic‑shape deep learning workloads and introduces BladeDISC++, an AI compiler that uses symbolic shape graphs, operation scheduling, and just‑in‑time auto‑rematerialization to dramatically reduce GPU memory peaks while maintaining training throughput.
This article explains how the Go library BigCache uses sharding, efficient hash functions, and memory‑optimised data structures to deliver high‑concurrency, low‑latency caching, while avoiding GC bottlenecks and providing practical code examples and performance benchmarks.
This article walks through the mathematics of Bloom filters, showing how to model false‑positive probability, derive optimal bit array size and hash‑function count, and apply the formulas to a 4‑million‑item dataset with concrete examples and performance tables.
This article presents a collection of lesser‑known Python tricks—including string cleaning with translate, iterator slicing via itertools.islice, skipping header lines, keyword‑only functions, custom context managers, memory‑saving __slots__, resource limits, import control, and total_ordering—to help developers write cleaner, more efficient code.
The article explains the Flyweight design pattern in Java, detailing intrinsic vs extrinsic state, the role of a FlyweightFactory, step‑by‑step code examples for sharing Circle objects, and discusses its advantages, suitable scenarios, and how it reduces memory usage while boosting performance.
Redis’s evolution from linked lists to ziplist, quicklist, and the newer listpack is examined in depth, highlighting their structures, memory‑performance trade‑offs, operational details, and practical guidance for Java developers on choosing the right data structure for different workloads.
Go 1.23 introduces the unique package, which provides generic interning for comparable values, allowing canonicalization of strings and structs, reducing memory usage and speeding up equality checks; the article walks through a simple map‑based implementation, its limitations, the advanced Handle[T] design, real‑world netip usage, and future prospects.
This article explains Rust's Clone‑On‑Write (COW) feature, showing how it reduces unnecessary memory allocations, detailing its internal enum variants, typical use cases, performance benefits, limitations, and providing practical code examples for function return values and string concatenation.
This article dissects Python's dictionary implementation, explaining the PyDictObject layout, the combined and split hash‑table designs, memory‑optimisation techniques, insertion and lookup processes, ordered iteration, and precise memory‑size calculations with concrete code examples.
This article reviews the principles and practical implementations of data, pipeline, tensor, sequence, and context parallelism together with memory‑saving strategies such as recomputation and ZeRO, and demonstrates how the QLM framework leverages these techniques to accelerate large‑model training and fine‑tuning on multi‑GPU clusters.
Facing massive FullGC alarms due to an oversized in‑memory cache, we leveraged Java’s String.intern() via a custom fastjson deserializer and a rewritten HashMap to deduplicate low‑entropy strings, slashing heap usage by over 1 GB while preserving functionality.
This article explains Hugging Face's KV cache quantization technique, detailing how it reduces memory usage for long‑context LLM generation, the underlying quantization methods, implementation steps in 🤗 Transformers, benchmark results versus fp16, and the trade‑offs between speed, memory, and accuracy.