The article explains why ordinary MyBatisPlus pagination becomes slow and memory‑hungry when processing millions of rows, and demonstrates three alternatives—regular query, stream query, and cursor query—detailing their implementations, trade‑offs, and practical code snippets for MySQL and Oracle environments.
This article shares practical strategies for optimizing MyBatis‑Plus CRUD operations in massive‑data scenarios, covering query, insert, update, delete, batch processing, and database‑level tuning to keep performance stable when tables exceed ten million rows.
The article explains how PHP generator functions can create infinite iterable objects, such as a Fibonacci sequence, to process large data sets efficiently by yielding values on demand, reducing memory usage and improving performance, with practical code examples and discussion of advantages and use cases.
When processing massive MySQL result sets, loading all rows into JVM memory can cause OOM and slow performance, so this guide compares three approaches—regular pagination, streaming queries using server-side cursors, and cursor‑based fetchSize control—detailing their implementations, MyBatis configurations, and trade‑offs.
This article explains how PHP generator functions can create infinite iterable objects, demonstrating with a Fibonacci sequence example to efficiently handle large data sets while reducing memory usage and improving performance in backend development.
This article examines the challenges of querying massive follower lists stored in MySQL, compares limit‑based pagination with tag‑record and range‑limit methods, and offers practical indexing and query‑optimization recommendations for handling tables containing hundreds of millions of rows.
This article explains how PHP developers can use generators to iterate over large data sets without exhausting memory, covering the concept, syntax with the yield keyword, step‑by‑step examples, converting regular functions, handling key‑value pairs, sending data back, returning values, and a real‑world file‑reading use case.
This article analyzes the performance problems of LIMIT‑based pagination on massive MySQL tables and presents three progressively more efficient solutions—including a simple LIMIT approach, a tag‑record method using the last primary key, and a range‑limit method with cached min‑ID—along with best‑practice indexing recommendations to keep query latency in the tens of milliseconds even for tables containing billions of rows.
The article analyzes the performance problems of deep pagination on massive MySQL tables storing billions of fan records and presents three progressive solutions—simple LIMIT, tag‑record (maxId) pagination, and range‑limited pagination with async and offline minId caching—along with general indexing best‑practices for high‑throughput queries.
This article demonstrates several techniques—including server‑side data simulation, batch rendering with timers, requestAnimationFrame, pagination components, infinite scroll, and virtual lists—to efficiently display and interact with 100,000 records in a Vue.js front‑end without causing UI freezes or performance degradation.
This article details a real‑world test of loading over 20 billion records into MySQL with XeLabs TokuDB, covering the demand, configuration tweaks, table schema, bulk‑loader commands, performance metrics, comparison with InnoDB, and practical conclusions for large‑scale data ingestion.
This article explains how to use MyBatis streaming queries via the Cursor interface, showing configuration, code examples, batch processing techniques, practical use cases, and important precautions to efficiently handle large result sets without exhausting JVM memory.
This article demonstrates how to generate, insert, and query millions of rows in a MySQL 5.7 table, measures pagination performance under varying offsets and result sizes, and presents several optimization techniques—including sub‑queries, indexed look‑ups, and column selection—to dramatically reduce query latency.
This article demonstrates how to create a 10‑million‑row MySQL table, batch‑insert data via a stored procedure, measure ordinary LIMIT pagination versus offset‑optimized queries, and provides practical tips for improving query speed on massive tables.
This article demonstrates how to generate, insert, and query ten‑million‑row MySQL tables, measures the latency of ordinary LIMIT pagination, analyzes the impact of offset size and result set size, and presents practical optimization techniques such as sub‑query pagination and ID‑range filtering to dramatically improve query speed.
This article walks through creating a 10‑million‑row MySQL table, measuring pagination query times under different offsets and result sizes, and presents practical optimizations such as sub‑queries, ID‑range filtering, and column selection to dramatically improve performance while explaining why using SELECT * is discouraged.
The article explains how to efficiently retrieve and process massive MySQL result sets in Java using MyBatis, comparing regular pagination, streaming queries via Cursor, and cursor-based fetchSize techniques, and provides practical code examples and best‑practice tips to avoid OOM and improve performance.
The article explains how to process massive MySQL data sets—covering data migration, export, and batch handling—by comparing regular pagination, streaming queries using server‑side cursors, and cursor‑based fetchSize techniques, and provides concrete MyBatis code examples for each approach.
Processing massive datasets in MySQL can cause OOM and slow performance; this article explains when to use regular pagination, streaming queries, and cursor-based fetching, compares their trade‑offs, and provides MyBatis code examples with @Options and @ResultType to handle millions of rows efficiently.
This article details how Ant Financial’s front‑end team created a fast, extensible React big‑data table component using the open‑source AntV S2 Canvas library, covering performance‑driven technical choices, custom cell editing, virtual scrolling, freezing, filtering, sorting, and integration challenges that replaced costly commercial spreadsheet solutions.
This article explains why traditional Excel export methods cause memory overflow, introduces Alibaba's EasyExcel library that keeps memory usage in kilobytes, and provides step‑by‑step Maven setup, POJO definitions, and code examples for exporting small, medium, and massive data sets in a Spring Boot environment.
This article explains the challenges of exporting massive datasets from backend systems, compares a custom SXSSFWorkbook solution with Alibaba's EasyExcel library, and provides detailed code examples for handling small, medium, and huge data volumes while keeping memory usage low and performance high.
This article explains how to replace a custom, memory‑intensive Excel export implementation with Alibaba's EasyExcel library, demonstrating Maven setup, POJO definition, and three test scenarios for small, medium, and huge data sets, while offering performance tips and production‑ready code for Java backend systems.
This article examines the performance drawbacks of MySQL's LIMIT pagination on large tables, presents six practical query methods—including direct LIMIT, primary‑key indexing, index‑based ordering, prepared statements, covering indexes, and sub‑query/join techniques—provides extensive benchmark results, and offers concrete indexing recommendations to achieve fast, stable pagination even with millions of rows.
This article details a real‑world test of loading over 200 million rows into MySQL using XeLabs TokuDB, showing configuration tweaks, bulk‑loader commands, performance metrics (≈570 k rows per second), file‑size reductions, and comparisons with InnoDB under a 8‑core, 8 GB RAM, 500 GB SSD environment.
This article describes a real‑world requirement to import over 2 billion records into MySQL, analyzes the challenges, introduces XeLabs TokuDB with its optimizations, details the test schema and configuration, demonstrates bulk loading commands, presents performance metrics showing up to 570 k rows per second, and concludes with practical recommendations and environment details.
This article explains how to prevent frequent full GC and out‑of‑memory errors when using Apache POI for massive Excel imports and exports by leveraging SXSSFWorkbook for XLSX, choosing appropriate processing models for XLS/XLSX, and provides performance test results and code samples.
This article details how to transform a painfully slow SSRS report that scanned billions of rows into a high‑performance solution by analyzing the original SQL, fixing indexing and partitioning mistakes, and rewriting the logic as a flexible stored procedure that runs in seconds.
This article details a real‑world performance test of loading over two billion rows from a big‑data platform into MySQL using XeLabs TokuDB, covering requirements, TokuDB features, table schema, bulk‑load commands, results, and practical recommendations for high‑throughput data ingestion.
This article explores MySQL performance when storing and querying billions of financial transaction records, detailing table creation, massive data generation, the impact of indexes on query and insert speed, pagination limits, and a practical time‑based sharding approach for bank‑level cash‑flow logs.
This article describes how to ingest more than two billion records into MySQL by leveraging XeLabs TokuDB’s bulk‑loader, detailing configuration, table schema, performance metrics, and a comparison with InnoDB to demonstrate a three‑to‑fourfold speed improvement.
This article details a real‑world test of loading over 2 billion rows into MySQL using XeLabs TokuDB, compares InnoDB and TokuDB performance, shows configuration, table schema, bulk‑loader commands, and provides practical conclusions for high‑throughput data ingestion.
This article describes how to quickly import more than two billion rows from a big‑data source into MySQL by leveraging XeLabs TokuDB’s bulk‑loader, showing configuration, code examples, performance results, and practical recommendations for handling large‑scale data ingestion.
To prevent memory overflow when processing massive arrays in PHP, read large files line‑by‑line, batch database queries, promptly unset variables, and prefer a pre‑counted for‑loop over foreach for mutable collections, thereby freeing memory gradually and ensuring stable script execution.
This article evaluates MongoDB and MySQL for storing and querying large‑scale precious‑metal market data, detailing hardware setup, version configurations, schema designs, and extensive performance benchmarks across million‑ to billion‑record datasets, and concludes with analysis of memory architectures, data models, and suitable application scenarios.
To meet a demanding monitoring project that required storing up to 400 million records per day in SQL Server 2012, the author details a step‑by‑step journey involving bulk‑copy insertion, removing indexes during load, hourly partitioned tables, strategic index creation, and query optimizations to achieve sub‑second query times.
This article examines three SQL pagination strategies for a 100‑million‑row favorites table, compares their correctness and performance using EXPLAIN analysis, and demonstrates why a GROUP BY approach with proper indexing yields the most reliable and fast results.
