batch insert | BestHub

Code Ape Tech Column

May 9, 2025 · Databases

Efficient Strategies for Importing One Billion Records into MySQL

This article explains how to import 1 billion 1 KB log records stored in HDFS or S3 into MySQL by analyzing single‑table limits, using batch inserts, choosing storage engines, sharding, optimizing file‑reading methods, and coordinating distributed tasks with Redis, Redisson, and Zookeeper to ensure ordered, reliable, and high‑throughput data loading.

KafkaMySQLPerformance Tuning

0 likes · 19 min read

Efficient Strategies for Importing One Billion Records into MySQL

Java Tech Enthusiast

May 5, 2025 · Backend Development

Advanced MyBatis-Plus Features: Batch Insert, Logical Delete, Auto Field Fill, Type Handlers, Dynamic Table Names, Multi-Tenant

The article demonstrates advanced MyBatis‑Plus techniques—including high‑speed batch inserts with rewriteBatchedStatements, logical deletes via global config, automatic timestamp and user fields through MetaObjectHandler, JSON serialization with JacksonTypeHandler, dynamic table naming via an interceptor, and a brief multi‑tenant overview—to streamline development and reduce boilerplate code.

Dynamic TableJavaMyBatis-Plus

0 likes · 16 min read

Advanced MyBatis-Plus Features: Batch Insert, Logical Delete, Auto Field Fill, Type Handlers, Dynamic Table Names, Multi-Tenant

Java Tech Enthusiast

Feb 9, 2025 · Databases

Optimizing Database Insert Operations and Batch Insertion Strategies

The article explains how inserts work in relational databases, why fixed‑size pages and batch transactions dramatically reduce I/O and lock contention, and provides practical MyBatis guidelines—using the foreach tag, ExecutorType.BATCH, and an appropriate batchSize while estimating memory and disk limits to choose an optimal batch size and commit only once.

DatabaseMyBatisMySQL

0 likes · 12 min read

Optimizing Database Insert Operations and Batch Insertion Strategies

Zhuanzhuan Tech

Nov 28, 2024 · Backend Development

Boosting MyBatis-Plus Batch Insert Performance by 2000% with rewriteBatchedStatements and Pre‑Generated IDs

This article explains how to dramatically improve MyBatis-Plus batch insertion speed—up to 2000%—by enabling rewriteBatchedStatements, pre‑generating primary keys to handle foreign‑key relationships, applying proper JDBC batch settings, and using asynchronous multithreaded insertion with optimized connection‑pool and executor configurations.

ID generationJDBCMyBatis-Plus

0 likes · 21 min read

Boosting MyBatis-Plus Batch Insert Performance by 2000% with rewriteBatchedStatements and Pre‑Generated IDs

Architect's Tech Stack

Nov 6, 2024 · Backend Development

Optimizing MyBatis Batch Insert Performance with ExecutorType.BATCH and Proper Value Chunking

This article explains why using MyBatis foreach for bulk inserts can cause severe performance degradation, analyzes the underlying cost of large prepared statements, and demonstrates how switching to ExecutorType.BATCH or limiting each INSERT to 20‑50 rows dramatically improves insertion speed.

ExecutorType.BATCHJavaMyBatis

0 likes · 8 min read

Optimizing MyBatis Batch Insert Performance with ExecutorType.BATCH and Proper Value Chunking

Top Architect

Nov 2, 2024 · Backend Development

Optimizing MyBatis-Plus Batch Insert Performance with rewriteBatchedStatements and Multithreading

This article explains how to dramatically speed up MyBatis-Plus batch inserts by enabling the rewriteBatchedStatements JDBC property, creating custom batch insert/update methods, and applying multithreading, providing code examples, performance test results, and practical recommendations for handling large data synchronizations.

DatabaseJavaMyBatis-Plus

0 likes · 15 min read

Optimizing MyBatis-Plus Batch Insert Performance with rewriteBatchedStatements and Multithreading

Architect's Guide

Oct 31, 2024 · Databases

Designing an Efficient Pipeline for Importing One Billion Records into MySQL

This article presents a comprehensive engineering guide for importing one billion 1 KB unstructured log records stored in HDFS or S3 into MySQL, covering data sizing, B‑tree limits, batch insertion strategies, storage‑engine choices, sharding, file‑reading techniques, concurrency control, and reliable task coordination using Redis, Redisson, and Zookeeper.

Data ShardingMySQLPerformance Optimization

0 likes · 17 min read

Designing an Efficient Pipeline for Importing One Billion Records into MySQL

Java Architect Essentials

Oct 29, 2024 · Backend Development

Optimizing Batch Insert Performance with MyBatis-Plus: Configuration, Custom Methods, and Multithreading

This article explains why MyBatis-Plus's saveBatch() can be slow for large data sets, shows how enabling rewriteBatchedStatements in the JDBC URL dramatically speeds up inserts, and demonstrates custom batch insert/update methods and multithreaded execution to further improve performance, complete with code examples and test results.

DatabaseJavaMyBatis-Plus

0 likes · 14 min read

Optimizing Batch Insert Performance with MyBatis-Plus: Configuration, Custom Methods, and Multithreading

Architect's Guide

Sep 5, 2024 · Databases

Strategies for Fast Import of 1 Billion Records into MySQL

To import one billion 1 KB log records stored in HDFS or S3 into MySQL efficiently, the article examines data partitioning, B‑tree index limits, batch insertion, storage engine choices, concurrency control, file‑reading methods, task scheduling with Redis, Redisson, and Zookeeper for reliable, ordered, high‑throughput loading.

Data PartitioningMySQLRedisson

0 likes · 18 min read

Strategies for Fast Import of 1 Billion Records into MySQL

Top Architect

Aug 11, 2024 · Backend Development

Efficient Insertion of 300,000 Records Using MyBatis and JDBC

This article demonstrates how to efficiently insert 300,000 rows into a MySQL table by using MyBatis batch operations, JDBC batch processing, and various performance optimizations such as batch size tuning, transaction management, and connection pooling.

JDBCJavaMyBatis

0 likes · 17 min read

Efficient Insertion of 300,000 Records Using MyBatis and JDBC

Architecture Digest

Jul 24, 2024 · Backend Development

Improving Million-Scale Data Insertion Efficiency with Multithreaded Batch Processing in Spring Boot

This article demonstrates how to boost the insertion speed of over two million records by using Spring Boot with MyBatis‑Plus, a ThreadPoolTaskExecutor for multithreaded batch inserts, and detailed performance testing that shows a reduction from 5.75 minutes to 1.67 minutes.

Performance TestingPostgreSQLSpring Boot

0 likes · 6 min read

Improving Million-Scale Data Insertion Efficiency with Multithreaded Batch Processing in Spring Boot

Selected Java Interview Questions

Jul 12, 2024 · Backend Development

Efficient Large File Upload and Database Insertion with Spring Boot, Vue, and Multithreading

This article demonstrates how to implement a high‑performance file upload system using Vue's Element Plus component on the frontend and Spring Boot on the backend, comparing single‑row inserts, batch inserts, and a multithreaded producer‑consumer approach to dramatically reduce MySQL insertion time.

File UploadMySQLPerformance

0 likes · 10 min read

Efficient Large File Upload and Database Insertion with Spring Boot, Vue, and Multithreading

Top Architect

Jul 5, 2024 · Databases

Efficient Insertion of 300,000 Records Using MyBatis and JDBC

This article demonstrates how to insert 300,000 rows into a MySQL table efficiently by defining a User entity, configuring MyBatis and JDBC, comparing direct batch insertion, row‑by‑row insertion, and optimized batch strategies with configurable batch sizes and wait times, and provides performance results and best‑practice recommendations.

DatabaseJDBCJava

0 likes · 15 min read

Code Ape Tech Column

Jun 14, 2024 · Databases

Designing an Efficient Import of 1 Billion Records into MySQL: Architecture, Batch Loading, Sharding, and Concurrency Control

This article analyzes how to import one billion 1 KB log records stored in HDFS or S3 into MySQL by evaluating single‑table limits, choosing storage engines, designing sharding, batch insertion, file‑reading strategies, task coordination, and distributed locking to achieve high‑throughput and ordered writes.

MySQLPerformancebatch insert

0 likes · 19 min read

Designing an Efficient Import of 1 Billion Records into MySQL: Architecture, Batch Loading, Sharding, and Concurrency Control

macrozheng

Apr 26, 2024 · Databases

Boost MySQL Insert Performance with Batch Techniques and MyISAM

This article explains how to dramatically improve MySQL insert speed by adjusting database settings, choosing the MyISAM engine, and applying batch insertion methods such as concatenated SQL, transaction‑wrapped inserts, and MyBatis‑Flex saveBatch, complete with test code and performance results.

Database OptimizationInsert PerformanceMyBatis

0 likes · 10 min read

Boost MySQL Insert Performance with Batch Techniques and MyISAM

IT Services Circle

Apr 18, 2024 · Databases

Improving MySQL Insert Performance with Batch Operations and MyBatis Flex

This article explains how to boost MySQL insertion speed by adjusting server hardware, tuning database parameters, choosing the MyISAM engine, and especially applying three batch‑insert techniques—SQL concatenation, MyBatis‑Flex saveBatch, and manual transaction batching—while showing test results and configuration tips.

Database OptimizationMyBatisMySQL

0 likes · 8 min read

Improving MySQL Insert Performance with Batch Operations and MyBatis Flex

Selected Java Interview Questions

Apr 8, 2024 · Databases

Designing High‑Performance Import of 1 Billion Records into MySQL

This article analyzes how to import one billion 1 KB log records stored in HDFS or S3 into MySQL efficiently, covering constraints, single‑table limits, batch insertion, storage‑engine choices, sharding, file‑reading techniques, task reliability, and concurrency control to achieve optimal throughput.

MySQLPerformance Tuningbatch insert

0 likes · 19 min read

Designing High‑Performance Import of 1 Billion Records into MySQL

Java Tech Enthusiast

Apr 5, 2024 · Databases

Efficient Insertion of 300,000 Records Using MyBatis and JDBC

The article shows how to efficiently load 300,000 MySQL rows in Java by comparing MyBatis and plain-JDBC batch inserts, demonstrating that batching 1,000‑5,000 records per transaction with proper transaction control, connection pooling, and MySQL tuning reduces insertion time from hours to seconds.

DatabaseJDBCJava

0 likes · 13 min read

Top Architect

Mar 31, 2024 · Databases

Inserting 300,000 Records with MyBatis and JDBC: Batch vs Loop Approaches and Performance Optimization

This article demonstrates how to insert 300,000 rows into a MySQL table using MyBatis and JDBC, compares bulk batch insertion with per‑row looping, discusses packet size limits, transaction handling, performance metrics, and provides practical code snippets and optimization tips for efficient large‑scale data loading.

JDBCMyBatisMySQL

0 likes · 16 min read

Inserting 300,000 Records with MyBatis and JDBC: Batch vs Loop Approaches and Performance Optimization

Selected Java Interview Questions

Mar 18, 2024 · Backend Development

Efficient Insertion of 300,000 Records Using MyBatis and JDBC in Java

This article demonstrates various methods for inserting 300,000 rows into a MySQL database using Java, including direct batch insertion with MyBatis, row‑by‑row insertion, and optimized batch strategies for both MyBatis and JDBC, along with performance results and practical tips.

JDBCJavaMyBatis

0 likes · 15 min read

Efficient Insertion of 300,000 Records Using MyBatis and JDBC in Java