Skip to content
Tagged articles

batch processing

273 articles · Page 1 of 3
LuTiao Programming
LuTiao Programming
Nov 16, 2026 · Backend Development

Why 50K Records Took 40 Seconds: The Hidden Cost of List.contains() in Loops

A permission sync task processing 50,000 records ran in 40 seconds despite sub-second SQL queries; profiling revealed the bottleneck was an O(n²) List.contains() inside a stream filter, and replacing the List with a HashSet cut the diff calculation from 38 seconds to milliseconds, demonstrating how data structure choice dominates batch performance.

Data StructuresHashSetJava
0 likes · 16 min read
Why 50K Records Took 40 Seconds: The Hidden Cost of List.contains() in Loops
Raymond Ops
Raymond Ops
Nov 8, 2026 · Operations

SaltStack Automation from Zero to One: Practical Batch Management Implementation

This article provides a step-by-step guide to implementing SaltStack for operations automation, covering master/minion deployment, key-based authentication, state tree and pillar configuration, grain-based targeting, canary deployments with dry-run verification, batched rollouts, health checks, troubleshooting techniques, and Git-based rollback procedures on Ubuntu 22.04.

Configuration ManagementInfrastructure as CodeOperations Automation
0 likes · 21 min read
SaltStack Automation from Zero to One: Practical Batch Management Implementation
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Oct 30, 2026 · Big Data

Automaker's Large-Scale Multimodal Labeling with EMR Serverless Spark AI Functions

A leading automaker processes hundreds of millions of road images using Alibaba Cloud EMR Serverless Spark AI Functions for scenario extraction and vector embedding, integrating multimodal model calls into Spark jobs via SQL, PySpark, and Ray to enable efficient retrieval and sample selection for autonomous driving training.

AI FunctionEMR Serverless SparkSpark SQL
0 likes · 12 min read
Automaker's Large-Scale Multimodal Labeling with EMR Serverless Spark AI Functions
Data Practical Notes
Data Practical Notes
Oct 27, 2026 · Big Data

Real-Time vs Batch Processing: Why Enterprises Need Both Data Pipelines

This article contrasts real-time and offline computing, showing how they serve distinct business needs—real-time for immediate alerts, batch for accurate periodic analysis—and provides a practical framework for choosing and combining both while keeping key metrics consistent.

FineDataLinkReal-time Computingbatch processing
0 likes · 13 min read
Real-Time vs Batch Processing: Why Enterprises Need Both Data Pipelines
Architecture Development Notes
Architecture Development Notes
Oct 21, 2026 · Artificial Intelligence

Claude Code 2.1.302: Fork Subagents Default On — Batch Tasks Lose Automatic Isolation

Claude Code 2.1.302 enables fork subagents by default in headless and editor entry points, breaking the assumption that batch subtasks run in isolated contexts; the article details trade-offs between fork (shared history, tools, model, cache, permissions) and custom subagents (fresh context, configurable tools/model), and guides choosing based on whether subtasks need shared context or independent isolation.

Agent SDKClaude Codebackground execution
0 likes · 26 min read
Claude Code 2.1.302: Fork Subagents Default On — Batch Tasks Lose Automatic Isolation
Fei's Miscellaneous Talks
Fei's Miscellaneous Talks
Oct 20, 2026 · Backend Development

Feeder Pipelines: Bridging Offline Data Factory to Online Recommendation Engine

This article details Feeder pipelines that transport diverse data types—user features, content tags, embeddings, popularity rankings, and operational rules—from offline computation to online serving, covering batch atomic switches, NRT incremental updates, cursor-based recovery, scheduling platform design, and latency observability practices.

Data PipelineFeature StoreFeeder
0 likes · 30 min read
Feeder Pipelines: Bridging Offline Data Factory to Online Recommendation Engine
Java Tech Workshop
Java Tech Workshop
Oct 19, 2026 · Backend Development

One @Trans Annotation Solves N+1 Queries and Data Translation in Spring Boot

This article introduces Easy-Trans, a Dromara open-source library that uses a single @Trans annotation to automatically handle five types of data translation—dictionary, simple join, cross-service RPC, enum, and custom—while eliminating N+1 queries through batch ID collection and single IN-clause or cache lookups, with configuration details and best practices for Spring Boot applications.

DromaraEasy-TransMyBatis Plus
0 likes · 14 min read
One @Trans Annotation Solves N+1 Queries and Data Translation in Spring Boot
Cilantro Loves Linux
Cilantro Loves Linux
Oct 18, 2026 · Backend Development

Batch Processing Scheduling Architecture: Reliability Beyond Scheduled Triggers

This chapter distinguishes scheduling from batch execution, covering single-machine, cluster, distributed, and pipeline batch patterns, emphasizing reliability through state tracking, idempotency, sharding, checkpoints, and explicit dependencies over fixed timing. It also outlines practical platform capabilities like versioned task definitions, unique run instances, state machines, leases, business idempotency keys, resource quotas, and end-to-end reconciliation.

Distributed SystemsIdempotencyPipeline
0 likes · 9 min read
Batch Processing Scheduling Architecture: Reliability Beyond Scheduled Triggers
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Oct 14, 2026 · Backend Development

Spring Boot Large Batch Request Body? 6 Optimization Strategies

This article presents six optimization strategies for handling oversized batch request bodies in Spring Boot, covering network compression with GZIP, Protobuf adoption, streaming JSON parsing with Jackson JsonParser, early validation layers, asynchronous processing via Kafka, and request chunking to reduce memory pressure and improve throughput.

Chunked RequestsGZIP CompressionJSON Streaming
0 likes · 12 min read
Spring Boot Large Batch Request Body? 6 Optimization Strategies
ByteDance Data Platform
ByteDance Data Platform
Oct 13, 2026 · Artificial Intelligence

LAS AI Operator Automates Viral Clip Editing from Raw Short Dramas

Volcano Engine's LAS Hit Material Editing Operator automates the full pipeline from scene detection and plot understanding to editing decisions and video synthesis for short drama ad campaigns, enabling batch production of differentiated creatives with structured outputs for retrieval and audit.

AI video editingAPILAS
0 likes · 12 min read
LAS AI Operator Automates Viral Clip Editing from Raw Short Dramas
samdeepthink
samdeepthink
Oct 12, 2026 · Backend Development

Java CyclicBarrier Deep Dive: Batch Synchronization Patterns & Design Improvements

This article explains Java's CyclicBarrier through two practical examples: coordinating concurrent API stress tests and batch-importing one million Excel rows, then reveals a design flaw where the main thread participates in the barrier and shows how to separate concerns using CountDownLatch for cleaner batch processing.

API Stress TestingCountDownLatchCyclicBarrier
0 likes · 10 min read
Java CyclicBarrier Deep Dive: Batch Synchronization Patterns & Design Improvements
LuTiao Programming
LuTiao Programming
Sep 30, 2026 · Backend Development

Spring Boot Excel Export OOM: Replacing XSSFWorkbook with SXSSFWorkbook and Batch Processing

The author describes how exporting 500k order records caused a 2GB heap OOM, then fixed it by replacing XSSFWorkbook with streaming SXSSFWorkbook, switching from full entity loading to cursor-based batch queries with DTOs, reusing cell styles, and finally decoupling large exports from HTTP requests via async task processing.

Apache POICursor PaginationExcel Export
0 likes · 14 min read
Spring Boot Excel Export OOM: Replacing XSSFWorkbook with SXSSFWorkbook and Batch Processing
DataFunTalk
DataFunTalk
Sep 21, 2026 · Artificial Intelligence

Jev's First Week: 15 Projects Show How Agents Separate Generation from Judgment

In Jev's first six days, 239 public builds demonstrate a shift where developers offload structured judgments — yes/no, choice, scoring — to a specialized model while reserving LLMs for generation and code for execution, across browser automation, coding agents, batch classification, content scoring, and SQL integration.

Coding AgentJevSystem One model
0 likes · 25 min read
Jev's First Week: 15 Projects Show How Agents Separate Generation from Judgment
Xiaolin Talks Programming
Xiaolin Talks Programming
Sep 18, 2026 · Backend Development

Engineering Large-Scale Excel Imports: Spring Boot + EasyExcel Streaming & Task Isolation

This article details a production-ready engineering approach for importing hundreds of thousands of Excel rows using Spring Boot and EasyExcel, covering streaming parsing, batch validation, async task isolation, idempotency, error reporting, and operational concerns like logging and temp file cleanup.

EasyExcelIdempotencyProduction Engineering
0 likes · 23 min read
Engineering Large-Scale Excel Imports: Spring Boot + EasyExcel Streaming & Task Isolation
Rain's Path to Java Mastery
Rain's Path to Java Mastery
Aug 26, 2026 · Backend Development

Million-Row Excel Import/Export: Streaming, Batching & Async Patterns for Zero-OOM Performance

This guide details a production-grade architecture for million-row Excel import/export using EasyExcel streaming reads, SXSSF streaming writes, JDBC batch inserts with rewriteBatchedStatements, async MQ processing, and resource isolation — achieving 10-100x speedups and sub-100MB memory.

EasyExcelExcel streamingJDBC optimization
0 likes · 32 min read
Million-Row Excel Import/Export: Streaming, Batching & Async Patterns for Zero-OOM Performance
Architect's Guide
Architect's Guide
Aug 26, 2026 · Backend Development

10 Powerful Performance‑Optimization Techniques You Should Try

The article surveys ten practical performance‑optimization tactics—from classic indexing, compression, and caching to prefetching, peak‑shaving, batch processing, and advanced methods such as resource squeezing, horizontal scaling, sharding, and lock‑free designs—explaining their trade‑offs, concrete examples, and when to apply each in real‑world systems.

CompressionHorizontal ScalingIndexing
0 likes · 36 min read
10 Powerful Performance‑Optimization Techniques You Should Try
Linyb Geek Road
Linyb Geek Road
Aug 1, 2026 · Artificial Intelligence

Practical Guide to Cutting LLM Token Costs

This article systematically explains how large‑language‑model token pricing works, identifies eight high‑consumption usage patterns, presents nine actionable optimization principles, and offers a tiered model‑selection framework so engineering teams can reduce token spend by up to 80% without sacrificing result quality.

LLMbatch processingcaching
0 likes · 22 min read
Practical Guide to Cutting LLM Token Costs
YiSu Grain
YiSu Grain
Jul 28, 2026 · Big Data

Day 39: Big Data Architecture – Distributed Storage, Batch vs Stream Processing, and Compute‑Storage Separation

This lesson explains why a single database cannot scale for massive e‑commerce data, introduces the three core pillars of distributed storage—sharding, replication, and horizontal scaling—covers batch and stream processing differences with Spark, Hive, Flink and Storm, compares compute‑storage integration versus separation, and shows how Kafka, HDFS, Spark and Flink fit together in a real‑time data platform.

FlinkKafkaSpark
0 likes · 25 min read
Day 39: Big Data Architecture – Distributed Storage, Batch vs Stream Processing, and Compute‑Storage Separation
Subtle Storm
Subtle Storm
Jul 19, 2026 · Big Data

Core Big Data Technologies Explained with a Restaurant Analogy

The article breaks down the essential big data components—data ingestion, distributed storage, resource management, batch and stream processing, analytics, search, and cluster operations—using a restaurant metaphor and lists typical tools such as Flume, Kafka, HDFS, Spark, Flink, Elasticsearch, and Kubernetes.

Data Ingestionbatch processingbig data
0 likes · 7 min read
Core Big Data Technologies Explained with a Restaurant Analogy
Random Bulletin
Random Bulletin
Jul 17, 2026 · Backend Development

Why Moving from Real‑Time to Batch Is Essential for Scaling to Tens of Millions QPS

Scaling a service from millions to tens of millions of queries per second fails not because of data size but due to per‑request fixed costs, and the article shows how batching aggregates these costs, dramatically boosts throughput, reduces latency, and introduces new challenges such as memory pressure and partial failures.

batch processinghigh QPSlatency tradeoff
0 likes · 17 min read
Why Moving from Real‑Time to Batch Is Essential for Scaling to Tens of Millions QPS
samdeepthink
samdeepthink
Jul 16, 2026 · Databases

Checkpoint‑Based Resumption for Billions‑Row Full Migration

When migrating tens of billions of rows, the article explains how a simple progress‑tracking table and failure‑log table enable automatic checkpoint‑based resumption, stateless execution, and dynamic batch tuning without restarting or rewriting code.

Nacosbatch processingcheckpoint
0 likes · 5 min read
Checkpoint‑Based Resumption for Billions‑Row Full Migration
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Jul 11, 2026 · Backend Development

Boost Performance in Spring Boot with a Single Batch‑Processing Annotation

The article demonstrates how to create a custom @BatchProcess annotation combined with AOP to aggregate high‑frequency requests into batches, persist metadata in Redis, and process them efficiently, thereby reducing connection, I/O, and CPU overhead in distributed, high‑concurrency Spring Boot 3.5.0 applications.

AOPCustom AnnotationPerformance Optimization
0 likes · 14 min read
Boost Performance in Spring Boot with a Single Batch‑Processing Annotation
Xiaolin Talks Programming
Xiaolin Talks Programming
Jul 7, 2026 · Backend Development

Spring Batch for Massive Data Migration: Partitioning, Fault Tolerance & Checkpoint Restart

This article details production-ready Spring Batch architecture for massive data migration, covering core Job/Step/Chunk model, partitioning for parallelism, streaming file reads, JDBC batch tuning, retry/skip fault tolerance, checkpoint restart via BATCH_* tables, memory/cursor trade-offs, dynamic chunk sizing, monitoring metrics, and scheduler-executor decoupling with XXL-JOB or Airflow.

Checkpoint RestartFault ToleranceJDBC optimization
0 likes · 19 min read
Spring Batch for Massive Data Migration: Partitioning, Fault Tolerance & Checkpoint Restart
Cloud Architecture
Cloud Architecture
Jul 6, 2026 · Backend Development

Designing a High‑Concurrency Coupon Expiration System with Task Tables and Batch Processing

The article explains why coupon expiration cannot be handled by a simple scheduled scan and presents a production‑grade architecture that uses a task‑table, four‑plane design, fine‑grained splitting, lease‑based worker coordination, idempotent updates, and observability to reliably expire billions of coupons under peak load.

Concurrencybatch processingcoupon
0 likes · 37 min read
Designing a High‑Concurrency Coupon Expiration System with Task Tables and Batch Processing
Advanced AI Application Practice
Advanced AI Application Practice
Jul 1, 2026 · Operations

Still Copy‑Pasting Bugs? This Skill Inserts Bugs into Feishu Base in 10 Seconds

The bug‑report‑writer v1.1.1 upgrade adds Feishu Base one‑click API writing, cutting single‑bug entry time from two minutes to 10‑15 seconds, boosting overall bug‑submission efficiency by 80% over v1.0 and up to 72‑fold versus manual entry, while supporting batch imports and a six‑dimension quality score.

EfficiencyFeishuautomation
0 likes · 11 min read
Still Copy‑Pasting Bugs? This Skill Inserts Bugs into Feishu Base in 10 Seconds
Machine Learning Algorithms & Natural Language Processing
Machine Learning Algorithms & Natural Language Processing
Jun 28, 2026 · Artificial Intelligence

DSpark Explained in 10 Essential Concepts: System‑Level Engineering Insights

DSpark, DeepSeek’s new LLM inference framework, combines batch processing, speculative decoding, Eagle‑style draft models and DFlash‑style parallel generation with a lightweight sequential head and hardware‑aware scheduling, delivering 60‑85% speedups while preserving model quality.

DeepSeekGPU optimizationLLM inference
0 likes · 12 min read
DSpark Explained in 10 Essential Concepts: System‑Level Engineering Insights
Niu Liu
Niu Liu
Jun 25, 2026 · Backend Development

Don’t Mix Them Up: When to Use RPC, MQ, or Offline Scripts

The article breaks down three fundamental mechanisms—synchronous RPC, asynchronous MQ, and offline scripts—by comparing their lifecycles, IO models, bottlenecks, and consistency guarantees, and provides a concrete decision framework for choosing the right tool for different business scenarios.

ConsistencyMessage QueueRPC
0 likes · 12 min read
Don’t Mix Them Up: When to Use RPC, MQ, or Offline Scripts
Shepherd Advanced Notes
Shepherd Advanced Notes
Jun 24, 2026 · Backend Development

Boosting Throughput 10×: Architecture Evolution and Tuning for High‑Concurrency Batch Processing

The article details how a sluggish batch‑processing system handling millions of records was redesigned with XXL‑JOB sharding, Redis‑based dynamic task distribution, cursor pagination, and selective transaction scopes, achieving nearly ten‑fold throughput improvement while addressing resource contention, load‑balancing, and reliable result reconciliation.

JavaPerformance TuningRedis
0 likes · 19 min read
Boosting Throughput 10×: Architecture Evolution and Tuning for High‑Concurrency Batch Processing
Java Tech Workshop
Java Tech Workshop
Jun 9, 2026 · Backend Development

Spring Batch Basics: Building Efficient SpringBoot Batch Jobs

This article explains why naive for‑loop DB operations fail on large data sets, introduces Spring Batch’s chunk, transaction, retry, skip and restart features, and provides step‑by‑step SpringBoot configurations, code samples for Tasklet and Chunk jobs, database and CSV readers/writers, and manual or scheduled job triggering.

ChunkRetryScheduling
0 likes · 18 min read
Spring Batch Basics: Building Efficient SpringBoot Batch Jobs
Cloud Architecture
Cloud Architecture
May 30, 2026 · Operations

How to Build Production‑Grade Observability Metrics and Alerting for Batch Jobs

The article explains why batch processing tasks often slip out of control, defines a four‑layer observability model covering status, progress, quality and performance, proposes a unified task state machine and event flow, and provides concrete metric, logging, tracing and alerting designs—including Go and Java SDK examples—for reliable production‑level batch job monitoring.

GoJavaMetrics
0 likes · 34 min read
How to Build Production‑Grade Observability Metrics and Alerting for Batch Jobs
Cloud Architecture
Cloud Architecture
May 29, 2026 · Cloud Native

Batch Task Platform on Kubernetes: From Job Wrappers to Scalable Control Plane

The article explains how to design a production‑grade, unified batch‑task platform on Kubernetes that goes beyond a simple job UI, covering unified abstractions, multi‑tenant governance, state‑machine modeling, scalable scheduling, high‑concurrency handling, observability, security, and a phased roadmap for incremental implementation.

Cloud NativeKubernetesScheduling
0 likes · 36 min read
Batch Task Platform on Kubernetes: From Job Wrappers to Scalable Control Plane
Cloud Architecture
Cloud Architecture
May 29, 2026 · Cloud Native

DAG‑as‑Code: Building a Production‑Grade Batch Orchestration System with Argo Workflows

This article explains how to design and operate a scalable, multi‑tenant batch processing platform on Kubernetes using Argo Workflows, covering core concepts, DAG scheduling, concurrency control, controller scaling, artifact handling, event‑driven triggers, and practical best‑practice patterns for production reliability and cost efficiency.

Argo WorkflowsCloud NativeKubernetes
0 likes · 33 min read
DAG‑as‑Code: Building a Production‑Grade Batch Orchestration System with Argo Workflows
Cloud Architecture
Cloud Architecture
May 28, 2026 · Cloud Native

Production‑Ready Guide to Kubernetes Jobs & CronJobs: Controller Mechanics to Batch Platform Design

This article explains how Kubernetes Jobs and CronJobs work under the hood, outlines production‑grade design principles such as idempotency, failure modeling, scaling, observability, and security, and provides concrete YAML configurations and Go code examples for building a reliable, high‑throughput batch processing platform.

CronJobGoJob
0 likes · 46 min read
Production‑Ready Guide to Kubernetes Jobs & CronJobs: Controller Mechanics to Batch Platform Design
Cloud Architecture
Cloud Architecture
Apr 27, 2026 · Backend Development

Building an Enterprise‑Level MyBatis Persistence Layer from Zero to One

The article walks through a real production incident caused by a massive IN‑list query, then presents a complete methodology for designing, implementing, and tuning an enterprise‑grade MyBatis persistence layer—including core execution chain, caching strategies, batch processing, read/write splitting, sharding, observability, and deployment best practices.

MyBatisPerformance TuningSQL
0 likes · 39 min read
Building an Enterprise‑Level MyBatis Persistence Layer from Zero to One
Architect's Guide
Architect's Guide
Apr 27, 2026 · Backend Development

How to Optimize Large Transactions in API Endpoints

The article examines why large database transactions inside API endpoints degrade performance, outlines issues such as data inconsistency, lock contention, undo‑log overhead, and DB pressure, and presents practical optimizations including removing remote RPC calls, using programmatic transactions, batch processing, splitting into smaller transactions, and asynchronous parallel execution with code examples.

Asynchronous ProcessingTransactionsbackend development
0 likes · 9 min read
How to Optimize Large Transactions in API Endpoints
JD Tech
JD Tech
Apr 16, 2026 · Industry Insights

How JD Revolutionized Coupon Search with a Stream‑Batch Unified Architecture

This article analyzes JD's end‑to‑end upgrade of its retail coupon search infrastructure, detailing the business drivers, data‑skew challenges, the shift from dual KV and batch pipelines to a unified stream‑batch model built on Apache Doris, and the resulting performance, resource and stability gains across multiple scenarios.

Apache DorisCoupon SearchPerformance Optimization
0 likes · 12 min read
How JD Revolutionized Coupon Search with a Stream‑Batch Unified Architecture
AI Explorer
AI Explorer
Mar 16, 2026 · Artificial Intelligence

How to Unlock NotebookLM’s Python API for Batch Automation and AI Workflows

The unofficial notebooklm-py library provides a Python API and CLI that give programmatic access to Google NotebookLM’s hidden capabilities, enabling batch import of PDFs or videos, automated generation of podcasts, mind‑maps, and other outputs, and integration into AI agents for scalable research workflows.

AI agentsAI automationNotebookLM
0 likes · 6 min read
How to Unlock NotebookLM’s Python API for Batch Automation and AI Workflows
LuTiao Programming
LuTiao Programming
Mar 14, 2026 · Backend Development

Why Your Spring Boot App Freezes at One Million Records – 5 Proven Techniques to Double Performance

When a Spring Boot application reaches millions of rows, it often suffers from OutOfMemoryErrors, slow queries, and high CPU, but by applying five proven strategies—pagination, streaming, batch processing, indexing, and asynchronous execution—you can halve memory usage and achieve up to ten‑fold speed gains.

IndexingPaginationPerformance Optimization
0 likes · 11 min read
Why Your Spring Boot App Freezes at One Million Records – 5 Proven Techniques to Double Performance
Lakehouse Research Base
Lakehouse Research Base
Feb 12, 2026 · Big Data

Spark & Flink in Lakehouse: Latency, Accuracy & Cost Trade-offs for Batch-Stream Design

This article analyzes how Spark and Flink should divide responsibilities in lakehouse architectures by evaluating three core dimensions—data latency, computation accuracy, and resource cost—arguing that mature batch-stream integration balances business needs against technical overhead rather than pursuing universal real-time processing.

Computation AccuracyData LatencyFlink
0 likes · 9 min read
Spark & Flink in Lakehouse: Latency, Accuracy & Cost Trade-offs for Batch-Stream Design
Architect's Guide
Architect's Guide
Jan 16, 2026 · Databases

How to Safely Update Billions of MySQL Rows Without Overloading Binlog

This article explains why a naïve full‑table UPDATE on massive MySQL tables can cripple replication, explores deep‑pagination and IN‑clause inefficiencies, and presents a batch‑processing strategy using NO_CACHE, FORCE INDEX, and rate‑controlled scripts to perform safe, high‑performance updates.

Full Table UpdatePerformance OptimizationSQL
0 likes · 8 min read
How to Safely Update Billions of MySQL Rows Without Overloading Binlog
SpringMeng
SpringMeng
Jan 8, 2026 · Backend Development

How to Cut API Latency from Seconds to Milliseconds: Elegant Optimization Techniques

This article walks through a series of practical backend techniques—batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelization, indexing, transaction management, pagination, and lock granularity—to dramatically reduce API response times from several seconds to just a few milliseconds.

API optimizationasynchronousbatch processing
0 likes · 9 min read
How to Cut API Latency from Seconds to Milliseconds: Elegant Optimization Techniques
DeWu Technology
DeWu Technology
Dec 25, 2025 · Frontend Development

Build a High‑Performance H5 PAG Player: SDK, Image Editing, Batch Synthesis

This guide details how to implement a full‑stack H5 PAG player for the “Use Basketball to Know Me” activity, covering SDK loading, canvas‑based image manipulation (drag, scale, rotate), dynamic layer and text replacement, real‑time preview synchronization, snapshot export, batch synthesis, performance tuning, and fallback strategies.

CanvasPAGPerformance Optimization
0 likes · 30 min read
Build a High‑Performance H5 PAG Player: SDK, Image Editing, Batch Synthesis
Architect's Tech Stack
Architect's Tech Stack
Dec 24, 2025 · Backend Development

10 Proven Techniques to Supercharge API Performance in Java Applications

This article presents a comprehensive, step‑by‑step guide to optimizing Java‑based API interfaces, covering batch processing, asynchronous execution, caching, pre‑processing, pooling, transaction handling, pagination, SQL tuning, lock granularity, and code restructuring, with practical code examples and diagrams.

API optimizationJavaSQL Tuning
0 likes · 10 min read
10 Proven Techniques to Supercharge API Performance in Java Applications
FunTester
FunTester
Dec 23, 2025 · Backend Development

Mastering Delayed, Priority, and Retry Tasks with River – A Go Queue Deep Dive

This article explains how River, a Go job‑queue library, implements delayed execution, priority handling, exponential‑backoff retries, batch inserts, monitoring, and best‑practice patterns, and compares it with other queue solutions to help developers build reliable, high‑performance background processing pipelines.

GoRiverbatch processing
0 likes · 14 min read
Mastering Delayed, Priority, and Retry Tasks with River – A Go Queue Deep Dive
Ray's Galactic Tech
Ray's Galactic Tech
Dec 12, 2025 · Cloud Native

Mastering Kubernetes Jobs and CronJobs: Complete Guide & Practical Examples

Learn how Kubernetes Jobs and CronJobs enable one‑off and scheduled batch processing, understand their core concepts, key differences, YAML specifications, typical use cases, advanced configurations, monitoring, logging, and cleanup strategies, and see real‑world examples with complete YAML snippets and command‑line tips.

Cloud NativeCronJobJob
0 likes · 8 min read
Mastering Kubernetes Jobs and CronJobs: Complete Guide & Practical Examples
Chen Tian Universe
Chen Tian Universe
Nov 27, 2025 · Fundamentals

A Comprehensive Overview of Modern Bank Core Systems Architecture

This article provides a detailed walkthrough of bank core system architecture, covering its evolution, global overview, key modules such as customer information, account types, deposit and loan cores, payment and settlement, general ledger, card management, and batch processing, illustrated with diagrams and real‑world examples.

Ledgeraccount managementbanking architecture
0 likes · 20 min read
A Comprehensive Overview of Modern Bank Core Systems Architecture
Tech Freedom Circle
Tech Freedom Circle
Nov 16, 2025 · Databases

How Redis Pipeline Can Boost Performance 3‑12× and Impress Interviewers

This article explains Redis Pipeline’s core principle of batching commands to reduce network round‑trips, presents benchmark data showing up to 17‑fold speedups, details real‑world use cases such as cache warm‑up, heartbeat reporting, and high‑traffic events, and provides best‑practice guidelines on batch sizing, error handling, cluster constraints, and comparisons with transactions and Lua scripts.

BenchmarkDistributed SystemsJava
0 likes · 36 min read
How Redis Pipeline Can Boost Performance 3‑12× and Impress Interviewers
dbaplus Community
dbaplus Community
Nov 12, 2025 · Databases

Mastering Data Sync: From Full Loads to Real‑Time CDC in E‑Commerce

This guide walks a new e‑commerce developer through the evolution of order data synchronization—from naïve full‑table loads, through incremental and batch strategies, cursor‑based pagination, performance tuning, and finally to real‑time CDC with message queues—highlighting pitfalls and practical solutions.

CDCCursor PaginationMessage Queue
0 likes · 12 min read
Mastering Data Sync: From Full Loads to Real‑Time CDC in E‑Commerce
Architect-Kip
Architect-Kip
Nov 4, 2025 · Backend Development

How to Optimize Batch Jobs for Better System and Database Performance

This article explains why batch tasks significantly affect system and database performance, and outlines three key strategies—task simplification, resource and performance tuning, and proactive monitoring with fault recovery—to design and optimize batch jobs effectively in enterprise development.

batch processingdatabaseresource isolation
0 likes · 4 min read
How to Optimize Batch Jobs for Better System and Database Performance
Su San Talks Tech
Su San Talks Tech
Oct 29, 2025 · Databases

How to Safely Delete Millions of Rows from a Large Table: Proven Strategies & Code

This article explains why deleting tens of millions of rows is challenging, analyzes transaction, lock, resource, and business impacts, and presents five practical solutions—including batch deletion, table rebuild, partition dropping, online migration, and professional tools—along with best‑practice guidelines and a decision matrix.

Partitioningbatch processingdata deletion
0 likes · 28 min read
How to Safely Delete Millions of Rows from a Large Table: Proven Strategies & Code
Architect-Kip
Architect-Kip
Oct 28, 2025 · Operations

Mastering Failure Recovery: Fast‑Fail, Auto‑Retry, and Resilience Patterns for Distributed Systems

This guide outlines core principles and practical solutions for building resilient backend systems, covering fast‑failure handling, automatic retries with exponential back‑off, circuit‑breaker usage, idempotency, batch job strategies, online transaction patterns, and robust message‑queue processing.

IdempotencyMessage Queuebatch processing
0 likes · 17 min read
Mastering Failure Recovery: Fast‑Fail, Auto‑Retry, and Resilience Patterns for Distributed Systems
Top Architect
Top Architect
Oct 28, 2025 · Backend Development

How I Cut XML‑to‑MySQL Import Time from 300 s to 4 s

This article details a step‑by‑step performance overhaul for importing 60,000+ XML records into MySQL, covering baseline measurements, MySQL batch processing, asynchronous writes with Disruptor, XML parsing optimizations, and tuning MySQL buffers, ultimately reducing total runtime from 300 seconds to just four seconds.

DisruptorJavaPerformance Optimization
0 likes · 14 min read
How I Cut XML‑to‑MySQL Import Time from 300 s to 4 s
Chen Tian Universe
Chen Tian Universe
Oct 14, 2025 · Operations

How Banks Turn Every Transaction into Precise Accounting Records

This article explains the core banking accounting system, covering the transaction‑driven entry engine, multidimensional accounting, internal account management, end‑of‑day batch processing, and a complete loan lifecycle example that shows how each step is recorded, reconciled, and reported.

bankingbatch processingcore accounting
0 likes · 16 min read
How Banks Turn Every Transaction into Precise Accounting Records
Architect's Guide
Architect's Guide
Oct 3, 2025 · Backend Development

Merge Duplicate Requests Using Hystrix Collapser, BatchCollapser, and ConcurrentHashMultiset

This article explains how merging similar or duplicate requests upstream with Hystrix Collapser, a custom BatchCollapser, and Guava's ConcurrentHashMultiset can dramatically reduce downstream load, improve system throughput, and outlines their implementations, configurations, and ideal use‑cases.

HystrixJava Concurrencybackend optimization
0 likes · 15 min read
Merge Duplicate Requests Using Hystrix Collapser, BatchCollapser, and ConcurrentHashMultiset
Baidu Geek Talk
Baidu Geek Talk
Sep 24, 2025 · Big Data

How Feed Real‑Time Data Warehouse Was Re‑Engineered for Speed and Cost Savings

This article explains how Baidu’s Feed real‑time data warehouse was rebuilt using a pure streaming architecture, detailing the limitations of the previous stream‑batch design, the technical solutions—including core/non‑core data separation, metric calculation in streaming, and Parquet storage with Apache Arrow—and the resulting cost reductions, latency improvements, and future roadmap.

Apache ArrowParquetbatch processing
0 likes · 17 min read
How Feed Real‑Time Data Warehouse Was Re‑Engineered for Speed and Cost Savings
Architect's Guide
Architect's Guide
Sep 18, 2025 · Backend Development

Master Spring Batch: From Basics to Advanced Job Configurations

This article provides a comprehensive guide to Spring Batch, covering its purpose, supported business scenarios, core components and interfaces, Maven setup, sample job definitions, parallel execution, decision flows, nested jobs, data reading and writing, validation processing, and REST‑based job scheduling, all illustrated with complete code examples.

JavaSpring BatchSpring Boot
0 likes · 15 min read
Master Spring Batch: From Basics to Advanced Job Configurations
Architect's Guide
Architect's Guide
Aug 22, 2025 · Backend Development

Can JDBC Batch Insertion Speed Up MySQL? 100M Row Performance Test

This article evaluates the efficiency of inserting massive data into MySQL using Java, comparing three strategies—MyBatis without transactions, plain JDBC with and without transactions, and JDBC batch processing—showing that batch processing combined with transactions yields the fastest insertion speed for hundreds of millions of rows.

JDBCJavabatch processing
0 likes · 13 min read
Can JDBC Batch Insertion Speed Up MySQL? 100M Row Performance Test
Data Practical Notes
Data Practical Notes
Aug 13, 2025 · Big Data

Stream vs Batch: Why Unified Stream-Batch Architecture Is the Future of Big Data

This article examines the evolution from separate stream and batch processing to unified stream-batch architecture, explaining the three key unifications (logic, storage, resources), debunking three common misconceptions, and arguing that stream-batch unification is essential for modern real-time decision-making and data-driven business growth.

Data EngineeringFlinkbatch processing
0 likes · 14 min read
Stream vs Batch: Why Unified Stream-Batch Architecture Is the Future of Big Data
mikechen
mikechen
Aug 6, 2025 · Big Data

How to Diagnose and Fix Kafka Message Backlog in High‑Concurrency Environments

In high‑concurrency systems, Kafka message backlog occurs when producers outpace consumers, leading to unprocessed messages that threaten stability and real‑time performance, and this article explains the root causes and provides practical producer‑side and consumer‑side optimization techniques to resolve the issue.

batch processingconsumer optimizationhigh concurrency
0 likes · 5 min read
How to Diagnose and Fix Kafka Message Backlog in High‑Concurrency Environments
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
Jul 22, 2025 · Big Data

How Apache SeaTunnel Revolutionizes Heterogeneous Data Integration with Decoupled Connectors

This article explores how Apache SeaTunnel addresses modern data integration challenges by providing a high‑performance, distributed, plugin‑based platform that decouples connectors from execution engines, enabling seamless batch and streaming synchronization across heterogeneous sources such as databases, message queues, and data lakes.

Apache SeaTunnelConnector Architecturebatch processing
0 likes · 24 min read
How Apache SeaTunnel Revolutionizes Heterogeneous Data Integration with Decoupled Connectors
macrozheng
macrozheng
Jul 14, 2025 · Backend Development

Master Spring Batch: Core Concepts, Architecture, and Best Practices

This article provides a comprehensive guide to Spring Batch, covering its purpose, architecture, core components such as Job, Step, ItemReader/Writer/Processor, chunk processing, skip strategies, and practical tips for configuring and optimizing batch jobs in Java applications.

ChunkJavaJob
0 likes · 21 min read
Master Spring Batch: Core Concepts, Architecture, and Best Practices
Instant Consumer Technology Team
Instant Consumer Technology Team
Jul 9, 2025 · Cloud Native

Scaling a Financial Accounting System to 100k TPS with Cloud‑Native Microservices

This article examines how a ten‑year‑old financial accounting platform transformed from a monolithic design into a cloud‑native, micro‑service architecture that achieved massive scalability, high availability, and 24‑hour real‑time processing through distributed batch scheduling, elastic scaling, and intelligent fault‑tolerance.

batch processingcloud-nativefinancial systems
0 likes · 14 min read
Scaling a Financial Accounting System to 100k TPS with Cloud‑Native Microservices
Code Ape Tech Column
Code Ape Tech Column
Jul 8, 2025 · Backend Development

Mastering Spring Batch: Real-World Use Cases and Hands‑On Guide

This comprehensive guide explains why batch processing is essential, walks through typical banking, e‑commerce, logging and medical data scenarios, details Spring Batch's core architecture and components, provides step‑by‑step setup and code examples, and presents a production‑grade bank reconciliation case with monitoring and troubleshooting tips.

Performance OptimizationSpring BatchSpring Boot
0 likes · 27 min read
Mastering Spring Batch: Real-World Use Cases and Hands‑On Guide
Top Architect
Top Architect
Jul 5, 2025 · Databases

How I Cut XML‑to‑MySQL Import Time from 300 s to 4 s with Batch, RewriteBatchedStatements, and Multithreading

This article walks through optimizing a 60,000‑row XML‑to‑MySQL import by profiling the environment, measuring baseline performance, and applying JDBC batch rewriting, write aggregation, and asynchronous writes with LMAX Disruptor, ultimately reducing execution time from 300 seconds to about four seconds while keeping memory usage reasonable.

DisruptorJDBCJava
0 likes · 15 min read
How I Cut XML‑to‑MySQL Import Time from 300 s to 4 s with Batch, RewriteBatchedStatements, and Multithreading
Java Interview Crash Guide
Java Interview Crash Guide
Jul 3, 2025 · Backend Development

10 Proven Strategies to Supercharge API Performance in Java

This article presents a comprehensive, step‑by‑step guide to optimizing API latency by applying batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelism, indexing, transaction management, code refactoring, pagination, SQL tuning, and fine‑grained locking techniques.

API optimizationSQL Tuningbatch processing
0 likes · 10 min read
10 Proven Strategies to Supercharge API Performance in Java
Code Ape Tech Column
Code Ape Tech Column
Jun 30, 2025 · Backend Development

Mastering Software Performance: 6 Time‑Space Trade‑offs and 4 Advanced Parallel Techniques

This article explores practical performance‑optimization techniques, covering six fundamental time‑for‑space trade‑offs such as indexing, compression, caching, prefetching, peak‑shaving, and batch processing, followed by four advanced methods that boost parallelism like resource draining, horizontal scaling, sharding, and lock‑free programming.

BackendCompressionIndexing
0 likes · 22 min read
Mastering Software Performance: 6 Time‑Space Trade‑offs and 4 Advanced Parallel Techniques
Architect's Tech Stack
Architect's Tech Stack
Jun 22, 2025 · Backend Development

Boost API Performance: 12 Proven Backend Optimization Techniques

This article presents a comprehensive set of twelve backend optimization strategies—including batch processing, asynchronous execution, caching, pooling, parallelism, indexing, transaction management, and SQL tuning—to dramatically reduce API latency and improve overall system efficiency.

API optimizationBackend PerformanceDatabase Indexing
0 likes · 9 min read
Boost API Performance: 12 Proven Backend Optimization Techniques
Java Captain
Java Captain
Jun 10, 2025 · Backend Development

Why Spring Batch? Real‑World Scenarios, Core Architecture and Hands‑On Guide

This article explains the necessity of batch processing, presents typical use cases such as daily interest calculation, e‑commerce order archiving, log analysis and medical data migration, then dives deep into Spring Batch's core components, provides step‑by‑step code examples, performance‑tuning tips, production‑grade fault‑tolerance, monitoring solutions and a comprehensive FAQ.

JavaSpring Batchbatch processing
0 likes · 20 min read
Why Spring Batch? Real‑World Scenarios, Core Architecture and Hands‑On Guide
Senior Tony
Senior Tony
Jun 8, 2025 · Backend Development

Why Kafka Handles Millions of Messages Per Second: Batch, Partition, Zero‑Copy, and Compression Explained

This article breaks down the core techniques that give Kafka its high‑throughput capability, including producer batch settings (batch.size, linger.ms), broker append‑only writes, consumer poll configuration, partition distribution, zero‑copy data transfer, dual‑thread processing, and configurable compression algorithms.

CompressionConsumerPartitioning
0 likes · 8 min read
Why Kafka Handles Millions of Messages Per Second: Batch, Partition, Zero‑Copy, and Compression Explained
Tencent Cloud Developer
Tencent Cloud Developer
May 8, 2025 · Big Data

How Setats Unifies Stream, Batch, and Incremental Processing for Real‑Time Data Lakes

At the 2025 DA Data+AI Conference in Shanghai, Tencent Cloud unveiled Setats—a unified stream‑batch‑incremental engine that cuts system costs, delivers second‑level data visibility and real‑time changelog generation, and demonstrates measurable performance gains in automotive IoT analytics while integrating tightly with the WeData platform.

batch processingbig data architecturedata lake
0 likes · 5 min read
How Setats Unifies Stream, Batch, and Incremental Processing for Real‑Time Data Lakes
Wukong Talks Architecture
Wukong Talks Architecture
Apr 27, 2025 · Backend Development

Evolution of a Batch Processing System: From Centralized to Configurable and Localized Architectures

This article examines the evolution of a merchant batch processing system, detailing its transition from a centralized, tightly‑coupled architecture to a configurable, SPI‑based design and finally to a localized task‑reporting model, while discussing code reuse, scheduling strategies, isolation techniques, and performance challenges.

JavaTask Schedulingbackend development
0 likes · 18 min read
Evolution of a Batch Processing System: From Centralized to Configurable and Localized Architectures
Architect
Architect
Apr 9, 2025 · Backend Development

Merging Requests and Batch Querying in Spring Boot to Reduce Database Connections

This article explains how to merge concurrent user requests into a single batch SQL query using Java's LinkedBlockingQueue, ScheduledThreadPoolExecutor and CompletableFuture in a Spring Boot application, thereby saving database connections and improving performance under high concurrency.

Spring Bootbatch processingdatabase optimization
0 likes · 13 min read
Merging Requests and Batch Querying in Spring Boot to Reduce Database Connections
DataFunTalk
DataFunTalk
Apr 9, 2025 · Big Data

Highlights of the Apache Hudi Asia Technical Salon Hosted by Kuaishou – Practices and Innovations from Leading Companies

The Kuaishou‑hosted Apache Hudi Asia technical salon gathered over 230 attendees and featured seven experts from Kuaishou, Meituan, TikTok, Huawei, JD and others, who shared best practices, architecture designs, and performance optimizations for large‑scale data lake applications across AI, BI, and real‑time workloads.

AIApache Hudibatch processing
0 likes · 14 min read
Highlights of the Apache Hudi Asia Technical Salon Hosted by Kuaishou – Practices and Innovations from Leading Companies
Su San Talks Tech
Su San Talks Tech
Apr 2, 2025 · Backend Development

How to Import Millions of Excel Rows in Seconds: 4 Proven Performance Hacks

This article analyzes why traditional Excel import methods crash under massive loads and presents four practical optimization techniques—including streaming parsing, batch inserts, asynchronous processing, and parallel sharding—backed by code samples, configuration tips, and real‑world performance benchmarks for importing millions of rows efficiently.

BackendJavaSpring
0 likes · 10 min read
How to Import Millions of Excel Rows in Seconds: 4 Proven Performance Hacks
JavaEdge
JavaEdge
Mar 30, 2025 · Artificial Intelligence

How GenAI Can Transform E‑Commerce Product Review Analysis

This article examines the critical role of product reviews for buyers and sellers, outlines the limitations of traditional review processing, and proposes a GenAI‑powered solution—including platform and model choices, batch inference, and semantic search—to efficiently analyze large‑scale e‑commerce feedback.

GenAINLPSemantic Search
0 likes · 12 min read
How GenAI Can Transform E‑Commerce Product Review Analysis
macrozheng
macrozheng
Mar 28, 2025 · Backend Development

Boost API Performance: 12 Proven Backend Optimization Techniques

This article presents a comprehensive set of backend optimization strategies—including batch processing, asynchronous execution, caching, pre‑processing, pooling, parallelism, indexing, transaction management, program refactoring, pagination, SQL tuning, and fine‑grained locking—to dramatically reduce API latency and improve system efficiency.

API optimizationBackend PerformanceSQL indexing
0 likes · 10 min read
Boost API Performance: 12 Proven Backend Optimization Techniques
Java Captain
Java Captain
Mar 21, 2025 · Backend Development

Request Merging and Batch Processing in Java Spring Boot to Reduce Database Connections

This article explains how to merge multiple user‑detail requests into a single database query using a blocking queue, scheduled thread pool, and CompletableFuture in Spring Boot, providing code examples, a high‑concurrency test, and discussion of trade‑offs such as added latency and timeout handling.

CompletableFutureConcurrencyJava
0 likes · 13 min read
Request Merging and Batch Processing in Java Spring Boot to Reduce Database Connections
Architect's Tech Stack
Architect's Tech Stack
Mar 17, 2025 · Backend Development

Comprehensive Interface Performance Optimization Strategies

This article presents a systematic guide to improving API response times by applying batch processing, asynchronous execution, caching, pooling, parallelism, indexing, transaction management, pagination, SQL tuning, and proper lock granularity, supplemented with practical Java code examples and diagrams.

Lock GranularitySQL paginationasynchronous
0 likes · 10 min read
Comprehensive Interface Performance Optimization Strategies
Java Backend Technology
Java Backend Technology
Mar 8, 2025 · Backend Development

How to Merge Concurrent Requests in Spring Boot and Save Database Connections

This article explains how to batch multiple user‑info requests on the server side, merge them into a single SQL query using a blocking queue and ScheduledThreadPoolExecutor, and return the results individually, thereby reducing database connection usage and improving performance under high concurrency.

CompletableFutureJava ConcurrencySpring Boot
0 likes · 13 min read
How to Merge Concurrent Requests in Spring Boot and Save Database Connections
macrozheng
macrozheng
Mar 5, 2025 · Backend Development

How to Merge Concurrent Requests in Spring Boot to Save Database Connections

This article explains how to combine multiple simultaneous user requests on the server side using a queue, scheduled thread pool and CompletableFuture in Spring Boot, reducing database connections while handling high concurrency, and discusses implementation details, testing, and potential pitfalls.

ConcurrencySpring Bootbatch processing
0 likes · 15 min read
How to Merge Concurrent Requests in Spring Boot to Save Database Connections
Top Architect
Top Architect
Jan 16, 2025 · Backend Development

Optimizing XML‑to‑MySQL Data Import: Reducing Execution Time from 300 s to 4 s

This article describes how to dramatically speed up the import of 60,000+ XML records into MySQL by analyzing the original environment, measuring baseline performance, and applying a series of backend optimizations—including MySQL batch mode, rewriteBatchedStatements, multithreaded asynchronous writes with Disruptor, and XML parsing improvements—ultimately cutting the total runtime from five minutes to just a few seconds.

DisruptorJavaMultithreading
0 likes · 14 min read
Optimizing XML‑to‑MySQL Data Import: Reducing Execution Time from 300 s to 4 s