Master Spring Batch: Core Concepts, Architecture, and Practical Tips

Spring Batch Overview

Spring Batch is a lightweight, comprehensive batch‑processing framework built on the Spring ecosystem. It enables automated, high‑volume data handling without user interaction, supporting time‑based events, large‑scale repetitive processing, and integration of internal or external data sources.

Typical batch use cases include reading massive records from databases, files, or queues, processing them, and writing the transformed data back to a target system.

Core Concepts

Job and Step

A Job represents the entire batch process and is composed of one or more Step objects. Each step defines its own ItemReader, ItemProcessor, and ItemWriter.

JobInstance and JobParameters

JobInstance identifies a logical execution of a job, while JobParameters provide the runtime data (e.g., execution date) that distinguishes different instances.

JobExecution and StepExecution

JobExecution records a single attempt to run a JobInstance, exposing status, start/end times, and exit codes. StepExecution tracks the execution details of an individual step, including commit counts and timestamps.

ExecutionContext

Both JobExecution and StepExecution have an ExecutionContext , a key‑value store for persisting state between restarts.

JobRepository and JobLauncher

JobRepository persists metadata for jobs, steps, and executions. JobLauncher starts a job with a given set of JobParameters.

ItemReader / ItemWriter / ItemProcessor

These abstractions separate data input, transformation, and output. Spring Batch supplies many implementations, such as JdbcPagingItemReader, JdbcCursorItemReader, and various writers for databases, files, or messaging systems.

@Bean
public JdbcPagingItemReader<CustomerCredit> itemReader(DataSource dataSource, PagingQueryProvider queryProvider) {
    Map<String, Object> params = new HashMap<>();
    params.put("status", "NEW");
    return new JdbcPagingItemReaderBuilder<CustomerCredit>()
            .name("creditReader")
            .dataSource(dataSource)
            .queryProvider(queryProvider)
            .parameterValues(params)
            .rowMapper(customerCreditMapper())
            .pageSize(1000)
            .build();
}

Chunk Processing

Spring Batch groups records into a configurable chunk . The framework reads items until the chunk size is reached, then writes them in a single transaction, improving throughput.

Skip Strategy and Failure Handling

Batch steps can be configured to skip a limited number of exceptions using skipLimit(), skip(), and noSkip(). For example, setting skipLimit(10) allows up to ten tolerated exceptions before the step fails.

Practical Tips

Design batch jobs with clear, simple step logic to avoid unnecessary complexity.

Keep data processing close to storage to reduce I/O overhead.

Allocate sufficient memory at startup to prevent runtime reallocations.

Implement checksum or record‑count validation to ensure data integrity.

Perform load testing with realistic data volumes before production deployment.

Disabling Automatic Job Launch

Spring Boot automatically runs all defined jobs on application start. To prevent this, add the following property:

spring.batch.job.enabled=false

Memory Exhaustion Issue

If a reader loads the entire dataset into memory, the JVM may throw a "Resource exhaustion event: the JVM was unable to allocate memory from the heap" error. Mitigation strategies include:

Switch to a paging reader (e.g., JdbcPagingItemReader) to fetch data in smaller batches.

Increase the JVM heap size or allocate more service memory.

Source: blog.csdn.net/topdeveloperr/article/details/84337956