Master Spring Batch: Build a File‑to‑Database Batch Job with Spring Boot
This tutorial walks through using Spring Batch with Spring Boot 2.6.12 to create a batch job that reads CSV data, processes each record, and writes the results into a MySQL database, covering architecture, configuration, code components, and execution steps.
Environment: Spring Boot 2.6.12 + Spring Batch 4.2.7.
Spring Batch is a lightweight, fully Spring‑based batch processing framework that provides enterprise‑grade features such as logging, transaction management, job restart, skip, and resource handling.
Business scenarios
Periodic batch submission
Parallel batch processing
Stage‑based, message‑driven processing
Large‑scale parallel batch
Manual or scheduled restart after failure
Ordered step execution (workflow‑driven batch)
Partial processing with record skipping
Whole‑batch transaction for small batches or existing scripts
Technical goals
Let batch developers focus on business logic while the framework handles infrastructure.
Clear separation between infrastructure, execution environment, and batch applications.
Provide a common core execution service that can be implemented across projects.
Offer simple, out‑of‑the‑box core interfaces with default implementations.
Enable easy configuration, customization, and extension via Spring.
Make core services replaceable or extensible without affecting the infrastructure layer.
Simple deployment model using a Maven‑built JAR that is completely separate from the application.
Spring Batch architecture
The layered architecture highlights three high‑level components: Application, Core, and Infrastructure. The Application layer contains all batch jobs and custom code written by developers. The Core layer provides runtime classes such as JobLauncher, Job, and Step. Both layers sit on top of the shared Infrastructure, which offers common readers, writers, and services (e.g., RetryTemplate).
Development process
The example reads a file, processes each record, and stores the data in a database.
1. Add dependencies
<code><dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
<version>6.0.7.Final</version>
</dependency></code>2. Application configuration (application.yml)
<code>spring:
datasource:
driverClassName: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://localhost:3306/batch?serverTimezone=GMT%2B8
username: root
password: ******
type: com.zaxxer.hikari.HikariDataSource
hikari:
minimumIdle: 10
maximumPoolSize: 200
autoCommit: true
idleTimeout: 30000
poolName: MasterDatabookHikariCP
maxLifetime: 1800000
connectionTimeout: 30000
connectionTestQuery: SELECT 1
---
spring:
jpa:
generateDdl: false
hibernate:
ddlAuto: update
openInView: true
show-sql: true
---
spring:
batch:
job:
enabled: false # disable auto‑run
initialize-schema: always # auto‑create DB scripts</code>3. Enable batch processing
<code>@Configuration
@EnableBatchProcessing
public class BatchConfig extends DefaultBatchConfigurer {
}</code>4. Job launcher bean
<code>@Override
protected JobLauncher createJobLauncher() throws Exception {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(createJobRepository());
jobLauncher.afterPropertiesSet();
return jobLauncher;
}</code>5. Job repository bean
<code>@Resource
private PlatformTransactionManager transactionManager;
@Override
protected JobRepository createJobRepository() throws Exception {
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDatabaseType("mysql");
factory.setTransactionManager(transactionManager);
factory.setDataSource(dataSource);
factory.afterPropertiesSet();
return factory.getObject();
}</code>6. Define the Job
<code>@Bean
public Job myJob(JobBuilderFactory builder, @Qualifier("myStep") Step step) {
return builder.get("myJob")
.incrementer(new RunIdIncrementer())
.flow(step)
.end()
.listener(jobExecutionListener)
.build();
}</code>7. ItemReader (CSV file)
<code>@Bean
public ItemReader<Person> reader() {
FlatFileItemReader<Person> reader = new FlatFileItemReader<>();
reader.setResource(new ClassPathResource("cvs/persons.cvs"));
reader.setLineMapper(new DefaultLineMapper<Person>() {{
setLineTokenizer(new DelimitedLineTokenizer(",") {{ setNames("id", "name"); }});
setFieldSetMapper(new BeanWrapperFieldSetMapper<Person>() {{ setTargetType(Person.class); }});
}});
return reader;
}</code>8. ItemProcessor
<code>@Bean
public ItemProcessor<Person, Person2> processorPerson() {
return new ItemProcessor<Person, Person2>() {
@Override
public Person2 process(Person item) throws Exception {
Person2 p = new Person2();
p.setId(item.getId());
p.setName(item.getName() + ", pk");
return p;
}
};
}</code>9. ItemWriter (JPA)
<code>@Resource
private Validator<Person> validator;
@Resource
private EntityManagerFactory entityManagerFactory;
@Bean
public ItemWriter<Person2> writerPerson() {
JpaItemWriterBuilder<Person2> builder = new JpaItemWriterBuilder<>();
builder.entityManagerFactory(entityManagerFactory);
return builder.build();
}</code>10. Step definition
<code>@Bean
public Step myStep(StepBuilderFactory stepBuilderFactory,
ItemReader<Person> reader,
ItemWriter<Person> writer,
ItemProcessor<Person, Person> processor) {
return stepBuilderFactory.get("myStep")
.<Person, Person>chunk(2)
.reader(reader).faultTolerant().retryLimit(3).retry(Exception.class)
.skip(Exception.class).skipLimit(2).listener(new MyReadListener())
.processor(processor)
.writer(writer).faultTolerant().skip(Exception.class).skipLimit(2)
.listener(new MyWriteListener())
.build();
}</code>11. Listeners
<code>public class MyReadListener implements ItemReadListener<Person> {
private Logger logger = LoggerFactory.getLogger(MyReadListener.class);
@Override public void beforeRead() {}
@Override public void afterRead(Person item) {
System.out.println("reader after: " + Thread.currentThread().getName());
}
@Override public void onReadError(Exception ex) {
logger.info("读取数据错误：{}", ex);
}
}
@Component
public class MyWriteListener implements ItemWriteListener<Person> {
private Logger logger = LoggerFactory.getLogger(MyWriteListener.class);
@Override public void beforeWrite(List<? extends Person> items) {}
@Override public void afterWrite(List<? extends Person> items) {
System.out.println("writer after: " + Thread.currentThread().getName());
}
@Override public void onWriteError(Exception exception, List<? extends Person> items) {
try {
logger.info(String.format("%s%n", exception.getMessage()));
for (Person item : items) {
logger.info(String.format("Failed writing BlogInfo : %s", item.toString()));
}
} catch (Exception e) { e.printStackTrace(); }
}
}</code>12. Entity class
<code>@Entity
@Table(name = "t_person")
public class Person {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Integer id;
private String name;
}</code>13. Controller to launch the job
<code>@RestController
@RequestMapping("/demo")
public class DemoController {
@Resource @Qualifier("myJob") private Job job;
@Resource private JobLauncher launcher;
@GetMapping("/index")
public Object index() {
JobParameters jobParameters = new JobParametersBuilder().toJobParameters();
try {
launcher.run(job, jobParameters);
} catch (JobExecutionAlreadyRunningException | JobRestartException |
JobInstanceAlreadyCompleteException | JobParametersInvalidException e) {
e.printStackTrace();
}
return "success";
}
}</code>Result
All steps complete successfully.
Spring Full-Stack Practical Cases
Full-stack Java development with Vue 2/3 front-end suite; hands-on examples and source code analysis for Spring, Spring Boot 2/3, and Spring Cloud.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.