Spring Batch: Chunk Size

In Spring Batch, when configuring a step you can set the chunk size.

This is a very critical and often overlooked settings.

When developing locally, it’s difficult to catch performance problems because typically local data sets are very small. However once deployed the performance problems can be crippling.

The chunk size determines how many records are processed before a commit is triggered.

So if your chunk size is set to 10 and you have 1 million records to process, the application will trigger 100,000 commits. This will be very slow.

If you set the chunk size too high, and a record fails the commit, you will rollback all the other records not committed. Additionally, setting the chunk size too high doesn’t increase the performance but will increase how many records have to be in memory.

That being said, the number of records are memory will be controlled mostly by the reader portion of the step.

It is best to monitor the performance of the batch job against a high volume of data, and try a few different chunk sizes to see what works best.

Example

return stepBuilderFactory
	.reader(reader())
	.writer(writer())
	.chunkSize(1000)
	.build();

Also remember you can inject a variable into the application from your environment so this can set from a configuration file.