Spring Book – Chapter 21 – Spring Batch

Scaling and Parallel Processing

In any modern application, executing a job in stipulated time with good capability of scaling are very important factors in overall functioning of an application. Spring Batch provides various options by which to address scaling and parallel processing in an application as summarized below:

  • Single process, making a step multi-threaded (Multi-Threaded Step)
  • Single process, by doing parallel steps (Parallel Step)
  • Multiple process, making a step remote chunking (Remote Chunking)
  • Single or multiple process, making a step partitioned (Partitioning)

We will now see these options in some detail (not in exhaustive manner) in the following sections.

Multi-Threaded Step

Pictorially a multi-threaded step can be as shown in Figure 23-24 below.

Figure 23-24. Multi-Threaded Step

Figure 23-24. Multi-Threaded Step

Just by adding a TaskExecutor to the “tasklet” in the Spring configuration file, you can make a step multi-threaded as shown in Listing 23-30 below.

Listing 23-30. Making a step multi-threaded using TaskExecutor in the Spring configuration file

Parallel Step

Pictorially a multi-threaded step can be as shown in Figure 23-25 below.

 Figure 23-25. Parallel Step

Figure 23-25. Parallel Step

Listing 23-31 below shows how we can achieve parallel step by mere configuration in the Spring configuration file.

Listing 23-31. Parallel Step configuration of job in Spring configuration file

Remote Chunking

In Remote Chunking the Step processing is split across multiple processes, communicating with each other through some middleware. This is one of the approaches to parallelization which allows spreading processing across multiple JVM’s. Figure 23-26 below shows Chunk Remoting in action.

 Figure 23-26. Remote Chunking

Figure 23-26. Remote Chunking

Reader reads chunks and then sends for remote processing. It requires a durable middleware similar to JMS for communication between the master and the slaves. In the remote slave ItemProcessor is configured as a message driven POJO which processes the message and then sends the updated item back to the master for writing.

The Master component is a single process, and the Slaves are multiple remote processes. Spring Batch has a sister project Spring Batch Admin, which provides implementations of various patterns using Spring Integration. These are implemented in a module called Spring Batch Integration.

Partitioning

This is another approach of parallelization in which you use a master/slave configuration, but in this approach you don’t need a durable method of communication and the master serves only as a controller for a collection of slave steps as shown in Figure 23-27 below.

Figure 23-27. Partitioning

Figure 23-27. Partitioning

Listing 23-32 below shows how we can achieve this approach of parallelization by mere step configuration in the Spring configuration file.

Listing 23-32. Parallelization using partitioning of step in Spring configuration file

Benefits of Using Spring Batch

The benefits of Spring Batch can be summarized as follows:

  • Spring Batch is completely based on Spring Framework; you will automatically get all the benefits of Spring Framework like dependency injection and bean management based on simple POJO’s.
  • Developers well-versed with Spring Framework will become productive in Spring Batch in very less time and become productive.
  • The technical difficulties surrounding batch based application is solved, making developers concentrate on the actual business requirement rather than creating the batch framework itself.
  • Leveraging the additional functionality of Spring Integration using Spring Batch Integration project, you can further increase the scalability of more distributed processes.

Summary

Spring Batch is a new implementation of some very old ideas. It brings one of the oldest programming models in software industry into the mainstream through the use of the Spring Framework. Being built over the Spring Framework, it enables a batch project to enjoy the same clean architecture and lightweight programming model, supported by industry-proven patterns, operations, templates, callbacks and other idioms. Spring Batch is an exciting initiative that offers the potential of standardizing Batch Architectures.

In this Chapter, we started off with various concepts in a typical batch project and delved into these into some detail. We haven’t covered each concept in complete detail as this is out-of-scope of this book. This chapter intended to introduce you to Spring Batch as a whole with the various functionalities which you could do using this. It has covered various uses of this framework but didn’t go into detail.

After reading this Chapter, you would have a clear idea of Spring Batch as a whole and I am sure you will be able to convert typical business use cases in your application to appropriate use case in Spring Batch.

Page Visitors: 25939

The following two tabs change content below.
Tomcy John

Tomcy John

Blogger & Author at javacodebook
He is an Enterprise Java Specialist holding a degree in Engineering (B-Tech) with over 10 years of experience in several industries. He's currently working as Principal Architect at Emirates Group IT since 2005. Prior to this he has worked with Oracle Corporation and Ernst & Young. His main specialization is on various web technologies and acts as chief mentor and Architect to facilitate incorporating Spring as Corporate Standard in the organization.
Tomcy John

Latest posts by Tomcy John (see all)

2 thoughts on “Spring Book – Chapter 21 – Spring Batch

  1. Thanks for sharing the information but what’s the difference between CursorItemReader’s setFetchSize() and PagingItemReader’s setPageSize()? isn’t it the same?

  2. Please provide an example for StoredProcedureItem Reader – which returns cursor and how to process cursor in processor – I am new to Spring Batch as part of my work I need to invoke Oracle Stored proc which takes one param as input and returns result set which I need to process in Spring Batch Processor.

    Thanks
    Laxmi

Leave a Reply

Your email address will not be published. Required fields are marked *