With the release of Spring Boot 3, Spring Batch 5 was released as well, which includes some interesting new features. The probably most important one is the ability to create native executables and images from your Spring Batch 5 applications. These will require less memory, have a faster startup time and will run a lot quicker than on a regular JVM.
In this post, I will first showcase a simple example application build on Spring Boot 3 / Spring Batch 5. In the second part I will implement the changes required to compile it to a native image and compare the runtime performance of the native application versus the regular JVM based application.
The Example App
For the use case of my application, I picked a typical scenario for a Spring Batch application. I have a file containing some data that should be read by the application and written into a relational database. In my use case the file contains customer data that will be written into an H2 in-memory database.
To create the basic app, go to the Spring Initializr and pick the following Spring modules/settings:
- Spring Batch
- Spring Data JPA for the database part of the application
- Project Lombok
- H2 in-memory database
- Maven as the build tool
- Spring Boot Version 3.1.2. or higher
The following picture shows a common architecture for a Spring batch application:
In general, a Spring Batch application is a regular Spring Boot application with some small modification that are specific to the batch module.
The main object you need to create to run a batch is the Job class. The itself Job consist of one or more Steps classes. Each Step usually consist of a Reader to read the data which we want to process, an optional Processor to modify the data (if required) and a Writer to write the modified data to some other system. Furthermore, you can register a subclass of JobExecutionListener in the Job class to get notified when the Job has been finished.
In our case the Reader will read a CSV file to create a bunch of Customer objects. We won’t need a Processor because we can simply write the Customer objects to the database in the writer class.
For many basic read and write operations, Spring batch already has Reader and Writer implementations that you can use, like database writer classes for various databases and readers for files or databases.
For my use case, there is already a reader implementation available. The class is called FlatFileItemReader, which can read CSV files. Here is the code to create one:
@Bean
public FlatFileItemReader<Customer> reader() {
return new FlatFileItemReaderBuilder<Customer>()
.name("customerReader")
.resource(new ClassPathResource("data.csv"))
.delimited()
.names(new String[] { "firstName","lastName",
"birthday","gender","married" })
.fieldSetMapper(new CustomerMapper())
.build();
}You can use the FileItemReaderBuilder class to specify the parameters for your reader. In my example, I specify the following parameters to build the FlatFileItemReader:
- a name for the bean in line(4)
- the file to be read in line (5)
- use the default delimiter in the file, which is the comma in line (6)
- the names of the fields that will be in the CSV file in line (7)
- a custom FieldSetMapper to map the CVS data into a Java class
To further process the data in the application, I will use the Customer POJO to store the data and the CustomerMapper, to create Customer objects from the CSV file. To avoid unnecessary boilerplate code, I use Lombok to create the getters and setters of the Customer class.
@Data
@Entity
public class Customer {
@Id
@GeneratedValue
Integer id;
String firstName;
String lastName;
LocalDate birthday;
String gender;
boolean married;
}public class CustomerMapper implements FieldSetMapper<Customer> {
public Customer mapFieldSet(FieldSet fieldSet) {
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("dd.MM.yyyy");
Customer customer = new Customer();
customer.setFirstName(fieldSet.readString("firstName"));
customer.setLastName(fieldSet.readString("lastName"));
String dateString = fieldSet.readString("birthday");
customer.setBirthday(LocalDate.parse(dateString, formatter));
customer.setGender(fieldSet.readString("gender").equals("male") ? "m" : "f");
customer.setMarried(Boolean.parseBoolean(fieldSet.readString("married")));
return customer;
}
}The Customer POJO specifies the same fields as imported in the CSV file and is also used as a JPA Entity. By this, I can easily write the data into the H2 database later in the batch job.
The CustomerMapper translates the raw data from the CSV file into a Customer object. I changed some of the String properties in the file to different data types in the Customer class to showcase what can be done in a custom mapper class. Notice that the field names were specified in the FileItemReaderBuilder class and used in this mapper to easily access the fields.
Next, we can specify a custom Processor to further change our data. Since I don’t really need anything changed in my example, I will simple print the received data into the console.
@Component
public class CustomerItemProcessor implements ItemProcessor<Customer, Customer> {
private static final Logger LOG = LoggerFactory.getLogger(CustomerItemProcessor.class);
@Override
public Customer process(@NonNull final Customer customer) throws Exception {
LOG.info("Customer received:"
+ customer.getFirstName() + " "
+ customer.getLastName());
return customer;
}The base interface to implement for the processor is the ItemProcessor<A,B>. The two generics specify the input and output classes of the processor. In my case both are Customer objects since I don’t need to change anything in the processor.
The final component to make the example work is a H2 writer implementation that writes our data to a in-memory database. There is a predefined JpaItemWriter class that we can use to write entities to a database. Since we have the H2 dependency in the project pom, Spring Boot will automatically configure everything required for the H2 database and EntityManager in the background without any changes of the base configuration. The following code creates the JPAItemWriter for our Customer objects.
@Autowired
private EntityManagerFactory emf;
@Bean
public JpaItemWriter<Customer> writer() {
JpaItemWriter<Customer> writer = new JpaItemWriter<>();
writer.setEntityManagerFactory(emf);
return writer;
}With all business logic in place, we can now create our Step and Job definitions for the batch job:
@Bean
public Step step1(JobRepository jobRepository,
PlatformTransactionManager transactionManager, JpaItemWriter<Customer> writer,
CustomerItemProcessor processor, FlatFileItemReader<Customer> reader) {
return new StepBuilder("step1", jobRepository)
.<Customer, Customer>chunk(10, transactionManager)
.reader(reader)
.processor(processor)
.writer(writer)
.build();
}
@Bean
public Job importUserJob(JobRepository jobRepository,
JobCompletionNotificationListener listener, Step step1) {
return new JobBuilder("importUserJob", jobRepository)
.incrementer(new RunIdIncrementer())
.flow(step1)
.end()
.build();
}To create the Step, we can use a StepBuilder class that accepts the Reader, Writer and Processor objects that we specified earlier. It also need a JobRepository to track the progress of the job in the background. This will also be saved in our H2 database by default. Finally, the Job is created by a JobBuilder, which takes the Step to execute. The JobBuilder again needs the JobRepository for status tracking and needs a way to assign a number to the Job execution. This is done by the existing class RunIdIncrementer, which simply uses an ascending numbering for the job executions.
With all this in place, we can start our batch job by simply starting the Spring Boot application. Since the spring-boot-starter-batch is on the classpath, the application will look for a bean of type Job and execute it automatically.
This will produce the following output:
The 16384 customers from the data.csv file were successfully inserted into the H2 database in 33 seconds.
Building a Spring Batch Native Image
Next step is to compile the application into a native docker image. Spring Boot 3 offers an easy way to do this by integrating Packeto build packs. The build packs are integrated in the Spring Boot Maven plugin and the native compile build pack can be started by the following command:
mvn -Pnative spring-boot:build-imageYou will need to have a Docker deamon running on your machine to execute the command. This will start a Docker container in which the build is started. Inside the container, the code will be compiled into a native executable and after that a docker image.
There are some limitations to native executables, with the most important one, that no dynamic class loading can be done after the compile. This was a problem for Spring applications for a long time but with Spring 3 all Spring modules are now adapted to work with the native compiler. However, we use a Lombok enhanced class and a text file in our app that we need to load during runtime. These two need to be included into the native executable as well.
We can tell the native compiler to include these two files by a so called compile hint. For Spring apps, you can use an annotation that will tell the compiler to include these files. By this, they can be loaded dynamically when we execute the application. This is the class we need to insert into our application to create the compile hints:
public class NativeCompileHints implements RuntimeHintsRegistrar {
@Override
public void registerHints(RuntimeHints hints, ClassLoader classLoader) {
// Register CSV file
hints.resources().registerPattern("data.csv");
// add lombok generated methods for Customer type
hints.reflection().registerType(Customer.class, MemberCategory.values());
}
}A native compile hint just has to implement the RuntimeHintsRegistrar interface and implement the method registerHints. The class can be in any package of the Spring application. To register a class or file, use the RuntimeHints class, which offers a lot of different methods to give hints to the compiler. For any file use the code in line (5) and for any class that needs to be loaded dynamically use the syntax in line (8). You can also only include partial classes, but this would go into to much detail for this simple examples.
Now we can use the above Maven command to compile the application. This will take quite some time depending on your machine.
After the build finishes you can start the batch by the following command:
docker run --rm -p 8080:8080 docker.io/library/spring-batch-5-native-images:0.0.1-SNAPSHOTThis will start the docker container created in the Maven build and run the native batch executable.
On the same machine the native job only takes around 6 seconds to insert the 16384 customers in the H2 db. This is only 18% of the time it took the regular JVM based batch to complete.
As you can see, with only very little changes to the application, we managed to improve the performance by more than 80%. Looking at this number there is very little reason to not upgrade your batch jobs to native images.
You can find the source code for the example here on my github page.
Schreibe einen Kommentar