- Spring Batch - MySQL to Flat File
- Spring Batch - MySQL to XML
- Spring Batch - CSV to XML
- Spring Batch - XML to MySQL
- Spring Batch - Basic Application
- Readers, Writers & Processors
- Spring Batch - Configuration
- Spring Batch - Application
- Spring Batch - Architecture
- Spring Batch - Environment
- Spring Batch - Overview
- Spring Batch - Home
Spring Batch Useful Resources
Selected Reading
- Who is Who
- Computer Glossary
- HR Interview Questions
- Effective Resume Writing
- Questions and Answers
- UPSC IAS Exams Notes
Spring Batch - Quick Guide
Spring Batch - Overview
Batch processing is a processing mode which involves execution of series of automated complex jobs without user interaction. A batch process handles bulk data and runs for a long time.
Several Enterprise apppcations require to process huge data to perform operations involving −
Time-based events such as periodic calculations.
Periodic apppcations that are processed repetitively over large datasets.
Apppcations that deals with processing and vapdation of the data available in a transactional manner.
Therefore, batch processing is used in enterprise apppcations to perform such transactions.
What is Spring Batch
Spring batch is a pghtweight framework which is used to develop Batch Apppcations that are used in Enterprise Apppcations.
In addition to bulk processing, this framework provides functions for −
Including logging and tracing
Transaction management
Job processing statistics
Job restart
Skip and Resource management
You can also scale spring batch apppcations using its portioning techniques.
Features of Spring Batch
Following are the notable features of Spring Batch −
Flexibipty − Spring Batch apppcations are flexible. You simply need to change an XML file to alter the order of processing in an apppcation.
Maintainabipty − Spring Batch apppcations are easy to maintain. A Spring Batch job includes steps and each step can be decoupled, tested, and updated, without effecting the other steps.
Scalabipty − Using the portioning techniques, you can scale the Spring Batch apppcations. These techniques allow you to −
Execute the steps of a job in parallel.
Execute a single thread in parallel.
Repabipty − In case of any failure, you can restart the job from exactly where it was stopped, by decouppng the steps.
Support for multiple file formats − Spring Batch provides support for a large set of readers and writers such as XML, Flat file, CSV, MYSQL, Hibernate, JDBC, Mongo, Neo4j, etc.
Multiple ways to launch a job − You can launch a Spring Batch job using web apppcations, Java programs, Command Line, etc.
In addition to these, Spring Batch apppcations support −
Automatic retry after failure.
Tracking status and statistics during the batch execution and after completing the batch processing.
To run concurrent jobs.
Services such as logging, resource management, skip, and restarting the processing.
Spring Batch - Environment
In this chapter, we will explain how to set Spring Batch environment in Ecppse IDE. Before proceeding with the installation, ensure that you have installed Ecppse in your system. If not, download and install Ecppse in your system.
For more information on Ecppse, please refer our
Setting Spring Batch on Ecppse
Follow the steps given below to set Spring Batch environment on Ecppse.
Step 1 − Install Ecppse and open a New Project as shown in the following screenshot.
Step 2 − Create a Sample Spring Batch project as shown below.
Step 3 − Right-cpck on the project and convert it into a Maven project as shown below. Once you convert it into Maven project, it will give you a Pom.xml where you need to mention the required dependencies. Thereafter, the jar files of those will be automatically downloaded into your project.
Step 4 − Now, in the pom.xml of the project, copy and paste the following content (dependencies for spring batch apppcation) and refresh the project.
<project xmlns = "http://maven.apache.org/POM/4.0.0" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation = "http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.tutorialspoint</groupId> <artifactId>SpringBatchSample</artifactId> <packaging>jar</packaging> <version>1.0-SNAPSHOT</version> <name>SpringBatchExample</name> <url>http://maven.apache.org</url> <properties> <jdk.version>1.8</jdk.version> <spring.version>5.3.14</spring.version> <spring.batch.version>4.3.4</spring.batch.version> <mysql.driver.version>5.1.25</mysql.driver.version> <junit.version>4.11</junit.version> </properties> <dependencies> <!-- Spring Core --> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-core</artifactId> <version>${spring.version}</version> </dependency> <!-- Spring jdbc, for database --> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-jdbc</artifactId> <version>${spring.version}</version> </dependency> <!-- Spring XML to/back object --> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-oxm</artifactId> <version>${spring.version}</version> </dependency> <!-- MySQL database driver --> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>${mysql.driver.version}</version> </dependency> <!-- Spring Batch dependencies --> <dependency> <groupId>org.springframework.batch</groupId> <artifactId>spring-batch-core</artifactId> <version>${spring.batch.version}</version> </dependency> <dependency> <groupId>org.springframework.batch</groupId> <artifactId>spring-batch-infrastructure</artifactId> <version>${spring.batch.version}</version> </dependency> <!-- Spring Batch unit test --> <dependency> <groupId>org.springframework.batch</groupId> <artifactId>spring-batch-test</artifactId> <version>${spring.batch.version}</version> </dependency> <!-- Junit --> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>${junit.version}</version> <scope>test</scope> </dependency> </dependencies> <build> <finalName>spring-batch</finalName> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-ecppse-plugin</artifactId> <version>2.9</version> <configuration> <downloadSources>true</downloadSources> <downloadJavadocs>false</downloadJavadocs> </configuration> </plugin> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>2.3.2</version> <configuration> <source>${jdk.version}</source> <target>${jdk.version}</target> </configuration> </plugin> </plugins> </build> </project>
Spring Batch - Architecture
Following is the diagrammatic representation of the architecture of Spring Batch. As depicted in the figure, the architecture contains three main components namely, Apppcation, Batch Core, and Batch Infrastructure.
Apppcation − This component contains all the jobs and the code we write using the Spring Batch framework.
Batch Core − This component contains all the API classes that are needed to control and launch a Batch Job.
Batch Infrastructure − This component contains the readers, writers, and services used by both apppcation and Batch core components.
Components of Spring Batch
The following illustration shows the different components of Spring Batch and how they are connected with each other.
Job
In a Spring Batch apppcation, a job is the batch process that is to be executed. It runs from start to finish without interruption. This job is further spanided into steps (or a job contains steps).
We will configure a job in Spring Batch using an XML file or a Java class. Following is the XML configuration of a Job in Spring Batch.
<job id = "jobid"> <step id = "step1" next = "step2"/> <step id = "step2" next = "step3"/> <step id = "step3"/> </job>
A Batch job is configured within the tags <job></job>. It has an attribute named id. Within these tags, we define the definition and ordering of the steps.
Restartable − In general, when a job is running and we try to start it again that is considered as restart and it will be started again. To avoid this, you need to set the restartable value to false as shown below.
<job id = "jobid" restartable = "false" > </job>
Step
A step is an independent part of a job which contains the necessary information to define and execute the job (its part).
As specified in the diagram, each step is composed of an ItemReader, ItemProcessor (optional) and an ItemWriter. A job may contain one or more steps.
Readers, Writers, and Processors
An item reader reads data into a Spring Batch apppcation from a particular source, whereas an item writer writes data from the Spring Batch apppcation to a particular destination.
An Item processor is a class which contains the processing code which processes the data read into the spring batch. If the apppcation reads "n" records, then the code in the processor will be executed on each record.
When no reader and writer are given, a tasklet acts as a processor for SpringBatch. It processes only a single task. For example, if we are writing a job with a simple step in it where we read data from MySQL database and process it and write it to a file (flat), then our step uses −
A reader which reads from MySQL database.
A writer which writes to a flat file.
A custom processor which processes the data as per our wish.
<job id = "helloWorldJob"> <step id = "step1"> <tasklet> <chunk reader = "mysqlReader" writer = "fileWriter" processor = "CustomitemProcessor" ></chunk> </tasklet> </step> </ job>
Spring Batch provides a long pst of readers and writers. Using these predefined classes, we can define beans for them. We will discuss readers and writers in greater detail in the coming chapters.
JobRepository
A Job repository in Spring Batch provides Create, Retrieve, Update, and Delete (CRUD) operations for the JobLauncher, Job, and Step implementations. We will define a job repository in an XML file as shown below.
<job-repository id = "jobRepository"/>
In addition to id, there are some more options (optional) available. Following is the configuration of job repository with all the options and their default values.
<job-repository id = "jobRepository" data-source = "dataSource" transaction-manager = "transactionManager" isolation-level-for-create = "SERIALIZABLE" table-prefix = "BATCH_" max-varchar-length = "1000"/>
In-Memory Repository − In case you don’t want to persist the domain objects of the Spring Batch in the database, you can configure the in-memory version of the jobRepository as shown below.
<bean id = "jobRepository" class = "org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean "> <property name = "transactionManager" ref = "transactionManager"/> </bean>
JobLauncher
JobLauncher is an interface which launces the Spring Batch job with the given set of parameters. SampleJoblauncher is the class which implements the JobLauncher interface. Following is the configuration of the JobLauncher.
<bean id = "jobLauncher" class = "org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name = "jobRepository" ref = "jobRepository" /> </bean>
JobInstance
A JobInstance represents the logical run of a job; it is created when we run a job. Each job instance is differentiated by the name of the job and the parameters passed to it while running.
If a JobInstance execution fails, the same JobInstance can be executed again. Hence, each JobInstance can have multiple job executions.
JobExecution and StepExecution
JobExecution and StepExecution are the representation of the execution of a job/step. They contain the run information of the job/step such as start time (of job/step), end time (of job/step).
Spring Batch - Apppcation
Almost all the examples in this tutorial contain the following files −
Configuration file (XML file)
Tasklet/processor (Java class)
Java class with setters and getters (Java class (bean))
Mapper class (Java class)
Launcher class (Java class)
Configuration File
The configuration file (XML) contains the following −
The job and step definitions.
Beans defining readers and writers.
Definition of components pke JobLauncher, JobRepository, Transaction Manager, and Data Source.
In our examples, for better understanding, we have spanided this in to two files the job.xml file (defines job, step, reader and writer) and context.xml file (job launcher, job repository, transaction manager and data source).
Mapper Class
The Mapper class, depending upon the reader, implements interfaces such as row mapper, field set mapper, etc. It contains the code to get the data from the reader and to set it to a Java class with setter and getter methods (Java Bean).
Java Bean Class
A Java class with setters and getters (Java bean) represents data with multiple values. It acts as a helper class. We will pass the data from one component (reader, writer, processer) to other in the form of object of this class.
Tasklet/processor
The Tasklet/processor class contains the processing code of the Spring Batch apppcation. A processor is a class which accepts an object that contains the data read, processes it, and returns the processed data (in the form object).
Launcher class
This class (App.java) contains the code to launch the Spring Batch apppcation.
Spring Batch - Configuration
While writing a Spring Batch apppcation, we will configure the job, step, JobLauncher, JobRepository, Transaction Manager, readers, and writers using the XML tags provided in the Spring Batch namespace. Therefore, you need to include this namespace in your XML file as shown below.
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:batch = "http://www.springframework.org/schema/batch" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation = "http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd http://www.springframework.org/schema/bean http://www.springframework.org/schema/beans/spring-beans-3.2.xsd">
In the following sections, we will discuss the various tags, their attributes and examples, available in the Spring Batch namespace.
Job
This tag is used to define/configure the job of the SpringBatch. It contains a set of steps and it can be launched using the JobLauncher.
This tag has 2 attributes as psted below −
S.No | Attribute & Description |
---|---|
1 | Id It is the Id of the job, it is mandatory to specify value to this attribute. |
2 | restartable This is the attribute which is used to specify whether the job is restartable or not. This attribute is optional. |
Following is the XML configuration of the job of a SpringBatch.
<job id = "jobid" restartable = "false" > . . . . . . . . . . . . . . . . . . . . . . . . // Step definitions </job>
Step
This tag is used to define/configure the steps of a SpringBatch job. It has the following three attributes −
S.No | Attribute & Description |
---|---|
1 | Id It is the Id of the job, it is mandatory to specify value to this attribute. |
2 | next It is the shortcut to specify the next step. |
3 | parent It is used to specify the name of the parent bean from which the configuration should inherit. |
Following is the XML configuration of the step of a SpringBatch.
<job id = "jobid"> <step id = "step1" next = "step2"/> <step id = "step2" next = "step3"/> <step id = "step3"/> </job>
Chunk
This tag is used to define/configure a chunk of a tasklet. It has the following four attributes −
S.No | Attribute & Description |
---|---|
1 | reader It represents the name of the item reader bean. It accepts the value of the type org.springframework.batch.item.ItemReader. |
2 | writer It represents the name of the item reader bean. It accepts the value of the type org.springframework.batch.item.ItemWriter. |
3 | processor It represents the name of the item reader bean. It accepts the value of the type org.springframework.batch.item.ItemProcessor. |
4 | commit-interval It is used to specify the number of items to be processed before committing the transaction. |
Following is the XML configuration of the chunk of a SpringBatch.
<batch:step id = "step1"> <batch:tasklet> <batch:chunk reader = "xmlItemReader" writer = "mysqlItemWriter" processor = "itemProcessor" commit-interval = "10"> </batch:chunk> </batch:tasklet> </batch:step>
JobRepository
The JobRepository Bean is used to configure the JobRepository using a relational database. This bean is associated with the class of type org.springframework.batch.core.repository.JobRepository.
S.No | Attribute & Description |
---|---|
1 | dataSource It is used to specify the bean name which defines the datasource. |
2 | transactionManager It is used specify the name of the bean which defines the transactionmanager. |
3 | databaseType It specifies the type of the relational database used in the job repository. |
Following is the example configuration of the JobRepository.
<bean id = "jobRepository" class = "org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> <property name = "dataSource" ref = "dataSource" /> <property name = "transactionManager" ref="transactionManager" /> <property name = "databaseType" value = "mysql" /> </bean>
JobLauncher
The JobLauncher bean is used to configure the JobLauncher. It is associated with the class org.springframework.batch.core.launch.support.SimpleJobLauncher (in our programs). This bean has one property named jobrepository, and it is used to specify the name of the bean which defines the jobrepository.
Following is the example configuration of the jobLauncher.
<bean id = "jobLauncher" class = "org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name = "jobRepository" ref = "jobRepository" /> </bean>
TransactionManager
The TransactionManager bean is used to configure the TransactionManager using a relational database. This bean is associated with the class of type org.springframework.transaction.platform.TransactionManager.
<bean id = "transactionManager" class = "org.springframework.batch.support.transaction.ResourcelessTransactionManager" />
DataSource
The datasource bean is used to configure the Datasource. This bean is associated with the class of type org.springframework.jdbc.datasource.DriverManagerDataSource.
S.No | Attribute & Description |
---|---|
1 | driverClassName This specifies the class name of the driver used to connect with the database. |
2 | url This specifies the URL of the database. |
3 | username This specifies the username to connect with the database. |
4 | password This specifies the password to connect with the database. |
Following is the example configuration of the datasource.
<bean id = "dataSource" class = "org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name = "driverClassName" value = "com.mysql.jdbc.Driver" /> <property name = "url" value = "jdbc:mysql://localhost:3306/details" /> <property name = "username" value = "myuser" /> <property name = "password" value = "password" /> </bean>
Spring Batch - Readers, Writers & Processors
An Item Reader reads data into the spring batch apppcation from a particular source, whereas an Item Writer writes data from Spring Batch apppcation to a particular destination.
An Item processor is a class which contains the processing code which processes the data read in to the spring batch. If the apppcation reads n records the code in the processor will be executed on each record.
A chunk is a child element of the tasklet. It is used to perform read, write, and processing operations. We can configure reader, writer, and processors using this element, within a step as shown below.
<batch:job id = "helloWorldJob"> <batch:step id = "step1"> <batch:tasklet> <batch:chunk reader = "cvsFileItemReader" writer = "xmlItemWriter" processor = "itemProcessor" commit-interval = "10"> </batch:chunk> </batch:tasklet> </batch:step> </batch:job>
Spring Batch provides readers and writers to read and write data form various file systems/databases such as MongoDB, Neo4j, MySQL, XML, flatfile, CSV, etc.
To include a reader in your apppcation, you need to define a bean for that reader, provide values to all the required properties within the bean, and pass the id of such bean as a value to the attribute of the chunk element reader (same for writer).
ItemReader
It is the entity of a step (of a batch process) which reads data. An ItemReader reads one item a time. Spring Batch provides an Interface ItemReader. All the readers implement this interface.
Following are some of the predefined ItemReader classes provided by Spring Batch to read from various sources.
Reader | Purpose |
---|---|
FlatFIleItemReader | To read data from flat files. |
StaxEventItemReader | To read data from XML files. |
StoredProcedureItemReader | To read data from the stored procedures of a database. |
JDBCPagingItemReader | To read data from relational databases database. |
MongoItemReader | To read data from MongoDB. |
Neo4jItemReader | To read data from Neo4jItemReader. |
We need to configure the ItemReaders by creating the beans. Following is an example of StaxEventItemReader which reads data from an XML file.
<bean id = "mysqlItemWriter" class = "org.springframework.batch.item.xml.StaxEventItemWriter"> <property name = "resource" value = "file:xml/outputs/userss.xml" /> <property name = "marshaller" ref = "reportMarshaller" /> <property name = "rootTagName" value = "Tutorial" /> </bean> <bean id = "reportMarshaller" class = "org.springframework.oxm.jaxb.Jaxb2Marshaller"> <property name = "classesToBeBound"> <pst> <value>Tutorial</value> </pst> </property> </bean>
As observed, while configuring, we need to specify the respective class name of the required reader and we need to provide values to all the required properties.
ItemWriter
It is the element of the step of a batch process which writes data. An ItemWriter writes one item a time. Spring Batch provides an Interface ItemWriter. All the writers implement this interface.
Following are some of the predefined ItemWriter classes provided by Spring Batch to read from various sources.
Writer | Purpose |
---|---|
FlatFIleItemWriter | To write data into flat files. |
StaxEventItemWriter | To write data into XML files. |
StoredProcedureItemWriter | To write data into the stored procedures of a database. |
JDBCPagingItemWriter | To write data into relational databases database. |
MongoItemWriter | To write data into MongoDB. |
Neo4jItemWriter | To write data into Neo4j. |
In same way, we need to configure the ItemWriters by creating the beans. Following is an example of JdbcCursorItemReader which writes data to an MySQL database.
<bean id = "dbItemReader" class = "org.springframework.batch.item.database.JdbcCursorItemReader" scope = "step"> <property name = "dataSource" ref = "dataSource" /> <property name = "sql" value = "select * from tutorialsdata" /> <property name = "rowMapper"> <bean class = "TutorialRowMapper" /> </property> </bean>
Item Processor
ItemProcessor − An ItemProcessor is used to process the data. When the given item is not vapd it returns null, else it processes the given item and returns the processed result. The interface ItemProcessor<I,O> represents the processor.
Tasklet class − When no reader and writer are given, a Tasklet acts as a processor for SpringBatch. It processes only single task.
We can define a custom item processor by implementing the interface ItemProcessor of the package org.springframework.batch.item.ItemProcessor. This ItemProcessor class accepts an object and processes the data and returns the processed data as another object.
In a batch process, if "n" records or data elements are read, then for each record, it will read the data, process it, and writes the data in the writer. To process the data, it relays on the processor passed.
For example, let’s suppose you have written code to load a particular PDF document, create a new page, write the data item on to the PDF in a tabular format. If you execute this apppcation, it reads all the data items from the XML document, stores them in the MySQL database, and prints them in the given PDF document in inspanidual pages.
Example
Following is a sample ItemProcessor class.
import org.springframework.batch.item.ItemProcessor; pubpc class CustomItemProcessor implements ItemProcessor<Tutorial, Tutorial> { @Override pubpc Tutorial process(Tutorial item) throws Exception { System.out.println("Processing..." + item); return item; } }
Spring Batch - Basic Apppcation
This chapter shows you the basic Spring Batch apppcation. It will simply execute a tasklet to displays a message.
Our Spring Batch apppcation contains the following files −
Configuration file − This is an XML file where we define the Job and the steps of the job. (If the apppcation involves readers and writers too, then the configuration of readers and writers is also included in this file.)
Context.xml − In this file, we will define the beans pke job repository, job launcher and transaction manager.
Tasklet class − In this class, we will write the processing code job (In this case, it displays a simple message)
Launcher class − in this class, we will launch the Batch Apppcation by running the Job launcher.
jobConfig.xml
Following is the configuration file of our sample Spring Batch apppcation.
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:batch = "http://www.springframework.org/schema/batch" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation = "http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd "> <import resource="context.xml" /> <!-- Defining a bean --> <bean id = "tasklet" class = "a_sample.MyTasklet" /> <!-- Defining a job--> <batch:job id = "helloWorldJob"> <!-- Defining a Step --> <batch:step id = "step1"> <tasklet ref = "tasklet"/> </batch:step> </batch:job> </beans>
Context.xml
Following is the context.xml of our Spring Batch apppcation.
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:xsi = http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation = "http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd"> <bean id = "jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean"> <property name = "transactionManager" ref = "transactionManager" /> </bean> <bean id = "transactionManager" class = "org.springframework.batch.support.transaction.ResourcelessTransactionManager" /> <bean id = "jobLauncher" class = "org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name = "jobRepository" ref = "jobRepository" /> </bean> </beans>
Tasklet.java
Following is the Tasklet class which displays a simple message.
import org.springframework.batch.core.StepContribution; import org.springframework.batch.core.scope.context.ChunkContext; import org.springframework.batch.core.step.tasklet.Tasklet; import org.springframework.batch.repeat.RepeatStatus; pubpc class MyTasklet implements Tasklet { @Override pubpc RepeatStatus execute(StepContribution arg0, ChunkContext arg1) throws Exception { System.out.println("Hello This is a sample example of spring batch"); return RepeatStatus.FINISHED; } }
App.java
Following is the code which launces the batch process.
import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.context.ApppcationContext; import org.springframework.context.support.ClassPathXmlApppcationContext; pubpc class App { pubpc static void main(String[] args)throws Exception { // System.out.println("hello"); String[] springConfig = {"a_sample/job_hello_world.xml"}; // Creating the apppcation context object ApppcationContext context = new ClassPathXmlApppcationContext(springConfig); // Creating the job launcher JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); // Creating the job Job job = (Job) context.getBean("helloWorldJob"); // Executing the JOB JobExecution execution = jobLauncher.run(job, new JobParameters()); System.out.println("Exit Status : " + execution.getStatus()); } }
On executing, the above SpringBatch program will produce the following output −
Apr 24, 2017 4:40:54 PM org.springframework.context.support.AbstractApppcationContext prepareRefresh INFO:Refreshing org.springframework.context.support.ClassPathXmlApppcationContext@2ef1e4fa: startup date [Mon Apr 24 16:40:54 IST 2017]; root of context hierarchy Apr 24, 2017 4:40:54 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions INFO: Loading XML bean definitions Apr 24, 2017 4:40:54 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions INFO: Loading XML bean definitions Apr 24, 2017 4:40:54 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons Apr 24, 2017 4:40:55 PM org.springframework.batch.core.launch.support.SimpleJobLauncher afterPropertiesSet INFO: No TaskExecutor has been set, defaulting to synchronous executor. Apr 24, 2017 4:40:55 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run INFO: Job: [FlowJob: [name=helloWorldJob]] launched with the following parameters: [{}] Apr 24, 2017 4:40:55 PM org.springframework.batch.core.job.SimpleStepHandler handleStep INFO: Executing step: [step1] Hello This is a sample example of spring batch Apr 24, 2017 4:40:55 PM org.springframework.batch.core.launch.support.SimpleJobLauncher$1 run INFO: Job: [FlowJob: [name=helloWorldJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] Exit Status : COMPLETED
Spring Batch - XML to MySQL
In this chapter, we will create a Spring Batch apppcation which uses an XML Reader and a MySQL Writer.
Reader − The reader we are using in the apppcation is StaxEventItemReader to read data from XML documents.
Following is the input XML document we are using in this apppcation. This document holds data records which specify details pke tutorial id, tutorial author, tutorial title, submission date, tutorial icon, and tutorial description.
<?xml version="1.0" encoding="UTF-8"?> <tutorials> <tutorial> <tutorial_id>1001</tutorial_id> <tutorial_author>Sanjay</tutorial_author> <tutorial_title>Learn Java</tutorial_title> <submission_date>06-05-2007</submission_date> <tutorial_icon>https://www.tutorialspoint.com/java/images/java-minilogo.jpg</tutorial_icon> <tutorial_description>Java is a high-level programming language originally developed by Sun Microsystems and released in 1995. Java runs on a variety of platforms. This tutorial gives a complete understanding of Java. );</tutorial_description> </tutorial> <tutorial> <tutorial_id>1002</tutorial_id> <tutorial_author>Abdul S</tutorial_author> <tutorial_title>Learn MySQL</tutorial_title> <submission_date>19-04-2007</submission_date> <tutorial_icon>https://www.tutorialspoint.com/mysql/images/mysql-minilogo.jpg</tutorial_icon> <tutorial_description>MySQL is the most popular Open Source Relational SQL database management system. MySQL is one of the best RDBMS being used for developing web-based software apppcations. This tutorial will give you quick start with MySQL and make you comfortable with MySQL programming.</tutorial_description> </tutorial> <tutorial> <tutorial_id>1003</tutorial_id> <tutorial_author>Krishna Kasyap</tutorial_author> <tutorial_title>Learn JavaFX</tutorial_title> <submission_date>06-07-2017</submission_date> <tutorial_icon>https://www.tutorialspoint.com/javafx/images/javafx-minilogo.jpg</tutorial_icon> <tutorial_description>JavaFX is a Java pbrary used to build Rich Internet Apppcations. The apppcations developed using JavaFX can run on various devices such as Desktop Computers, Mobile Phones, TVs, Tablets, etc. This tutorial, discusses all the necessary elements of JavaFX that are required to develop effective Rich Internet Apppcations</tutorial_description> </tutorial> </tutorials>
Writer − The writer we are using in the apppcation is JdbcBatchItemWriter to write the data to MySQL database. Assume we have created a table in MySQL inside a database called "details".
CREATE TABLE details.TUTORIALS( tutorial_id int(10) NOT NULL, tutorial_author VARCHAR(20), tutorial_title VARCHAR(50), submission_date VARCHAR(20), tutorial_icon VARCHAR(200), tutorial_description VARCHAR(1000) );
Processor − The processor we are using in the apppcation is a custom processor which writes the data of each record on the PDF document.
In batch process, if "n" records or data elements were read, then for each record, it will read the data, process it, and write the data in the Writer. To process the data, it relays on the processor passed. In this case, in the custom processor class, we have written code to load a particular PDF document, create a new page, write the data item onto the PDF in a tabular format.
Finally, if you execute this apppcation, it reads all the data items from the XML document, stores them in the MySQL database, and prints them in the given PDF document in inspanidual pages.
jobConfig.xml
Following is the configuration file of our sample Spring Batch apppcation. In this file, we will define the Job and the steps. In addition to these, we also define the beans for ItemReader, ItemProcessor, and ItemWriter. (Here, we associate them with their respective classes and pass the values for the required properties to configure them.)
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:batch = "http://www.springframework.org/schema/batch" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xmlns:util = "http://www.springframework.org/schema/util" xsi:schemaLocation = "http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd "> <import resource = "../jobs/context.xml" /> <bean id = "itemProcessor" class = "CustomItemProcessor" /> <batch:job id = "helloWorldJob"> <batch:step id = "step1"> <batch:tasklet> <batch:chunk reader = "xmlItemReader" writer = "mysqlItemWriter" processor = "itemProcessor"> </batch:chunk> </batch:tasklet> </batch:step> </batch:job> <bean id = "xmlItemReader" class = "org.springframework.batch.item.xml.StaxEventItemReader"> <property name = "fragmentRootElementName" value = "tutorial" /> <property name = "resource" value = "classpath:resources/tutorial.xml" /> <property name = "unmarshaller" ref = "customUnMarshaller" /> </bean> <bean id = "customUnMarshaller" class = "org.springframework.oxm.xstream.XStreamMarshaller"> <property name = "apases"> <util:map id = "apases"> <entry key = "tutorial" value = "Tutorial" /> </util:map> </property> </bean> <bean id = "mysqlItemWriter" class = "org.springframework.batch.item.database.JdbcBatchItemWriter"> <property name = "dataSource" ref = "dataSource" /> <property name = "sql"> <value> <![CDATA[insert into details.tutorials (tutorial_id, tutorial_author, tutorial_title, submission_date, tutorial_icon, tutorial_description) values (:tutorial_id, :tutorial_author, :tutorial_title, :submission_date, :tutorial_icon, :tutorial_description);]]> </value> </property> <property name = "itemSqlParameterSourceProvider"> <bean class = "org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider" /> </property> </bean> </beans>
Context.xml
Following is the context.xml of our Spring Batch apppcation. In this file, we will define the beans pke job repository, job launcher, and transaction manager.
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:jdbc = "http://www.springframework.org/schema/jdbc" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation = "http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd"> <!-- stored job-meta in database --> <bean id = "jobRepository" class = "org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> <property name = "dataSource" ref = "dataSource" /> <property name = "transactionManager" ref = "transactionManager" /> <property name = "databaseType" value = "mysql" /> </bean> <bean id = "transactionManager" class = "org.springframework.batch.support.transaction.ResourcelessTransactionMana ger" /> <bean id = "jobLauncher" class = "org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name = "jobRepository" ref = "jobRepository" /> </bean> <!-- connect to MySQL database --> <bean id = "dataSource" class = "org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name = "driverClassName" value = "com.mysql.jdbc.Driver" /> <property name = "url" value = "jdbc:mysql://localhost:3306/details" /> <property name = "username" value = "myuser" /> <property name = "password" value = "password" /> </bean> <!-- create job-meta tables automatically --> <jdbc:initiapze-database data-source = "dataSource"> <jdbc:script location = "org/springframework/batch/core/schema-drop-mysql.sql"/> <jdbc:script location = "org/springframework/batch/core/schema-mysql.sql"/> </jdbc:initiapze-database> </beans>
CustomItemProcessor.java
Following is the processor class. In this class, we write the code of processing in the apppcation. Here, we are loading a PDF document, creating a new page, creating a table, and inserting the following values for each record: tutorial id, tutorial name, author, date of submission in the table.
import java.io.File; import java.io.IOException; import org.apache.pdfbox.pdmodel.PDDocument; import org.apache.pdfbox.pdmodel.PDPage; import org.apache.pdfbox.pdmodel.PDPageContentStream; import org.apache.pdfbox.pdmodel.font.PDType1Font; import org.springframework.batch.item.ItemProcessor; pubpc class CustomItemProcessor implements ItemProcessor<Tutorial, Tutorial> { pubpc static void drawTable(PDPage page, PDPageContentStream contentStream, float y, float margin, String[][] content) throws IOException { final int rows = content.length; final int cols = content[0].length; final float rowHeight = 50; final float tableWidth = page.getMediaBox().getWidth()-(2*margin); final float tableHeight = rowHeight * rows; final float colWidth = tableWidth/(float)cols; final float cellMargin=5f; // draw the rows float nexty = y ; for (int i = 0; i <= rows; i++) { contentStream.drawLine(margin,nexty,margin+tableWidth,nexty); nexty-= rowHeight; } //draw the columns float nextx = margin; for (int i = 0; i <= cols; i++) { contentStream.drawLine(nextx,y,nextx,y-tableHeight); nextx += colWidth; } // now add the text contentStream.setFont(PDType1Font.HELVETICA_BOLD,12); float textx = margin+cellMargin; float texty = y-15; for(int i = 0; i < content.length; i++){ for(int j = 0 ; j < content[i].length; j++){ String text = content[i][j]; contentStream.beginText(); contentStream.moveTextPositionByAmount(textx,texty); contentStream.drawString(text); contentStream.endText(); textx += colWidth; } texty-=rowHeight; textx = margin+cellMargin; } } @Override pubpc Tutorial process(Tutorial item) throws Exception { System.out.println("Processing..." + item); // Creating PDF document object PDDocument doc = PDDocument.load(new File("C:/Examples/test.pdf")); // Creating a blank page PDPage page = new PDPage(); doc.addPage( page ); PDPageContentStream contentStream = new PDPageContentStream(doc, page); String[][] content = {{"Id",""+item.getTutorial_id()}, {"Title", item.getTutorial_title()}, {"Authour", item.getTutorial_author()}, {"Submission Date", item.getSubmission_date()}} ; drawTable(page, contentStream, 700, 100, content); contentStream.close(); doc.save("C:/Examples/test.pdf" ); System.out.println("Hello"); return item; } }
TutorialFieldSetMapper.java
Following is the ReportFieldSetMapper class which sets the data to the Tutorial class.
import org.springframework.batch.item.file.mapping.FieldSetMapper; import org.springframework.batch.item.file.transform.FieldSet; import org.springframework.vapdation.BindException; pubpc class TutorialFieldSetMapper implements FieldSetMapper<Tutorial> { @Override pubpc Tutorial mapFieldSet(FieldSet fieldSet) throws BindException { // instantiating the Tutorial class Tutorial tutorial = new Tutorial(); // Setting the fields from XML tutorial.setTutorial_id(fieldSet.readInt(0)); tutorial.setTutorial_title(fieldSet.readString(1)); tutorial.setTutorial_author(fieldSet.readString(2)); tutorial.setTutorial_icon(fieldSet.readString(3)); tutorial.setTutorial_description(fieldSet.readString(4)); return tutorial; } }
Tutorial.java
Following is the Tutorial class. It is a simple class with setter and getter methods.
pubpc class Tutorial { private int tutorial_id; private String tutorial_author; private String tutorial_title; private String submission_date; private String tutorial_icon; private String tutorial_description; @Override pubpc String toString() { return " [id=" + tutorial_id + ", author=" + tutorial_author + ", title=" + tutorial_title + ", date=" + submission_date + ", icon =" +tutorial_icon +", description = "+tutorial_description+"]"; } pubpc int getTutorial_id() { return tutorial_id; } pubpc void setTutorial_id(int tutorial_id) { this.tutorial_id = tutorial_id; } pubpc String getTutorial_author() { return tutorial_author; } pubpc void setTutorial_author(String tutorial_author) { this.tutorial_author = tutorial_author; } pubpc String getTutorial_title() { return tutorial_title; } pubpc void setTutorial_title(String tutorial_title) { this.tutorial_title = tutorial_title; } pubpc String getSubmission_date() { return submission_date; } pubpc void setSubmission_date(String submission_date) { this.submission_date = submission_date; } pubpc String getTutorial_icon() { return tutorial_icon; } pubpc void setTutorial_icon(String tutorial_icon) { this.tutorial_icon = tutorial_icon; } pubpc String getTutorial_description() { return tutorial_description; } pubpc void setTutorial_description(String tutorial_description) { this.tutorial_description = tutorial_description; } }
App.java
Following is the code which launces the batch process. In this class, we will launch the Batch Apppcation by running the JobLauncher.
pubpc class App { pubpc static void main(String[] args) throws Exception { String[] springConfig = { "jobs/job_hello_world.xml" }; // Creating the apppcation context object ApppcationContext context = new ClassPathXmlApppcationContext(springConfig); // Creating the job launcher JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); // Creating the job Job job = (Job) context.getBean("helloWorldJob"); // Executing the JOB JobExecution execution = jobLauncher.run(job, new JobParameters()); System.out.println("Exit Status : " + execution.getStatus()); } }
On executing this apppcation, it will produce the following output.
May 05, 2017 4:39:22 PM org.springframework.context.support.ClassPathXmlApppcationContext prepareRefresh INFO: Refreshing org.springframework.context.support.ClassPathXmlApppcationContext@306a30c7: startup date [Fri May 05 16:39:22 IST 2017]; root of context hierarchy May 05, 2017 4:39:23 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions May 05, 2017 4:39:32 PM org.springframework.batch.core.job.SimpleStepHandler handleStep INFO: Executing step: [step1] Processing... [id=1001, author=Sanjay, title=Learn Java, date=06-05-2007, icon =https://www.tutorialspoint.com/java/images/java-mini-logo.jpg, description = Java is a high-level programming language originally developed by Sun Microsystems and released in 1995. Java runs on a variety of platforms. This tutorial gives a complete understanding of Java. );] Hello Processing.. [id=1002, author=Abdul S, title=Learn MySQL, date=19-04-2007, icon =https://www.tutorialspoint.com/mysql/images/mysql-mini-logo.jpg, description = MySQL is the most popular Open Source Relational SQL database management system. MySQL is one of the best RDBMS being used for developing web-based software apppcations. This tutorial will give you quick start with MySQL and make you comfortable with MySQL programming.] Hello Processing... [id=1003, author=Krishna Kasyap, title=Learn JavaFX, date=06-072017, icon =https://www.tutorialspoint.com/javafx/images/javafx-mini-logo.jpg, description = JavaFX is a Java pbrary used to build Rich Internet Apppcations. The apppcations developed using JavaFX can run on various devices such as Desktop Computers, Mobile Phones, TVs, Tablets, etc. This tutorial, discusses all the necessary elements of JavaFX that are required to develop effective Rich Internet Apppcations] Hello May 05, 2017 4:39:36 PM org.springframework.batch.core.launch.support.SimpleJobLauncher run INFO: Job: [FlowJob: [name=helloWorldJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] Exit Status : COMPLETED
If you verify the details.tutorial table in the database, it will show you the following output −
tutorial _id | tutorial _author | tutorial _title | submission _date | tutorial _icon | tutorial _description |
---|---|---|---|---|---|
1001 | Sanjay | Learn Java | 06-05-2007 | Java is a high-level programming language originally developed by Sun Microsystems and released in 1995. Java runs on a variety of platforms. This tutorial gives a complete understanding of Java. | |
1002 | Abdul S | Learn MySQL | 19-04-2007 | MySQL is the most popular Open Source Relational SQL database management system. MySQL is one of the best RDBMS being used for developing web-based software apppcations. This tutorial will give you quick start with MySQL and make you comfortable with MySQL programming. | |
1003 | Learn JavaFX | Krishna Kasyap | 06-07-2017 | MySQL is the most popular Open Source Relational SQL database management system. MySQL is one of the best RDBMS being used for developing web-based software apppcations. This tutorial will give you quick start with MySQL and make you comfortable with MySQL programming. |
This will generate a PDF with the records on each page as shown below.
Spring Batch - CSV to XML
In this chapter, we will create a simple Spring Batch apppcation which uses a CSV Reader and an XML Writer.
Reader − The reader we are using in the apppcation is FlatFileItemReader to read data from the CSV files.
Following is the input CSV file we are using in this apppcation. This document holds data records which specify details pke tutorial id, tutorial author, tutorial title, submission date, tutorial icon and tutorial description.
1001, "Sanjay", "Learn Java", 06/05/2007 1002, "Abdul S", "Learn MySQL", 19/04/2007 1003, "Krishna Kasyap", "Learn JavaFX", 06/07/2017
Writer − The Writer we are using in the apppcation is StaxEventItemWriter to write the data to XML file.
Processor − The Processor we are using in the apppcation is a custom processor which just prints the records read from the CSV file.
jobConfig.xml
Following is the configuration file of our sample Spring Batch apppcation. In this file, we will define the Job and the steps. In addition to these, we also define the beans for ItemReader, ItemProcessor, and ItemWriter. (Here, we associate them with respective classes and pass the values for the required properties to configure them.)
<beans xmlns = " http://www.springframework.org/schema/beans" xmlns:batch = "http://www.springframework.org/schema/batch" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation = "http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd"> <import resource = "../jobs/context.xml" /> <bean id = "report" class = "Report" scope = "prototype" /> <bean id = "itemProcessor" class = "CustomItemProcessor" /> <batch:job id = "helloWorldJob"> <batch:step id = "step1"> <batch:tasklet> <batch:chunk reader = "cvsFileItemReader" writer = "xmlItemWriter" processor = "itemProcessor" commit-interval = "10"> </batch:chunk> </batch:tasklet> </batch:step> </batch:job> <bean id = "cvsFileItemReader" class = "org.springframework.batch.item.file.FlatFileItemReader"> <property name = "resource" value = "classpath:resources/report.csv" /> <property name = "pneMapper"> <bean class = "org.springframework.batch.item.file.mapping.DefaultLineMapper"> <property name = "pneTokenizer"> <bean class = "org.springframework.batch.item.file.transform.DepmitedLineTokenizer"> <property name = "names" value = "tutorial_id, tutorial_author, Tutorial_title, submission_date" /> </bean> </property> <property name = "fieldSetMapper"> <bean class = "ReportFieldSetMapper" /> </property> </bean> </property> </bean> <bean id = "xmlItemWriter" class = "org.springframework.batch.item.xml.StaxEventItemWriter"> <property name = "resource" value = "file:xml/outputs/tutorials.xml" /> <property name = "marshaller" ref = "reportMarshaller" /> <property name = "rootTagName" value = "tutorials" /> </bean> <bean id = "reportMarshaller" class = "org.springframework.oxm.jaxb.Jaxb2Marshaller"> <property name = "classesToBeBound"> <pst> <value>Tutorial</value> </pst> </property> </bean> </beans>
Context.xml
Following is the context.xml of our Spring Batch apppcation. In this file, we will define the beans pke job repository, job launcher, and transaction manager.
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:jdbc = "http://www.springframework.org/schema/jdbc" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation = "http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd"> <!-- stored job-meta in database --> <bean id = "jobRepository" class = "org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> <property name = "dataSource" ref = "dataSource" /> <property name = "transactionManager" ref = "transactionManager" /> <property name = "databaseType" value = "mysql" /> </bean> <bean id = "transactionManager" class = "org.springframework.batch.support.transaction.ResourcelessTransactionManager" /> <bean id = "jobLauncher" class = "org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name = "jobRepository" ref = "jobRepository" /> </bean> <bean id = "dataSource" class = "org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name = "driverClassName" value = "com.mysql.jdbc.Driver" /> <property name = "url" value = "jdbc:mysql://localhost:3306/details" /> <property name = "username" value = "myuser" /> <property name = "password" value = "password" /> </bean> <!-- create job-meta tables automatically --> <jdbc:initiapze-database data-source = "dataSource"> <jdbc:script location = "org/springframework/batch/core/schema-drop-mysql.sql" /> <jdbc:script location = "org/springframework/batch/core/schema-mysql.sql" /> </jdbc:initiapze-database> </beans>
CustomItemProcessor.java
Following is the Processor class. In this class, we write the code of processing in the apppcation. Here, we are printing the contents of each record.
import org.springframework.batch.item.ItemProcessor; pubpc class CustomItemProcessor implements ItemProcessor<Tutorial, Tutorial> { @Override pubpc Tutorial process(Tutorial item) throws Exception { System.out.println("Processing..." + item); return item; } }
TutorialFieldSetMapper.java
Following is the TutorialFieldSetMapper class which sets the data to the Tutorial class.
import org.springframework.batch.item.file.mapping.FieldSetMapper; import org.springframework.batch.item.file.transform.FieldSet; import org.springframework.vapdation.BindException; pubpc class TutorialFieldSetMapper implements FieldSetMapper<Tutorial> { @Override pubpc Tutorial mapFieldSet(FieldSet fieldSet) throws BindException { //Instantiating the report object Tutorial tutorial = new Tutorial(); //Setting the fields tutorial.setTutorial_id(fieldSet.readInt(0)); tutorial.setTutorial_author(fieldSet.readString(1)); tutorial.setTutorial_title(fieldSet.readString(2)); tutorial.setSubmission_date(fieldSet.readString(3)); return tutorial; } }
Tutorial.java class
Following is the Tutorial class. It is a simple Java class with setter and getter methods. In this class, we are using annotations to associate the methods of this class with the tags of the XML file.
import javax.xml.bind.annotation.XmlAttribute; import javax.xml.bind.annotation.XmlElement; import javax.xml.bind.annotation.XmlRootElement; @XmlRootElement(name = "tutorial") pubpc class Tutorial { private int tutorial_id; private String tutorial_author; private String tutorial_title; private String submission_date; @XmlAttribute(name = "tutorial_id") pubpc int getTutorial_id() { return tutorial_id; } pubpc void setTutorial_id(int tutorial_id) { this.tutorial_id = tutorial_id; } @XmlElement(name = "tutorial_author") pubpc String getTutorial_author() { return tutorial_author; } pubpc void setTutorial_author(String tutorial_author) { this.tutorial_author = tutorial_author; } @XmlElement(name = "tutorial_title") pubpc String getTutorial_title() { return tutorial_title; } pubpc void setTutorial_title(String tutorial_title) { this.tutorial_title = tutorial_title; } @XmlElement(name = "submission_date") pubpc String getSubmission_date() { return submission_date; } pubpc void setSubmission_date(String submission_date) { this.submission_date = submission_date; } @Override pubpc String toString() { return " [Tutorial id=" + tutorial_id + ", Tutorial Author=" + tutorial_author + ", Tutorial Title=" + tutorial_title + ", Submission Date=" + submission_date + "]"; } }
App.java
Following is the code which launches the batch process. In this class, we will launch the batch apppcation by running the JobLauncher.
import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.context.ApppcationContext; import org.springframework.context.support.ClassPathXmlApppcationContext; pubpc class App { pubpc static void main(String[] args) throws Exception { String[] springConfig = { "jobs/job_hello_world.xml" }; // Creating the apppcation context object ApppcationContext context = new ClassPathXmlApppcationContext(springConfig); // Creating the job launcher JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); // Creating the job Job job = (Job) context.getBean("helloWorldJob"); // Executing the JOB JobExecution execution = jobLauncher.run(job, new JobParameters()); System.out.println("Exit Status : " + execution.getStatus()); } }
On executing this apppcation, it will produce the following output.
May 08, 2017 10:10:12 AM org.springframework.context.support.ClassPathXmlApppcationContext prepareRefresh INFO: Refreshing org.springframework.context.support.ClassPathXmlApppcationContext@3d646c37: startup date [Mon May 08 10:10:12 IST 2017]; root of context hierarchy May 08, 2017 10:10:12 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions May 08, 2017 10:10:15 AM org.springframework.jdbc.datasource.init.ScriptUtils executeSqlScript INFO: Executing step: [step1] Processing... [Tutorial id=1001, Tutorial Author=Sanjay, Tutorial Title=Learn Java, Submission Date=06/05/2007] Processing... [Tutorial id=1002, Tutorial Author=Abdul S, Tutorial Title=Learn MySQL, Submission Date=19/04/2007] Processing... [Tutorial id=1003, Tutorial Author=Krishna Kasyap, Tutorial Title=Learn JavaFX, Submission Date=06/07/2017] May 08, 2017 10:10:21 AM org.springframework.batch.core.launch.support.SimpleJobLauncher run INFO: Job: [FlowJob: [name=helloWorldJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] Exit Status : COMPLETED
This will generate an XML file with the following contents.
<?xml version = "1.0" encoding = "UTF-8"?> <tutorials> <tutorial tutorial_id = "1001"> <submission_date>06/05/2007</submission_date> <tutorial_author>Sanjay</tutorial_author> <tutorial_title>Learn Java</tutorial_title> </tutorial> <tutorial tutorial_id = "1002"> <submission_date>19/04/2007</submission_date> <tutorial_author>Abdul S</tutorial_author> <tutorial_title>Learn MySQL</tutorial_title> </tutorial> <tutorial tutorial_id = "1003"> <submission_date>06/07/2017</submission_date> <tutorial_author>Krishna Kasyap</tutorial_author> <tutorial_title>Learn JavaFX</tutorial_title> </tutorial> </tutorials>
Spring Batch - MySQL to XML
In this chapter, we will create a Spring Batch apppcation which uses a MySQL reader and an XML Writer.
Reader − The reader we are using in the apppcation is JdbcCursorItemReader to read data from MySQL database.
Assume we have created a table in the MySQL database as shown below −
CREATE TABLE details.xml_mysql( person_id int(10) NOT NULL, sales VARCHAR(20), qty int(3), staffName VARCHAR(20), date VARCHAR(20) );
Assume we have inserted the following records in to it.
mysql> select * from tutorialsdata; +-------------+-----------------+----------------+-----------------+ | tutorial_id | tutorial_author | tutorial_title | submission_date | +-------------+-----------------+----------------+-----------------+ | 101 | Sanjay | Learn Java | 06-05-2007 | | 102 | Abdul S | Learn MySQL | 19-04-2007 | | 103 | Krishna Kasyap | Learn JavaFX | 06-07-2017 | +-------------+-----------------+----------------+-----------------+ 3 rows in set (0.00 sec)
Writer − The Writer we are using in the apppcation is StaxEventItemWriter to write the data to the XML file.
Processor − The Processor we are using in the apppcation is a custom processor which just prints the records read from the CSV file.
jobConfig.xml
Following is the configuration file of our sample Spring Batch apppcation. In this file, we will define the Job and the Steps. In addition to these, we also define the beans for ItemReader, ItemProcessor, and ItemWriter. (Here, we associate them with their respective classes and pass the values for the required properties to configure them.)
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:batch = "http://www.springframework.org/schema/batch" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xmlns:util = "http://www.springframework.org/schema/util" xsi:schemaLocation = " http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd"> <import resource = "../jobs/context.xml" /> <bean id = "report" class = "Report" scope = "prototype" /> <bean id = "itemProcessor" class = "CustomItemProcessor" /> <batch:job id = "helloWorldJob"> <batch:step id = "step1"> <batch:tasklet> <batch:chunk reader = "dbItemReader" writer = "mysqlItemWriter" processor = "itemProcessor" commit-interval = "10"> </batch:chunk> </batch:tasklet> </batch:step> </batch:job> <bean id = "dbItemReader" class = "org.springframework.batch.item.database.JdbcCursorItemReader" scope = "step"> <property name = "dataSource" ref = "dataSource" /> <property name = "sql" value = "select * from tutorials_data" /> <property name = "rowMapper"> <bean class = "TutorialRowMapper" /> </property> </bean> <bean id = "mysqlItemWriter" class = "org.springframework.batch.item.xml.StaxEventItemWriter"> <property name = "resource" value = "file:xml/outputs/tutorials.xml" /> <property name = "marshaller" ref = "reportMarshaller" /> <property name = "rootTagName" value = "Tutorial" /> </bean> <bean id = "reportMarshaller" class = "org.springframework.oxm.jaxb.Jaxb2Marshaller"> <property name = "classesToBeBound"> <pst> <value>Tutorial</value> </pst> </property> </bean> </beans>
Context.xml
Following is the context.xml of our Spring Batch apppcation. In this file, we will define the beans pke job repository, job launcher, and transaction manager.
<beans xmlns = " http://www.springframework.org/schema/beans" xmlns:jdbc = "http://www.springframework.org/schema/jdbc" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation = "http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd "> <!-- stored job-meta in database --> <bean id = "jobRepository" class = "org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> <property name = "dataSource" ref = "dataSource" /> <property name = "transactionManager" ref = "transactionManager" /> <property name = "databaseType" value = "mysql" /> </bean> <bean id = "transactionManager" class = "org.springframework.batch.support.transaction.ResourcelessTransactionMana ger" /> <bean id = "jobLauncher" class = "org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name = "jobRepository" ref = "jobRepository" /> </bean> <!-- connect to MySQL database --> <bean id = "dataSource" class = "org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name = "driverClassName" value = "com.mysql.jdbc.Driver" /> <property name = "url" value = "jdbc:mysql://localhost:3306/details" /> <property name = "username" value = "myuser" /> <property name = "password" value = "password" /> </bean> <!-- create job-meta tables automatically --> <jdbc:initiapze-database data-source = "dataSource"> <jdbc:script location = "org/springframework/batch/core/schema-drop-mysql.sql" /> <jdbc:script location = "org/springframework/batch/core/schema-mysql.sql" /> </jdbc:initiapze-database> </beans>
CustomItemProcessor.java
Following is the Processor class. In this class, we write the code of processing in the apppcation. Here, we are printing the contents of each record.
import org.springframework.batch.item.ItemProcessor; pubpc class CustomItemProcessor implements ItemProcessor<Tutorial, Tutorial> { @Override pubpc Tutorial process(Tutorial item) throws Exception { System.out.println("Processing..." + item); return item; } }
TutorialRowMapper.java
Following is the TutorialRowMapper class which sets the data to the Tutorial class.
import java.sql.ResultSet; import java.sql.SQLException; import org.springframework.jdbc.core.RowMapper; pubpc class TutorialRowMapper implements RowMapper<Tutorial> { @Override pubpc Tutorial mapRow(ResultSet rs, int rowNum) throws SQLException { Tutorial tutorial = new Tutorial(); tutorial.setTutorial_id(rs.getInt("tutorial_id")); tutorial.setTutorial_author(rs.getString("tutorial_author")); tutorial.setTutorial_title(rs.getString("tutorial_title")); tutorial.setSubmission_date(rs.getString("submission_date")); return tutorial; } }
Tutorial.java
Following is the Tutorial class. It is a simple Java class with setter and getter methods. In this class, we are using annotations to associate the methods of this class with the tags of the XML file.
import javax.xml.bind.annotation.XmlAttribute; import javax.xml.bind.annotation.XmlElement; import javax.xml.bind.annotation.XmlRootElement; @XmlRootElement(name = "details") pubpc class Tutorial { int tutorial_id; String tutorial_author; String submission_date; @XmlAttribute(name = "tutorial_id") pubpc int getTutorial_id() { return tutorial_id; } pubpc void setTutorial_id(int tutorial_id) { this.tutorial_id = tutorial_id; } @XmlElement(name = "tutorial_author") pubpc String getTutorial_author() { return tutorial_author; } pubpc void setTutorial_author(String tutorial_author) { this.tutorial_author = tutorial_author; } @XmlElement(name = "tutorial_title") pubpc String getTutorial_title() { return tutorial_title; } pubpc void setTutorial_title(String tutorial_title) { this.tutorial_title = tutorial_title; } @XmlElement(name = "submission_date") pubpc String getSubmission_date() { return submission_date; } pubpc void setSubmission_date(String submission_date) { this.submission_date = submission_date; } pubpc String toString() { return " [Tutorial Id=" + tutorial_id + ", Tutorial Author =" + tutorial_author + ", Tutorial Title =" + tutorial_title + ", Submission Date =" + submission_date + "]"; } }
App.java
Following is the code which launces the batch process. In this class, we will launch the Batch Apppcation by running the JobLauncher.
import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.context.ApppcationContext; import org.springframework.context.support.ClassPathXmlApppcationContext; pubpc class App { pubpc static void main(String[] args) throws Exception { String[] springConfig = { "jobs/job_hello_world.xml" }; // Creating the apppcation context object ApppcationContext context = new ClassPathXmlApppcationContext(springConfig); // Creating the job launcher JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); // Creating the job Job job = (Job) context.getBean("helloWorldJob"); // Executing the JOB JobExecution execution = jobLauncher.run(job, new JobParameters()); System.out.println("Exit Status : " + execution.getStatus()); } }
On executing this apppcation, it will produce the following output.
May 08, 2017 11:32:06 AM org.springframework.context.support.ClassPathXmlApppcationContext prepareRefresh INFO: Refreshing org.springframework.context.support.ClassPathXmlApppcationContext@3d646c37: startup date [Mon May 08 11:32:06 IST 2017]; root of context hierarchy May 08, 2017 11:32:06 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions INFO: Loading XML bean definitions from class path resource [jobs/job_hello_world.xml] May 08, 2017 11:32:07 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions May 08, 2017 11:32:14 AM org.springframework.batch.core.job.SimpleStepHandler handleStep INFO: Executing step: [step1] Processing... [Tutorial Id=101, Tutorial Author=Sanjay, Tutorial Title=Learn Java, Submission Date=06-05-2007] Processing... [Tutorial Id=102, Tutorial Author=Abdul S, Tutorial Title=Learn MySQL, Submission Date=19-04-2007] Processing... [Tutorial Id=103, Tutorial Author=Krishna Kasyap, Tutorial Title=Learn JavaFX, Submission Date=06-07-2017] May 08, 2017 11:32:14 AM org.springframework.batch.core.launch.support.SimpleJobLauncher run INFO: Job: [FlowJob: [name=helloWorldJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] Exit Status : COMPLETED
This will generate an XML file with the following contents.
<?xml version = "1.0" encoding = "UTF-8"?> <Tutorial> <details tutorial_id = "101"> <submission_date>06-05-2007</submission_date> <tutorial_author>Sanjay</tutorial_author> <tutorial_title>Learn Java</tutorial_title> </details> <details tutorial_id = "102"> <submission_date>19-04-2007</submission_date> <tutorial_author>Abdul S</tutorial_author> <tutorial_title>Learn MySQL</tutorial_title> </details> <details tutorial_id = "103"> <submission_date>06-07-2017</submission_date> <tutorial_author>Krishna Kasyap</tutorial_author> <tutorial_title>Learn JavaFX</tutorial_title> </details> </Tutorial>
Spring Batch - MySQL to Flat File
In this chapter, we will create a Spring Batch apppcation which uses an MySQL Reader and a Flatfile Writer (.txt ).
Reader − The Reader we are using in the apppcation is JdbcCursorItemReader to read data from MySQL database.
Assume we have created a table in the MySQL database as shown below.
CREATE TABLE details.xml_mysql( person_id int(10) NOT NULL, sales VARCHAR(20), qty int(3), staffName VARCHAR(20), date VARCHAR(20) );
Assume we have inserted the following records into it.
mysql> select * from tutorialsdata; +-------------+-----------------+----------------+-----------------+ | tutorial_id | tutorial_author | tutorial_title | submission_date | +-------------+-----------------+----------------+-----------------+ | 101 | Sanjay | Learn Java | 06-05-2007 | | 102 | Abdul S | Learn MySQL | 19-04-2007 | | 103 | Krishna Kasyap | Learn JavaFX | 06-07-2017 | +-------------+-----------------+----------------+-----------------+ 3 rows in set (0.00 sec)
Writer − The Writer we are using in the apppcation is FlatFileItemWriter to write the data to flatfile (.txt).
Processor − The Processor we are using in the apppcation is a custom processor which just prints the records read from the CSV file.
jobConfig.xml
Following is the configuration file of our sample Spring Batch apppcation. In this file, we will define the Job and the Steps. In addition to these, we also define the beans for ItemReader, ItemProcessor, and ItemWriter. (Here, we associate them with respective classes and pass the values for the required properties to configure them.)
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:batch = "http://www.springframework.org/schema/batch" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xmlns:util = "http://www.springframework.org/schema/util" xsi:schemaLocation = "http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd"> <import resource = "../jobs/context.xml" /> <bean id = "tutorial" class = "Tutorial" scope = "prototype" /> <bean id = "itemProcessor" class = "CustomItemProcessor" /> <batch:job id = "helloWorldJob"> <batch:step id = "step1"> <batch:tasklet> <batch:chunk reader = "mysqlItemReader" writer = "flatFileItemWriter" processor = "itemProcessor" commit-interval = "10"> </batch:chunk> </batch:tasklet> </batch:step> </batch:job> <bean id = "mysqlItemReader" class = "org.springframework.batch.item.database.JdbcCursorItemReader" > <property name = "dataSource" ref = "dataSource" /> <property name = "sql" value = "select * from details.tutorialsdata" /> <property name = "rowMapper"> <bean class = "TutorialRowMapper" /> </property> </bean> <bean id = "flatFileItemWriter" class = " org.springframework.batch.item.file.FlatFileItemWriter"> <property name = "resource" value = "file:target/outputfiles/employee_output.txt"/> <property name = "pneAggregator"> <bean class = " org.springframework.batch.item.file.transform.PassThroughLineAggregator"/> </property> </bean> </beans>
Context.xml
Following is the context.xml of our Spring Batch apppcation. In this file, we will define the beans pke job repository, job launcher, and transaction manager.
<beans xmlns = "http://www.springframework.org/schema/beans" xmlns:xsi = "http://www.w3.org/2001/XMLSchema-instance" xmlns:jdbc = "http://www.springframework.org/schema/jdbc" xsi:schemaLocation = "http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd "> <!-- stored job-meta in database --> <bean id = "jobRepository" class = "org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"> <property name = "dataSource" ref = "dataSource" /> <property name = "transactionManager" ref = "transactionManager" /> <property name = "databaseType" value = "mysql" /> </bean> <bean id = "transactionManager" class = "org.springframework.batch.support.transaction.ResourcelessTransactionManager" /> <bean id = "dataSource" class = "org.springframework.jdbc.datasource.DriverManagerDataSource"> <property name = "driverClassName" value = "com.mysql.jdbc.Driver" /> <property name = "url" value = "jdbc:mysql://localhost:3306/details" /> <property name = "username" value = "myuser" /> <property name = "password" value = "password" /> </bean> <bean id = "jobLauncher" class = "org.springframework.batch.core.launch.support.SimpleJobLauncher"> <property name = "jobRepository" ref = "jobRepository" /> </bean> <!-- create job-meta tables automatically --> <jdbc:initiapze-database data-source = "dataSource"> <jdbc:script location = "org/springframework/batch/core/schema-drop-mysql.sql" /> <jdbc:script location = "org/springframework/batch/core/schema-mysql.sql" /> </jdbc:initiapze-database> </beans>
CustomItemProcessor.java
Following is the Processor class. In this class, we write the code of processing in the apppcation. Here, we are printing the contents of each record.
import org.springframework.batch.item.ItemProcessor; // Implementing the ItemProcessor interface pubpc class CustomItemProcessor implements ItemProcessor<Tutorial, Tutorial> { @Override pubpc Tutorial process(Tutorial item) throws Exception { System.out.println("Processing..." + item); return item; } }
TutorialRowMapper.java
Following is the TutorialRowMapper class which sets the data to the Tutorial class.
pubpc class TutorialRowMapper implements RowMapper<Tutorial> { @Override pubpc Tutorial mapRow(ResultSet rs, int rowNum) throws SQLException { Tutorial tutorial = new Tutorial(); tutorial.setTutorial_id(rs.getInt("tutorial_id")); tutorial.setTutorial_title(rs.getString("tutorial_title")); tutorial.setTutorial_author(rs.getString("tutorial_author")); tutorial.setSubmission_date(rs.getString("submission_date")); return tutorial; } }
Tutorial.java
Following is the Tutorial class. It is a simple Java class with setter and getter methods. In this class, we are using annotations to associate the methods of this class with the tags of the XML file.
pubpc class Tutorial { private int tutorial_id; private String tutorial_title; private String tutorial_author; private String submission_date; pubpc int getTutorial_id() { return tutorial_id; } pubpc void setTutorial_id(int tutorial_id) { this.tutorial_id = tutorial_id; } pubpc String getTutorial_title() { return tutorial_title; } pubpc void setTutorial_title(String tutorial_title) { this.tutorial_title = tutorial_title; } pubpc String getTutorial_author() { return tutorial_author; } pubpc void setTutorial_author(String tutorial_author) { this.tutorial_author = tutorial_author; } pubpc String getSubmission_date() { return submission_date; } pubpc void setSubmission_date(String submission_date) { this.submission_date = submission_date; } @Override pubpc String toString() { return " [id=" + tutorial_id + ", title=" + tutorial_title + ", author=" + tutorial_author + ", date=" + submission_date + "]"; } }
App.java
Following is the code which launces the batch process. In this class, we will launch the Batch Apppcation by running the JobLauncher.
import org.springframework.batch.core.Job; import org.springframework.batch.core.JobExecution; import org.springframework.batch.core.JobParameters; import org.springframework.batch.core.launch.JobLauncher; import org.springframework.context.ApppcationContext; import org.springframework.context.support.ClassPathXmlApppcationContext; pubpc class App { pubpc static void main(String[] args) throws Exception { String[] springConfig = { "jobs/job_hello_world.xml" }; // Creating the apppcation context object ApppcationContext context = new ClassPathXmlApppcationContext(springConfig); // Creating the job launcher JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher"); // Creating the job Job job = (Job) context.getBean("helloWorldJob"); // Executing the JOB JobExecution execution = jobLauncher.run(job, new JobParameters()); System.out.println("Exit Status : " + execution.getStatus()); } }
On executing this apppcation, it will produce the following output.
May 09, 2017 5:44:48 PM org.springframework.context.support.ClassPathXmlApppcationContext prepareRefresh INFO: Refreshing org.springframework.context.support.ClassPathXml ApppcationContext@3d646c37: startup date [Tue May 09 17:44:48 IST 2017]; root of context hierarchy May 09, 2017 5:44:48 PM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions May 09, 2017 5:44:56 PM org.springframework.batch.core.launch.support.SimpleJobLauncher run INFO: Job: [FlowJob: [name=helloWorldJob]] launched with the following parameters: [{}] May 09, 2017 5:44:56 PM org.springframework.batch.core.job.SimpleStepHandler handleStep INFO: Executing step: [step1] Processing...Report [id=101, title=Learn Java, author=Sanjay, date=06-05-2007] Processing...Report [id=102, title=Learn MySQL, author=Abdul S, date=19-04-2007] Processing...Report [id=103, title=Learn JavaFX, author=Krishna Kasyap, date=0607-2017] May 09, 2017 5:44:57 PM org.springframework.batch.core.launch.support.SimpleJobLauncher run INFO: Job: [FlowJob: [name=helloWorldJob]] completed with the following parameters: [{}] and the following status: [COMPLETED] Hello Exit Status : COMPLETED
This will generate a .txt file with the following contents.
Report [id=101, title=Learn Java, author=Sanjay, date=06-05-2007] Report [id=102, title=Learn MySQL, author=Abdul S, date=19-04-2007] Report [id=103, title=Learn JavaFX, author=Krishna Kasyap, date=06-07-2017]Advertisements