Runing batch job in mule

Mule ESB has the ability to process messages in batches.
Within an application, we can initiate a batch job which is basically a block of code that
splits a large messages into individual records, then performs actions upon each record,
then reports on the results and potentially pushes the processed output to other systems
or queues.
This functionality is particularly useful when working with a large set of data for example
Database where a large set of Data is required to retrieve or insert in a bulk.
Source:- MuleSoft

A batch job is a top-level element in Mule which exists outside all Mule flows.
Generally a batch jobs split a large messages into small parts which is called records and
which Mule processes asynchronously in a batch job; similarly just as flows process
messages, batch jobs process records.
A batch job contains one or more batch steps which, in turn, can contain any number of
message processors or Mule component that act upon records as they move through the
batch job.
A batch job executes when triggered by either a batch executor in a Mule flow or
a message source in a batch-accepting input which is when triggered, will create a
new batch job instance.
After all record are processed and passed though all the batch steps, the batch job
instances is ended and result of all the batch jobs are accumulated and summarised in a
report that reflect which batch job as succeed and which job has failed at the time of
processing. Source:- MuleSoft

Structure of batch job in Mule Config :-
<batch:job name="myBatchJobSteps">
<batch:process-records>
<batch:step name="Step1"/>
…………………….
</batch:process-records>
</batch:job>

The batch processing is useful in particular scenario :-
• When a large message fails, for example a database insertion in bulk, we can continue
with the rest.
• When integrating a data sets, that may be small or large, streaming or not, to parallel
process records.
• When it require to handle a large quantities of incoming data from an API into
a legacy system.
• Synchronising data sets between various business applications. Example Netsuite and
Salesforce
Source:- MuleSoft

There are basically 4 parts in Mule batch Job:-
Input phase: In Input phase, we can place a one-way message source and/or message
processors to prepare the data that is actually going to be fed into the job. Here we are
processing synchronously at a message level and this is an optional phase.
Loading phase: This phase is automatic and implicit and we don’t have to do anything
here. In this phase payload from Input phase are split into records and stored in
persistent queues.
Process phase: In this phase, each records is processed separately and independently
and after that it is moved across the steps in an asynchronous and paralleled fashion.
On Complete phase: In this phase you get a result object that tells you how many records
were processed, how many succeeded, which ones failed (and in which step), etc. This is
useful for generating reports and sending out notifications
Source:- MuleSoft

Let’s consider we have a simple Mule flow as following:-
Here , we can see there are 3 phases :- In Input phase it is retrieving the data from databases at
a fixed interval of time
In Process phase it is executing 3 steps .. In step 1, it is inserting data into database.. In step
2 it is logging messages if any and in step 3, it is logging failed messages if any.
In complete phase it log the Number of failed Records , Number of successful Records and
Elapsed Time

So, our Mule config will be as following:-

So, if we run our flow, we will see the following in log :-
Here you can see the Number of failed Records , Number of successful Records and
Elapsed Time in the log

So, here you can see how to use a Mule batch module. It’s pretty simple and effective in
handling larges set of data 

In my next slide I will bring some other techniques in Mule implementation .
Hope you have enjoyed this simpler version.
Keep sharing your knowledge and let our Mule community grow 

Runing batch job in mule

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Runing batch job in mule (20)

More from Son Nguyen (20)

Recently uploaded (20)

Runing batch job in mule