Asynchronous Messaging with Retry Using the JMS Adapter

In many cases integration scenarios have to be decoupled asynchronously between sender and receiver
message processing to ensure that a retry is done from the integration system rather than the sender
system. This can be achieved, for example, by using JMS queues to temporarily store the messages in
the Cloud Integration system if the receiver system cannot be reached, and retry them from there.
Follow the steps described below to activate the message broker and to setup the scenario. Use the
monitoring tools described to check the processing and view the stored messages in the queue monitor.

Setup Scenario With Asynchronous Decoupling

To configure the decoupling of inbound and outbound message processing you need to configure two
processes:, one process to receive the inbound message and store it in the JMS queue and a second
process to trigger the message from the JMS queue to the receiver backend. The blog describes the
configuration using two separate integration flows.

Configure the Integration Flow Receiving the Message

The first integration flow will be configured to receive the message via any inbound adapter. In this
sample setup we use the HTTP adapter.

Configure the JMS Receiver Channel

Create the integration flow with the inbound channel required by your scenario, and use the JMS
adapter as the outbound adapter. You only have to configure the Queue Name for the JMS queue that
will be used to store the messages. It is Important to set an Expiration Period that suits your scenario.
The default value is 90 days', which means that after 90 days any messages, that were not delivered
within this period will be deleted from the queue.
With the 10-December 2017 release, you will have the option to configure, if exchange properties are
forwarded to the JMS queue. Before, the properties were always forwarded along with the message,
which caused severe issues in some scenarios. The default value of the option will be that properties are
not forwarded, because this could lead to runtime issues if the properties are too large. In detail the size
restrictions are described in the Blog 'JMS Resource and Size Limits'.

This option is available in adapter version 1.1.

Configure Transaction Handling

You need to configure the correct transaction manager in the integration process for transactional end-
to-end processing. Our process does not contain any JDBC resources (data store, variables, aggregator),
and only one direct JMS Receiver without splitter or sequential multicast; so we don't need a specific
transaction handler. Select the integration process and switch to the Processing tab. In Transaction
Handling drop down select Not Required.
More details about the different transaction handling options and existing limitations are described in
the blog 'How to Configure Transaction Handling in Integration Flow'.

Deploy the Integration Flow

Now you can deploy the integration flow. During deployment, the specified queue will be created
automatically in the message broker. You can check that the queue has been created in the Queue
Monitor available in the operations view in the Manage Storages section. The monitor is described in
more detail in the Monitoring section below.

If a message broker has not yet been provisioned, the deployment will end with an error. In Manage
Integration Content the error details for your integration flow tell you that no messaging host is
available.
Queue Name : Name of queue in CPI
When we use POSTMAN to run this integration flow We will receive message in Message Queues
  JMS Sender adaptor: in this the message will be picked from JMS queue
 JMS receiver adaptor : you need JMS receiver adaptor to put the message in the queue once
the message is available in the queue you can fetch the same using Sender JMS adaptor

Error handling in JMS queues

This is integration flow which using JMS adapter for sender

Configure the Integration Flow Doing the Retry

To consume the messages from the JMS queue, you configure a second integration flow with a JMS
sender channel and the outbound adapter needed for your scenario. In this sample configuration we use
the HTTP adapter.

Configure the JMS Sender Channel

Create the integration flow with the outbound channel required by your scenario, and use the JMS
adapter as the inbound adapter. You only have to configure the Queue Name for the JMS queue that
you want to poll the messages from. Use the same queue name used in the receiving integration flow.

Number of Concurrent Processes is set to 1 per default and should only be increased if more parallel
processing is needed. Because the JMS resources are limited, you need to keep the number of parallel
processes as low as possible. Especially if large messages are processed in the scenario, increasing the
parallelism may lead to out of memory problems.

In the JMS sender channel you also have to configure the retry handling. Set the Retry Interval as
required for your scenario (, the default value is 1 minute). We also recommend using Exponential
Backoff, which means that the retry interval is doubled after each unsuccessful retry. By using this
setting you can avoid calling the receiver backend every minute if it is unavailable for a longer time
period. The Maximum Retry Interval defines the maximum time between two retries if exponential
backoff is used. The recommendation is to either keep the 60 minutes or even increase it if this is
acceptable by the scenario.

Note, that the JMS sender does not guarantee the order in which the messages are consumed. If several
processes are configured to consume messages and/or multiple worker nodes process messages, the
messages are processed in parallel. Furthermore, in case the message is in retry because of an error it is
taken out from processing until the retry interval is reached.

Optional: Explicit Retry Configuration

In the JMS sender channel, you can configure the time intervals between retries, but you cannot
configure that processing will end after a specific number of retries. If required, you can configure this in
an exception subprocess that calls a local process for retry handling using the header SAPJMSRetries set
by the JMS sender adapter.

Note that the header SAPJMSRetriesmay not indicate the exact number of retries in clusters with
multiple worker nodes. It may happen that a retry is triggered on multiple worker nodes shortly one
after the other before the message is moved to the error queue and the counter in the header is
increased.
To configure this, add a Local Integration Process to the integration flow. In this process you configure
the specific retry handling. In this sample, we check the number of retries executed and after 6
unsuccessful retries, we end the processing and trigger a mail to alert someone to the problem.

In the local integration process configure a Router after the start event, add an Error End Event and
configure the router as shown in the picture. The header SAPJMSRetries set by the JMS adapter is used
to decide if the message processing continues or if it ends and a mail is triggered. If message processing
is to continue, an error is raised by the process so that the message stays in the JMS queue and goes
into retry status.

We set the transaction handling to From Calling Process in the local process, and as there are neither
JMS nor JDBC transacted resources configured in the local process, we take over the Not
Required setting from main process.
Furthermore, you need to add an Exception Subprocess in the main process, and add a Local Process
Call, where you select the configured local process, within the exception subprocess. Add a Receiver for
the mail to be sent. Connect the Message End Event of the exception subprocess to the receiver using a
Mail adapter receiver channel. In this channel, you configure the mail address the mail is to be sent to
and the details to be sent.

With this configuration the message will be removed from the JMS queue after 5 unsuccessful retries, so
that no further retries are executed. A mail is triggered to notify the appropriate person to take manual
action.

If you use the explicit retry configuration, you are free to configure retries as required for your scenario.
For example, you can notify someone after 3 retries but not stop the processing until 10 retries have
been made, to allow enough time for manual actions. Or, you can send the message after 3 retries via
an alternative receiver channel.

Deploy the Integration Flow

You can now deploy the integration flow. This completes the scenario configuration in SAP Cloud
Integration and you can start processing messages. Test the configuration by sending a test message via
the configured inbound adapter and check that it is received by the receiver backend.

JMS Resource and Size Limits

The connected JMS messaging instance, that is used in the asynchronous messaging scenario has limited
resources. The Enterprise License sets a limit on the queues, storage and connections that you can use
in the messaging instance. The limits are described in more detail in the blog 'JMS Resource and Size
Limits'.

Performance Optimization Options

If a higher throughput is required for your scenario and you identify JMS as the bottleneck, the following
options for performance increase are possible:

 Do not use intermediate JMS queues in one end-to-end scenario. The recommendation is to
use one queue after the message is received from the sender system and one directly before the
message is sent to the receiver system. Depending on the scenario, one JMS queue could be
sufficient as well.

 Increase the number of parallel processes in the JMS sender channel to make sure more
messages are consumed in parallel. But keep an eye on the JMS resources, see blog 'JMS
Resource and Size Limits'.
  Switch off Encryption in the JMS receiver channel. For scenarios without sensitive data this
option could be used to increase performance.

 Do not use the Dead-Letter Queue. The option to switch off the dead-letter queue you should
only use if you can make sure that no out-of-memory errors can occur, for example because of
large messags. If you have only small messages and don't see a risk for out-of-memory situations
in your scenario switching off the Dead-Letter Queue could increase the performance. More
details can be found in the blog How to configure Dead Letter Handling in JMS Adapter.

 Use the correct transaction handling. It is for example important to use a JMS transaction
handler if JMS sender and JMS receiver channels are used in the same integration process. More
details can be found in the blog 'How to configure Transaction Handling in Integration Flow'.

Monitoring

To check the processing in detail, you can use the monitoring tools provided by Cloud Integration.

Integration Flow Monitoring

In the operations view, in the section Manage Integration Content, you can find the deployed
integration flows and their status. Check that the flows have the status Started.
Poll Status (available with the 16-Feb-2020 update)

In the Status Details area you may find the status and the details about the current consumption:

Message Monitoring

Message processing can be checked in detail in the section Monitor Message Processing. Select the
message you sent and check the processing status. There should be two message processing logs, one
for the inbound messaging that saves the message to the JMS queue, and one for processing the
message from the queue and sending it to the receiver system.

Those two messages are correlated via a Correlation ID. With the 8th-July-2018 update of Cloud
Integration, this Correlation ID is shown in the Properties of a message processing log. You can also
search for all messages having the same Correlation ID using the ID search field at the top of the
monitor:
In the event of a failure when sending to the receiver system the outbound message will go into
status Retry. The message will then be retried as configured in the JMS sender channel. The detailed
error information and the retries are available under the link Message Processing Log.

Queue Monitor

If there are problems during processing and your message does not arrive at the receiver backend, the
message status is Retry and the messages can be checked in the Queue Monitor in the section Manage
Storages in the Operations view. This monitor shows all queues existing in your tenant, with their stored
messages. Select your queue to check the messages contained in the queue.
You can see the number of retries that have already been executed and the time of the next retry. You
can also trigger an immediate retry of the message or delete specific messages if you no longer want
them to be processed.

If a message broker has not yet been provisioned, the Queue Monitor tile is not shown.

Queue Deletion

If you do no longer need a specific queue, (for example if you un-deploy the integration flow), you have
to delete the queue manually in the Queue Monitor. Unused queues, that do not contain messages will
be deleted automatically after the May-13-2018 update. Queues containing messages cannot be deleted
automatically because this could cause inconsistencies, as the messages in the queue may still have to
be processed.
JMS feature is mainly required to Retry/Resend a message from SAP CPI.

We have a requirement where a government agency would send some information to CPI. CPI will have
to always deliver this message to 2 different systems. In case of any failure there is no option of asking
the sender to resend the message.

Since there is no default retry mechanism available in CPI, what is the recommended approach/solution
available to ensure message is delivered to all the target systems?

Conclusion
From this, it is clear that whenever you send any data via JMS adapter, in case the receiver is not
available/not reachable, the JMS will store the message in its queue and retries to deliver until either the
receiver is available or it times out.

If you check the Default Connection settings of the JMS adapter, the Adapter will retry sending the
message every 1 min. If you have checked the Exponential Backoff button, The retry interval doubles for
each successive retry. ( Eg., 1min,2 min,4 min, 8 min, and so on) till the value specified in the Maximum
Retry interval is reached ( 60 mins in this case ). After that, the message is deleted from the queue.

Checking the Dead letter Queue will let your message be in a locked state if there are 2
successive memeory errors/ worker node failure in case of large messages to avoid unnecessary traffic
congestion. The lock can be removed from the Monitoring window under Manage Locks Tile.
Monitor Tips :

 Only when the iflow fails the info log is enabled Monitor-> Monitor Message Processing
 Only in failed status Info log is enable and if the iflow is in completed status the info log is not
enabled

If you click on Info log link it will display where the iflow has been failed

Info log is not enables as the iflow is completed status :

----------------------------------------------------------------------------------------------------------------------------------

When you change the log level to trace and click on the track link below is the screenshot how it looks
like:

Click on trace link to see the below Message processed flow