Spring-Integration Default Retry Configuration - spring-integration

I have a spring-integration project which does the following
1.) Read messages from a queue
2.) Transform messages
3.) Send transformed messages to an Api
Relevant Config for Step 1
<bean id="cachingConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="MQConnectionFactory" />
<property name="sessionCacheSize" value="10"/>
</bean>
<bean id="requestQueue" class="com.ibm.mq.jms.MQQueue">
<constructor-arg index="0" value="${queuemanager}"/>
<constructor-arg index="1" value="${incoming.queue}"/>
</bean>
<integration:poller id="poller" default="true" fixed-delay="1000"/>
<jms:message-driven-channel-adapter id="jmsIn"
destination="requestQueue"
channel="inputJsonConversionChannel"
connection-factory="cachingConnectionFactory" />
Step 3 is a Service Activator, and in case of a failure (not 201 HTTP status) I am throwing a custom exception.
Relevant config for step 3
<int:service-activator input-channel="ApiChannel" ref="EventApiClient" method="post"/>
<int:service-activator input-channel="errorChannel" ref="PListenerExceptionHandler" method="handleFailure"/>
The behaviour that occurs is that, it keeps trying to connect and gets the same errors over and over again.
I wanted to know if someone could explain to me
how is this default retry being configured/triggered?
how can I redirect the errors to an error channel, because right now errors in Step 1 use the global error channel and the default error handler I created. But errors from the Service Activator are not.
Cheers
Kris

It's retrying because the message-driven channel adapter is transactional by default, which means the exception causes the message to be rolled back onto the queue.
You can add an error-channel to the adapter and any exceptions will be sent in the form of an ErrorMessage which has a payload MessagingException which has properties failedMessage and cause.
If the integration flow downstream of the error channel "consumes" the error, the transaction will be committed and the message removed. If the error flow throws an exception, the transaction will be rolled back, as before.
There is a default errorChannel which just logs the exception by default.
error-channel="errorChannel"
or you can use a custom channel and put your own logic in a subscriber to that channel.

Related

Migration Of Spring Batch from 2.2 to 4.x (XML Configuration Of Partition Jobs

I am migrating Spring Batch Partition Jobs with XML configuration to Spring batch 4.x. I am trying to take advantage to an improvement in the MessageChannelPartitionHandler where it looks for completion of remote steps with both a reply channel and a datasource polling.
When I use this configuration:
<int:channel id="partitioned.jms.requests">
<int:dispatcher task-executor="springbatch.partitioned.jms.taskExecutor"/>
</int:channel>
<int:channel id="partitioned.jms.reply" />
<bean id="partitioned.jms.handler" class="org.springframework.batch.integration.partition.MessageChannelPartitionHandler">
<property name="messagingOperations">
<bean class="org.springframework.integration.core.MessagingTemplate">
<property name="defaultChannel" ref="partitioned.jms.requests"/>
</bean>
</property>
<property name="stepName" value="process.partitioned.step"/>
<property name="gridSize" value="${process.step.partitioned.gridSize}"/>
<property name="dataSource" ref="springbatch.repositoryDataSource" />
<property name="pollInterval" value="${springbatch.partition.verification.interval}"/>
</bean>
The step completes but I see an error in the logs.
no output-channel or replyChannel header available
I looked at the class and see I can add a replyChannel property to the MessageChannelPartitionHandler class. If I add the following:
<property name="replyChannel" ref="claim.acp.process.partitioned.jms.reply"/>
I get error back that a pollable channel is needed.
How do I create a pollable channel (assuming from the same JMS queue)?
You need to show the rest of your configuration.
If you are using DB polling for the results, set the output-channel on the jms outbound gateway to "nullChannel" and the replies received over JMS will be discarded.
Or, use an outbound channel adapter (instead of a gateway) (and an inbound-channel-adapter on the slaves). That avoids the replies being returned altogether.
You have to set pollRepositoryForResults to true.
To answer your specific question
<int:channel id="replies>
<int:queue />
<int:channel>

Spring Integration AMQP - retries continuously despite retry advice

I've used Spring Integration with JMS very successfully before, but we're now using with RabbitMQ / AMQP and having some issues with error handling.
I have an int-amqp:inbound-channel-adapter with with a errorChannel set up to receive any exception, here an ErrorTransformer class inspects the failed Message's cause exception. Then depending on the type of exception either :-
suppresses the exception and transforms into a JSON object that can go to AMQP outbound-channel-adapter as a business reply explaining the failure. Here I want the original message consumed/ACKed.
Or Re-throws the causing exception to let RabbitMQ re-deliver the message, a certain number of times.
I found that re-throwing caused the message to be re-delivered continuously, I then read about StatefulRetryOperationsInterceptorFactoryBean, so added an advice chain to retry 3 times, then I got exception about no message-id, so also added a 'MissingMessageIdAdvice' at the start of the advice chain.
Despite the advice, I still get continuous re-tries for a RuntimeException that is re-thrown from errorChannel's ErrorTransformer. I'm publishing the message via RabbitMQ admin just using the defaults. Not sure if the lack of a message-id is making this not work, if so how do I get a message to have an id ?
I'm confused about the differences between the :-
A) ConditionalRejectingErrorHandler (I've set as the inbound adapter's error-handler) which allows me to provide a customFatalExceptionStrategy to implement isFatal(). Where I believe fatal=true (means DISCARD) and message is consumed and discarded, but how can I still send an outbound failure message ?
B) And the errorChannel I have on the inbound adapter which I'm using to inspect an Exception and transform into a outbound fail response message. Here I guess I could throw AmqpRejectAndDontRequeueException, but then why have the ConditionalRejectingErrorHandler as well ? and will thorwing AmqpRejectAndDontRequeueException work
<int-amqp:inbound-channel-adapter id="amqpInRequestPatternValuation" channel="requestAmqpIn" channel-transacted="true" transaction-manager="transactionManager"
queue-names="requestQueue" error-channel="patternValuationErrorChannel" connection-factory="connectionFactory"
receive-timeout="59000" concurrent-consumers="1"
advice-chain="retryChain" error-handler="customErrorHandler" />
<bean id="customErrorHandler" class="org.springframework.amqp.rabbit.listener.ConditionalRejectingErrorHandler">
<constructor-arg ref="customFatalExceptionStrategy"/>
</bean>
<bean id="customFatalExceptionStrategy" class="abc.common.CustomFatalExceptionStrategy"/>
<!-- ADVICE CHAIN FOR CONTROLLING NUMBER OF RE-TRIES before sending to DLQ (or discarding if no DLQ) without this any re-queued fatal message will retry forever -->
<util:list id="retryChain">
<bean class="org.springframework.amqp.rabbit.retry.MissingMessageIdAdvice">
<constructor-arg>
<bean class="org.springframework.retry.policy.MapRetryContextCache" />
</constructor-arg>
</bean>
<ref bean="retryInterceptor" />
</util:list>
<bean id="retryInterceptor"
class="org.springframework.amqp.rabbit.config.StatefulRetryOperationsInterceptorFactoryBean">
<property name="retryOperations" ref="retryTemplate" />
<property name="messageRecoverer" ref="messageRecoverer"/>
</bean>
<bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="retryPolicy" ref="simpleRetryPolicy" />
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="10000" />
</bean>
</property>
</bean>
<bean id="simpleRetryPolicy" class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="3" />
</bean>
You have to use RejectAndDontRequeueRecoverer to stop re-delivery in the end of retries:
* MessageRecover that causes the listener container to reject
* the message without requeuing. This enables failed messages
* to be sent to a Dead Letter Exchange/Queue, if the broker is
* so configured.
Yes, the messageId is important for that retry use-case.
You can inject custom MessageKeyGenerator strategy to determine a unique key from message if you can't supply it manually during sending.
I never got around to posting the solution, so here it is.
Once I had configured the retry advice chain to the AMQP inbound channel adapter which must include a messageRecoverer of RejectAndDontRequeueRecoverer (which I believe is also the dafault). The important point I was missing was to ensure that when sender's send a message they include a message_id. So if publishing via the RabbitMQ Admin Console I needed to include the predefined message_id property and supply a value.
Using the 'MissingMessageIdAdvice' doesn't help (so I removed) as it will generate a different message_id on each re-deliver on the message leading to the re-try count not incrementing as each delivery was considered different from the last

Logging SOAP message on http gateway spring integration

I am using spring integration (4.0.6) to make SOAP calls from rest service using int-http:outbound-gateway and int-ws:outbound-gateway. Is there a way to log SOAP request message only in case of exception.
<int:gateway id="myGateway" service-interface="myservice">
<int:method name="getData" request-channel="reqChannel" reply-channel="repChannel">
</int-gateway>
my outbound-gateway configured as below
<int-http:outbound-gateway id="og1" request-channel="reqChannel" url="xxx" http-method="POST" reply-channel="repChannel" reply-timeout="10000" message-converters="converter" request-factory="reqFactory">
<bean id="reqFactory"
class="org.springframework.http.client.SimpleClientHttpRequestFactory">
<property name="cTimeout" value="20000"/>
<property name="rTimeout" value="20000"/>
</bean>
Ok, Try this method.
Create your own error Handler which implements the spring Error Handler Interface. In your context file
<bean id="soapErrorHandlingSyncTaskExecutor"
class="org.springframework.integration.util.ErrorHandlingTaskExecutor">
<constructor-arg>
<bean class="org.springframework.core.task.SyncTaskExecutor" />
</constructor-arg>
<constructor-arg>
<bean class="ErrorHandle"
p:errorSender-ref="errorSender"/>
</constructor-arg>
</bean>
<bean id="errorSender" class="org.springframework.integration.core.MessagingTemplate"
p:defaultChannel-ref="errors" />
When ever your WS adapter throws any exception then org.springframework.integration.util.ErrorHandlingTaskExecutor catches this exception now its upto you how you handle the exception in your error handler.
The SOAP faultcode and faultstring is found in the MessagingException's cause (if a SoapFaultClientException).
Here i am sending the error messages to a channel for me to monitor.
Hope this helps.
Would be better to share some config and the point where would you like to get a log/exception.
Both, <int-http:outbound-gateway> and <int-ws:outbound-gateway> have <request-handler-advice-chain> ability, where you could use ExpressionEvaluatingRequestHandlerAdvice and send an ErrorMessage with the request (guilty?) message for any diagnostics.
You should configure the readTimeout on the appropriate ClientHttpRequestFactory, e.g. HttpComponentsClientHttpRequestFactory#setReadTimeout(int timeout).
Regarding converters and the wish to log their results in case of error...
How about to implement ClientHttpRequestInterceptor with the desired logic:
try {
return execution.execute(request, body);
catch (Exception e) {
this.errorChannel.send(...);
}
I'm prety sure that the byte[] body is your SOAP request.
See ClientHttpRequestInterceptor JavaDocs for more info.
The ClientHttpRequestInterceptor is part of RestTemplate, which can be injected to the http:outbound-gateway.
For the ws:outbound-gateway you should use ClientInterceptor with the handleFault implementation and extract MessageContext.getRequest() for your purpose.

how to add a retry advice with jms:message-driven-channel-adapter

I am new to spring integration. My requirement is that if there is a connection problem to the jms q then it should try to connect 3 times then log it and exit the process. I am not able to do it. It throws an error saying it needs the ref attribute for service:activator. But I don't have/know reference of which class to provide here. Is there any other way of doing it?
<int-jms:message-driven-channel-adapter id="msgIn" channel="toRoute" container="messageListenerContainer" />
<int:service-activator id="service" input-channel="toRoute" >
<int:request-handler-advice-chain>
<bean class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<property name="recoveryCallback">
<bean class="org.springframework.integration.handler.advice.ErrorMessageSendingRecoverer">
<constructor-arg ref=“errorChannel" />
</bean>
</property>
</bean>
</request-handler-advice-chain>
</int:service-activator>
You seem to have completely misunderstood what the framework does.
The service-activator gets a message when one is received from JMS (which implies the connection is good), and needs "something" (a reference to a bean or an expression) to invoke as a result of receiving that message.
The retry advice is to retry calling that service if it fails to process the message for some reason. It is unrelated to whatever is the source of the message (JMS in this case).
It's not clear why you are trying to use Spring Integration for something as simple as testing whether a JMS broker is available.
Perhaps if you can provide some larger context someone might be able to help.

timeout issues in spring integration gateway for soap request and reply

I am facing some chellenge while setting the timeout for the soap request to ESB interface from my interface(Using spring integration). While sending the request to ESB interface, I am setting the request timeout = 5000ms and reply timeout = 5000ms but in case the services is down on ESB interface, the request is not timeout in the desired timeout time of 5000ms(5 sec) and timeout sometimes in 40sec or more. I tried to use the default request timeout and default reply timeout options also with the int:gateway configuration but the same issue. Please see the below configuration done:
<int:gateway id="SoapESBGateway"
service-interface="test.soap.service.ServiceSoap">
<int:method name="ServiceResponse"
request-channel="RequestChannel" reply-channel="ReplyChannel"
request-timeout="5000" reply-timeout="5000" />
</int:gateway>
<int:chain input-channel="RequestChannel" output-channel="ReplyChannel">
<int-xml:marshalling-transformer marshaller="marshaller" result-transformer="resultTransformer"></int-xml:marshalling-transformer>
<int:transformer ref="TransformerBean" method="transformMethod"></int:transformer>
<int-ws:outbound-gateway id="ws-SoapESB-gateway" ignore-empty-responses="true" uri="${soap.URI}"></int-ws:outbound-gateway>
<int:transformer ref="TransformerBean" method="transformMethod"></int:transformer>
<int-xml:unmarshalling-transformer unmarshaller="marshaller"></int-xml:unmarshalling-transformer>
</int:chain>
<bean id="resultTransformer" class="org.springframework.integration.xml.transformer.ResultToStringTransformer" />
<bean id="TransformerBean" class="test.TransformerImpl"></bean>
Please let me know if I am missing something while setting the timeout option.
Thanks,
Vinay A
updated:
<int:gateway id="SoapESBGateway"
service-interface="test.soap.service.ServiceSoap">
<int:method name="ServiceResponse"
request-channel="RequestChannel" reply-timeout="5000" />
</int:gateway>
<int:chain input-channel="RequestChannel">
<int-xml:marshalling-transformer marshaller="marshaller" result-transformer="resultTransformer"></int-xml:marshalling-transformer>
<int:transformer ref="TransformerBean" method="transformMethod"></int:transformer>
<int-ws:outbound-gateway id="ws-SoapESB-gateway" ignore-empty-responses="true" uri="${soap.URI}" message-sender="messageSender"></int-ws:outbound-gateway>
<int:transformer ref="TransformerBean" method="transformMethod"></int:transformer>
<int-xml:unmarshalling-transformer unmarshaller="marshaller"></int-xml:unmarshalling-transformer>
</int:chain>
<bean id="resultTransformer" class="org.springframework.integration.xml.transformer.ResultToStringTransformer" />
<bean id="TransformerBean" class="test.TransformerImpl"></bean>
<bean id ="messageSender" class="org.springframework.ws.transport.http.HttpComponentsMessageSender">
<property name="connectionTimeout" value="2000"></property>
</bean>
As above, I have added messageSender bean reference with the int-ws:outbound-gateway and set the property connectiontimeout = 2sec but I can still see that this is waiting around for 19sec for first hit when network is down and for the next hits, less then 19 sec but not the desired timeout for the first hit at least. So do we not have any option where we may ascertain that the desired timeout work always.
Thanks,
Vinay
First of all, please, read this chapter: http://docs.spring.io/spring-integration/docs/latest-ga/reference/html/messaging-endpoints-chapter.html#d4e4197
According to your config you have direct flow, so there is no reason to have request-timeout and reply-channel.
The issue of your 40sec is under that WS gateway's WebServiceMessageSender. In your case it is HttpUrlConnectionMessageSender by default. And there is some connectTimeout attribute in the HttpURLConnection.
I'm not sure how we can change it, but I prefer to use HttpComponentsMessageSender to have more control over HTTP connections.
Now the question Why is it 40sec, but not those 5sec from reply-timeout?. Because your thread is blocked on the send and it hasn't reached to the wait reply part yet.

Resources