Spring Integration Java DSL using JMS retry/redlivery - spring-integration

How can I effectively support JMS redelivery when msg handling throws an exception?
I have a flow using JMS (ActiveMQ) with a connectionFactory that is configured to allow n redelivery attempts.
I would like to have any error that occurs while handling the msg cause the msg to get put back for redelivery as many times as the connectionFactory config allows and then when max redelivery attempts are exhausted, deliver to DLQ. per usual with AMQ.
An answer to a related SO question implies that I could have an errorChannel that re-throws which should trigger redelivery: Spring Integration DSL ErrorHandling
But, with the following that isnt happening:
/***
* Dispatch msgs from JMS queue to a handler using a rate-limit
* #param connectionFactory
* #return
*/
#Bean
public IntegrationFlow flow2(#Qualifier("spring-int-connection-factory") ConnectionFactory connectionFactory) {
IntegrationFlow flow = IntegrationFlows.from(
Jms.inboundAdapter(connectionFactory)
.configureJmsTemplate(t -> t.receiveTimeout(1000))
.destination(INPUT_DIRECT_QUEUE),
e -> e.poller(Pollers
.fixedDelay(5000)
.errorChannel("customErrorChannel")
//.errorHandler(this.msgHandler)
.maxMessagesPerPoll(2))
).handle(this.msgHandler).get();
return flow;
}
#Bean
public MessageChannel customErrorChannel() {
return MessageChannels.direct("customErrorChannel").get();
}
#Bean
public IntegrationFlow customErrorFlow() {
return IntegrationFlows.from(customErrorChannel())
.handle ("simpleMessageHandler","handleError")
.get();
}
The errorChannel method impl:
public void handleError(Throwable t) throws Throwable {
log.warn("got error from customErrorChannel");
throw t;
}
When an exception is thrown from the handler in flow2, the errorChannel does get the exception but then the re-throw causes a MessageHandlingException:
2018-08-13 09:00:34.221 WARN 98425 --- [ask-scheduler-5] c.v.m.i.jms.SimpleMessageHandler : got error from customErrorChannel
2018-08-13 09:00:34.224 WARN 98425 --- [ask-scheduler-5] o.s.i.c.MessagePublishingErrorHandler : Error message was not delivered.
org.springframework.messaging.MessageHandlingException: nested exception is org.springframework.messaging.MessageHandlingException: error occurred in message handler [simpleMessageHandler]; nested exception is java.lang.IllegalArgumentException: dont want first try, failedMessage=GenericMessage [payload=Enter some text here for the message body..., headers={jms_redelivered=false, jms_destination=queue://_dev.directQueue, jms_correlationId=, jms_type=, id=c2dbffc8-8ab0-486f-f2e5-e8d613d62b6a, priority=0, jms_timestamp=1534176031021, jms_messageId=ID:che2-39670-1533047293479-4:9:1:1:8, timestamp=1534176034205}]
at org.springframework.integration.handler.MethodInvokingMessageProcessor.processMessage(MethodInvokingMessageProcessor.java:107) ~[spring-integration-core-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.integration.handler.BeanNameMessageProcessor.processMessage(BeanNameMessageProcessor.java:61) ~[spring-integration-core-5.0.7.RELEASE.jar:5.0.7.RELEASE]
at org.springframework.integration.handler.ServiceActivatingHandler.handleRequestMessage(ServiceActivatingHandler.java:93) ~[spring-integration-core-5.0.7.RELEASE.jar:5.0.7.RELEASE]

It would work with a message-driven channel adapter but I presume that's not what you want because of this question.
Since the polled adapter uses a JmsTemplate.receive() operation, the message is already ack'd by the time the flow is called.
You need to use a transactional poller with a JmsTransactionManager so that the exception thrown by the error flow rolls back the transaction and the message will be redelivered.

Related

Spring Integration Infinite Loop on any Exception in Custom Error Channel

We have a custom error channel for our application which mainly notifies integration failures via Email. When there is an exception thrown in the SendEmail Http call (WebFlux.outboundGateway) of the custom error channel, the integration flow goes on an infinite loop.
Please help stop this behavior
#Bean("appErrorChannel")
public MessageChannel appErrorChannel() {
return new DirectChannel();
}
#Bean("appErrorFlow")
public IntegrationFlow errorFlow() {
// #formatter:off
return IntegrationFlows.from(appErrorChannel())
.<MessagingException> log(ERROR, message -> "Exception: " + ExceptionUtils.getStackTrace(message.getPayload()))
.transform(transformer, "errorMessage")
.transform(transformer, "emailRequest")
.log(INFO, message -> "Email Request: " + message)
.enrichHeaders(headerEnricherSpec -> headerEnricherSpec.header(CONTENT_TYPE, APPLICATION_JSON_VALUE))
.enrichHeaders(Collections.singletonMap(MessageHeaders.ERROR_CHANNEL, errorFlowErrorChannel()))
.handle(WebFlux.outboundGateway(sendEmailURL, webClient)
.httpMethod(POST)
.expectedResponseType(String.class)
.mappedRequestHeaders(CONTENT_TYPE))
.log(INFO, message -> "Email Response: " + message)
.get();
// #formatter:on
}
#Bean("errorFlowErrorChannel")
public MessageChannel errorFlowErrorChannel() {
return new DirectChannel();
}
#Bean("errorChannelErrorFlow")
public IntegrationFlow errorChannelErrorFlow() {
// #formatter:off
return IntegrationFlows.from(errorFlowErrorChannel())
.<MessagingException> log(FATAL, message -> "Exception in Error Flow: " + ExceptionUtils.getStackTrace(message.getPayload()))
.get();
// #formatter:on
}
Exception:
2022-04-09 11:45:11,198[0;39m [boundedElastic-1 ] [31mERROR[0;39m [36mo.s.i.w.o.WebFluxRequestExecutingMessageHandler[0;39m - [35me2abc8b6eba0119c[0;39m Failed to send async reply
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'application-1.appErrorChannel'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers, failedMessage=ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: nested exception is org.springframework.web.reactive.function.client.WebClientResponseException$BadRequest: 400 Bad Request from POST ****** URL ******, failedMessage=GenericMessage [payload=******* SUPPRESSED ********}, headers={b3=e2abc8b6eba0119c-c4254a192695705d-1, nativeHeaders={}, errorChannel=bean 'appErrorChannel'; defined in: 'class path resource [**************}]
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:76)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:317)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:272)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:457)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendErrorMessage(AbstractMessageProducingHandler.java:496)
at org.springframework.integration.handler.AbstractMessageProducingHandler$ReplyFutureCallback.onFailure(AbstractMessageProducingHandler.java:555)
at org.springframework.util.concurrent.ListenableFutureCallbackRegistry.notifyFailure(ListenableFutureCallbackRegistry.java:86)
at org.springframework.util.concurrent.ListenableFutureCallbackRegistry.failure(ListenableFutureCallbackRegistry.java:158)
.....
.....
Caused by: org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:139)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72)
It is really not recommended to have a single global error channel for the whole application.
Consider to have its own error channel for this specific error handling flow.
Set it via enrichHeaders(MessageHeaders.ERROR_CHANNEL, "errorFlowErrorChannel") and there is not going to be an infinite loop because an error from the WebFlux.outboundGateway() is not going to be forwarded to the appErrorChannel any more.

How to pass object associated with message past outbound channel adapter

I have the following:
[inbound channel adapter] -> ... -> foo -> [outbound channel adapter] -> bar
How can I write my spring-integration app so that foo can an extra object that's not part of the message the [outbound channel adapter] is to consume, such that bar gets it?
My app basically receives messages from AWS SQS (using spring-integration-aws), does some filtering / transformations, then publishes a message to Apache Kafka (using spring-integration-kafka), and if and only if that succeeds, deletes the original message off the SQS queue.
For that reason, when I receive the SQS message, I want to hold onto the receipt handle / acknowledgement object, transform the rest of the message into the Kafka message to be published, and then if that succeeds, make use of that receipt handle / acknowledgement object to dequeue the original message.
So say I'm using this example code off the spring-integration-kafka docs:
#Bean
#ServiceActivator(inputChannel = "toKafka", outputChannel = "result")
public MessageHandler handler() throws Exception {
KafkaProducerMessageHandler<String, String> handler =
new KafkaProducerMessageHandler<>(kafkaTemplate());
handler.setTopicExpression(new LiteralExpression("someTopic"));
handler.setMessageKeyExpression(new LiteralExpression("someKey"));
handler.setFailureChannel(failures());
return handler;
}
#Bean
public KafkaTemplate<String, String> kafkaTemplate() {
return new KafkaTemplate<>(producerFactory());
}
#Bean
public ProducerFactory<String, String> producerFactory() {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, this.brokerAddress);
// set more properties
return new DefaultKafkaProducerFactory<>(props);
}
With the above, if I have a message message and some extra, unrelated info extra, what do I send to the toKafka channel such that handler will consume message, and if that was successful, the result channel will receive extra?
Outbound channel adapters don't produce output - they are one-way only and end the flow.
You can make toKafka a PublishSubscribeChannel and add a second service activator; by default, the second will only be called if the first is successful.

Spring-Integration Webflux exception handling

If an exception occurs in a spring-integration webflux flow, the exception itself (with stacktrace) is sent back to the caller as payload through MessagePublishingErrorHandler, which uses an error channel from the "errorChannel" header, not the default error channel.
How can I set up an error handler similar to WebExceptionHandler? I want to produce an Http status code and possibly a DefaultErrorAttributes object as response.
Simply defining a flow that starts from the errorChannel doesn't work, the error message won't end up there. I tried to define my own fluxErrorChannel, but it appears that it is also not used as error channel, the errors do not end up in my errorFlow:
#Bean
public IntegrationFlow fooRestFlow() {
return IntegrationFlows.from(
WebFlux.inboundGateway("/foo")
.requestMapping(r -> r.methods(HttpMethod.POST))
.requestPayloadType(Map.class)
.errorChannel(fluxErrorChannel()))
.channel(bazFlow().getInputChannel())
.get();
}
#Bean
public MessageChannel fluxErrorChannel() {
return MessageChannels.flux().get();
}
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from(fluxErrorChannel())
.transform(source -> source)
.enrichHeaders(h -> h.header(HttpHeaders.STATUS_CODE, HttpStatus.BAD_GATEWAY))
.get();
}
#Bean
public IntegrationFlow bazFlow() {
return f -> f.split(Map.class, map -> map.get("items"))
.channel(MessageChannels.flux())
.<String>handle((p, h) -> throw new RuntimeException())
.aggregate();
}
UPDATE
In MessagingGatewaySupport.doSendAndReceiveMessageReactive my error channel defined on the WebFlux.inboundGateway is never used to set the error channel, rather the error channel is always the replyChannel which is being created here:
FutureReplyChannel replyChannel = new FutureReplyChannel();
Message<?> requestMessage = MutableMessageBuilder.fromMessage(message)
.setReplyChannel(replyChannel)
.setHeader(this.messagingTemplate.getSendTimeoutHeader(), null)
.setHeader(this.messagingTemplate.getReceiveTimeoutHeader(), null)
.setErrorChannel(replyChannel)
.build();
The error channel is ultimately being reset to the originalErrorChannelHandler in Mono.fromFuture, but that error channel is ˋnullˋ in my case. Also, the onErrorResume lambda is never invoked:
return Mono.fromFuture(replyChannel.messageFuture)
.doOnSubscribe(s -> {
if (!error && this.countsEnabled) {
this.messageCount.incrementAndGet();
}
})
.<Message<?>>map(replyMessage ->
MessageBuilder.fromMessage(replyMessage)
.setHeader(MessageHeaders.REPLY_CHANNEL, originalReplyChannelHeader)
.setHeader(MessageHeaders.ERROR_CHANNEL, originalErrorChannelHeader)
.build())
.onErrorResume(t -> error ? Mono.error(t) : handleSendError(requestMessage, t));
How is this intended to work?
It's a bug; the ErrorMessage created for the exception by the error handler is sent to the errorChannel header (which has to be the replyChannel so the gateway gets the result). The gateway should then invoke the error flow (if present) and return the result of that.
https://jira.spring.io/browse/INT-4541

Handling errors after a message splitter with direct channels

I'm working on a service which sends emails using the spring integration java dsl.
I have a batch message which is split into a collection of individual messages which will be turned into emails.
The issue I am experiencing is that if one of these individual messages throws an error, the other messages in the batch are not processed.
Is there a way to configure the flow so that when a message throws an exception, the exception is handled gracefully and the next message in the batch is processed?
The following code achieves the functionality I would like but I'm wondering if there is an easier / better way to achieve this, ideally in a single IntegrationFlow? :
#Bean
public MessageChannel individualFlowInputChannel() {
return MessageChannels.direct().get();
}
#Bean
public IntegrationFlow batchFlow() {
return f -> f
.split()
.handle(message -> {
try {
individualFlowInputChannel().send(message);
} catch (Exception e) {
e.printStackTrace();
}
});
}
#Bean
public IntegrationFlow individualFlow() {
return IntegrationFlows.from(individualFlowInputChannel())
.handle((payload, headers) -> {
throw new RuntimeException("BOOM!");
}).get();
}
You can add ExpressionEvaluatingRequestHandlerAdvice to the last handle() definition with its trapException option:
/**
* If true, any exception will be caught and null returned.
* Default false.
* #param trapException true to trap Exceptions.
*/
public void setTrapException(boolean trapException) {
On the other hand, if you are talking about "sends emails", wouldn't it be better to consider to do that in the separate thread for each splitted item? In this case the ExecutorChannel after .split() comes to the rescue!

Are there any code samples for spring integration dsl error handling?

The JmsTests.java was very helpful in understanding ways to structure my code. Code is at https://github.com/spring-projects/spring-integration-java-dsl/blob/master/src/test/java/org/springframework/integration/dsl/test/jms/JmsTests.java
But, for error handling, I am kind of figuring it out as we go. Are there any tests or reference code that show good ways to structure error channels or using sub-flows to delegate error handling?
Adding more context to the question:
jmsMessageDrivenFlow: reads from incoming queue and prints payload. If there is an error, then it is sent to errorChannel()
handleErrors(): Is supposed to listen to errorChannel and send anything there to the fatalErrorQueue
jmsMessageDrivenFlow() works fine and prints the message. I am expecting the error handler to be called only if there is an error during the jmsMessageDrivenFlow. However, the errorhandler() also gets called, and it puts a 'Dispatcher failed to deliver Message' message in the fatal queue.
I am hoping that the error channel shouldn't even be invoked in this scenario since jmsMessageDrivenFlow() didn't create any errors. Obviously, if I leave out .errorChannel(errorChannel()) in the jmsMessageDrivenFlow(), the error flow is not invoked and I get only jmsMessageDrivenFlow() executing as expected.
#EnableIntegration
#IntegrationComponentScan
#Component
#Configuration
public class MessageReceiver {
private static final Logger logger = LoggerFactory.getLogger(MessageReceiver.class);
String fatalErrorQueue = "fatal";
String incomingQueue = "incoming";
#Autowired
private JmsMessagingTemplate jmsMessagingTemplate;
#Bean
public Queue incomingQueue() {
return new ActiveMQQueue(incomingQueue);
}
#Bean
public MessageChannel errorChannel() {
return new DirectChannel();
}
#Bean
public IntegrationFlow handleErrors() {
return IntegrationFlows
.from("errorChannel")
.handle((payload,headers) -> {
System.out.println("processing error: "+payload.toString());
return payload;
})
.handle(Jms.outboundAdapter(jmsMessagingTemplate.getConnectionFactory()).destination(fatalErrorQueue))
.get();
}
#Bean
public IntegrationFlow jmsMessageDrivenFlow() {
return IntegrationFlows
.from(
Jms.messageDriverChannelAdapter(jmsMessagingTemplate.getConnectionFactory())
.destination(incomingQueue)
.errorChannel(errorChannel())
)
.handle((payload,headers) ->{
System.out.println("processing payload: "+payload.toString());
return payload;
})
.get();
}
}
Log of the execution:
2016-01-22 10:18:20,531 DEBUG DefaultMessageListenerContainer-1 ServiceActivatingHandler.handleMessage - ServiceActivator for [org.springframework.integration.dsl.LambdaMessageProcessor#522d5d91] (org.springframework.integration.handler.ServiceActivatingHandler#1) received message: GenericMessage [payload=SAMPLE QUEUE MESSAGE, headers={jms_redelivered=false, jms_correlationId=, jms_type=, id=78d5456e-4442-0c2b-c545-870d2c177802, priority=0, jms_timestamp=1453479493323, jms_messageId=ID:crsvcdevlnx01.chec.local-47440-1453310266960-6:2:1:1:1, timestamp=1453479500531}]
processing payload: SAMPLE QUEUE MESSAGE
2016-01-22 10:18:20,535 DEBUG DefaultMessageListenerContainer-1 DirectChannel.send - preSend on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, headers={id=dc87f71d-a38c-4718-a2e9-2fba2a7c847c, timestamp=1453479500535}]
2016-01-22 10:18:20,535 DEBUG DefaultMessageListenerContainer-1 ServiceActivatingHandler.handleMessage - ServiceActivator for [org.springframework.integration.dsl.LambdaMessageProcessor#5545b00b] (org.springframework.integration.handler.ServiceActivatingHandler#0) received message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, headers={id=dc87f71d-a38c-4718-a2e9-2fba2a7c847c, timestamp=1453479500535}]
processing error: org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available
2016-01-22 10:18:20,535 DEBUG DefaultMessageListenerContainer-1 DirectChannel.send - preSend on channel 'handleErrors.channel#0', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, headers={id=f434348c-f659-1e88-2eba-3d467761ab47, timestamp=1453479500535}]
2016-01-22 10:18:20,535 DEBUG DefaultMessageListenerContainer-1 JmsSendingMessageHandler.handleMessage - org.springframework.integration.jms.JmsSendingMessageHandler#0 received message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, headers={id=f434348c-f659-1e88-2eba-3d467761ab47, timestamp=1453479500535}]
2016-01-22 10:18:20,537 DEBUG DefaultMessageListenerContainer-1 DynamicJmsTemplate.execute - Executing callback on JMS Session: ActiveMQSession {id=ID:MACD13-60edd8d-49463-1453479492653-1:1:1,started=true} java.lang.Object#858db12
2016-01-22 10:18:20,568 DEBUG DefaultMessageListenerContainer-1 DynamicJmsTemplate.doSend - Sending created message: ActiveMQObjectMessage {commandId = 0, responseRequired = false, messageId = null, originalDestination = null, originalTransactionId = null, producerId = null, destination = null, transactionId = null, expiration = 0, timestamp = 0, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = null, replyTo = null, persistent = false, type = null, priority = 0, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence#559da78d, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {timestamp=1453479500535}, readOnlyProperties = false, readOnlyBody = false, droppable = false, jmsXGroupFirstForConsumer = false}
2016-01-22 10:18:20,570 DEBUG DefaultMessageListenerContainer-1 DirectChannel.send - postSend (sent=true) on channel 'handleErrors.channel#0', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, headers={id=f434348c-f659-1e88-2eba-3d467761ab47, timestamp=1453479500535}]
2016-01-22 10:18:20,571 DEBUG DefaultMessageListenerContainer-1 DirectChannel.send - postSend (sent=true) on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, headers={id=dc87f71d-a38c-4718-a2e9-2fba2a7c847c, timestamp=1453479500535}]
OK. Thanks. Investigating... But from the big height everything works as expected:
Your jmsMessageDrivenFlow() is one-way flow
You have non-end .handle() in the end of that flow:
.handle((payload,headers) ->{
System.out.println("processing payload: "+payload.toString());
return payload;
})
Since you return anything, the flow expect the output-channel or replyChannel in headers. The ErrorMessage says you about that.
I tell that about the approach to fix the error altogether. For your case you can just change .handle((payload,headers) -> to the .handle(message ->
which is one-way to stop your flow exactly there.
Since you really have that issue for the next endpoint after that replying .handle(), the messagingGateway reacts for that correctly:
this.messagingTemplate.convertAndSend(requestChannel, object, this.historyWritingPostProcessor);
}
catch (Exception e) {
MessageChannel errorChannel = getErrorChannel();
if (errorChannel != null) {
this.messagingTemplate.send(errorChannel, new ErrorMessage(e));
}
And that's why your handleErrors() is able to handle that error.
Without errorChannel() on the Jms.messageDriverChannelAdapter() we end up in the AbstractMessageListenerContainer:
protected void executeListener(Session session, Message message) {
try {
doExecuteListener(session, message);
}
catch (Throwable ex) {
handleListenerException(ex);
}
}
where its default ErrorHandler is ... null and we see only:
logger.warn("Execution of JMS message listener failed, and no ErrorHandler has been set.", ex);
Isn't it?
UPDATE
Can you please explain why messagingGateway sends the payload to errorChannel() just because I have defined an error channel for non-happy-path scnenarios.
??? You will end up with the same error even with the XML definition. The Java DSL uses the same components on the background. I won't explain the regular Java code: you see the try...catch around convertAndSend and your condition at runtime defines that there is something wrong. That's why you get that DestinationResolutionException. we can't identify non-happy-path during init phase, because your return in the .handle() can be lucky null only at runtime. So, from the configuration perspective everything is OK. From other side even without outputChannel you still can be happy there with the request/reply scenarion, when we have replyChannel header on the matter.

Resources