error-channel and reply-channel vanishes during header aggregation - spring-integration

I have the following workflow.
inbound-channel
splitter
task executor for split channels - all the threads execute the same workflow.
3.a. construct the request
3.b. service activator wrapper for a gateway message endpoint.
3.c. gateway wrapper over the http-outbound-gateway with the configuration of error-channel (to handle exceptions while invoking http-outbound-gateway)
3.d. http-outbound-gateway
aggregator
response out of spring integration workflow.
If an exception occurs in 3.d, the control goes to the service-activator configured for the gateway error channel.
I copy just the following from the failed message to the new header to the header passed to the error channel.
a. correlationId
b. sequenceNumber
c. sequenceSize
But while aggregating the splitter response, the DefaultAggregatingMessageGroupProcessor.java removes the conflicting headers and by that it removes the error-channel and reply-channel before providing the control to aggregator.
So once the aggregator completes it's operation it is unable to find the reply or error channel and it results in an exception.
I'm using spring-integration-core version 2.2.1 and I'm not sure why the reply-channel and error-channel is being removed during header aggregation.
Any input on resolving this issue will be of great help.
Thank You :)
EDIT 1:
Thank You very much Gary for helping me out with this scenario. I'm sharing my current configuration
<!-- SPLITTER -->
<int:splitter id="dentalSplitter" ref="dentalServiceSplitter"
method="getDentalServiceList" input-channel="dentalServiceSplitterChannel"
output-channel="dentalSplitterTaskChannel" />
<int:channel id="dentalSplitterTaskChannel">
<int:dispatcher task-executor="dentalTaskExecutor" />
</int:channel>
<int:chain input-channel="dentalSplitterTaskChannel" output-channel="dentalGatewayChannel">
<int:header-enricher>
<int:header name="CHAIN_START_TIME" expression="T(System).currentTimeMillis()" overwrite="true" />
<int:object-to-json-transformer content-type="application/json"/>
</int:chain>
<int:service-activator input-channel="dentalGatewayChannel" ref="dentalGatewayWrapper" output-channel="dentalReplyChannel" />
<int:gateway id="dentalGatewayWrapper" default-request-channel="dentalCostEstimateRequestChannel" error-channel="dentalErrorChannel"/>
<int-http:outbound-gateway id="dentalGateway"
url-expression="#urlBuilder.build('${service.endpoint}')"
http-method="POST" request-factory="clientHttpRequestFactory"
request-channel="dentalCostEstimateRequestChannel" extract-request-payload="true"
expected-response-type="com.dental.test.DentalResponse">
<int-http:request-handler-advice-chain>
<ref bean="logChainTimeInterceptor" />
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
<!-- EXCEPTION -->
<int:chain input-channel="dentalErrorChannel" output-channel="dentalAggregatorChannel">
<int:transformer ref="commonErrorTransformer" method="dentalGracefulReturn"/>
</int:chain>
<!-- SUCCESS -->
<int:chain input-channel="dentalReplyChannel" output-channel="dentalAggregatorChannel">
<int:filter discard-channel="dentalErrorChannel"
expression="T(com.dental.util.InvocationOutcomeHelper).isOutcomeSuccess(payload?.metadata?.outcome?.code,payload?.metadata?.outcome?.message)" />
</int:chain>
<!-- AGGREGATION -->
<int:chain input-channel="dentalAggregatorChannel" output-channel="wsDentalServiceOutputChannel" >
<int:aggregator ref="dentalServiceAggregator" />
<int:service-activator ref="dentalResponseServiceActivator" />
</int:chain>
What I noticed was this, every split channel when passing through the gateway creates a new temporary channel for error and reply and after getting the response back from the gateway, it retains the preserved (original inbound) error and reply channel header. And as you had mentioned, after the control gets to the error transformer that flow of retaining the preserved headers gets broken and the aggregating message group processor receives three different instances of temporary channel and hence removes them.
I was planning to have a custom message group processor and modify the conflict resolution strategy for aggregating the header and came up with this config.
<bean id="channelPreservingAggregatingMessageHandler" class="org.springframework.integration.aggregator.AggregatingMessageHandler">
<constructor-arg name="processor" ref="channelPreservingMessageGroupProcessor"/>
</bean>
I'm yet to test this out though. But based on this discussion, this does not look like a viable solution.
And looks like my configuration for error handling in gateway is incorrect.
However, I'm confused on this statement of yours "Instead of forwarding the message directly, simply handle the error on your error flow and return the result normally to the gateway "wrapper"". If I remove the error channel how will I get the control back when an exception occurs? May be I'm missing to understand something here. Can you elaborate more on this please?

When asking questions about scenarios such as this, you generally need to show your configuration. However, I suspect you are forwarding the message from the error flow directly to the aggregator.
This is like doing a GOTO in code and breaks the scoping.
It won't work because the replyChannel header in the error message is for the gateway "wrapper", not the original upstream inbound gateway. When the aggregator gets conflicting headers, it has no choice but to drop the headers (you will see a DEBUG log message to that effect).
Instead of forwarding the message directly, simply handle the error on your error flow and return the result normally to the gateway "wrapper" (simply omit the error channel on the last element on the error flow).
The gateway will then fix up the reply so it is consistent with other messages (good and bad) and forward it to the aggregator.
You don't need to mess with headers in your error flow, just return the value you want to be aggregated along with the good results.
You should really update to a current release, or at least the latest in the 2.2.x line (2.2.6).

Related

Error is not propagated to my error handler while we are using dispatcher task-executor in Channel

Below is code snippets, where I am trying to handle an error.
<int:channel id="errorGateWayChannel"></int:channel>
<int:service-activator ref="errorHandler"
input-channel="errorGateWayChannel" />
...
<int:service-activator.. />
<int-http:outbound-gateway ..>
<int:channel id="routerInputChannel">
<int:dispatcher task-executor="sharedPool"/></int:channel>
<int:header-enricher input-channel="routerInputChannel" output-channel="routerInputChannel1">
<int:error-channel ref="errorGateWayChannel"/>
</int:header-enricher>
<int:recipient-list-router input-channel="routerInputChannel1"
default-output-channel="nullChannel">
<int:recipient channel="datasetInputChannel"/>
</int:recipient-list-router>
<int:service-activator id="datasetInputChannelSA" input-channel="datasetInputChannel"/> <!-- now this service activator giving me error -->
Now if "datasetInputChannelSA" service activator throw any exception I am getting WARN and that not propagate to error channel.
WARN org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel - Reply message received but the receiving thread has already received a reply:ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: nested exception is java.lang.RuntimeException:
I can see similar threads where they are getting WARN but not found how to handle it.
Update:
After providing overwrite="true" in "int:error-channel" it is working and sovle my issue.
<int:error-channel ref="errorGateWayChannel"/>
to
<int:error-channel ref="errorGateWayChannel" overwrite="true"/>
Your configuration is not full enough to play with on our side, so, would be great to have some small project from you somewhere on GitHub to let us to reproduce an issue if that.
However, if you would like to handle errors from the datasetInputChannelSA and don't have any other problems, you need to consider to use a <request-handler-advice-chain> for that <service-activator> and include an ExpressionEvaluatingRequestHandlerAdvice with its trapException as true. There you can configure a failureChannel to catch service exceptions and handle them in the consumer of that channel: https://docs.spring.io/spring-integration/docs/current/reference/html/#expression-advice
It's already late to set a error-channel header in an Executor context: the failed message is not going to consult downstream headers. So, your error is propagated to the TemporaryReplyChannel header populated by the upstream gateway call (not shown in your code snippet though...).

Spring integration handler error in outbound-channel-adapter --> confirm-ack-channel

I want to do:
Receive message from web service
Send xml message to rabbit
Validate xml to xsd --> Launch exception (validation incorrect)
Return custom error message to web service
Step 1: Receive message from web service --> RESULT OK
<ws:inbound-gateway id="ws-inbound-gateway"
request-channel="requestChannel" reply-channel="replyChannel" reply-timeout="300000"
error-channel="errorChannel" />
<int:chain input-channel="requestChannel" output-channel="inbound">
<int:service-activator ref="defaultLogger" method="logger"/>
</int:chain>
Step 2: Write a xml message in fanout exchange rabbit --> RESULT OK
<int-amqp:outbound-channel-adapter
channel="inbound" amqp-template="amqpTemplate" return-channel="outbound"
exchange-name="es.queue.test"
confirm-ack-channel="confirmAck" confirm-nack-channel="confirmNack" confirm-correlation-expression="#this" />
Step 3: Validate xml to xsd --> Result ok XsdValidationException
<int:chain input-channel="confirmAck" output-channel="outbound">
<int:service-activator ref="defaultLogger" method="logger"/>
<int-xml:validating-filter schema-type="xml-schema"
schema-location="classpath:/schema/prueba.xsd"
throw-exception-on-rejection="true" discard-channel="errorChannel" />
</int:chain>
In this step the message will be sent to errorChannel but I have the next error:
63863 [AMQP Connection 10.0.9.155:5672] ERROR o.s.a.r.s.PublisherCallbackChannelImpl - Exception delivering confirm
org.springframework.integration.MessageRejectedException: Message was rejected due to XML Validation errors; nested exception is org.springframework.integration.xml.AggregatedXmlMessageValidationException: Multiple causes:
cvc-elt.1: No se ha encontrado la declaración del elemento 'ns2:***'.
at org.springframework.integration.xml.selector.XmlValidatingMessageSelector.accept(XmlValidatingMessageSelector.java:134)
at org.springframework.integration.filter.MessageFilter.doHandleRequestMessage(MessageFilter.java:161)
at org.springframework.integration.handler.AbstractReplyProducingPostProcessingMessageHandler.handleRequestMessage(AbstractReplyProducingPostProcessingMessageHandler.java:46)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:109)
Step 4: The web service client never receive response
I think that it's error "Exception delivering confirm" occurs because the exception is throw in thread AMQP when try send confirm ack.
Can you help me?
Everything looks good unless you should explain to us why do you use throw-exception-on-rejection="true" and break your flow?
I even think that confirm-correlation-expression="#this" really passes a requestMessage to the confirm-ack-channel, so you can send reply back to the WS Gateway from that sub-flow.
But! Since you throw-exception-on-rejection="true" nothing is going to be sent to the outbound channel.
Also return-channel is not for reply. It is another error state, when Broker can't deliver the message to the queue because of some misconfiguration there: http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/

Implementing Gateway causing Problems in Message Flow

I have a scenario in which i am receiving a message and routing it to different message channels using Recipient-list-router.
<int:recipient-list-router input-channel="inputChannel1"
apply-sequence="true" id="router1">
<int:recipient channel="service1" />
<int:recipient channel="service2" />
</int:recipient-list-router>
This aggregator is used to aggregate the response from above 2 services and redirects the aggregated output to some other channel for further processing.
<int:aggregator input-channel="aggregator1" output-channel="channel2" >
<bean class="wsintg.service.AggregatorClass" />
</int:aggregator>
The code for individual services looks like this :
<!--service 1 code -->
<int:service-activator input-channel="service1" ref="service1Ref" />
<int:gateway id="service1Ref" default-request-channel="success_channel" error-channel="error_channel" />
<int:chain id="chainid1" input-channel="success_channel" output-channel="aggregator1">
step 1 :xsl transformation
step 2 :call to a service using ws:outbound-gateway
<int:service-activator id="sa1">
<bean class="samplePkg.returnMessage"></bean> <!--return message simply returns the message without doing anything -->
</int:service-activator>
</int:chain>
<int:chain id="chainid2" input-channel="error_channel" output-channel="aggregator1" >
<int:service-activator>
<!-- some custom exception handling -->
</int:service-activator>
</int:chain>
<!--service 2 code --> same as service 1
Problem :
If i don't use gateway in Service 1 and Service 2 and simply do the transformations, the code works perfectly.
But, as i need to do some error handling using error_channel, so i put a gateway in service1 and service 2 code. Now the flow
stucks after service activator in service 1 without throwing any
exception. and does not calls the service 2 flow.
You need to show real configuration, not just comments; we can't guess what your application looks like.
That said, you can't just send error messages to the aggregator because the aggregation information is in the failedMessage headers and needs to be promoted.
See error handling in this sample for an example.
As always, the first step to debug situations like this is to turn on DEBUG logging for org.springframework.integration and follow the messages through the flow.
You will usually be able to determine the problem that way.

Reply Time out for StoredProcOutboundGateway

I have issue with reply-timeout property of StoredProcOutboundGateway in spring integration. I have gone through the documentation and it says reply-time out will not work in case of direct channel for the Gateway . But in case of external integration gateways ie StoredProcOutboundGateway it says
"when sending to a DirectChannel, the invocation will occur in the sender’s thread so the failing of the send operation may be caused by other components further downstream "
As per my design my StoredProcOutboundGateway reply is sending to a Direct channel.Will the StoredProcOutboundGateway throw exception after the reply-time out expiration ? .For re creating this scenario I have put a debug point and wait for configured reply time out in the first line of StoredProcOutboundGateway.handleRequestMessage method ,But it didn't thrown any exception and it returned successsful response even after configured time .
<int:gateway id="idGateway" service-interface="Gateway">
<int:method name="getStoreProcData" request-channel="store.req.channel" reply-channel="store.reply.channel" />
</int:gateway>
<int-jdbc:stored-proc-outbound-gateway stored-procedure-name="GE_PKG.GET_PRC"
ignore-column-meta-data="true" reply-timeout="2000" request-channel="store.req.channel" reply-channel="store.reply.channel" is-function="false"
............other cursor config
</int-jdbc:stored-proc-outbound-gateway>enter code here
<int:channel id="store.req.channel" />
<int:channel id="store.reply.channel" />
Spring-Integration V4..1.2 DB oracle ojdbc6
The reply-timeout only applies if the reply-channel can block - such as a bounded QueueChannel that is full.
i.e. it is a timeout for the send to the reply-channel.
It will never occur with a DirectChannel.

Spring Integration - Missing Messages

I have an integration application that mostly works, but noticed yesterday a message was lost. At the time, the service-activator endpoint was extremely busy processing a previous message.
Basically, my integration flow is:
jdbc:inbound-channel-adapter --> splitter -> aggregator (based on a type field) -> [pollable channel] --> service-activator
From the log, I can see a group of messages polled, split, aggregated, expired (via a message store reaper), and placed on the creation channel. There is a preSend and postSend event in the log for placing the message on the creationChannel, but that's the end of it. It never gets to my service-activator, there's no messages in the jdbc message store and I dont see any errors.
Could there an issue with how I have either the service-activator or poller configured (or a combination)?
Here is the configuration applicable to the issue.
<int:poller default="true" fixed-delay="1000" receive-timeout="0" />
<int-jdbc:message-store id="jdbc-messageStore" data-source="dataSource" />
<int:channel id="creationChannel" >
<int:queue message-store="jdbc-messageStore" />
</int:channel>
<!-- Endpoint responsible for creating Archive from List of Document objects -->
<bean id="archiveCreator" class="org.abc.ArchiveCreator"/>
<int:service-activator ref="archiveCreator" method="createArchive" input-channel="creationChannel" />
First of all you should mark your <poller> as <transactional>.
Another point switch on the org.springframework.integration DEBUG logging level and show here the log, when a message is lost.

Resources