I am trying to design a message processing pipeline that needs to process a message using multiple end-points. At each stage, the endpoints returns either the payload (could be transformed) or an error message. Here's a generic example of what I have in my mind:
<int:payload-type-router input-channel="preprocessing-output">
<int:mapping type="com.example.Error" channel="error" />
<int:mapping type="com.example.PreprocessedDomainObject" channel="validation-input"/>
</int:payload-type-router>
<int:service-activator input-channel="validation-input"
ref="validationService" method="validate" output-channel="validation-output"/>
<int:payload-type-router input-channel="validation-output">
<int:mapping type="com.example.Error" channel="error" />
<int:mapping type="com.example.CouldBeAnotherObject" channel="processor-input"/>
</int:payload-type-router>
So on, this processing chain can be quite long... Is there a better way to design this instead of a payload type router after every stage? Kinda feels redundant.
Well, I'd make it based on the exceptions. The validation service should just throw exception and all you need is to catch it on the caller or have an errorChannel if it is Inbound Channel Adapter.
That way you don't need those routers definitions at all and just plain services calling chain.
Related
I have a scenario in which i am receiving a message and routing it to different message channels using Recipient-list-router.
<int:recipient-list-router input-channel="inputChannel1"
apply-sequence="true" id="router1">
<int:recipient channel="service1" />
<int:recipient channel="service2" />
</int:recipient-list-router>
This aggregator is used to aggregate the response from above 2 services and redirects the aggregated output to some other channel for further processing.
<int:aggregator input-channel="aggregator1" output-channel="channel2" >
<bean class="wsintg.service.AggregatorClass" />
</int:aggregator>
The code for individual services looks like this :
<!--service 1 code -->
<int:service-activator input-channel="service1" ref="service1Ref" />
<int:gateway id="service1Ref" default-request-channel="success_channel" error-channel="error_channel" />
<int:chain id="chainid1" input-channel="success_channel" output-channel="aggregator1">
step 1 :xsl transformation
step 2 :call to a service using ws:outbound-gateway
<int:service-activator id="sa1">
<bean class="samplePkg.returnMessage"></bean> <!--return message simply returns the message without doing anything -->
</int:service-activator>
</int:chain>
<int:chain id="chainid2" input-channel="error_channel" output-channel="aggregator1" >
<int:service-activator>
<!-- some custom exception handling -->
</int:service-activator>
</int:chain>
<!--service 2 code --> same as service 1
Problem :
If i don't use gateway in Service 1 and Service 2 and simply do the transformations, the code works perfectly.
But, as i need to do some error handling using error_channel, so i put a gateway in service1 and service 2 code. Now the flow
stucks after service activator in service 1 without throwing any
exception. and does not calls the service 2 flow.
You need to show real configuration, not just comments; we can't guess what your application looks like.
That said, you can't just send error messages to the aggregator because the aggregation information is in the failedMessage headers and needs to be promoted.
See error handling in this sample for an example.
As always, the first step to debug situations like this is to turn on DEBUG logging for org.springframework.integration and follow the messages through the flow.
You will usually be able to determine the problem that way.
I have the following workflow.
inbound-channel
splitter
task executor for split channels - all the threads execute the same workflow.
3.a. construct the request
3.b. service activator wrapper for a gateway message endpoint.
3.c. gateway wrapper over the http-outbound-gateway with the configuration of error-channel (to handle exceptions while invoking http-outbound-gateway)
3.d. http-outbound-gateway
aggregator
response out of spring integration workflow.
If an exception occurs in 3.d, the control goes to the service-activator configured for the gateway error channel.
I copy just the following from the failed message to the new header to the header passed to the error channel.
a. correlationId
b. sequenceNumber
c. sequenceSize
But while aggregating the splitter response, the DefaultAggregatingMessageGroupProcessor.java removes the conflicting headers and by that it removes the error-channel and reply-channel before providing the control to aggregator.
So once the aggregator completes it's operation it is unable to find the reply or error channel and it results in an exception.
I'm using spring-integration-core version 2.2.1 and I'm not sure why the reply-channel and error-channel is being removed during header aggregation.
Any input on resolving this issue will be of great help.
Thank You :)
EDIT 1:
Thank You very much Gary for helping me out with this scenario. I'm sharing my current configuration
<!-- SPLITTER -->
<int:splitter id="dentalSplitter" ref="dentalServiceSplitter"
method="getDentalServiceList" input-channel="dentalServiceSplitterChannel"
output-channel="dentalSplitterTaskChannel" />
<int:channel id="dentalSplitterTaskChannel">
<int:dispatcher task-executor="dentalTaskExecutor" />
</int:channel>
<int:chain input-channel="dentalSplitterTaskChannel" output-channel="dentalGatewayChannel">
<int:header-enricher>
<int:header name="CHAIN_START_TIME" expression="T(System).currentTimeMillis()" overwrite="true" />
<int:object-to-json-transformer content-type="application/json"/>
</int:chain>
<int:service-activator input-channel="dentalGatewayChannel" ref="dentalGatewayWrapper" output-channel="dentalReplyChannel" />
<int:gateway id="dentalGatewayWrapper" default-request-channel="dentalCostEstimateRequestChannel" error-channel="dentalErrorChannel"/>
<int-http:outbound-gateway id="dentalGateway"
url-expression="#urlBuilder.build('${service.endpoint}')"
http-method="POST" request-factory="clientHttpRequestFactory"
request-channel="dentalCostEstimateRequestChannel" extract-request-payload="true"
expected-response-type="com.dental.test.DentalResponse">
<int-http:request-handler-advice-chain>
<ref bean="logChainTimeInterceptor" />
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
<!-- EXCEPTION -->
<int:chain input-channel="dentalErrorChannel" output-channel="dentalAggregatorChannel">
<int:transformer ref="commonErrorTransformer" method="dentalGracefulReturn"/>
</int:chain>
<!-- SUCCESS -->
<int:chain input-channel="dentalReplyChannel" output-channel="dentalAggregatorChannel">
<int:filter discard-channel="dentalErrorChannel"
expression="T(com.dental.util.InvocationOutcomeHelper).isOutcomeSuccess(payload?.metadata?.outcome?.code,payload?.metadata?.outcome?.message)" />
</int:chain>
<!-- AGGREGATION -->
<int:chain input-channel="dentalAggregatorChannel" output-channel="wsDentalServiceOutputChannel" >
<int:aggregator ref="dentalServiceAggregator" />
<int:service-activator ref="dentalResponseServiceActivator" />
</int:chain>
What I noticed was this, every split channel when passing through the gateway creates a new temporary channel for error and reply and after getting the response back from the gateway, it retains the preserved (original inbound) error and reply channel header. And as you had mentioned, after the control gets to the error transformer that flow of retaining the preserved headers gets broken and the aggregating message group processor receives three different instances of temporary channel and hence removes them.
I was planning to have a custom message group processor and modify the conflict resolution strategy for aggregating the header and came up with this config.
<bean id="channelPreservingAggregatingMessageHandler" class="org.springframework.integration.aggregator.AggregatingMessageHandler">
<constructor-arg name="processor" ref="channelPreservingMessageGroupProcessor"/>
</bean>
I'm yet to test this out though. But based on this discussion, this does not look like a viable solution.
And looks like my configuration for error handling in gateway is incorrect.
However, I'm confused on this statement of yours "Instead of forwarding the message directly, simply handle the error on your error flow and return the result normally to the gateway "wrapper"". If I remove the error channel how will I get the control back when an exception occurs? May be I'm missing to understand something here. Can you elaborate more on this please?
When asking questions about scenarios such as this, you generally need to show your configuration. However, I suspect you are forwarding the message from the error flow directly to the aggregator.
This is like doing a GOTO in code and breaks the scoping.
It won't work because the replyChannel header in the error message is for the gateway "wrapper", not the original upstream inbound gateway. When the aggregator gets conflicting headers, it has no choice but to drop the headers (you will see a DEBUG log message to that effect).
Instead of forwarding the message directly, simply handle the error on your error flow and return the result normally to the gateway "wrapper" (simply omit the error channel on the last element on the error flow).
The gateway will then fix up the reply so it is consistent with other messages (good and bad) and forward it to the aggregator.
You don't need to mess with headers in your error flow, just return the value you want to be aggregated along with the good results.
You should really update to a current release, or at least the latest in the 2.2.x line (2.2.6).
I am planning to use a buffered queue channel backed by IBM MQ Queue. I would like the Spring message to be put into a MQ Queue so that each can be handled in a separate transaction and if the downstream webservice fails it can be re-tried and ultimately put in a backout queue. This way the messages will never be lost.
I see there are implementations for JDBC Message Store and others but I cannot find anything for MQ Queue. Can you please point me if I am missing
<si:service-activator id="eventHandler"
input-channel="channel1l" output-channel="amountDataChannel"
method="processEvent" ref="eventService" >
</si:service-activator>
<si:channel id="amountDataChannel">
<si:queue message-store="queueMessageStore ???"/>
</si:channel>
<si:chain id="dataChain1" input-channel="amountDataChannel" output-
channel="outputChannel">
<si:poller fixed-rate="10000" max-messages-per-poll="1" />
<si:transformer ref="transformer" method="transformEvent"/>
..Make webservice call
</si:chain>
Update
After reading through the documentation here is what I am doing though I have a question on transaction manger Should I be giving the transaction-manager which I declared or will it automatically participate using acknowldege attribute. If yes then what is the difference in both of them?
<jee:jndi-lookup id="amountQueue" jndi-name="jms/amountQueue" />
<si-jms:channel id="amountDataChannel" queue="amountQueue" connection-
factory="queueConnectionFactory" transaction-manager="txManager" />
Also for production ready is there any other attribute which needs to be given ?
Appreciate your help on this
Please refer to the JMS section of the documentation for JMS-backed Channels.
I have the following requirement:
A message could come in on one of several message-driven-channel-adapter definitions, all obviously mapped to different incoming queues.
All the channel adapters then forward to the same internal Spring Integration channel where they are handled.
How can it be determined exactly which channel adapter - and therefore queue - the message was received on? For instance, is there a way in the channel adapter configuration to specify that a property be added to the message header at that point, which would be one solution.
Thanks
One way to do it is to write a ChannelInterceptor to add a header to the message. The preSend() method provides the Message and MessageChannel as arguments. MessageChannel may be cast to NamedComponent (An interface implemented be AbstracMessageChannel) to get the channel name.
You can use a header enrichier with a different identifier after each message-driven-channel-adapter.
For example :
<jms:message-driven-channel-adapter id="jmsIn1" destination="inQueue1" channel="in1"/>
<int:header-enricher input-channel="in1" output-channel="out1">
<int:header name="fromAdapter" value="1"/>
</int:header-enricher>
<jms:message-driven-channel-adapter id="jmsIn2" destination="inQueue2" channel="in1"/>
<int:header-enricher input-channel="in2" output-channel="out2">
<int:header name="fromAdapter" value="2"/>
</int:header-enricher>
I have several "int-http:inbound-gateway" and I need one of them to point at different services based on http method provided by the request.
<int-http:inbound-gateway path="....." supported-methods="POST,PUT"/>
At this moment I have 2 different endpoints and I was looking for some rest-method-based router but I have found nothing about this topic.
Any help?
you can use a header-value-router for this one as the http method is automatically set in the message header.
something like this
<int-http:inbound-channel-adapter channel="input.channel"
path="/log" supported-methods="PUT,POST" request-payload-type="java.lang.String"/>
<int:channel id="input.channel"/>
<int:header-value-router input-channel="input.channel" header-name="#{T(org.springframework.integration.http.HttpHeaders).REQUEST_METHOD">
<int:mapping value="PUT" channel="put.input.channel"/>
<int:mapping value="POST" channel="post.input.channel"/>
</int:header-value-router>
hope that helps