I have a scenario in which i am receiving a message and routing it to different message channels using Recipient-list-router.
<int:recipient-list-router input-channel="inputChannel1"
apply-sequence="true" id="router1">
<int:recipient channel="service1" />
<int:recipient channel="service2" />
</int:recipient-list-router>
This aggregator is used to aggregate the response from above 2 services and redirects the aggregated output to some other channel for further processing.
<int:aggregator input-channel="aggregator1" output-channel="channel2" >
<bean class="wsintg.service.AggregatorClass" />
</int:aggregator>
The code for individual services looks like this :
<!--service 1 code -->
<int:service-activator input-channel="service1" ref="service1Ref" />
<int:gateway id="service1Ref" default-request-channel="success_channel" error-channel="error_channel" />
<int:chain id="chainid1" input-channel="success_channel" output-channel="aggregator1">
step 1 :xsl transformation
step 2 :call to a service using ws:outbound-gateway
<int:service-activator id="sa1">
<bean class="samplePkg.returnMessage"></bean> <!--return message simply returns the message without doing anything -->
</int:service-activator>
</int:chain>
<int:chain id="chainid2" input-channel="error_channel" output-channel="aggregator1" >
<int:service-activator>
<!-- some custom exception handling -->
</int:service-activator>
</int:chain>
<!--service 2 code --> same as service 1
Problem :
If i don't use gateway in Service 1 and Service 2 and simply do the transformations, the code works perfectly.
But, as i need to do some error handling using error_channel, so i put a gateway in service1 and service 2 code. Now the flow
stucks after service activator in service 1 without throwing any
exception. and does not calls the service 2 flow.
You need to show real configuration, not just comments; we can't guess what your application looks like.
That said, you can't just send error messages to the aggregator because the aggregation information is in the failedMessage headers and needs to be promoted.
See error handling in this sample for an example.
As always, the first step to debug situations like this is to turn on DEBUG logging for org.springframework.integration and follow the messages through the flow.
You will usually be able to determine the problem that way.
Related
I have a flow that
1. Starts with a config map -> MainGateway.start(configMap) -> void
2. Splits map into multiple messages per entry
3. For every config entry do the following using an orchestrator java class:
BEGIN LOOP (offset and limit)
Data d = HTTPGateway.getData();
PublishGateway.sendMessage(d); -> Send to 2 SQS queues
END LOOP
Requirement
I have to schedule this flow via cron. One option is to provide an HTTP endpoint that will start the flow. But then the second HTTP request should wait/timeout/error until the first is complete.
Question
I was looking at barrier to implement blocking for the flow thread until it completes and have only a single thread http processor, so at one time only 1 request is processed and I can know when the flow is complete. (The LOOP ends for all config entry objects and all messages to SQS are acked). How can I achieve this? I have a loop and am using pub-sub channel with executors for parallel configs and parallel SQS dispatch.
I have trimmed down the XML config below for clarity.
<!-- Bring in list of Configs to process -->
<int:gateway service-interface="Gateway"
default-request-channel="configListChannel" />
<int:chain input-channel="configListChannel" output-channel="configChannel">
<!-- Split the list to one instance of config per message -->
<int:splitter/>
<int:filter expression="payload.enablePolling" />
</int:chain>
<!-- Manually orchestrate a loop to query a system as per config and publish messages to SQS -->
<bean class="Orchestrator" id="orchestrator" />
<int:service-activator ref="orchestrator" method="getData" input-channel="configChannel" />
<!-- The flow from this point onwards is triggered inside a loop controlled by the Orchestrator
The following Gateway calls are inside Orchestrators loop -->
<!-- Create a Http request from the Orchestrator using a Gateway -->
<int:gateway service-interface="HttpGateway">
<int:method name="getData"
request-channel="requestChannel"
payload-expression="#args[0]">
</int:method>
</int:gateway>
<!-- Transform request object to json and invoke Http endpoint -->
<int:chain input-channel="requestChannel" id="httpRequestChain">
<int:object-to-json-transformer />
<int-http:outbound-gateway url-expression="headers['config'].url"
http-method="POST"
expected-response-type="java.lang.String"
/>
</int:chain>
<!-- Publish Messages to Outbound Gateway -->
<task:executor id="executor" pool-size="5" />
<int:publish-subscribe-channel id="publishChannel" task-executor="executor" />
<int:gateway service-interface="PublishGateway" >
<int:method name="publishToOutbound" payload-expression="#args[0]" request-channel="publishChannel" />
</int:gateway>
<!-- Route to System A SQS with transformations (omitted here)-->
<int-aws:sqs-outbound-channel-adapter sqs="amazonSQS" channel="publishChannel" queue="system-a-queue" success-channel="successChannel" failure-channel="errorChannel"/>
<!-- Route to System B SQS with transformations (omitted here)-->
<int-aws:sqs-outbound-channel-adapter sqs="amazonSQS" channel="publishChannel" queue="system-b-queue" success-channel="successChannel" failure-channel="errorChannel"/>
<int:logging-channel-adapter logger-name="sqsCallbackLogger" log-full-message="true" channel="successChannel" />
In the meanwhile, I am trying to adapt the A B C barrier example from spring-integration-samples to my use case.
As you pointed out in your comment, an aggregator approach could be used in your solution.
This way you aggregated results of those parallel SQS requests and wait for an aggregation reply in the original requestor. This way it is really going to be blocked even if internals of your flow still concurrent. You call a gateway and the reply for it is going to be from the aggregator.
Currently we are using Spring Integration 2.1.0 Release in our application.
Application flow is as below:
Some operation is performed in application and we got the output string in String via Active MQ.
I have used message-driven-channel-adapter and service-activator to read the data from queue.
That data is displayed successfully on Server(application is working as client) using tcp-outbound-gateway.
Problem is while getting the acknowledgement from server.
Created a new channel and entered in reply-channel in tcp-outbound-gateway
Passing the same channel in service-activator as input channel.
It is showing below error:
[task-scheduler-5] 2017-10-05 18:32:20,732 ERROR org.springframework.integration.handler.LoggingHandler - org.springframework.integration.MessageDeliveryException: Dispatcher has no subscribers.
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:108)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:101)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:61)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:157)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:128)
at org.springframework.integration.core.MessagingTemplate.doSend(MessagingTemplate.java:288)
at org.springframework.integration.core.MessagingTemplate.send(MessagingTemplate.java:149)
Code is as below
<context:property-placeholder />
<!-- Gateway and connection factory setting -->
<int:channel id="telnetLandingChannel" />
<int:channel id="telnetReplyChannel" />
<beans:bean id="clientCustomSerializer"
class="com.telnet.core.serializer.CustomSerializer">
<beans:property name="terminatingChar" value="10" />
<beans:property name="maxLength" value="65535" />
</beans:bean>
<int:gateway id="gw" default-reply-channel="telnetReplyChannel" default-reply-timeout="100000"
service-interface="com.telnet.core.integration.connection.ParseTcpConfiguration$SimpleGateway"
default-request-channel="telnetLandingChannel"/>
<ip:tcp-connection-factory id="clientFactory"
type="client" host="localhost" port="7777" single-use="false" using-nio="false"
serializer="${client.serializer}" deserializer="${client.serializer}" />
<ip:tcp-outbound-gateway id="clientInGw"
request-channel="telnetLandingChannel"
connection-factory="clientFactory"
reply-channel="telnetReplyChannel"
reply-timeout="100000"/>
<!-- To send the messege over server via JMS and serviceActivator -->
<int:channel id="incidentDispatchMessageChannel" />
<int:channel id="jmsChannel" />
<beans:bean id="customClientServiceActivator"
class= "com.telnet.core.integration.CustomClientServiceActivator">
</beans:bean>
<int-jms:message-driven-channel-adapter id="incidentDispatchMessageChannelAdapter" error-channel="errorChannel"
connection-factory="mqConnectionFactory"
destination-name="${incident.processing.messaging.dispatch.queues}"
channel="incidentDispatchMessageChannel"/>
<int:service-activator id="incidentMessageActivator"
input-channel="incidentDispatchMessageChannel"
output-channel="jmsChannel"
ref="customClientServiceActivator" method="getOutboundMessage">
</int:service-activator>
<int:object-to-string-transformer id="clientBytes2String"
input-channel="jmsChannel"
output-channel="telnetLandingChannel"/>
<!-- To receive the acknowledgement message on server via serviceActivator -->
<int:service-activator id="incidentAck"
input-channel="telnetReplyChannel"
ref="customClientServiceActivator" method="getAck">
</int:service-activator>
I have studied various article on stackverFlow but not able to get any solution
Yeah... That isn't clear by the error what channel is guilty.
On the other hand you really use very old Spring Integration version.
Would be great to consider to upgrade to the latest: http://projects.spring.io/spring-integration/.
However I think that issue is somehow around exactly that reply-channel, which you use not only for the <service-activator> but for the <int:gateway> as well.
I suggest you to remove default-reply-channel="telnetReplyChannel" from the gateway definition, remove reply-channel="telnetReplyChannel" from the <ip:tcp-outbound-gateway> definition. And let them communicate via replyChannel header populated by the gateway during request.
Regarding your <int-jms:message-driven-channel-adapter> flow which leads to the same <ip:tcp-outbound-gateway>, I would suggest to still stay with the replyChannel header but here populate it via <header-enricher> before sending message to the telnetLandingChannel. That replyChannel via <header-enricher> would be exactly an input-channel for the subsequent <int:service-activator> to handle ack from the <ip:tcp-outbound-gateway>.
I got the solution of this issue, there are multiple xmls in our code but i have added the code in one to show flow in stackOverflow.
Issue was i had defined in a xml which has only configuration part like Outbound adapter connection factory where as it should be defined in another xml where I am using service activator. Changed the place of channel definition and it worked.
I want to disconnect the TCP (as server) the moment i got the response message. As of now I am using so-timeout, so my TCP server will gets timedout after the time given in so-timeout, but requirement is to disconnect the connection the moment TCP print/display the acknowledgement. Please suggest how can I implement this.
I have the following workflow.
inbound-channel
splitter
task executor for split channels - all the threads execute the same workflow.
3.a. construct the request
3.b. service activator wrapper for a gateway message endpoint.
3.c. gateway wrapper over the http-outbound-gateway with the configuration of error-channel (to handle exceptions while invoking http-outbound-gateway)
3.d. http-outbound-gateway
aggregator
response out of spring integration workflow.
If an exception occurs in 3.d, the control goes to the service-activator configured for the gateway error channel.
I copy just the following from the failed message to the new header to the header passed to the error channel.
a. correlationId
b. sequenceNumber
c. sequenceSize
But while aggregating the splitter response, the DefaultAggregatingMessageGroupProcessor.java removes the conflicting headers and by that it removes the error-channel and reply-channel before providing the control to aggregator.
So once the aggregator completes it's operation it is unable to find the reply or error channel and it results in an exception.
I'm using spring-integration-core version 2.2.1 and I'm not sure why the reply-channel and error-channel is being removed during header aggregation.
Any input on resolving this issue will be of great help.
Thank You :)
EDIT 1:
Thank You very much Gary for helping me out with this scenario. I'm sharing my current configuration
<!-- SPLITTER -->
<int:splitter id="dentalSplitter" ref="dentalServiceSplitter"
method="getDentalServiceList" input-channel="dentalServiceSplitterChannel"
output-channel="dentalSplitterTaskChannel" />
<int:channel id="dentalSplitterTaskChannel">
<int:dispatcher task-executor="dentalTaskExecutor" />
</int:channel>
<int:chain input-channel="dentalSplitterTaskChannel" output-channel="dentalGatewayChannel">
<int:header-enricher>
<int:header name="CHAIN_START_TIME" expression="T(System).currentTimeMillis()" overwrite="true" />
<int:object-to-json-transformer content-type="application/json"/>
</int:chain>
<int:service-activator input-channel="dentalGatewayChannel" ref="dentalGatewayWrapper" output-channel="dentalReplyChannel" />
<int:gateway id="dentalGatewayWrapper" default-request-channel="dentalCostEstimateRequestChannel" error-channel="dentalErrorChannel"/>
<int-http:outbound-gateway id="dentalGateway"
url-expression="#urlBuilder.build('${service.endpoint}')"
http-method="POST" request-factory="clientHttpRequestFactory"
request-channel="dentalCostEstimateRequestChannel" extract-request-payload="true"
expected-response-type="com.dental.test.DentalResponse">
<int-http:request-handler-advice-chain>
<ref bean="logChainTimeInterceptor" />
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
<!-- EXCEPTION -->
<int:chain input-channel="dentalErrorChannel" output-channel="dentalAggregatorChannel">
<int:transformer ref="commonErrorTransformer" method="dentalGracefulReturn"/>
</int:chain>
<!-- SUCCESS -->
<int:chain input-channel="dentalReplyChannel" output-channel="dentalAggregatorChannel">
<int:filter discard-channel="dentalErrorChannel"
expression="T(com.dental.util.InvocationOutcomeHelper).isOutcomeSuccess(payload?.metadata?.outcome?.code,payload?.metadata?.outcome?.message)" />
</int:chain>
<!-- AGGREGATION -->
<int:chain input-channel="dentalAggregatorChannel" output-channel="wsDentalServiceOutputChannel" >
<int:aggregator ref="dentalServiceAggregator" />
<int:service-activator ref="dentalResponseServiceActivator" />
</int:chain>
What I noticed was this, every split channel when passing through the gateway creates a new temporary channel for error and reply and after getting the response back from the gateway, it retains the preserved (original inbound) error and reply channel header. And as you had mentioned, after the control gets to the error transformer that flow of retaining the preserved headers gets broken and the aggregating message group processor receives three different instances of temporary channel and hence removes them.
I was planning to have a custom message group processor and modify the conflict resolution strategy for aggregating the header and came up with this config.
<bean id="channelPreservingAggregatingMessageHandler" class="org.springframework.integration.aggregator.AggregatingMessageHandler">
<constructor-arg name="processor" ref="channelPreservingMessageGroupProcessor"/>
</bean>
I'm yet to test this out though. But based on this discussion, this does not look like a viable solution.
And looks like my configuration for error handling in gateway is incorrect.
However, I'm confused on this statement of yours "Instead of forwarding the message directly, simply handle the error on your error flow and return the result normally to the gateway "wrapper"". If I remove the error channel how will I get the control back when an exception occurs? May be I'm missing to understand something here. Can you elaborate more on this please?
When asking questions about scenarios such as this, you generally need to show your configuration. However, I suspect you are forwarding the message from the error flow directly to the aggregator.
This is like doing a GOTO in code and breaks the scoping.
It won't work because the replyChannel header in the error message is for the gateway "wrapper", not the original upstream inbound gateway. When the aggregator gets conflicting headers, it has no choice but to drop the headers (you will see a DEBUG log message to that effect).
Instead of forwarding the message directly, simply handle the error on your error flow and return the result normally to the gateway "wrapper" (simply omit the error channel on the last element on the error flow).
The gateway will then fix up the reply so it is consistent with other messages (good and bad) and forward it to the aggregator.
You don't need to mess with headers in your error flow, just return the value you want to be aggregated along with the good results.
You should really update to a current release, or at least the latest in the 2.2.x line (2.2.6).
I have an integration application that mostly works, but noticed yesterday a message was lost. At the time, the service-activator endpoint was extremely busy processing a previous message.
Basically, my integration flow is:
jdbc:inbound-channel-adapter --> splitter -> aggregator (based on a type field) -> [pollable channel] --> service-activator
From the log, I can see a group of messages polled, split, aggregated, expired (via a message store reaper), and placed on the creation channel. There is a preSend and postSend event in the log for placing the message on the creationChannel, but that's the end of it. It never gets to my service-activator, there's no messages in the jdbc message store and I dont see any errors.
Could there an issue with how I have either the service-activator or poller configured (or a combination)?
Here is the configuration applicable to the issue.
<int:poller default="true" fixed-delay="1000" receive-timeout="0" />
<int-jdbc:message-store id="jdbc-messageStore" data-source="dataSource" />
<int:channel id="creationChannel" >
<int:queue message-store="jdbc-messageStore" />
</int:channel>
<!-- Endpoint responsible for creating Archive from List of Document objects -->
<bean id="archiveCreator" class="org.abc.ArchiveCreator"/>
<int:service-activator ref="archiveCreator" method="createArchive" input-channel="creationChannel" />
First of all you should mark your <poller> as <transactional>.
Another point switch on the org.springframework.integration DEBUG logging level and show here the log, when a message is lost.
Hi this is my spring integration configuration..When i hit my service using mozilla fire fox rest console two times its getting called.For example if i hit some service in service-activator (CA request activator),On certaion un predicatable scenarios it s getting called twice.I dont know whetther it is mozilla issue or configuration issue.I tried using new window but this problem persists.In first case my service activator returning xml response properly but immediately it s getting called again.Only on rare scenarios its calling twice.
<int:channel id="accountRequest" />
<int:channel id="accountResponse" />
<int:channel id="catRequestChannel" />
<int:channel id="mataccountRequest" />
<int:channel id="errorChannel"/>
<int-http:inbound-gateway id="cwebAccountManagementGateway"
supported-methods="GET, POST"
request-channel="accountRequest"
reply-channel="accountResponse"
mapped-request-headers="*"
mapped-response-headers="*"
view-name="/policies"
path="/services/{class}/{method}"
reply-timeout="50000"
error-channel="errorChannel">
<int-http:header name="serviceClass" expression="#pathVariables.class"/>
<int-http:header name="serviceMethod" expression="#pathVariables.method"/>
</int-http:inbound-gateway>
<int:header-value-router input-channel="accountRequest"
header-name="state"
default-output-channel="accountRequest" resolution-required="false">
<int:mapping value="MA"
channel="mataccountRequest" />
<int:mapping value="CA"
channel="catRequestChannel" />
</int:header-value-router>
<int:service-activator id="accountServiceActivator"
input-channel="mataccountRequest"
output-channel="accountResponse"
ref="serviceGatewayAdapter"
method="requestHandler"
send-timeout="60000"/>
<int:service-activator id="caRequestActivator"
input-channel="catRequestChannel"
output-channel="accountResponse"
ref="caServiceGatewayAdapter"
method="requestHandler"
send-timeout="60000"/>
<int:service-activator id="errorRequestActivator"
input-channel="errorChannel"
output-channel="accountResponse"
ref="errorGatewayAdapter"
method="errorHandler"
send-timeout="60000"/>
for eg:This is my url
http://localhost:9085/springintegrationsample/create?mail=15999999#mail.com&idNumber=80010600010
if i edit the mail to some other values it ll get called twice
if i change agan it s working fine.I dont understand on what scenarios its getting called twice
You are using the same "requestHandler" for both of your service activators, so its obvious it will be called twice.
You can't know which activator is called for the handler method as when you debug you can only check for the handler method being called.
To better handle this, use different a handler method for each of your service activators. Even if they are using similar operations, it will be more clear and easy to debug.
I would suggest to add channel interceptors for logging of incoming and outgoing messages just to get a better idea.
cheers