processing multiple http outbound gateways - spring-integration

I am working on the spring integration http. Wanted to make multiple http calls and collect the response to one common java object. I am facing an issues which says no output/reply channel at the end of the aggregator.
This is the xml definition
<int:channel id="intermediateWChannel">
<int:interceptors>
<int:wire-tap channel="intermediateWLogger" />
</int:interceptors>
</int:channel>
<int:channel id="intermediateSChannel">
<int:interceptors>
<int:wire-tap channel="intermediateSLogger" />
</int:interceptors>
</int:channel>
<int:logging-channel-adapter id="intermediateWLogger" expression="'Fetch Wtms Trip : '.concat(payload)" level="INFO" />
<int:logging-channel-adapter id="intermediateSLogger" level="INFO" />
<int:chain input-channel="intermediateSChannel" output-channel="publishsubscribechannel">
<int-http:outbound-gateway
id="outboundGateway"
url="{url2}"
http-method="GET"
request-factory="requestFactory"
charset="UTF-8"
mapped-request-headers="Accept:application/json"
expected-response-type="java.lang.String"></int-http:outbound-gateway>
<int:object-to-json-transformer/>
</int:chain>
<int:chain id="chain2" input-channel="publishsubscribechannel" output-channel="aggregatorChannel">
<int:transformer ref="fetchTransformer" method="process" />
</int:chain>
<int:chain id="request-chain" input-channel="publishsubscribechannel" output-channel="aggregatorChannel">
<int-http:outbound-gateway
id="strideOutboundGateway1"
url="url3"
http-method="GET"
request-factory="requestFactory"
charset="UTF-8"
mapped-request-headers="Accept:application/json"
expected-response-type="java.lang.String"></int-http:outbound-gateway>
<int:transformer ref="fetchTransformer" method="process1" />
</int:chain>
<int:chain id="chain3" input-channel="aggregatorChannel" output-channel="outputChannel">
<int:aggregator id="tAggregator"
ref="tDataAggregator"
method="processAggregator"
correlation-strategy-expression="headers['id']"
release-strategy="aggregatorReleaseStrategy"
expire-groups-upon-completion="true"/>
</int:chain>
But, I have specified the output channel in the aggregator.
Below is the exception and even if I add the reply-channel in headers, application is going for stackoverflow.
org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:452) ~[spring-integration-core-5.5.13.jar:5.5.13]
at org.springframework.integration.handler.AbstractMessageProducingHandler.doProduceOutput(AbstractMessageProducingHandler.java:325) ~[spring-integration-core-5.5.13.jar:5.5.13]
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:268) ~[spring-integration-core-5.5.13.jar:5.5.13]
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:232) ~[spring-integration-core-5.5.13.jar:5.5.13]

The transformer is different from other endpoints that it does not modify a Message we return from its method call. It is counted as transformer has a full control over the reply message creation. So, if there is a need to preserve some request message headers, it is recommended to use a MessageBuilder.copyHeadersIfAbsent() API. This way in request-reply scenario, the necessary replyChannel header is present in the reply message.
We probably have to mention shouldCopyRequestHeaders() behavior for the transformer in docs: https://docs.spring.io/spring-integration/docs/current/reference/html/message-transformation.html#messaging-transformation-chapter. Feel free to raise a GH issue.

Related

Inconsistency Dequeue issues with Spring Integration with Oracle AQ

I am using Spring integration with Oracle AQ,configuration code is as below.
Currently with the below configuration,the service activator is not getting invoked consistently with even the dequeue was successful at oracle end, unable to trace the message after successful Dequeue with the application logs, not even single error message shown on the log. Tried Debug,Trace and info options in the jms adaptor, but no clue from the log details. I have verified with Oracle team on enque and deque messages, but the health check reports are clearing mentioning the messages are dequeued successfully.
Badly required your help to get rid of this inconsitency behaviour while invoking the service activator using spring
<int:logging-channel-adapter id="jmslogger" log-full-message="true" level="TRACE"/>
<!-- Oracle Advanced Queue Integration -->
<bean id="jdbc4NativeJdbcExtractor"
class="org.springframework.jdbc.support.nativejdbc.Jdbc4NativeJdbcExtractor"
p:connectionType="oracle.jdbc.driver.OracleConnection" />
<orcl:aq-jms-connection-factory
id="oracleAqConnectionFactory"
use-local-data-source-transaction="true"
native-jdbc-extractor="jdbc4NativeJdbcExtractor"
data-source="dataSource"/>
<!-- Siebel Atlas Service Request - Oracle Advanced Queue Integration -->
<bean id="jmsJsonMessageConverter" class="org.springframework.jms.support.converter.MappingJackson2MessageConverter"
p:typeIdPropertyName="javaDtoClass"
/>
<int:channel id="submitSiebelAtlasCreateServiceRequestOutboundRequestChannel" ></int:channel>
<int:channel id="submitSiebelAtlasCreateServiceRequestOutboundRequestEnrichedChannel" >
<int:interceptors>
<int:wire-tap channel="jmslogger"/>
</int:interceptors>
</int:channel>
<int:channel id="createSiebelAtlasCreateServiceRequestOutboundReplyChannel" />
<int:logging-channel-adapter id="createSiebelAtlasCreateServiceRequestOutboundReplyChannelLogger"
channel="createSiebelAtlasCreateServiceRequestOutboundReplyChannel" />
<int:gateway
id="submitSiebelAtlasCreateServiceRequestMessagingService"
service-interface="ServiceRequestOutboundGatewayMessagingService"
default-request-channel="submitSiebelAtlasCreateServiceRequestOutboundRequestChannel"
default-reply-channel="submitSiebelAtlasCreateServiceRequestOutboundReplyChannel" />
<int:header-enricher input-channel="submitSiebelAtlasCreateServiceRequestOutboundRequestChannel" output-channel="submitSiebelAtlasCreateServiceRequestOutboundRequestEnrichedChannel">
<int:correlation-id expression="payload.getRequestId()"/>
</int:header-enricher>
<!-- Outbound driven channel adapter, meaning messagings are being sent to / queued in AQ -->
<int-jms:outbound-channel-adapter
id="siebelAtlasCreateServiceRequestJmsOutboundChannelAdapter"
destination-name="Q_NAME"
channel="submitSiebelAtlasCreateServiceRequestOutboundRequestEnrichedChannel"
connection-factory="oracleAqConnectionFactory"
message-converter="jmsJsonMessageConverter"
auto-startup="true">
</int-jms:outbound-channel-adapter>
<int:service-activator
output-channel="createSiebelAtlasCreateServiceRequestOutboundReplyChannel"
input-channel="createSiebelAtlasCreateServiceRequestInboundRequestChannel"
ref="createCustomerRelationshipsSiebelAtlasServiceRequestService"
method="create">
</int:service-activator>
<int:channel id="createSiebelAtlasCreateServiceRequestInboundRequestChannel">
<int:interceptors>
<int:wire-tap channel="jmslogger"/>
</int:interceptors>
</int:channel>
<!-- Inbound message driven channel adapter, meaning messagings are being consumed / dequeued from AQ -->
<int-jms:message-driven-channel-adapter connection-factory="oracleAqConnectionFactory"
message-converter="jmsJsonMessageConverter"
destination-name="Q_NAME"
channel="createSiebelAtlasCreateServiceRequestInboundRequestChannel"
acknowledge="transacted"
max-concurrent-consumers="5"
transaction-manager="transactionManager"
auto-startup="true"
concurrent-consumers="2" />

Spring Integration Log SOAP Message Content of int-ws:outbound-gateway

Could you tell me how to log the SOAP Message send/received of int-ws:outbound-gateway ?
I have tried
<int-ws:outbound-gateway id="ais-outbound-gateway"
request-channel="aisRequestChannel" reply-channel="aisResponseChannel"
uri="http://localhost:8080/services/ONESHOT"
marshaller="aisMarshaller" unmarshaller="aisMarshaller" />
<int:channel id="aisRequestChannel">
<int:interceptors>
<int:wire-tap channel="logChannel"/>
</int:interceptors>
</int:channel>
<int:channel id="aisResponseChannel">
<int:interceptors>
<int:wire-tap channel="logChannel"/>
</int:interceptors>
</int:channel>
<int:logging-channel-adapter expression="payload" id="logChannel" level="DEBUG" />
But the payload is just java object, not SOAP message.
Best regards,
The <int-ws:outbound-gateway> supports interceptor injection:
<xsd:documentation>
Reference to the bean definition of a ClientInterceptor.
</xsd:documentation>
So, you should implement your own LoggingClientInterceptor.
The logging ideas you can borrow from the existing PayloadLoggingInterceptor.

Spring Integration Task Executor Memory Leakage

My module is on spring integration which pushes message to RabbitMQ.
<task:executor id="bulkChannelExecutor" keep-alive="50" poolsize="50-100" queue-capacity="500"
></task:executor>
<int:channel id="logIngesterRestEndpointBulk" >
<int:dispatcher task-executor="bulkChannelExecutor" failover="false" />
</int:channel>
While Load testing ist not able to handle heavy loads(100 concurrent users) it causing message or request lost ,if i remove pool-size ,having unbounded poolsize its able to handling heavy loads but its creating memory thread leaks?
Rest Gateway will get input as Json and pass it to Filter and then will to
Chain ,there Json messages will get parsed and splitted as indivdiual messages and then will get push to rabbitMQ
<task:executor id="bulkChannelExecutor" keep-alive="50" pool-size="100-500"
queue-capacity="500"
></task:executor>
<int:channel id="logIngesterRestEndpointBulk" >
<int:dispatcher task-executor="bulkChannelExecutor" failover="false" />
</int:channel>
<int-http:inbound-gateway id="logIngesterGatewayBulk" auto-startup="true"
supported-methods="POST" request-channel="logIngesterRestEndpointBulk"
path="/rest/log/bulk" error-channel="errorChannel" reply-timeout="50"
request-payload-type="java.lang.String">
</int-http:inbound-gateway>
<int:channel id="filterChannelbulk">
</int:channel>
<int:channel id="messageOutputChannel" >
</int:channel>
<int:filter input-channel="logIngesterRestEndpointBulk"
throw-exception-on-rejection="true" method="validate" ref="payloadValidation"
output-channel="filterChannelbulk">
</int:filter>
<int:chain input-channel="filterChannelbulk" output-channel="messageOutputChannel" id="chaining" >
<int:splitter id="splitter" ref="payloadSplitter" method="splitPayLoad" >
</int:splitter>
<int:transformer id="logMessageTransformerbulk" ref="logMessageHeaderTransformer"
method="transform">
</int:transformer>
</int:chain>
<int:service-activator input-channel="errorChannel"
ref="responseHandler" method="handleFailedPayLoad" >
</int:service-activator>
<!-- Start RabbitMQ Configuration -->
<int:channel id="ackchannel">
</int:channel>
<int-amqp:outbound-channel-adapter
id="amqpAdapter" channel="messageOutputChannel" amqp-template="amqpTemplate" lazy-connect="false" confirm-ack-channel="ackchannel" confirm-correlation-expression="headers['amqp_publishConfirm']"
exchange-name="dhp_exchange" routing-key-expression="headers['routingKey']" >
</int-amqp:outbound-channel-adapter>
<int:service-activator id="ackservice" input-channel="ackchannel" ref="responseHandler" method="confirmAck" />
Since you are using HTTP, you should remove the task executor and allow the web container to manage the threads. If you need more threads, do it through the web container configuration; don't use a thread handoff here; it really serves no purpose and can cause the issues you describe.

Spring Integration is not triggering int-http:outbound-gateway URL

I have the following spring integration configuration which starts from reading a file from drive, transforming it into java object and sending http GET request to a REST APT. The first 2 steps are working fine in my chain, but when it comes to int-http:outbound-gateway step, the URL is never triggered nor it is displaying any error message. The application remains running state without showing any error message. It never goes to kbbCvsReadResponseTransformer class where I can check the response received from this REST service. What could be the reason. I am using spring-integration-4.1.2
<int-file:inbound-channel-adapter id="kbbFileInbound"
channel="kbbInboundFileChannel"
directory="file:/ftpguest/kbb-gm-rem/data"
filename-pattern="GM_Remarketing_Pricing_Res_*.csv"
auto-startup="true"
prevent-duplicates="true">
<int:poller fixed-rate="5000"/>
</int-file:inbound-channel-adapter>
<int:chain input-channel="kbbInboundFileChannel" output-channel="kbbCvsReadRequest">
<int:transformer ref="kbbInputFileDataTransformer" />
</int:chain>
<int:chain input-channel="kbbCvsReadRequest" output-channel="cvsVehicleReadRequest">
<int:transformer ref="kbbCvsUpdateRequestTransformer" />
</int:chain>
<int:chain input-channel="cvsVehicleReadRequest" output-channel="cvsVehicleReadResponse">
<int:header-enricher>
<int:header name="Content-Type" expression="'application/json'" />
</int:header-enricher>
<int-http:outbound-gateway http-method="GET" expected-response-type="com.fasterxml.jackson.databind.JsonNode"
charset="UTF-8" request-factory="clientHttpRequestFactory" url="http://services.dev-sea.cobaltgroup.com/inventoryWebApp/rest/v1.0/vehicles/search?vin={vin};inventoryOwner={inventoryOwner}">
<int-http:uri-variable name="vin" expression="payload.getVin()"/>
<int-http:uri-variable name="inventoryOwner" expression="payload.getInventoryOwner()"/>
<int-http:request-handler-advice-chain>
<ref bean="retrier" />
</int-http:request-handler-advice-chain>
</int-http:outbound-gateway>
</int:chain>
<int:chain input-channel="cvsVehicleReadResponse">
<int:transformer ref="kbbCvsReadResponseTransformer" />
</int:chain>
<bean id="clientHttpRequestFactory"
class="org.springframework.http.client.HttpComponentsClientHttpRequestFactory">
<property name="httpClient" value="#{httpComponentsMessageSender.httpClient}" />
</bean>
<bean id="kbbInputFileDataTransformer" class="com.cdk.dmg.kbb.integration.transformer.KbbInputFileDataTransformer"/>
<bean id="kbbCvsUpdateRequestTransformer" class="com.cdk.dmg.kbb.integration.transformer.KbbCVSUpdateRequestTransformer" />
<bean id="kbbCvsReadResponseTransformer" class="com.cdk.dmg.kbb.integration.transformer.KbbCVSReadResponseTransformer" />
<!-- Need to use it from integration-context.xml, there are lot of issues with this file right now, so temporerly copied it here -->
<int:handler-retry-advice id="retrier" max-attempts="5" recovery-channel="errorChannel">
<int:exponential-back-off initial="1000" multiplier="5.0" maximum="60000" />
</int:handler-retry-advice>
We can't help you here without any logs confirmation.
Please, share DEBUG logs for the org.springframework.integration to have a picture how your messages are traveling.
From other side looks like you have some error there and just send it to the errorChannel according to the <int:handler-retry-advice> configuration.
And that's why you don't get any reply to the cvsVehicleReadResponse after that...

int-http:inbound-gateway same request Channel and different Response channel issue

i have two int-http:inbound-gateway with path as mentioned below.And points to same request channel but have different reply-channel.
http://localhost:8080/XYZ/ABCService/query -- i expected to call http:inbound-gateway with id ="XYZ"
http://localhost:8080/ABCService/query - i expected to call http:inbound-gateway with id ="default"
but what happing in its not consistence when i give request to
http://localhost:8080/XYZ/ABCService/query
it is calling "default" gateway other time its calling "XYZ" i.e not consistence. or not sure it may call correctly but instead gives response to different reply-channel ?
I am using DispatcherServlet.Below my spring-integration.xml
<int-http:inbound-gateway id="default"
path="/*Service/query"
request-channel="RequestChannel" reply-channel="ResponseChannel"
supported-methods="POST" reply-timeout="5000" request-payload-type="java.lang.String"
error-channel="ErrorChannel" mapped-request-headers="xyz-*, HTTP_REQUEST_HEADERS">
<int-http:header name="reply-type" expression="'DEFAULT'" />
</int-http:inbound-gateway>
<int-http:inbound-gateway id="XYZ"
path="/XYZ/*Service/query"
request-channel="RequestChannel" reply-channel="XYZExportTransformedChannel"
supported-methods="POST" reply-timeout="5000" request-payload-type="java.lang.String"
error-channel="ErrorChannel" mapped-request-headers="xyz-*, HTTP_REQUEST_HEADERS">
<int-http:header name="reply-type" expression="'ABC'" />
</int-http:inbound-gateway>
<!--All endpoints output chanlle is CommonResonseChannel -->
<int:channel id="CommonResponseChannel">
</int:channel>
<!-- final router -->
<int:header-value-router input-channel="CommonResponseChannel"
header-name="reply-type">
<int:mapping value="DEFAULT" channel="ResponseChannel" />
<int:mapping value="ABC" channel="XYZResponseChannel" />
</int:header-value-router>
<int:channel id="ResponseChannel">
</int:channel>
<int:channel id="XYZResponseChannel">
</int:channel>
<int:transformer input-channel="XYZResponseChannel"
output-channel="XYZExportTransformedChannel" id="TransformerChannel"
ref="objToCSVTransformer"></int:transformer>
<bean class="SomeTransformer"
id="objToCSVTransformer"></bean>
<int:channel id="XYZExportTransformedChannel" />
I have opened this question before not not very clear.Not sure how to update that.So opened new one.
You should not configure your reply channels that way; it may cause unpredictable results.
reply-channels are simply bridged to the message replyChannel header so it generally will work ok, but it's unpredictable because the bridge is set up when the gateway first receives a message.
Instead, simply omit the reply-channel attributes on the gateways and let the framework route the replies directly using the replyChannel header.
Instead of your router, configure a "bridge to nowhere" (a <bridge/> with input-channel= CommonResponseChannel" and no output-channel.
Or simply omit the output-channel on the last endpoints (instead of sending to CommonResponseChannel).

Resources