I have written code to read message from IBM MQ using Spring Integration JMS-message-driven-channel-adapter but not able to read message from queue can anybody help me out below is my configuration.
<beans:bean id="ibmJmsConnectionFactory" class="com.ibm.mq.jms.MQQueueConnectionFactory">
<beans:property name="transportType" value="1"/>
<beans:property name="queueManager" value="***"/>
<beans:property name="hostName" value="**"/>
<beans:property name="port" value="**" />
<beans:property name="channel" value="***"/>
</beans:bean>
<beans:bean id="receiverQueue" class="com.ibm.mq.jms.MQQueue">
<beans:constructor-arg index="0" value="****"/>
<beans:constructor-arg index="1" value="****"/>
</beans:bean>
<integration:channel id="componentInfoChannel" />
<jms:message-driven-channel-adapter
id="componentInfoAdapter" connection-factory="ibmJmsConnectionFactory" destination="receiverQueue" channel="componentInfoChannel"
/>
<integration:service-activator id="componentInfoActivator"
input-channel="componentInfoChannel" ref="componentInfoConsumer"
method="componentInfoListen" />
Above configuration is working but failing when I added logging adapter its continously printing warning on console .Detail are mention below .
<integration:logging-channel-adapter
id="componentInfologger" level="INFO" />
<integration:wire-tap id="componentInfoWireTap"
channel="componentInfologger" pattern="*" order="2" />
<integration:logging-channel-adapter
id="logger" log-full-message="true" level="INFO" />
2017-02-23 00:09:05.093 WARN [componentdatafiles,48f69e84ab395754,ebf4ae7f3b812a01,false] 90072 --- [ter.container-1] o.s.j.l.DefaultMessageListenerContainer : Execution of JMS message listener failed, and no ErrorHandler has been set.
java.lang.StackOverflowError: null
at java.util.Collections$SynchronizedMap.put(Unknown Source)
at ch.qos.logback.classic.util.LogbackMDCAdapter.put(LogbackMDCAdapter.java:110)
at org.slf4j.MDC.put(MDC.java:147)
at org.springframework.cloud.sleuth.log.Slf4jSpanLogger.logStartedSpan(Slf4jSpanLogger.java:48)
at org.springframework.cloud.sleuth.trace.DefaultTracer.createChild(DefaultTracer.java:170)
at org.springframework.cloud.sleuth.trace.DefaultTracer.createSpan(DefaultTracer.java:72)
at org.springframework.cloud.sleuth.instrument.messaging.TraceChannelInterceptor.startSpan(TraceChannelInterceptor.java:98)
at org.springframework.cloud.sleuth.instrument.messaging.TraceChannelInterceptor.preSend(TraceChannelInterceptor.java:78)
at org.springframework.integration.channel.AbstractMessageChannel$ChannelInterceptorList.preSend(AbstractMessageChannel.java:538)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:415)
at org.springframework.integration.channel.interceptor.WireTap.preSend(WireTap.java:168)
at org.springframework.integration.channel.AbstractMessageChannel$ChannelInterceptorList.preSend(AbstractMessageChannel.java:538)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:415)
at org.springframework.integration.channel.interceptor.WireTap.preSend(WireTap.java:168)
at org.springframework.integration.channel.AbstractMessageChannel$ChannelInterceptorList.preSend(AbstractMessageChannel.java:538)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:415)
at org.springframework.integration.channel.interceptor.WireTap.preSend(WireTap.java:168)
at org.springframework.integration.channel.AbstractMessageChannel$ChannelInterceptorList.preSend(AbstractMessageChannel.java:538)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:415)
at org.springframework.integration.channel.interceptor.WireTap.preSend(WireTap.java:168)
Sorry for delay. So, I guess your id="logger" is redundant component. And the problem is exactly with the <integration:wire-tap> and combination with the TraceChannelInterceptor.
I mean that you use here Spring Cloud Sleuth.
Would you mind sharing DEBUG for the org.springframework.integration category meanwhile I'm trying to reproduce an issue locally.
Plus you can exclude componentInfologger from the tracing to avoid that java.lang.StackOverflowError.
I think it happens somehow that we loop tracing via wire-tap.
Related
I'm consuming messages with spring-integration-kafka, using a message-driven-channel-adapter:
<int-kafka:message-driven-channel-adapter
id="kafkaListener"
listener-container="container1"
channel="outputFromKafka"
error-channel="errorChannel"/>
The container uses a JsonDeserializer to deserialize the incoming JSON to an object:
<beans:bean id="container1" class="org.springframework.kafka.listener.KafkaMessageListenerContainer">
<beans:constructor-arg>
<beans:bean class="org.springframework.kafka.core.DefaultKafkaConsumerFactory">
<beans:constructor-arg>
<beans:map>
<beans:entry key="bootstrap.servers" value="localhost:9092" />
<beans:entry key="key.deserializer" value="org.apache.kafka.common.serialization.StringDeserializer" />
<beans:entry key="group.id" value="mygroup" />
</beans:map>
</beans:constructor-arg>
<beans:property name="valueDeserializer">
<beans:bean class="org.springframework.kafka.support.serializer.JsonDeserializer">
<beans:constructor-arg value="com.foo.MyType"/>
</beans:bean>
</beans:property>
</beans:bean>
</beans:constructor-arg>
<beans:constructor-arg>
<beans:bean class="org.springframework.kafka.listener.config.ContainerProperties">
<beans:constructor-arg name="topics" value="foo" />
</beans:bean>
</beans:constructor-arg>
</beans:bean>
If the message can't be parsed successfully (e.g. because the consumer accidentally uses the wrong type), an exception is thrown:
ERROR org.apache.kafka.clients.NetworkClient - Uncaught error in request completion: org.apache.kafka.common.errors.SerializationException: Can't deserialize data ...
After this, the adapter is receiving the same message again (probably because the last one wasn't committed?), and fails in exactly the same way, resulting in an endless stream of exceptions.
It looks like the configured error-channel is not used.
What are the options to handle errors like this, and how is it configured in XML?
It looks like the configured error-channel is not used.
What makes you believe that?
I just ran a test with the error channel set to errorChannel (the default, with a logging adapter)...
12:06:08.366 [container-kafka-consumer-1] ERROR o.s.i.handler.LoggingHandler - ...
Can you provide a debug log snippet for org.springframework.integration showing a failure?
EDIT
Oh, sorry...
ERROR org.apache.kafka.clients.NetworkClient - Uncaught error in request completion: org.apache.kafka.common.errors.SerializationException: Can't deserialize data ...
That's much lower down in the stack before Spring Integration gets the message.
I have a message driven channel adapter configured to pick messages from queue with acknowledge set to transacted. Is it possible to send different messages to different queues down stream holding the same transaction also have database inserts between in the flow?. If message delivery fails to any queue , the transaction must roll back (including database entries) as well the message send to other queues.
Example : queue(receive)--->insert Db-->send to(queue1,queue2.. etc sending different message to each queue)
if any send call fails for queue1, queue2 etc the transaction should roll back..
I am able to do the configuration with single queue (ie only with queue1). But how to do if multiple queues are involved and holding the transaction boundaries.
Thanks
Vaidya
Below is the configuration
<int-jms:message-driven-channel-adapter
id="MessageDrivenAdapter" channel="injmsChannel" destination="testQ"
concurrent-consumers="5" max-concurrent-consumers="10" acknowledge="transacted"
error-channel="Error" />
<int:channel id="injmsChannel" />
<int:chain input-channel="injmsChannel" id="Chain1">
<int-jpa:retrieving-outbound-gateway
entity-class="entity.TestTable8"
entity-manager-factory="entityManagerFactory" id-expression="payload.getSno()">
<int-jpa:transactional transaction-manager="transactionManager" />
</int-jpa:retrieving-outbound-gateway>
<int:recipient-list-router id="ROUTE_1_2">
<int:recipient channel="SuccessChannel"
selector-expression="payload.getSno()==1" />
/> -->
</int:recipient-list-router>
</int:chain>
<int:channel id="SuccessChannel" />
<int:chain id="MessageProcessingChain" input-channel="SuccessChannel"
output-channel="putMsgChannel">
<int:service-activator id="a1" ref="taskexe"
method="processTable8_1" requires-reply="true" />
<int-jpa:retrieving-outbound-gateway
id="table7" entity-class="entity.TestTable7"
entity-manager-factory="entityManagerFactory" id-expression="payload.getSno()">
<int-jpa:transactional transaction-manager="transactionManager" />
</int-jpa:retrieving-outbound-gateway>
<int:service-activator id="a2" ref="taskexe"
method="processTable8_2" requires-reply="true" />
<int-jpa:updating-outbound-gateway
id="table6" entity-class="entity.TestTable6"
entity-manager-factory="entityManagerFactory" flush="true">
<int-jpa:transactional transaction-manager="transactionManager" />
</int-jpa:updating-outbound-gateway>
<int:service-activator id="a3" ref="taskexe"
method="processTable6_1" requires-reply="true" />
<int-jpa:updating-outbound-gateway
id="uptable6" entity-class="entity.TestTable6"
entity-manager-factory="entityManagerFactory" flush="true">
<int-jpa:transactional transaction-manager="transactionManager" />
</int-jpa:updating-outbound-gateway>
<int:service-activator id="a4" ref="taskexe"
method="processTable6_2" requires-reply="true" />
<int-jpa:updating-outbound-gateway
id="uptable4" entity-class="entity.TestTable4"
entity-manager-factory="entityManagerFactory" flush="true">
<int-jpa:transactional transaction-manager="transactionManager" />
</int-jpa:updating-outbound-gateway>
<int:service-activator ref="taskexe" method="processTable4_1"
requires-reply="true" />
</int:chain>
<int:channel id="putMsgChannel" />
<int-jms:outbound-channel-adapter id="sendsomemsg"
channel="putMsgChannel" connection-factory="connectionFactory"
session-transacted="true" destination-expression="headers['somequeue']" />
How to add another int-jms:outbound-channel-adapte for other queue having same transaction boundaries of message driven adapter?. Also flush=true been set so that message is not passed downstream if there is any jpa adapter exception.
As long as the queue sends are performed using a JmsTemplate (including the use of a JMS outbound channel adapter), on the same thread, they will be performed in the same transactional session as the message-driven adapter's message delivery.
If you add the JDBC transaction manager to the message-driven adapter, its transaction will be synchronized with the JMS transaction.
This provides "Best Effort 1PC" as discussed in Dave Syer's JavaWorld article: Distributed transactions in Spring, with and without XA.
There is a small possibility that the DB commit will succeed and the JMS commit fails, so you need to deal with duplicates. To avoid that you need a full XA solution.
Below is my config. In case of successful posting to queue the flow is not passing to confirm-ack-channel. Can anyone suggest where am going wrong?
<int-amqp:outbound-channel-adapter channel="fulfillmentOutboundChannel"
routing-key="xyz"
amqp-template="transactionalRabbitTemplate"
confirm-correlation-expression="payload"
confirm-ack-channel="successRespTransformChannel"
confirm-nack-channel="failureRespTransformChannel"
return-channel="failureRespTransformChannel"
mapped-request-headers="*"
/>
<int:channel id="successRespTransformChannel">
<int:interceptors><int:wire-tap channel="loggerChannel"/></int:interceptors>
</int:channel>
<chain input-channel="successRespTransformChannel">
<int:header-enricher>
<error-channel ref="failed-publishing" />
</int:header-enricher>
<service-activator id="successResp" expression="#abc.addRequestTracking(payload.id,'success')"/>
</chain>
In order to get acks/nacks, in addition to setting up the adapter with confirm-correlation, You also need to enable publisherConfirms on the CachingConnectionFactory.
From the documentation:
For Publisher Confirms (aka Publisher Acknowledgements), the template requires a CachingConnectionFactory that has its publisherConfirms property set to true.
Thanks for your response Gary. I have already made publish-confirms=true as below..
<rabbit:connection-factory id="fufillConnectionFactory"
connection-factory="rabbitClientConnectionFactory"
publisher-confirms="true"
publisher-returns="true"/>
<bean id="rabbitClientConnectionFactory" class="com.rabbitmq.client.ConnectionFactory" >
<property name="uri" value="${mq.uri}" />
<property name="requestedHeartbeat" value="30" />
</bean>
I am using Spring Batch 3.2 for bulk migration of data from XML to Database.
My XML contains around 140K users and I want to dump it into DB.
I do not want to proceed it in a single thread.
I tried using TaskExecutor but not able to succeed due to below error.
at java.lang.Thread.run(Thread.java:724)
Caused by: com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </consumerName>; expected </first>.
at [row,col {unknown-source}]: [4814,26]
at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:606)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:479)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:464)
at com.ctc.wstx.sr.BasicStreamReader.reportWrongEndElem(BasicStreamReader.java:3283)
at com.ctc.wstx.sr.BasicStreamReader.readEndElem(BasicStreamReader.java:3210)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2829)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1072)
at org.codehaus.stax2.ri.Stax2EventReaderImpl.peek(Stax2EventReaderImpl.java:367)
at org.springframework.batch.item.xml.stax.DefaultFragmentEventReader.nextEvent(DefaultFragmentEventReader.java:114)
at org.springframework.batch.item.xml.stax.DefaultFragmentEventReader.markFragmentProcessed(DefaultFragmentEventReader.java:184)
where consumerName and first are XML nodes.
I knew StaxEventItemReader is not a thread safe and multiple threads are using XML and due to that issue, there is some problem in marking fragments as processed and I am not able to get unique record as well as complete fragment to process.
Can any one suggest me how Can I use multi-threading/partitioning in my case.
What I want
By using multi-threading, how can I make sure that each thread get unique chuck i.e (Thread 1 - fragement 1-100, Thread 2 - fragement 101-200.... so on)
Each thread process unique chuck and dump into DB.
My configuration
<batch:job id="sampleJob">
<batch:step id="multiThreadStep" allow-start-if-complete="true">
<batch:tasklet transaction-manager="transactionManager" task-executor="taskExecutor" throttle-limit="10">
<batch:chunk reader="xmlItemReader" writer="itemWriter"
processor="itemProcessor" commit-interval="10" skip-limit="1500000">
<batch:skippable-exception-classes>
<batch:include class="java.lang.Exception" />
</batch:skippable-exception-classes>
</batch:chunk>
</batch:tasklet>
</batch:step>
</batch:job>
<!-- <bean id="taskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor">
<property name="concurrencyLimit" value="10"/>
</bean> -->
<bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor">
<property name="corePoolSize" value="10" />
<property name="maxPoolSize" value="10" />
<property name="allowCoreThreadTimeOut" value="true" />
</bean>
<bean id="xmlItemReader" class="org.springframework.batch.item.xml.StaxEventItemReader">
<property name="fragmentRootElementName" value="userItem" />
<property name="unmarshaller" ref="userDetailUnmarshaller" />
<property name="saveState" value="false" />
</bean>
Sample XML
<userItems>
<userItem>
<...>
<...>
</userItem>
<userItem>
<...>
<...>
</userItem>
...
...
</userItems>
I've got for a JPA Outbound-channel-adapter both transactional and request-handler-advice-chain. In the advice-chain I try to log the Exception, when it happens.
It iss not logged, but I know that it actually happend since the Message was sent to failover clickDbFailoverChannel . What can be a problem with it? Is it a bug in Spring Integration?
<int:channel id="clickDbWithFailoverChannelSite-1">
<int:dispatcher load-balancer="none" task-executor="clickDbSiteRouterExecutor"/>
</int:channel>
<int:bridge input-channel="clickDbWithFailoverChannelSite-1"
output-channel="jpaOutboundChannelSite-1" order="1" send-timeout="100" />
<int:bridge input-channel="clickDbWithFailoverChannelSite-1"
output-channel="clickDbFailoverChannel" order="2" />
<int-jpa:outbound-channel-adapter id="jpaOutboundChannelSite-1"
persist-mode="PERSIST" flush-size="100" entity-manager-factory="emfSite-1">
<int-jpa:transactional transaction-manager="transactionManagerSite-1" />
<int-jpa:request-handler-advice-chain>
<bean class="org.springframework.integration.handler.advice.ExpressionEvaluatingRequestHandlerAdvice">
<property name="failureChannel" ref="clickDbFailureLogger"/>
<property name="onFailureExpression" value="#exception"/>
</bean>
</int-jpa:request-handler-advice-chain>
</int-jpa:outbound-channel-adapter>
OK. I can guess where is your issue. The real exception to rollback the transaction is caused before an internal logic, where <request-handler-advice-chain> does the stuff. That's why your ExpressionEvaluatingRequestHandlerAdvice doesn't get a failure message.
To workaround your rollback issue, you should replace <int-jpa:transactional> with <tx:advice> within <int-jpa:request-handler-advice-chain>.
You should understand here that <int-jpa:transactional> is for entire MessageHandler.handleMessage, but <int-jpa:request-handler-advice-chain> is just for its part - AbstractReplyProducingMessageHandler.handleRequestMessage.
UPDATE
TX Advice should be like this:
<tx:advice transaction-manager="transactionManagerSite-1"/>
<tx:attributes>
<tx:method name="*"/>
</tx:attributes>
</tx:advice>