Synchronizing JMS Message Queue and JDBC transaction- Is Transaction proxy needed - spring-integration

We need to persist message from a JMS Queue to a database in a transaction ensuring that JMS message is not acknowledged in case any error is thrown during DB persist.
Based on the solution provided here- Transaction handling while using message driven channel adapter & service activator
Below is the approach I arrived at
<int-jms:message-driven-channel-adapter id="jmsIn"
transaction-manager="transactionManager"
connection-factory="sConnectionFactory"
destination-name="emsQueue"
acknowledge="transacted" channel="jmsInChannel"/>
<int:channel id=" jmsInChannel " />
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.hsqldb.jdbcDriver" />
<property name="url" value="jdbc:hsqldb:mem:testdb" />
</bean>
<!-- Transaction manager for a datasource -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="sconnectionFactory" class="org.springframework.jms.connection.TransactionAwareConnectionFactoryProxy">
<property name="targetConnectionFactory">
<bean class="project.TestConnectionFactory">
</bean>
</property>
<property name="synchedLocalTransactionAllowed" value="true" />
</bean>
Please confirm if the understanding is correct. Also, have the below queries
Is the TransactionAwareConnectionFactoryProxy necessary in this scenario
JMS Queue and JDBC are two separate transactional resources. Is injecting jdbc transaction manager into JMS adapter as shown in this example equivalent to using a ChainedTransactionManager(chaining JMS Transaction Mananger and JDBC transaction manager)

For TransactionAwareConnectionFactoryProxy, please, consult its JavaDocs:
* Proxy for a target CCI {#link javax.resource.cci.ConnectionFactory}, adding
* awareness of Spring-managed transactions. Similar to a transactional JNDI
* ConnectionFactory as provided by a Java EE server.
*
* <p>Data access code that should remain unaware of Spring's data access support
* can work with this proxy to seamlessly participate in Spring-managed transactions.
* Note that the transaction manager, for example the {#link CciLocalTransactionManager},
* still needs to work with underlying ConnectionFactory, <i>not</i> with this proxy.
I'm not sure how is that related to your request about JMS+DB transaction, but I think you should worry about transaction manager which supports both those transactional resources.
For this purpose Spring suggests JtaTransactionManager for XA transactions.
Or you can consider to use ChainedTransactionManager based on the Dave Syer article: http://www.javaworld.com/article/2077963/open-source-tools/distributed-transactions-in-spring--with-and-without-xa.html
UPDATE
If local transaction are fine for you, you really can go ahead with the TransactionAwareConnectionFactoryProxy and its synchedLocalTransactionAllowed to true. And, of course, use that DataSourceTransactionManager directly from the <int-jms:message-driven-channel-adapter>.
Otherwise you have to go with the ChainedTransactionManager. This solution really adds some overhead over the first TransactionAwareConnectionFactoryProxy-based one.

Related

Performance Issue using DefaultMessageListenerContainer with CachingConnectionFactory

In our application we are using SingleConnectionFactory with DefaultMessageListenerContainer consuming from IBM MQ server, performance wise the app is doing pretty good, however on MQ end our application is opening too many new connections randomly. Here is the current config we have.
Try with this instead of caching or single:
<bean id="connectionFactory" class="org.springframework.jms.connection.DelegatingConnectionFactory">
<property name="targetConnectionFactory" ref="primaryRawInputConnectionFactory" />
<property name="shouldStopConnections" value="true"/>
</bean>

Migration Of Spring Batch from 2.2 to 4.x (XML Configuration Of Partition Jobs

I am migrating Spring Batch Partition Jobs with XML configuration to Spring batch 4.x. I am trying to take advantage to an improvement in the MessageChannelPartitionHandler where it looks for completion of remote steps with both a reply channel and a datasource polling.
When I use this configuration:
<int:channel id="partitioned.jms.requests">
<int:dispatcher task-executor="springbatch.partitioned.jms.taskExecutor"/>
</int:channel>
<int:channel id="partitioned.jms.reply" />
<bean id="partitioned.jms.handler" class="org.springframework.batch.integration.partition.MessageChannelPartitionHandler">
<property name="messagingOperations">
<bean class="org.springframework.integration.core.MessagingTemplate">
<property name="defaultChannel" ref="partitioned.jms.requests"/>
</bean>
</property>
<property name="stepName" value="process.partitioned.step"/>
<property name="gridSize" value="${process.step.partitioned.gridSize}"/>
<property name="dataSource" ref="springbatch.repositoryDataSource" />
<property name="pollInterval" value="${springbatch.partition.verification.interval}"/>
</bean>
The step completes but I see an error in the logs.
no output-channel or replyChannel header available
I looked at the class and see I can add a replyChannel property to the MessageChannelPartitionHandler class. If I add the following:
<property name="replyChannel" ref="claim.acp.process.partitioned.jms.reply"/>
I get error back that a pollable channel is needed.
How do I create a pollable channel (assuming from the same JMS queue)?
You need to show the rest of your configuration.
If you are using DB polling for the results, set the output-channel on the jms outbound gateway to "nullChannel" and the replies received over JMS will be discarded.
Or, use an outbound channel adapter (instead of a gateway) (and an inbound-channel-adapter on the slaves). That avoids the replies being returned altogether.
You have to set pollRepositoryForResults to true.
To answer your specific question
<int:channel id="replies>
<int:queue />
<int:channel>

Spring Integration AMQP - retries continuously despite retry advice

I've used Spring Integration with JMS very successfully before, but we're now using with RabbitMQ / AMQP and having some issues with error handling.
I have an int-amqp:inbound-channel-adapter with with a errorChannel set up to receive any exception, here an ErrorTransformer class inspects the failed Message's cause exception. Then depending on the type of exception either :-
suppresses the exception and transforms into a JSON object that can go to AMQP outbound-channel-adapter as a business reply explaining the failure. Here I want the original message consumed/ACKed.
Or Re-throws the causing exception to let RabbitMQ re-deliver the message, a certain number of times.
I found that re-throwing caused the message to be re-delivered continuously, I then read about StatefulRetryOperationsInterceptorFactoryBean, so added an advice chain to retry 3 times, then I got exception about no message-id, so also added a 'MissingMessageIdAdvice' at the start of the advice chain.
Despite the advice, I still get continuous re-tries for a RuntimeException that is re-thrown from errorChannel's ErrorTransformer. I'm publishing the message via RabbitMQ admin just using the defaults. Not sure if the lack of a message-id is making this not work, if so how do I get a message to have an id ?
I'm confused about the differences between the :-
A) ConditionalRejectingErrorHandler (I've set as the inbound adapter's error-handler) which allows me to provide a customFatalExceptionStrategy to implement isFatal(). Where I believe fatal=true (means DISCARD) and message is consumed and discarded, but how can I still send an outbound failure message ?
B) And the errorChannel I have on the inbound adapter which I'm using to inspect an Exception and transform into a outbound fail response message. Here I guess I could throw AmqpRejectAndDontRequeueException, but then why have the ConditionalRejectingErrorHandler as well ? and will thorwing AmqpRejectAndDontRequeueException work
<int-amqp:inbound-channel-adapter id="amqpInRequestPatternValuation" channel="requestAmqpIn" channel-transacted="true" transaction-manager="transactionManager"
queue-names="requestQueue" error-channel="patternValuationErrorChannel" connection-factory="connectionFactory"
receive-timeout="59000" concurrent-consumers="1"
advice-chain="retryChain" error-handler="customErrorHandler" />
<bean id="customErrorHandler" class="org.springframework.amqp.rabbit.listener.ConditionalRejectingErrorHandler">
<constructor-arg ref="customFatalExceptionStrategy"/>
</bean>
<bean id="customFatalExceptionStrategy" class="abc.common.CustomFatalExceptionStrategy"/>
<!-- ADVICE CHAIN FOR CONTROLLING NUMBER OF RE-TRIES before sending to DLQ (or discarding if no DLQ) without this any re-queued fatal message will retry forever -->
<util:list id="retryChain">
<bean class="org.springframework.amqp.rabbit.retry.MissingMessageIdAdvice">
<constructor-arg>
<bean class="org.springframework.retry.policy.MapRetryContextCache" />
</constructor-arg>
</bean>
<ref bean="retryInterceptor" />
</util:list>
<bean id="retryInterceptor"
class="org.springframework.amqp.rabbit.config.StatefulRetryOperationsInterceptorFactoryBean">
<property name="retryOperations" ref="retryTemplate" />
<property name="messageRecoverer" ref="messageRecoverer"/>
</bean>
<bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="retryPolicy" ref="simpleRetryPolicy" />
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="10000" />
</bean>
</property>
</bean>
<bean id="simpleRetryPolicy" class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="3" />
</bean>
You have to use RejectAndDontRequeueRecoverer to stop re-delivery in the end of retries:
* MessageRecover that causes the listener container to reject
* the message without requeuing. This enables failed messages
* to be sent to a Dead Letter Exchange/Queue, if the broker is
* so configured.
Yes, the messageId is important for that retry use-case.
You can inject custom MessageKeyGenerator strategy to determine a unique key from message if you can't supply it manually during sending.
I never got around to posting the solution, so here it is.
Once I had configured the retry advice chain to the AMQP inbound channel adapter which must include a messageRecoverer of RejectAndDontRequeueRecoverer (which I believe is also the dafault). The important point I was missing was to ensure that when sender's send a message they include a message_id. So if publishing via the RabbitMQ Admin Console I needed to include the predefined message_id property and supply a value.
Using the 'MissingMessageIdAdvice' doesn't help (so I removed) as it will generate a different message_id on each re-deliver on the message leading to the re-try count not incrementing as each delivery was considered different from the last

how to add a retry advice with jms:message-driven-channel-adapter

I am new to spring integration. My requirement is that if there is a connection problem to the jms q then it should try to connect 3 times then log it and exit the process. I am not able to do it. It throws an error saying it needs the ref attribute for service:activator. But I don't have/know reference of which class to provide here. Is there any other way of doing it?
<int-jms:message-driven-channel-adapter id="msgIn" channel="toRoute" container="messageListenerContainer" />
<int:service-activator id="service" input-channel="toRoute" >
<int:request-handler-advice-chain>
<bean class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<property name="recoveryCallback">
<bean class="org.springframework.integration.handler.advice.ErrorMessageSendingRecoverer">
<constructor-arg ref=“errorChannel" />
</bean>
</property>
</bean>
</request-handler-advice-chain>
</int:service-activator>
You seem to have completely misunderstood what the framework does.
The service-activator gets a message when one is received from JMS (which implies the connection is good), and needs "something" (a reference to a bean or an expression) to invoke as a result of receiving that message.
The retry advice is to retry calling that service if it fails to process the message for some reason. It is unrelated to whatever is the source of the message (JMS in this case).
It's not clear why you are trying to use Spring Integration for something as simple as testing whether a JMS broker is available.
Perhaps if you can provide some larger context someone might be able to help.

Spring Integration JMS message driven channel adapter stops picking message after certain interval of time

I am using spring integration to read message from TIBCO EMS queue using jms-int:message-driven-channel-adapter. I am facing issue : After certain interval of time say 5-10 hours(occurs at random interval), jms channel adapter stops picking the message even if there is message in the jms queue
Below is my spring integration context :
<bean id="jmsConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="connectionFactory"/>
<property name="sessionCacheSize" value="${sessionCacheSize}"/>
<property name="cacheProducers" value="${cacheProducers}"/>
<property name="cacheConsumers" value="${cacheConsumers}"/>
</bean>
<bean id="jmsQueue" class="${queueClassName}">
<constructor-arg value="${jmsQueue}" />
</bean>
<int-jms:message-driven-channel-adapter
id="jmsMessageDrivenAdapter" connection-factory="jmsConnectionFactory" channel="jmsListenerChannel" destination="jmsQueue"
error-channel="integrationErrorChannel" max-concurrent-consumers="${maxConcurrentConsumers}" auto-startup="${jms.autostart}"/>
I have feature to start/stop jmsMessageDrivenAdapter, and the adapter can be started/stopped anytime while it is picking messages but at this point, start/stop feature does not works too. Please suggest !
This seems to be a duplicate of Spring Integration jms message driven channel adapter fails (but a different contributor, so I'll answer again here)...
This is generally caused by some component in the downstream flow hanging the container thread uninterruptibly (e.g. reading from a socket with no timeout, where no data arrives).
To diagnose, take a thread dump (e.g. jstack ) when the hang occurs, to find out what the listener container threads are doing.

Resources