I have a spring integration application with a requirement of 2 DefaultMessageListenerContainer (with transactionManager) and 2 jms:outbound-channel-adapter.
I have configured each listener container with "property name="connectionFactory" ref="jmsConnectionFactory" and also "property name="transactionManager" ref="platformTransactionManager" .
But for the transaction manager i have used again a different jmsConnectionFactory . I am not using a CachingConnectionFactory in the listener container but using it in platformTransactionManager . For jms:outbound-channel-adapter again i am using the same jmsConnectionFactory which i have used in listener container.
My problem is that with this configuration i am hitting the no. of connections exceeding the limit (150) to a MQ channel in a high load condition.
I tried many different ways but i don't see the no. of connection getting down once it gets up , using below configuration for cachingFactory
CachingConnectionFactory cachingConnectionFactory = new
CachingConnectionFactory(factory);
cachingConnectionFactory.setCacheConsumers(true);
cachingConnectionFactory.setCacheProducers(true);
cachingConnectionFactory.setSessionCacheSize(16);`
i tried
1)sharing the jmsConnectionFactory between the listenerContainer and the platformTransactionManager but the txn doesn't work correctly as i see the messages getting sent in incorrect order
2)using the cachingConenctionFactory in thelistenerContainer` , but still i see the same no. of connections
Should i use 3 different jmsConnectionFactory for the listener, txnmanager and the sender?
What would be best configuration here, please suggest , many thanks!
I think with IBM MQ you can't use CachingConnectionFactory, but instead this one as:
<bean id="connectionFactory" class="org.springframework.jms.connection.DelegatingConnectionFactory">
<property name="targetConnectionFactory" ref="imbMqFactory"/>
<property name="shouldStopConnections" value="true"/>
</bean>
This single connectionFactory must be used from the TX manager and from the containers.
Related
I have a Spring integration flow which bridges from ActiveMQ to OracleAQ.
See example project under GitHub - https://github.com/cknzl2014/springio-ora-xa/tree/atomikos.
When I run it without XA, it is blazingly fast.
With XA, it processes only 1 to 2 messages per second.
When profiling the application, I see that for every message a new physical connection is established, and with this, the metadata query is issued on the oracle db.
But I don't understand why it does this, and how I can prevent this from happening.
Does anyone of you guys have experience with OracleAQ and XA?
Could this be a problem with the XA transaction manager (I use Atomikos)?
Thanks for your help,
Chris
We found a solution to the problem.
It consists of four steps.
Step 1: Use the latest Oracle client libraries
The first step ist to use the lastest Oracle 12c client libraries.
There were significant improvements in the ojdbc8.jar, e.g. they use stored procedures to get the metadata now.
This increased the throughput to about 10 msgs/s.
Step 2: Setup connection pooling correctly
The second step was improving the connection pooling according to article http://thinkfunctional.blogspot.ch/2012/05/atomikos-and-oracle-aq-pooling-problem.html:
<bean id="oraXaDataSource" primary="true"
class="oracle.jdbc.xa.client.OracleXADataSource" destroy-method="close">
<property name="URL" value="${oracle.url}" />
<property name="user" value="${oracle.username}" />
<property name="password" value="${oracle.password}" />
</bean>
<bean id="atomikosOraclaDataSource"
class="org.springframework.boot.jta.atomikos.AtomikosDataSourceBean">
<property name="uniqueResourceName" value="xaOracleAQ" />
<property name="xaDataSource" ref="oraXaDataSource" />
<property name="poolSize" value="5" />
</bean>
<bean id="OracleAQConnectionFactory" class="oracle.jms.AQjmsFactory" factory-method="getConnectionFactory">
<constructor-arg ref="atomikosOraclaDataSource" />
</bean>
This configuration alone resultet in exceptions because of 'auto-commit' of the Oracle connection.
Step 3: Set autoCommit to false
The third step was to set the following java system property (see https://docs.oracle.com/database/121/JAJDB/oracle/jdbc/OracleConnection.html#CONNECTION_PROPERTY_AUTOCOMMIT):
-DautoCommit=false
But then the throughput went down to 1 to 2 msg/s again.
Step 4: Set oracle.jdbc.autoCommitSpecCompliant to false
The last step was to set the following java system property (see https://docs.oracle.com/database/121/JAJDB/oracle/jdbc/OracleConnection.html#CONNECTION_PROPERTY_AUTO_COMMIT_SPEC_COMPLIANT):
-Doracle.jdbc.autoCommitSpecCompliant=false
Now we get a throughput of 80 msgs/s.
Conclusion
The setting of oracle.jdbc.autoCommitSpecCompliant to false is not elegant, but solved the problem.
We have to investigate further to see how we can get around this problem without setting oracle.jdbc.autoCommitSpecCompliant to false.
Many thanks to Dani Steinmann (stonie) for the help!
P.S.: I updated the sample project under GitHub - https://github.com/cknzl2014/springio-ora-xa/tree/atomikos.
First of all you should be sure that you use pool for JDBC connections.
On the other hand you may consider to use ChainedTransactionManager isntead of XA for two target transaction managers - JMS and JDBC.
Also see some information in the JDBC extensions project.
There is also some Oracle AQ API as well in that project.
We need to persist message from a JMS Queue to a database in a transaction ensuring that JMS message is not acknowledged in case any error is thrown during DB persist.
Based on the solution provided here- Transaction handling while using message driven channel adapter & service activator
Below is the approach I arrived at
<int-jms:message-driven-channel-adapter id="jmsIn"
transaction-manager="transactionManager"
connection-factory="sConnectionFactory"
destination-name="emsQueue"
acknowledge="transacted" channel="jmsInChannel"/>
<int:channel id=" jmsInChannel " />
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.hsqldb.jdbcDriver" />
<property name="url" value="jdbc:hsqldb:mem:testdb" />
</bean>
<!-- Transaction manager for a datasource -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="sconnectionFactory" class="org.springframework.jms.connection.TransactionAwareConnectionFactoryProxy">
<property name="targetConnectionFactory">
<bean class="project.TestConnectionFactory">
</bean>
</property>
<property name="synchedLocalTransactionAllowed" value="true" />
</bean>
Please confirm if the understanding is correct. Also, have the below queries
Is the TransactionAwareConnectionFactoryProxy necessary in this scenario
JMS Queue and JDBC are two separate transactional resources. Is injecting jdbc transaction manager into JMS adapter as shown in this example equivalent to using a ChainedTransactionManager(chaining JMS Transaction Mananger and JDBC transaction manager)
For TransactionAwareConnectionFactoryProxy, please, consult its JavaDocs:
* Proxy for a target CCI {#link javax.resource.cci.ConnectionFactory}, adding
* awareness of Spring-managed transactions. Similar to a transactional JNDI
* ConnectionFactory as provided by a Java EE server.
*
* <p>Data access code that should remain unaware of Spring's data access support
* can work with this proxy to seamlessly participate in Spring-managed transactions.
* Note that the transaction manager, for example the {#link CciLocalTransactionManager},
* still needs to work with underlying ConnectionFactory, <i>not</i> with this proxy.
I'm not sure how is that related to your request about JMS+DB transaction, but I think you should worry about transaction manager which supports both those transactional resources.
For this purpose Spring suggests JtaTransactionManager for XA transactions.
Or you can consider to use ChainedTransactionManager based on the Dave Syer article: http://www.javaworld.com/article/2077963/open-source-tools/distributed-transactions-in-spring--with-and-without-xa.html
UPDATE
If local transaction are fine for you, you really can go ahead with the TransactionAwareConnectionFactoryProxy and its synchedLocalTransactionAllowed to true. And, of course, use that DataSourceTransactionManager directly from the <int-jms:message-driven-channel-adapter>.
Otherwise you have to go with the ChainedTransactionManager. This solution really adds some overhead over the first TransactionAwareConnectionFactoryProxy-based one.
I am trying to implement a connection between my application (Spring integration) with IBM-MQ, I did see the question spring-integration-support-for-clustered-high-availability-ibm-mq-manager, but in my case each host has different "queueManager" and "channels", it means I will have must have a configuration like follows but, the queueManager and channels properties support a String not a list values:
<bean id="connectionFactory" class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="connectionNameList" value="host1(6464),host1(6464)" />
<property name="clientReconnectTimeout" value="15000" />
<property name="transportType" value="1" />
<property name="queueManager" value="QM1, QM1," />
<property name="channel" value="channel1,channel1"/>
</bean>
The simplest thing would be to define a channel with the same name on both hosts and let the client try host1 first and then host2 using connectionNameList. In this setup it would always prefer host1. You would need to specify a queueManager that is blank so that the client will accept the queue manager it connects to. Example follows:
<property name="queueManager" value="" />
Another option that was pointed out in a comment from Morag on the other post you linked to is to use a CCDT (Client channel definition table).
See Using a client channel definition table with IBM WebSphere MQ classes for JMS. The property name is CCDTURL
The CCDT can have multiple CLNTCONN channel entries with different channel names all having the same QMNAME, this is called a Queue Manager Group, you would then specify the queueManager property as *QMNAME, this will allow the app to connect to which ever queue manager you are directed to with out regard to the actual queue manager name. There are other parameters of the CLNTCONN listed at the bottom of the link I provided that can help you to control if one queue manager is preferred over the other(s) as well as which queue manager to connect to if a reconnect is required.
I am new to spring integration. My requirement is that if there is a connection problem to the jms q then it should try to connect 3 times then log it and exit the process. I am not able to do it. It throws an error saying it needs the ref attribute for service:activator. But I don't have/know reference of which class to provide here. Is there any other way of doing it?
<int-jms:message-driven-channel-adapter id="msgIn" channel="toRoute" container="messageListenerContainer" />
<int:service-activator id="service" input-channel="toRoute" >
<int:request-handler-advice-chain>
<bean class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">
<property name="recoveryCallback">
<bean class="org.springframework.integration.handler.advice.ErrorMessageSendingRecoverer">
<constructor-arg ref=“errorChannel" />
</bean>
</property>
</bean>
</request-handler-advice-chain>
</int:service-activator>
You seem to have completely misunderstood what the framework does.
The service-activator gets a message when one is received from JMS (which implies the connection is good), and needs "something" (a reference to a bean or an expression) to invoke as a result of receiving that message.
The retry advice is to retry calling that service if it fails to process the message for some reason. It is unrelated to whatever is the source of the message (JMS in this case).
It's not clear why you are trying to use Spring Integration for something as simple as testing whether a JMS broker is available.
Perhaps if you can provide some larger context someone might be able to help.
I am using spring integration to read message from TIBCO EMS queue using jms-int:message-driven-channel-adapter. I am facing issue : After certain interval of time say 5-10 hours(occurs at random interval), jms channel adapter stops picking the message even if there is message in the jms queue
Below is my spring integration context :
<bean id="jmsConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="connectionFactory"/>
<property name="sessionCacheSize" value="${sessionCacheSize}"/>
<property name="cacheProducers" value="${cacheProducers}"/>
<property name="cacheConsumers" value="${cacheConsumers}"/>
</bean>
<bean id="jmsQueue" class="${queueClassName}">
<constructor-arg value="${jmsQueue}" />
</bean>
<int-jms:message-driven-channel-adapter
id="jmsMessageDrivenAdapter" connection-factory="jmsConnectionFactory" channel="jmsListenerChannel" destination="jmsQueue"
error-channel="integrationErrorChannel" max-concurrent-consumers="${maxConcurrentConsumers}" auto-startup="${jms.autostart}"/>
I have feature to start/stop jmsMessageDrivenAdapter, and the adapter can be started/stopped anytime while it is picking messages but at this point, start/stop feature does not works too. Please suggest !
This seems to be a duplicate of Spring Integration jms message driven channel adapter fails (but a different contributor, so I'll answer again here)...
This is generally caused by some component in the downstream flow hanging the container thread uninterruptibly (e.g. reading from a socket with no timeout, where no data arrives).
To diagnose, take a thread dump (e.g. jstack ) when the hang occurs, to find out what the listener container threads are doing.