I am trying to implement a connection between my application (Spring integration) with IBM-MQ, I did see the question spring-integration-support-for-clustered-high-availability-ibm-mq-manager, but in my case each host has different "queueManager" and "channels", it means I will have must have a configuration like follows but, the queueManager and channels properties support a String not a list values:
<bean id="connectionFactory" class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="connectionNameList" value="host1(6464),host1(6464)" />
<property name="clientReconnectTimeout" value="15000" />
<property name="transportType" value="1" />
<property name="queueManager" value="QM1, QM1," />
<property name="channel" value="channel1,channel1"/>
</bean>
The simplest thing would be to define a channel with the same name on both hosts and let the client try host1 first and then host2 using connectionNameList. In this setup it would always prefer host1. You would need to specify a queueManager that is blank so that the client will accept the queue manager it connects to. Example follows:
<property name="queueManager" value="" />
Another option that was pointed out in a comment from Morag on the other post you linked to is to use a CCDT (Client channel definition table).
See Using a client channel definition table with IBM WebSphere MQ classes for JMS. The property name is CCDTURL
The CCDT can have multiple CLNTCONN channel entries with different channel names all having the same QMNAME, this is called a Queue Manager Group, you would then specify the queueManager property as *QMNAME, this will allow the app to connect to which ever queue manager you are directed to with out regard to the actual queue manager name. There are other parameters of the CLNTCONN listed at the bottom of the link I provided that can help you to control if one queue manager is preferred over the other(s) as well as which queue manager to connect to if a reconnect is required.
Related
I have a spring integration application with a requirement of 2 DefaultMessageListenerContainer (with transactionManager) and 2 jms:outbound-channel-adapter.
I have configured each listener container with "property name="connectionFactory" ref="jmsConnectionFactory" and also "property name="transactionManager" ref="platformTransactionManager" .
But for the transaction manager i have used again a different jmsConnectionFactory . I am not using a CachingConnectionFactory in the listener container but using it in platformTransactionManager . For jms:outbound-channel-adapter again i am using the same jmsConnectionFactory which i have used in listener container.
My problem is that with this configuration i am hitting the no. of connections exceeding the limit (150) to a MQ channel in a high load condition.
I tried many different ways but i don't see the no. of connection getting down once it gets up , using below configuration for cachingFactory
CachingConnectionFactory cachingConnectionFactory = new
CachingConnectionFactory(factory);
cachingConnectionFactory.setCacheConsumers(true);
cachingConnectionFactory.setCacheProducers(true);
cachingConnectionFactory.setSessionCacheSize(16);`
i tried
1)sharing the jmsConnectionFactory between the listenerContainer and the platformTransactionManager but the txn doesn't work correctly as i see the messages getting sent in incorrect order
2)using the cachingConenctionFactory in thelistenerContainer` , but still i see the same no. of connections
Should i use 3 different jmsConnectionFactory for the listener, txnmanager and the sender?
What would be best configuration here, please suggest , many thanks!
I think with IBM MQ you can't use CachingConnectionFactory, but instead this one as:
<bean id="connectionFactory" class="org.springframework.jms.connection.DelegatingConnectionFactory">
<property name="targetConnectionFactory" ref="imbMqFactory"/>
<property name="shouldStopConnections" value="true"/>
</bean>
This single connectionFactory must be used from the TX manager and from the containers.
We need to persist message from a JMS Queue to a database in a transaction ensuring that JMS message is not acknowledged in case any error is thrown during DB persist.
Based on the solution provided here- Transaction handling while using message driven channel adapter & service activator
Below is the approach I arrived at
<int-jms:message-driven-channel-adapter id="jmsIn"
transaction-manager="transactionManager"
connection-factory="sConnectionFactory"
destination-name="emsQueue"
acknowledge="transacted" channel="jmsInChannel"/>
<int:channel id=" jmsInChannel " />
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.hsqldb.jdbcDriver" />
<property name="url" value="jdbc:hsqldb:mem:testdb" />
</bean>
<!-- Transaction manager for a datasource -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="sconnectionFactory" class="org.springframework.jms.connection.TransactionAwareConnectionFactoryProxy">
<property name="targetConnectionFactory">
<bean class="project.TestConnectionFactory">
</bean>
</property>
<property name="synchedLocalTransactionAllowed" value="true" />
</bean>
Please confirm if the understanding is correct. Also, have the below queries
Is the TransactionAwareConnectionFactoryProxy necessary in this scenario
JMS Queue and JDBC are two separate transactional resources. Is injecting jdbc transaction manager into JMS adapter as shown in this example equivalent to using a ChainedTransactionManager(chaining JMS Transaction Mananger and JDBC transaction manager)
For TransactionAwareConnectionFactoryProxy, please, consult its JavaDocs:
* Proxy for a target CCI {#link javax.resource.cci.ConnectionFactory}, adding
* awareness of Spring-managed transactions. Similar to a transactional JNDI
* ConnectionFactory as provided by a Java EE server.
*
* <p>Data access code that should remain unaware of Spring's data access support
* can work with this proxy to seamlessly participate in Spring-managed transactions.
* Note that the transaction manager, for example the {#link CciLocalTransactionManager},
* still needs to work with underlying ConnectionFactory, <i>not</i> with this proxy.
I'm not sure how is that related to your request about JMS+DB transaction, but I think you should worry about transaction manager which supports both those transactional resources.
For this purpose Spring suggests JtaTransactionManager for XA transactions.
Or you can consider to use ChainedTransactionManager based on the Dave Syer article: http://www.javaworld.com/article/2077963/open-source-tools/distributed-transactions-in-spring--with-and-without-xa.html
UPDATE
If local transaction are fine for you, you really can go ahead with the TransactionAwareConnectionFactoryProxy and its synchedLocalTransactionAllowed to true. And, of course, use that DataSourceTransactionManager directly from the <int-jms:message-driven-channel-adapter>.
Otherwise you have to go with the ChainedTransactionManager. This solution really adds some overhead over the first TransactionAwareConnectionFactoryProxy-based one.
I've used Spring Integration with JMS very successfully before, but we're now using with RabbitMQ / AMQP and having some issues with error handling.
I have an int-amqp:inbound-channel-adapter with with a errorChannel set up to receive any exception, here an ErrorTransformer class inspects the failed Message's cause exception. Then depending on the type of exception either :-
suppresses the exception and transforms into a JSON object that can go to AMQP outbound-channel-adapter as a business reply explaining the failure. Here I want the original message consumed/ACKed.
Or Re-throws the causing exception to let RabbitMQ re-deliver the message, a certain number of times.
I found that re-throwing caused the message to be re-delivered continuously, I then read about StatefulRetryOperationsInterceptorFactoryBean, so added an advice chain to retry 3 times, then I got exception about no message-id, so also added a 'MissingMessageIdAdvice' at the start of the advice chain.
Despite the advice, I still get continuous re-tries for a RuntimeException that is re-thrown from errorChannel's ErrorTransformer. I'm publishing the message via RabbitMQ admin just using the defaults. Not sure if the lack of a message-id is making this not work, if so how do I get a message to have an id ?
I'm confused about the differences between the :-
A) ConditionalRejectingErrorHandler (I've set as the inbound adapter's error-handler) which allows me to provide a customFatalExceptionStrategy to implement isFatal(). Where I believe fatal=true (means DISCARD) and message is consumed and discarded, but how can I still send an outbound failure message ?
B) And the errorChannel I have on the inbound adapter which I'm using to inspect an Exception and transform into a outbound fail response message. Here I guess I could throw AmqpRejectAndDontRequeueException, but then why have the ConditionalRejectingErrorHandler as well ? and will thorwing AmqpRejectAndDontRequeueException work
<int-amqp:inbound-channel-adapter id="amqpInRequestPatternValuation" channel="requestAmqpIn" channel-transacted="true" transaction-manager="transactionManager"
queue-names="requestQueue" error-channel="patternValuationErrorChannel" connection-factory="connectionFactory"
receive-timeout="59000" concurrent-consumers="1"
advice-chain="retryChain" error-handler="customErrorHandler" />
<bean id="customErrorHandler" class="org.springframework.amqp.rabbit.listener.ConditionalRejectingErrorHandler">
<constructor-arg ref="customFatalExceptionStrategy"/>
</bean>
<bean id="customFatalExceptionStrategy" class="abc.common.CustomFatalExceptionStrategy"/>
<!-- ADVICE CHAIN FOR CONTROLLING NUMBER OF RE-TRIES before sending to DLQ (or discarding if no DLQ) without this any re-queued fatal message will retry forever -->
<util:list id="retryChain">
<bean class="org.springframework.amqp.rabbit.retry.MissingMessageIdAdvice">
<constructor-arg>
<bean class="org.springframework.retry.policy.MapRetryContextCache" />
</constructor-arg>
</bean>
<ref bean="retryInterceptor" />
</util:list>
<bean id="retryInterceptor"
class="org.springframework.amqp.rabbit.config.StatefulRetryOperationsInterceptorFactoryBean">
<property name="retryOperations" ref="retryTemplate" />
<property name="messageRecoverer" ref="messageRecoverer"/>
</bean>
<bean id="retryTemplate" class="org.springframework.retry.support.RetryTemplate">
<property name="retryPolicy" ref="simpleRetryPolicy" />
<property name="backOffPolicy">
<bean class="org.springframework.retry.backoff.ExponentialBackOffPolicy">
<property name="initialInterval" value="10000" />
</bean>
</property>
</bean>
<bean id="simpleRetryPolicy" class="org.springframework.retry.policy.SimpleRetryPolicy">
<property name="maxAttempts" value="3" />
</bean>
You have to use RejectAndDontRequeueRecoverer to stop re-delivery in the end of retries:
* MessageRecover that causes the listener container to reject
* the message without requeuing. This enables failed messages
* to be sent to a Dead Letter Exchange/Queue, if the broker is
* so configured.
Yes, the messageId is important for that retry use-case.
You can inject custom MessageKeyGenerator strategy to determine a unique key from message if you can't supply it manually during sending.
I never got around to posting the solution, so here it is.
Once I had configured the retry advice chain to the AMQP inbound channel adapter which must include a messageRecoverer of RejectAndDontRequeueRecoverer (which I believe is also the dafault). The important point I was missing was to ensure that when sender's send a message they include a message_id. So if publishing via the RabbitMQ Admin Console I needed to include the predefined message_id property and supply a value.
Using the 'MissingMessageIdAdvice' doesn't help (so I removed) as it will generate a different message_id on each re-deliver on the message leading to the re-try count not incrementing as each delivery was considered different from the last
I have the same problem as several other questions here, none of which have really been answered; that is, with CAS 4.x (4.2.6 actually) I cannot get LDAP attributes to return to the client application.
Question 1 This seems like overkill; custom code to get around what is a "simple" problem.
Question 2 Had already done this, and it didn't work.
So, now it's my turn to ask... is there some magic to making it work? We've used 3.5 for a long time without any issues. I'm trying to convert those settings to the 4.x Maven overlayer and new context configuration of 4.x, and it's not doin' it.
I can see in the logs that CAS is requesting, and getting the properties I'm looking for from LDAP. But they are not getting put in the token back to the application.
What more needs to be done beyond what the Apereo documentation lays out? I'm thinking it's the attribute repository maybe??? If something would help you help me through this, just ask: Config, Logs (redacted of course)... anything.
Thanks.
Update #1. Here is my resolvers list. NOTE: I keep code/settings in place commented out until I get it to work before I clean stuff out.
<util:map id="authenticationHandlersResolvers">
<!--
<entry key-ref="proxyAuthenticationHandler" value-ref="proxyPrincipalResolver" />
<entry key-ref="primaryAuthenticationHandler" value-ref="primaryPrincipalResolver" />
-->
<!--<entry key-ref="ldapAuthenticationHandler" value-ref="primaryPrincipalResolver" /> -->
<entry key-ref="ldapAuthenticationHandler" value="#{null}" />
</util:map>
Update #2
I've done more testing, and still am unsuccessful. I think, it's coming down to the principalAttributeMap of the LdapAuthenticationHandler not working, OR, the attributeReleasePolicy of the serviceRegistryDao... anyone see any issues in this config?
<bean id="ldapAuthenticationHandler" class="org.jasig.cas.authentication.LdapAuthenticationHandler"
p:principalIdAttribute="sAMAccountName"
c:authenticator-ref="authenticator"
>
<property name="principalAttributeMap">
<map>
<entry key="cn" value="cn" />
<entry key="mail" value="Email" />
<entry key="memberOf" value="Groups" />
<entry key="displayName" value="displayName" />
</map>
</property>
</bean>
<bean id="serviceRegistryDao" class="org.jasig.cas.services.InMemoryServiceRegistryDaoImpl">
<property name="registeredServices">
<list>
<bean class="org.jasig.cas.services.RegexRegisteredService"
p:id="5"
p:name="All Servicesxxx"
p:description="Allow connections for all services and protocols"
p:serviceId="^(http|https|imaps)://.*"
p:evaluationOrder="5"
>
<property name="attributeReleasePolicy">
<bean class="org.jasig.cas.services.ReturnAllAttributeReleasePolicy" />
</property>
</bean>
</list>
</property>
</bean>
To quote from CAS Security Guide:
All communication with the CAS server MUST occur over a secure channel
(i.e. TLSv1). There are two primary justifications for this
requirement:
The authentication process requires transmission of security
credentials.
The CAS ticket-granting ticket is a bearer token.
Since the disclosure of either data would allow impersonation attacks, it’s
vitally important to secure the communication channel between CAS
clients and the CAS server.
Practically, it means that all CAS urls must use HTTPS, but it also
means that all connections from the CAS server to the application must
be done using HTTPS:
when the generated service ticket is sent back to the application on the “service” url when a
proxy callback url is called.
HTTPS is a must in CAS. However, there are possibilities to disable it but it is highly recommended not to do so. If you have struggle to handle the SSL configuration, I recommend you read my question where I explain in detail how to deal with the keystores.
I am using spring integration to read message from TIBCO EMS queue using jms-int:message-driven-channel-adapter. I am facing issue : After certain interval of time say 5-10 hours(occurs at random interval), jms channel adapter stops picking the message even if there is message in the jms queue
Below is my spring integration context :
<bean id="jmsConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="connectionFactory"/>
<property name="sessionCacheSize" value="${sessionCacheSize}"/>
<property name="cacheProducers" value="${cacheProducers}"/>
<property name="cacheConsumers" value="${cacheConsumers}"/>
</bean>
<bean id="jmsQueue" class="${queueClassName}">
<constructor-arg value="${jmsQueue}" />
</bean>
<int-jms:message-driven-channel-adapter
id="jmsMessageDrivenAdapter" connection-factory="jmsConnectionFactory" channel="jmsListenerChannel" destination="jmsQueue"
error-channel="integrationErrorChannel" max-concurrent-consumers="${maxConcurrentConsumers}" auto-startup="${jms.autostart}"/>
I have feature to start/stop jmsMessageDrivenAdapter, and the adapter can be started/stopped anytime while it is picking messages but at this point, start/stop feature does not works too. Please suggest !
This seems to be a duplicate of Spring Integration jms message driven channel adapter fails (but a different contributor, so I'll answer again here)...
This is generally caused by some component in the downstream flow hanging the container thread uninterruptibly (e.g. reading from a socket with no timeout, where no data arrives).
To diagnose, take a thread dump (e.g. jstack ) when the hang occurs, to find out what the listener container threads are doing.