I have a problem with int-jms:inbound-channel-adapter, when i start my application i see in activemq webadmin that i have two consumer for the queue.
<int-jms:inbound-channel-adapter
id="jmsAdapter"
acknowledge="transacted"
destination-name="${destinationName}"
connection-factory="cachedConnectionFactory"
channel="inboundChannel"
auto-startup="true" >
<int:poller fixed-delay="100" ></int:poller>
</int-jms:inbound-channel-adapter>
Somme thing wrong in my conf?
problem is that org.springframework.jms.connection.CachingConnectionFactory
create multiple session, and one consumer per session . Sometimes i have 3 consumers for the same queue.
To resolve this i use cachedConsumer à false.
Related
we used to configure the poller in int:chain like below, the inboundChannel is configured with queue
<int:chain id="messageProcessChain" input-channel="inboundChannel" >
<int:poller fixed-delay="1" max-messages-per-poll="1"/>
This time, we'd like to programmatically initialize int flows, so using DSL. my test code as below:
Jms.messageDrivenChannelAdapter(inConnFactory).destination(flowProperties.getInputQueue()))
.channel(MessageChannels.queue(mongoDbChannelMessageStore,"groupId"))
// .enrichHeaders(testHeaders)
.handle(m -> {
System.out.println("test handler");
logger.info("test dsl flow:"+m.getPayload().toString());
},c -> c.poller(Pollers.fixedRate(flowProperties.getInboudFixedRate()).maxMessagesPerPoll(10)))
.get()
it cannot work with the ".enrichHeaders(testHeaders)", due to'No poller has been defined for endpoint', but neither can I configure a poller for the header richer because it has an implicit SubscribableChannel?
In such case, can only use a bridge to connect the two channel? Is there some other approaches?
Use .enrichHeaders(headers, e -> e.poller(...)).
However, you should NOT use a queue channel with a message-driven channel adapter, you will likely lose messages in the event of a failure.
To achieve concurrency, increase concurrency on the adapter.
I am using spring integration, sometimes I got following exception:
org.springframework.messaging.MessageDeliveryException{failed to send Message to channel 'milestoneChannel';
nested exception is org.springframework.dao.CannotAcquireLockException: PreparedStatementCallback; SQL [UPDATE INT_MESSAGE_GROUP set UPDATED_DATE=? where GROUP_KEY=? and REGION=?];
Lock wait timeout exceeded;
try restarting transaction; nested exception is com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException:
Lock wait timeout exceeded; try restarting transaction, failedMessage=GenericMessage....
Here is my part spring configuration file
<int:channel id="milestoneChannel">
<int:queue message-store="dataMessageStore"/>
</int:channel>
.....
<si-xml:xpath-router id="messageRouter" default-output-channel="filterHistoryChannel" resolution-required="false">
<si-xml:xpath-expression expression="//messageType"/>
<si-xml:mapping value="MILESTONE" channel="mlestoneChannel"/>
<si-xml:mapping value="JOB_INFO" channel="jobInfoChannel"/>
</si-xml:xpath-router>
......
Consider to use a JdbcChannelMessageStore instead with specialized MySqlChannelMessageStoreQueryProvider: https://docs.spring.io/spring-integration/docs/current/reference/html/jdbc.html#jdbc-message-store-channels. This was especially designed for QueueChannel operations, so you should be good.
Another, not relevant, concern: do not try to use so many QueueChannels. Doesn't look like a logic of that router should start from queue. More over your mlestoneChannel is also QueueChannel. That's too much thread shifting and very stressful for default TaskScheduler.
I see one issue today.I have few messages dropped in a queue and one message is throwing error in service activator. the errorred message keep on trying again and again without other messages are processing from a queue. Is there any way I can set in below config to send error msg to end of queue messages. (ex if queue depth is 1000, error msg will be at 1000 pos)
<jms:message-driven-channel-adapter id="exch" destination="requestQueue" channel="jmsInChannel"
transaction-manager="txManager" acknowledge="auto" concurrent-consumers="3"/>
<int:service-activator input-channel="jmsInChannel" ref="messageService"/>
Pls advice.
thanks Gary..Unfortunately we can't change any MQ configuration.
I have tried as follows and going to add service activator for errorRetryChannel.
<int:service-activator input-channel="jmsInChannel" ref="messageService">
<int:request-handler-advice-chain>
<int:retry-advice max-attempts="4" recovery-channel="errorRetryChannel">
<int:exponential-back-off initial="1000" multiplier="5.0" maximum="60000" />
</int:retry-advice>
</int:request-handler-advice-chain>
</int:service-activator>
Typically, you would configure the broker to dump "poison" messages to a dead letter queue after some number of attempts.
If you can't do that, you can add a retry advice to your service activator together with a recoverer that can log and otherwise dispose of the bad message.
A custom recoverer could requeue the message at the tail of the queue but you will probably want to trash irrecoverable messages.
Considering any message bus can be deployed with spring-xd, is there any API provided by spring-xd that can write to the message bus deployed, say, Kafka/rabbitmq based on the configurations in xd/config/servers.yml or any other such place.
I am writing a processor (itemprocessor), which does some processing of the data and the writer will write the data to a rabbitmq queue for now. Since, in the current deployment scenario, rabbitmq may or may not be deployed, the processor should be able to write to the default Redis message bus.I know I can write to rabbitmq using apis provided by spring-rabbit, but that will tie my processor to RabbitMQ. I am looking for a way to generalize it. I tried to looked into spring-xd code to see if there were any examples like this. I found a MessageProcessor example, but this one is a stream processor, not sure how I can apply it or if I am on the right track.
https://github.com/spring-projects/spring-xd/blob/master/spring-xd-rxjava/src/test/java/org/springframework/xd/rxjava/PongMessageProcessor.java
I just started working with spring-xd, so please pardon my ignorance if this has already been discussed. Any pointers are greatly appreciated.
Update
Thanks Gary, based on your answer I tried out the spring-integration jms samples.
I have a spring batch job which has
<batch:chunk reader="reader" processor="processor" writer="writer" />
I want the output of the writer to be written to any underlying message bus, RabiitMQ to start with. So I added the following, based on what I saw in the examples:
<beans:bean id="writer" class="abc" scope="step">
</beans:bean>
<channel id="outputme"/>
<beans:bean id="requestQueue" class="org.apache.activemq.command.ActiveMQQueue">
<beans:constructor-arg value="queue.demo"/>
</beans:bean>
<beans:bean id="replyQueue" class="org.apache.activemq.command.ActiveMQQueue">
<beans:constructor-arg value="queue.reply"/>
</beans:bean>
<jms:outbound-gateway request-channel="outputme"
request-destination="requestQueue"
reply-channel="jmsReplyChannel"/>
<channel id="jmsReplyChannel" />
<beans:beans profile="default">
<stream:stdout-channel-adapter channel="jmsReplyChannel" append-newline="true"/>
</beans:beans>
When I execute this, I see the following output, which leads me to believe that something is getting written to the embedded ActiveMQ broker.
16:05:42,400 [AbstractApplicationContext] - Closing org.springframework.context.support.GenericApplicationContext#125a6d70: startup date [Tue Feb 03 16:05:40 PST 2015]; root of context hierarchy
16:05:42,401 [DefaultLifecycleProcessor$LifecycleGroup] - Stopping beans in phase 0
16:05:42,402 [EventDrivenConsumer] - Removing {jms:outbound-gateway} as a subscriber to the 'outputme' channel
16:05:42,402 [AbstractSubscribableChannel] - Channel 'org.springframework.context.support.GenericApplicationContext#125a6d70.outputme' has 0 subscriber(s).
16:05:42,402 [AbstractEndpoint] - stopped org.springframework.integration.config.ConsumerEndpointFactoryBean#0
16:05:42,402 [EventDrivenConsumer] - Removing {service-activator} as a subscriber to the 'jmsReplyChannel' channel
16:05:42,402 [AbstractSubscribableChannel] - Channel 'org.springframework.context.support.GenericApplicationContext#125a6d70.jmsReplyChannel' has 1 subscriber(s).
16:05:42,402 [AbstractEndpoint] - stopped org.springframework.integration.config.ConsumerEndpointFactoryBean#1
16:05:42,402 [EventDrivenConsumer] - Removing {stream:outbound-channel-adapter(character)} as a subscriber to the 'jmsReplyChannel' channel
16:05:42,403 [AbstractSubscribableChannel] - Channel 'org.springframework.context.support.GenericApplicationContext#125a6d70.jmsReplyChannel' has 0 subscriber(s).
16:05:42,403 [AbstractEndpoint] - stopped org.springframework.integration.config.ConsumerEndpointFactoryBean#2
However, when I try to change ActiveMQ with RabbitMQ by changing the connectionfactory like this:
<rabbit:connection-factory id="connectionFactory" />
I get an error saying:
Cannot convert value of type [org.springframework.amqp.rabbit.connection.CachingConnectionFactory] to required type [javax.jms.ConnectionFactory] for property 'connectionFactory'
which I understand based on what is mentioned in the schema file at http://www.springframework.org/schema/integration/jms/spring-integration-jms.xsd, as the required type is javax.jms.ConnectionFactory for the element connection-factory. I looked around, did not find a way to create the conenctionFactory for RabbitMQ the same way as we create the ConnectionFactory for ActiveMQ.
I looked around and am not sure how to fix this. Maybe I am missing something very basic. I am not even sure if this is the right approach. Can you please let me know what I am missing and if this is the right approach? I apologize in advance if this has been discussed already.
Thank you again for your time.
Thank you so much for your time.
regards,
Alice
The MessageBus SPI is very specific for XD intra-module commmunication; it's not designed for arbitrary application-level messaging.
That said, XD (and its message bus implementations) uses the Spring Integration project extensively.
That project provides the abstraction you need. You can send to a message channel (either using a MessagingGateway or a MessagingTemplate and, downstream of that channel, you can wire in any kind of channel adapter (rabbit [amqp], redis, etc).
So your item processor is decoupled from the actual technology that receives the message.
Take a look at the Spring Integration reference manual (there's a link on the project page).
1) Server sends a message to client.
2) Inbound channel adapter is configured to wait for "MANUAL" acknowledge mode operation from consumer
3) "TaskBundlereceiver" bean is implementing "ChannelAwareMessageListener" and in the implementation method, I am performing message acknowledgement.
I don't see "TaskBundlereceiver" getting executed. Am I missing something ?
Below is the configuration details of the steps that I have explained.
Appreciate your inputs.
#Override
public void onMessage(org.springframework.amqp.core.Message message, Channel channel) throws Exception
{
logger.debug("In onMessage method of the channel aware listener. message =["+message.getBody().toString()+"]");
channel.basicAck(message.getMessageProperties().getDeliveryTag(), true);
}
XML Configuration :
<!-- Channel that receives the task bundle from the server for execution -->
<int:channel id="fromKServerChannel"/>
<int-amqp:inbound-channel-adapter id="taskBundleReceiverAdapter"
channel="fromKServerChannel"
error-channel="taskBundleErrorChannel"
acknowledge-mode="MANUAL"
expose-listener-channel="true"
queue-names="kanga_task_queue"
connection-factory="connectionFactory"
concurrent-consumers="20"/>
<int:chain input-channel="fromKServerChannel" output-channel="nullChannel">
<int:service-activator ref="taskBundleReceiver" method="onMessage"/>
<int:service-activator ref="taskBundleExecutor" method="executeBundle"/>
</int:chain>
It doesn't work that way; the listener is the adapter, not the service invoked via the service-activator. The adapter currently does not support passing the channel to the client for manual acks. The expose-listener-channel attribute is for use when using transactions, so a down-stack rabbit template can participate in the transaction.
Why do you want MANUAL acks? AUTO (default) means the ack will be done automatically by the container when the thread returns normally; if your service throws an exception, the message will be nacked.
So, that's how to control the ack.
If you really want to use MANUAL acks, you'll have to use a <rabbit:listener-container/> to invoke your taskBundleReceiver directly. It could then send a message to the executor using a messaging gateway.