We have the following issue in our application. Messages come in on an inbound channel adapter, and are accumulated in an aggregator using a persistent message store. Once the condition defined in the release strategy returns true, the messages are sent to the next stage in the processing. If an exception is thrown at that next stage of processing, the transaction will be rolled back and the messages placed again in the persistent message store. However, the transaction will not place the messages back on the original queue, as messages are acked once they are placed in the aggregator. This is not what we want. Ideally, if an exception occurs when processing downstream one of the messages that the aggregator has batched, only that single message for which an error occurred would be rolled back by the transaction and placed back on the original queue. Is there anyway using Spring Integration to do this?
Thanks
Not built-in; you would have to do your own error-handling and requeue the failed message.
To insert error handling downstream of the aggregator you can add a service activator that invokes a gateway (with a method that returns void), and add an error-channel to it.
The ErrorMessage on the error channel will have a payload that's a MessagingException with cause and failedMessage properties.
Related
I have a spring integration application and I am using message driven channel adapter for consuming the messages. This is the definition of the adapter -
<jms:message-driven-channel-adapter id="messageAdapter" destination="inQueue"
connection-factory="connectionFactory"
error-channel="errorChannel"
concurrent-consumers="${consumer.concurrent-consumers}"
acknowledge="transacted"
transaction-manager="transactionManager"
channel="channel"
auto-startup="true"
receive-timeout="50000"/>
So this message goes to my core channel and then goes through a series of service activators. In between if there is a error than this message is moved to errorChannel where I handle the errors and decide on what needs to be done with this message. For one scenario I want the message to not rollback to the queue, is it possible? I am using 'transacted' in my adapter definition so I am not sure how to drive this behaviour. Any help is greatly appreciated!
You don't describe what the transactionManager bean is. If it's a JmsTransactionManager, remove it and the container will just use local transactions.
Then, the transaction will only roll back if the flow on the error-channel throws an exception. If that error flow exits normally ("consuming" the error), the transaction will not roll back.
If it's some other transaction manager (e.g. JDBC) then remove it and start the JDBC transaction later in the flow (i.e. don't synchronize the JMS and JDBC transactions; again using a local JMS transaction).
I'm working on a worker which is able to treat message from a RabbitMQ.
However, I am unsure of how to accomplish this.
If I receive a message and during my treating an error occurs, how can I put the message into the end of the queue?
I'm trying to using nack or reject, but the message is always re-put in the first position, and other messages stay frozen!
I don't understand why the message has to be put in the first position, I'm trying to "play" with other options like requeue or AllupTo but none of them seem to work.
Thank you in advance!
Documentation says:
Messages can be returned to the queue using AMQP methods that feature a requeue parameter (basic.recover, basic.reject and
basic.nack), or due to a channel closing while holding unacknowledged
messages. Any of these scenarios caused messages to be requeued at the
back of the queue for RabbitMQ releases earlier than 2.7.0. From
RabbitMQ release 2.7.0, messages are always held in the queue in
publication order, even in the presence of requeueing or channel
closure.
With release 2.7.0 and later it is still possible for individual
consumers to observe messages out of order if the queue has multiple
subscribers. This is due to the actions of other subscribers who may
requeue messages. From the perspective of the queue the messages are
always held in the publication order.
Remember to ack your successful messages, otherwise they will not be removed from the queue.
If you need more control over your rejected messages you should take a look to dead letter exchanges.
nack or reject either discard the message or re-queue the message.
For your requirement following could be suitable,
Once the consumer receives the message, just before start processing it, send ack() back to rabbitmq server.
Process the message then after, If found any error in the process then send ( publish ) the same message into the same queue. This will put the message at the back of the queue.
On successful processing do nothing. ack() has been already sent to rabbitmq server. Just take the next message and process it.
I'm trying to implement a job that reads messages from a queue and move them to dead letter when an error occurs. I've tried to use a transacted channel but it always requeue the message.
Is there any automatic way to do that with Spring Batch when an exception is thrown?
Please, refer to Spring AMQP about Dead Lettering.
There is RejectAndDontRequeueRecoverer which you can inject into the RabbitTemplate for the AmqpItemReader. So, all those .receive() will be wrapped with the retry advice and in case of exhausting the message will be rejected to the configured DLX.
I'm implementing Kafka consumer with custom acknowledgement mechanism using spring-integration-kafka.
The code from this example was used.
What I'm trying to achieve is when an exception is thrown, the acknowledgement should not be sent back to Kafka (i.e. no offset commit should be performed) so the next fromKafka.receive(10000) method call will return the same message as the previous one.
But I faced with a problem: even if the acknowledgement isn't sent to Kafka, the consumer knows somehow the offset of the next message and continues to read new messages in spite of the fact that offset value in offset topic remains unchanged.
How to make consumer reread message in case of some failures?
There's not currently any support for re-fetching failed messages
One thing you can do is add retry (e.g. using a request handler retry advice) downstream of the message-driven adapter.
By not acking, the message(s) will be delivered after a restart but not during the current instantiation.
Since messages are prefetched into the adapter, one thing you could do is detect the failure, stop the adapter, drain the prefetched messages and restart.
You could inject a custom ErrorHandler to stop the adapter and signal to your downstream flow that it should ignore the draining messages.
EDIT
There is now a SeekToCurrentErrorHandler.
Here is my channels set-up:
A jdbc message-store backed queue
A bridge connecting the queue to a pub-sub channel
The poller configured on the pub-sub channel is transactional
Now when there is an exception raised in any one of the subscribers then the transaction rolls back and the message is retried for ever. The message is again processed by all the subscribers again. If this is a permanent exception in at least subscriber then the message is not getting processed by none of the other subscribers.
What is the best exception handling strategy here?
I prefer exception handling on the subscriber i.e.only the failing subscriber will retry, other subscribers will process the message and move on.
How can this be implemented in spring integration?
More details here..
If the poller is made transactional and the message fails processing in at least one of the subscribers, then the message is rolled back to the message store and retried. I also configured a jdbc message store for the errorChannel. Every time the message processing fails, the message gets rolled back to the original message store and the error channel message store has one entry for each retry.
If the poller is made non-transactional and the message fails processing in the first subscriber, then the message is put to the error channel, but the second subscriber never gets the message.
It appears that there is something fundamentally wrong.. Is it with my configuration?
http://forum.springsource.org/archive/index.php/t-75000.html
The discussion in the above thread explains the ups and downsides of the framework with respect to pubsub impl.
We chose to go with the below approach:
Pollers will be transactional, meaning all subscribers process the message successfully or none of them. Message will be retried with all subs until all of them complete successfully.
Error handling is the subscribers responsibility
Only system exceptions will be bubbled back to the poller. Business exceptions will be handled by the subscriber and the message will be put to some error channel manually.