I am new to message queues. I read the documentation of rabbitmq today, about the setting of "prefetch_count", this is to set the number of unpacked messages that consumers can have the most. I looked at the official documentation and said that the order is guaranteed even if re-enqueued. Assuming a single consumer, the producer sends the message 1->2->3->4->5->6->7->8->9->10, if the prefetch_count is set to 30, then if the 5 message fails and it does't has manual ack, this failed message is requeue. However, 6, 7, 8, 9, 10 are successfully consumed and all acked. At this time, although the messages seem to be pushed to consumers in order, the order of consuming messages maybe 1->2->3-> 4->6->7->8->9->10->5 order. I have this question:
In the scenario where the order of consumption needs to be guaranteed, how do you all do the above situations in the real scenario?
If the order of consumption needs to be guaranteed, how to ensure that the next message will not be consumed if the previous message is not successfully consumed?
Is it necessary to persist the message to MySql? For example, record the message of consumption failure
Here is the rabbitmq doc about the order.
Message ordering guarantees
Section 4.7 of the AMQP 0-9-1 core specification explains the conditions under which order is guaranteed: messages published in one channel, passing through one exchange and one queue, and one outgoing channel will be received in the same order that they were sent. RabbitMQ offers stronger guarantees since release 2.7.0.
Messages can be returned to the queue using AMQP methods that feature a requeue parameter (basic. recover, basic. reject, and basic. nack), or due to a channel closing while holding unacknowledged messages. Any of these scenarios caused messages to be queued at the back of the queue for RabbitMQ releases earlier than 2.7.0. From RabbitMQ release 2.7.0, messages are always held in the queue in publication order, even in the presence of queueing or channel closure.
With release 2.7.0 and later it is still possible for individual consumers to observe messages out of order if the queue has multiple subscribers. This is due to the actions of other subscribers who may require messages. From the perspective of the queue the messages are always held in the publication order.
Related
I have a peculiar type of problem statement to be solved.
Configured RabbitMQ as message broker and its working but when there is failure in process in consume I'm now acknowledging with nack but it blindly re-queues with whatever already came in as payload but i want to add some-more fields to it and re-queue with simpler steps
For Example:
When consume gets payload data from RabbitMQ it will then process it and try to do some process based on it in multiple host machines, but due to some thing if one machine not reachable i need to process that alone after some time .
Hence I'm planning to re-queue failed data with one more fields with machine name again back to queue so it will be processed again with existing logic itself.
How to achieve this ? Can someone help on me
When a message is requeued, the message will be placed to its original position in its queue, if possible. If not (due to concurrent deliveries and acknowledgements from other consumers when multiple consumers share a queue), the message will be requeued to a position closer to queue head. This way you will end up in an infinite loop(consuming and requeuing the message). To avoid this, you can, positively acknowledge the message and publish it to the queue with the updated fields. Publishing the message puts it at the end of the queue, hence you will be able to process it after some time.
Reference https://www.rabbitmq.com/nack.html
We have recently experienced unexpected behaviour with our application that is powered by RabbitMQ.
RabbitMQ version is 3.6.12 and we are using .NET Client 5.0.1
The application subscribes to two queues, one for commands, and another for events - we also use manual acknowledgements.
Our application is configured to have 7 consumers. Each has its own channel(IModel) and each has its own EventingBasicConsumer
We end up processing messages when EventingBasicConsumer.Received is fired.
Our application must process messages as close as possible to when they are routed onto the queues and to date we have not had issues.
However recently, we have seen that when one of our messages being processed takes a long time to complete, it delays when another message is to be processed although there are many consumers available (6) that are not busy.
Note we have observed that this issue does not happen when an application is only subscribing to a single queue, it becomes an issue when there is multiple queues involved.
This is best illustrated using the following example:
We have a simple consuming application that subscribes to two queues,
one for commands and one for events. This application have 7
consumers, each with their own channel and EventingBasicConsumer We
start a simple publishing application, that publishes 20 messages, a
second apart. Every message is an event so is published to the event
queue except for the 5th and 10th messages, which are commands and
sent to the command queue. Note that every event is processed without
delay whereas commands take 30 seconds
The following table describes what we are observing in relation to assigning multiple channels to messages across multiple queues:
Once Message5 completes after 30 seconds with C1, then Messaqe9 is assigned immediately to C1 and is processed without delay
Once Message10 completes after 30 seconds with C2, then Messaqe11 is assigned immediately to C2 and is processed without delay
Hence, to us it looks like the assignment of channels is done independently per queue - meaning you can have delayed execution if some messages take a long time to process.
Is it possible that when multiple consumers are subscribing to multiple queues, RabbitMQ can assign a message to be handled by a consumer that is busy even if there are consumers that are currently idle?
Is there any documentation that explains the RabbitMQ algorithm that selects which consumers EventingBasicConsumer.received fires from a collection of consumers?
We have fixed this issue.
In the RMQ documentation (https://www.rabbitmq.com/api-guide.html#consuming) we came across the following:
"Each Channel has its own dispatch thread. For the most common use case of one Consumer per Channel, this means Consumers do not hold up other Consumers. If you have multiple Consumers per Channel be aware that a long-running Consumer may hold up dispatch of callbacks to other Consumers on that Channel.”
In our code, we had 2 consumers per channel, meaning consumers could hold up other consumers.
We changed to have one consumer per channel and that fixed the issue.
I'm working on a worker which is able to treat message from a RabbitMQ.
However, I am unsure of how to accomplish this.
If I receive a message and during my treating an error occurs, how can I put the message into the end of the queue?
I'm trying to using nack or reject, but the message is always re-put in the first position, and other messages stay frozen!
I don't understand why the message has to be put in the first position, I'm trying to "play" with other options like requeue or AllupTo but none of them seem to work.
Thank you in advance!
Documentation says:
Messages can be returned to the queue using AMQP methods that feature a requeue parameter (basic.recover, basic.reject and
basic.nack), or due to a channel closing while holding unacknowledged
messages. Any of these scenarios caused messages to be requeued at the
back of the queue for RabbitMQ releases earlier than 2.7.0. From
RabbitMQ release 2.7.0, messages are always held in the queue in
publication order, even in the presence of requeueing or channel
closure.
With release 2.7.0 and later it is still possible for individual
consumers to observe messages out of order if the queue has multiple
subscribers. This is due to the actions of other subscribers who may
requeue messages. From the perspective of the queue the messages are
always held in the publication order.
Remember to ack your successful messages, otherwise they will not be removed from the queue.
If you need more control over your rejected messages you should take a look to dead letter exchanges.
nack or reject either discard the message or re-queue the message.
For your requirement following could be suitable,
Once the consumer receives the message, just before start processing it, send ack() back to rabbitmq server.
Process the message then after, If found any error in the process then send ( publish ) the same message into the same queue. This will put the message at the back of the queue.
On successful processing do nothing. ack() has been already sent to rabbitmq server. Just take the next message and process it.
Quite new to RabbitMQ and I'm trying to see if I can achieve what I need with it.
I am looking for the Worker Queues pattern but with one caveat. I want to have only a single worker running concurrently per routing key.
An example for clarification:
If i send the following messages with routing keys by order: a, a, b, c, I want to have only 3 workers running concurrently. When the first a message is received a worker picks it up and handles it.
When the next a message is received and the previous a message is still handled (not acknowledged) the new a message should wait in queue. When the b and c messages are received they each get a worker handling them. When the first a message is acknowledged any worker can pick up the next a message.
Would that pattern be possible using RabbitMQ in a natural way (without writing any application code on my side to handle the locking and stuff...)
Edit:
Another clarification. All workers can and should handle all messages, and I don't want to have a queue per Worker as I want to share the load between them, and the Publisher doesn't know which Worker should process the message. But I do want to make sure that no 2 Workers are working on messages sharing the same key at the same time.
For example, if I have a Publisher publishing messages with a userId field, I want to make sure no 2 Workers are handling messages with the same userId at the same time.
Edit 2
Expanding on the userId example. Let's say I have a single Publisher and 3 Workers. The publisher publishes messages like these: { userId: 1, text: 'Hello' }, with varying userIds. My 3 Workers all do the same thing to this messages, so I can have any of them handle the messages coming in. But what I'm trying to achieve is to have only a single worker processing a message from a certain user at the same time. If a Worker has received a message with userId 1 and is still processing it, and another message with userId 1 is received I want to make sure no other Worker picks up that message. But other messages coming in with different userIds should be processed by other available Workers.
userIds are not known beforehand, and the publisher doesn't know how many workers are or anything specific about them, he just wants to schedule the messages for processing.
what your asking is not possible with routing keys, but is built into queues with a few settings.
if you define "queue_a" for a messages, "queue_b" for b messages, etc, you can then have as many consumers connect to it as you want.
RabbitMQ will only deliver a given message to a single consumer of a given queue.
The way it works with multiple consumers on a single queue is basic round-robin style dispatch of the messages. that is, the first message will be delivered to one of the consumers, and the next message (assuming the first consumer is still busy) will be delivered to the next consumer.
So, that should satisfy the need to deliver the message to any given consumer of the queue.
To ensure your messages have an equal chance of getting to any of the consumer (and are not all delivered to the same consumer all the time), there are a few other settings you should put in place.
First, make sure to set the message consumer no ack setting to false (sometimes called "auto ack"). This will force you to ack the message from your code.
Lastly, set the "consumer prefetch" limit of the consumer to 1.
With this combination of settings, a single consumer will retrieve a single message and begin working on it. While that consumer is working, any message waiting in the queue will be delivered to other consumers if any are available. If there are none available, the message will wait in the queue until a consumer is available.
With this, you should be able to achieve the behavior you are wanting, on a given queue.
...
Keep in mind this only applies to queues, though. routing keys cannot be managed this way. all matched routing keys from an exchange will cause a copy of the message to be sent to the destination queue.
I would appreciate some clarification about the meaning of "send-timeout" configuration parameter for an Aggregator. Based on Spring documentation, this configuration is The timeout interval for sending the aggregated messages to the output or reply channel. Optional.
Now, based on my understanding the Aggregator is a passive component and only decides to send a message or not based on the result of the release strategy after the receipt of a message; it won't release messages based on timeout events, for that, a separate Reaper component is needed. Is this correct?
Assuming, the send-timeout is the maximum amount of time that the Aggregator can spend sending the completed group of messages to the output channel. What would happen if time runs out (due to this parameter set up) while sending a message. How will the Aggregator handle that message group that was ready to release, started to be sent but never finished? Will it be marked complete?
Thanks
This is a fairly commonly misunderstood attribute. In many places (but not all) we have explained it clearly in the XSD and docs.
Bottom line is it rarely is applied. It only applies when the output channel can block. For example, if the output-channel is a QueueChannel with a capacity and the queue is full; it is the time we will wait to send the message to the channel.
If the output channel is, for example, a DirectChannel, it never applies.
If it does apply, the exception will be thrown back to the caller and the group will remain. Attempts to re-release such groups will occur if you configure a MessageGroupStoreReaper; if the group is still eligible for release, the reaper will again attempt to send the group to the output channel.
The "stuck" group will also be released if new messages arrives for the same group and the release strategy still considers the group eligible (e.g. it uses size >= n rather than size == n).
BTW, while the aggregator is generally a passive component, we did introduce the group-timeout (and group-timeout-expression) in 4.0 which allows partial groups to be released after a timeout, even without a reaper.
However, if such a release fails to happen because of the send-timeout, a new release attempt will only be made if a reaper is configured.