I need help interpreting this graph:
It looks like messages are coming in, then being processed, and going back to 0. The graph is not continually rising.
However, the outgoing messages has "--". Does this mean 0?
Does an outgoing message represent a message being read by a service?
If the messages are not being read, then what is happening to them? The dead letter queue has 0 messages.
Yes, an outgoing message represents a message being read by a service.
In your case, I see you've disabled the dead letter queue(in your screenshot, the Dead lettering option is disabled.), so there is no messages in the DLQ. If DLQ is disabled, the messages will be got deleted after expired.
The docs describes them like this:
The number of events or messages received from Service Bus over a specified period.
So, the incoming messages are message that are sent to the service bus. The outgoing messages are messages that are picked up by message processors (your application). So the answer to your question
Does an outgoing message represent a message being read by a service?
is: Yes!
Related
When I try to click getMessages button in RabbitMQManagement I get a response which says it is empty, as in the picture belove.
However, I send 4 messages which are unacked because I make the acknowledgment manually.
If the callback function in my consumer does not/cannot add the message in the database it does not acknowledges the message.
So the queue has persistent=true but here I see that the queue is empty.
How can I prevent this? How can I have the message in the queue until i aknowledge it?
The message is "still in the queue", just not available for consumption.
Rabbitmq knows the messages have been given to consumer, and is waiting for the ack to delete them.
If you reject, you have the option to requeue the message (where it will given for consumption again if a consumer is availavble), or not.
I'd typically recommend configuring a dead letter exchange and reject the message without requeue as a more sturdy solution (you'd have configured a queue bound to the defined dead letter exchange).
In such case the rejected message would be published to the dead letter exchange and finish in the dead letter queue (the queue bound to the dead letter exchange) where you can consume them for review or other secondary processing.
So I have an azure function acting as a queue trigger that calls an internally hosted API.
There doesn't seem to be a definitive answer online on how to handle a message that could not be processed due to issues other than being poisonous.
An example:
My message is received and the function attempts to call the API. The message payload is correct and could be handled however the API/service is down for whatever reason (this time will likely be upwards of 10 minutes). Currently what happens is the message delivery count is reaching its max(10) and then getting pushed to the dead letter queue, which in turn happens for each message after.
I need a way to either not increment the delivery count or reset it upon reaching max. Alternatively I could abandon the peek lock on the message without increment the delivery count as I want to stop processing any message on the queue until the API/service is back up and running.
This way I would ensure that all messages that can be processed will be and will not fall on the dead letter because of connection issues between services.
Any ideas on how to achieve this?
Currently what happens is the message delivery count is reaching its max(10) and then getting pushed to the dead letter queue, which in turn happens for each message after.
As this document states about Exceeding MaxDeliveryCount:
Queues and subscriptions have a QueueDescription.MaxDeliveryCount/SubscriptionDescription.MaxDeliveryCount setting; the default value is 10. Whenever a message has been delivered under a lock (ReceiveMode.PeekLock), but has been either explicitly abandoned or the lock has expired, the message's BrokeredMessage.DeliveryCount is incremented. When the DeliveryCount exceeds the MaxDeliveryCount, the message gets moved to the DLQ specifying the ``MaxDeliveryCountExceeded``` reason code.
This behavior cannot be turned off, but the MaxDeliveryCount can set to a very large number.
According to your requirement, I assumed that you could follow the approaches below to achieve your purpose:
For receiving messages under ReceiveMode.PeekLock
You could specify the Maximum Delivery Count between 1 and 2147483647 under the "SETTINGS > Properties" of your service bus queue on Azure Portal.
For receiving messages under ReceiveMode.ReceiveAndDelete
You could try-catch the exception when your API/service is down, then you could re-send the message to your queue.
I have huge number of messages in azure service bus dead letter queue. When I see the messages, I see that most of the messages are expired.
I want to know what happens when we try to re-submit the expired deadletter queue message back to its original queue?
Can anyone help me out in explaining this ?
Thank you !
I am trying to answer two of your questions below,
when you receive an expired message from the dead letter queue to process/resubmit to main queue(Using ReceiveAsync() to receive a message), the state of the message will be changed to deferred state. So, the message won't be available for receiving in the Dead-Letter queue anymore.
For your question on, what happens to the message when you resubmit, it would be submitted as a new message into the target queue.
We could use FormatDeadLetterPath() method to build the format name for the specified dead letter queue path and create a receiver and retrieve messages from a DLQ. If you’d like to resubmit message back into the main queue, you could create and send a new message based on retrieved message in DLQ. And you could investigate why the message has been dead-lettered via checking DeadLetterReason and DeadLetterErrorDescription properties.
This link explained Dead-Letter Queues with a sample, please refer to it.
I would appreciate your thoughts on this.
I have a node app which subscribes to a RabbitMQ queue. When it receives a message, it checks it for something and then saves it to a database.
However, if the message is missing some information or some other criteria is not yet met, I would like the subscriber to publish the message back onto the RabbitMQ queue.
I understand logically this is just connecting to the queue and publishing the message, but is it really this simple or is this a bad practice or potentially dangerous?
Thanks for your help.
As I point out in the comment, When you create connection with queue, and set autoAck = true, to enable message acknowledge. The message in the queue will be deleted until receive acknowledge.
When the received message meets requirement, then send ack message to this queue, and this message will be deleted from queue. Otherwise, no ack message is sent to queue, this message will stay in the queue.
As for you mentioned in comment, the valid process may take 5 minutes, just set the send ack message as callback function of validation function.
In your question, you describe two criterion for when a message may not be processed:
if the message is missing some information or
some other criteria is not yet met
The first of these appears to be an issue with the message, and it doesn't seem that it makes much sense to re-queue a message that has a problem. The appropriate action is to log an error and drop the message (or invoke whatever error-handling logic your application contains).
The second of these is rather vague, but for the purposes of this answer, we will assume that the problem is not with the message but with some other component in the system (e.g. perhaps a network connection issue). In this case, the consuming application can send a Nack (negative acknowldegement) which can optionally requeue the message.
Keep in mind that in the second case, it will be necessary to shut down the consumer until the error condition has resolved, or the message will be redelivered and erroneously processed ad infinitum until the system is back up, thus wasting resources on an unprocessable message.
Why use a nack instead of simply re-publishing?
This will set the "redelivered" flag on the message so that you know it was delivered once already. There are other options as well for handling bad messages.
What is the difference between Receive and Peek in the context of Azure Service Bus Topics?
Peek: This method enables you to view messages without locking or receiving them.
Receive: Can work in 2 modes: PeekLock (receives the message but keeps it peek-locked until the receiver abandons the message. The maximum timeout is 5 minutes before message expiration) and ReceiveAndDelete (deletes the message after it is received).
So this means you'll use the Peek when your goal is to look at the messages without actually consuming them (maybe you're building a "Queue browser", ... or your process needs to decided if it wants/can consume the message). And you'll use Receive when you're really planning to consume the message and do whatever you need to do with it.
Here is documentation on the PeakLock and ReceiveAndDelete values for ReceiveMode. Note that PeakLock is the default.