How to move a service bus messge to deadletter in service bus queue trigger function - azure

How can we move a service bus queue message to the dead letter through service bus queue trigger function

https://github.com/Azure/azure-webjobs-sdk/issues/1986#issuecomment-433960534
In v3, you can bind to the MessageReceiver class, which exposes methods like DeadLetter, Abaondon, Complete, etc. Example:
public static async Task ProcessMessage(
[ServiceBusTrigger("myqueue")] string message, int deliveryCount,
MessageReceiver messageReceiver,
string lockToken)
{
. . .
await messageReceiver.DeadLetterAsync(lockToken);
. . .
}
In this example, the message is bound as a string and the various message properties including lockToken are bound as params. You can also bind the message as a Message Type and access the requisite message properties from there. In v2 the ServiceBus SDK exposed these message methods directly on the BrokeredMessage class itself, but in the latest version of their SDK those methods no longer exist, meaning you have to bind to MessageReceiver to access them.
Edit you also need to set AutoComplete to false when you do this. https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-service-bus-trigger?tabs=csharp#configuration

I have not tried it but what you can do is set the MaxDeliveryCount property on the queue to 1 and then throw an exception in the function as soon as it is triggered. That way the message's delivery count will increase by 1 and Service Bus will automatically dead letter the message.

In latest versions (5.5.1 for me), you must use the ServiceBusMessageActions class from the Microsoft.Azure.WebJobs.ServiceBus namespace. It looks like this:
[FunctionName("MyFunction")]
public static async Task Run(
[ServiceBusTrigger("myQueue", Connection = "myConnection")]
ServiceBusReceivedMessage message,
ServiceBusMessageActions messageActions)
{
...
await messageActions.DeadLetterMessageAsync(message);
...
}
The NuGet package you want to use is Microsoft.Azure.WebJobs.Extensions.ServiceBus.

Reading the Dead Letter Queue messages is by creating the Azure Function Trigger in the Azure portal. In the Function, provide the name of the DLQ as “QueueName/$DeadLetterQueue” as shown in the below image
Note: If you want to access the undelivered message from the Topic then, the syntax of reading the Dead Letter Queue will be “TopicName/Subscriptions/SubscriptionName/$DeadLetterQueue”.
Now, define what should be done with the DLQ messages. Here, as shown in the below screenshot, we are sending the DLQ messages of “myqueue” to the Topic named of “queue” using the Azure Service Bus
In this manner, we can handle the DLQ messages as required very easily using the Azure Functions.

Related

webjob QueueTrigger does not delete message from the queue

.NET Core 2.2, WebJobs SDK 3.0
I have a webjob that takes the messages from a queue. A standard QueueTrigger like this
public void ProcessQueueMessage(
[QueueTrigger("%WebJobs:WorkerQueueName%")] CloudQueueMessage queueMessage,
ILogger log)
At the end of the process I write the message to another queue (archive).
The function finishes successfully but the message is kept in the source queue
In Storage Explorer I see this (in this example I had 3 messages pending)
and the message is dequeued once again after 10 minutes.
How can I make it so the message is dequeued when my function is successful?
Btw my queue config is
BatchSize 1
MaxDequeueCount 2
MaxPollingInterval 00:00:04
VisibilityTimeout 02:00:00
The SDK should already handle that.
The message will be leased (or become invisible) for 30 seconds by default. If the job takes longer than that, then the lease will be renewed. The message will not become available to another instance of the function unless the host crashes or the function throws an exception. When the function completes successfully, the message is deleted by the SDK.
When a message is retrieved from the queue, the response includes the message and a pop receipt value, which is required to delete the message. The message is not automatically deleted from the queue, but after it has been retrieved, it is not visible to other clients for the time interval specified by the visibilitytimeout parameter. The client that retrieves the message is expected to delete the message after it has been processed, and before the time specified by the TimeNextVisible element of the response, which is calculated based on the value of the visibilitytimeout parameter. The value of visibilitytimeout is added to the time at which the message is retrieved to determine the value of TimeNextVisible.
So you shouldn't need to write any special code for deleting message from queue.
For more details you could refer to this article and this one.
It turns out that I was using the queueMessage object I got as a parameter to directly put it in another queue which probably confused the SDK.
public void ProcessQueueMessage(
[QueueTrigger("%WebJobs:WorkerQueueName%")] CloudQueueMessage queueMessage,
ILogger log)
So I changed that and I create a new CloudQueueMessage object before I put it in another queue.
var newMessage = new CloudQueueMessage(queueMessage.AsString);
Now the message is properly deleted from the queue when my function returns.

Can I use Azure's ServiceBusTrigger with ReceiveMode?

I want to handle dead letters of my azure service bus queue. For this I thought I could use a azure function with a ServiceBusTrigger:
[FunctionName("DoSomethingWithDeadLetters")]
public static async Task DoSomething(
[ServiceBusTrigger("myqueue/$DeadLetterQueue", Connection = "ServiceBusConnection")]
Message deadLetter)
Unfortunately with this the messages are processed in PeekLockMode.
If my code is faulty and the function fails with an exception the dead-letter-message is abandoned.
The ServiceBusTrigger will start the function again short time later... of course it will fail due to the same exception and abandon the message again... a endless loop is born.
Is it possible to use the ServiceBusTrigger with ReceiveAndDelete mode? This would be sufficient for my dead letter handling.

how to exclude queue messages like azure storage explorer?

Our code is just a copy paste from some online tutorial in getting the messages from an azure storage queue.
public int? GetQueueMessageCount(CloudQueue queue, TextWriter textWriter)
{
int? messageCount;
try
{
queue.FetchAttributes();
// Retrieve the cached approximate message count.
messageCount = queue.ApproximateMessageCount;
}
catch (Exception exception)
{
LogHelper.LogInfo(logger, textWriter, $"GetQueueMessageCount failed for {queue.Name}." + exception);
throw;
}
return messageCount;
}
However, we found that randomly some messages may get stuck in the queue and our queue trigger never got fired.
public static void ProcessUnitsForCacheItem(
[QueueTrigger(QueueClient.RefreshUnitsQueue)] string projectUnitsMessage, TextWriter textWriter)
When I open my queue with storage explorer, I can see the explorer will not show any message, instead just display a status text "displaying 0 of 199 messages". So storage explorer must somehow know that these messages are not right (expired or something).
Is there some status I can retrieve to see the status of the message or anyone know how storage explorer decide to show a message or not?
Storage explorer shows info exactly what it retrieves from Storage account/emulator.
displaying 0 of 199 messages means the messages are invisible for now because they have been dequeued and being processed, it's a feature of queue message and handled by Storage service automatically once your queue trigger gets messages from a queue. See Storage queue doc.
Typically, when a consumer retrieves a message via Get Messages, that message is usually reserved for deletion until the visibilitytimeout interval expires, but this behavior is not guaranteed. After the visibilitytimeout interval expires, the message again becomes visible to other consumers.
As for the problem
get stuck in the queue and our queue trigger never got fired
If I understand correctly, your code from some tutorial is a custom queuetrigger, which may have no guarantee on the behavior. Have a look at Azure Function Queuetrigger example.

How to intentionally deadletter a message is azure functions v2

This question is similar to others out there with answers, but I think things may have changed with AF v2 (I'm jumping into v2 without v1 experience). It seems that the assembly that AF v2 is integrated with for interacting with ServiceBus is Microsoft.Azure.ServiceBus, which has a class called "Message", which is not quite the same as "BrokeredMessage" found in other microsoft servicebus assemblies. One key difference between the two is that BrokeredMessage (which is referenced in almost all of the documentation, examples, and any other thread out there that I can find) has a .Deadletter() method on it, while Message does not. How can I deadletter a message intentionally when I don't have access to the Client or Receiver that received it?
(I have seen suggestions out there related to just cloning the message, putting on my own "dead letter" queue, and letting AF commit the original - I do not feel this is an adequate solution.)
I have yet to prove it out in the functions environment, but according to the ticket I opened on github (https://github.com/Azure/azure-webjobs-sdk/issues/1986), webjobs v3 supports binding both the Message and MessageReceiver that received it in the function trigger, and then the MessageReceiver can be used to deadletter the message.
This was a real pain to find out as it's still not documented. The API reference on the Azure SDK for .NET project readme even points to the documentation from the old namespace 🤦‍♂️
I had to search through the source code to find the correct way to get the LockToken.
[FunctionName("MyFunction")]
public static async Task Run([ServiceBusTrigger("myqueue", Connection = "myconnectionstring")] Message message,
ILogger log, MessageReceiver messageReceiver)
{
await messageReceiver.DeadLetterAsync(message.SystemProperties.LockToken);
}
Putting MessageReceiver as a parameter with automatically bind it, no additional set up needed.

How to abandon or deadletter messages on ServiceBus BrokeredMessage level on AzureFunction V2?

I am encountering one major road block issue when trying to use ServiceBusTrigger in azureFunction. I am trying to abandon, or deadletter, a service bus message in V2 ServiceBusTrigger, How can I do so? I've tried the following solution, but I didn't get anywhere.
Here is the codeSample I used:
public async static Task Run(Message myQueueItem, TraceWriter log, ExecutionContext context)
{
log.Info($"C# ServiceBus queue trigger function processed message delivery count: {myQueueItem.SystemProperties.DeliveryCount}");
QueueClient queueClient = new QueueClient("[connectionstring]","[queueName]");
////await queueClient.DeadLetterAsync(myQueueItem.SystemProperties.LockToken);
await queueClient.AbandonAsync(myQueueItem.SystemProperties.LockToken);
}
Solution 1: I tried to substitute Message myQueueItem for BrokeredMessage like in V1, I then can call myQueueItem.Abandon, or deadletter, on the message lever. However It came back with exception:
Microsoft.Azure.WebJobs.Host: Exception binding parameter 'myQueueItem'. System.Private.DataContractSerialization: There was an error deserializing the object of type Microsoft.ServiceBus.Messaging.BrokeredMessage. The input source is not correctly formatted. System.Private.DataContractSerialization: The input source is not correctly formatted."
At least I can go one step further. to
solution 2. Solution 2: is to use:
QueueClient queueClient = new QueueClient("[connectionstring]","[queueName]");
////await queueClient.DeadLetterAsync(myQueueItem.SystemProperties.LockToken);
await queueClient.AbandonAsync(myQueueItem.SystemProperties.LockToken);
I can use the lock provided in the Message Object, however, when I try to send it with queueClient, It said the message gone from the queue. or no longer available.
Can anybody let me know if i am on the right track? If I am not, please kindly guide me in the right track.
Service Bus messages are automatically completed or abandoned by Azure Functions runtime based on the success/failure of the function call, docs:
The Functions runtime receives a message in PeekLock mode. It calls Complete on the message if the function finishes successfully, or calls Abandon if the function fails.
So, the suggested way to Abandon your message is to throw an exception from function invocation.

Resources