This question is similar to others out there with answers, but I think things may have changed with AF v2 (I'm jumping into v2 without v1 experience). It seems that the assembly that AF v2 is integrated with for interacting with ServiceBus is Microsoft.Azure.ServiceBus, which has a class called "Message", which is not quite the same as "BrokeredMessage" found in other microsoft servicebus assemblies. One key difference between the two is that BrokeredMessage (which is referenced in almost all of the documentation, examples, and any other thread out there that I can find) has a .Deadletter() method on it, while Message does not. How can I deadletter a message intentionally when I don't have access to the Client or Receiver that received it?
(I have seen suggestions out there related to just cloning the message, putting on my own "dead letter" queue, and letting AF commit the original - I do not feel this is an adequate solution.)
I have yet to prove it out in the functions environment, but according to the ticket I opened on github (https://github.com/Azure/azure-webjobs-sdk/issues/1986), webjobs v3 supports binding both the Message and MessageReceiver that received it in the function trigger, and then the MessageReceiver can be used to deadletter the message.
This was a real pain to find out as it's still not documented. The API reference on the Azure SDK for .NET project readme even points to the documentation from the old namespace 🤦♂️
I had to search through the source code to find the correct way to get the LockToken.
[FunctionName("MyFunction")]
public static async Task Run([ServiceBusTrigger("myqueue", Connection = "myconnectionstring")] Message message,
ILogger log, MessageReceiver messageReceiver)
{
await messageReceiver.DeadLetterAsync(message.SystemProperties.LockToken);
}
Putting MessageReceiver as a parameter with automatically bind it, no additional set up needed.
Related
How can we move a service bus queue message to the dead letter through service bus queue trigger function
https://github.com/Azure/azure-webjobs-sdk/issues/1986#issuecomment-433960534
In v3, you can bind to the MessageReceiver class, which exposes methods like DeadLetter, Abaondon, Complete, etc. Example:
public static async Task ProcessMessage(
[ServiceBusTrigger("myqueue")] string message, int deliveryCount,
MessageReceiver messageReceiver,
string lockToken)
{
. . .
await messageReceiver.DeadLetterAsync(lockToken);
. . .
}
In this example, the message is bound as a string and the various message properties including lockToken are bound as params. You can also bind the message as a Message Type and access the requisite message properties from there. In v2 the ServiceBus SDK exposed these message methods directly on the BrokeredMessage class itself, but in the latest version of their SDK those methods no longer exist, meaning you have to bind to MessageReceiver to access them.
Edit you also need to set AutoComplete to false when you do this. https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-service-bus-trigger?tabs=csharp#configuration
I have not tried it but what you can do is set the MaxDeliveryCount property on the queue to 1 and then throw an exception in the function as soon as it is triggered. That way the message's delivery count will increase by 1 and Service Bus will automatically dead letter the message.
In latest versions (5.5.1 for me), you must use the ServiceBusMessageActions class from the Microsoft.Azure.WebJobs.ServiceBus namespace. It looks like this:
[FunctionName("MyFunction")]
public static async Task Run(
[ServiceBusTrigger("myQueue", Connection = "myConnection")]
ServiceBusReceivedMessage message,
ServiceBusMessageActions messageActions)
{
...
await messageActions.DeadLetterMessageAsync(message);
...
}
The NuGet package you want to use is Microsoft.Azure.WebJobs.Extensions.ServiceBus.
Reading the Dead Letter Queue messages is by creating the Azure Function Trigger in the Azure portal. In the Function, provide the name of the DLQ as “QueueName/$DeadLetterQueue” as shown in the below image
Note: If you want to access the undelivered message from the Topic then, the syntax of reading the Dead Letter Queue will be “TopicName/Subscriptions/SubscriptionName/$DeadLetterQueue”.
Now, define what should be done with the DLQ messages. Here, as shown in the below screenshot, we are sending the DLQ messages of “myqueue” to the Topic named of “queue” using the Azure Service Bus
In this manner, we can handle the DLQ messages as required very easily using the Azure Functions.
I'm not sure of the proper way to manage the lifespans of the various clients necessary to interact with the Azure Service Bus. From my understanding there are three different but similar clients to manage: ServiceBusClient, a Topic/Queue/Subscription Service, and then a Sender of some sort. In my case, its TopicService and a Sender. Should I close the sender after every message? After a certain amount of downtime? And same with all the others? I feel like I should keep the ServiceBusClient open until the function is entirely complete, so that probably carries over to the Topic Client as well. There's just so many ways to skin this one, I'm not sure where to start to draw the line. I'm pretty sure it's not this extreme:
function sendMessage(message: SendableMessageInfo) {
let client=createServiceBusClientFromConnectionString(connectionString)
let tClient = createTopicClient(client);
const sender = tClient.createSender();
sender.send(message);
sender.close();
tClient.close();
client.close();
}
But leaving everything open all the time seems like a memory leak waiting to happen. Should I handle this all through error handling? Try-catch, then close everything in a finally block?
I could also just use the Azure Function binding, correct me if I'm wrong:
const productChanges: AzureFunction = async function (context: Context, products: product[]): Promise<void> {
context.bindings.product_changes = []
for (let product of product) {
if(product.updated) {
let message = this.createMessage(product)
context.bindings.product_changes.push(message)
}
}
context.done();
}
I can't work out from the docs or source which would be better (both in terms of performance and finances) for an extremely high throughput Topic (at surge, ~100,000 requests/sec).
Any advice would be appreciated!
In my opinion, we'd better use Azure binding or set the client static but not create the client every time. If use Azure binding, we will not consider the problem about close the sender, if set the client static, it's ok too. Both of the solutions have good performance and there is no difference in cost (you can refer to this tutorial for servicebus price: https://azure.microsoft.com/en-us/pricing/details/service-bus/) between these twos. Hope it would be helpful to your question.
I know this is a late reply, but I'll try to explain the concepts behind the clients below in case someone lands here looking for answers.
Version 1
_ ServiceBusClient (maintains the connection)
|_ TopicClient
|_ Sender (sender link)
Version 7
_ ServiceBusClient (maintains the connection)
|_ ServiceBusSender (sender link)
In both version 1 and version 7 of #azure/service-bus SDK, when you use the sendMessages method or the equivalent send method for the first time, a connection is created on the ServiceBusClient if there was none and the new sender link is created.
The sender link remains active for a while and is cleared on its own(by the SDK) if there is no activity. Even if it is closed by inactivity, the subsequent send call even after waiting for a long duration would work just fine since it creates a new sender link.
Once you're done using the ServiceBusClient, you can close the client and all the internal senders, receivers are also closed with this if they are not already closed individually.
The latest version 7.0.0 of #azure/service-bus has been released recently.
#azure/service-bus - 7.0.0
Samples for 7.0.0
Guide to migrate from #azure/service-bus v1 to v7
I have a Azure Function using Queue Trigger, the method looks like this
[FunctionName("ProcessNotifications")]
public static async Task ExecuteAsync(
[QueueTrigger("sharepointnotifications", Connection = "QueueConnectionString")] CloudQueueMessage queueMessage,
ILogger logger,
ExecutionContext context)
{
logger.LogInformation("Message received..");
var azureFuncContext = InitializationService.Initialize(context.FunctionAppDirectory, logger);
await ExecuteInternalAsync(queueMessage, logger, azureFuncContext);
}
I'm using a Azure Storage Queue Service, which acts a queue. What's really interesting: In around 80% of the time, when a new message arrives, it is removed, but the Azure Function is not called. I've also removed all my code except the LogInformation to assure it's not an internal problem, but even then, it doesn't work.
Googling about this issue, I found this thread: https://github.com/Azure/azure-functions-host/issues/3493
But this error seems different, since in this case, the message remains on the queue, but in my case the message is removed, but just nothing is happening.
To assure the problem isn't about the messages, I've stopped the Azure Function and, as expected, the messages are put in the queue.
Unfortunately, I really can't get more insights in the issues, as no warnings or such are thrown. The GitHub issue is still active, therefore I'm inclined to believe there is a problem, but having unreliable queue triggers would be such a big issue, I'm sure it would be fixed by now.
Is there any possibility to get more information about this?
I have also asked this question in the MSDN Azure forums, but have not received any guidance as to why my function goes idle.
I have an Azure function running on a Consumption plan that goes idle (i.e. does not respond to new messages on the ServiceBus trigger queue) despite following the instructions outlined in this GitHub issue:
The configuration for the function is the following json:
{
"ConnectionStrings": {
"MyConnectionString": "Server=tcp:project.database.windows.net,1433;Database=myDB;User ID=user#project;Password=password;Encrypt=True;Connection Timeout=30;"
},
"Values": {
"serviceBusConnection": "Endpoint=sb://project.servicebus.windows.net/;SharedAccessKeyName=SharedAccessKeyName;SharedAccessKey=KEY_HERE",
}
}
And the function signature is:
public static void ProcessQueue([ServiceBusTrigger("queueName", AccessRights.Listen, Connection = "serviceBusConnection")] ...)
Based on the discussion in the GitHub issue, I believed that having either a serviceBusConnection entry OR an AzureWebJobServiceBus entry should be enough to ensure that the central listener triggers the function when a new message is added to the ServiceBusQueue, but that is proving to not be the case.
Can anyone clarify the difference between how those two settings are used, or notice anything else with the settings I provided that might be causing the function to not properly be triggered after a period of inactivity?
I suggest there are several possible causes for this behavior. I have several Azure subs and only one of them had issues with Storage/Service Bus-based triggers only popping up when app is not idle. So far I have observed that actions listed below will prevent triggers from working correctly:
Creating any Storage-based trigger, deleting (for any reason) the triggering object and re-creating it.
Corrupting azure function input parameters by deleting/altering associated objects without recompiling a function
Restarting functions app when one of the functions fails to compile/bind to trigger OR input parameter and hangs may cause same problems.
It has also been observed that using legacy Connection Strings setting for trigger binding will not work.
Clean deploy of an affected function app will most likely solve the problem if it was caused by any of the actions described above.
EDIT:
It looks like this is also caused by setting Authorization/Authentication on the functions app, but I have not yet figured out if it happens in general or when Auth has specific configuration. Tested on affected Azure sub by disabling auth at all - function going idle after 30-40 mins, queue trigger still initiates an execution, though with a delay as expected. I have found an old bug related to this, but it says issue resolved.
I like to change the timespan a Q-message is invisible in the scenario where a webjob is triggered upon arrival of a message in the Queue.
I need this in the dev-stage while debugging. Sometimes an old Q-msg re-appears due to a bug but I already cleaned up the blob-storage-item the message was refering to. Hence WebJob crashes because it tries to get that not existing blob-object.
If I could set that the Q-messages are again visible after 1 min, that would be helpfull.
public partial class Functions
{
public static void ProcessBulkFileQ(
[QueueTrigger(AppConst.Q_BULKFILES)] JobInfo JobInfo,
[Blob(AppConst.CONTAINER_BULKFILES + "/{BlobName}", FileAccess.Read)] Stream InputStream,
[Blob(AppConst.CONTAINER_BULKFILES + "/{BlobName}")] CloudBlockBlob BlobToDelete
)
{
In the latest v1.1.0 release of the SDK we've added the ability for you to add custom QueueProcessors to control various aspects of message processing. You can see an example of what you're looking to do here in our tests.
In that example, for failed messages, you can explicitly set the visibility to your own value based on the error details. You can register custom QueueProcessors by creating your own factory and setting it on JobHostConfiguration.Queues.QueueProcessorFactory.