In a pervious project I have managed to setup a Alert rule that looks at poison queue message count and alerts using a webhook into slack when something is in the queue (once per day).
I was trying to find where this exists in Azure as it looks like things have moved around. If this is not a feature provided by azure could you give guidance on what's the best route in implementing something similar.
Thanks
I was trying to find where this exists in Azure as it looks like things have moved around. If this is not a feature provided by azure could you give guidance on what's the best route in implementing something similar.
As far as I know, currently there is no a feature provided by azure to send the a alert rule to check the poison queue count.
You need write you own logic to achieve this requirement.
I suggest you could considerusing webjob/azure function timer trigger or queue trigger.
If you want to check the poison count every 5 minutes(for example), you could choose timer trigger.
Then in the timer trigger method, you could use ApproximateMessageCount method to get the queue messages' count.
At last, you could use sendgrid to send the notification email to special account.
Codes:
//get the storage account from the connection string
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
//instantiate the client
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
CloudQueue q = queueClient.GetQueueReference("queue-poison");
q.FetchAttributes();
var qCnt = q.ApproximateMessageCount;
If you want to get the count when the new queue message has added into the poison queue. You could choose queue trigger. The codes is as same as the timer trigger, just change the parameters.
Related
Problem: I have a Azure Topic Topic1 with Subscriptions Subscription-A, Subscription-B, Subscription-C
Each Subscription is listen to by an Azure function and it uses the message to perform unique action like charging customer or Updating another subsystem etc.
If the execution in one of the functions results in error , I would like to requeue the message into that specific subscription for execution after a delay say 1 min. not right way. [ Currently, If the function results in error, message is retried right away couple of times before pushing to dead letter. Looks like default behavior by Azure service bus and functions]
I can not resend a message to Topic1 as that message would be read by all the 3 subscriptions. I don't want this as the other message were processed successfully by the respective subscription/functions. This requeued message should be filtered out by other subscriptions on this topic.
Any ideas or suggestion would be of great help.
I had multiple Azure Functions which are deployed in different Azure Function slots. The functions are called based on the CloudStorage QueueMessage as below.
// Get storage account
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(Microsoft.Azure.CloudConfigurationManager.GetSetting("StorageConnectionString"));
// Next, create a queue client
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
// Then retrieve a reference to a queue – here’s where you give your queue a name
CloudQueue queue = queueClient.GetQueueReference(queueName);
// Create the queue if it doesn’t already exist
queue.CreateIfNotExists();
// Penultimate step – create a message and add it to the queue.
CloudQueueMessage message = new CloudQueueMessage(bordereauxId.ToString());
// Finally, add your message to the queue
queue.AddMessage(message);
// Fetch the queue attributes.
queue.FetchAttributes();
Is there any way to queue the message based on the Slots. For example, if the user is using development site then the function queue should call the function that belongs to Devlopment Slot using the same storage account.
As #MurrayFoxcroft suggested, you should use per-slot App Settings to configure your storage accounts per App Slot.
To add, you should replace all the code you are quoting with an Storage Queue output binding:
{
"type": "queue",
"direction": "out",
"name": "$return",
"queueName": "outqueue",
"connection": "StorageConnectionString",
}
Then the function will just return the queue message (you can use out params too).
StorageConnectionString should be an app setting pinned to App Slot.
You can determine if your function is running in a slot by using the following:
var slot = System.Environment.GetEnvironmentVariable("APPSETTING_WEBSITE_SLOT_NAME", EnvironmentVariableTarget.Process);
You can then use the result to direct your code down the correct path.
However, where possible, I'd avoid coding in logic to handle different scenarios based on slot. Try to drive your logic from configuration where possible. For example, in your AppSettings you can configure sticky settings (Slot Settings) to provide configuration per slot.
When running locally you should be using the Azure Functions Core Tools for debugging. See this link. Put your sample app settings in localsettings.json to test your function. Change them appropriately or use multiple files to simulate your slots.
I got the solution by making the queue name based on the slots
So in my console application now the queue name is based on the slot I want to call and added in the config file.
// Then retrieve a reference to a queue – here’s where you give your queue a name
CloudQueue queue = queueClient.GetQueueReference(queueName + "_" + slotname);
And the queue name in the function is defined based on the slot application settings
Hence in the Azure function method the queue name will be defined as below
public static void Run([QueueTrigger("%slot_queue_name%", Connection = "AzureWebJobsStorage")]string myQueueItem)
Where the "slot_queue_name" will be defined in the Azure Function slot Application settings.
I am using azure service bus topic and subscription mechanism and want to process the messages which are all in the dead letter queue.
Moreover i want to process the messages via azure web job in C# and send them back to queue. So i want to know how I can process the messages on the deadletter queue through my application?
When a message is deadlettered it goes onto the dead letter queue for the subscription from which it was read. You access that just like you'd access the original subscription except that you append /$DeadLetterQueue to the subscription name.
Moreover i want to process the messages via azure web job in C# and send them back to queue.
As spodger pointed that the path of your deadletter subscription would be:
{topic-path}/Subscriptions/{subcription-name}/$DeadLetterQueue
You could use the WebJobs SDK for Service Bus and leverage the ServiceBusTrigger to access your dead letter queue message(s) as follows:
public void ProcessDeadletterQueue(
[ServiceBusTrigger("topicName", "subscriptionName/$DeadLetterQueue")] BrokeredMessage message)
{
//TODO:
}
For more details, you could refer to here.
When a message is dead-lettered from a Service Bus Entity(Queue or Topic Subscription), it will be moved to the dead-letter path of the same entity. The reason for dead-lettering will be available in the message's custom properties DeadLetterReason and DeadLetterErrorDescription.
In order to receive the dead-letter messages,
string path = Microsoft.ServiceBus.Messaging.SubscriptionClient.FormatDeadLetterPath(topicPath, subscriptionName);
var subscriptionClient = SubscriptionClient.CreateFromConnectionString(connectionString, path);
BrokeredMessage message = subscriptionClient.Receive();
I've create an Azure Function to listen to an Azure IoT Hub instance. When I post message to the IoT Hub I set a property like so:
Message iotMessage = new Message([myMessage]);
iotMessage.Properties.Add("Type", [MessageType]);
At the Azure Function end I want the Azure function only to receive/process messages that have a Type property and where the Type property equals "MessageType1".
I cannot see a way to do this in an Azure function. Can someone advise if this is possible?
EDIT: this appears to be what you're looking for:
https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-csharp-csharp-process-d2c#add-a-queue-to-your-iot-hub-and-route-messages-to-it
Your condition would be Type=MessageType1 and you would have the function trigger off of the output queue.
What type of input binding are you using for your Azure Function?
AFAIK, this isn't currently possible in one step. However, there are a few options you have:
connect IoT Hub to a ServiceBus Topic/Subscription, which allows you to do some filtering based on properties. Trigger on the subscription which filters by MessageType1
have a function dedicated to filtering IoT Hub messages. When it matches a MessageType1 message, put that message into a queue. This queue triggers another function which only processes matched messages
I send a message to ASB, some data (a long task) will start being processed, but the processing fails miserably (someone turns off the computer/out of memory/whatever). How should I handle this situation? Like putting the queue back on the bus? Do I need to create my own monitoring/requeueing unit?
Azure ServiceBus has an internal retry policy. If a message fails to deliver, it will be send back to the queue automatically.
When creating a queue or a topic/subscription you can specify the MaxDeliveryCount.
QueueDescription.MaxDeliveryCount
SubscriptionDescription.MaxDeliveryCount
Default value is 10. A message is automatically deadlettered after this number of deliveries.