I have a .NET isolated function with a queue trigger.
When triggering that queue from the storage explorer or from another function using a QueueServiceClient, a new operationId is made up. Thus I cannot correlate it, it seems.
Is it possible to do distributed tracing using W3C standard for Azure Function Queue trigger? I can not find any source on this.
If so, how?
Currently not supported.
Azure Functions team will evaluate this scenario (at some unmentioned point in time) whether or not it can be/will be supported. This has to do with their dependency on the team creating the Azure.Storage.Queues SDK.
https://github.com/Azure/azure-functions-dotnet-worker/issues/1126
Related
I have a requirement where I need to transfer file (20-150 MB) between two systems. For this requirement , is it better to use Durable function instead of Azure data factory (ADF). As per my understanding , ADF execution will costlier as compared to durable functions. Note : durable function trigger is eventGrid trigger. Any suggestion will be helpful. File transfer will be simple pass through, no transformation is involved.
Also, for my requirement even simple azure function will work right instead of durable function? There is no need of function orchestration as file is not processed in batch. Since, file will be processed based on event trigger.
As of my experience, I would like to recommend using Azure functions over ADF is a good idea because of the following reasons:
Azure Data Factory is too expensive. ADF costs way more than azure functions.
Custom Logic: ADF is not built to perform cleansing logics or any custom code. Its primary goal is for data integration from external systems using its vast connector pool
Latency: ADF has much higher latency due to the large overhead of its job framework
Durable function is just related to the maximum execution time of a single call. For "out of the box" functions, that timeout is 10min, for durable functions this limitation gets removed. In this case, where you simply need to copy the data, there might be timeout issue and therefore you can consider the Durable function. Otherwise, simple function should also work fine. Moreover, Durable functions and normal functions share the same billing pattern.
For more details: https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-overview?tabs=csharp
I am having trouble to identify the best "tool" to solve the problem. I am using a python library which publishes to its data via Sever-Sent-Events (SSE) (see https://github.com/wattsight/wapi-python/blob/development/wapi/events.py).
I would like to constantly listen for new events. However, I am not sure which tool in Azure in appropriate. An Azure Function would have to run continously which seems like a misuse, SignalR requires control over the "sender" of events and I don't know if EventHub would be able to manage that job.
Thank you for letting my learn from your experience.
The azure Eventhub is the right service to receive new events. Besides that, it can also provide other benefits like scalability, events storage etc.
And you can also consider using azure function with eventhub trigger. But it has a limitation about the incoming message max size.
I've recently started experimenting with Azure functions and I'd like to get some info on whether the following makes sense and is possible..
How feasible is it to build a normal .NET Core Web API following DDD patterns but replacing the API endpoints with durable azure functions? Is this a possible use case for Azure Functions, to make the web API "serverless"?
And how would the whole thing be structured? Does each Azure function need its own project or can they all be placed in one project?
Thanks
As I wrote in comments why not?
You can define bounded context and deploy to one azure function as microservice for instance service which will be responsible for orders, other azure function of delivery and so on.
Use durable function when you need to orchestrate actions, for instance you have buy flow when in first action you lock products, take payments and unlock so you have kind of dependency on each other.
You can use azure functions with service bus or azure queue storage for event processing.
One thing keep in mind that when you design function you have time limitations is up to 5 minutes on provisioning plan. So when you design newsletter for instance keep in mind that you would need to send email in batches.
I am looking for a simple Scheduler to execute a task inside my Java web application deployed in Azure cloud. I am evaluating Azure functions with TimerTrigger for my requirement. Here, I am planning to define a Azure function with a callback API URL to invoke my application for executing the task inside my application.
I have some queries in this approach. Can anyone help me If you are familiar with Azure functions please?
1) Is it possible to initiate/reschedule/cancel a Azure TimerTrigger function from a Java application through API at runtime?
2) If yes, Is it possible to pass a call back URL to the timer Trigger?
3) Is there any known drawback in using Azure functions?
Thanks!
TimerTriggers don't have an api to control this (you could try to hack one in by uploading a new function.json with the schedule you want and whether or not the timer is disabled, but I don't recommend that at all).
Instead, I'd suggest using a QueueTrigger. This would allow you to pass the function any data you need in the queue item (the callback url) and you could add items to the queue with a visibility timeout in order to create your schedule. If you need to cancel pending executions, just remove the item(s) from the queue. The function is also more durable - if a queue item fails, it will automatically retry (unlike timers).
3) is way too broad of a question to have an answer.
I'm simply trying to work out how best to retrieve messages as quickly as possible from an Azure Service Bus Queue.
I was shocked that there wasn't some way to properly subscribe to the queue for notifications and that I'm going to have to poll. (unless I'm wrong in which case the documentation is terrible).
I got long polling working, but checking a single message every 60 seconds looks like it'll cost around £900 per month (again, unless I've misunderstood that). And if I add a redundant/second service to poll it'll double.
So I'm wondering what the best/most cost efficient way of doing it is.
Essentially I just want to take a message from the queue, perform an API lookup on some internally held data (perhaps using hybrid services?) and then perhaps post a message back to a different queue with some additional information .
I looked at worker roles(?) -- is that something that could do it?
I should mention that I've been looking at doing this with node.js.
Check out these videos from Scott Hanselman and Mark Simms on Azure Queues.
It's C# but you get the idea.
https://channel9.msdn.com/Search?term=azure%20queues%20simms#ch9Search
Touches on:
Storage Queues vs. Service Bus Queues
Grabbing messages in bulk vs. one by one (chunky vs. chatty)
Dealing with poison messages (bad actors)
Misc implementation details
Much more stuff i can't remember now
As for your compute, you can either do a VM, a Worker Role (Cloud Services), App Service Webjobs, or Azure Functions.
The Webjobs SDK and Azure Functions bot have a way to subscribe to Queue events (notify on message).
(Listed from IaaS to PaaS to FaaS - Azure Functions - if such a thing exists).
Azure Functions already has sample code provided as templates to do all that with Node. Just make a new Function and follow the wizard.
If you need to touch data on-prem you either need to look at integrating with a VNET that has site-to-site connectivity back to your prem, or Hybrid Connections (App Service only!). Azure Functions can't do that yet, but every other compute is a go.
https://azure.microsoft.com/en-us/documentation/articles/web-sites-hybrid-connection-get-started/
(That tutorial is Windows only but you can pull data from any OS. The Hybrid Connection Manager has to live on a Windows box, but then it acts as a reverse proxy to any host on your network).
To deal with Azure ServiceBus Queue easily, the best option seems to be Azure Webjob.
There is a ServiceBusTrigger that allows you to get messages from an Azure ServiceBus queue.
For node.js integration, you should have a look at Azure Function. It is built on top of the webjob SDK and have node.js integration :
Azure Functions NodeJS developer reference
Azure Functions Service Bus triggers and bindings for queues and topics
In the second article, there is an example on how get messages from a queue using Azure Function and nodejs :
module.exports = function(context, myQueueItem) {
context.log('Node.js ServiceBus queue trigger function processed message', myQueueItem);
context.done();
};