I have a background worker that listens to a service bus (Azure Service bus) for messages.
Each message stands for an async task that the service should work on, but for the case that no event is reaching the bus, I also want to trigger the service automatically each day.
The service bus is currently triggered by user events that are generated in different APIs.
This works fine, but who should trigger my service with a certain schedule?
I could of course write a second service that sends a message to the bus each week, but it feels kind of overkill to have a service running only for this task.
I am wondering if there is a better solution how I could do this? Even an Azure Function seems overkill for me...
How would you address this issue?
Azure Functions are ideal because you can create both a timer triggered, and service bus triggered function in the same Function App.
If you feel it is excessive, I suggest the other option is to use the Azure Web jobs which can run in the App Services.
You can have timer-triggered Web jobs and use that Web job SDK to trigger them whenever there is a message in the Azure Service Bus.
Refer to the Scheduled Webjobs and Service Bus triggered webjobs for more information.
Related
I have an Azure function processing a file uploaded by end user which could take a long time. I'd like to notify the end user when process is completed. I'm not clear whether Azure event grid topic, event hub, service bus or other services would be best for this?
I'd like the user to be able to register how they would like to get notified as well. Is there an Azure service that would accomodate this?
You can use Azure Logic Apps to run your functions at a specific time and notify end users by email when finished for instance. Azure Logic Apps are used to run automated workflows that integrate your apps, data, services, and systems.
Link that could be helpful: Create and run your own code from workflows in Azure Logic Apps by using Azure Functions
You can use Signal R service by Azure. It will be useful in case you want to notify user on any real-time update from the server.
you can refer Signal R using Azure Function to implement it with azure function.
I have a function set up with ServiceBusTrigger which runs when it is deployed and then keeps running while I'm testing and sending messages to a topic. However, if I wait an hour or so and then send more messages they are not processed until I either restart the function app or disable and reenable the actual function.
How can I change this so that the function is always "on"?
Sounds like you are using a dedicated App Service Plan. Make sure you have "Always On" enabled. You need to have at least a "Basic" plan. If you don't want to pay for a basic plan, I suggest you use a consumption plan.
https://github.com/Azure/Azure-Functions/wiki/Enable-Always-On-when-running-on-dedicated-App-Service-Plan
How do I turn on "always-on" for an Azure Function?
We are currently using Azure more like IAAS, rather than cloud services.
As a start, I would like to utilise Azure Messaging Queues to process some database actions and Web API calls.
I am assuming I would need to write another piece of code that subscribes to the queues, so when messages arrive, it knows to process the transaction?
Is that piece of code, a console app? runs on a scheduled task? a windows service? or a function app within azure?
What is the Best Practice for this architecture?
You can write console app and schedule web job to monitor the queue. However better way is to use Azure Functions. You don't have to monitor the queue then. Whenever the message arrives in the queue it will trigger Azure Function and you can process the message. The benefit is it's server-less.
I am using Azure Functions V2 with a Service Bus trigger using 1.0.23 of the C# Functions SDK. I'm using the following approach to get secrets from KeyVault and use them within the settings of the triggers: How to map Azure Functions secrets from Key Vault automatically
The function, especially when it has done nothing for a while, doesn't fire when there are messages on the subscription. If I then go to the portal and execute manually (yes, that particular execution is fired with a null message) it kicks it into life and picks up the other messages on the queue and processes them correctly.
This obviously isn't ideally for our automated tests. Has anybody seen this, or know of anything that will help?
Also, the Function App is running on a consumption plan.
App Service Plan
If you're using App Service plan then it's simple, just make use of Always on
Consumption Plan
If you're using Consumption plan, the issue could be that your triggers did not sync properly with the Azure Infrastructure (Central Listener). It could have happened due to the way you deployed/edited your trigger related settings as explained in issue #210 below.
When you access the function directly from Portal, it might be forcing your function app to come alive, but as you can see that's only a workaround. Something similar is mentioned here
Take a look at these issues:
Service Bus Topic Trigger goes to sleep - Consumption Plan
They also mention that it wakes up only on accessing it via the portal or calling a HTTP triggered function in the same app, which is similar to the behavior you are seeing.
Issue #210
Issue #681
There are 3 suggested ways to resolve it, mentioned as part of Issue #210 above
In order to synchronize triggers when these deployment options are
used, open the Azure Portal and click the Refresh button, or make a
API call to the sync triggers endpoint:
https://github.com/davidebbo/AzureWebsitesSamples/blob/master/ARMTemplates/FunctionsWebDeploy.json#L90
Powershell sample:
https://github.com/davidebbo/AzureWebsitesSamples/blob/master/PowerShell/HelperFunctions.ps1#L360-L365
I've had a similar issue. ServiceBus connection was injected using ServiceBus value in ConnectionStrings section of Function configuration. This is enough when Function is in hot state but after transitioning to cold state AzureWebJobsServiceBus value is used to connect to service bus. So in my case setting AzureWebJobsServiceBus to ServiceBus connection string in Function configuration fixed this.
I'm simply trying to work out how best to retrieve messages as quickly as possible from an Azure Service Bus Queue.
I was shocked that there wasn't some way to properly subscribe to the queue for notifications and that I'm going to have to poll. (unless I'm wrong in which case the documentation is terrible).
I got long polling working, but checking a single message every 60 seconds looks like it'll cost around £900 per month (again, unless I've misunderstood that). And if I add a redundant/second service to poll it'll double.
So I'm wondering what the best/most cost efficient way of doing it is.
Essentially I just want to take a message from the queue, perform an API lookup on some internally held data (perhaps using hybrid services?) and then perhaps post a message back to a different queue with some additional information .
I looked at worker roles(?) -- is that something that could do it?
I should mention that I've been looking at doing this with node.js.
Check out these videos from Scott Hanselman and Mark Simms on Azure Queues.
It's C# but you get the idea.
https://channel9.msdn.com/Search?term=azure%20queues%20simms#ch9Search
Touches on:
Storage Queues vs. Service Bus Queues
Grabbing messages in bulk vs. one by one (chunky vs. chatty)
Dealing with poison messages (bad actors)
Misc implementation details
Much more stuff i can't remember now
As for your compute, you can either do a VM, a Worker Role (Cloud Services), App Service Webjobs, or Azure Functions.
The Webjobs SDK and Azure Functions bot have a way to subscribe to Queue events (notify on message).
(Listed from IaaS to PaaS to FaaS - Azure Functions - if such a thing exists).
Azure Functions already has sample code provided as templates to do all that with Node. Just make a new Function and follow the wizard.
If you need to touch data on-prem you either need to look at integrating with a VNET that has site-to-site connectivity back to your prem, or Hybrid Connections (App Service only!). Azure Functions can't do that yet, but every other compute is a go.
https://azure.microsoft.com/en-us/documentation/articles/web-sites-hybrid-connection-get-started/
(That tutorial is Windows only but you can pull data from any OS. The Hybrid Connection Manager has to live on a Windows box, but then it acts as a reverse proxy to any host on your network).
To deal with Azure ServiceBus Queue easily, the best option seems to be Azure Webjob.
There is a ServiceBusTrigger that allows you to get messages from an Azure ServiceBus queue.
For node.js integration, you should have a look at Azure Function. It is built on top of the webjob SDK and have node.js integration :
Azure Functions NodeJS developer reference
Azure Functions Service Bus triggers and bindings for queues and topics
In the second article, there is an example on how get messages from a queue using Azure Function and nodejs :
module.exports = function(context, myQueueItem) {
context.log('Node.js ServiceBus queue trigger function processed message', myQueueItem);
context.done();
};