I have an Azure WebJob that I created from the WebJobSDK, Service Bus monitor. Basically the WebJob continually monitors a Service Bus, processes the message, and continues looking for new messages. When I debug (F5) the application locally, the process works perfectly. It basically grabs the message from the Bus, transposes it into an XML file, copies that file to an Azure BLOB container, and then call Azure SQL to insert a control row.
My problem is that when I publish the WebJob, I have an error that I don't know how to debug. I can look at the Service Bus messages and see that the process is peeking the message, failing, doing that 9 more times, and then the messages goes into the dead message queue. That is expected when a message is peeked 10 times and never completed (I left this defaulted to 10), so I know my process is running but failing abnormally.
I have Try/Catches around all of my code. What do I need to place within the Catch to have something surfaced in the Azure Portal so I can debug what is wrong? I actually have a call in the catch handler that reads the exception.InnerException and attempts to write it to a DB table, but surprisingly, that never happens.
I appreciate any insight. Thanks a lot...
You can do this as follows:
In VS, go to Cloud Explorer (make sure you have the latest Azure tools)
Find your Web App under App Service
Under it, go under WebJobs and find your Continuous WebJob
Right Click it and choose Attach Debugger
Related
Summary: I have 2 different Azure Function Apps (Node.js), sharing a single file storage account, however if I go into the Kudu Invocation Logs for either of them I see the entries from both Apps.
Here is my setup:
1 File Storage (shared by both Function Apps)
Service Bus 1 (sb-prod), with a single queue (somequeue)
Service Bus 2 (sb-staging), with a single queue (somequeue)
Function App 1 (func-prod), with a single function (somefunc)
Function App 2 (func-staging), with a single function (somefunc)
Both func-prod and func-staging are setup for continuous deployment from the same Bitbucket repo, but different branches
When a message is received in sb-prod it triggers somefunc in func-prod
When a message is received in sb-staging it triggers somefunc in func-staging
Note that the queue name and function name are the same in both prod and staging. That all seems to work fine. However if I go into Kudu and look at the Invocation Logs for debugging, it shows the execution of functions across both Function Apps (prod and staging shown in the logs for both). It is not respecting the folder structure on the file storage to only show the logs from the appropriate App. As far as I can tell, this is only a log viewing issue, and the functions aren't being run twice or messages being sent to the wrong function app. Any ideas on how to fix this? Or is this a bug and I would need to add a second storage account to fix it so that Kudu doesn't get confused? Is there any risk with this setup that messages from staging service bus end up in the prod app or vice versa?
By 'Kudu', I assume you mean the WebJobs Dashboard (not related to Kudu). The behavior you are seeing is quirky, but is in fact by design. See https://github.com/Azure/azure-webjobs-sdk/issues/1541 for more info.
Workarounds:
The best is to use App Insights instead of the WebJobs Dashboard
If you must use the WebJobs Dashboard, use distinct storage accounts
I have a web job that is supposed to be a triggered web job. I have it been deployed to azure fine and it is listed as a triggered webjob. However when I add things to the azure service bus that it has a function for. When i trigger it from the UI it works and will respond to my messages.
My host is configured like so
var config = new JobHostConfiguration
{
JobActivator = new MyActivator(container)
};
config.UseServiceBus();
var host = new JobHost(config);
host.RunAndBlock();
My Function looks something like this
public void ProcessQueueMessage([ServiceBusTrigger("recipetest")] ProductName message, TextWriter log)
{
//code
}
I have been looking for a while now but all google searches have given me back continuous web jobs with function triggers. Can anyone tell me how to get the web job to wake up and handle messages. I have found most other answers talk about always on but triggered jobs should work without always on.
In my opinion, the differences between continuous webJob and trigger webjob is as below:
Continuous webJob: Always run a backend exe in the web application.
In webjob SDK ServiceBusTrigger scenarios, even though your individual functions are 'triggered', the WebJob as a while runs continuously (i.e. your exe keeps running and does its own internal triggering).
Triggered webjob: Triggered by schedule or manually.
Notice: Webjobs are all run by the web app's process and web apps are unloaded if they are idle for some period of time. This lets the system conserve resources.
So both Continuous and Triggered (scheduled CRON) webjobs require 'Always on'.
To use ServiceBusTrigger, your WebJob has to be continuous.
To add a little more detail, the ServiceBusTriggerAttribute actually does poll the Message Queue that it monitors using a backoff if no messages are found. So you can see that messages added to the queue do not "wake up" a WebJob and tell it to do something - instead the ServiceBusTriggerAttribute polls and will find when new messages are waiting to be processed. This is why the WebJob can't be Triggered if you're using ServiceBusTriggerAttribute.
We have a WebJob in our Azure website that was supposed to listen to some messages on Azure Service Bus (ASB), so based on that we were expecting to run a process continuously so that it can listen to bus messages and process them as soon as they arrive. so with this background we chose “Run continuously” as WebJobs’s schedule at deployment time.
After some time, it appears that the WebJob does not work! and as usual we checked the Azure portal in that environment and checked the status of the job and noticed that it is running!
But the observation was, as soon as we open the WebJob tab in the azure portal we could see that it says “Web Job Starting” and after a few seconds it says “Web job Running”!
Any ideas about this behaviour and how to fix it?
There is a setting in the Configure tab called “ALWAYS ON“ which it has a very bold message on top of it that says “Should be turned on if you have a job that runs continuously”.
Here is how this setting looks like:
I also have a blog post about this issue:
https://koukia.ca/microsoft-windows-azure-webjobs-and-how-to-keep-them-awake-16283c28f19f#.nentaowwo
I have an Azure WebJob built using the Azure Web Jobs SDK. It consumes messages off of a queue and produces messages on another queue. It has thousands of successes and a handful of failures. Unfortunately, the only way I can find too look back over the function calls is by paging through all of them.
Is there some way I can get a list of the failures and get additional details about them such as what was logged or what exception was thrown?
An option is to log/save the message that failed and then retry it with your debugger attached to the webjob.
Also, Whatever is written to console output and console error will go to a log file for the specific triggered webjob run. So before throwing an exception in your try/catch block make sure you log the exception with the details so that you can acceess it later via FTP or the UI.
Here are the details on how/where is everything logged in azure websites. You can access the logs via FTP.
You can also access the information via the REST API. Details here.
Hope this helps,
I've got a continuously running WebJob on my auto-scale Azure website.
My WebJob is a simple console application with a while(true) loop that subscribes to certain messages on the Azure service bus and processes them. I don't want to process the same message twice, so when the web site is scaled an another WebJob is started I need for it to detect that another instance is already running and just sit there doing nothing until it's either killed (by scaling down again) or the other instance is killed. In the last scenario the second WebJob should detect that's the other is no longer take over.
Any takers?
You should create a queue (either using the Service Bus or storage queues) and pull the jobs off (create and manage a lease to the message) and process them from there. If that lease is managed properly, the job should only get processed once although you should make sure it's idempotent just in case as there are fringe cases where it will be processed more than once.