I have deployed my web job on production environment and suddenly blob trigger stops working(Looking into App Insight I know that blob trigger is not called).
If I debug the same code from the local machine then it triggers blob trigger.But stopped working in the production environment.
Always On is enabled.
Also, I have these containers are present
azure-jobs-host-archive,
azure-jobs-host-output,
azure-webjobs-dashboard,
Microsoft.Azure.WebJobs.Host,
Installed packages:-
Microsoft.Azure.Webjobs installed version is 2.0.0,
Microsoft.Azure.WebJobs.Host installed version is 2.0.0
According to your description, we couldn't directly find out why the blob trigger doesn't work. I suggest you could follow below way to troubleshoot by yourself.
If I debug the same code from the local machine then it triggers blob trigger.But stopped working in the production environment.
Since you found your webjob could work well in local, I guess maybe there are something wrong with the connection string config in your web app.
I suggest you could try to follow below way to change the production environment appsetting to set the storage connection string in it.
1.Open the portal and edit the appsetting as below.
2.Create another web app and publish the web job to it and run.
If these two ways all doesn't help you solve the error, I suggest you could post the log/error message and the details codes about the web job.
More details about how to find the log/error message, you could follow below way.
Related
I have made a task for Azure Devops Server pipeline.
On the local Azure Dev-Ops server, I can upload extensions from the local machine wrapped in VSIX file and examine the agent on the machine itself. The agent has the deployed DLL's from the extension and I always change specific DLL that has Logger inside and it prints to console on the pipeline itself.
How can I check if my task is compatible with the cloud version?
And off course if it is, how can I debug it?
The only possible way I found to insert my task in VSO is through the Microsoft DevOps marketplace, but I cannot upload the whole extension once per each change in code.
Also, didn't find a way to upload personal tasks to the marketplace. The private version is not helping because I can see it from within the server just like my clients.
Any advice or debug experience on that platform is appreciated.
Please follow this devblogs, it described how you can work and debug extension directly inside your IDE without publishing it to the marketplace.
Since this blog is very detailed, just mention the key steps.
Based on debugging purposes, you need create a special manifest, for sample I specify it as vss-extension-debug.json:
{
...
"id": "dev-extension",
"name": "Dev-Extension",
"baseUri": "https://localhost:44362",
...
}
You can customized the id and name by yourself. The most important and magic is baseUri, this will tell the VSTS server to load the extension from localhost where we can run and debug it in IISExpress from Visual Studio.
Note: Since our server need extension served from a secure source, please ensure enable SSL mode to run IISExpress
I'm running an Azure function in Azure, the function gets triggered by a file being uploaded to blob storage container. The function detects the new blob (file) but then outputs the following message - Did not find any initialized language workers.
Setup:
Azure function using Python 3.6.8
Running on linux machine
Built and deployed using azure devops (for ci/cd capability)
Blob Trigger Function
I have run the code locally using the same blob storage container, the same configuration values and the local instance of the azure function works as expected.
The functions core purpose is to read in the .xml file uploaded into blob storage container and parse and transform the data in the xml to be stored as Json in cosmos db.
I expect the process to complete like on my local instance with my documents in cosmos db, but it looks like the function doesn't actually get to process anything due to the following error:
Did not find any initialized language workers
Troy Witthoeft's answer was almost certainly the right one at the time the question was asked, but this error message is very general. I've had this error recently on runtime 3.0.14287.0. I saw the error on many attempted invocations over about 1 hour, but before and after that everything worked fine with no intervention.
I worked with an Azure support engineer who gave some pointers that could be generally useful:
Python versions: if you have function runtime version ~3 set under the Configuration blade, then the platform may choose any of python versions 3.6, 3.7, or 3.8 to run your code. So you should test your code against all three of these versions. Or, as per that link's suggestion, create the function app using the --runtime-version switch to specify a specific python version.
Consumption plans: this error may be related to a consumption-priced app having idled off and taking a little longer to warm back up again. This depends, of course, on the usage pattern of the app. (I infer (but the Engineer didn't say this) that perhaps if the Azure datacenter my app is in happens to be quite busy when my app wants to restart, it might just have to wait for some resources to become available.). You could address this either by paying for an always-on function app, or by rigging some kind of heartbeat process to stop the app idling for too long. (Easiest with a HTTP trigger: probably just ping it?)
The Engineer was able to see a lower-level error message generated by the Azure platform, that wasn't available to me in Application Insights: ARM authentication token validation failed. This was raised in Microsoft.Azure.WebJobs.Script.WebHost.Security.Authentication.ArmAuthenticationHandler.HandleAuthenticate() at /src/azure-functions-host/src/WebJobs.Script.WebHost/Security/Authentication/Arm/ArmAuthenticationHandler.cs. There was a long stack trace with innermost exception being: System.Security.Cryptography.CryptographicException : Padding is invalid and cannot be removed.. Neither of us were able to make complete sense of this and I'm not clear whether the responsibility for this error lies within the HandleAuthenticate() call, or outside (invalid input token from... where?).
The last of these points may be some obscure bug within the Azure Functions Host codebase, or some other platform problem, or totally misleading and unrelated.
Same error but different technology, environment, and root cause.
Technology Net 5, target system windows. In my case, I was using dependency injection to add a few services, I was getting one parameter from the environment variables inside the .ConfigureServices() section, but when I deployed I forget to add the variable to the application settings in azure, because of that I was getting this weird error.
This is due to SDK version, I would suggest to deploy fresh function App in Azure and deploy your code there. 2 things to check :
Make sure your local function app SDK version matches with Azure function app.
Check python version both side.
This error is most likely github issue #4384. This bug was identified, and a fix was released mid-june 2020. Apps running on version 3.0.14063 or greater should be fine. List of versions is here.
You can use azure application insights to check your version. KUSTO Query the logs. The exception table, azure SDK column has your version.
If you are on the dedicated App Service plan, you may be able to "pull" the latest version from Microsoft by deleting and redeploying your app. If you are on consumption plan, then you may need to wait for this bugfix to rollout to all servers.
Took me a while to find the cause as well, but it was related to me installing a version of protobuf explicitly which conflicted with what was used by Azure Functions. Fair, there was a warning about that in the docs. How I found it: went to <your app name>.scm.azurewebsites.net/api/logstream and looked for any errors I could find.
I am attempting to use an Azure App Service Deploy task in Azure DevOps to deploy a file to an app service in Azure. I have been able to use this release pipeline to deploy to the same app service in the past. Two things have changed: I had to re-make the build machine (which exists in Azure), and the source code moved from Azure DevOps to GitLab.
The error I am running into is:
Error: (3/21/2019 4:34:45 PM) An error occurred when the request was processed on the remote computer.
Error: The server experienced an issue processing the request. Contact the server administrator for more information.
Well, I'm the server admin, and I don't have a clue.
Is there some log file I can look at for more information? Can someone please offer some suggestions?
I'm happy to provide additional information, I'm just not sure what would help at this point.
Thank you Tom Sun for directing me to the system.debug variable. That was very helpful. When I turned on the debugging the release worked, witch really doesn't make sense. I started the build machine after it had automatically shut down the day before, but then I had also rebooted the machine eight or nine times before that.
I removed the debug variable and the release still works.
I have no satisfying answer to how this issue was resolved and I don't know how to get the system in a state to reproduce the error.
An additional finding I have is that the artifact produced by the original build was a corrupt war file. Perhaps that had something to do with it?
I am trying to deploy a Cloud Service with 1 Web Role to Azure.
When I do so, I get this message:
Your role instances have recycled a number of times during an update or upgrade operation. This indicates that the new version of your service or the configuration settings you provided when configuring the service prevent the role instances from running. Verify your code does not throw unhandled exceptions and that your configuration settings are correct and then start another update or upgrade operation.
The project runs just fine locally, and I'm having a hard time figuring out how to start debugging this issue. Are there any common problems that cause this message or steps to figure out what is causing it?
See https://learn.microsoft.com/en-us/archive/blogs/kwill/windows-azure-paas-compute-diagnostics-data. This will walk through all of the diagnostic data available as well as how to troubleshoot the most common issues.
We also had this annoying problem and in our case:
We use local storage, but it wasn't defined in service definition (or Worker Role's properties)
Our worker role project has reference to a service project which has reference to data layer project. But, the worker role project doesn't have reference to the data layer project. As soon as we added reference to data layer project in worker role project, it deploys successfully.
Problem #1 can be easily noticed if you first run the project in your local machine. Exception will be thrown.
Problem #2, however, is more difficult, mainly because it runs just fine in local machine. After 5 days of trouble shooting, we finally found the problem. So, check all references and try to add sub-reference projects, those that are referenced by other references.
We had similar problem, and it was due to some DDLs failed to load. (due to different version from the one MS have deployed to the VM)
Try to set CopyLocal to "true" for all the References in the project, and re-deploy.
I would either remote desktop to the cloud instance and review the Windows Event Logs for exceptions or redeploy with IntelliTrace Enabled. If you choose the later, you can download the IntelliTrace logs from Visual Studio and debug
http://msdn.microsoft.com/en-us/library/windowsazure/ff683671.aspx
One way to find out the actual error is to click on the " 1 instance" at the top of Dashboard after trying to deploy your web role. It will tell you the status of the role instance. The status should include more information about the type of error which blocks your deployment.
It depends on what your case is. For me, the status claimed that I had an unhandled Security exception. After some investigation, it turned out that under my role's OnStart(), I tried to create a event source. However, Azure service doesn't have the permission to create an event source.
For more possible issues, check http://blogs.msdn.com/b/kwill/archive/2013/09/06/troubleshooting-scenario-3-role-stuck-in-busy.aspx
For me, the issue was with my SQL Azure DB firewall rules. My Azure SQL Database servers are not set to "Allow Access to Azure services", so I have to explicitly list IPs that are allowed.
I discovered this after wrapping my code in a try/catch that swallowed all exceptions, refactoring my OnStart() and RunAsync() methods, and setting all my references to Copy Local = True. None of that worked, then I saw that I had this line in my RunAsync() method:
log4net.Config.XmlConfigurator.Configure();
I am using the AdoNetAdapter for log4net and connecting to an Azure SQL DB for logging, so that led me to check the firewall rules.
For me, I had some differing version of nuget packages in my various projects. Once I consolidated everything to the same version(s), it worked fine.
With the release of Windows Azure SDK version 2.2 for Visual Studio 2012 and 2013, Now you can Remote Debug Cloud Resources within Visual Studio.
Once your cloud service is published and running live in the cloud, you can simply set a breakpoint in your local source code. This may help you in digging out what's going wrong!
I have a Worker Role which isn´t working on Cloud environment.
When I run it locally, it runs perfectly, but, when I deploy it to Azure I´m having some troubles. The deploy, itself, occurs seamlessy, after the VM starts, my app doesn´t run. There is nothing on Event Log, and, even after I setted up the app to flush all Trace Messages to Azure Table, nothing is wrote there too.
How can I check if my app is really running on the VM? Why my app isn´t working there as it works locally?
Have tried to implement diagnostics on your webrole? This is the best way to find any errors in your code. An other solution is to install sysinternals during startup. Patriek van Dorp has made a nu get package thad adds the sysinternal suite as a plugin for your cloud project.
The best way is to enable RDP and remote into the machine. Then you can look at the processes running and ensure that things are running as you expect. It is odd that there is nothing in the event log if it is failing to run. Does the portal show the deployment as Ready?
Maybe too late, but i had similar issue.When i run it locally, it was running.After deploy did not run anything.Problem was that, i deploying into website,where worker not available.You should deploy into CloudService(there are available both roles) or make Sheduler, which will do request on your page, where process job you needed.In your custom intreval, ofcourse.
BTW sorry about my english ...
Regards