I'm using Right-Click publish in Visual Studio to publish directly to my staging slot in Azure. I've added Application Insights that added a WebJob to my app.
I have enabled Remove additional files at destination and Precompile during Publishing turned on.
But when a Web Job is running and my staging site is not 'stopped', because it's still warmed up - I get the following error:
Web deployment task failed. ((18-Jan-18 22:47:23) An error occurred
when the request was processed on the remote computer.)
(18-Jan-18 22:47:23) An error occurred when the request was processed
on the remote computer. An error was encountered when processing
operation 'Delete Directory' on
'D:\home\site\wwwroot\App_Data\jobs\continuous'. The error code was
0x80070091. The directory is not empty.
at
Microsoft.Web.Deployment.NativeMethods.RaiseIOExceptionFromErrorCode(Win32ErrorCode
errorCode, String maybeFullPath) at
Microsoft.Web.Deployment.DirectoryEx.Delete(String path) at
Microsoft.Web.Deployment.DirPathProviderBase.Delete(Boolean
whatIf) ProjectX.Web 7 0
I can fix this by manually stopping the staging slot, upload it, and then start it again. This is quite cumbersome though.
Is there a fix?
I read somewhere that you can automatically take the (staging) app offline during publish but I can't find the setting.
UPDATE 1
<PropertyGroup>
<EnableMSDeployAppOffline>true</EnableMSDeployAppOffline>
</PropertyGroup>
...is not working for me.
UPDATE 2
Adding the WEBJOBS_STOPPED setting on the Staging slot is not working for me.
As you have enabled Application Insights on your Azure App Service, it has internally installed an extension which started a continuous web job and content corresponding to web job got copied to App_Data folder. The actual problem is because extension data get copied to App_Data folder where only user's data should reside instead of infrastructure data.
Now when you are selecting the option "Remove Additional Files From Destination" in visual studio, it tries to remove all contents including Application Insights data from App_Data folder. Since Application Insights web job is already running in background hence corresponding dll files are locked.
Even if you go with option of "App Offline" or "Stop web job" and successfully able to delete all contents then your deployment will succeed but this way your Application Insights will get corrupted and hence you may have to configure Application Insights again.
Suggestion:
Select skip "App_Data" folder along with option "Remove Additional Files From Destination".
It will delete all content except App_Data folder content and hence dll files corresponding to Application Insights continuous web job will be skipped.
Reference :
VSTS - Azure App Service Task Documentation
Limitation
Though this solution have a limitation - If you want to remove/modify existing user defined web jobs then can't skip App_Data folder. In such case either have to perform two step deployment or have to reinstall application insight extension again.
I found this in another SO question:
<PropertyGroup>
<EnableMSDeployAppOffline>true</EnableMSDeployAppOffline>
...
</PropertyGroup>
... adding to your publishing profile (.pubxml).
The full answer is here: https://stackoverflow.com/a/20888597/647845
UPDATE
This is not working for me. Maybe for others. Seems to ignore the setting.
Related
I've created a web job using the webjobs sdk that is linked to my web application. Well it isn't anymore (I removed the webjobs-list.json file from the web app project), because when it was linked, it just copied the source code files into the websites' app-data folder, not the bin folder/binaries/exe. But now that my web job is deployed from visual studio online to azure with my website, and tries to run, it's not reading the connection strings (AzureWebJobsStorage and AzureWebJobsDashboard) from the connection strings in my web application's settings in the azure portal. It looks for them in the config file that is deployed with the exe that is generated from compiling the webjob. If I manually put those strings in that config and check in, and redploy it will use them from there, so it's not checking the portal at all (even though my web app is running fine using what is defined in the portal). I thought that all web jobs were supposed to look for connection strings in the azure portal first? That isn't happening at all for me. Has anyone else experienced this? Could it have something to do with how it's deployed by visual studio online?
Part of the error getting logged for the webjob process when trying to start:
Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexingException: Error indexing method 'Functions.ProcessTestQueueMessage' ---> System.InvalidOperationException: Microsoft Azure WebJobs SDK 'Storage' connection string is missing or empty. The Microsoft Azure Storage account connection string can be set in the following ways:
[04/01/2017 02:27:40 > ceef22: INFO] 1. Set the connection string named 'AzureWebJobsStorage' in the connectionStrings section of the .config file in the following format , or
[04/01/2017 02:27:40 > ceef22: INFO] 2. Set the environment variable named 'AzureWebJobsStorage', or
[04/01/2017 02:27:40 > ceef22: INFO] 3. Set corresponding property of JobHostConfiguration.
[04/01/2017 02:27:40 > ceef22: INFO] at Microsoft.Azure.WebJobs.Host.Executors.StorageAccountParser.ParseAccount(String connectionString, String connectionStringName, IServiceProvider services)
Also, the only way I could get this to really deploy was to remove the webjobs-list.json file from the properties folder in the web project. I know I'm not supposed to do that, that everything should just work and deploy with that in there, but when I left it in there, it wouldn't copy the binaries/bin/executable files into the App_Data\Jobs\continuous[webjob] folder on the website...it would literally copy all of the source code\files for the web job project into there (which of course wouldn't work/run, since it's just code, not an exe). After removing that webjobs-list.json file, and then adding a second task to my release to find the [webjob].zip file from the build, then it would copy just the binaries and executable into the App_Data\Jobs\continuous[webjob] folder. Maybe this is just an issue with trying to build and release from visual studio online, and it doesn't handle web job deployment well...not sure, I haven't been able to find a lot of information on setting that up to deploy a web job with a web app.
Normally, this should just work. The flow is:
The WebJobs exe doesn't run in-place, but first gets copied to a temp folder
As part of that copying process, if any of the AppSettings & Conn Strings in the foo.exe.config file are set as Azure settings, they get replaced (only in the temp copy, not under App_Data).
Can you make sure that you are deploying both your foo.exe and foo.exe.config?
Update: as we found on chat, the problem was that you had a typo in the name of the connection string in Azure.
Since yesterday I'm suddenly having consistent deployment failures for my existing WebJobs. The error I'm getting is
Start Web Deploy Publish the Application/package to https://octotrip-api-test-dev.scm.azurewebsites.net/msdeploy.axd?site=octotrip-api-test__dev ...
Adding sitemanifest (sitemanifest).
Adding directory (XXX\app_data).
Adding directory (XXX\app_data\jobs).
Adding directory (XXX\app_data\jobs\continuous).
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\Web\Microsoft.Web.Publishing.targets(4295,5): Error : Web deployment task failed. ((3/16/2017 1:32:23 PM) An error occurred when the request was processed on the remote computer.)
(3/16/2017 1:32:23 PM) An error occurred when the request was processed on the remote computer.
An error was encountered when processing operation 'Create Directory' on 'D:\home\site\wwwroot\app_data\jobs\continuous'.
The error code was 0x800700B7.
Cannot create "D:\home\site\wwwroot\app_data\jobs\continuous" because a file or directory with the same name already exists.
at Microsoft.Web.Deployment.NativeMethods.RaiseIOExceptionFromErrorCode(Win32ErrorCode errorCode, String maybeFullPath)
at Microsoft.Web.Deployment.DirectoryEx.CreateDirectory(String path)
at Microsoft.Web.Deployment.DirPathProviderBase.CreateDirectory(String fullPath, DeploymentObject source)
at Microsoft.Web.Deployment.DirPathProviderBase.Add(DeploymentObject source, Boolean whatIf)
Publish failed to deploy.
Important to note that at the time of the deployment; there is no app_data folder (I removed it manually using FTP). You can see it is creating the folder structure step by step; app_data, then the jobs folder, then the continuous folder. I confirmed using FTP that he succesfully creates the app_data/jobs folder; but creating the continuous folder consistently crashes.
This happens when deploying using a build definition in VSTS as well as deploying as a WebJob from within Visual Studio; with all the WebJob NuGet up to date.
Thanks
It is very odd that can't create folder for WebJob. I can't repro it in my side. Do you have a try to deploy it to another website to check whether it can be repro?
Based on my experience, we could troubleshoot with following ways:
Restart the Website
If it is possible try to redeploy the Website and remove additional files at destination
Scale up and scale down back Service plan
Note:change will be applied to all WebApps in the ServicePlan
Please contact Azure Support team for help if it is not resolved .
Worked around the issue by simply creating a new Azure Web App. Something messed up going on with the file system. Take a look.
app_data or subdirectories of app_data don't exist on the server. (confirmed using CMD and FTP).
This only happens for
app_data\jobs\triggered and
app_data\jobs\continuous
(app_data\jobs\triggereddddd works for example)
These folders have existed on this server before (they contain the data for the WebJobs that we're running)
open Azure Portal and go to your web app
Under "development tools" select "K Advanced Tools" this will open the Kudu control panel in a new browser window.
Select "Debug Console" -> "CMD"
in the Items table, locate the wwwroot entry and delete it.
continue working ...
In Visual Studio, I am deploying a web site to Azure via the Publish menu option, and the project has an existing web job added to it (via the Add menu). It was added as a continuous job, and has the correct entries in the 'webjob-publish-settings.json' and 'webjobs-list.json' files.
When I publish the web site, the job is compiled and put into place in the 'jobs' folder and published with the web site. However, it is not started. No entry is added automatically to the WebJobs settings page in Azure, which remains empty. There are no errors in the output window.
The Azure webjobs documentation states that it will attempt to run a selection of files from within the webjob directory, and one of these is any file that ends with ".exe". The webjob project is a console project, and compiles into an exe which is published in the directory.
In the past, I have published this job by zipping the contents of the 'bin\Release' directory, and adding it manually.
How can I debug why it does not start when deployed automatically?
First thing you should do is look at exactly what got deployed, to make sure it looks like what you expect. To do this:
Go to Kudu Console
Go to D:\home\site\wwwroot\App_Data\jobs\continuous
You should see a folder named after your WebJob in there. Go in that folder
Check that it has all the right files
If it doesn't, then you likely have some kind of deployment issue, and it is not an issue with the WebJobs runtime.
I think when you published the Web site, it just copied over the Webjob files but Azure does not know that there's a Webjob that has been published. Could you please try publishing the WebJob separately by selecting "Publish as Webjob" option. If it has problems publishing that way then there's some problem in your Webjob, like app settings etc, and if it is successfully published then I guess you have to publish it separately every time or find a hack in Publish settings of the Website to publish both the projects.
Hope it helps. Thanks
I had a web job that was working fine, I stopped it, changed a value in the config file and started it again. It didn't start, no errors, no log issues, nothing at all.
How I fixed it? I clicked "Restart" button in the app service that contains all the web jobs.
Note: stop and start did not work, only "Restart" did the trick. It really makes no sense but that's what happened. I hope it helps others with the same issue.
Very often, when I hit Publish in VS13, I get the site to compile but when uploading I get the error saying that a file is busy.
Updating file (MyAzureSite\PrecompiledApp.config).
C:...\v12.0\Web\Microsoft.Web.Publishing.targets(4255,5):
Error ERROR_FILE_IN_USE: Web deployment task failed.
(The file 'PrecompiledApp.config' is in use.
Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_FILE_IN_USE.)
When I follow the link provided, it's suggested that I should go for enabling the appOffline rule. "Sure!", I think to myself. But how?! I've googled it, only to get a bunch of hits on the file that's supposed to replace the site while publishing. However, I get no info on how to get rid of my little problem.
I went the easy way and downloaded a publishing profile from my Azure web site and now I'm using it (you know, ALT+B+H).
Right now I resolve the problem by going to the portal for Azure and manually take the site off-line. Then I can publish and after that I take the site on-line. Highly impractical and painfully tedious.
What is causing this and how do I kill it?
You actually configure it in the publishing profile (.pubxml). Just add the element to the PropertyGroup like this:
<PropertyGroup>
<EnableMSDeployAppOffline>true</EnableMSDeployAppOffline>
...
</PropertyGroup>
More in this MSDN document
The Azure App Service Deployment Task has a checkbox for this (from version 2.0 onwards), under Additional Deployment Options: Take App Offline. Check that and you should be good to go.
Take Application Offline: Select the option to take the AzureRM Web App offline by placing an app_offline.htm file in the root directory of the Web App before the sync operation begins. The file will be removed after the sync operation completes successfully.
Screenshot:
I am getting the following error:
Error: The process cannot access the file 'C:\DWASFiles\Sites\mywebsitename\VirtualDirectory0\site\wwwroot\newrelic\NewRelic.Agent.Core.dll' because it is being used by another process.
In the Running deployment command... log file when attempting to deploy an Azure website from Github.
Would appreciate any pointers as to what could be causing this.
UPDATE: Turns out this is also failing when publishing directly from VS.NET with the following:
1>C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\Microsoft.Web.Publishing.targets(4196,5): Warning : An error was encountered when processing operation 'Create File' on 'NewRelic.Agent.Core.dll'.
1>Retrying operation 'Update' on object filePath (mywebsitename\newrelic\NewRelic.Agent.Core.dll). Attempt 1 of 2.
1>C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\Microsoft.Web.Publishing.targets(4196,5): Error : Web deployment task failed. ((06/07/2013 23:54:58) An error occurred when the request was processed on the remote computer.)
This was working before and I am not sure why it would have stopped.
NewRelic recommend stopping the website to unload the file and allow the deployment to go through.
As an alternative, you can set COR_ENABLE_PROFILING to 0 in your app settings on the configure tab to temporarily disable the profiling, which should then allow you to continue with the deployment while leaving the website operational throughout.
Instead of stopping the website you can temporarily turn off New Relic monitoring via the Configure tab on manage.windowsazure.com:
Configure > developer analytics > select "OFF" > Save
Deploy
Configure > developer analytics > select "ADD-ON" > Choose Add-on from dropdown > Save
Worked for me, both with a regular deployment from VS and an automatic build from VSO.
This is a known issue with the New Relic .NET agent for Azure Websites when performing an upgrade of the agent. The workaround is to stop the website to release the dll, finish the deployment and then restart the instance.
https://newrelic.com/docs/dotnet/azure-web-sites#h2-1
Not really a solution but more of a work-around, in the publish dialog view a preview of the changes and uncheck the NewRelic.Agent.Core.dll file so that it doesn't get published.
None of these answers work for me anymore. I have an Azure Basic tier website plan, which hosts multiple actual websites.
If I don't stop the website, I get the error mentioned above (newrelic.agent.core.dll is in use)...
If I do stop the website (or all of them), I get an error saying that the publishing endpoint isn't available.
If I go to the configure tab and disable the AddOn, we still get the error mentioned above (newrelic.agent.core.dll is in use)...
Pretty much we just republish over and over again with different permutations of the above until if works. It took me hours the other day, took me 10 minutes today.
If you are using webdeploy, then you can configure your webdeploy settings so that it ignores the file. However, if you do that, you will manually have to deploy any updates to the new relic agent.
I had a similar issue with the new relic log file being locked, and solved it by:
Moving the new relic log file to a subdirectory of the web root (e.g. \newreliclogs)
Adding 2 lines to my powershell script that configured the skip directive to ignore that whole directory. e.g. (where destBaseOptions is of type Microsoft.Web.Deployment.DeploymentBaseOptions
$skipDirective = new-object Microsoft.Web.Deployment.DeploymentSkipDirective("NewRelicLog","objectName=dirPath,absolutePath=.*\newreliclogs$")
$destBaseOptions.SkipDirectives.Add($skipDirective)
Depending on how you are using webdeploy, the configuration is achieved slightly differently, I used the following links to help me piece it together:
https://technet.microsoft.com/en-us/library/dd569089(WS.10).aspx
https://msdn.microsoft.com/en-us/library/microsoft.web.deployment.deploymentskipdirective(v=vs.90).aspx
https://msdn.microsoft.com/en-us/library/dd543313(v=vs.90).aspx
http://blogs.iis.net/jamescoo/archive/2009/11/03/msdeploy-api-scenarios.aspx
http://forums.iis.net/p/1192163/2031814.aspx#2031813
And I used the powershell script from the Octopus Deploy Library at https://library.octopusdeploy.com/#!/step-template/actiontemplate-web-deploy-publish-website-(msdeploy).