I had a site running asp.net 5 beta4, and decided to upgrade to beta5. The site runs locally fine. I pushed the changes to master and it was picked up from bitbucket and deployed successfully.
When I try to hit the site in azure, I get a 500 Internal Server Error. I've tried a number of things, but can't seem to track down the root cause of the failure. I'm looking for suggestions as I'm hitting a wall. From what I've tried below it seems like some fundamental initialization is failing.
Here's what I've tried:
Enabling customerrors="off". I added a web.config to the wwwroot folder with system.web/customErrors mode="Off". I've verified that the web.config is populated correctly in the deployed wwwroot and had the appsettings containing the dnxversion etc merged correctly.
Customizing the custom error page, adding runtimeinfo. I have the following set in my Startup.cs:
app.UseErrorHandler("/Home/Error");. I also have set the error page to display the exception. This doesn't seem to be hit.
Attached to the remote process to debug. Visual studio eventually freezes, so haven't gotten anywhere with this.
Enabled application insights. This registers events when I debug locally, but doesn't capture anything from the azure instance.
Enabled application logs and request failure tracing. The detailed errors show a 500.0, without much detailed information.
Imgur
Imgur
I've also verified through the console that the runtime is set correctly to beta5.
Update:
I set the ASPNET_ENV to Development and it loaded with appsettings loaded via the azure portal. Setting ASPNET_ENV to something else isn't working. I also removed any custom code from startup.cs pertaining to the non-development environments, with no help. I'm still looking for a means of capturing the original error.
Assuming you are targeting DNX451 and not dnxcore50, there is a good chance Azure it still trying to run it against the beta4 runtime instead of beta5. If that's the case, you won't get a decent error message.
Try adding an environment variable in Azure "SCM_DNX_VERSION" and set it to 1.0.0-beta5. It looks like kudu was recently upgraded to support beta5 https://github.com/projectkudu/kudu/commit/55175a017779bf493ff8e6ce87b96dd1451f7d7b, so you might want to try to redeploy from bitbucket in the case that the Kudu team has already deployed this change.
For a little more detail, you can check out my previous answer (although it is very dated and references the old "K" names) here:
Deploying ASP.NET vNext beta 2 on Azure with Kudu
Every time you update to a new beta, you will have to update your SCM_DNX_VERSION environment variable.
Related
I have a Azure App Service app that I'm trying to get deployed.
Today I ran into an issue where .NET informed me (via the yellow screen of death when I browse to the URL of my app) that I had a missing DLL (for the purposes of this question I don't think it really matters).
I used FileZilla to publish my changes in an attempt to do a manual deployment first and then work my way to automate it.
After so many attempts to fix it I later realized that the error message never changed. I did something more severe and renamed my bin folder into something completely different and the exact same error message would appear.
I've stopped the service, restarted it, and as mentioned, renamed folders, etc. and still the exact same error message persisted.
I also decided to open up the Azure Portal Console for my App Service app to browse a bit and to my amazement, nothing seemed to have reflected at all. The FTP shows one thing and the Console shows another.
Would anyone have any idea as to why this is happening?
I eventually got it to work and I will share what I tried.
I deleted the web app and created it again (I found this to be important the first time around). This was quite time consuming and did help but it wasn't long before the same problem happened again.
Then I finally found a solution that seems to give me consistent results:
I kept on editing the Web.config which seems to force a recompile and clear some sort of cache. So each time the web app stopped updating, I would make a slight change in the Web.config, upload it via FTP and the app finally updates.
If anyone has any more details on this, it would be greatly appreciated.
ASP.NET Core 2.1
We have a shared dll, that is used in a few projects (via project reference)
We have class in that was named Proofer, and is now named ProofReader.
This was done through mass find and replace.
Nowhere in our source files locally exists any files containing the text Proofer.
VSTS builds the application without issue, then we send it via a release pipeline to Azure. All succeed.
Despite this we attempt to run the web app and get the following:
An unhandled exception occurred while processing the request.
TypeLoadException: Could not load type
'MainShared.Models.Proofer' from assembly 'MainShared,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.
System.Reflection.CustomAttribute._CreateCaObject(RuntimeModule
pModule, IRuntimeMethodInfo pCtor, Byte** ppBlob, Byte* pEndBlob,
Int32* pcNamedArgs)
We are having trouble determining if this is in the build, maybe something not being rebuilt when it should be, or something still cached on Azure that isn't being overwritten.
A few places I can suggest you look into -
If you are using app service web apps in azure , we can use the kudu (web app extension to remove/clean the files) to make sure you have a clean destination.
Check the VSTS Release logs if you have access to , it shows the files which are overwritten and the ones which could not be. Once i had a similar issue and found that the logs were already warning me that a few could not be overwritten.
If you are using VM's for your app deployment if has access to it , remove/backup the files under your site folder.
I had a similar problem after some redesign for a Web App using ASP.NET Core 2.2 (hosted on Azure), that some old not even anymore existing references causing the Load type exceptions to happen and the Web App simply refused to come up. I tried to clean everything up in Visual Studio, even deleted the bin and object directories, but the old reference was still looked for. In Kudu on the Azure portal I saw the *.PrecompiledViews.dll and *.PrecompiledViews.pdb files, which were very old. I simply removed them directly on the Azure server and could then simply start up my app and everything came up again. This fault should be fixed from the source. If I do a complete rebuild, I expect this old garbage to be taken care of during the publishing to the website. This is something that Microsoft should fix and as I can see this happened to others over half a year ago. Thanks Mark Johnson for your very useful hint about those files!
When I publish to Azure my tiny, tiny test project, configured in the latest vNext RC2, I get the following error upon first load after an extremely long wait:
The specified CGI application encountered an error and the server terminated the process.
Subsequently if the app is anything more than the very simple "Hello World" project below, i.e. it uses some MVC etc. then the app is extremely unresponsive, failing to load some images, taking minutes to load each page. Although sometimes it's suddenly fast for a little while, then slow again.
In RC2 there were some changes to the hosting setup, but all these have been implemented in my tiny test project.
I have also seen this question and ensured I am publishing the exact correct version of the CLR, in fact for information the same result happens if I use full or core CLR.
Here is the example project (publishing profiles removed):
https://www.dropbox.com/s/hpkrj6c74eaytjz/TinyProject.zip?dl=1
If I create a new RC1 project, the problem doesn't surface, but as soon as I update it to RC2 the problem persists.
In the end I solved this by creating an App Service Plan that was anything other than the free or shared option, in my case B1 (screenshot from Visual Studio Azure SDK):
Has your Azure Web app instance had a RC1 instance uploaded to it prior to your RC2? Your project looks ok to me, I can't see anything wrong at first glimpse with your project.json, Startup.cs or hosting.json files. I had an instance of RC1 on a Web App, and when trying to upload RC2, nothing would work, just a long long wait until eventually it would time out with a 503 error. I deleted the Web App, and just published the RC2 (using same DNX build as yourself) and everything works fine (so far!).
Also, if you turn on Diagnostic logging in your Web App, does that provide any more info?
I created a cloud service and tested it successfully locally. I added service configurations for stage and production. Here is a snippet of my staging-configuration:
and here my configuration-settings:
Then when I publish I set up the deployment as follows:
All this worked like 2 weeks ago. But now he deploys in VS and when I look into Azure Service Configure area it looks like this:
I played a little bit with the "Update development ..."-checkbox on the second screen but the result is the same.
So it ignores all the settings I made and just won't tranistion my configuration to the ine I named "CloudStage". My current Web PI tells me that I use Windows Azure SDK for .NET (VS 2013) 2.3. I don't get the point.
Edit
Some more things I observed:
No WADLogsTable and WADWindowsEventLogsTable is generated automatically in the staging storage.
I deactivated Remote Desktop because it was one of the changes I made to monitor the event log (which wasn't useful here)
I manually changed the connection strings in Azure Portal but it seems as if the worker is totally unaware of the storage (rebooted it with no success).
Edit
I recognized another thing. Here you can see a running deployment of my service:
See the warning-mark on the left? If I go to my Error list this is shown:
This warning is senseless since it tells me that I did everything the right way. My *.Local.csfg-files are pointing to the local storage. So?!?
This seems weird. Please check the in your ServiceConfiguration.CloudStage.cscfg to verify the expected values.
Have you tried updating any other property like Enabling Remote desktop? Does that get updated on your deployment? You should select the "Deployment Update" check box in the publish dialog. Now, when deploying to an existing Cloud Service, it should ask you if you want to replace it.
If you get the Object reference error every time you right click on project, there might be some issue with the Azure SDK set up.
I'm a little bit further now. What I did was:
Deleted all Services in Azure.
Deleted all Storage Accounts in Azure
Removed my Service-Project completely from solution (not the library containing the worker-logic).
Re-added storage-accounts in Azure.
Re-added services in Azure.
Re-added a project in the solution and added the worker-logic inside it.
Builded up all the publishing-stuff again.
Published it.
The first publish ended like the one described in my question. After I checked the "Update development..."-option in properties of my worker it finally took my transitions into the stage!
Now I recognized, that WADLogsTable was still empty. I hit the instance right in server-explorer and choosse "Update diagnostic settings...". There was an option "Transfer period" suddenly set to "None". This explained to me, why my table was empty and after I set it back to "1" my table is filling again!
Another funny thing beside: When I right-click my Cloud-project in the solution I get "Object reference not set to an instance...". When I just click it left and choose Build->Publish it works.
I just hope that I can help somebody with this. Lets see if it's stable now.
Edit: Yesterday it worked - today is still the same issue :-(.
When you get "Object reference not set to an instance.." for a CloudService project you usually have some kind of mismatch. It could be that a setting in the ServiceConfiguration is not defined in the ServiceDefinition. It could also be that there is a publish profile defined in the .ccproj file for the CloudService that doesn't exist. This might also be what is causing your problems with the different configurations.
So it turns out that the problem is completely on client-side. My Visual Studio (now with SDK 2.4) is doing something wrong. I set up a fresh installation with all the stuff needed :-( and there it works perfect. I'll try to determine if one of my extensions is causing the strange "Object reference not set..."-bug.
Repair-Installation of VS does not solve the problem btw.
I am trying to deploy a Cloud Service with 1 Web Role to Azure.
When I do so, I get this message:
Your role instances have recycled a number of times during an update or upgrade operation. This indicates that the new version of your service or the configuration settings you provided when configuring the service prevent the role instances from running. Verify your code does not throw unhandled exceptions and that your configuration settings are correct and then start another update or upgrade operation.
The project runs just fine locally, and I'm having a hard time figuring out how to start debugging this issue. Are there any common problems that cause this message or steps to figure out what is causing it?
See https://learn.microsoft.com/en-us/archive/blogs/kwill/windows-azure-paas-compute-diagnostics-data. This will walk through all of the diagnostic data available as well as how to troubleshoot the most common issues.
We also had this annoying problem and in our case:
We use local storage, but it wasn't defined in service definition (or Worker Role's properties)
Our worker role project has reference to a service project which has reference to data layer project. But, the worker role project doesn't have reference to the data layer project. As soon as we added reference to data layer project in worker role project, it deploys successfully.
Problem #1 can be easily noticed if you first run the project in your local machine. Exception will be thrown.
Problem #2, however, is more difficult, mainly because it runs just fine in local machine. After 5 days of trouble shooting, we finally found the problem. So, check all references and try to add sub-reference projects, those that are referenced by other references.
We had similar problem, and it was due to some DDLs failed to load. (due to different version from the one MS have deployed to the VM)
Try to set CopyLocal to "true" for all the References in the project, and re-deploy.
I would either remote desktop to the cloud instance and review the Windows Event Logs for exceptions or redeploy with IntelliTrace Enabled. If you choose the later, you can download the IntelliTrace logs from Visual Studio and debug
http://msdn.microsoft.com/en-us/library/windowsazure/ff683671.aspx
One way to find out the actual error is to click on the " 1 instance" at the top of Dashboard after trying to deploy your web role. It will tell you the status of the role instance. The status should include more information about the type of error which blocks your deployment.
It depends on what your case is. For me, the status claimed that I had an unhandled Security exception. After some investigation, it turned out that under my role's OnStart(), I tried to create a event source. However, Azure service doesn't have the permission to create an event source.
For more possible issues, check http://blogs.msdn.com/b/kwill/archive/2013/09/06/troubleshooting-scenario-3-role-stuck-in-busy.aspx
For me, the issue was with my SQL Azure DB firewall rules. My Azure SQL Database servers are not set to "Allow Access to Azure services", so I have to explicitly list IPs that are allowed.
I discovered this after wrapping my code in a try/catch that swallowed all exceptions, refactoring my OnStart() and RunAsync() methods, and setting all my references to Copy Local = True. None of that worked, then I saw that I had this line in my RunAsync() method:
log4net.Config.XmlConfigurator.Configure();
I am using the AdoNetAdapter for log4net and connecting to an Azure SQL DB for logging, so that led me to check the firewall rules.
For me, I had some differing version of nuget packages in my various projects. Once I consolidated everything to the same version(s), it worked fine.
With the release of Windows Azure SDK version 2.2 for Visual Studio 2012 and 2013, Now you can Remote Debug Cloud Resources within Visual Studio.
Once your cloud service is published and running live in the cloud, you can simply set a breakpoint in your local source code. This may help you in digging out what's going wrong!