Service Fabric deployment hanging on GetApplicationExistence - azure

I have created a Service Fabric cluster in Azure from scratch, with security provided by an X509 certificate. There is one node type with five nodes, all of which appear healthy when I look in Service Fabric Explorer.
I have a solution in Visual Studio 2017 (v15.3.5) containing a Service Fabric project along with one other project which is a Stateless Service. When I attempt to deploy this, I select the cluster in the publish window, and hit go.
It then sits indefinately displaying "Started executing script 'GetApplicationExistence'" in the Build output window in VS. No amount of waiting will result in either success nor an error message. I can't find anything in the logs in Azure.
Does anyone have any idea what could be going wrong?

Check if your LocalCluster is healthy. There should not be any error/warning. I had warning System/DnsService was not up.
Reset the LocalCluster worked for me.

It turns out that there was something wrong with my development environment; the service fabric installation had got messed up somehow. When I tried with the exact same code from a different machine it all worked properly. I updated the Service Fabric elements of Visual Studio to the latest version and all started working again.

Related

Vague error with Azure DevOps web app deploy

I am attempting to use an Azure App Service Deploy task in Azure DevOps to deploy a file to an app service in Azure. I have been able to use this release pipeline to deploy to the same app service in the past. Two things have changed: I had to re-make the build machine (which exists in Azure), and the source code moved from Azure DevOps to GitLab.
The error I am running into is:
Error: (3/21/2019 4:34:45 PM) An error occurred when the request was processed on the remote computer.
Error: The server experienced an issue processing the request. Contact the server administrator for more information.
Well, I'm the server admin, and I don't have a clue.
Is there some log file I can look at for more information? Can someone please offer some suggestions?
I'm happy to provide additional information, I'm just not sure what would help at this point.
Thank you Tom Sun for directing me to the system.debug variable. That was very helpful. When I turned on the debugging the release worked, witch really doesn't make sense. I started the build machine after it had automatically shut down the day before, but then I had also rebooted the machine eight or nine times before that.
I removed the debug variable and the release still works.
I have no satisfying answer to how this issue was resolved and I don't know how to get the system in a state to reproduce the error.
An additional finding I have is that the artifact produced by the original build was a corrupt war file. Perhaps that had something to do with it?

Web deployment task failed. Destination not reachable

I've been successfully using Azure for months. Today I'm getting the following error when I publish from via Web Deploy from Visual Studio 2013
Error 5 Web deployment task failed. (Could not connect to the remote computer ("waws-prod-hk1-001.publish.azurewebsites.windows.net"). On the remote computer, make sure that Web Deploy is installed and that the required process ("Web Management Service") is started. Learn more at: http://go.microsoft.com/fwlink/?LinkId=221672#ERROR_DESTINATION_NOT_REACHABLE.)
I tried downloaded a new Publishing Profile from the Azure Dashboard and turning off my firewall - no change.
I can't ping the listed server.
It seems to be the same issue as http://social.msdn.microsoft.com/Forums/windowsazure/en-US/83d4e635-2851-4526-b21b-31101d00aa86/web-deployment-task-failed-errorcouldnotconnecttoremotesvc?forum=windowsazurewebsitespreview
Any ideas? Thank you.
I had the exact same problem for a couple of days (sometimes I could published and sometimes not).
The problem was my internet configuration (not related to Microsoft or Azure in anyway).
To solve this issue I disconnected my modem (its actually modem+router) for 10 min and reconnect it and it was fixed!
Hope it will help someone someday...
Nevermind. It would appear that Azure was in a 'challenged' state.
It's working fine now..
What worked for me was disabling and re-enabling my network adapter.
The reasoning is, quite possibly naive, that having Fiddler running during a deployment will fail but it also changes the network state and that state continues
to exist even after exiting Fiddler (which disables the proxy).

Your role instances have recycled a number of times during an update or upgrade operation

I am trying to deploy a Cloud Service with 1 Web Role to Azure.
When I do so, I get this message:
Your role instances have recycled a number of times during an update or upgrade operation. This indicates that the new version of your service or the configuration settings you provided when configuring the service prevent the role instances from running. Verify your code does not throw unhandled exceptions and that your configuration settings are correct and then start another update or upgrade operation.
The project runs just fine locally, and I'm having a hard time figuring out how to start debugging this issue. Are there any common problems that cause this message or steps to figure out what is causing it?
See https://learn.microsoft.com/en-us/archive/blogs/kwill/windows-azure-paas-compute-diagnostics-data. This will walk through all of the diagnostic data available as well as how to troubleshoot the most common issues.
We also had this annoying problem and in our case:
We use local storage, but it wasn't defined in service definition (or Worker Role's properties)
Our worker role project has reference to a service project which has reference to data layer project. But, the worker role project doesn't have reference to the data layer project. As soon as we added reference to data layer project in worker role project, it deploys successfully.
Problem #1 can be easily noticed if you first run the project in your local machine. Exception will be thrown.
Problem #2, however, is more difficult, mainly because it runs just fine in local machine. After 5 days of trouble shooting, we finally found the problem. So, check all references and try to add sub-reference projects, those that are referenced by other references.
We had similar problem, and it was due to some DDLs failed to load. (due to different version from the one MS have deployed to the VM)
Try to set CopyLocal to "true" for all the References in the project, and re-deploy.
I would either remote desktop to the cloud instance and review the Windows Event Logs for exceptions or redeploy with IntelliTrace Enabled. If you choose the later, you can download the IntelliTrace logs from Visual Studio and debug
http://msdn.microsoft.com/en-us/library/windowsazure/ff683671.aspx
One way to find out the actual error is to click on the " 1 instance" at the top of Dashboard after trying to deploy your web role. It will tell you the status of the role instance. The status should include more information about the type of error which blocks your deployment.
It depends on what your case is. For me, the status claimed that I had an unhandled Security exception. After some investigation, it turned out that under my role's OnStart(), I tried to create a event source. However, Azure service doesn't have the permission to create an event source.
For more possible issues, check http://blogs.msdn.com/b/kwill/archive/2013/09/06/troubleshooting-scenario-3-role-stuck-in-busy.aspx
For me, the issue was with my SQL Azure DB firewall rules. My Azure SQL Database servers are not set to "Allow Access to Azure services", so I have to explicitly list IPs that are allowed.
I discovered this after wrapping my code in a try/catch that swallowed all exceptions, refactoring my OnStart() and RunAsync() methods, and setting all my references to Copy Local = True. None of that worked, then I saw that I had this line in my RunAsync() method:
log4net.Config.XmlConfigurator.Configure();
I am using the AdoNetAdapter for log4net and connecting to an Azure SQL DB for logging, so that led me to check the firewall rules.
For me, I had some differing version of nuget packages in my various projects. Once I consolidated everything to the same version(s), it worked fine.
With the release of Windows Azure SDK version 2.2 for Visual Studio 2012 and 2013, Now you can Remote Debug Cloud Resources within Visual Studio.
Once your cloud service is published and running live in the cloud, you can simply set a breakpoint in your local source code. This may help you in digging out what's going wrong!

Azure Worker Role not working properly

I have a Worker Role which isn´t working on Cloud environment.
When I run it locally, it runs perfectly, but, when I deploy it to Azure I´m having some troubles. The deploy, itself, occurs seamlessy, after the VM starts, my app doesn´t run. There is nothing on Event Log, and, even after I setted up the app to flush all Trace Messages to Azure Table, nothing is wrote there too.
How can I check if my app is really running on the VM? Why my app isn´t working there as it works locally?
Have tried to implement diagnostics on your webrole? This is the best way to find any errors in your code. An other solution is to install sysinternals during startup. Patriek van Dorp has made a nu get package thad adds the sysinternal suite as a plugin for your cloud project.
The best way is to enable RDP and remote into the machine. Then you can look at the processes running and ensure that things are running as you expect. It is odd that there is nothing in the event log if it is failing to run. Does the portal show the deployment as Ready?
Maybe too late, but i had similar issue.When i run it locally, it was running.After deploy did not run anything.Problem was that, i deploying into website,where worker not available.You should deploy into CloudService(there are available both roles) or make Sheduler, which will do request on your page, where process job you needed.In your custom intreval, ofcourse.
BTW sorry about my english ...
Regards

How do I debug a Worker Role using Remote Desktop with Windows Azure?

I now have my Windows Azure environment set up so that I can access my Worker Role with Remote Desktop. However, I'm not sure how to proceed at the moment. After much digging I found a web site that was offline but in Google's cache there was mention of attaching to the Worker Role running in the Azure Cloud from the Visual Studio debugger. But I only have Visual Developer (not studio) 2010 and I have searched all over and as far as I can see there is no such option to attach to a remote server. I am able to publish my project to the Azure Cloud without error and I have a "healthy" instance of my Worker Role showing as active and running.
I did connect with RDP through the Azure Management portal. The login worked fine and up came the remote desktop window. I searched through much of what I could find and was unable to find my Worker Role. I must have the wrong impression of RDP, because I had hoped to see the Worker Role's main display form when I logged in, just like I do when I debug it locally in the Cloud Emulator. But instead all I saw was a blank desktop with some base level server inspection and management routines. I even checked the Event Viewer for Application related messages and saw none.
So now I'm stuck wondering if my Worker Role is actually running or not, despite the seemingly positive status messages from the Management Portal, and I still want to attach to my Worker Role for debugging through Visual Developer, if it's possible, but I am unable to figure out how.
Anyone with experience in this area that can give me some solid tips on what to do next, please respond.
UPDATE: I believe my worker role may be running because I opened a command window and did a Netstat and saw it listening on the correct port. However, that may just be my Worker Role shell class that starts the custom EXE I have it launch as a spawned proces. I still haven't confirmed if my custom EXE is running yet.
UPDATE-2: Just ran TaskList from a command window and the custom EXE is listed.
UPDATE-3: Everything is working as I just ran a remote test of the service so that's not a problem. Still want to know how to attach to the Worker Role from Visual Developer 2010 for remote debugging, and if it's possible to see the custom EXE's display form like I do when doing local debugging in the Cloud Emulator.
-- roschler
There is a set of articles here which goes in length on how to set up for remote debugging in Azure:
http://blogs.u2u.be/peter/post/2011/06/21/Remote-debugging-an-Azure-Worker-role-using-Azure-Connect-Remote-desktop-and-the-remote-debugger.aspx
http://blogs.u2u.be/peter/post/2011/06/24/Remote-debugging-an-Azure-worker-role-using-Azure-Connect-remote-desktop-and-remote-debugger-part-2.aspx
http://blogs.u2u.be/peter/post/2011/06/26/Remote-debugging-a-Windows-Azure-Worker-Role-using-Azure-Connect-Remote-desktop-and-the-remote-debugger-part-3.aspx
The key takeaway is that you don't need to actually install Visual Studio on Azure, you only need to copy the Remote Debugger bits and then use Azure Connect to add your developer machine to the Virtual Network.
You can setup Remote Debugging with Visual Studio 2012
http://code.msdn.microsoft.com/Remote-Debugging-Windows-dedaaec9
When you say:
But instead all I saw was a blank desktop with some base level server inspection and management routines.
this is exactly what you get with an Azure VM. It's a basic OS install, plus the bare minimum of Azure stuff it needs to run and the code you've uploaded. There's no fancy monitoring or health checks available on the machine by default, you're expected to have provided those yourself to have them available without having to RDP into the machine to check on it.
RDP is very good for tracking down certain problems, like checking that a startup task will run, checking which directories items are installed in and just generally being nosey. If you need extra tools to track down a problem, you can just install them while you're connected to the server. For example I have RDPed into a server and installed the Microsoft Debugging Tools, to track down a memory issue.
I suppose you could remote into your VM, install Visual Studio there, and debug the process...
I also suppose it might be possible to enable remote debugging (not sure what's involved there, but such a thing exists, and it works over TCP) and debug from a local instance of Visual Studio.
To my knowledge, neither is commonly done.
Based on other answers, you would be better off writing a log file to a local storage. You can read the file from RDP if you reallyhace to. Keep in mind, debugging on Azure isn't really simple, and rightly so.
What I was thinking though was, maybe you could run the process using the user's credentials. I can't verify at the moment, but you have a better shot of seeing the ui when you rdp.

Resources