Letsencrypt auto-renewal is failing with a 409 (Conflict) - azure-web-app-service

I set up the letsencrypt extension in Azure Portal several months ago by following this: https://github.com/sjkp/letsencrypt-siteextension/wiki/How-to-install
Everything worked fine. But now when it's trying to renew the cert, I see the logs in Web Jobs showing that it failed. The first error was due to:
The Lets Encrypt ACME server was probably unable to reach 'http://<site>/.well-known/acme-challenge/<stuff>'
I found this post: https://blog.nicholasrogoff.com/2017/02/21/lets-encrypt-extension-for-azure-app-services/
Then added some settings that I was missing. And now I'm getting a 409 (Conflict) error.
What else do I need to check?

I had this exact problem for one of my azure functions.
The error actually comes from Kudu that refuses editing when the site it is "run from package"
The solution was simply to set WEBSITE_RUN_FROM_PACKAGE to 0 in application settings.

Related

Problem connecting to azure gitlab vm instance after changing external url

I am trying to create a self-managed version of gitlab that runs on azure using this link: https://docs.gitlab.com/ee/install/azure/
It all works fine until I get to the "Change the GitLab external URL" section. I follow the instructions exactly: I replace the external url and I comment out the lines and I run the reconfigure command. But this breaks connections to the vm. I can no longer connect to it at all (previously I could connect, but I would always be redirected to the public unsecure url as the article says).
Now I simply get a "this site can't be reached error" [public ip] refused to connect.
Any ideas what step I'm missing.
I also think the article is slightly outdated because of the section that tells us to rename the utility bitnami uses:
"sudo mv /opt/bitnami/apps/gitlab/bnconfig /opt/bitnami/apps/gitlab/bnconfig.bak"
There is no longer a bnconfig file that exists in the gitlab azure instance.
I would greatly appreciate any help!
I just had the exact same problem. What worked for me was to edit the config by:
sudo vim /etc/gitlab/gitlab.rb
And then enable letsencrypt by changing this line to true:
letsencrypt['enable'] = true
Save, and then do the usual
sudo gitlab-ctl reconfigure
According to the docs it is supposed to be enabled automatically:
Using https in the URL automatically enables, Let’s Encrypt, and sets HTTPS by default
...but that statement no longer holds, it seems.
Also, I did not figure out the missing /opt/bitnami/apps/gitlab/bnconfig file that should be renamed (it is also missing for me), but I don't seem to loose the config after restarting the VM, so this part of the docs just seem outdated.

502 WAF to Appservice connection issue (domain name fails and Azure name passes)

When we call domain url, www.foo.com it gives 502 error
"502 - Web server received an invalid response while acting as a
gateway or proxy server."
But the technical url www.foo.azurewebsites.net is working fine.
How do we diagnose this?
Since the technical url is working fine we can rule out application or code error
We ran a Diagnosis of WAF (pdf removed the spaces, sorry)
DegradedBackendServerHealth:ApplicationGateway:'FOO',BackendServer:'foo.azurewebsites.net',HealthStatus:Down,BackendservercertificateisnotwhitelistedwithApplication
Gateway.,Reportedat:7/14/20199:34:40AM.Mitigation:Reviewthehealthofthebackendserverfirst.Ifthebackend
serverishealthyandcanrespondwithHTTP200viaotheraccesspaths,troubleshootnetworkconnectivityfromthe
ApplicationGatewayinstancestothebackendserver.Troubleshootingincludes(butisnotlimitedto):SecurityRules,
routing,networkperformance,andgeneralTCPconnectivitytroubleshooting.
WehavefoundthatalltheinstancesofBackendAddressPoolareunhealthy.Ensurethattheinstancesarehealthyandthe
applicationisproperlyconfigured.Checkiftheback-endinstancescanrespondtoapingfromanotherVMinthesame
VNet.Ifconfiguredwithapublicendpoint,ensureabrowserrequesttothewebapplicationisserviceable.
We checked the certifcates configured, it is working fine
Pretty much all recommendation are verified and working fine
How can we diagnose this further and find rootcause?
It was resolved on slot swap. For some reason, one slot had some security restrictions in place. Not sure who did it or Why it was done. This is the immediate answer.
There seemed to have some script run which affected all live servers but the slots/staging were 'protected'. When the staging became production this issue was reverted(as they were 'protected')
Will update once more info available on this

Azure Website "Always On"

I've a CakePHP Website hosted on Azure Website (Standard). When I switched "Always On" on. It seems to keep giving me a 403 error after logged in my WebApp. When I turned off and the error goes away. How can I turn on "Always On" and resolve the 403 error? . This error pop out after I logged in my Webapp. Do I need to put any additional code to web.config to resolve this issue?
It's seems unlikely that Always On would cause this. The only things Always On does is send a ping request to the root of your app every couple minutes (with an AlwaysOn user agent). It's really not much different from sending similar requests yourself at the same interval.
If you enable http logging, you should see those requests.

Debugging ERR_CONNECTION_RESET error

Recently I started getting The connection was reset. error message (error code is ERR_CONNECTION_RESET) when I open one of my websites. This has nothing to do with application(node.js) level afaik as no changes or deploys were made in the application. To add to this, there were no error logs so I am assuming the request did not even receive the node app. The website is hosted on godaddy and its backend is a elastic beanstalk application. How do I fix this? On the web browser, in the second attempt it automatically loads correctly. But in a iframe which embeds my website, this has become a nasty issue as the browser does not even retry. I did the DNS analysis using dig command and all that but could not find anything relevant. Interesting part is this only happens after some interval (it does not happen continuously). On reload it works without fail.

IIS bindings keep being removed

I'm having a problem where the security certificate for a site is being periodically unbound from port 443 and replaced with another certificate which is sitting on the server. So whenever a user tries to access the site they are met with a 'untrusted' warning.
So when this first happened, I investigated and found the wrong certificate in place so I changed it back. This worked fine for a while but then it happened again. I checked the event logs and the following two warnings are fired:
SSL Certificate Settings deleted for endpoint : 0.0.0.0:443
SSL Certificate Settings created by an admin process for endpoint : 0.0.0.0:443
This happens once or twice a day, and I have to keep rebinding the correct certificate, and I haven't been able to find a solution yet.
The site is running on Windows Server 2012/ IIS 8
According to a couple of online support forums/articles there was an old legacy setting in the ApplicationHost.config file which was supposed to cause this. All references to this that I found referred to a property in the 'customMetaData' section, the property had a specific Id (5506). I couldn't find this specific property anywhere in our ApplicationHost.config file on the server.
Has anyone encountered a similar issue? Or can anyone shed any light on potential causes of this? Having looked around online I'm finding it hard to find much related to my problem, but perhaps I'm not searching for the right thing...
Any advice on this issue would be greatly appreciated.
NOTE:
Have since realised that this happens at 13:00 each day, cant see any significant events that are occurring on the server that might trigger it though...
Resolution
Locate the following property in the section of the applicationHost.config file, and delete it:
<property id="5506" dataType="Binary" userType="1" attributes="None" value="oXiHOzFAMOF0YxIuI7soWvDFEzg=" />
This property is a legacy feature from Internet Information Services (IIS) 6.0 and is no longer needed.
Link to MS Article
If the other answer (property id) doesn't work, follow these steps:
Check if there is an antivirus software in the server. Look for especially HIPS feature. Disable the antivirus and try to reproduce the issue
Check if the site is using a wildcard certificate. This issue occurs when the wildcard certificate has been imported without marking the keys as exportable. In order to solve it, the affected certificate should be uninstalled and it should be imported back again with marking the keys as exportable
Look for System Center Virtual Machine Manager Agent in the server. If it is enabled in the server, disable it and try to reproduce the issue (Reference)
Another process might be using 443 port in the server (Example: Windows Admin Center. Check this post out: 503 Service Unavailable error related to Windows Admin Center)
Check if insecure protocols are enabled. Registry settings are below. Disable these protocols if they are enabled and try to reproduce the issue
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL
2.0\Server HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL
3.0\Client HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL
3.0\Server HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS
1.0\Client
Source: SSL Certificate Settings deleted for endpoint (Event ID 15300)

Resources