Azure Application Gateway API Management probe cannot connect to back end - azure

I hope somebody can help to understand what I am doing wrong here because I am totally confused and lost.
I am trying to build an API Management in internal mode, and have in front of it a application gateway.
Following Microsoft Documentation I build the following resource:
API Management
Application Gateway
Virtual network
In the virtual network I set 2 subnets(application gateway and API Management)
2 Network Security groups one for each resources
As per the documentation and general advice I found online. I created a Keyvault and generated a certificate. In the Subject I set this CN:
api.test.com
I assigned a Managed identity to this KeyVault.
After this step I created a API Management Service. and the only api inside is a
/configurations
Once this was done. In the Newtork Tab I set the Api management to the internal mode and selected my virtual network and the subnet I designed for this service. So far everything went smooth. When the update completed I set the custom domain in the api management.
In the Tab Custom Domain I added a new domain, in the hostname I set the same CN I set in the KeyVault
api.test.com
and selected the KeyVault from which it has to fetch the right cert.
Everything is done here.
I created the Application gateway in the designed virtual network and subnet.
the first thing I set the backend pool to the gateway url of the API Management
api.test.com
I set a HTTP settings over protocol HTTPS port 443 as follow
Still in the application Gateway I set the Listeners on port 443 and selected my certificate from the KeyVault
In the Rules I configured the listeners and the back end targets to target the backend pool.
At this point, when I test the Probe:
I get the following error
Cannot connect to backend server. Check whether any NSG/UDR/Firewall is blocking access to the server. Check if application is running on correct port.
I checked both my security groups which are set as follow
this is the msg for the apim
and this for the application gateway
Can please anyone help understand what I am doing wrong here? Because I have no clue anyomore what could be the issue.
And please, if you need anymore info don't hesitate to let me know. And if is easy, I can post here my terraform script to deploy this infra.

You said: the first thing I set the backend pool to the gateway url of the API Management "api.test.com"
That url is inaccessible and points to a public IP (which I guess should be the app gateway IP)
The backend pool should be the private SLB IP address of the api management service. the listener should be listening to the host name so when it receives a request with that host name, it forwards it to the api management service through its private IP and a host header holding the same listener host name as a value.

Related

External api management service can't resolve Azure function service name

Currently I am trying to use Api Management to expose an Azure Function http trigger that is within a vnet.
After setting everything up, I tried a simple GET request and I'm getting a 500 error telling me
"messages": [
"Error occured while calling backend service.",
"The remote name could not be resolved: '<function-name>.azurewebsites.net'"
]
My Azure function was:
Created with a premium plan.
enabled inbound traffic with a private endpoint.
deployed in the same Vnet as my api management service.
Added a httpTrigger template from the portal
Was set up with a Azure managed DNS name
My api management service was created in external mode and I linked my function to Api Management in the portal.
Things I've tried:
I've double checked that the correct security group rules have been opened for api management to work
I've ensured that both my api management (external) is in the same vnet as my Serverless function
I deployed a vm to the same virtual network and was unable to resolve the dns name of my function there. Used ping, telnet, nslookup.
Tried adding application settings to my function to specify the azure dns server, among other settings.
Ive check that my private dns zone is linked with my vnet.
Seems like a DNS name resolving issue, but I can't seem to fix it. Any ideas on what could be causing this error?
update
So it seems to be an issue with my private endpoint. Every function I create without one works fine, but I would still like to have the private endpoint so it isn't accessible from the internet.
I know I could probably lockdown my function to only be called from the api management gateway ip, but I would rather not have to hard code IPs if I don't have too.
Thank you MayankBargali-MSFT | Microsoft Docs Posting your suggestion as answer to help other community members.
As per the error, the APIM is not able to resolve your azure function
app. Can you please verify if the custom DNS is correctly setup and
you can refer to this
document
for more details. Outbound access on port 53 is required for
communication with DNS servers. If a custom DNS server exists on the
other end of a VPN gateway, the DNS server must be reachable from the
subnet hosting API Management. I will also suggest you to review this
document for the setup part.
Reference: External api management service can't resolve Azure function service name - Microsoft Q&A

Azure App Gateway with Internal API Management 503 backend server error

I am following up this doc series to set up an internal API management instance integrated with APP Gateway in azure. I followed everything to the detail:
Created a new resource group
Setup a Vnet with 3 subnets
And setup a private dns zone and link the vnet
And then created self-signed certificates to be used with the dns created in private dns zone
Created API management instance and added custom domains
Created a App Gateway with public IP and setup routing rules and backends and setup health probes with path /status-0123456789abcdef for APIM
But now I am getting this backend health error as below:
Can someone tell me what I am doing wrong?
Are there any security groups to be configured? I am using an internl mode for the APIM, and when I even try to test the default API (which is echo test) it gives the below error:
Why this is not working? If you need any more information, I will let you know (update the question). Can someone please help me?
I have a similar situation which was driving me insane. I must have changed everything I possibly could. The answer, was to create a custom health probe and at the very bottom of the HTTP settings it was an option to use the custom probe.
Since the Gateway URL is not registered on the public DNS, the test console available on the Azure portal will not work for Internal VNET deployed service. Instead, use the test console provided on the Developer portal.
You can find more details here.

Azure App Service behind Azure Application Gateway

I'm trying to serve multiple Azure App Services behind an Application Gateway. These services should only be reachable through the application gateway endpoint. To achieve this, I've done the following:
Added a VNet
Added a subnet, and added all app services to it
Added an application gateway with appropriate rules for pointing to the app service
Added access restrictions to the app service to only allow the subnet of the application gateway, and the public IP of the application gateway
The health probe of the application gateway indicates the service is healthy (and thus reachable), but when I try to make a request to the service trough the gateway using Postman, I get a 403 IP forbidden error. However, when I add my local IP as an allowed IP address to the app service, the request comes through just fine
From what I've read online, adding the public IP of the gateway should be sufficient for allowing access through the gateway, but it seems that requests from my local machine are blocked. Am I getting something fundamentally wrong here?
Edit: I also tried assigning a front-end private IP address to the gateway and adding access for that in the app service, but that also did not work
Edit 2: My configuration
Access restrictions for app service:
access restrictions
Backend pool:
Backend pool
HTTP settings:
HTTP settings
Listener:
Listener
Rule:
Rule
From your description "Added a subnet, and added all app services to it", I assume that you are meaning integrating app with Azure VNet or enable the service endpoint with Microsoft.Web for the subnet. If so, you could remove them. Both are useless in this scenario.
To restrict the access through the gateway, you only need to add the frontend IP address to the Azure App Service access restrictions. For more details, you could refer to this blog https://www.cloudmanav.com/azure/restricting-appservice-accessible-via-appgateway/#
If you have NSG associated with the subnets, you could follow the prerequisites here
https://learn.microsoft.com/en-us/azure/application-gateway/configuration-overview#network-security-groups-on-the-application-gateway-subnet
Edit
You may have redirection configured on your app service or have Azure Active Directory authentication, which causes the redirection. So when redirection happens, the client makes the request directly to app service URL Path contoso.azurewebsites.net/xxx instead of going through the application gateway URL path contoso.com/xxx.
To fix it, you could use a custom domain name to pass the same host name that the application gateway receives to the app service as well, instead of doing a host override. Get more details about this solution here.

Set kubernetes VM with nodeports as backend for application gateway

I have two VMs that are part of a kubernetes cluster. I have a single service that is exposed as NodePort (30001). I am able to reach this service on port 30001 through curl on each of these VMs. When I create an Azure application gateway, the gateway is not directing traffic to these VMs.
I've followed the steps for setting up the application gateway as listed in the Azure documentation.
I constantly get a 502 from the gateway.
In order for the Azure Application Gateway to redirect or route traffic to the NodePort you need to add the Backend servers to the backend pool inside the Azure Application Gateway.
There are options to choose Virtual Machines as well.
A good tutorial explaining how to configure an application gateway in azure and direct web traffic to the backend pool is:
https://learn.microsoft.com/en-us/azure/application-gateway/quick-create-portal
I hope this solves your problem.
So I finally ended up getting on a call with the support folks. It turned out that the UI on Azure's portal is slightly tempremental.
For the gateway to be able to determine which of your backends are healthy it needs to have a health probe associated with the HTTP setting (the HTTP Setting is the one that determines how traffic from the gateway flows to your backends).
Now, when you are configuring the HTTP setting, you need to select the "Use Custom Probe" but when you do that it doesn't show the probe that you have already created. Hence, I figured that wasn't required.
The trick to first check the box below "Use Custom probe" which reads "Pick hostname from backend setttings", and then click on custom probe and your custom probe will show up and things will work.

Azure application Gateway WAF

I am trying to configure Azure application gateway WAF with a backendpool set to a VM in a different Azure tenant using its public IP address on port 443. All the SSL certificates are configured properly. However, I keep getting following error while browsing the site via WAF.
502 - Web server received an invalid response while acting as a gateway or proxy server.
I have confirmed that the NSG is on the mentioned back-end VM is allowing all traffic on port 443. What could be going wrong here?
Generally, you can check the status of Backend health in the monitoring of Application Gateway and compare with the DETAILS referring to these possible reasons on your side.
NSG, UDR or Custom DNS is blocking access to backend pool members.
Back-end VMs or instances of virtual machine scale set are not responding to the default health probe.
Invalid or improper configuration of custom health probes.
Azure Application Gateway's back-end pool is not configured or empty.
None of the VMs or instances in virtual machine scale set are healthy.
Request time-out or connectivity issues with user requests.
For each reason, you can get a solution from that link. I think you could make sure you can directly access the backend with public IP from one tenant to another tenant. Then if you have an NSG in the app gateway subnet, you must include exceptions for incoming traffic on ports 65503-65534 for the Application Gateway v1 SKU, and ports 65200-65535 for the v2 SKU. You could get more details here. You could also whitelist the app gateway public IP address in the NSG of backend VM.

Resources