Azure App Gateway with Internal API Management 503 backend server error - azure

I am following up this doc series to set up an internal API management instance integrated with APP Gateway in azure. I followed everything to the detail:
Created a new resource group
Setup a Vnet with 3 subnets
And setup a private dns zone and link the vnet
And then created self-signed certificates to be used with the dns created in private dns zone
Created API management instance and added custom domains
Created a App Gateway with public IP and setup routing rules and backends and setup health probes with path /status-0123456789abcdef for APIM
But now I am getting this backend health error as below:
Can someone tell me what I am doing wrong?
Are there any security groups to be configured? I am using an internl mode for the APIM, and when I even try to test the default API (which is echo test) it gives the below error:
Why this is not working? If you need any more information, I will let you know (update the question). Can someone please help me?

I have a similar situation which was driving me insane. I must have changed everything I possibly could. The answer, was to create a custom health probe and at the very bottom of the HTTP settings it was an option to use the custom probe.

Since the Gateway URL is not registered on the public DNS, the test console available on the Azure portal will not work for Internal VNET deployed service. Instead, use the test console provided on the Developer portal.
You can find more details here.

Related

Getting error while accessing Azure API developer portal while using Azure Application Gateway with Azure API Management service

We have following setup in Azure configured using this guide (https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/apis/protect-apis)
App Gateway with Public IP calling ===> Azure APIM Service configured in Internal mode (development tier)
Details:
Both App Gateway and APIM are in same VNet but in separate subnets.
App Gateway has only public IP with Firewall disabled (for testing purpose).
APIM is configured in Internal mode.
Three listeners, one for API, one for developer portal and one for API management endpoint) are configured in App Gateway using custom domains and SSL certificates.
There is VM deployed and configure in same VNet.
APP Gateway Domains are: (Internet Accessible)
dev.apiportal.xxxx.com ==> API Portal
dev.api.xxxx.com ==> API
Gateway
dev.apimanagement.xxxx.com ==> API Management
APIM Service is using default domains: (Not accessible on internet. APIM is configured in Internal mode)
dev-apim.developer.azure-api.net ==> API Portal
dev-apim.azure-api.net ==> API Gateway
dev-apim.management.azure-api.net ==> API Management
There probes defined on App GW for these endpoint mentioned above are healthy and in green status
Problem Statement:
API portal is accessible from VM mentioned in #5 above using default links. I am able to login using user added in portal. VM's host file has been modified with IP address and default host name of APIM service. Portal works fine when accessed from VM.
When I try to access portal from internet using custom domain (configured on listeners mentioned in #4 above of App GW) portal loads fine.
When I try to login using same user (mentioned in #1) doesn't work and error I get is "Please provide a valid email and password". This error appears after almost 15-20 seconds after clicking sign in button.
When I tried to check error in browser developer tools found that even though portal is accessed using internet facing URL one request for management Api is pointed to internal URL of Api management. .
Wanted to understand if we need to have custom domains for APIM service as well in order to get this setup working? Any pointer to fix this error?
• Thank you #Pankaj Kapare for your question and the subsequent resolution that you have provided in the comments section. I am posting the same comment of yours elaborately and some additional probable resolution as an answer.
Setting the APIM service’s URls with the custom domain that is mentioned in the application gateway while still the APIM service operational in internal mode helped login to the APIM developer portal successfully, i.e., setting the same custom domain that are defined in application gateway in the APIM’s endpoints.
• You can also resolve this DNS mapping issue by creating a CNAME record for the custom domain configured in application gateway and mentioning an alias to the default domain URL in the APIM service. Thus, by creating a CNAME DNS record for the default domain in the APIM service, you can redirect the requests for the application gateway’s custom domain URL to the default domain URL in the APIM service.
But for this also, you would need to create a private DNS zone in Azure and create the above said in it which will restrict the internet access to the APIM URLs and redirect the requests internally.

Azure Application Gateway API Management probe cannot connect to back end

I hope somebody can help to understand what I am doing wrong here because I am totally confused and lost.
I am trying to build an API Management in internal mode, and have in front of it a application gateway.
Following Microsoft Documentation I build the following resource:
API Management
Application Gateway
Virtual network
In the virtual network I set 2 subnets(application gateway and API Management)
2 Network Security groups one for each resources
As per the documentation and general advice I found online. I created a Keyvault and generated a certificate. In the Subject I set this CN:
api.test.com
I assigned a Managed identity to this KeyVault.
After this step I created a API Management Service. and the only api inside is a
/configurations
Once this was done. In the Newtork Tab I set the Api management to the internal mode and selected my virtual network and the subnet I designed for this service. So far everything went smooth. When the update completed I set the custom domain in the api management.
In the Tab Custom Domain I added a new domain, in the hostname I set the same CN I set in the KeyVault
api.test.com
and selected the KeyVault from which it has to fetch the right cert.
Everything is done here.
I created the Application gateway in the designed virtual network and subnet.
the first thing I set the backend pool to the gateway url of the API Management
api.test.com
I set a HTTP settings over protocol HTTPS port 443 as follow
Still in the application Gateway I set the Listeners on port 443 and selected my certificate from the KeyVault
In the Rules I configured the listeners and the back end targets to target the backend pool.
At this point, when I test the Probe:
I get the following error
Cannot connect to backend server. Check whether any NSG/UDR/Firewall is blocking access to the server. Check if application is running on correct port.
I checked both my security groups which are set as follow
this is the msg for the apim
and this for the application gateway
Can please anyone help understand what I am doing wrong here? Because I have no clue anyomore what could be the issue.
And please, if you need anymore info don't hesitate to let me know. And if is easy, I can post here my terraform script to deploy this infra.
You said: the first thing I set the backend pool to the gateway url of the API Management "api.test.com"
That url is inaccessible and points to a public IP (which I guess should be the app gateway IP)
The backend pool should be the private SLB IP address of the api management service. the listener should be listening to the host name so when it receives a request with that host name, it forwards it to the api management service through its private IP and a host header holding the same listener host name as a value.

External api management service can't resolve Azure function service name

Currently I am trying to use Api Management to expose an Azure Function http trigger that is within a vnet.
After setting everything up, I tried a simple GET request and I'm getting a 500 error telling me
"messages": [
"Error occured while calling backend service.",
"The remote name could not be resolved: '<function-name>.azurewebsites.net'"
]
My Azure function was:
Created with a premium plan.
enabled inbound traffic with a private endpoint.
deployed in the same Vnet as my api management service.
Added a httpTrigger template from the portal
Was set up with a Azure managed DNS name
My api management service was created in external mode and I linked my function to Api Management in the portal.
Things I've tried:
I've double checked that the correct security group rules have been opened for api management to work
I've ensured that both my api management (external) is in the same vnet as my Serverless function
I deployed a vm to the same virtual network and was unable to resolve the dns name of my function there. Used ping, telnet, nslookup.
Tried adding application settings to my function to specify the azure dns server, among other settings.
Ive check that my private dns zone is linked with my vnet.
Seems like a DNS name resolving issue, but I can't seem to fix it. Any ideas on what could be causing this error?
update
So it seems to be an issue with my private endpoint. Every function I create without one works fine, but I would still like to have the private endpoint so it isn't accessible from the internet.
I know I could probably lockdown my function to only be called from the api management gateway ip, but I would rather not have to hard code IPs if I don't have too.
Thank you MayankBargali-MSFT | Microsoft Docs Posting your suggestion as answer to help other community members.
As per the error, the APIM is not able to resolve your azure function
app. Can you please verify if the custom DNS is correctly setup and
you can refer to this
document
for more details. Outbound access on port 53 is required for
communication with DNS servers. If a custom DNS server exists on the
other end of a VPN gateway, the DNS server must be reachable from the
subnet hosting API Management. I will also suggest you to review this
document for the setup part.
Reference: External api management service can't resolve Azure function service name - Microsoft Q&A

Azure App Service behind Azure Application Gateway

I'm trying to serve multiple Azure App Services behind an Application Gateway. These services should only be reachable through the application gateway endpoint. To achieve this, I've done the following:
Added a VNet
Added a subnet, and added all app services to it
Added an application gateway with appropriate rules for pointing to the app service
Added access restrictions to the app service to only allow the subnet of the application gateway, and the public IP of the application gateway
The health probe of the application gateway indicates the service is healthy (and thus reachable), but when I try to make a request to the service trough the gateway using Postman, I get a 403 IP forbidden error. However, when I add my local IP as an allowed IP address to the app service, the request comes through just fine
From what I've read online, adding the public IP of the gateway should be sufficient for allowing access through the gateway, but it seems that requests from my local machine are blocked. Am I getting something fundamentally wrong here?
Edit: I also tried assigning a front-end private IP address to the gateway and adding access for that in the app service, but that also did not work
Edit 2: My configuration
Access restrictions for app service:
access restrictions
Backend pool:
Backend pool
HTTP settings:
HTTP settings
Listener:
Listener
Rule:
Rule
From your description "Added a subnet, and added all app services to it", I assume that you are meaning integrating app with Azure VNet or enable the service endpoint with Microsoft.Web for the subnet. If so, you could remove them. Both are useless in this scenario.
To restrict the access through the gateway, you only need to add the frontend IP address to the Azure App Service access restrictions. For more details, you could refer to this blog https://www.cloudmanav.com/azure/restricting-appservice-accessible-via-appgateway/#
If you have NSG associated with the subnets, you could follow the prerequisites here
https://learn.microsoft.com/en-us/azure/application-gateway/configuration-overview#network-security-groups-on-the-application-gateway-subnet
Edit
You may have redirection configured on your app service or have Azure Active Directory authentication, which causes the redirection. So when redirection happens, the client makes the request directly to app service URL Path contoso.azurewebsites.net/xxx instead of going through the application gateway URL path contoso.com/xxx.
To fix it, you could use a custom domain name to pass the same host name that the application gateway receives to the app service as well, instead of doing a host override. Get more details about this solution here.

Azure private DNS zone resolving

We have a private DNS zone setup for the zone project.local. For app service instances living in an app service environment, each service has its own record pointing to the load balancer in front of the service (so all have the same IP).
We have an App Gateway instance linked to a public IP in front to make this all publiccally available. The gateway is available via a public URL and routes the request to the load balancer.
Now what we see is the following:
From external, everything is fine. We can get to the services using the external URL, gateway forwards it and all is well.
From internal, we want to use the internal DNS address set in the private zone. This is not working, calls from service to service throw an error stating that the host URL could not be resolved.
When I log into a VM in the same vNET or use the Kudu console, I'm able to resolve the DNS address to the correct IP. What I do notice is that when using nslookup, it says it's getting a non authoritative answer.
It's very hard to get any more information for debug purposes. We're not sure why resolving isn't working as per documentation these records should work for all of the components in the same vNET. The authoritative error might be related, but again: not sure. So any ideas on what else to check would be highly appreciated.
Disclaimer: I also have a support ticket open for the same question, but wanted to put this out there to see if there's anyone else who might have encountered the same since this is pretty new tech.
Azure DNS Private Zones are able to resolve names between VMs and Cloud services. It does not look like it can be used by Azure Web Apps or Azure App Services at this time. 
You can see more information on name resolution for resources in Azure Virtual Networks Here.
If you would like to request this feature be added to DNS Private Zones, you can leave your feedback Here.

Resources