Our AKS and Application Gateway are in different Vnets. From the AGIC documentation, the connection between AKS and Application Gateway is through a route table, as can be seen here AGIC Github.
However, we have a requirement that the route table should not exist between these two resources. AKS is using Kubenet and we cannot change it.
Is there another way to connect AKS and Application Gateway? Thanks.
To connect AKS and Application Gateway, please try the below:
You can enable AGIC on existing AKS cluster through Azure portal:
Enable the Application Gateway ingress controller and add the created application gateway like below:
If the above suggestion doesn't work in your scenario, then please refer the below links:
Managing traffic to AKS through Azure Application Gateway using Application gateway Ingress Controller by SUDHAKARA RAO SAJJA
Enable ingress controller add-on for existing AKS cluster with existing Azure application gateway | Microsoft Docs
Related
While adding subnet for creating a WAFV2 application gateway in azure, I tried to use the subnet used in WAFV1 gateway.
But it is showing an error "Subnet does not support application gateway tier".
In your virtual network, an application gateway is a dedicated deployment. The application gateway requires its own subnet within your virtual network.
In a subnet, you can have many instances of the same application gateway deployment.
Both v2 and v1 Application Gateway SKUs cannot be supported by the same subnet.
please refer this document.
I am deploying an API application on an existing application AKS cluster which uses the Azure CNI plugin. The deployment manifest are native k8s with kustomize. The resources getting deployed are API deployment with an NGINX Ingress controller and couple ingress routes for API itself and grafana and prometheus(through prom operator). I have on 1 ingress route added so far which just for accessing the API.
When I deploy the resources all of them get successfully deployed and a Public IP get assigned to the controller. However, when I try to hit public IP to fetch the response for the endpoint. And I am looking for some help to troubleshoot the issue.
After looking at setup a little but I realized a couple of things:
Load Balancer's resource group and the nodes in the agent pools have different resource groups.
The NSG inbound and outbound rules are not in the same resource group.
I am not sure what piece is missing. I tried changing the resource group of the load balancer it didn't do that failing with a validation error. I also ran the same setup with the default kubenet network plugin and it worked successfully. Any help on this will be greatly appreciated.
Q1: Load Balancer's resource group and the nodes in the agent pools have different resource groups.
The Azure AKS is an individual resource, but its components are not. You need to create the AKS cluster in a resource group. When creating the AKS cluster, Azure will create another resource group to put the components of the AKS. So there will be two resource groups when the AKS is created. Here are the details to help you better understand it.
And I guess you want to assign a static public IP address to the Ingress controller and create the static public IP in the resource group which the AKS resource in, not the components. If I'm right, then you need to assign the service principal of the AKS with a network role. Here are the steps that how to create a static public IP in another resource group.
Q2: The NSG inbound and outbound rules are not in the same resource group.
You do not need to care about the NSG inbound and outbound rules for AKS, Azure manage them itself. You just need to focus on the things that how to deploy applications in the AKS cluster.
I have an application setup on AKS (Azure Kubernetes Service) and I’m currently using Azure Application gateway as ingress resource for my application running on AKS.
Now after setting up ISTIO for my cluster the graphs are coming up fine except one part. Since the Azure APP gateway is unknown to ISTIO it is showing the resource as “unknown”. I even tried launching a virtual service and pointed it to the ingress resource but that didn’t have any effect on the graph. How shall I establish to ISTIO that it is Azure app gateway and not “unknown” resource.
This is because Azure Application gateway is not part of Istio Mesh. Depending on how You have Your Azure Application Gateway configured You might not even get any benefits of using istio.
Getting istio to work with Azure Application Gateway is lot more complicated than it seems.
There is a Github issue that uses istio and Azure Application Gateway at the same time.
With the following statement:
You may wonder why I chose to put the ingress resource into the istio-system namespace. Im doing so because in my understanding the istio-ingress must be the endpoint for each app-gateway redirect. If I would let it redirect to the echo-server service, AGKI(application-gateway-kubernetes-ingress) would point to the ip-address of the deployed pod, which would completely disregard istios servicemesh.
So if don't already have configuration like that and You want to use Istio I suggest setting Istio Ingress Gateway as an endpoint for Your Azure Application Gateway and treat it as traffic comming from outside mesh.
Here is an explanation why Azure Application gateway is "unknown" resource.
In an this article you can find the following statement:
Ingress traffic
Istio expects traffic to go via the the Ingress Gateway. When you see ‘unknown’ traffic it can simply be the case that you use the standard Kubernetes Ingress or an OpenShift route to send traffic from the outside to Istio.
Azure Application gateway uses custom ingress controller:
Application Gateway Ingress Controller (AGIC) allows you to use Application Gateway as the ingress for an Azure Kubernetes Service (AKS) cluster.
The ingress controller runs as a pod within the AKS cluster and consumes Kubernetes Ingress Resources and converts them to an Application Gateway configuration which allows the gateway to load-balance traffic to the Kubernetes pods. The ingress controller only supports Application Gateway V2 SKU.
For more information, see Application Gateway Ingress Controller (AGIC).
According to Kiali documentation:
In some situations you can see a lot of connections from an "Unknown" node to your services in the graph, because some software external to your mesh might be periodically pinging or fetching data. This is typically the case when you setup Kubernetes liveness probes, or have some application metrics pushed or exposed to a monitoring system such as Prometheus. Perhaps you wouldn’t like to see these connections because they make the graph harder to read.
To address Your additional question:
How shall I establish to ISTIO that it is Azure app gateway and not “unknown” resource.
As far as I know there is no way to make Custom (non-istio) Ingress Gateway be part of istio mesh. Leaving Azure Application Gateway labelled as “unknown”.
Hope this helps.
AFAIK, istio needs its own ingress gateway for apps.
Create an istio VirtualService and point it to istio's ingress gateway. The steps to do it are here and here.
Istio's ingress gateway for the app can be seen in the output of kubectl get gateway:
$ kubectl get gateway
NAME AGE
bookinfo-gateway 32s
I am able to create aks cluster with advance networking. able to integrate application load balancer also with this AKS cluster but i am unable to find any way to integrate azure api gateway with aks.
Using Application Gateway as an Ingress controller for AKS is in a beta state at the moment (as shown on the Github page - https://github.com/Azure/application-gateway-kubernetes-ingress) and so I don't believe there will be any support for setting it up with Terraform until it gets to GA.
You might be able to do something with exec resources to set it up, but that would be up to you to figure out.
Unfortunately, it seems there is no way to integrate the application load balancer with the AKS cluster directly. And you can see all the things you can set for AKS here.
But you can integrate the application load balancer with AKS cluster when you take knowledge of AKS internal load balancer and Application gateway backend pool addresses. You can take a look at the steps that how to integrate application gateway with AKS cluster.
First of all, you need to make a plan for the AKS cluster network and take an exact IP address for the application gateway backend pool address in the Terraform. Hope this will help you if there any more question you can give me the message.
I am in process to integrate Azure API gateway with AKS.
in order to achieve this.
i had
created AKS cluster with advance networking. consider it is in
aks-vnet
created different vnet for API gateway.
peer both vnet i.e. vnet for aks and for api gateway.
created internal load balancer. by
running azure voting app yaml file also added annotation for using
internal load balancer
metadata:
name: azure-vote-front
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
added kubernates internal load balancer public ip in address probe of api gateway.
when i navigate in browser with public ip address of api gateway or dns name associated with api gateway i am getting following error
502 - Web server received an invalid response while acting as a gateway or proxy server.
There is a problem with the page you are looking for, and it cannot be displayed. When the Web server (while acting as a gateway or proxy) contacted the upstream content server, it received an invalid response from the content server.
First, there is something wrong described in your case: You should provide add the kubernetes internal load balancer private IP to the application gateway backend pool.
Then I did the test as the steps in Integrate Application Gateway with AKS cluster. As the error shows that you should make the check if the application in the AKS works fine, in your case the application is azure voting app.
If you follow the Azure Kubernetes Service tutorial and push the image to Azure Container Registry, you must grant the AKS permission to access your ACR. See Authenticate with Azure Container Registry from Azure Kubernetes Service.
And on my side, I can access the azure voting app from the public IP of the application gateway. Hope this will help you.