We are trying to set up a replacement kube cluster that has a couple of micro services on it but the majority of code still lives in a cloud service. We are using the app-gateway ingress controller as our routing mechanism. Is it possible to have an app gateway ingress controller route all requests to the external (does not live in the kub cluster) resource except those paths as defined for the micoroservices? And if so, how do we go about setting it up?
this is dotnetcore for linux with helm as the orchestration.
the layout would look something like this...
default: /* => cloud service
microservice 1: /ms1/route1 or /ms1/route2 => internal kube deployment 1
microservice 2: /ms2/route1 => internal kube deployment 2
Is it possible to have an app gateway ingress controller route all
requests to the external (does not live in the kub cluster) resource
except those paths as defined for the micoroservices?
I don't think it's possible to route the requests to the resources that outside the Kubernetes. You can take a look at What is Ingress:
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.
internet
|
[ Ingress ]
--|-----|--
[ Services ]
I think you can route the outside requests to your microservices or your Kubernetes services through Azure Application Gateway.
it is possible. create a service with selectors or external service and point it to your cloud service and route requests from /* to it.
apiVersion: v1
kind: Service
metadata:
name: name
spec:
type: ExternalName
externalName: url-goes-here
Related
We are trying to deploy Microservices in AKS cluster and use Azure API Management as API Gateway. In order to prevent direct access to backend Microservices, planning to have both API Management and AKS part of a single VNet and force all API calls to be routed through API Management.
Following are the steps done to provision the services:
Create a new Virtual Network with Subnet configuration in Azure
Created Network Security Group and configured all required.
inbound/outbound port rules exclusions.
Configured Azure API Management in "external" networking mode, and completed steps as given in https://learn.microsoft.com/en-us/azure/api-management/api-management-using-with-vnet, by enabling "External" mode in Azure API Management and linked the newly created Virtual Network and Subnet in Networking section.
Created new AKS Cluster in the same Virtual Network and Since we plan to
use Istio as Service Mesh, created a new Istio Ingress Gateway with
below configuration to provision a new Azure Load Balancer to accept
requests on the Internal IP routed by API Management (Note the
inclusion of annotations to create a new Internal IP based Azure
Load Balancer):
apiVersion: v1
kind: Service
metadata:
name: istio-internal-ingressgateway
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
namespace: poc
spec:
type: LoadBalancer
selector:
istio: internal-ingressgateway
ports:
- port: 80
name: http
- port: 443
name: https
With the above setup completed, all backend API calls from API Management routed through the internal IP (through Azure Load Balancer) works as expected. As given in the below screenshot, calls made to http://10.228.80.227/api/k8s/GetClientIP does return with HTTP 200 (expected result).
Question:
Instead of using an interal IP (10.228.80.227, in this case), is there a way to define a (Private) DNS for the IP and make it resolve from API Management? For example, define a DNS alias entry for 10.228.80.227 as "api.contoso.com" and use http://api.contoso.com/api/k8s/GetClientIP and still make it resolve from API Management.
Based on some research tried the following but couldn't make it work all the way:
Provisioned a new Azure Private DNS Zone and configured the "Virtual network links" to point to the same Virtual Network.
Added a new "A" record set in the Private DNS Zone for 10.228.80.227 and api.contoso.com.
Updated the API Management to use the FQDN instead of IP in Settings:
As a final step, Initiated the Test from API Management but ended up with the following HTTP 500 error:
Looking into the Trace, got the following error that indicates that calls from API Management failed at DNS resolution:
forward-request (68.296 ms)
{
"messages": [
"Error occured while calling backend service.",
"The remote name could not be resolved: 'x3api.jha.com'"
] }
Note: All required host entries in Istio Ingress Gateway are added for new host to allow traffic routing and the service itself runs fine when accessed from within the POD/cluster using CURL.
Can someone throw light if any steps are being missed to make the DNS resolution work from API Management?
First of all I am pretty new on Kubernetes and containerized world.
My scenario is as follows:
I have a application which is deployed to AKS, we are using AGIC as ingress. The application is consuming endpoints hosted outside the AKS. The consumed application is publicly accessible but it has IP whitelisting. I am whitelisting the Application Gateway IP. Also I created a External Service as such.
kind: Service
apiVersion: v1
metadata:
name: service-endpoint
spec:
type: ExternalName
externalName: endpointname.something.com
ports:
- protocol: TCP
port: 433
But it does not work.
Additionally I tried to ping the direct endpoint URL(https://endpointname.something.com) from the pod, and I receive 403.
Can someone advice what would be the correct steps in order to achieve this connectivity?
Please note that we fixed this issue by whitelisting the public IP of the AKS load balancer to the target system.
I have deployed my kubernetes application on AKS, it uses the Internal Loadbalancer without a separate Ingress Controller.
Currently the service is accessible using http protocol and I want to secure the service. How do I enable https and where should I install the certificate?
The internal loadbalancer is a layer 4 service, you will need a layer 7 ingress controller like Azure Application Gateway Ingress Controller or roll your own ingress controller for SSL.
see:
Configuring an AKS load balancer for HTTPS access
what is the best way to access a web app running in aks container from outside the cluster with a name, which is already defined in Azure DNS zone? and an external DNS server can be helpful for this?
I would setup an ingress that would point to your service which exposes the web app.
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: simple-fanout-example
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: your.web.app.address
http:
paths:
- path: /
backend:
serviceName: service
servicePort: 8080
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource.
internet
|
[ Ingress ]
--|-----|--
[ Services ]
An Ingress may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name based virtual hosting. An Ingress controller is responsible for fulfilling the Ingress, usually with a load balancer, though it may also configure your edge router or additional frontends to help handle the traffic.
An Ingress does not expose arbitrary ports or protocols. Exposing services other than HTTP and HTTPS to the internet typically uses a service of type Service.Type=NodePort or Service.Type=LoadBalancer.
I would recommend reading Create an ingress controller in Azure Kubernetes Service (AKS), or use Azure Application Gateway as an ingress this is explained here and you can find tutorials on GitHub
I have an application setup on AKS (Azure Kubernetes Service) and I’m currently using Azure Application gateway as ingress resource for my application running on AKS.
Now after setting up ISTIO for my cluster the graphs are coming up fine except one part. Since the Azure APP gateway is unknown to ISTIO it is showing the resource as “unknown”. I even tried launching a virtual service and pointed it to the ingress resource but that didn’t have any effect on the graph. How shall I establish to ISTIO that it is Azure app gateway and not “unknown” resource.
This is because Azure Application gateway is not part of Istio Mesh. Depending on how You have Your Azure Application Gateway configured You might not even get any benefits of using istio.
Getting istio to work with Azure Application Gateway is lot more complicated than it seems.
There is a Github issue that uses istio and Azure Application Gateway at the same time.
With the following statement:
You may wonder why I chose to put the ingress resource into the istio-system namespace. Im doing so because in my understanding the istio-ingress must be the endpoint for each app-gateway redirect. If I would let it redirect to the echo-server service, AGKI(application-gateway-kubernetes-ingress) would point to the ip-address of the deployed pod, which would completely disregard istios servicemesh.
So if don't already have configuration like that and You want to use Istio I suggest setting Istio Ingress Gateway as an endpoint for Your Azure Application Gateway and treat it as traffic comming from outside mesh.
Here is an explanation why Azure Application gateway is "unknown" resource.
In an this article you can find the following statement:
Ingress traffic
Istio expects traffic to go via the the Ingress Gateway. When you see ‘unknown’ traffic it can simply be the case that you use the standard Kubernetes Ingress or an OpenShift route to send traffic from the outside to Istio.
Azure Application gateway uses custom ingress controller:
Application Gateway Ingress Controller (AGIC) allows you to use Application Gateway as the ingress for an Azure Kubernetes Service (AKS) cluster.
The ingress controller runs as a pod within the AKS cluster and consumes Kubernetes Ingress Resources and converts them to an Application Gateway configuration which allows the gateway to load-balance traffic to the Kubernetes pods. The ingress controller only supports Application Gateway V2 SKU.
For more information, see Application Gateway Ingress Controller (AGIC).
According to Kiali documentation:
In some situations you can see a lot of connections from an "Unknown" node to your services in the graph, because some software external to your mesh might be periodically pinging or fetching data. This is typically the case when you setup Kubernetes liveness probes, or have some application metrics pushed or exposed to a monitoring system such as Prometheus. Perhaps you wouldn’t like to see these connections because they make the graph harder to read.
To address Your additional question:
How shall I establish to ISTIO that it is Azure app gateway and not “unknown” resource.
As far as I know there is no way to make Custom (non-istio) Ingress Gateway be part of istio mesh. Leaving Azure Application Gateway labelled as “unknown”.
Hope this helps.
AFAIK, istio needs its own ingress gateway for apps.
Create an istio VirtualService and point it to istio's ingress gateway. The steps to do it are here and here.
Istio's ingress gateway for the app can be seen in the output of kubectl get gateway:
$ kubectl get gateway
NAME AGE
bookinfo-gateway 32s