I have deployed the Kubernetes cluster in the Azure platform. My application is hosted in azure docker and i deployed the docker image into my Kubernetes cluster. After deploying the pods are running fine. But I was not able to access the application from outside (postman/talentapi). Nginx Ingress controller is installed inside the cluster but still not able to access the application and getting no response error. Our application uses 2 different ports (5000 & 7003). Port 7003 is used to connect the application inside docker and port 5000 used to connect outside docker. Here by im sharing my service.yaml file,
kind: Service
apiVersion: v1
metadata:
name: kyc-service
namespace: kyc
spec:
#type: ClusterIP
selector:
app: kyc-app
ports:
- name: http
protocol: TCP
port: 5000
targetPort: 5000
- name: kycapp
protocol: TCP
port: 7003
targetPort: 7003
Related
I have an application that relies on a kafka service.
With Kafka connect, I'm getting an error when trying to curl localhost:8083, on the Linux VM that's running the kubernetes pod for Kafka connect.
curl -v localhost:8083 gives:
Rebuilt URL to: localhost:8083/
Trying 127.0.0.1...
connect to 127.0.0.1 port 8083 failed: Connection refused
Failed to connect to localhost port 8083: Connection refused
Closing connection 0
curl: (7) Failed to connect to localhost port 8083: Connection refused
kubectl get po -o wide for my kubernetes namespace gives:
When I check open ports using sudo lsof -i -P -n | grep LISTEN I don't see 8083 listed. The kafka connect pod is running and there's nothing suspicious in the logs for the pod.
There's a kubernetes manifest that I think was probably used to set up the Kafka connect service, these are the relevant parts. I'd really appreciate any advice about how to figure out why I can't curl localhost:8083
apiVersion: apps/v1
kind: Deployment
metadata:
name: kafka-connect
namespace: my-namespace
spec:
...
template:
metadata:
labels:
app: connect
spec:
containers:
- name: kafka-connect
image: confluentinc/cp-kafka-connect:3.0.1
ports:
- containerPort: 8083
env:
- name: CONNECT_REST_PORT
value: "8083"
- name: CONNECT_REST_ADVERTISED_HOST_NAME
value: "kafka-connect"
volumes:
- name: connect-plugins
persistentVolumeClaim:
claimName: pvc-connect-plugin
- name: connect-helpers
secret:
secretName: my-kafka-connect-config
---
apiVersion: v1
kind: Service
metadata:
name: kafka-connect
namespace: my-namespace
labels:
app: connect
spec:
ports:
- port: 8083
selector:
app: connect
You can't connect to a service running inside your cluster, from outside your cluster, without a little bit of tinkering.
You have three possible solutions:
Use a service with type NodePort or LoadBalancer to make the service reachable outside the cluster.
See the services and kubectl expose documentation.
Be aware, depending on your environment, this may expose the service to the internet.
Access using Proxy Verb: (see here)
This only works for HTTP/HTTPS. Use this if your service is not secure
enough to be exposed to the internet.
Access from pod running inside your cluster.
As you have noticed in the comments, you can curl from inside the pod. You can also do this from any other pod running in the same cluster. Pods can communicate with each other without any additional configuration.
Why can I not curl 8083 when I ssh onto the VM?
Pods/services are not reachable from outside the cluster, if not exposed using aforementioned methods (point 1 or 2).
Why isn't the port exposed on the host VM that has the pods?
It's not exposed on your VM, it's exposed inside your cluster.
I would strongly recommend going through Cluster Networking documentation to learn more.
I have a Kubernetes cluster inside Azure which holds some services and pods. I want to make those pods communicate with each other but when I try to execute a CURL/WGET from one to another, a timeout occurs.
The service YAMLs can be found below:
First service:
apiVersion: v1
kind: Service
metadata:
labels:
app: core-node
name: core-node
spec:
ports:
- name: "9001"
port: 9001
targetPort: 8080
selector:
app: core-node
status:
loadBalancer: {}
Second service:
apiVersion: v1
kind: Service
metadata:
labels:
app: core-python
name: core-python
spec:
ports:
- name: "9002"
port: 9002
targetPort: 8080
selector:
app: core-python
status:
loadBalancer: {}
When I am connecting to the "core-node" pod for example through sh and try to execute the following command, it gets a timeout. It happens also if I try for "core-python" pod to the other one.
wget core-python:9002
wget: can't connect to remote host (some ip): Operation timed out
I also tried using the IP directly and also trying to switch from ClusterIP to LoadBalancer, but the same thing happens. I have some proxy configuration as well but this is done mainly at Ingress level and should not affect the communication between PODS via service names, at least from what I know.
Pods are in running status and their APIs can be accessed through the public URLs exposed through Ingress.
#EDIT1:
I connected also to one of the PODs and checked if port 8080 is listening and it seems ok from my perspective.
netstat -nat | grep LISTEN
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN
#EDIT2:
When I do an endpoints check for this service, it returns the following:
kubectl get ep core-node
NAME ENDPOINTS AGE
core-node 10.x.x.x:8080 37m
If I try to wget this IP from the other pod, it responds:
wget 10.x.x.x:8080
Connecting to 10.x.x.x:8080 (10.x.x.x:8080)
wget: server returned error: HTTP/1.1 404 Not Found
Here is my docker image
FROM mcr.microsoft.com/dotnet/core/aspnet:2.2-alpine3.8 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443
COPY ./xyz/publish .
ENV ASPNETCORE_URLS=https://+:443;http://+80
ENTRYPOINT ["dotnet","abc/xyz.dll"]
Here is my Deployment.yaml file
apiVersion: apps/v1
kind: Deployment
metadata:
name: xyzdemo
namespace: default
spec:
replicas: 1
selector:
matchLabels:
papi: web
template:
metadata:
labels:
papi: web
spec:
containers:
- name: xyzdemo-site
image: xyz.azurecr.io/abc:31018
ports:
- containerPort: 443
imagePullSecrets:
- name: secret
---
apiVersion: v1
kind: Service
metadata:
name: xyzdemo-entrypoint
namespace: default
spec:
type: LoadBalancer
selector:
papi: web
ports:
- port: 44328
targetPort: 443
Here is my appsettings file
"Server": "xyz.database.windows.net",
"Database": "pp",
"User": "ita",
"Password": "password",
using all these i deployed the application in to the k8s cluster and am able to open the application from the browser, however when i try to get the info from the database, application is getting the network related error after a while.
System.Data.SqlClient.SqlException (0x80131904): A network-related or
instance-specific error occurred while establishing a connection to
SQL Server
I tried going inside to POD and did the ls command, i can see my application setting file as well as when Cat the application settings, i can see the correct credentials and i dont know what to do and not sure why is not able to connect to the database.
So finally i tried adding the sql connections as the env variables to the pod , then it started working. when i remove those its not connecting.
Now i removed the env variables which has the sql connections then did the log on the pod.
it says can't connect to the database: 'Empty' and server: 'Empty'
not sure why is it taking the empty when it has the details inside the applicationsettings.json file.
Well, I do not see what is the config for your k8's application to connect to database. Importantly, where is your database hosted? How can papi:web connect to database?
I also suspect your service is not having appropriate port re-direction. From your service.yaml above, https port of 443 is internally mapped to 44328. What is 44328? What is listening on that port? Your application seems to have no mention of 44328. (Refer Dockerfile)
I would improvise your service.yaml to look something like this:
apiVersion: v1
kind: Service
metadata:
name: xyzdemo-entrypoint
namespace: default //This is inferred anyways
spec:
selector:
papi: web
ports:
- name: http
protocol: TCP
port: 80
targetPort: xxxx //Where your web-server is listening. (From your dockerfile, this is also 80 but it can be any valid TCP port)
- name: https
protocol: TCP
port: 443
targetPort: xxxx //https for your web-server. (From your dockerfile, this is also 443. Again, can be any TCP port)
Opening up database server to internet is not a good practise. It's a big security threat. Good pattern is to facilitate your web-server communicate to database-server via internal dns that k8's maintain (This is assuming your database server is also a container - something like kubedb. If not, you're database server will have to be available via some sort of proxy that whitelists known hosts and only allows known hosts. eg - cloudsql proxy in GCP).
Depending on how your database server is hosted, you'll have to configure your db config to allow or whitelist your containerised application (The IP you get after applying service.yaml) Only then will your k8's app be able to talk/achieve connectivity to respective db.
I suspect you need to Allow connections on the Azure SQL firewall for this to work. Using the portal would be the easiest way. You can just allow all or allow Azure services for starters (assuming your Kubernetes is inside Azure). And narrow it down later (if this is the culprit).
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-firewall-configure#use-the-azure-portal-to-manage-server-level-ip-firewall-rules
I'm trying to expose a pod using a load balancer service. The service was created successfully and an external IP was assigned. When I tried accessing the external in the browser the site is no and I got ERR_CONNECTION_TIMED_OUT. Please see the yaml below:
apiVersion: v1
kind: Service
metadata:
labels:
name: service-api
name: service-api
spec:
externalTrafficPolicy: Cluster
ports:
- nodePort: 30868
port: 80
protocol: TCP
targetPort: 9080
name: http
selector:
name: service-api
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer: {}
I also tried creating the service using kubernetes CLI still no luck.
It looks like I have a faulty DNS on my k8s cluster. In order to resolve the issue, I have to restart the cluster. But before restarting the cluster, you can also delete all the pods in kube-system to refresh the DNS pods and if it's still not working I suggest restarting the cluster.
Battling with Kubernetes manifest on Azure. I have a simple api app running on port 443 (https). I simply want to run and replicate this app 3 times within a kubernetes cluster with a load balancer.
Kubernetes cluster:
My manifest file:
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: apiApp
spec:
replicas: 3
template:
metadata:
labels:
app: apiApp
spec:
containers:
- name: apiApp
image: {image name on Registry}
ports:
- containerPort: 443
hostPort: 443
---
apiVersion: v1
kind: Service
metadata:
name: apiApp
spec:
type: LoadBalancer
ports:
- name: https
port: 443
targetPort: 443
selector:
app: apiApp
In the above manifest the loadbalancer does not seem to find the app on port 443 within the container.
1) How can I create this manifest to link load balancer to port 443 of the containers and also expose the load balancer to the outside world on port 443.
2)How would manifest look like in multi cluster environment (same conditions as above)
For your issue, I did the test with the load balancer follow the document Deploy an Azure Kubernetes Service (AKS) cluster.
This example only has one pod, so I scale up the pod in to 3 with the command kubectl scale --replicas=3 deployment/azure-vote-front. The yaml file about scales and Load Balancer will like the screenshot below.
When the Cluster finish, I can access the service from Internet via Web Browse. And you can use the command az aks browse to go into the Kubernets dashboard to get a overview of the Kubernets Cluster.
Update
The Azure Kubernets Cluster is just a resource group like below and so as the load balancer: