So I've read a bunch of these similar questions/issues on stackoverflow and I understand it enough but not sure what I am missing.
deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: dev-namespace
labels:
web: nginx
spec:
replicas: 2
selector:
matchLabels:
web: nginx
template:
metadata:
labels:
web: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 8080
service.yml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: NodePort
selector:
web: nginx
ports:
- protocol: TCP
port: 80
targetPort: 8080
This is my minikube ip:
$ minikube ip
192.168.49.2
This is the service
$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-service NodePort 10.104.139.228 <none> 80:30360/TCP 14
This is the deployment
$ kubectl get deployments.apps
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 2/2 2 2 14h
This is the pods
$ kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deployment-5b78696cc8-9fpmr 1/1 Running 0 14h 172.17.0.6 minikube <none> <none>
nginx-deployment-5b78696cc8-h4m72 1/1 Running 0 14h 172.17.0.4 minikube <none> <none>
This is the endpoints
$ kubectl get endpoints
NAME ENDPOINTS AGE
nginx-service 172.17.0.4:8080,172.17.0.6:8080 14h
But when I try to curl 10.104.139.228:30360 it just hangs. When I try to curl 192.168.49.2:30360 I get the Connection refused
I am sure that using NodePort means I need to use the node ip and that would be the server's local IP since I am using minikube and control plane and worker are in the same server.
What am I missing here? Please help, this is driving me crazy. I should mention that I am able to kubectl exec -ti pod-name -- /bin/bash and if I do a curl localhost I do get the famous response "Welcome to NGINX"
Nevermind :/ I feel very foolish I see that the mistake was the container ports :/ my nginx pods are listening on port 80 not port 8080
For anyone out there, I updated my config files to this:
service.yml
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: busy-qa
spec:
type: NodePort
selector:
web: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
deployment.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: busy-qa
labels:
web: nginx
spec:
replicas: 2
selector:
matchLabels:
web: nginx
template:
metadata:
labels:
web: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
Now when I curl I get the NGINX response
$ curl 192.168.49.2:31168
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
nginx.org.<br/>
Commercial support is available at
nginx.com.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Related
I have install kubeadm in master and connected 2 worker node to it, after that i deploy nginx pod and ssh into that nginx pod after that i type nslookup google.com or apt update its not working got connection timeout it think due its not connecting to internet. How to solve it, The 3 VM is running in azure portal and 3 VM are connected to together. kubectl v1.24.2 im using it
The 3 VM is running in azure portal and 3 VM are connected to together. kubectl v1.24.2 im using it and calico network also im using
nginx pod is running in worker2 and services of containerd container runtime/Docker Application Container Engine both are in running state. if i type lsmod | grep br_netfilter i got
br_netfilter 28672 0
bridge 266240 1 br_netfilter
here is my nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
# command: ["/bin/sh","-c"]
# args: ["apt update"]
# securityContext:
# privileged: true
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
type: ClusterIP
selector:
app: nginx
ports:
- port: 8080
targetPort: 80
name: nginx-http
And Here is the screenshot of coredns install in kube-system naming space
Seems like name is not resolving, please check if coredns is working properly. You can break down the issue like this:
from inside the pod first check you have network reachability to the internet: curl -v telnet://8.8.8.8:53
check you have connectivity to core DNS: curl -v telnet://coredns_service_name:53 or curl -v telnet://coredns_cluster_ip:53
I am trying to create a multicontainer pod for a simple demo. I have an app that is build in docker containers. There are 3 containers
1 - redis server
1 - node/express microservice
2 - node/express/react front end
All 3 containers are deployed successfully and running.
I have created a public load balancer, which is running without any errors.
I cannot connect to the front end from the public ip.
I have also run tcpdump in the frontend container and there is no traffic getting in.
Here is my yaml file used to create the deployment and service
apiVersion: apps/v1
kind: Deployment
metadata:
name: mydemoapp
spec:
replicas: 1
selector:
matchLabels:
app: mydemoapp
template:
metadata:
labels:
app: mydemoapp
spec:
nodeSelector:
"beta.kubernetes.io/os": linux
containers:
- name: microservices-web
image: mydemocr.azurecr.io/microservices_web:v1
ports:
- containerPort: 3001
- name: redislabs-rejson
image: mydemocr.azurecr.io/redislabs-rejson:v1
ports:
- containerPort: 6379
- name: mydemoappwebtest
image: mydemocr.azurecr.io/jsonformwebtest:v1
ports:
- containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
name: mydemoappservice
spec:
type: LoadBalancer
ports:
- protocol: TCP
port: 80
targetPort: 3000
selector:
app: mydemoapp
This is what a describe of my service looks like :
Name: mydemoappservice
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"mydemoappservice","namespace":"default"},"spec":{"ports":[{"p...
Selector: app=mydemoapp
Type: LoadBalancer
IP: 10.0.104.159
LoadBalancer Ingress: 20.49.172.10
Port: <unset> 80/TCP
TargetPort: 3000/TCP
NodePort: <unset> 31990/TCP
Endpoints: 10.244.0.17:3000
Session Affinity: None
External Traffic Policy: Cluster
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal EnsuringLoadBalancer 24m service-controller Ensuring load balancer
Normal EnsuredLoadBalancer 24m service-controller Ensured load balancer
One more weirdness is that when I run the docker container from the front end I can get a shell and run curl localhost:3000 and get some output but when I do it in the az container I get the following response after some delay
curl: (52) Empty reply from server
As to why this container works on my machine and not in azure is another layer to the mystery.
Referring from docs here the container need to listen on 0.0.0.0 instead of 127.0.0.1 because
any port which is listening on the default 0.0.0.0 address inside a
container will be accessible from the network
.
I have a service which exposes a "hello world" web deployment in "develop" namespace.
Service YAML
kind: Service
apiVersion: v1
metadata:
name: hello-v1-svc
spec:
selector:
app: hello-v1
ports:
- protocol: TCP
port: 80
targetPort: 8080
To test if the page is working properly, I run "kubectl port-forward" and the page is displayed successfully using the public IP.
Edit:
Then the Ingress is deployed but the page only displays within vnet address space
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
app: app
version: 1.0.0
name: dev-ingress
namespace: develop
spec:
rules:
- http:
paths:
- backend:
serviceName: hello-v1-svc
servicePort: 80
path: /
Ingress Rules
Rules:
Host Path Backends
---- ---- --------
*
/ hello-v1-svc:80 (10.1.1.13:8080,10.1.1.21:8080,10.1.1.49:8080)
What step am I skipping for the page to display?
First of all to answer your comment:
Maybe it is related to this annotation:
"service.beta.kubernetes.io/azure-load-balancer-internal". Controller
is set to "True"
service.beta.kubernetes.io/azure-load-balancer-internal: "true" annotation is typicaly used when you create an ingress controller to an internal virtual network. In case of this annotation , the ingress controller is configured on an internal, private virtual network and IP address. No external access is allowed.
You can find more info in Create an ingress controller to an internal virtual network in Azure Kubernetes Service (AKS) article.
Reproduced your case, created AKS cluster and applied below yamls. It works as expected so please use
apiVersion: v1
kind: Namespace
metadata:
name: develop
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world-deployment
namespace: develop
labels:
app: hello-v1
spec:
selector:
matchLabels:
app: hello-v1
replicas: 2
template:
metadata:
labels:
app: hello-v1
spec:
containers:
- name: hello-v1
image: paulbouwer/hello-kubernetes:1.8
ports:
- containerPort: 8080
---
kind: Service
apiVersion: v1
metadata:
name: hello-v1-svc
spec:
type: ClusterIP
selector:
app: hello-v1
ports:
- protocol: TCP
port: 80
targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: dev-ingress
namespace: develop
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- http:
paths:
- backend:
serviceName: hello-v1-svc
servicePort: 80
path: /
List of my services
vitalii#Azure:~$ kubectl get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default hello-v1-svc ClusterIP 10.0.20.206 <none> 80/TCP 19m
default kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 83m
ingress-basic nginx-ingress-controller LoadBalancer 10.0.222.156 *.*.*.* 80:32068/TCP,443:30907/TCP 53m
ingress-basic nginx-ingress-default-backend ClusterIP 10.0.193.198 <none> 80/TCP 53m
kube-system dashboard-metrics-scraper ClusterIP 10.0.178.224 <none> 8000/TCP 83m
kube-system healthmodel-replicaset-service ClusterIP 10.0.199.235 <none> 25227/TCP 83m
kube-system kube-dns ClusterIP 10.0.0.10 <none> 53/UDP,53/TCP 83m
kube-system kubernetes-dashboard ClusterIP 10.0.115.184 <none> 443/TCP 83m
kube-system metrics-server ClusterIP 10.0.199.200 <none> 443/TCP 83m
Sure for security reasons I hide EXTERNAL-IP of nginx-ingress-controller. Thats the IP you should use to access page.
MOre information and exampke you can find in Create an ingress controller in Azure Kubernetes Service (AKS) article
I have created the Kubernetes Cluster using two Azure Ubuntu VMs. I am able to deploy and access pods and deployments using the Nodeport service type. I have also checked the pod's status in Kube-system namespace. All of the pod's status showing as running. but, whenever I mention service type to Loadbalancer, it was not creating the LoadBalancer IP and it's status always showing as pending. I have also created an Ingress controller for the Nginx service. still, it is not creating an ingress Address. While initializing the Kubernetes master, I am using the following command.
kubeadm init
Below is deployment, svc and Ingress manifest files.
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
replicas: 3
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
spec:
backend:
serviceName: nginx
servicePort: 80
$ kubectl describe svc nginx
Name: nginx
Namespace: default
Labels: app=nginx
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx"},"name":"nginx","namespace":"default"},"spec":{"p...
Selector: app=nginx
Type: ClusterIP
IP: 10.96.107.97
Port: http 80/TCP
TargetPort: 80/TCP
Endpoints: 10.44.0.4:80,10.44.0.5:80,10.44.0.6:80
Session Affinity: None
Events: <none>
$ kubectl describe ingress nginx
Name: test-ingress
Namespace: default
Address:
Default backend: nginx:80 (10.44.0.4:80,10.44.0.5:80,10.44.0.6:80)
Rules:
Host Path Backends
---- ---- --------
`*` `*` nginx:80 (10.44.0.4:80,10.44.0.5:80,10.44.0.6:80)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"networking.k8s.io/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"test-ingress","namespace":"default"},"spec":{"backend":{"serviceName":"nginx","servicePort":80}}}
Events: `<none>`
Do we need to mention any IP ranges(private or public) of VMs while initializing the kubeadm init? or
Do we need to change any network settings in Azure Ubuntu VMs?
As you created your own Kubernetes cluster rather than AWS, Azure or GCP provided one, there is no load balancer integrated. Due to this reason, you are getting IP status pending.
But with the use of Ingress Controller or directly through NodePort you can circumvent this problem.
However, I also observed in your nginx service you are using an annotation service.beta.kubernetes.io/aws-load-balancer-type: nlb and you said you are using Azure and those are platform specific annotations for the service and that annotation is AWS specific.
However, you can give something like this a try, if you would like to experiment directly with public IPs, you can define your service by providing externalIPs in your service if you have a public ip allocated to your node and allows ingress traffic from somewhere.
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app: MyApp
ports:
- name: http
protocol: TCP
port: 80
targetPort: 9376
externalIPs:
- 80.11.12.10
But, a good approach to get this done is using an ingress controller if you are planning to build your own Kubernetes cluster.
Hope this helps.
I am using AKS cluster on Azure. I am trying to discover service using DNS (http://my-api.default.svc.cluster.local:3000/) but, it's not working (This site can’t be reached). With service IP endpoint everything is working fine.
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-api
labels:
app: my-api
spec:
replicas: 1
selector:
matchLabels:
app: my-api
template:
metadata:
labels:
app: my-api
spec:
containers:
- name: my-api
image: test.azurecr.io/my-api:latest
ports:
- containerPort: 3000
imagePullSecrets:
- name: testsecret
---
apiVersion: v1
kind: Service
metadata:
name: my-api
spec:
selector:
app: my-api
ports:
- protocol: TCP
port: 3000
targetPort: 3000
kubectl describe services kube-dns --namespace kube-system
Name: kube-dns
Namespace: kube-system
Labels: addonmanager.kubernetes.io/mode=Reconcile
k8s-app=kube-dns
kubernetes.io/cluster-service=true
kubernetes.io/name=KubeDNS
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"addonmanager.kubernetes.io/mode":"Reconcile","k8s-app":"kube-dns","kubernet...
Selector: k8s-app=kube-dns
Type: ClusterIP
IP: 10.10.110.110
Port: dns 53/UDP
TargetPort: 53/UDP
Endpoints: 10.10.100.54:53,10.10.100.64:53
Port: dns-tcp 53/TCP
TargetPort: 53/TCP
Endpoints: 10.10.100.54:53,10.10.100.64:53
Session Affinity: None
Events: <none>
kubectl describe svc my-api
Name: my-api
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"my-api","namespace":"default"},"spec":{"ports":[{"port":3000,"protocol":...
Selector: app=my-api
Type: ClusterIP
IP: 10.10.110.104
Port: <unset> 3000/TCP
TargetPort: 3000/TCP
Endpoints: 10.10.100.42:3000
Session Affinity: None
Events: <none>
From Second POD
kubectl exec -it second-pod /bin/bash
curl my-api.default.svc.cluster.local:3000
Response: {"value":"Hello world2"}
From Second POD website is running which is using the same endpoint but it's not connecting to the service.
Fixing the indentation of your yaml file, I was able to launch the deployment and service successfully. Also the DNS resolution worked fine.
Differences:
Fixed indentation
Used test1 namespaces instead of default
Used containerPort 80 instead of 3000
Used my image
Deployment:
apiVersion: apps/v1beta2
kind: Deployment
metadata:
labels:
app: my-api
name: my-api
namespace: test1
spec:
replicas: 1
selector:
matchLabels:
app: my-api
template:
metadata:
labels:
app: my-api
spec:
containers:
- image: leodotcloud/swiss-army-knife
name: my-api
ports:
- containerPort: 80
protocol: TCP
Service:
apiVersion: v1
kind: Service
metadata:
name: my-api
namespace: test1
spec:
ports:
- port: 3000
protocol: TCP
targetPort: 80
selector:
app: my-api
type: ClusterIP
Debugging steps:
Install tcpdump inside both of the kube-dns containers and start capturing DNS traffic (with filters from the second pod IP)
From inside the second pod, run curl or dig command using the FQDN.
Check if the DNS query packets are reaching the kube-dns containers.
If not, check for networking issues.
If the DNS resolution is working, then start tcpdump inside your application container and check if the curl packet is reaching the container.
Check the source and destination IP address of the packets.
Check the iptables rules on the hosts.
Check sysctl settings.
If you use Deployment to deploy your application onto cluster where it will be consumed via a Service you should have no need at all to manually set Endpoints. Just rely on kubernetes and define normal selector in your Service object.
Other then that, when it makes sense (external service consumed from within cluster), you need to make sure your Endpoints ports definition fully matches the one on service (incl. protocol and potentially name). This incomplete matching is a most common reason for endpoints to be not visible as a part of service.
From the above discussion, what I understood is, you want to expose a service but not using the IP address.
Service can be exposed in many ways. you should look for Service type LoadBalancer.
Try modifying your service is follow :
apiVersion: v1
kind: Service
metadata:
name: my-api
spec:
type: LoadBalancer
selector:
app: my-api
ports:
- protocol: TCP
port: 3000
targetPort: 3000
This will create a loadbalancer and map your service to the same.
Later you can add this loadbalancer to your DNS mapping service provided by Azure to give the domain name you like. ex: http:\\my-api.example.com:3000
Also I would like to add, if you define your ports as follow :
ports:
- name: http
port: 80
targetPort: 3000
This will redirect traffic coming to port 80 to 3000 and your service call would look much cleaner for ex. http:\\my-api.example.com