I am attempting to create a service for creating training datasets using the Prodigy UI tool. I would like to do this using a Kubernetes cluster which is running in Azure cloud. My Prodigy UI should be reachable on 0.0.0.0:8880 (on the container).
As such, I created a deployment as follows:
kind: Deployment
apiVersion: apps/v1beta2
metadata:
name: prodigy-dply
spec:
replicas: 1
selector:
matchLabels:
app: prodigy_pod
template:
metadata:
labels:
app: prodigy_pod
spec:
containers:
- name: prodigy-sentiment
image: bdsdev.azurecr.io/prodigy
imagePullPolicy: IfNotPresent
command: ["/bin/bash"]
args: ["-c", "prodigy spacy textapi -F training_recipe.py"]
ports:
- name: prodigyport
containerPort: 8880
This should (should being the operative word here) expose that 8880 port at the pod level aliased as prodigyport
Following that, I have created a Service as below:
kind: Service
apiVersion: v1
metadata:
name: prodigy-service
spec:
type: LoadBalancer
selector:
app: prodigy_pod
ports:
- protocol: TCP
port: 8000
targetPort: prodigyport
At this point, when I run the associated kubectl create -f <deployment>.yaml and kubectl create -f <service>.yaml, I get an ExternalIP and associated Port: 10.*.*.*:34672.
This is not reachable by browser, and I'm assuming I have a misunderstanding of how my browser would interact with this Service, Pod, and the underlying Container. What am I missing here?
Note: I am willing to accept that kubernetes may not be the tool for the job here, it seems enticing because of the ease of scalability and updating images to reflect more recent configurations
You can find public IP address(LoadBalancer Ingress) with this command:
kubectl get service azure-vote-front
Result like this:
root#k8s-master-79E9CFFD-0:~# kubectl get service azure
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
azure 10.0.136.182 52.224.219.190 8080:31419/TCP 10m
Then you can browse it with external IP and port, like this:
curl 52.224.219.190:8080
Also you can find the Load Balaner rules via Azure portal:
Hope this helps.
You can find the IP address created for your service by getting the service information through kubectl:
kubectl describe services prodigy-service
The IP address is listed next to LoadBalancer Ingress.
Also, you can use port forwarding to access your pod:
kubectl port-forward <pod_name> 8880:8880
After that you can access Prodigy UI by localhost:8880 in your browser.
Related
I have install kubeadm in master and connected 2 worker node to it, after that i deploy nginx pod and ssh into that nginx pod after that i type nslookup google.com or apt update its not working got connection timeout it think due its not connecting to internet. How to solve it, The 3 VM is running in azure portal and 3 VM are connected to together. kubectl v1.24.2 im using it
The 3 VM is running in azure portal and 3 VM are connected to together. kubectl v1.24.2 im using it and calico network also im using
nginx pod is running in worker2 and services of containerd container runtime/Docker Application Container Engine both are in running state. if i type lsmod | grep br_netfilter i got
br_netfilter 28672 0
bridge 266240 1 br_netfilter
here is my nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
# command: ["/bin/sh","-c"]
# args: ["apt update"]
# securityContext:
# privileged: true
---
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
type: ClusterIP
selector:
app: nginx
ports:
- port: 8080
targetPort: 80
name: nginx-http
And Here is the screenshot of coredns install in kube-system naming space
Seems like name is not resolving, please check if coredns is working properly. You can break down the issue like this:
from inside the pod first check you have network reachability to the internet: curl -v telnet://8.8.8.8:53
check you have connectivity to core DNS: curl -v telnet://coredns_service_name:53 or curl -v telnet://coredns_cluster_ip:53
Afaik, the K8s NetworkPolicy can only allow pods matching a label to do something. I do not want to:
Deny all traffic
Allow traffic for all pods except the ones matching my label
but instead:
Allow all traffic
Deny traffic for pods matching my label
How do I do that?
From kubectl explain NetworkPolicy.spec.ingress.from:
DESCRIPTION:
List of sources which should be able to access the pods selected for this
rule. Items in this list are combined using a logical OR operation. If this
field is empty or missing, this rule matches all sources (traffic not
restricted by source). If this field is present and contains at least one
item, this rule allows traffic only if the traffic matches at least one
item in the from list.
As far as I understand this, we can only allow, not deny.
As you mentioned in the comments, you are using the Kind tool for running Kubernetes. Instead of kindnet CNI plugin (default CNI plugin for Kind) which does not support Kubernetes network policies, you can use Calico CNI plugin which support Kubernetes network policies + it has its own, similar solution called Calico network policies.
Example - I will create cluster with disabled default kind CNI plugin + enabled NodePort for testing (assuming that you have kind + kubectl tools already installed):
kind-cluster-config.yaml file:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
disableDefaultCNI: true # disable kindnet
podSubnet: 192.168.0.0/16 # set to Calico's default subnet
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30000
hostPort: 30000
listenAddress: "0.0.0.0" # Optional, defaults to "0.0.0.0"
protocol: tcp # Optional, defaults to tcp
Time for create a cluster using above config:
kind create cluster --config kind-cluster-config.yaml
When cluster is ready, I will install Calico CNI plugin:
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
I will wait until all calico pods are ready (kubectl get pods -n kube-system command to check). Then, I will create sample nginx deployment + service type NodePort for accessing:
nginx-deploy-service.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 2 # tells deployment to run 2 pods matching the template
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.14.2
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: NodePort
selector:
app: nginx
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30000
Let's apply it: kubectl apply -f nginx-deploy-service.yaml
So far so good. Now I will try to access nginx-service using node IP (kubectl get nodes -o wide command to check node IP address):
curl 172.18.0.2:30000
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
Okay, it's working.
Now time to install calicoctl and apply some example policy - based on this tutorial - to block ingress traffic only for pods with label app with value nginx:
calico-rule.yaml:
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
name: default-deny
spec:
selector: app == "nginx"
types:
- Ingress
Apply it:
calicoctl apply -f calico-rule.yaml
Successfully applied 1 'GlobalNetworkPolicy' resource(s)
Now I can't reach the address 172.18.0.2:30000 which was working previously. The policy is working fine!
Read more about calico policies:
Get started with Calico network policy
Calico policy tutorial
Also check this GitHub topic for more information about NetworkPolicy support in Kind.
EDIT:
Seems like Calico plugin supports as well Kubernetes NetworkPolicy, so you can just install Calico CNI plugin and apply the following policy:
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: default-deny
spec:
podSelector:
matchLabels:
app: nginx
policyTypes:
- Ingress
I tested it and seems it's working fine as well.
I have created a sample spring boot app and did the following:-
1.created a docker image
2.created an Azure container registry and did a docker push to this
3.Created a cluster in Azure Kubernetes service and deployed it successfully.I have chosen external endpoint option for this.
Kubernetes external end point
say for service to service call i dont want to use IP like http://20.37.134.68:80 but another custom name how can i do it?
Also if i chose internal then is there any way to replace the name.
Tried editing YAML with endpoint name property but failed.Any ideas?
I think you mixing some concept, so I'll try to explain and help you to reach what you want.
When you deploy a container image in a Kubernetes cluster, in the most cases you will use a pod or deployment spec, that basically is a yaml file with all your deployment/pod configuration, name, image name etc. Here is an example of a simple echo-server app:
apiVersion: apps/v1
kind: Deployment
metadata:
name: echo
spec:
selector:
matchLabels:
app: echo
template:
metadata:
labels:
app: echo
spec:
containers:
- name: echo
image: mendhak/http-https-echo
ports:
- name: http
containerPort: 80
Observe the fields name in the file. Here you can configure the name for your deployment and for your containers.
In order to expose your application, you will need to use a service. Services can be internal and external. Here you can find all service types.
For a internal service, you need to use the service type ClusterIP (default), it means only your cluster will reach the pods. To reach your service from other pods, you can use the service name composed by my-svc.my-namespace.svc.cluster-domain.example.
Here is an example of a service for the deployment above:
apiVersion: v1
kind: Service
metadata:
name: echo-svc
spec:
selector:
app: echo
ports:
- protocol: TCP
port: 80
targetPort: 80
To expose your service externally, you have the option to use a service type NodePort, LoadBalancer or use an ingress.
You can configure your DNS name in the ingress rules and make path rules if you want, or even configure a HTTPS for your application. There are few options to ingresses in kubernetes, and one of the most popular is nginx-ingress.
Here is an example of how to configure a simple ingress for our example service:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "false"
name: echo-ingress
spec:
rules:
- host: myapp.mydomain.com
http:
paths:
- path: "/"
backend:
serviceName: echo-svc
servicePort: 80
In the example, i'm using the dns name myapp.mydomain.com, so it means you can only will reach your application by this name.
After create the ingress, you can see the external ip with the command kubectl get ing, and you can create a A entry in your dns server.
I have a mysql pod in my cluster that I want to expose to a public IP. Therefor I changed it to be a loadbalancer by doing
kubectl edit svc mysql-mysql --namespace mysql
release: mysql
name: mysql-mysql
namespace: mysql
resourceVersion: "646616"
selfLink: /api/v1/namespaces/mysql/services/mysql-mysql
uid: cd1cce11-890c-11e8-90f5-869c0c4ba0b5
spec:
clusterIP: 10.0.117.54
externalTrafficPolicy: Cluster
ports:
- name: mysql
nodePort: 31479
port: 3306
protocol: TCP
targetPort: 3306
selector:
app: mysql-mysql
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 137.117.40.121
changing ClusterIP to LoadBalancer.
However I can't seem to reach it by going to mysql -h137.117.40.121 -uroot -p*****
Anyone have any idea? Is it because i'm trying to forward it over TCP?
For your issue, you want to expose your mysql pod to a public IP. So you need to take a look at Ingress in Kubernets. It's an API object that manages external access to the services in a cluster, typically HTTP. For the Ingress, you need both ingress controller and ingress rules. For more details, you can read the document I posted.
In Azure, you can get more details from HTTPS Ingress on Azure Kubernetes Service (AKS).
As pointed out by #aurelius, your config seems correct it's possible that the traffic is getting blocked by your firewall rules.
Also make sure, the cloud provider option is enabled for your cluster.
kubectl get svc -o wide would show the status of the LoadBalancer and the IP address allocated.
#charles-xu-msft, using Ingress is definitely an option but there is nothing wrong in using LoadBalancer kind of Service when the cloud provider is enabled for the kubernetes cluster.
Just for reference, here is test config:
apiVersion: v1
kind: Pod
metadata:
name: mysql-pod
labels:
name: mysql-pod
spec:
containers:
- name: mysql:5
image: mysql
ports:
- containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
value: mysqlpassword
---
apiVersion: v1
kind: Service
metadata:
name: test-mysql-lb
spec:
type: LoadBalancer
ports:
- port: 3306
targetPort: 3306
protocol: TCP
selector:
name: mysql-pod
As an experiment I'm trying to run a docker container on Azure using the Azure Container Service and Kubernetes as the orchestrator. I'm running the official nginx image. Here are the steps I am taking:
az group create --name test-group --location westus
az acs create --orchestrator-type=kubernetes --resource-group=test-group --name=k8s-cluster --generate-ssh-keys
I created Kubernetes deployment and service files from a docker compose file using Kompose.
deployment file
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
kompose.service.type: LoadBalancer
creationTimestamp: null
labels:
io.kompose.service: test
name: test
spec:
replicas: 1
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
io.kompose.service: test
spec:
containers:
- image: nginx:latest
name: test
ports:
- containerPort: 80
resources: {}
restartPolicy: Always
status: {}
service file
apiVersion: v1
kind: Service
metadata:
annotations:
kompose.service.type: LoadBalancer
creationTimestamp: null
labels:
io.kompose.service: test
name: test
spec:
ports:
- name: "80"
port: 80
targetPort: 80
selector:
io.kompose.service: test
type: LoadBalancer
status:
loadBalancer: {}
I can then start everything up:
kubectl create -f test-service.yaml,test-deployment.yaml
Once an IP has been exposed I assign a dns prefix to it so I can access my running container like so: http://nginx-test.westus.cloudapp.azure.com/.
My question is, how can I access the service using https? At https://nginx-test.westus.cloudapp.azure.com/
I don't think I'm supposed to configure nginx for https, since the certificate is not mine. I've tried changing the load balancer to send 443 traffic to port 80, but I receive a timeout error.
I tried mapping port 443 to port 80 in my Kubernetes service config.
ports:
- name: "443"
port: 443
targetPort: 80
But that results in:
SSL peer was not expecting a handshake message it received. Error code: SSL_ERROR_HANDSHAKE_UNEXPECTED_ALERT
How can I view my running container at https://nginx-test.westus.cloudapp.azure.com/?
If I understand it correctly, I think you are looking for Nginx Ingress controller.
If we need TLS termination on Kubernetes, we can use ingress controller, on Azure we can use Nginx Ingress controller.
To archive this, we can follow those steps:
1 Deploy the Nginx Ingress controller
2 Create TLS certificates
3 Deploy test http service
4 configure TLS termination
More information about configure Nginx Ingress Controller for TLS termination on Kubernetes on Azure, please refer to this blog.
root#k8s-master-6F403744-0:~/ingress/examples/deployment/nginx# kubectl get services --namespace kube-system -w
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default-http-backend 10.0.113.185 <none> 80/TCP 42m
heapster 10.0.4.232 <none> 80/TCP 1h
kube-dns 10.0.0.10 <none> 53/UDP,53/TCP 1h
kubernetes-dashboard 10.0.237.125 <nodes> 80:32229/TCP 1h
nginx-ingress-ssl 10.0.92.57 40.71.37.243 443:30215/TCP 13m