I run JHipster app on Kubernetes [microk8s] with ingress on remote host.
Ingress sets by default https.
In browser console I get:
WebSocket connection to 'wss://XXXXXX/websocket/tracker/041/tjk33h4t/websocket?access_token=XXXXXXXXXXXXXXXX' failed: Error during WebSocket handshake: Unexpected response code: 403
followed by
Refused to display 'https://XXXXX/websocket/tracker/iframe.html?access_token=XXXXXXXXXX' in a frame because it set 'X-Frame-Options' to 'deny'
In application-prod.yml I have:
port: 8080
compression:
enabled: true
mime-types: text/html,text/xml,text/plain,text/css, application/javascript, application/json
min-response-size: 1024
When I type http [not https] url followed by port set by kubernetes service - works fine.
In microk8s ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
doesn't seem to work anyway.
How can I get to work websocket from https front-end?
you can use try adding annotation for websocket service
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
certmanager.k8s.io/cluster-issuer: tls-issuer
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/proxy-body-size: 50m
nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/secure-backends: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/websocket-services: SERVICE_NAME
nginx.org/websocket-services: SERVICE_NAME
Related
I've just finished setting up AKS with AGIC and using Azure CNI. I'm trying to deploy NGINX to test if I set the AKS up correctly with the following configuration:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
kubernetes.io/ingress.class: azure/application-gateway
kubernetes.io/ingress.allow-http: "false"
appgw.ingress.kubernetes.io/use-private-ip: "false"
appgw.ingress.kubernetes.io/override-frontend-port: "443"
spec:
tls:
- hosts:
- my.domain.com
secretName: aks-ingress-tls
rules:
- host: my.domain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-service
port:
number: 80
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
component: nginx
template:
metadata:
labels:
component: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: ClusterIP
selector:
component: nginx
ports:
- port: 80
protocol: TCP
There's no error or any other log message on apply the above configurations.
> k apply -f nginx-test.yml
deployment.apps/nginx-deployment created
service/nginx-service created
ingress.networking.k8s.io/nginx-ingress created
But after a further investigation in the Application Gateway I found these entries in the Activity log popped up at the same time I applied the said configuration.
Further details in one of the entries is as follows:
Operation name: Create or Update Application Gateway
Error code: RequestDisallowedByPolicy
Message: Resource 'my-application-gateway' was disallowed by policy.
[
{
"policyAssignment": {
"name": "Encryption In Transit",
"id": "/providers/Microsoft.Management/managementGroups/***/providers/Microsoft.Authorization/policyAssignments/EncryptionInTransit"
},
"policyDefinition": {
"name": "HTTPS protocol only on Application Gateway listeners",
"id": "/providers/microsoft.management/managementgroups/***/providers/Microsoft.Authorization/policyDefinitions/HttpsOnly_App_Gateways"
},
"policySetDefinition": {
"name": "Encryption In Transit",
"id": "/providers/Microsoft.Management/managementgroups/***/providers/Microsoft.Authorization/policySetDefinitions/EncryptionInTransit"
}
}
]
My organization have a policy to enforce TLS but from my configuration I'm not sure what I did wrong as I have already configured the ingress to only use HTTPS and also have certificate (from the secret) installed.
I'm not sure where to look and wish someone could guide me in the correct direction. Thanks!
• As you said, your organization has a policy for enforcing TLS for securing encrypted communication over HTTPS. Therefore, when you create an ‘NGINX’ deployment through the ‘yaml’ file posted, you can see that the nginx application is trying to connect to the application gateway ingress controller over Port 80 which is reserved for HTTP communications. Thus, your nginx application has also disallowed the usage of private IPs with the AGIC due to which the nginx application is directly overriding the HTTPS 443 port for reaching out to the domain ‘my.domain.com’ over port 80 without using the SSL/TLS certificate-based port for communication.
Thus, would suggest you to please configure NGINX application for port 443 as the frontend port for the cluster IP and ensure ‘SSL redirection’ is set to enabled due to which when the NGINX application is deployed, it will be not face the policy restrictions and get failed. Also, refer to the below snapshot of the listeners in application gateway and load balancer when provisioning an AGIC for an AKS cluster.
Also, for more detailed information on deploying the NGINX application in AKS cluster on ports, kindly refer to the below documentation link: -
https://learn.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli
I have a Service configured to be accessible via HTTP.
kind: Service
apiVersion: v1
metadata:
name: myservice
spec:
ports:
- name: http
protocol: TCP
port: 80
targetPort: 8080
type: ClusterIP
And an Ngynx Ingress configured to make that internal service accessible from a specific secure subdomain.domain
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: myservice-ingress
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/myservice-ingress
annotations:
certmanager.k8s.io/issuer: letsencrypt-prod
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: HTTP
spec:
tls:
- hosts:
- myservice.mydomain.com
secretName: myservice-ingress-secret-tls
rules:
- host: myservice.mydomain.com
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 80
status:
loadBalancer:
ingress:
- {}
So when I reach https://myservice.mydomain.com I can access to my service through HTTPS.
Is it safe enough or should I configure my service and pods to communicate only through HTTPS?
It's expected behaviour since you've set TLS in your Ingress.
Note that by default the controller redirects (308) to HTTPS if TLS is enabled for that ingress. If you want to disable this behavior globally, you can use ssl-redirect: "false" in the NGINX ConfigMap.
To configure this feature for specific ingress resources, you can use the nginx.ingress.kubernetes.io/ssl-redirect: "false" annotation in the particular resource.
About your question: "Is it safe enough.." - it's opinion based question, so I can answer to use better HTTPS, rather than HTTP, but it's just my opinion. You can always find the difference between HTTP and HTTPS
I have deployed Kubernetes Dashboard with a command:
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml
and I've edited the Service as a Nodeport and configured Ingress object accordingly. I could able to login to dashboard with http but getting issue while login the same URL with https:
"TLS handshake error from 10.244.0.0:44950: remote error: tls: unknown certificate" .
When i configured ingress rule with ssl it is giving error:
"Client sent an HTTP request to an HTTPS server."
I have jenkins application running on same cluster with real certificate and i could able to login the jenkins url with https .
Cluster Information:
k8s cluster running on (Linux Server release 7.9)
kubernetes version (v1.19.6)
Request you to confirm if any suggestion to fix this issue
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: kube-system-ingress
namespace: kubernetes-dashboard
annotations:
kubernetes.io/ingress.class: "haproxy"
ingress.kubernetes.io/ssl-passthrough: "false"
spec:
tls:
- hosts:
- console.qa.test.com
secretName: qa-pss-dashboard
rules:
- host: console.qa.test.com
http:
paths:
- path: /
backend:
serviceName: kubernetes-dashboard
servicePort: 8443
I think you have to add the annotation
ingress.kubernetes.io/backend-protocol: "HTTPS"
Please not that the kubernetes dashboard service is exposed in 443 port, not 8443 that is related to the deployment (pod port).
so:
backend:
service:
name: kubernetes-dashboard
port:
number: 443
I am having issues with request to my NodeJS app running in my kubernetes cluster in digital ocean. Every request returns a 502 Bad Gateway Error. I am not sure what I am missing.
This is what the service config looks like
apiVersion: v1
kind: Service
metadata:
name: service-api
namespace: default
labels:
app: service-api
spec:
type: ClusterIP
ports:
- name: http
protocol: TCP
port: 80
targetPort: 3000
selector:
app: service-api
The Ingress.yml looks like this
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: service-api-ingress
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/limit-connections: '2'
nginx.ingress.kubernetes.io/limit-rpm: '60'
service.beta.kubernetes.io/do-loadbalancer-protocol: "http"
service.beta.kubernetes.io/do-loadbalancer-algorithm: "round_robin"
service.beta.kubernetes.io/do-loadbalancer-http2-ports: "443,80"
service.beta.kubernetes.io/do-loadbalancer-tls-ports: "443"
service.beta.kubernetes.io/do-loadbalancer-tls-passthrough: "true"
spec:
tls:
- hosts:
- dev-api.service.com
secretName: service-api-tls
rules:
- host: "dev-api.service.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: service-api
port:
number: 80
Whenever I visit the host url I get a 502 error.
This is what appears in the nginx ingress log
2021/01/13 08:41:34 [error] 319#319: *31338 connect() failed (111: Connection refused) while connecting to upstream, client: IP, server: dev-api.service.com, request: "GET /favicon.ico HTTP/2.0", upstream: "http://10.244.0.112:3000/favicon.ico", host: "dev-api.service.com", referrer: "https://dev-api.service.com/status"
As we ( with #Emmanuel Amodu ) have discussed in comment:
mistake was to connect to app using wrong port, port 4000 instead of 3000 as defined in service-api.
For community which will have similar problem please - most important steps for debugging:
Checking netstat -plant output table
Checking your Nginx Configuration: $ kubectl exec -it -n <namespace-of-ingress-controller> <nginx-ingress-controller-pod> -- cat /etc/nginx/nginx.conf
Checking service: $ kubectl describe svc service-api
Could it be the annotation that configures SSL passthru?
If SSL passthru has been configured on your ingress controller then your service needs to expose port 443 in addition to port 80. You're basically saying the pod is terminating the secure connection not nginx.
If this is the issue would explain 50X error which indicates a problem with the backend
No special setup is needed, that 404 is most likely coming from your actual backend.
I am getting 502 Bad Gateway nginx/1.13.9 for my angular app when accessing in the browser in k8's. My service and ingress is as below.
Angular app pod logs show all successful and in-fact port forwarding is working fine. This same image is also working fine in my local machine with docker.
From k8s logs, I could see this:
[error] 1534#1534: *32272457 SSL_do_handshake() failed (SSL:
error:1408F10B:SSL routines:ssl3_get_record:wrong version number)
while SSL handshaking to upstream
Service:
Name: test-portal
Namespace: testproject
Labels: app=test-portal
chart=test-portal-1.0.0
environment=dev
heritage=Tiller
release=test-portal
version=dev
Annotations: <none>
Selector: app=test-portal,release=test-portal
Type: ClusterIP
IP: x.x.x.x
Port: <unset> 80/TCP
TargetPort: 4200/TCP
Endpoints: x.x.x.x:4200
Session Affinity: None
Events: <none>
Ingress:
Name: test-portal
Namespace: testproject
Address:
Default backend: default-http-backend:80 (<none>)
TLS:
SNI routes test-portal.us-west-2.xxxxx.xxxxxx.delivery
Rules:
Host Path Backends
---- ---- --------
test-portal.us-west-2.xxxxx.xxxxxx.delivery
/ test-portal:80 (<none>)
Annotations:
secure-backends: true
ssl-redirect: true
Events: <none>
Ingress Config map
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
ingress.kubernetes.io/secure-backends: "false"
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"ingress.kubernetes.io/secure-backends":"false","kubernetes.io/ingress.class":"nginx"},"labels":{"app":"test-portal","chart":"test-portal-1.0.0","environment":"dev","heritage":"test-portal","release":"Helm","version":"1.0.0"},"name":"test-portal","namespace":"testproject"},"spec":{"rules":[{"host":"test-portal.us-west-2.xxxx.xxxxxxxxx.delivery","http":{"paths":[{"backend":{"serviceName":"test-portal","servicePort":80},"path":"/"}]}}],"tls":[{"hosts":["test-portal.us-west-2.xxxx.xxxxxxxxx.delivery"]}]}}
kubernetes.io/ingress.class: nginx
creationTimestamp: 2020-01-08T11:24:18Z
generation: 1
labels:
app: test-portal
chart: test-portal-1.0.0
environment: dev
heritage: test-portal
release: Helm
version: 1.0.0
name: test-portal
namespace: testproject
resourceVersion: "2379156945"
selfLink: /apis/extensions/v1beta1/namespaces/testproject/ingresses/test-portal
uid: 6925819b-3209-11ea-80fb-02fb0c9060d8
spec:
rules:
- host: test-portal.us-west-2.xxxx.xxxxxxxxx.delivery
http:
paths:
- backend:
serviceName: test-portal
servicePort: 80
path: /
tls:
- hosts:
- test-portal.us-west-2.xxxx.xxxxxxxxx.delivery
status:
loadBalancer:
ingress:
- {}
You seem to be running the angular dev server on port 4200 in your pod.
The angular app is served using http, not https, therefore you must configure the ingress to not use https (secure-backends: false) for backend communication.
Besides, the angular dev server should not be used for prod serving. Build a container image with the angular prod build to benefit from much increased performance.