Nginx-Ingress for TCP entrypoint not working - logstash

Question

I am using Nginx as Kubernetes Ingress controller. After following this simple example I was able to setup this example
Now I am trying to setup TCP entrypoint for logstash with following config
Logstash :
apiVersion: v1
kind: Secret
metadata:
name: logstash-secret
namespace: kube-logging
type: Opaque
data:
tls.crt: "<base64 encoded>" #For logstash.test.domain.com
tls.key: "<base64 encoded>" #For logstash.test.domain.com
---
apiVersion: v1
kind: ConfigMap
metadata:
name: logstash-config
namespace: kube-logging
labels:
app: logstash
data:
syslog.conf: |-
input {
tcp {
port => 5050
type => syslog
}
}
filter {
grok {
match => {"message" => "%{SYSLOGLINE}"}
}
}
output {
elasticsearch {
hosts => ["http://elasticsearch:9200"] #elasticsearch running in same namespace (kube-logging)
index => "syslog-%{+YYYY.MM.dd}"
}
stdout { codec => rubydebug }
}
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: logstash
namespace: kube-logging
labels:
app: logstash
spec:
replicas: 1
selector:
matchLabels:
app: logstash
strategy:
type: RollingUpdate
template:
metadata:
labels:
app: logstash
spec:
#serviceAccountName: logstash
containers:
- name: logstash
image: docker.elastic.co/logstash/logstash:7.2.1
imagePullPolicy: Always
env:
- name: ELASTICSEARCH_HOST
value: elasticsearch
- name: ELASTICSEARCH_PORT
value: "9200"
- name: ELASTICSEARCH_USERNAME
value: elastic
- name: ELASTICSEARCH_PASSWORD
value: changeme
- name: ELASTIC_CLOUD_ID
value:
- name: ELASTIC_CLOUD_AUTH
value:
ports:
- name: logstash
containerPort: 5050
protocol: TCP
securityContext:
runAsUser: 0
volumeMounts:
- name: config
mountPath: /usr/share/logstash/pipeline/syslog.conf
readOnly: true
subPath: syslog.conf
volumes:
- name: config
configMap:
defaultMode: 0600
name: logstash-config
---
kind: Service
apiVersion: v1
metadata:
name: logstash
namespace: kube-logging
labels:
app: logstash
spec:
ports:
- name: tcp-port
protocol: TCP
port: 5050
targetPort: 5050
selector:
app: logstash
Nginx-Ingress:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-ingress
namespace: kube-ingress
spec:
replicas: 1
selector:
matchLabels:
app: nginx-ingress
template:
metadata:
labels:
app: nginx-ingress
spec:
serviceAccountName: nginx-ingress
containers:
- image: nginx/nginx-ingress:1.5.7
imagePullPolicy: Always
name: nginx-ingress
ports:
- name: http
containerPort: 80
- name: https
containerPort: 443
- name: tcp5050
containerPort: 5050
securityContext:
allowPrivilegeEscalation: true
runAsUser: 101 #nginx
capabilities:
drop:
- ALL
add:
- NET_BIND_SERVICE
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
args:
- -nginx-configmaps=$(POD_NAMESPACE)/nginx-config
- -default-server-tls-secret=$(POD_NAMESPACE)/default-server-secret
- -v=3 # Enables extensive logging. Useful for troubleshooting.
#- -report-ingress-status
#- -external-service=nginx-ingress
#- -enable-leader-election
#- -enable-prometheus-metrics
#- -enable-custom-resources
LoadBalancer :
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress-external
namespace: kube-ingress
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
spec:
type: LoadBalancer
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
- name: https
protocol: TCP
port: 443
targetPort: 443
- name: tcp5050
protocol: TCP
port: 5050
targetPort: 5050
selector:
app: nginx-ingress
Ingress :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: logstash-ingress
namespace: kube-logging
spec:
tls:
- hosts:
- logstash.test.domain.com
secretName: logstash-secret #This has self-signed cert for logstash.test.domain.com
rules:
- host: logstash.test.domain.com
http:
paths:
- path: /
backend:
serviceName: logstash
servicePort: 5050
With this config it shows following ,
NAME HOSTS ADDRESS PORTS AGE
logstash-ingress logstash.test.domain.com 80, 443 79m
Why port 5050 not listed here ?
Just want to expose logstash service through public endpoint. When I use
openssl s_client -connect logstash.kube-logging.svc.cluster.local:5050 within the cluster I get
$ openssl s_client -connect logstash.kube-logging.svc.cluster.local:5050
CONNECTED(00000005)
But from outside of the cluster openssl s_client -connect logstash.test.domain.com:5050 I get
$ openssl s_client -connect logstash.test.domain.com:5050
connect: Connection refused
connect:errno=61
and
$ openssl s_client -cert logstash_test_domain_com.crt -key logstash_test_domain_com.key -servername logstash.test.domain.com:5050
connect: Connection refused
connect:errno=61
What I need to do to get this working ?

Answer 1

It seems like you are kind of confused. So let's start by ordering your services and ingress.
First, there are 3 types of services in kubernetes. ClusterIP which allows you to expose your deployments internally in k8s. Nodeport which is the same as ClusterIP but also exposes your deployment through every node external IP and a PORT which is in the range ~30K-32K. Finally there is the LoadBalancer service which is the same as ClusterIP but also exposes your app in a specific external IP address assigned by the cloud provider LoadBalancer.
The NodePort service you created will make logstash accessible through every node external IP in a random port in the range 30K to 32K; find the port running kubectl get services | grep nginx-ingress and check the last column. To get your node's external ip addresses run kubectl get node -o wide. The LoadBalancer service you created will make logstash accessible through an external IP address in the port 5050. To find out the IP run kubectl get services | grep nginx-ingress-external. Finally, you have also created an ingress resource to reach logstash. For this you have defined a host which given TLS will be accessible in port 443 and inbound traffic there will be redirected to logstash's service type ClusterIP in port 5050. So there you go you have 3 ways to reach logstash. I would go for the LoadBalancer given that it is a specific port.