Python/Pod cannot reach the internet - python-3.x

Question

I'm using python3 with microk8s to develop a simple web service.
The service is working properly (with docker in my local development machine), but the production machine (Ubuntu18.04 LTS with microk8s in Azure) cannot reach the internet (SMTP/Web REST API) once the pod was started (all internal service is working).
Problem
The pod cannot ping the hostname but the IP address. After investigation, the pod is working as expected except for the external resource. When executing the nslookup, it seems to be ok. But the ping is not working.
bash-5.1# ping www.google.com
ping: bad address 'www.google.com'
bash-5.1# nslookup www.google.com
Server: 10.152.183.10
Address: 10.152.183.10:53
Non-authoritative answer:
Name: www.google.com
Address: 74.125.68.103
Name: www.google.com
Address: 74.125.68.106
Name: www.google.com
Address: 74.125.68.99
Name: www.google.com
Address: 74.125.68.104
Name: www.google.com
Address: 74.125.68.105
Name: www.google.com
Address: 74.125.68.147
Non-authoritative answer:
Name: www.google.com
Address: 2404:6800:4003:c02::93
Name: www.google.com
Address: 2404:6800:4003:c02::63
Name: www.google.com
Address: 2404:6800:4003:c02::67
Name: www.google.com
Address: 2404:6800:4003:c02::69
bash-5.1# ping 74.125.68.103
PING 74.125.68.103 (74.125.68.103): 56 data bytes
64 bytes from 74.125.68.103: seq=0 ttl=55 time=1.448 ms
64 bytes from 74.125.68.103: seq=1 ttl=55 time=1.482 ms
^C
--- 74.125.68.103 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1.448/1.465/1.482 ms
bash-5.1# python3
>>> import socket
>>> socket.gethostname()
'projects-dep-65d7b8685f-jzmxx'
>>> socket.gethostbyname('www.google.com')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
socket.gaierror: [Errno -3] Try again
Environments/Settings
host $ #In Host
host $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
host $ microk8s is running
high-availability: no
datastore master nodes: 127.0.0.1:19001
datastore standby nodes: none
addons:
enabled:
dashboard
dns
ha-cluster
ingress
metrics-server
registry
storage
disabled:
ambassador
cilium
fluentd
gpu
helm
helm3
host-access
istio
jaeger
keda
knative
kubeflow
linkerd
metallb
multus
portainer
prometheus
rbac
traefik
# In Pod
bash-5.1 # python3
>>> import sys
>>> print({'version':sys.version, 'version-info': sys.version_info})
{'version': '3.9.3 (default, Apr 2 2021, 21:20:32) \n[GCC 10.2.1 20201203]', 'version-info': sys.version_info(major=3, minor=9, micro=3, releaselevel='final', serial=0)}
bash-5.1 #
bash-5.1 # cat /etc/resolv.conf
search default.svc.cluster.local svc.cluster.local cluster.local ngqy0alqbw2elndk2awonodqmd.ix.internal.cloudapp.net
nameserver 10.152.183.10
options ndots:5

Answer 1

You can confirm your pod network namespace can connect to external and internal vnet ips or not through the following commands: -
kubectl --namespace=kube-system exec -it ${KUBE-DNS-POD-NAME} -c kubedns -- sh
#run ping/or nslookup using metadata endpoint
If you restart the pod or container, it can fix the issue of hostname not resolving for external IP addresses or else, you can move the pod to a different node. Also, edit the Kubernetes dns add on master (repeat for every master) as below : -
vi /etc/kubernetes/addons/kube-dns-deployment.yaml
And change the arguments for the health container as below: -
"--cmd=nslookup bing.com 127.0.0.1 >/dev/null"
"--url=/healthz-dnsmasq"
"--cmd=nslookup bing.com 127.0.0.1:10053 >/dev/null"
"--url=/healthz-kubedns"
"--port=8080"
"--quiet"
You can also try restarting the kube coredns through the following command: -
kubectl -n kube-system rollout
This will force the kubedns container to restart if the above condition occurs.
Thanking you,