Packets not getting forwarded on Centos7 between GRE tunnels [closed] - linux

Question

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 years ago.
Improve this question
I'm have configured GRE tunnels between centos machines and corresponding routing tables on individual centos machines as shown in the image:
Im able to
Ping from Router-1 to gre1 tunnels other end:
worker]# ping 10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=1.43 ms
64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.472 ms
64 bytes from 10.0.0.2: icmp_seq=3 ttl=64 time=0.291 ms
64 bytes from 10.0.0.2: icmp_seq=4 ttl=64 time=0.319 ms
The traffic reaches Transit Router over the GRE tunnel(this is verified by tcpdump proto gre)
Ping from Router-2 to gre2 tunnels other end:
worker]# ping 11.0.0.2
PING 11.0.0.2 (11.0.0.2) 56(84) bytes of data.
64 bytes from 11.0.0.2: icmp_seq=1 ttl=64 time=1.10 ms
64 bytes from 11.0.0.2: icmp_seq=2 ttl=64 time=0.392 ms
64 bytes from 11.0.0.2: icmp_seq=3 ttl=64 time=0.369 ms
64 bytes from 11.0.0.2: icmp_seq=4 ttl=64 time=0.258 ms
This traffic too flows on tunnel
and on the transit router I'm able to ping the private address of both Router-1 and Router-2 after adding the routing entry:
Transit Router:
[root#vmc-centos conf]# ping 10.2.32.1
PING 10.2.32.1 (10.2.32.1) 56(84) bytes of data.
64 bytes from 10.2.32.1: icmp_seq=1 ttl=64 time=0.589 ms
64 bytes from 10.2.32.1: icmp_seq=2 ttl=64 time=0.380 ms
64 bytes from 10.2.32.1: icmp_seq=3 ttl=64 time=0.383 ms
Router-1:
worker]# tcpdump -i any proto gre -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
04:54:36.684864 IP 10.206.83.3 > 10.206.90.103: GREv0, length 88: IP 10.0.0.2 > 10.2.32.1: ICMP echo request, id 20445, seq 34, length 64
04:54:36.684951 IP 10.206.90.103 > 10.206.83.3: GREv0, length 88: IP 10.2.32.1 > 10.0.0.2: ICMP echo reply, id 20445, seq 34, length 64
04:54:37.684776 IP 10.206.83.3 > 10.206.90.103: GREv0, length 88: IP 10.0.0.2 > 10.2.32.1: ICMP echo request, id 20445, seq 35, length 64
Transit Router:
[root#vmc-centos conf]# ping 10.4.32.1
PING 10.4.32.1 (10.4.32.1) 56(84) bytes of data.
64 bytes from 10.4.32.1: icmp_seq=1 ttl=64 time=0.553 ms
64 bytes from 10.4.32.1: icmp_seq=2 ttl=64 time=0.325 ms
64 bytes from 10.4.32.1: icmp_seq=3 ttl=64 time=0.354 ms
Router-2:
worker]# sudo tcpdump -i any proto gre -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
04:56:57.549823 IP 10.206.83.3 > 10.206.86.199: GREv0, length 88: IP 11.0.0.2 > 10.4.32.1: ICMP echo request, id 20690, seq 24, length 64
04:56:57.549896 IP 10.206.86.199 > 10.206.83.3: GREv0, length 88: IP 10.4.32.1 > 11.0.0.2: ICMP echo reply, id 20690, seq 24, length 64
But now when I try to reach the private network of Router-2(10.4.32.1) from Router-1, the packets reach till Transit Router but are not being forwarded from there to Router-2:
Router-1:
worker]# ping 10.4.32.1
PING 10.4.32.1 (10.4.32.1) 56(84) bytes of data.
Transit Router:
[root#vmc-centos conf]# tcpdump -i any proto gre -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
04:59:06.382024 IP 10.206.90.103 > 10.206.83.3: GREv0, length 88: IP 10.0.0.1 > 10.4.32.1: ICMP echo request, id 36131, seq 40, length 64
04:59:07.382007 IP 10.206.90.103 > 10.206.83.3: GREv0, length 88: IP 10.0.0.1 > 10.4.32.1: ICMP echo request, id 36131, seq 41, length 64
Router-2:
[root#wdc-10-206-86-199 worker]# sudo tcpdump -i any proto gre -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
Route forwarding is enabled on all the machines:
[root#vmc-centos conf]# sudo sysctl -p
net.ipv4.ip_forward = 1
iptables on transit router:
[root#vmc-centos ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT gre -- anywhere anywhere
ACCEPT gre -- anywhere anywhere
Chain FORWARD (policy DROP)
target prot opt source destination
DOCKER-USER all -- anywhere anywhere
DOCKER-ISOLATION-STAGE-1 all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT gre -- anywhere anywhere
Chain DOCKER (1 references)
target prot opt source destination
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
target prot opt source destination
DOCKER-ISOLATION-STAGE-2 all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
target prot opt source destination
DROP all -- anywhere anywhere
RETURN all -- anywhere anywhere
Chain DOCKER-USER (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere
Note: I have tried this before and the packets were reaching the other private network. Now Im trying on another setup, theres some config I'm missing.

Answer 1

Got the answer here:
https://serverfault.com/questions/1010565/packets-not-getting-forwarded-on-centos7-between-gre-tunnels
The Docker daemon seems to be running on the forwarding machine. By default to isolate containers on different bridges and the host machine, Docker will install a default DROP policy on the forwarding chain in iptables. There is a setting in Docker daemon to not do this. Set iptables to false in /etc/docker/daemon.json. See Docker and iptables.
If you change default policy to ACCEPT, that will work.
iptables --policy FORWARD ACCEPT
BUT, when you (or a package upgrade of docker, or a reboot) restarts the Docker daemon the default policy will again change to DROP, if you didn't change the setting of the docker daemon.