I have an azure Linux VM say VM1 having only one network interface with private IP 10.3.0.5, I have another azure Linux VM say VM2 with two network interfaces, the private IP on primary network interface is 10.3.5.4, the private IP on the secondary network interface is 10.3.4.4.
Now I am able to ping VM2 from VM1 on primary network interface of VM2 as
ping 10.3.5.4 but I am not able to ping it in on secondary network interface as ping 10.3.4.4.
After reading azure docs - https://learn.microsoft.com/en-gb/azure/virtual-machines/linux/multiple-nics#configure-guest-os-for-multiple-nics, they say that you would have to manually add required routes to achieve this.
Similar problem in windows VM - https://support.microsoft.com/en-in/help/4048050/troubleshooter-for-azure-vm-connectivity-problems
here is the result of command route -n on VM2
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.3.5.1 0.0.0.0 UG 0 0 0 eth0
10.3.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
10.3.5.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
168.63.129.16 10.3.5.1 255.255.255.255 UGH 0 0 0 eth0
169.254.169.254 10.3.5.1 255.255.255.255 UGH 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker_gwbridge
Now as per the above azure docs, it is missing an entry something like this
0.0.0.0 10.3.4.1 0.0.0.0 UG 0 0 0 eth1
I tried adding this route to my VM2 but after running the following command, my vm just stopped responding
route add -net 0.0.0.0 netmask 0.0.0.0 gw 10.3.4.1 dev eth1
How do I add the correct route for my problem ? Please help !
My suggestion is adding another network interface to your VM1. For example, I have VMA with two network interfaces. The private IP of the primary network interface is 10.0.3.4.The private IP on the secondary network interface is 10.0.2.5.
Result of command route -n on VMA
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Ifac
0.0.0.0 10.0.3.1 0.0.0.0 UG 100 0 0 eth0
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
10.0.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
168.63.129.16 10.0.3.1 255.255.255.255 UGH 100 0 0 eth0
169.254.169.254 10.0.3.1 255.255.255.255 UGH 100 0 0 eth0
The VMB also has two network interfaces.The private IP of the primary network interface is 10.0.1.4.The private IP on the secondary network interface is 10.0.2.4. The private IP 10.0.2.4 of VMB and private IP 10.0.2.5 are in the same subnet.
Result of command route -n on VMB
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Ifa
0.0.0.0 10.0.1.1 0.0.0.0 UG 100 0 0 eth
10.0.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth
168.63.129.16 10.0.1.1 255.255.255.255 UGH 100 0 0 eth
169.254.169.254 10.0.1.1 255.255.255.255 UGH 100 0 0 eth
So in the VMA, I can ping 10.0.1.4 through the primary network interface and ping 10.0.2.4 through the secondary network interface.
Related
Long question in short:
Ping over r1-r4-r2 path works using 10.0.1.* or 10.0.2.* IP addresses, but fails if we alter the path to r1-r3-r2 using 1.0.0.* or 1.0.1.* IP addresses for the exactly same packets (except for the fact that packets' src and dst IP fields are changed from 10.* to 1.* and vice-versa at s1 and s2 respectively). Why?
Question in detail:
I have a small topology as below
h1 -- s1 -- r1 -- r4 -- r2 -- s2 -- h2
\ /
\ /
\ /
r3
The s's are OpenvSwitch instances while r's are Ubuntu 16 Linux machines.
IP Addresses are:
h1-eth0 - 10.0.1.10/24
s1 - 10.0.1.50/24
h2-eth0 - 10.0.2.10/24
s2 - 10.0.2.50/24
r1-eth0 - 10.0.1.1/24
r1-eth1 - 10.0.11.2/24
r1-eth2 - 10.0.12.2/24
r2-eth0 - 10.0.2.1/24
r2-eth1 - 10.0.13.1/24
r2-eth2 - 10.0.5.1/24
r3-eth0 - 10.0.12.1/24
r3-eth1 - 10.0.5.2/24
r4-eth0 - 10.0.11.1/24
r4-eth1 - 10.0.13.2/24
As you can see, there are two similar paths between r1 and r2. I add the following static entries.
r1
sudo ip route add 10.0.2.0/24 via 10.0.11.1
r2
sudo ip route add 10.0.1.0/24 via 10.0.13.2
r4
sudo ip route add 10.0.1.0/24 via 10.0.11.2
sudo ip route add 10.0.2.0/24 via 10.0.13.1
The ping between h1 and h2 works as expected. Now, since the switches are OVS (and thus are OpenFlow-enabled) I install entries in s1 to map the destination IPs to a different subnet.
i.e. the IP 10.0.1.10 would be mapped to 1.0.0.10 while the IP 10.0.2.10 would be mapped to 1.0.1.10 when such a packet is received at s1, while the destination IPs would be mapped back to original at s2.
(I have checked that these entries are indeed correct and are working as expected. Also I have added this entry only to match ICMP packets). Similar procedure would be done when ping reply is sent by h1.
Along with these, I install static routes in the routers to route these IPs.
r1
sudo ip route add 1.0.0.0/24 via 10.0.1.50
sudo ip route add 1.0.1.0/24 via 10.0.12.1
r2
sudo ip route add 1.0.0.0/24 via 10.0.5.2
sudo ip route add 1.0.1.0/24 via 10.0.2.50
r3
sudo ip route add 1.0.0.0/24 via 10.0.12.2
sudo ip route add 1.0.1.0/24 via 10.0.5.1
Now if I ping h1 from h2, the packet starts with destination IP 10.0.1.10, which is mapped to 1.0.0.10 at s2, r2 routes this and sends it to r3, r3 routes it and sends to r1. But r1, even after receiving the packet at one interface and having the matching entry in the Linux routing table does not route and forward packet.
Even ip route get outputs the correct port to which the packet should be forwarded. There are no firewall entries in ip tables as well.
Some additional information:
If I change the newly added routing entries to use the original path
of r1-r4-r2 (i.e., we route on this path with mapped ip's) , it behaves as expected and the ping works as expected.
Alternatively, if I change the old routing entries for 10.0.2.0/24 in r1 and
10.0.1.0/24 in r2 (which now ideally don't even have to be matched by the new
packets as their Destination IPs are in 1.0.0.* range or 1.0.1.* only) to use
the new path r1-r3-r4 along with this mapped-IP packets, the ping between
r2 and r1 works as expected.
Details that may be required:
The final routing tables are as follows:
r1
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.11.1 0.0.0.0 UG 0 0 0 eth1
1.0.0.0 10.0.1.10 255.255.255.0 UG 0 0 0 eth0
1.0.1.0 10.0.12.1 255.255.255.0 UG 0 0 0 eth2
10.0.1.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
10.0.2.0 10.0.11.1 255.255.255.0 UG 0 0 0 eth1
10.0.11.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1
10.0.12.0 0.0.0.0 255.255.255.0 U 1 0 0 eth2
r2
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.13.2 0.0.0.0 UG 0 0 0 eth1
1.0.0.0 10.0.5.2 255.255.255.0 UG 0 0 0 eth1
1.0.1.0 10.0.2.50 255.255.255.0 UG 0 0 0 eth0
10.0.1.0 10.0.13.2 255.255.255.0 UG 0 0 0 eth1
10.0.2.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
10.0.5.0 0.0.0.0 255.255.255.0 U 1 0 0 eth2
10.0.13.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1
r3
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.5.1 0.0.0.0 UG 0 0 0 eth1
1.0.0.0 10.0.12.2 255.255.255.0 UG 0 0 0 eth0
1.0.1.0 10.0.5.1 255.255.255.0 UG 0 0 0 eth1
10.0.1.0 10.0.12.2 255.255.255.0 UG 0 0 0 eth0
10.0.2.0 10.0.5.1 255.255.255.0 U 1 0 0 eth1
10.0.5.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1
10.0.12.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
r4
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth4
1.0.0.0 10.0.11.2 255.255.255.0 UG 0 0 0 eth0
1.0.1.0 10.0.13.1 255.255.255.0 UG 0 0 0 eth1
10.0.1.0 10.0.11.2 255.255.255.0 UG 0 0 0 eth0
10.0.2.0 10.0.13.1 255.255.255.0 UG 0 0 0 eth1
10.0.11.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
10.0.13.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 1 0 0 eth4
Note: 192.168.0.* is a subnet connected to outside Internet.
What do you think is the problem ? I am completely baffled looking at this problem.
The behavior of Linux routing here was as expected.
The flag for reverse path filter
i.e. /proc/sys/net/ipv4/conf/<interfacename>/rp_filter was turned ON (by setting value to 1) by default.
Reverse path filter is provided as a security feature of Linux kernel.
A common example is private IP space escaping onto the Internet. If you have an interface with a route of 195.96.96.0/24 to it, you do not expect packets from 212.64.94.1 to arrive there. So kernel drops such a packet if the flag is set to 1.
More formally,
Reverse path filtering is a mechanism adopted by the Linux kernel to
check whether the source IP address of the packet that is been
received is routable.
So in other words, when a machine with reverse path filtering enabled
receives a packet, the machine will first check whether the source of
the received packet is reachable through the interface it came in.
If it is routable through the interface which it came, then the machine will accept the packet.
If it is not routable through the interface, which it came, then the machine will drop that packet.
Latest kernels provide one more option value of 2. This option is slightly
more liberal in terms of accepting traffic.
If the received packet's source address is routable through any of the
interfaces on the machine, the machine will accept the packet.
To make that work, use this on all machines:
# for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
> echo 0 > $i
> done
Or make the following entry in your /etc/sysctl.conf
net.ipv4.conf.all.rp_filter = 0
rp_filter will filter packets in the three modes: 0 disabled, 1 strict and 2 loose.
Example
Client A - 192.168.2.10 - connected to router via eth0
Router
eth0 - 192.168.2.150
routes - 192.168.2.0/24
eth1 - 10.42.43.1
routes - 10.42.43.0/24
Note: No default route
Client C - 10.42.43.50 - connected to router via eth1
With this setup and rp_filter on the router set to “loose mode” (2) a packet on eth0 from 1.2.3.4 to 10.42.43.50 will be blocked.
With rp_filter on the router set to “strict mode” (1) a packet on eth0 from source address 10.42.43.2 will be blocked.
When set to “disabled” (0) both packets would go through.
First off your topology details is incomplete you are missing r3 and r4 details but they can be inferred.
Instead of trying to troubleshoot your issue I'm just going to try to explain what needs to happen. However it would be much easier if you just used a routing protocol like OSPF which is designed to make this easy so you don't have to do it by hand.
Each device that is routing needs to know how to get to every other subnet if its to be accessible. So this means you can either add in default routes (ie routes that match 0.0.0.0/0) or you can enter in each subnet with corresponding next-ip into each router (see below). Usually you do not need to add routes for subnets that are connected (IE you have an ip on that router in that subnet)
R1 routes
10.0.13.0/24 -> 10.0.11.1
10.0.5.0/24 -> 10.0.11.1
10.0.2.0/24 -> 10.0.11.1
R2 routes
10.0.1.0/24 -> 10.0.13.2
10.0.12.0/24 -> 10.0.13.2
10.0.11.0/24 -> 10.0.13.2
R3 routes
10.0.1.0/24 -> 10.0.12.2
10.0.11.0/24 -> 10.0.12.2
10.0.13.0/24 -> 10.0.5.1
10.0.2.0/24 -> 10.0.5.1
R4 Routes
10.0.1.0/24 -> 10.0.11.2
10.0.12.0/24 -> 10.0.11.2
10.0.2.0/24 -> 10.0.13.1
10.0.5.0/24 -> 10.0.13.1
For devices H1, S1, H2, and S2 they should have a default route that points to the gateway 10.0.1.1 and 10.0.2.1.
Have the next problem in my LINUX ASTERISK ELASTIX SERVER
Have two interfacaces:
eth0:
inet addr:192.168.1.240 Bcast:192.168.1.255 Mask:255.255.255.0
eth1:
inet addr:10.7.227.110 Bcast:10.7.227.111 Mask:255.255.255.252
The eth0 is the Internet and SIP Phones Network
The eth1 is the TRUNK of the TELMEX PROVIDIER
If make route show:
[root#localhost ~]# /sbin/route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.7.227.108 * 255.255.255.252 U 0 0 0 eth1
192.168.1.0 192.168.1.1 255.255.255.0 UG 0 0 0 eth0
192.168.1.0 * 255.255.255.0 U 0 0 0 eth0
169.254.0.0 * 255.255.0.0 U 0 0 0 eth1
default 10.7.227.109 0.0.0.0 UG 0 0 0 eth1
The problem:
The server dont connect to internet, if traceroute all time out by the eth1
DONT WORK TO MY INTERNET GATEWAY OR ANY OTHER PUBLIC IP
[root#localhost ~]# traceroute 192.168.1.1
traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 40 byte packets
1 * * *
2 * * *
WORK TO INTERNAL TELMEX IP
[root#localhost ~]# traceroute 10.7.224.1
traceroute to 10.7.224.1 (10.7.224.1), 30 hops max, 40 byte packets
1 (10.7.227.109) 0.886 ms 1.119 ms 1.428 ms
2 (10.170.40.1) 0.611 ms 0.617 ms 0.850 ms
3 (10.7.224.12) 1.993 ms 1.994 ms 1.990 ms
If delete the Route eth1, the calls dont out, but have internet,
Can you help me please?
Any ideas?
Thanks for your help
If your internet interface is eth0, then you have to remove default 10.7.227.109 0.0.0.0 UG 0 0 0 eth1 from a route table.
And add eth0 with its gateway as default.
I created a custom Buildroot 2015.01 distribution w/ 3.10.0 Linux Kernel with enabled IPv6 support in the Toolchain and the uClibc library.
There is a strange behaviour that takes place when trying to contact any IPv6 routed address, for example:
# wget google.com
Connecting to google.com ([2a00:1450:4002:809::200e]:80)
wget: can't connect to remote host: Network is unreachable
But, if I try the same thing on a non-IPv6 address, everything works correctly:
# wget abc.com
Connecting to abc.com (199.181.132.250:80)
Connecting to abc.go.com (23.21.209.55:80)
index.html 100% |*******************************| 46408 0:00:00 ETA
This is the IPv6 kernel routing table:
# route -A inet6
Destination Next Hop Flags Metric Ref Use Iface
::1/128 :: U 0 0 1 lo
fe80::20a:a5ff:fe12:3446/128 :: U 0 0 1 lo
fe80::/64 :: U 256 0 0 eth0
ff02::1:ffdc:adf/128 :: UC 0 1 0 eth0
ff02::1:fff4:5e3d/128 :: UC 0 1 0 eth0
ff00::/8 :: U 256 0 0 eth0
IPv4 one:
# route -A inet
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.0.230 0.0.0.0 UG 0 0 0 eth0
192.168.0.0 * 255.255.255.0 U 0 0 0 eth0
Any help or suggestion would be very appreciated.
I solved the issue by installing standard wget instead of the Busybox's integrated one.
As it seems our internal network was not IPv6 capable and Busybox's wget stopped when it couldn't reach the destination. Standard wget, instead, falls back to the IPv4 address, completing the download.
EDIT: apart from our network, seems Busybox's wget has some issues
I'm newbie in networking field. I have trouble with my web server Network configuration (OS is Centos).
I have 2 NICs (eth0 + eth2 - physically) running 2 public IP which have the same subnet, same gateway.
When I configure nginx to listen on these 2 NICs, everything works just fine. But when I monitor the traffic, all traffic is on the eth0 only, nothing on eth2.
My question is: How can I configure so that traffic goes in a NIC, go out on that NIC, too?
This is my ethernet card config:
DEVICE="eth0"
ONBOOT=yes
BOOTPROTO=static
IPADDR=x.x.x.38
PREFIX=27
GATEWAY=x.x.x.x.33
DNS1=8.8.8.8
DNS2=8.8.4.4
NAME="System eth0"
DEVICE="eth2"
ONBOOT=yes
BOOTPROTO=static
IPADDR=x.x.x.39
PREFIX=27
GATEWAY=x.x.x.33
DNS1=8.8.8.8
DNS2=8.8.4.4
NAME="System eth2"
This is my route -n result
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.14.8.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
y.z.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
y.z.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1
y.z.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 eth2
0.0.0.0 x.x.x.33 0.0.0.0 UG 0 0 0 eth0
Hope you can help, thanks in advanced!
In Linux, routing is performed by looking at the destination address only, so a packet will follow whichever route can be used to reach the packet's destination, with no regard to the source address.
The behaviour you want requires choosing a route depending not only on the destination address, but also on the source address — this is sometimes called source-sensitive routing or SADR (source-address dependent routing). The most portable way of implementing source-sensitive routing under Linux is to define routing rules across multiple routing tables using the ip rule and ip route ... table ... commands.
This is described in detail in Section 4 of the Linux Advanced Routing and Traffic Control HOWTO
Probably, the problem can be solved even with NAT.
ip tuntap add dev tap0 mode tap
ip tuntap add dev tap1 mode tap
Then you can assign separate ip addresses to these devices:
ifconfig tap0 10.10.10.1 netmask 255.255.255.255
ifconfig tap1 10.10.10.2 netmask 255.255.255.255
And finally - redirect incoming traffic to specific virtual device
iptables -t nat -A PREROUTING -i eth0 -j DNAT --to-destination 10.10.10.1
iptables -t nat -A PREROUTING -i eth2 -j DNAT --to-destination 10.10.10.2
In this case, all traffic will be routed definetely to the interface it came from.
Quite easy use-case: I have two Ethernet interfaces which should both be connected using DHCP. It does not matter which one the default route is, since it is just a dedicated proof-of-concept setup. However, hot-plugging must work for both interfaces (one of them is a CDC-NCM connection).
connman 1.17 seems only to try to establish a connection to one of the interfaces. If I unplug this connection, DHCP is started on the second interface and everything continues to work. I can connect the second interface always by using “connmanctl connect” manually. But…
… is there any way to configure connman to connect AUTOMATICALLY to both networks via DHCP?
is there any way to configure connman to connect AUTOMATICALLY to both networks via DHCP?
By default most DHCP servers are configured to issue a default route to the client. If you get two default routes, it's impossible for the linux networking stack to function properly.
Since you would want DHCP clients on both networks to be able to function properly, perhaps the best thing you can do is scan the system routing table and remove one of the default routes.
Typically you'd use netstat -rn to find the duplicate default...
[mpenning#tsunami micro]$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
172.16.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.16.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
239.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
0.0.0.0 172.16.1.1 0.0.0.0 UG 0 0 0 eth0 <--
0.0.0.0 172.16.2.1 0.0.0.0 UG 0 0 0 eth1 <--
[mpenning#tsunami micro]$
Then remove one of them...
[mpenning#tsunami micro]$ sudo route del default dev eth1
[mpenning#tsunami micro]$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
172.16.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.16.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
239.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
0.0.0.0 172.16.1.1 0.0.0.0 UG 0 0 0 eth0
[mpenning#tsunami micro]$
It's not so hard to write a shell script to check for this issue. However, a better solution is just to get a static address on one of the networks so you don't have to manage around the DHCP issue.