Quite easy use-case: I have two Ethernet interfaces which should both be connected using DHCP. It does not matter which one the default route is, since it is just a dedicated proof-of-concept setup. However, hot-plugging must work for both interfaces (one of them is a CDC-NCM connection).
connman 1.17 seems only to try to establish a connection to one of the interfaces. If I unplug this connection, DHCP is started on the second interface and everything continues to work. I can connect the second interface always by using “connmanctl connect” manually. But…
… is there any way to configure connman to connect AUTOMATICALLY to both networks via DHCP?
is there any way to configure connman to connect AUTOMATICALLY to both networks via DHCP?
By default most DHCP servers are configured to issue a default route to the client. If you get two default routes, it's impossible for the linux networking stack to function properly.
Since you would want DHCP clients on both networks to be able to function properly, perhaps the best thing you can do is scan the system routing table and remove one of the default routes.
Typically you'd use netstat -rn to find the duplicate default...
[mpenning#tsunami micro]$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
172.16.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.16.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
239.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
0.0.0.0 172.16.1.1 0.0.0.0 UG 0 0 0 eth0 <--
0.0.0.0 172.16.2.1 0.0.0.0 UG 0 0 0 eth1 <--
[mpenning#tsunami micro]$
Then remove one of them...
[mpenning#tsunami micro]$ sudo route del default dev eth1
[mpenning#tsunami micro]$ netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
172.16.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
172.16.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
239.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 eth0
0.0.0.0 172.16.1.1 0.0.0.0 UG 0 0 0 eth0
[mpenning#tsunami micro]$
It's not so hard to write a shell script to check for this issue. However, a better solution is just to get a static address on one of the networks so you don't have to manage around the DHCP issue.
Related
I have an azure Linux VM say VM1 having only one network interface with private IP 10.3.0.5, I have another azure Linux VM say VM2 with two network interfaces, the private IP on primary network interface is 10.3.5.4, the private IP on the secondary network interface is 10.3.4.4.
Now I am able to ping VM2 from VM1 on primary network interface of VM2 as
ping 10.3.5.4 but I am not able to ping it in on secondary network interface as ping 10.3.4.4.
After reading azure docs - https://learn.microsoft.com/en-gb/azure/virtual-machines/linux/multiple-nics#configure-guest-os-for-multiple-nics, they say that you would have to manually add required routes to achieve this.
Similar problem in windows VM - https://support.microsoft.com/en-in/help/4048050/troubleshooter-for-azure-vm-connectivity-problems
here is the result of command route -n on VM2
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.3.5.1 0.0.0.0 UG 0 0 0 eth0
10.3.4.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
10.3.5.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
168.63.129.16 10.3.5.1 255.255.255.255 UGH 0 0 0 eth0
169.254.169.254 10.3.5.1 255.255.255.255 UGH 0 0 0 eth0
172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0
172.18.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker_gwbridge
Now as per the above azure docs, it is missing an entry something like this
0.0.0.0 10.3.4.1 0.0.0.0 UG 0 0 0 eth1
I tried adding this route to my VM2 but after running the following command, my vm just stopped responding
route add -net 0.0.0.0 netmask 0.0.0.0 gw 10.3.4.1 dev eth1
How do I add the correct route for my problem ? Please help !
My suggestion is adding another network interface to your VM1. For example, I have VMA with two network interfaces. The private IP of the primary network interface is 10.0.3.4.The private IP on the secondary network interface is 10.0.2.5.
Result of command route -n on VMA
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Ifac
0.0.0.0 10.0.3.1 0.0.0.0 UG 100 0 0 eth0
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
10.0.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
168.63.129.16 10.0.3.1 255.255.255.255 UGH 100 0 0 eth0
169.254.169.254 10.0.3.1 255.255.255.255 UGH 100 0 0 eth0
The VMB also has two network interfaces.The private IP of the primary network interface is 10.0.1.4.The private IP on the secondary network interface is 10.0.2.4. The private IP 10.0.2.4 of VMB and private IP 10.0.2.5 are in the same subnet.
Result of command route -n on VMB
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Ifa
0.0.0.0 10.0.1.1 0.0.0.0 UG 100 0 0 eth
10.0.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth
10.0.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth
168.63.129.16 10.0.1.1 255.255.255.255 UGH 100 0 0 eth
169.254.169.254 10.0.1.1 255.255.255.255 UGH 100 0 0 eth
So in the VMA, I can ping 10.0.1.4 through the primary network interface and ping 10.0.2.4 through the secondary network interface.
=== Background stuff ===
I have two wired ethernet interface on my Debian 9 server, both of them have internet access but through different ISP.
enp11s0 192.168.0.9
eth0 192.168.2.2
iptables -L and iptables -L -t nat is empty and accepts all incoming and outgoing. Here is my route:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 192.168.0.1 0.0.0.0 UG 0 0 0 enp11s0
116.xxx.xxx.0 192.168.2.1 255.255.255.0 UG 0 0 0 eth0
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 enp11s0
192.168.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
=== End of Background ===
I would like to let most traffic go through enp11s0 since the eth0 is metered, but few connection go through eth0 (e.g. 116.xxx.xxx.xxx). This works, but when I try to let a program bind to a specific interface, it becomes weird.
curl icanhazip.com --interface enp11s0 gives my ip address which is expected, however curl icanhazip.com --interface eth0 does not work. With -v flag here is the output:
* Rebuilt URL to: icanhazip.com/
* Trying 144.202.71.30...
* TCP_NODELAY set
* Local Interface eth0 is ip 192.168.2.2 using address family 2
* Local port: 0
* Trying 2001:19f0:6401:18fc:2709:e14e:21cd:4e41...
* TCP_NODELAY set
* Trying 2001:19f0:5c01:1aed:dcdf:7e47:e74b:1197...
* TCP_NODELAY set
And it just keeps repeating forever. I added -4 flag and Trying 2001:19f0:6401:18fc:2709:e14e:21cd:4e41 disappears, but it still doesn't work. How do I fix eth0? Thanks in advance!
see this question and answer here for a good explanation about how what you are attempting with curl is not what is happenning with curl.
You are curling an external address from eth0 via the server default gateway which is on the link enp11s0 . check that forwarding is enabled on the server.
You need to set up policy based routing. depending on what traffic you wan't to send out eth0, look into iproute2, ip rule, and
/etc/iproute2/rt_tables
Long question in short:
Ping over r1-r4-r2 path works using 10.0.1.* or 10.0.2.* IP addresses, but fails if we alter the path to r1-r3-r2 using 1.0.0.* or 1.0.1.* IP addresses for the exactly same packets (except for the fact that packets' src and dst IP fields are changed from 10.* to 1.* and vice-versa at s1 and s2 respectively). Why?
Question in detail:
I have a small topology as below
h1 -- s1 -- r1 -- r4 -- r2 -- s2 -- h2
\ /
\ /
\ /
r3
The s's are OpenvSwitch instances while r's are Ubuntu 16 Linux machines.
IP Addresses are:
h1-eth0 - 10.0.1.10/24
s1 - 10.0.1.50/24
h2-eth0 - 10.0.2.10/24
s2 - 10.0.2.50/24
r1-eth0 - 10.0.1.1/24
r1-eth1 - 10.0.11.2/24
r1-eth2 - 10.0.12.2/24
r2-eth0 - 10.0.2.1/24
r2-eth1 - 10.0.13.1/24
r2-eth2 - 10.0.5.1/24
r3-eth0 - 10.0.12.1/24
r3-eth1 - 10.0.5.2/24
r4-eth0 - 10.0.11.1/24
r4-eth1 - 10.0.13.2/24
As you can see, there are two similar paths between r1 and r2. I add the following static entries.
r1
sudo ip route add 10.0.2.0/24 via 10.0.11.1
r2
sudo ip route add 10.0.1.0/24 via 10.0.13.2
r4
sudo ip route add 10.0.1.0/24 via 10.0.11.2
sudo ip route add 10.0.2.0/24 via 10.0.13.1
The ping between h1 and h2 works as expected. Now, since the switches are OVS (and thus are OpenFlow-enabled) I install entries in s1 to map the destination IPs to a different subnet.
i.e. the IP 10.0.1.10 would be mapped to 1.0.0.10 while the IP 10.0.2.10 would be mapped to 1.0.1.10 when such a packet is received at s1, while the destination IPs would be mapped back to original at s2.
(I have checked that these entries are indeed correct and are working as expected. Also I have added this entry only to match ICMP packets). Similar procedure would be done when ping reply is sent by h1.
Along with these, I install static routes in the routers to route these IPs.
r1
sudo ip route add 1.0.0.0/24 via 10.0.1.50
sudo ip route add 1.0.1.0/24 via 10.0.12.1
r2
sudo ip route add 1.0.0.0/24 via 10.0.5.2
sudo ip route add 1.0.1.0/24 via 10.0.2.50
r3
sudo ip route add 1.0.0.0/24 via 10.0.12.2
sudo ip route add 1.0.1.0/24 via 10.0.5.1
Now if I ping h1 from h2, the packet starts with destination IP 10.0.1.10, which is mapped to 1.0.0.10 at s2, r2 routes this and sends it to r3, r3 routes it and sends to r1. But r1, even after receiving the packet at one interface and having the matching entry in the Linux routing table does not route and forward packet.
Even ip route get outputs the correct port to which the packet should be forwarded. There are no firewall entries in ip tables as well.
Some additional information:
If I change the newly added routing entries to use the original path
of r1-r4-r2 (i.e., we route on this path with mapped ip's) , it behaves as expected and the ping works as expected.
Alternatively, if I change the old routing entries for 10.0.2.0/24 in r1 and
10.0.1.0/24 in r2 (which now ideally don't even have to be matched by the new
packets as their Destination IPs are in 1.0.0.* range or 1.0.1.* only) to use
the new path r1-r3-r4 along with this mapped-IP packets, the ping between
r2 and r1 works as expected.
Details that may be required:
The final routing tables are as follows:
r1
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.11.1 0.0.0.0 UG 0 0 0 eth1
1.0.0.0 10.0.1.10 255.255.255.0 UG 0 0 0 eth0
1.0.1.0 10.0.12.1 255.255.255.0 UG 0 0 0 eth2
10.0.1.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
10.0.2.0 10.0.11.1 255.255.255.0 UG 0 0 0 eth1
10.0.11.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1
10.0.12.0 0.0.0.0 255.255.255.0 U 1 0 0 eth2
r2
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.13.2 0.0.0.0 UG 0 0 0 eth1
1.0.0.0 10.0.5.2 255.255.255.0 UG 0 0 0 eth1
1.0.1.0 10.0.2.50 255.255.255.0 UG 0 0 0 eth0
10.0.1.0 10.0.13.2 255.255.255.0 UG 0 0 0 eth1
10.0.2.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
10.0.5.0 0.0.0.0 255.255.255.0 U 1 0 0 eth2
10.0.13.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1
r3
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.0.5.1 0.0.0.0 UG 0 0 0 eth1
1.0.0.0 10.0.12.2 255.255.255.0 UG 0 0 0 eth0
1.0.1.0 10.0.5.1 255.255.255.0 UG 0 0 0 eth1
10.0.1.0 10.0.12.2 255.255.255.0 UG 0 0 0 eth0
10.0.2.0 10.0.5.1 255.255.255.0 U 1 0 0 eth1
10.0.5.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1
10.0.12.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
r4
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth4
1.0.0.0 10.0.11.2 255.255.255.0 UG 0 0 0 eth0
1.0.1.0 10.0.13.1 255.255.255.0 UG 0 0 0 eth1
10.0.1.0 10.0.11.2 255.255.255.0 UG 0 0 0 eth0
10.0.2.0 10.0.13.1 255.255.255.0 UG 0 0 0 eth1
10.0.11.0 0.0.0.0 255.255.255.0 U 1 0 0 eth0
10.0.13.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1
192.168.0.0 0.0.0.0 255.255.255.0 U 1 0 0 eth4
Note: 192.168.0.* is a subnet connected to outside Internet.
What do you think is the problem ? I am completely baffled looking at this problem.
The behavior of Linux routing here was as expected.
The flag for reverse path filter
i.e. /proc/sys/net/ipv4/conf/<interfacename>/rp_filter was turned ON (by setting value to 1) by default.
Reverse path filter is provided as a security feature of Linux kernel.
A common example is private IP space escaping onto the Internet. If you have an interface with a route of 195.96.96.0/24 to it, you do not expect packets from 212.64.94.1 to arrive there. So kernel drops such a packet if the flag is set to 1.
More formally,
Reverse path filtering is a mechanism adopted by the Linux kernel to
check whether the source IP address of the packet that is been
received is routable.
So in other words, when a machine with reverse path filtering enabled
receives a packet, the machine will first check whether the source of
the received packet is reachable through the interface it came in.
If it is routable through the interface which it came, then the machine will accept the packet.
If it is not routable through the interface, which it came, then the machine will drop that packet.
Latest kernels provide one more option value of 2. This option is slightly
more liberal in terms of accepting traffic.
If the received packet's source address is routable through any of the
interfaces on the machine, the machine will accept the packet.
To make that work, use this on all machines:
# for i in /proc/sys/net/ipv4/conf/*/rp_filter ; do
> echo 0 > $i
> done
Or make the following entry in your /etc/sysctl.conf
net.ipv4.conf.all.rp_filter = 0
rp_filter will filter packets in the three modes: 0 disabled, 1 strict and 2 loose.
Example
Client A - 192.168.2.10 - connected to router via eth0
Router
eth0 - 192.168.2.150
routes - 192.168.2.0/24
eth1 - 10.42.43.1
routes - 10.42.43.0/24
Note: No default route
Client C - 10.42.43.50 - connected to router via eth1
With this setup and rp_filter on the router set to “loose mode” (2) a packet on eth0 from 1.2.3.4 to 10.42.43.50 will be blocked.
With rp_filter on the router set to “strict mode” (1) a packet on eth0 from source address 10.42.43.2 will be blocked.
When set to “disabled” (0) both packets would go through.
First off your topology details is incomplete you are missing r3 and r4 details but they can be inferred.
Instead of trying to troubleshoot your issue I'm just going to try to explain what needs to happen. However it would be much easier if you just used a routing protocol like OSPF which is designed to make this easy so you don't have to do it by hand.
Each device that is routing needs to know how to get to every other subnet if its to be accessible. So this means you can either add in default routes (ie routes that match 0.0.0.0/0) or you can enter in each subnet with corresponding next-ip into each router (see below). Usually you do not need to add routes for subnets that are connected (IE you have an ip on that router in that subnet)
R1 routes
10.0.13.0/24 -> 10.0.11.1
10.0.5.0/24 -> 10.0.11.1
10.0.2.0/24 -> 10.0.11.1
R2 routes
10.0.1.0/24 -> 10.0.13.2
10.0.12.0/24 -> 10.0.13.2
10.0.11.0/24 -> 10.0.13.2
R3 routes
10.0.1.0/24 -> 10.0.12.2
10.0.11.0/24 -> 10.0.12.2
10.0.13.0/24 -> 10.0.5.1
10.0.2.0/24 -> 10.0.5.1
R4 Routes
10.0.1.0/24 -> 10.0.11.2
10.0.12.0/24 -> 10.0.11.2
10.0.2.0/24 -> 10.0.13.1
10.0.5.0/24 -> 10.0.13.1
For devices H1, S1, H2, and S2 they should have a default route that points to the gateway 10.0.1.1 and 10.0.2.1.
I'm newbie in networking field. I have trouble with my web server Network configuration (OS is Centos).
I have 2 NICs (eth0 + eth2 - physically) running 2 public IP which have the same subnet, same gateway.
When I configure nginx to listen on these 2 NICs, everything works just fine. But when I monitor the traffic, all traffic is on the eth0 only, nothing on eth2.
My question is: How can I configure so that traffic goes in a NIC, go out on that NIC, too?
This is my ethernet card config:
DEVICE="eth0"
ONBOOT=yes
BOOTPROTO=static
IPADDR=x.x.x.38
PREFIX=27
GATEWAY=x.x.x.x.33
DNS1=8.8.8.8
DNS2=8.8.4.4
NAME="System eth0"
DEVICE="eth2"
ONBOOT=yes
BOOTPROTO=static
IPADDR=x.x.x.39
PREFIX=27
GATEWAY=x.x.x.33
DNS1=8.8.8.8
DNS2=8.8.4.4
NAME="System eth2"
This is my route -n result
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.14.8.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
y.z.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
y.z.0.0 0.0.0.0 255.255.0.0 U 1003 0 0 eth1
y.z.0.0 0.0.0.0 255.255.0.0 U 1004 0 0 eth2
0.0.0.0 x.x.x.33 0.0.0.0 UG 0 0 0 eth0
Hope you can help, thanks in advanced!
In Linux, routing is performed by looking at the destination address only, so a packet will follow whichever route can be used to reach the packet's destination, with no regard to the source address.
The behaviour you want requires choosing a route depending not only on the destination address, but also on the source address — this is sometimes called source-sensitive routing or SADR (source-address dependent routing). The most portable way of implementing source-sensitive routing under Linux is to define routing rules across multiple routing tables using the ip rule and ip route ... table ... commands.
This is described in detail in Section 4 of the Linux Advanced Routing and Traffic Control HOWTO
Probably, the problem can be solved even with NAT.
ip tuntap add dev tap0 mode tap
ip tuntap add dev tap1 mode tap
Then you can assign separate ip addresses to these devices:
ifconfig tap0 10.10.10.1 netmask 255.255.255.255
ifconfig tap1 10.10.10.2 netmask 255.255.255.255
And finally - redirect incoming traffic to specific virtual device
iptables -t nat -A PREROUTING -i eth0 -j DNAT --to-destination 10.10.10.1
iptables -t nat -A PREROUTING -i eth2 -j DNAT --to-destination 10.10.10.2
In this case, all traffic will be routed definetely to the interface it came from.
I'm writing a program for my embedded Debian Linux system. It has a cell modem that is brought up as ppp0 and also has eth0 that may or may not be plugged in. How I want the system to work is for it to use ppp0 as the default route, unless it can establish a connection to the outside world over eth0, in that case eth0 should be made the default route. The eth0 cable could be unplugged at any time. To do this I've written a C++ program that uses curl to try and establish a connection to a public web server over eth0. If successful it then tries to go and make the default route eth0. I've borrowed some code from the Linux pppd project to manipulate the routing information. The system boots up with ppp0 active. I then run my program, and it detects it can make a connection via eth0 (as the cable is plugged in), and it then changes the default route to eth0. After that networking somehow stops working, I can no longer ping and public servers. Somehow my changing the route has killed networking. Before I run my program, the output of "route" is:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
80.sub-66-174-2 * 255.255.255.255 UH 0 0 0 ppp0
192.168.98.0 * 255.255.255.0 U 0 0 0 eth0
loopback * 255.0.0.0 U 0 0 0 lo
default * 0.0.0.0 U 0 0 0 ppp0
After I run my program, the output of "route" is:
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
66.174.217.80 * 255.255.255.255 UH 0 0 0 ppp0
192.168.98.0 * 255.255.255.0 U 0 0 0 eth0
loopback * 255.0.0.0 U 0 0 0 lo
default * 0.0.0.0 U 0 0 0 eth0
So I can't see any obvious signs of what's wrong. The code is posted at http://pastebin.com/AwKdvSf1
Thanks for any help or insights on how to fix this.
your default gateway ip address is not defined.
type route add default gw <gateway ip> as root (or sudo)
NOTE: using dhcp to assign ip settings would be best way I think to solve your problem otherwise you'll have to define the default route statically.
You'll have to add DEFROUTE=no in your ppp0 configuration.