I am using namespaces to separate a few physical interfaces on a server. The routing works perfectly.
Also I have a folder for each namespace in /etc/netns/ e.g. /etc/netns/namespaceA/resolv.conf so that DNS works fine as well.
The problem arises when using DHCP with dhclient.
I am running dhclient from inside a namespace and am getting this error.
(namespaceA)root#tc-vm:~#dhclient
RTNETLINK answers: File exists
mv: cannot move '/etc/resolv.conf.dhclient-new.2740' to '/etc/resolv.conf': Device or resource busy
I found out that the mv in /etc/resolvconf/update.d/libc contains a mv which might cause the problem.
How can dhclient be made namespace aware?
I looked into the issue myself.
What happens is that when you create a network namespace, you see /etc/resolv.conf of the host machine unless you create explicitly /etc/netns/<namespace_name>/resolv.conf, which will bind mount automatically to /etc/resolv.conf when looked up inside the network namespace. Therefore, by simply creating that path, the resolv.conf of the host won't be visibile any more on the network namespace, which will have its own resolv.conf.
The manual page of ip netns explains this:
For applications that are aware of network namespaces, the convention
is to look for global network configuration files first in
/etc/netns/NAME/ then in /etc/. For example, if you want a different
version of /etc/resolv.conf for a network namespace used to isolate
your vpn you would name it /etc/netns/myvpn/resolv.conf.
Ip netns exec automates handling of this configuration, file
convention for network namespace unaware applications, by creating a
mount namespace and bind mounting all of the per network namespace
configure files into their traditional location in /etc.
As far as updating resolv.conf, dhclient doesn't work in network namespaces out of the box when /etc/netns/<namespace_name>/resolv.conf exists (on the other hand, when it doesn't exist, it will overwrite the resolv.conf of the host machine, since it's the only one available, but that's not really desirable). As the error in the question above shows, what happens is that dhclient prepares a temporary file with the new nameserver details in /etc/resolv.conf.dhclient-new.2740 and then tries to rename it as /etc/resolv.conf. It generates an error because /etc/resolv.conf is already bind-mounted and apparently mv isn't allowed to do this trick.
In order to make dhclient work in network namespaces, /sbin/dhclient-script should be modified.
I removed this:
mv -f $new_resolv_conf /etc/resolv.conf
And replaced it with:
cat $new_resolv_conf > /etc/resolv.conf
rm -f $new_resolv_conf
Otherwise, dhcpcd seems to do this job correctly.
Related
This seems to be a new issue with network-manager-openconect-gnome in Ubuntu 18.04+
I install sudo apt install network-manager-openconnect-gnome to get gnome integration with opeconnect and Cisco AnyConnect Compatible VPN (openconnect)
As an aside (which may actually be relevant) I do this to get *.local addresses to resolve:
sudo ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf as per systemd docs
Move dns before mdns4_minimal in /etc/nsswitch.conf
If I connect to the VPN with openconnect through the gnome network manager, VPN addresses (sites for work) do not resolve. Regular sites continue to work as expected.
If I connect to the VPN with openconnect on the command line with sudo openconnect vpn.mycompany.com, VPN addresses (sites for work) do resolve. Regular sites continue to work as expected.
I thought I would check to see if there were any differences between /etc/resolv.conf with each of these VPN connection methods and sure enough, there is one:
openconnect on the command line (working):
##VPNC_GENERATED# -- this file is generated by vpnc
# and will be overwritten by vpnc
# as long as the above mark is intact
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.
nameserver 10.10.10.10
nameserver 10.10.10.11
search broadband mycompany.com
openconnect gnome integration (not working):
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.
nameserver 192.168.1.1
nameserver 10.10.10.10
nameserver 10.10.10.11
search broadband mycompany.com
If I remove (or comment out) the nameserver 192.168.1.1, which is the difference in content between the working and not working files... everything works as expected. I can resolve addresses within the company and regular sites work as expected.
This does not happen with Fedora. Everything works out of the box. I'm not sure why the network-manager-openconnect-gnome package works differently or if there's a way I can make it work without either
Editing the file by hand.
Using the openconnect tool from the command line and keeping a terminal open running that command.
I installed Linux Mint 19 after using Ubuntu for a while,
I a have a local domain with subdomains,
I can ping mydomain.local
but I can't ping or access my subdomains with their names,
my DNS server IP = 10.0.0.4
Here's my /etc/resolv.conf generated with resolvconf:
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.
nameserver 10.0.0.4
And here's my /run/systemd/resolve/stub-resolv.conf :
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "systemd-resolve --status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs must not access this file directly, but only through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.
nameserver 10.0.0.4
Please help me.
I found the problem, it was in the file /etc/nsswitch.conf in the line :
hosts: files mdns4_minimal [NOTFOUND=return] dns myhostname
remark the "dns" is after the "[NOTFOUND=return]" part, so that is why the DNS resolution is not going correctly, so what I have done is moving "dns" before "[NOTFOUND=return]" part, and everything is good now.
I'm no Mint expert, don't know if I can help, but I'll certainly try... It's a bit 'blind leading the blind' in support forums :/
I recently installed Mint 19 but the only DNS issues I've had were caused by systemd's resolver. I have since replaced it with unbound.
I used this guy's instructions and it worked perfectly fine for me. YMMV.
https://blobfolio.com/2017/05/fix-linux-dns-issues-caused-by-systemd-resolved/
tl;dr: With systemd-resolve, LLMNR is used for short name resolution, which doesn't failover to a canonicalised DNS resolution. LLMNR is tried a few times and then it just fails without touching DNS.
I think there's a severe bug in it, TBQH - Others agree. Systemd's resolver behaves in an unexpected and possibly standards-breaking way and has done for 3-4 years.
HTH :D
I got two network interfaces (ethernet and wlan). Now I found a little script on github (https://github.com/Intika-Linux-Firewall/App-Route-Jail) which seems to allow me to route specific applications through the none default gateway to loadbalance the traffic a little bit.
The script is using the following call:
setsockopt(sd, SOL_SOCKET, SO_MARK, &mark, sizeof(mark));
As I found out via an strace I got: '-1 EPERM Permission denied' (as show on http://man7.org/linux/man-pages/man7/capabilities.7.html this command needs cap_net_admin rights)
The tool works when I use "sudo" infront of the command the socket is created with the none default gateway and works like intended (e.g. wget a file)
Example:
MARK=10 LD_PRELOAD=./mark.so wget -qO- ifconfig.me uses the default gateway (so not what I wanted)
sudo MARK=10 LD_PRELOAD=./mark.so wget -qO- ifconfig.me returns the IP of the none default gateway (what I want but without using sudo)
Some things I found on the internet and tried:
Setting File Capabilities (setcap cap_net_admin+eip) (still requires root somehow)
Using SUID Bit (chmod u+s) (no change at all)
I expect the command to run without root privileges, so every application can use the none default gateway, but currently I need to use sudo to gain enough permissions to run the command on the correct interface.
What am I missing to archive my goals?
You are looking to your problem from the wrong angle. Instead of marking the packets, you should try to force the application to bind to the correct interface in the first place.
You could try the retro-solution explained here, which overrides bind() and connect() instead of socket().
But more modern solution would be to create a separate network namespace and then run applications in their own namespace. Google ip netns for the examples. Creating and setting up the network namespace still requires root, but these privileges can be dropped before running the application. There are likely to be tools available to do that also.
I am deploying a Kops Kubernetes cluster on AWS with Debian Jessie image.
Mine is a hybrid environment where my artifactory is in a physical env in our DC. Now I have been facing an issue, my worker nodes are unable to pull images from my artifactory unless I specify the artifactory FQDN and IP in the /etc/hosts file.
So this is a manual edit, it works all fine after I do this. So I went ahead and added the data in my additional userdata of the Kops worker node group, but I am seeing after some time the hosts file on worker nodes is getting overwritten and also this is evident upon node reboot.
So how can I resolve this!!
The real answer is to run your own DNS server, or at least use DNS hostnames to resolve. If your router supports it, you can set local hostnames (machine-1.local)
If that isn't possible, you could try a solution like puppet if you own the virtual machines. Also, I believe Kubernetes does have a DNS addon. Also, you could use a crontab for on boot to write to the hosts file, but that's a dirty solution.
In addition, your hosts file would get rewritten for every DHCP renew. You could use static IPs, but again, DNS is the way to go.
Another workaround for this is to put it in your /etc/rc.local file:
If the file exists add this to the end:
echo '<ip-address-of-artifactory> <fqdn-of-artifactory>' >> /etc/hosts
If the file doesn't exist, create it:
$ cat << EOF > /etc/rc.local
#!/bin/sh -e
#
echo '<ip-address-of-artifactory> <fqdn-of-artifactory>' >> /etc/hosts
EOF
$ chmod 755 /etc/rc.local
$ reboot # check that it works
I am trying to mount a CephFS volume on CoreOS. It currently works by specifying the IP address of one of my monitors:
mount -t ceph 1.2.3.4:6789:/ /mnt
However, I have three monitors for redundancy and have configured my DNS server to resolve to all three of them so I want to specify the domain name instead of the IPs:
mount -t ceph ceph-mon:6789:/ /mnt
However, this doesn't work. I think mount relies on mount.ceph being installed properly in order to resolve hostnames but I'm not certain about that.
How can I achieve this with CoreOS?