Thanks for the read.
I'd like to preface this with the fact that I am weak on dev-ops! :)
First off, my environment:
3 node cluster each of which are CentOS 7 VMs running in VirtualBox 5.1.3. I have an additional network adaptor enabled running an internal network visible only to the VMs.
I have configured the VMs to use the additional adaptor and am able to ping all nodes via it's network.
etcd config:
{ "Network": "172.30.0.0/24", "Backend": { "Type": "vxlan" } }
nmcli output:
enp0s3: connected to enp0s3
"Intel 82540EM Gigabit Ethernet Controller (PRO/1000 MT Desktop Adapter)"
ethernet (e1000), 08:00:27:DD:CB:CA, hw, mtu 1500
ip4 default
inet4 10.0.2.15/24
inet6 fe80::49e4:5aa5:65c8:6e48/64
enp0s8: connected to enp0s8
"Intel 82540EM Gigabit Ethernet Controller (PRO/1000 MT Desktop Adapter)"
ethernet (e1000), 08:00:27:40:11:AE, hw, mtu 1500
inet4 192.168.0.10/24
inet6 fe80::90cc:f3b4:7e73:cf3f/64
I am working through the installation/configuration instructions in the following documentation.
I wanted to run through this full installation to grasp the overall topology if the product.
Installation seems to be fine and I am able to deploy pods to the cluster. The issue is that flanneld does not seem to be working. I have it explicitly bound to the internal network adapter in the config (enp0s8), however it's showing:
flannel.1: disconnected
"flannel.1"
vxlan, 0A:91:01:BC:7D:7D, sw, mtu 1450
Any insight on this issue would be appreciated. I have done due diligence before reaching out to the community!
Thanks in advance.
Braden
Related
I am facing an issue while configuring rdma and Infiniband on my two nodes. Both of these two nodes are connected and I have installed the recommended software libraries and packages required.
But my port status is down and physical state is Disabled. I tried to enable the state but I get the error of can’t open MAD PORT
:~# ibportstate -L 1 3 enable
ibwarn: [6772] mad_rpc_open_port: can't open UMAD port ((null):0)
ibportstate: iberror: failed: Failed to open '(null)' port '0'
Infiniband ibstatus returns this:
Infiniband device ‘mlx5_0’ port 1 status:
default gid: fe80:0000:0000:0000:1270:fdff:fe6e:43e0
base lid: 0x0
sm lid: 0x0
state: 1: DOWN
phys state: 3: Disabled
rate: 100 Gb/sec (4X EDR)
link_layer: Ethernet
I don't understand what seems to be the issue here, I also upgraded the firmware but still the problem persists.
I figured it out and I am sharing the answers for others to see so the issue was the network interface, you need to see which network interface the Infiniband and check the status.
root#dtn0:~# /etc/init.d/openibd status
HCA driver loaded
Configured Mellanox EN devices:
ens11np0
Currently active Mellanox devices:
The following OFED modules are loaded:
rdma_ucm
rdma_cm
ib_ipoib
mlx5_core
mlx5_ib
ib_uverbs
ib_umad
ib_cm
ib_core
mlxfw
After that, I just assigned Ip and netmask on the interface and I was able to use the interface and reach the network.
root#dtn0:~# ifconfig ens11np0 10.0.0.50/24
I have a VM backed by vCenter.
vCenter ESXi have physical adapter "Broadcom BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller" and SR-IOV enabled on this.
VM is connected to 1mgmt network (vmxnet3) and 2 SR-IOV adapters (SRIOVPassthrough).
Upon booting of the VM, only 2 networks shown up. (1mgmt and 1SR-IOV).
Journalctl -k logs showed following error.
[ 4832.408471] bnxt_en 0000:13:00.0 (unnamed net_device) (uninitialized): Error (timeout: 500015) msg {0x0 0x0} len:0
[ 4832.408930] bnxt_en: probe of 0000:13:00.0 failed with error -1
Reboot of machine did not help at all.
For the successful one adapter
bnxt_en 0000:03:00.0 eth1: NIC Link is Up, 25000 Mbps full duplex, Flow control: ON - receive & transmit
bnxt_en 0000:03:00.0 eth1: FEC autoneg off encodings: None
I did rescan of the pci devices and did multiple times reboot without any success.
Any pointers would be really helpful
We've got a similar issue and were able to fix it.
In our case we had the same error message on Debian 10, 11 and Oracle Linux 8 but we installed it directly on hardware without an hypervisor.
But it could be the same issue cause you're using passthrough.
There are two ways to fix it:
Usage of UEFI Boot
Disable PXE Boot and keep Bios / Legacy Boot
Both options fixed it.
Disabling PXE didn't work for us, but we can get the ports back online, by running
echo 0000:af:00.0 > /sys/bus/pci/drivers/bnxt_en/bind
Where 0000:af:00.0 is the PCI number for the port, which can be gotten from dmesg | grep bnxt_en and looking for the port or ports that failed.
is it possible to reply all incoming packets/request on same network interface?
This is my setting: I have a headless Raspberry Pi (raspbian) with two network interfaces (eth0 and eth1). The fist interface (eth0) uses a public IP address, which is static. This interface is intended to provide access to the Pi (time- and web server, SSH) via the Internet. The second interface (eth1) uses the Raspberry Pi for general Internet connection (perform updates, sync own time or whatever) and uses a dynamic IP via DHCP. A general Internet connectivity over eth0 is not possible, so I have to use eth1 on the Pi.
My problem is that Internet (on the Pi) and the Internet access to the Pi are not working correctly.
first configuration (/etc/dhcpcd.conf):
interface eth0
static ip_address=141.41.241.68/28
static routers=141.41.241.65 192.168.0.1
after reboot 'ifconfig' shows the correct IP settings:
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 141.41.241.68 netmask 255.255.255.240 broadcast 141.41.241.79
...
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.101 netmask 255.255.255.0 broadcast 192.168.0.255
...
result:
Internet on the Raspberry Pi (eth1): doesn't work
access to Raspberry Pi over Internet (via eth0):
SSH (TCP): works fine
Apache web server (TCP): works fine
NTP time server (UDP): works fine
Now I tried to change the metric of the interfaces, hoping that the change of prioritization is successful
second configuration (/etc/dhcpcd.conf):
interface eth1
metric 200
interface eth0
static ip_address=141.41.241.68/28
static routers=141.41.241.65 192.168.0.1
metric 201
result:
Internet on the Raspberry Pi (eth1): works fine
access to Raspberry Pi over Internet (via eth0):
SSH (TCP): doesn't work
Apache web server (TCP): doesn't work
NTP time server (UDP): doesn't work
Ok. I think all traffic going out to eth1 at default. With the tool 'iptraf-ng' I was able to see the problem:
TCP Connections (Source Host:Port) Iface
--------------------------------------------------
80.187.108.126:53024 eth0
141.41.241.68:80 eth0
141.41.241.68:80 eth1
80.187.108.126:53024 eth1
80.187.108.126:53025 eth0
141.41.241.68:80 eth0
141.41.241.68:80 eth1
80.187.108.126:53025 eth1
UDP Connections
--------------------------------------------------
UDP (76 bytes) from 80.187.108.126:28599 to 141.41.241.68:123 on eth0
UDP (76 bytes) from 192.168.0.101:123 to 80.187.108.126:28599 on eth1
We see:
On TCP: some connections going out to the wrong interface (eth1).
On UDP: The request from 80.187.108.126 came over eth0 and the response was sent over eth1.
Next, I defined the routing table to reply incoming packets on same network interface...
echo 100 public >> /etc/iproute2/rt_tables
ip rule add from 141.41.241.68/32 table public
ip route add default via 141.41.241.65 dev eth0 table public
result:
Internet on the Raspberry Pi (eth1): works fine
access to Raspberry Pi over Internet (via eth0):
SSH (TCP): works fine
Apache web server (TCP): works fine
NTP time server (UDP): doesn't work
and 'iptraf-ng' shows:
TCP Connections (Source Host:Port) Iface
--------------------------------------------------
141.41.241.68:80 eth0
80.187.108.126:52083 eth0
141.41.241.68:80 eth0
80.187.108.126:52084 eth0
141.41.241.68:80 eth0
80.187.108.126:52085 eth0
141.41.241.68:80 eth0
80.187.108.126:52086 eth0
141.41.241.68:80 eth0
80.187.108.126:52087 eth0
UDP Connections
--------------------------------------------------
UDP (76 bytes) from 80.187.108.126:28599 to 141.41.241.68:123 on eth0
UDP (76 bytes) from 192.168.0.101:123 to 80.187.108.126:28599 on eth1
We see:
On TCP: now it works correctly
On UDP: same problem :(
What can I do to send UDP responses over the correct interface (eth0)? I have no idea why TCP works fine but UDP fails :(
Its very frustrating and I have no more ideas.
I hope someone can help.
best regards,
SBond
My Dear All the Greatest Lords,
Some expert listed the details of connecting to a wireless network as,
This is a step-to-step guide for connecting to a WPA/WPA2 WiFi network via the Linux command line interface. The tools are:
wpa_supplicant
iw
ip
ping
iw is the basic tool for WiFi network-related tasks, such as finding the WiFi device name, and scanning access points. wpa_supplicant is the wireless tool for connecting to a WPA/WPA2 network. ip is used for enabling/disabling devices, and finding out general network interface information.
The steps for connecting to a WPA/WPA2 network are:
Find out the wireless device name.
$ /sbin/iw dev
phy#0
Interface wlan0
ifindex 3
type managed
The above output showed that the system has 1 physical WiFi card, designated as phy#0. The device name is wlan0. The type specifies the operation mode of the wireless device. managed means the device is a WiFi station or client that connects to an access point.
Check that the wireless device is up.
$ ip link show wlan0
3: wlan0: (BROADCAST,MULTICAST) mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
link/ether 74:e5:43:a1:ce:65 brd ff:ff:ff:ff:ff:ff
Look for the word "UP" inside the brackets in the first line of the output.
In the above example, wlan0 is not UP. Execute the following command to bring it up:
$ sudo ip link set wlan0 up
[sudo] password for peter:
Note: you need root privilege for the above operation.
If you run the show link command again, you can tell that wlan0 is now UP.
$ ip link show wlan0
3: wlan0: (NO-CARRIER,BROADCAST,MULTICAST,UP) mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
link/ether 74:e5:43:a1:ce:65 brd ff:ff:ff:ff:ff:ff
Check the connection status.
$ /sbin/iw wlan0 link
Not connected.
The above output shows that you are not connected to any network.
Scan to find out what WiFi network(s) are detected
$ sudo /sbin/iw wlan0 scan
BSS 00:14:d1:9c:1f:c8 (on wlan0)
... sniped ...
freq: 2412
SSID: stanford
RSN: * Version: 1
* Group cipher: CCMP
* Pairwise ciphers: CCMP
* Authentication suites: PSK
* Capabilities: (0x0000)
... sniped ...
The 2 important pieces of information from the above are the SSID and the security protocol (WPA/WPA2 vs WEP). The SSID from the above example is stanford. The security protocol is RSN, also commonly referred to as WPA2. The security protocol is important because it determines what tool you use to connect to the network.
Connect to WPA/WPA2 WiFi network.
This is a 2 step process. First, you generate a configuration file for wpa_supplicant that contains the pre-shared key ("passphrase") for the WiFi network.
$ sudo -s
[sudo] password for peter:
$ wpa_passphrase stanford >> /etc/wpa_supplicant.conf
...type in the passphrase and hit enter...
wpa_passphrase takes the SSID as the single argument. You must type in the passphrase for the WiFi network stanford after you run the command. Using that information, wpa_passphrase will output the necessary configuration statements to the standard output. Those statements are appended to the wpa_supplicant configuration file located at /etc/wpa_supplicant.conf.
Note: you need root privilege to write to /etc/wpa_supplicant.conf.
$ cat /etc/wpa_supplicant.conf
# reading passphrase from stdin
network={
ssid="stanford"
#psk="testtest"
psk=4dfe1c985520d26a13e932bf0acb1d4580461dd854ed79ad1a88ec221a802061
}
The second step is to run wpa_supplicant with the new configuration file.
$ sudo wpa_supplicant -B -D wext -i wlan0 -c /etc/wpa_supplicant.conf
-B means run wpa_supplicant in the background.
-D specifies the wireless driver. wext is the generic driver.
-c specifies the path for the configuration file.
Use the iw command to verify that you are indeed connected to the SSID.
$ /sbin/iw wlan0 link
Connected to 00:14:d1:9c:1f:c8 (on wlan0)
SSID: stanford
freq: 2412
RX: 63825 bytes (471 packets)
TX: 1344 bytes (12 packets)
signal: -27 dBm
tx bitrate: 6.5 MBit/s MCS 0
bss flags: short-slot-time
dtim period: 0
beacon int: 100
Obtain IP address by DHCP
$ sudo dhclient wlan0
Use the ip command to verify the IP address assigned by DHCP. The IP address is 192.168.1.113 from below.
$ ip addr show wlan0
3: wlan0: mtu 1500 qdisc mq state UP qlen 1000
link/ether 74:e5:43:a1:ce:65 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.113/24 brd 192.168.1.255 scope global wlan0
inet6 fe80::76e5:43ff:fea1:ce65/64 scope link
valid_lft forever preferred_lft forever
Add default routing rule.
The last configuration step is to make sure that you have the proper routing rules.
$ ip route show
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.113
The above routing table contains only 1 rule which redirects all traffic destined for the local subnet (192.168.1.x) to the wlan0 interface. You may want to add a default routing rule to pass all other traffic through wlan0 as well.
$ sudo ip route add default via 192.168.1.254 dev wlan0
$ ip route show
default via 192.168.1.254 dev wlan0
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.113
ping external ip address to test connectivity
$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_req=1 ttl=48 time=135 ms
64 bytes from 8.8.8.8: icmp_req=2 ttl=48 time=135 ms
64 bytes from 8.8.8.8: icmp_req=3 ttl=48 time=134 ms
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 134.575/134.972/135.241/0.414 ms
The above series of steps is a very verbose explanation of how to connect a WPA/WPA2 WiFi network. Some steps can be skipped as you connect to the same access point for a second time. For instance, you already know the WiFi device name, and the configuration file is already set up for the network. The process needs to be tailored according to your situation.
Thoroughly followed the above tutorial, I failed to connect the wireless router.
(working as root)
......
#wpa_supplicant -B -i wlan0 -c /etc/wpa_supplicant.conf -D wext
#iw wlan0 link
Not connected.
Even I disable the WPA authentication using,
iwconfig wlan0 essid XXXXXXXXXXXXX
of no avail.
But the GNOME wireless tray is functioning(can select, connect, disconnect etc.)
Thank you a lot in advance.
Latest wpa_supplicant is able to do all the job itself.
The wpa_supplicant option you wrote seem to me ok.
But please, check the options in the file "/etc/wpa_supplicant.conf", if it readable and is well written (ssid,wpa,password correct....)
Problem
I tried to set up a VirtualBox Client with Ubuntu (Server Edition) running a neo4j database for web development.
It all seemed to work (eg. SSH and SFT connection work), until I wanted to access the web frontend of the former mentioned DB.
In detail, i want to open port 7474 on my client through my webbrowser on my host machine.
I already crawled the web for answers, but none where applicable or solved my problem.
Setup
My host system is an Ubuntu 14.04 LTS 64 bit machine, running the VirtualBox GUI in Version 4.3.10
The client VM is a fresh install of Ubuntu 14.10 server 64 bit including the SSH package.
I use the Network-Bridge connection to eth0 of my host (see details below) with promiscuous Mode: allow for all VMs and Host.
The only other packages installed are neo4j and its dependencies.
I installed neo4j and, as far as I can tell, it's up and running. At least that's what service neo4j-service status said (* neo4j is running).
I don't have any apache or lighttp installed.
That's what ifconfig says on my host machine (sorry, it's in German)
eth0 Link encap:Ethernet Hardware Adresse d4:3d:7e:50:3b:2d
inet Adresse:192.168.0.45 Bcast:192.168.0.255 Maske:255.255.255.0
inet6-Adresse: fe80::d63d:7eff:fe50:3b2d/64 Gültigkeitsbereich:Verbindung
UP BROADCAST RUNNING MULTICAST MTU:1500 Metrik:1
RX-Pakete:71671 Fehler:0 Verloren:0 Überläufe:0 Fenster:0
TX-Pakete:53988 Fehler:0 Verloren:0 Überläufe:0 Träger:0
Kollisionen:0 Sendewarteschlangenlänge:1000
RX-Bytes:46107644 (46.1 MB) TX-Bytes:10170816 (10.1 MB)
lo Link encap:Lokale Schleife
inet Adresse:127.0.0.1 Maske:255.0.0.0
inet6-Adresse: ::1/128 Gültigkeitsbereich:Maschine
UP LOOPBACK RUNNING MTU:65536 Metrik:1
RX-Pakete:1836 Fehler:0 Verloren:0 Überläufe:0 Fenster:0
TX-Pakete:1836 Fehler:0 Verloren:0 Überläufe:0 Träger:0
Kollisionen:0 Sendewarteschlangenlänge:0
RX-Bytes:185486 (185.4 KB) TX-Bytes:185486 (185.4 KB)
And on my client
eth0 Link encap:Ethernet Hardware Adresse 08:00:27:15:31:ca
inet Adresse:192.168.0.18 Bcast:192.168.0.255 Maske:255.255.255.0
inet6-Adresse: fe80::a00:27ff:fe15:31ca/64 Gültigkeitsbereich:Verbindung
UP BROADCAST RUNNING MULTICAST MTU:1500 Metrik:1
RX-Pakete:9690 Fehler:0 Verloren:0 Überläufe:0 Fenster:0
TX-Pakete:96 Fehler:0 Verloren:0 Überläufe:0 Träger:0
Kollisionen:0 Sendewarteschlangenlänge:1000
RX-Bytes:828688 (828.6 KB) TX-Bytes:12046 (12.0 KB)
lo Link encap:Lokale Schleife
inet Adresse:127.0.0.1 Maske:255.0.0.0
inet6-Adresse: ::1/128 Gültigkeitsbereich:Maschine
UP LOOPBACK RUNNING MTU:65536 Metrik:1
RX-Pakete:999 Fehler:0 Verloren:0 Überläufe:0 Fenster:0
TX-Pakete:999 Fehler:0 Verloren:0 Überläufe:0 Träger:0
Kollisionen:0 Sendewarteschlangenlänge:0
RX-Bytes:97214 (97.2 KB) TX-Bytes:97214 (97.2 KB)
So I was trying to open http://192.168.0.18:7474 in my browser (Firefox and Opera as well), but none of both could establish a connection.
I can ping to 8.8.8.8 (google) and to my host / client from either one, so the whole problem should be in the ports, I presume.
I also already tried to sudo ufw disable, but it didn't helped.
The clients sudo iptables -L -n says now
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
I would appreciate any help, thank you for your time.
By default Neo4j just listens on the localhost interface for security reasons. You can open that up to the world by setting org.neo4j.server.webserver.address=0.0.0.0 in your $NEO4J_HOME/conf/neo4j-server.properties.
I installed the Neo4j 3.4.0 Community Edition in a Ubuntu 16.04 virtual box instance using the instructions provided in here and in order to open up the Neo4j browser to my OSX Host I needed to
Navigate to /etc/neo4j
vim neo4j.conf - the only file that is there
Find the property dbms.connectors.default_listen_address=0.0.0.0 and uncomment it
Restart and test in your host (assuming your vm's network adapter settings are configured - in my case it is Host-only Adapter)
I could not find the neo4j-server.properties and also the property seem to be have different name than what is mentioned in Stefan's answer. Hopefully I did not turn on the wrong property.