On one of my systems at work good ol' netstat shows multicast group membership information, and ss is missing a couple. I wonder why. For example:
[root#myhost ~]# netstat -gn | egrep "Inter|239.192"
Interface RefCnt Group
em4.204 1 239.192.33.183
em1.16 2 239.192.35.1
em1.16 2 239.192.12.98
em1.16 1 239.192.32.1
[root#myhost ~]# ss -apu | egrep "State|239.192"
State Recv-Q Send-Q Local Address:Port Peer Address:Port
UNCONN 0 0 239.192.35.1:12965 *:*
UNCONN 0 0 239.192.12.98:12965 *:*
UNCONN 0 0 239.192.35.1:12965 *:*
UNCONN 0 0 239.192.12.98:12965 *:*
Notice that ss shows only those groups with a RefCnt of 2.
Technically, ip maddr show is the replacement for netstat -gn but it doesn't include the RefCnt, and its output is more cumbersome. Also, we're interested in the more detailed output of ss which can include the PID of the listening processes (not seen here because no processes are actually currently listening to the multicast, as show by netstat -ulpn:
[root#myhost ~]# netstat -ulpn | egrep "Proto|239.192"
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
udp 0 0 239.192.35.1:12965 0.0.0.0:* -
udp 4480 0 239.192.12.98:12965 0.0.0.0:* -
udp 0 0 239.192.35.1:12965 0.0.0.0:* -
udp 0 0 239.192.12.98:12965 0.0.0.0:* -
I tried to look for a good doc on netstat especially for the options -an but cannot find anything detailed enough.
example 1:
tcp 0 126 android.home:58797 par10s34-in-x0e.1e100.net:https ESTABLISHED
what is par10s34-in-x0e.1e100.net? it is not a domain name? nslookup returns: Can't find par10s34-in-x0e.1e100.net: No answer
example 2:
tcp 0 0 2a01:cb1d:192:e100:f167:9ba7:494f:61fa:45634 par21s03-in-x14.1e100.net:https ESTABLISHED
what is 2a01:cb1d:192:e100:f167:9ba7:494f:61fa:45634?
2a01:cb1d:192:e100:f167:9ba7:494f:61fa is a IPv6 address.
45634 is a port number.
par10s34-in-x0e is a hostname.
1e100.net is a domain name.
I am not able to find an answer to a simple thing I will try to achive:
once a tcp connection is established to my linux server, let's say ssh / tcp 22 or x11 / tcp 6000 display -> how do I close this connection without killing the process (sshd / x11 display server).
I saw also some suggestoin to use iptables, but it does not work for me, the connection is still visible in netstat -an.
would be good if someone can point me to the right direction.
what I tried so far
tcpkill: kills the process, not good for me
iptables: does not close the established connection, but prevent further connections.
Thanks in adavnce
DJ
Ok, I found at least one solution (killcx) which is working. Maybe we will be able to find an easier solution.
Also, i saw the comment from "zb" - thanks - which might also work, but I was not able to find a working syntax, since this tool seems to be really useful but complex.
So here is an example how to work with the 1. solution which is working for me:
netstat -anp | grep 22
output: tcp 0 0 192.168.0.82:22 192.168.0.77:33597 VERBUNDEN 25258/0
iptables -A INPUT -j DROP -s 192.168.0.77 (to prevent reconnect)
perl killcx.pl 192.168.0.77:33597 (to kill the tcp connection)
killcx can be found here: http://killcx.sourceforge.net/
it "steals" the connection from the foreign host (192.168.0.77) and close it. So that solution is working fine, but to complex to setup quickly if you are under stress. Here are the required packages:
apt-get install libnetpacket-perl libnet-pcap-perl libnet-rawip-perl
wget http://killcx.sourceforge.net/killcx.txt -O killcx.pl
however, would be good to have an easier solution.
tcpkill wont work, since it will only kill any new connection, it doesnt kill existing ESTABLISHED connections
heres how you remove an Established TCP connection
find the PID of the process and the IP of the client connecting,
lets say you are on serverA and someone is connecting from serverB
root#A> netstat -tulpan | grep ssh | grep serverB
should see something like,
tcp 0 0 <serverA IP>:<port> <serverB>:<port> ESTABLISHED 221955/sshd
use lsof utility to get the File Descriptor of this connection using the parent PID
root#A> lsof -np 221995 | grep serverB IP
should see something like this
sshd 221955 <user> 17u IPv4 2857516568 0t0 TCP <serverA IP>:<port>-><serverB IP>:<port> (ESTABLISHED)
get the File Descriptor number (4th column) = 17u
use GDB to shut down this connection, w/out killing sshd
root#A> gdb -p 211955 --batch -ex 'call shutdown(17u, 2)'
should see something similar,
0x00007f0b138c0b40 in __read_nocancel () from /usr/lib64/libc.so.6
$1 = 0
[Inferior 1 (process 211955) detached]
that TCP connection should now be closed
How to create a TCP connection using nc which listens to multiple hosts?
nc -l -p 12345
Simultaneous connections are not possible with netcat. You should use something like ucspi-tcp's tcpserver tool or leverage xinetd since you're on Linux.
See: https://superuser.com/questions/232747/netcat-as-a-multithread-server
Consecutive connections could be handled through a shell script that restarts netcat after it finishes.
ncat can do it.
E.g. ncat --broker --listen -p 12345 will distribute all incoming messages to all other clients (think of it as a hub).
I recommend socat as nc alternative.
For OP's problem, socat - TCP-LISTEN:12345,fork,reuseaddr can do the job.
-k
Forces nc to stay listening for another connection after its current connection is completed. It is an error to use this option without the -l option.
using nc it is not possible to open parallel connections to same port, however you can trick nc to open multiple connections to same port.
To understand this, lets say you start listening on 4444 port using $ nc -l -p 4444 -v. Now, if you check output of $ netstat -anp | grep 4444 you will get its state as LISTEN and in here its pid is 3410.
tcp 0 0 0.0.0.0:4444 0.0.0.0:* LISTEN 3410/nc
Now, after it gets connected to client, lets say you run $ nc localhost 4444 -v, its state will get changed into ESTABLISHED. Now, try running $ netstat -anp | grep 4444 you will get its state as ESTABLISHED, see for same pid 3410, and a client process with pid 3435
tcp 0 0 127.0.0.1:46678 127.0.0.1:4444 ESTABLISHED 3435/nc
tcp 0 0 127.0.0.1:4444 127.0.0.1:46678 ESTABLISHED 3410/nc
Please note that there is no available listening port, so you can't have another client process. However if you run again $ nc -l -p 4444 -v you can have a listening port and can have multiple client process.
see netstat -anp | grep 4444 output after you start listening to same port.
tcp 0 0 0.0.0.0:4444 0.0.0.0:* LISTEN 3476/nc
tcp 0 0 127.0.0.1:46678 127.0.0.1:4444 ESTABLISHED 3435/nc
tcp 0 0 127.0.0.1:4444 127.0.0.1:46678 ESTABLISHED 3410/nc
see netstat -anp | grep 4444 output after you attach new client to same port.
tcp 0 0 127.0.0.1:4444 127.0.0.1:46694 ESTABLISHED 3476/nc
tcp 0 0 127.0.0.1:46678 127.0.0.1:4444 ESTABLISHED 3435/nc
tcp 0 0 127.0.0.1:4444 127.0.0.1:46678 ESTABLISHED 3410/nc
tcp 0 0 127.0.0.1:46694 127.0.0.1:4444 ESTABLISHED 3483/nc
You can say connections behavior is like:
SERVER_PROCESS_1 <---> CLIENT_PROCESS_1
SERVER_PROCESS_2 <---> CLIENT_PROCESS_2
so, you can write some script to simulate this behavior, or use this bash script to modify.
#!/usr/bin/bash
lport="4444"
i=0;
while [ true ]; do
echo "opening socket $(( i++ ))";
if [[ "$(ss sport = :$lport -l -H | wc -l)" -eq 0 ]]; then
nc -l -vv -p $lport &
#do something else to process or attach different command to each diff server process
fi;
if [[ "$(ss sport = :$lport -l -H | wc -l)" -ne 0 ]]; then
watch -n 0.1 -g "ss sport = :$lport -l -H" > /dev/null;
fi;
if [[ i -eq 10 ]]; then
break;
fi;
done;
in here every time client consume a connection this script will start new listen socket.
This behavior is however can be changed in ncat (here, using -k)as you can analyze the with below example:
server is started using $ ncat -l -p 4444 -v -4 -k and 3 clients are started using $ ncat -4 localhost 4444. Now output for $ netstat -anp | grep 4444 is:
tcp 0 0 0.0.0.0:4444 0.0.0.0:* LISTEN 3596/ncat
tcp 0 0 127.0.0.1:4444 127.0.0.1:46726 ESTABLISHED 3596/ncat
tcp 0 0 127.0.0.1:46726 127.0.0.1:4444 ESTABLISHED 3602/ncat
tcp 0 0 127.0.0.1:46722 127.0.0.1:4444 ESTABLISHED 3597/ncat
tcp 0 0 127.0.0.1:4444 127.0.0.1:46724 ESTABLISHED 3596/ncat
tcp 0 0 127.0.0.1:4444 127.0.0.1:46722 ESTABLISHED 3596/ncat
tcp 0 0 127.0.0.1:46724 127.0.0.1:4444 ESTABLISHED 3601/ncat
Every time new client connect, server fork its process to attach to client, so each server process is using same pid. So output of server in this way is shared to every attached clients, however each client can send individual message to server.
You can say connections behavior is like:
SERVER_PROCESS_1 <---> CLIENT_PROCESS_1
SERVER_PROCESS_1 <---> CLIENT_PROCESS_2
SERVER_PROCESS_1 <---> CLIENT_PROCESS_3
without -k, ncat will behave same as nc.
Benefits or loses can be defined on how they are to be needed.
For this example, i used nc or nc.traditional (v1.10-41.1+b1), and ncat (7.80).
This is an incomplete answer, because I haven't got it working. Arguably more of a question, in fact. Maybe someone else can finish it off.
First of all, it seems there are different versions of netcat. I'm on Ubuntu, so I've probably got the version that came with Ubuntu. When I nc -h, it says this:
OpenBSD netcat (Debian patchlevel 1.187-1ubuntu0.1)
When I run man nc, it says this:
-F Pass the first connected socket using sendmsg(2) to stdout and exit. This
is useful in conjunction with -X to have nc perform connection setup with
a proxy but then leave the rest of the connection to another program (e.g.
ssh(1) using the ssh_config(5) ProxyUseFdpass option).
It seems to me that this means that, instead of doing the usual thing with stdin and stdout, it just prints something to stdout. That something could then be used by another process to do the actual connection to the client.
Unfortunately, -F has no effect that I can see. So maybe I'm doing it wrong. Or maybe there's some secret pipe somewhere that I have to listen to, or a supplementary argument they forgot to document. Or maybe I happen to have a broken build of netcat, and it works for everyone else who's on Ubuntu.
In combination with the -k option (or, failing that, a while-true loop), this would allow many different clients to have separate connections. Suppose you have an executable called handle_connection, which takes as arguments an in file descriptor from a client and an out file descriptor to the client, and spawns a subprocess which communicates with the client. Then the server script might look like this:
nc -lkF $host $port | while read in out ; do
handle_connection $in $out ;
done
ncat can do it, but the correct command with ncat is:
ncat --keep-open --listen -p 12345
This will accept multiple connections at the same time.
You can then send the data with multiple clients. e.g. open in two or more terminals, and try typing there:
nc localhost 12345
developer#LinuxKernel:~> sudo netstat -elnopt
developer's password:
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State User Inode PID/Program name Timer
tcp 0 0 0.0.0.0:10080 0.0.0.0:* LISTEN 1003 3061421021 - off (0.00/0/0)
Regarding my experience, I have the authority to check the PID/Program name of netstat with sudo, how to explain this one, the kernel is 2.6.16.60, net-tools 1.6, netstat 1.42
A dash "-" is given when the kernel itself is listening, consequently there are no associated processes to show. I am not sure which kernel thread is using tcp/10080 though.