How to solve too many TCP connections on FIN_WAIT_2? - linux

Server and client are connected using port 8000. Clients are aborted unexpectively but the connections are still there. Besides restarting server, any suggestion to release the legacy connections?
$ net stat -an | grep 8000
tcp4 0 0 127.0.0.1.8000 127.0.0.1.58761 CLOSE_WAIT
tcp4 0 0 127.0.0.1.58761 127.0.0.1.8000 FIN_WAIT_2
tcp4 0 0 127.0.0.1.8000 127.0.0.1.58755 CLOSE_WAIT
tcp4 0 0 127.0.0.1.58755 127.0.0.1.8000 FIN_WAIT_2
tcp46 0 0 *.8000 *.* LISTEN

Related

Linux: who is listening on tcp port 22?

I have a AST2600 evb board. After power on (w/ RJ45 connected), it boots into a OpenBMC kernel. From serial port, using ip command I can obtain its IP address. From my laptop, I can ssh into the board using account root/0penBmc:
bruin#gen81:/$ ssh root#192.168.6.132
root#192.168.6.132's password:
Then I want to find out which tcp ports are open. As there is no ss/lsof/netstat utilities, I cat /proc/net/tcp:
root#AMIfa7ba648f62e:/proc/net# cat /proc/net/tcp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000:14EB 00000000:0000 0A 00000000:00000000 00:00000000 00000000 997 0 9565 1 0c202562 100 0 0 10 0
1: 3500007F:0035 00000000:0000 0A 00000000:00000000 00:00000000 00000000 997 0 9571 1 963c8114 100 0 0 10 0
The strange thing puzzled me is that that tcp port 22 is not listed in /proc/net/tcp, which suggests that no process is listening on tcp port 22. If this is true, how the ssh connection is established?
Btw, as tested using ps, it's the dropbear process who is handling the ssh connection, and the dropbear is spawned dynamically (i.e., if no ssh connection, no such process exist; if I made two ssh connection, two dropbear processes were spawned).
PS: as suggested by John in his reply, I added the ss utilities into the image, and it shows what I expected:
root#AMI8287361b9c6f:~# ss -antp
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 0 0.0.0.0:5355 0.0.0.0:* users:(("systemd-resolve",pid=239,fd=12))
LISTEN 0 0 127.0.0.1:5900 0.0.0.0:* users:(("obmc-ikvm",pid=314,fd=5))
LISTEN 0 0 127.0.0.53:53 0.0.0.0:* users:(("systemd-resolve",pid=239,fd=17))
LISTEN 0 0 *:443 *:* users:(("bmcweb",pid=325,fd=3),("systemd",pid=1,fd=41))
LISTEN 0 0 *:5355 *:* users:(("systemd-resolve",pid=239,fd=14))
LISTEN 0 0 *:5900 *:* users:(("obmc-ikvm",pid=314,fd=6))
LISTEN 0 0 *:22 *:* users:(("systemd",pid=1,fd=49))
LISTEN 0 0 *:2200 *:* users:(("systemd",pid=1,fd=50))
ESTAB 0 0 [::ffff:192.168.6.89]:22 [::ffff:192.168.6.98]:34906 users:(("dropbear",pid=485,fd=2),("dropbear",pid=485,fd=1),("dropbear",pid=485,fd=0),("systemd",pid=1,fd=20))
Good question.
First, it is pretty straigt forward to add common tools/utitlies to an image.
It could be added (for local testing only) by adding a line
OBMC_IMAGE_EXTRA_INSTALL:append = " iproute2 iproute2-ss"
to the https://github.com/openbmc/openbmc/blob/master/meta-aspeed/conf/machine/evb-ast2600.conf file (or to your own testing/deveopment layer). Adding useful tools is often worth it.
Second, if you are using ipv6 you will need to check /proc/net/tcp6
Third, you can also look for a port by looking up the pid of your application ps | grep <application name>. Then reading the port used by that pid cat /proc/<pid>/net/tcp
Last, if you have any more question or these steps don't work. Please reach out to us on discord https://discord.com/invite/69Km47zH98 or Email https://lists.ozlabs.org/listinfo/openbmc (they are the preferred place to ask questions)

Tight VNC server and Gucamole

I have a VM in which I installed the VNC server (TightVNC) using the link : https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-vnc-on-ubuntu-18-04
It is installed successfully and I can see the port 5901 running
/etc/tigervnc$ netstat -tulpn
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:5901 0.0.0.0:* LISTEN 16460/Xtigervnc
tcp 0 0 127.0.0.1:5902 0.0.0.0:* LISTEN 16183/Xtigervnc
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN -
tcp6 0 0 ::1:5901 :::* LISTEN 16460/Xtigervnc
tcp6 0 0 ::1:5902 :::* LISTEN 16183/Xtigervnc
tcp6 0 0 :::22 :::* LISTEN -
tcp6 0 0 ::1:631 :::* LISTEN -
udp 0 0 0.0.0.0:36618 0.0.0.0:* -
udp 29184 0 127.0.0.53:53 0.0.0.0:* -
udp 0 0 0.0.0.0:68 0.0.0.0:* -
udp 0 0 0.0.0.0:631 0.0.0.0:* -
udp 7680 0 0.0.0.0:5353 0.0.0.0:* -
udp6 0 0 :::37372 :::* -
udp6 20736 0 :::5353 :::*
Now from my local machine, I tried to do the port binding to my local from VM (as per the link https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-vnc-on-ubuntu-18-04)
ssh -L 5901:127.0.0.1:5901 -C -N -l test 172.1.1.1
In my local machine, I able to see the port is binded to 5901
/etc/guacamole$ fuser 5901/tcp
5901/tcp: 22049
Now when I try to take the VNC connection using 127.0.0.1:5901, It promopts for VM's password and shows only the blank page.
Could someone help me with this?
Thanks,
Hari
edit your ~/.vnc/xstartup file thus:
#!/bin/sh
startxfce4 &
I had the same problem and this solved it
For reference i got it from here:
https://www.raspberrypi.org/forums/viewtopic.php?t=52557
You can also try killing and restarting your VNC server
kill $(pgrep Xvnc)
vncserver
Are you trying to VNC from the local machine to the local machine? I am assuming just for testing correct?
If you are not getting a rejection, at least it should be talking to the service.

nrpe : Network server bind failure (98: Address already in use)

I have installed iCinga and nrpe in same machine. I am using nrpe for monitor many linux machine, so I installed nrpe locally also.
When I start my nrep locally service nrpe start it sows error like in /var/log/messages
nrpe : Network server bind failure (98: Address already in use)
I have google it that issue, and find the 5666 port usage
[root#cosrh6-74 conf.d]# netstat -apn | grep :5666
tcp 0 0 127.0.0.1:50539 10.104.16.212:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:50608 10.104.16.212:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:41987 10.104.16.210:5666 TIME_WAIT -
tcp 0 1 127.0.0.1:42001 10.104.16.210:5666 SYN_SENT -
tcp 0 0 127.0.0.1:50576 10.104.16.212:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:41927 10.104.16.210:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:52598 10.3.81.172:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:52624 10.3.81.172:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:41962 10.104.16.210:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:41979 10.104.16.210:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:52566 10.3.81.172:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:41928 10.104.16.210:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:52569 10.3.81.172:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:41955 10.104.16.210:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:52587 10.3.81.172:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:50586 10.104.16.212:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:50547 10.104.16.212:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:52588 10.3.81.172:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:50609 10.104.16.212:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:50567 10.104.16.212:5666 TIME_WAIT -
tcp 0 0 127.0.0.1:52592 10.3.81.172:5666 TIME_WAIT -
tcp 0 0 :::5666 :::* LISTEN 757/xinetd
I I have changed /etc/nagios/nrpe.cfg port to 56666 from 5666.
How can I configure different port in host configuration(different port for different host) in icinga2 server to monitor machines with nrpe running in different ports?
Is this right to change port? Or any other way to do this? Please correct me if I did anything wrong?
In each host definition add:
vars.nrpe_port = <host_nrpe_port>
Ref: docs.icinga.org
Added port in command.conf file like this,
object CheckCommand "check-nrpe" {
import "plugin-check-command"
command = ["/usr/local/nagios/libexec/check_nrpe"]
"-p" ="56666"
"-H" ="$host$"
"-c" = "$nrpe_command$"
"-a" = $nrpe_arguments$"
}
"-p" ="56666" Works for me!!
EDIT:
Or we can pass like arguments from host configuration ( keeping port number in host configuration like #7171u answer) .

telnet refused on specific port on AWS instances

I'm tryign to telnet from one linux env (10.205.116.141) to 10.205.117.246 on port 7199 but keep getting a connection refused. I did a chkconfig iptables off on both servers and even make sure iptables if stopped as well.
what else should I be looking at?
[root#ip-10-205-116-141 bin]# telnet 10.205.117.246 7199
Trying 10.205.117.246...
telnet: connect to address 10.205.117.246: Connection refused
trace route seems to be working as well...
[root#ip-10-205-116-141 bin]# traceroute 10.205.117.246 -p 7199
traceroute to 10.205.117.246 (10.205.117.246), 30 hops max, 60 byte packets
1 ip-10-205-117-246.xyz.cxcvs.com (10.205.117.246) 0.416 ms 0.440 ms 0.444 ms
also, I'm on a aws vpc so we don't get public IPs provisioned for use...
checked my security group and it looks like all ports are open as well
EDIT:
here is netstat as well, they look the same on both nodes:
[ec2-user#ip-10-205-116-141 ~]$ netstat -an | grep LISTEN
tcp 0 0 127.0.0.1:46626 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:9160 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:36523 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:9042 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:2738 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
tcp 0 0 10.205.116.141:7000 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:8089 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:4445 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:7199 0.0.0.0:* LISTEN
shouldn't 127.0.0.1:7199 really be 10.205.116.141:7199?
sorry, can't post a sc of the security groups...

Weird TCP connection on Oracle Linux

On Oracle Linux "Linux bjzv0880 3.8.13-16.2.1.el6uek.x86_64 #1 SMP Thu Nov 7 17:01:44 PST 2013 x86_64 x86_64 x86_64 GNU/Linux"
I have 1 TCP Server and 2 TCP Clients running with the connection status below:
******[root]# netstat -anp | grep 58000
tcp 0 0 192.168.250.102:58000 0.0.0.0:* LISTEN 3614/AppServer
tcp 0 0 192.168.250.102:44500 192.168.250.102:58000 ESTABLISHED 3673/AppClient1
tcp 0 0 192.168.250.102:44488 192.168.250.102:58000 ESTABLISHED 3631/AppClient2
tcp 0 0 192.168.250.102:58000 192.168.250.102:44500 ESTABLISHED 3614/AppServer
tcp 0 0 192.168.250.102:58000 192.168.250.102:44488 ESTABLISHED 3614/AppServer******
Then I forcefully stop the AppServer without cleaning up the socket. And make the AppClient* to try to reconnect to AppServer very quickly. After a little moment, I got a weird connection:
*[root]# netstat -anp | grep 58000
tcp 0 0 192.168.250.102:58000 192.168.250.102:58000 ESTABLISHED 3673/AppClient1*
Note: I have done a wireshark capture on the tcp communication, and from the traffic log
1. there are 2 rounds of retries to connect from source port selected by OS
2. In the 1st round, 58000 was not selected by OS
3. but in the 2nd round, 58000 was selected and it happened to be able to established
How could it be possible? Appreciated for your advice.

Resources