Fetching the TCP RTT in Linux - linux

I need to fetch the RTT for TCP flow.
I have looked into the proc file system but not able to get the RTT value of TCP .If any one having any idea regarding it that, in which file RTT would be stored pleae share.
Thanks in advance.

Maybe the ss (socket statistics) util available in the iproute utils can help you with this.
# ss -i 'src 1.1.1.1:1234 and dst 2.2.2.2:1234'
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 1.1.1.1:1234 2.2.2.2:1234
reno wscale:2,7 rto:3380 rtt:855/602.5 ato:40 ssthresh:2 send 27.3Kbps rcv_space:5840
If you want more information what the rtt field is i think it is best to take a look at ss.c.

You can so this using tcpprobe (inserts a hook into the tcp_recv processing path using kprobe and records the state of a TCP connection in response to incoming packets).
Explained here: Extract TCP round trip time (RTT) estimations on linux

It also possible to print the cached rtts (and rttvar, cwnd) for previous destinations using the ip command:
sudo ip tcp_metrics

Related

Extract TCP round trip time (RTT) estimations on linux

I have apache server running on Ubuntu. Client connects and downloads an image. I need to extract RTT estimations for the underlying TCP connection. Is there a way to do this? Maybe something like running my tcp stack in debug mode to have it log this info somewhere?
Note that I don't want to run tcpdump and extract RTTs from the recorded trace! I need the TCP stack's RTT estimations (apparently this is part of the info you can get with TCP_INFO socket option). Basically need something like tcpprob (kprobe) to insert a hook and record the estimated RTT of the TCP connection on every incoming packet (or on every change).
UPDATE:
I found a solution. rtt, congestion window and more can be logged using tcpprobe. I posted an answer below.
This can be done without the need for any additional kernel modules using the ss command (part of the iproute package), which can provide detailed info on open sockets. It won't show it for every packet but most of this info is calculated over a number of packets. E.g. To list the currently open TCP (t option) sockets and associated internal TCP info (i) information - including congestion control algorithm, rtt, cwnd etc:
ss -ti
Here's some example output:
State Recv-Q Send-Q Local Address:Port Peer Address:Port
ESTAB 0 0 192.168.56.102:ssh 192.168.56.1:46327
cubic wscale:6,7 rto:201 rtt:0.242/0.055 ato:40 mss:1448 rcvmss:1392
advmss:1448 cwnd:10 bytes_acked:33169 bytes_received:6069 segs_out:134
segs_in:214 send 478.7Mbps lastsnd:5 lastrcv:6 lastack:5
pacing_rate 955.4Mbps rcv_rtt:3 rcv_space:28960
This can be done using tcpprobe, which is a module that inserts a hook into the tcp_recv processing path using kprobe records the state of a TCP connection in response to incoming packets.
Let's say you want to probe tcp connection on port 443, you need to do the following:
sudo modprobe tcp_probe port=443 full=1
sudo chmod 444 /proc/net/tcpprobe
cat /proc/net/tcpprobe > /tmp/output.out &
pid=$!
full=1: log on every ack packet received
full=0: log on only condo changes (if you use this your output might be empty)
Now pid is the process which is logging the probe. To stop, simply kill this process:
kill $pid
The format of output.out (according to the source at line 198):
[time][src][dst][length][snd_nxt][snd_una][snd_cwnd][ssthresh][snd_wnd][srtt][rcv_wnd]

measuring TX retransmission amount per interface

i'm trying to get the TX re-transmission amount for a specific network interface on a Linux 2.6.33.3 kernel.
is there any Linux raw data i can use in order to get/calculate this kind of information?
i've tried netstat, but i couldn't find a way to get the retransmissions value for a specific interface.
with no other "quick" choice, what would it require from me in order to build a small app especially for that? listen to all outgoing packets using any capturing library and count.. what? duplicate packets? by some kind of a packet id?
i'm not a network pro.. am i suppose to find troubles in this kind of mission? (btw, i'll probably try to do it with python. any reason why not to?)
You can check how your interface is configured with the following commands:
mii-tool eth0
ethtool eth0
mii-diag eth0
Not all of them are installed by default in any Linux system, but they are definitely in the distributive and installable.
When you want to measure, how fast is your interface really, you can use dd + netcat:
host1$ cat /dev/zero | nc -l -p 3000
host2$ nc host1 3000 | dd of=/dev/null
^C
dd will write you with which speed was the retransmission between host1 and host2.
If you want per interface, try ifconfig eth0, you probably have what you want there. Just parse it through python.

How to automate measuring of bandwidth usage between two hosts

I have an application that has a TCP client and a server. I set up the client and server on separate machines. Now I want to measure how much bandwidth is being consumed ( bytes sent and received during a single run of the application). I have discovered that wireshark is one such tool that can help me get this statistic. However, wireshark seems to be GUI dependent. What I wanted was a way to automate the measuring and reporting of this statistic. I dont care about the information about individual packets captured by wireshark. I dont need that information. Is there some way to run wireshark so that all it does is write to a file, the total bytes sent and received between two hosts while the application was running on both ends?
Also, is there a better way to capture this statistic ? Through netstat or /proc/dev/net or any other tool ?
Both my machines have ubuntu 10.04 or later running on them.
Bro is an appropriate tool to measure connection-oriented statistics. You can either record a trace of your application communication or analyze it in realtime:
bro -r <trace>
bro -i <interface>
Thereafter, have a look at the connection log (conn.log) in the same directory for the amount of bytes sent and received by the application. Specifically, you're interested in the TCP payload size, which conn.log exposes via the columns orig_bytes and resp_bytes. Here is an example:
bro-cut id.orig_h id.resp_h conn_state orig_bytes resp_bytes < conn.log | head
which yields the following output:
192.168.1.102 192.168.1.1 SF 301 300
192.168.1.103 192.168.1.255 S0 350 0
192.168.1.102 192.168.1.255 S0 350 0
192.168.1.103 192.168.1.255 S0 560 0
192.168.1.102 192.168.1.255 S0 348 0
192.168.1.104 192.168.1.255 S0 350 0
192.168.1.104 192.168.1.255 S0 549 0
192.168.1.103 192.168.1.1 SF 303 300
192.168.1.102 192.168.1.255 S0 - -
192.168.1.104 192.168.1.1 SF 311 300
Each row represents a single connection, transport-layer ports omitted. The last two columns represent the bytes sent by the originator (first column) and responder (second column). The column conn_state represents the connection status. Please refer to the documentation for all possible field values. Some important values are:
S0: Connection attempt seen, no reply.
S1: Connection established, not terminated.
SF: Normal establishment and termination. Note that this is the same symbol as for state S1. You can tell the two apart because for S1 there will not be any byte counts in the summary, while for SF there will be.
REJ: Connection attempt rejected.

How to programmatically increase the per-socket buffer for UDP sockets on LInux?

I'm trying to understand the correct way to increase the socket buffer size on Linux for our streaming network application. The application receives variable bitrate data streamed to it on a number of UDP sockets. The volume of data is substantially higher at the start of the stream and I've used:
# sar -n UDP 1 200
to show that the UDP stack is discarding packets and
# ss -un -pa
to show that each socket Recv-Q length grows to the nearly the limit (124928. from sysctl net.core.rmem_default) before packets are discarded. This implies that the application simply can't keep up with the start of the stream. After discarding enough initial packets the data rate slows down and the application catches up. Recv-Q trends towards 0 and remains there for the duration.
I'm able to address the packet loss by substantially increasing the rmem_default value which increases the socket buffer size and gives the application time to recover from the large initial bursts. My understanding is that this changes the default allocation for all sockets on the system. I'd rather just increase the allocation for the specific UDP sockets and not modify the global default.
My initial strategy was to modify rmem_max and to use setsockopt(SO_RCVBUF) on each individual socket. However, this question makes me concerned about disabling Linux autotuning for all sockets and not just UDP.
udp(7) describes the udp_mem setting but I'm confused how these values interact with the rmem_default and rmem_max values. The language it uses is "all sockets", so my suspicion is that these settings apply to the complete UDP stack and not individual UDP sockets.
Is udp_rmem_min the setting I'm looking for? It seems to apply to individual sockets but global to all UDP sockets on the system.
Is there a way to safely increase the socket buffer length for the specific UDP ports used in my application without modifying any global settings?
Thanks.
Jim Gettys is armed and coming for you. Don't go to sleep.
The solution to network packet floods is almost never to increase buffering. Why is your protocol's queueing strategy not backing off? Why can't you just use TCP if you're trying to send so much data in a stream (which is what TCP was designed for).

How do I get min/avg/var rtt for all TCP connections in Linux?

I'm trying to implement a software that tracks open TCP connections and classify them based on TCP round trip time estimates, on Linux. I'm looking for similar information that the program nettop shows on MacOS X.
$ nettop -m tcp
It shows a list of open connections by the process that owns them. It includes the current round trip time min, mean and variance estimates for each connection.
For listening to a program's own connections it could be done something like in http://linuxgazette.net/136/pfeiffer.html but I'm looking for something like nettop that shows the information for all connections on the machine. On OS X that does not require root access but it is fine if the answer does.
I'd prefer a Python compatible version but if not available, I can live with C. If there is an existing command-line utility like nettop for Linux, that's also interesting.
Related:
Wikipedia: Karn's algorithm
Some of this information is available in the command:
ss -i -t
If you want to do this with your own code you can look at the output of libpcap or tcpdump and compare the timestamp on packets with corresponding sequence and ack numbers and average those out for the last few seconds.
12:19:39.331248 IP 10.0.60.243.ssh > 192.168.50.22.21950: P 11952:12180(228)
12:19:39.331388 IP 10.0.60.243.ssh > 192.168.50.22.21950: P 12328:12476(148)
12:19:39.380981 IP 192.168.50.22.21950 > 10.0.60.243.ssh: . ack 11952 win 65535
12:19:39.381039 IP 10.0.60.243.ssh > 192.168.50.22.21950: P 12624:12772(148)
12:19:39.381054 IP 192.168.50.22.21950 > 10.0.60.243.ssh: . ack 12328 win 65159
12:19:39.381058 IP 192.168.50.22.21950 > 10.0.60.243.ssh: . ack 12624 win 64863
This would average out to about 50ms rtt

Resources