Counting TCP retransmissions - linux

I would like to know if there is a way to count the number of TCP retransmissions that occurred in a flow, in LINUX. Either on the client side or the server side.

Looks like netstat -s solves my purpose.

You can see TCP retransmissions for a single TCP flow using Wireshark. The "follow TCP stream" filter will allow you to see a single TCP stream. And the tcp.analysis.retransmission one will show retransmissions.
For more details, this serverfault question may be useful: https://serverfault.com/questions/318909/how-passively-monitor-for-tcp-packet-loss-linux

The Linux kernel provides an interface through the pseudo-filesystem proc for counters to track the TCPSynRetrans
For example:
awk '$1 ~ "Tcp:" { print $13 }' /proc/net/snmp
Per documentation:
* TCPSynRetrans
This counter is explained by `kernel commit f19c29e3e391`_, I pasted the
explanation below::
--
TCPSynRetrans: number of SYN and SYN/ACK retransmits to break down
retransmissions into SYN, fast-retransmits, timeout retransmits, etc.
You can also adjust these settings also through the pseudo-filesystem procfs but under the sys directory. There is a handy utility that does this short-hand for you.
sysctl -a | grep retrans
net.ipv4.neigh.default.retrans_time_ms = 1000
net.ipv4.neigh.docker0.retrans_time_ms = 1000
net.ipv4.neigh.enp1s0.retrans_time_ms = 1000
net.ipv4.neigh.lo.retrans_time_ms = 1000
net.ipv4.neigh.wlp6s0.retrans_time_ms = 1000
net.ipv4.tcp_early_retrans = 3
net.ipv4.tcp_retrans_collapse = 1
net.ipv6.neigh.default.retrans_time_ms = 1000
net.ipv6.neigh.docker0.retrans_time_ms = 1000
net.ipv6.neigh.enp1s0.retrans_time_ms = 1000
net.ipv6.neigh.lo.retrans_time_ms = 1000
net.ipv6.neigh.wlp6s0.retrans_time_ms = 1000
net.netfilter.nf_conntrack_tcp_max_retrans = 3
net.netfilter.nf_conntrack_tcp_timeout_max_retrans = 300

Related

ACK packets forged issues: "This frame is a (suspected) retransmission"

I'm playing with scapy. I'm trying to forge JUST PSH/ACK and ACK packets in sequence
I coded two tools: A which sends PSH/ACK packets and then sniffs the resulting ACK, writing the sequence in a file to use it later
.....
bitack = random.randrange(1,656787969)
bitseq = random.randrange(1,4294967295)
if os.path.exists('test.txt'):
with open('test.txt','r') as f:
bitseq = int(f.read())
else:
with open('test.txt','w') as f:
f.write(str(bitseq))
.....
text = "Ok"
TSval = int(time.time())
TSecr = TSval
acker = IP(src="127.0.0.1",dst=destinazione"127.0.0.1")/TCP(sport=88,dport=8888,
flags="PA", seq=bitseq, ack=bitack, options=[('Timestamp', (TSval, TSecr))])/text
send(acker)
.....
rx = sniff(filter="host 127.0.0.1 and src port 8888", iface="lo", count=1)
seqcc = rx[0].getlayer(TCP).seq
ackcc = rx[0].getlayer(TCP).ack
with open('test.txt','w') as f:
f.write(str(ackcc))
print("SEQFINALE=", ackcc)
B: which sends ACK packets AFTER it sniffs a PSH/ACK packet from A. I know the ack packets contain text ( in this example the same of A), but this is what I want
....
rx = sniff(filter="host 127.0.0.1 and dst port 8888", iface="lo", count=1)
seqcc = rx[0].getlayer(TCP).seq
print("seq:", seqcc)
ackcc = rx[0].getlayer(TCP).ack
print("ack:", ackcc)
var = rx[0][Raw].load.decode(encoding='utf-8', errors='ignore')
acker = IP(src="127.0.0.1",dst="127.0.0.1")/TCP(sport=8888,dport=88, flags="A",
seq=ackcc, ack=seqcc + int(len(var)), options=[('Timestamp', (TSval, TSecr))])/var
send(acker)
.....
Everything works fine expect that wireshark gives some warning and I don't understand why:
"Expert Info (Note/Sequence): This frame is a (suspected) retransmission"
The first two packets are perfect:
Is there any issue in how I handle the sequence number/ ack number?
This makes me crazy
It is a retransmission. Your capture shows a frame from 8888 to 88 at seq=1 with 52 bytes of data (len=52). If you ever send another frame from 8888 to 88 at seq=1, it's a retransmission. TCP streams are in a single direction: A sends to B, B ACK's what A sent. (in this case, there should be an ACK=53 in a frame from 88 to 8888, either alone or piggybacking data.)

SOLVED - Debug Darkice to understand why is not connecting to shoutcast

I'm trying to connect to a shoutcast server from a darkice client using Ubuntu. This is my configuration:
#this section describes general aspects of the live streaming session
[general]
duration = 0 # duration of encoding, in seconds. 0 means forever
bufferSecs = 10 # size of internal slip buffer, in seconds
reconnect = yes # reconnect to the server(s) if disconnected
realtime = no # run the encoder with POSIX realtime priority
rtprio = 3 # scheduling priority for the realtime threads
# this section describes the audio input that will be streamed
[input]
device = hw:CARD=PCH,DEV=0
sampleRate = 44100 # sample rate in Hz. try 11025, 22050 or 44100
bitsPerSample = 16 # bits per sample. try 16
channel = 2 # channels. 1 = mono, 2 = stereo
# this section describes a streaming connection to an IceCast2 server
# there may be up to 8 of these sections, named [icecast2-0] ... [icecast2-7]
# these can be mixed with [icecast-x] and [shoutcast-x] sections
[shoutcast-0]
bitrateMode = cbr
format = mp3
bitrate = 96
quality = 1.0
server = xxxxxxxxxxxxxxx
port = 8020
password = xxxxxxxxxxxxxxx
name = Radio website
url = https://www.mywebsite.it
genre = live
public = no
But when I run
darkice -v 10 -c /etc/darkice-shoutcast.cfg
It only shows this, without errors or similar, but there is no streaming at the url. Using BUTT it works. I've also tested with 8021 instead of 8020 for port (8020 it's the port number given by the provider) but no luck.
DarkIce 1.4 live audio streamer, http://code.google.com/p/darkice/
Copyright (c) 2000-2007, Tyrell Hungary, http://tyrell.hu/
Copyright (c) 2008-2013, Akos Maroy and Rafael Diniz
This is free software, and you are welcome to redistribute it
under the terms of The GNU General Public License version 3 or
any later version.
Using config file: /etc/darkice-shoutcast.cfg
18-May-2021 12:02:28 Using ALSA DSP input device: hw:CARD=PCH,DEV=0
18-May-2021 12:02:28 buffer size: 1764000
18-May-2021 12:02:28 encoding
18-May-2021 12:02:28 MultiThreadedConnector :: transfer, bytes 0
18-May-2021 12:02:28 MultiThreadedConnector :: ThreadData :: threadFunction, was (thread, priority, type): 0x5568a502c010 0 SCHED_OTHER
18-May-2021 12:02:28 MultiThreadedConnector :: ThreadData :: threadFunction, now is (thread, priority, type): 0x5568a502c010 0 SCHED_OTHER
ADDENDUM
I've used tcpdump to understand what could be and I just see something similar to "invalid password"
: Flags [P.], cksum 0xc379 (correct), seq 1:19, ack 3090, win 294, options [nop,nop,TS val 3348978428 ecr 531576376], length 18
E..F1O#.1.3'.}.......T.Jq.k.D......&.y.....
..Z...68Invalid Passwor
Suggestions on how to better debug or fix this?
SOLVED
It seems the error is related to the password and wrong parsing of the config file, so I've written it without spaces
[shoutcast-0]
bitrateMode = cbr
format = mp3
bitrate = 96
quality = 1.0
server = xxxxxxxxxxxxxxx
port = 8020
password=xxxxxxxxxxxxxxx
name = Radio website
url = https://www.mywebsite.it
genre = live
public = no

Mysql seconds_behind master very high

Hi we have mysql master slave replication, master is mysql 5.6 and slave is mysql 5.7, seconds behind master is 245000, how I make it catch up faster. Right now it is taking more than 6 hours to copy 100 000 seconds.
My slave ram is 128 GB. Below is my my.cnf
[mysqld]
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
innodb_buffer_pool_size = 110G
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
# These are commonly set, remove the # and set as required.
basedir = /usr/local/mysql
datadir = /disk1/mysqldata
port = 3306
#server_id = 3
socket = /var/run/mysqld/mysqld.sock
user=mysql
log_error = /var/log/mysql/error.log
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
join_buffer_size = 256M
sort_buffer_size = 128M
read_rnd_buffer_size = 2M
#copied from old config
#key_buffer = 16M
max_allowed_packet = 256M
thread_stack = 192K
thread_cache_size = 8
query_cache_limit = 1M
#disabling query_cache_size and type, for replication purpose, need to enable it when going live
query_cache_size = 0
#query_cache_size = 64M
#query_cache_type = 1
query_cache_type = OFF
#GroupBy
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
#sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
enforce-gtid-consistency
gtid-mode = ON
log_slave_updates=0
slave_transaction_retries = 100
#replication related changes
server-id = 2
relay-log = /disk1/mysqllog/mysql-relay-bin.log
log_bin = /disk1/mysqllog/binlog/mysql-bin.log
binlog_do_db = brandmanagement
#replicate_wild_do_table=brandmanagement.%
replicate-wild-ignore-table=brandmanagement.t\_gnip\_data\_recent
replicate-wild-ignore-table=brandmanagement.t\_gnip\_data
replicate-wild-ignore-table=brandmanagement.t\_fb\_rt\_data
replicate-wild-ignore-table=brandmanagement.t\_keyword\_tweets
replicate-wild-ignore-table=brandmanagement.t\_gnip\_data\_old
replicate-wild-ignore-table=brandmanagement.t\_gnip\_data\_new
binlog_format=row
report-host=10.125.133.220
report-port=3306
#sync-master-info=1
read-only=1
net_read_timeout = 7200
net_write_timeout = 7200
innodb_flush_log_at_trx_commit = 2
sync_binlog=0
sync_relay_log_info=0
max_relay_log_size=268435456
Lots of possible solutions. But I'll go with the simplest one. Have you got enough network bandwidth to send all changes over the network? You're using "row" binlog, which may be good in case of random, unindexed updates. But if you're changing a lot of data using indexes only, then "mixed" binlog may be better.

Force lshosts command to return megabytes for "maxmem" and "maxswp" parameters

When I type "lshosts" I am given:
HOST_NAME type model cpuf ncpus maxmem maxswp server RESOURCES
server1 X86_64 Intel_EM 60.0 12 191.9G 159.7G Yes ()
server2 X86_64 Intel_EM 60.0 12 191.9G 191.2G Yes ()
server3 X86_64 Intel_EM 60.0 12 191.9G 191.2G Yes ()
I am trying to return maxmem and maxswp as megabytes, not gigabytes when lshosts is called. I am trying to send Xilinx ISE jobs to my LSF, however the software expects integer, megabyte values for maxmem and maxswp. By doing debugging, it appears that the software grabs these parameters using the lshosts command.
I have already checked in my lsf.conf file that:
LSF_UNIT_FOR_LIMTS=MB
I have tried searching the IBM Knowledge Base, but to no avail.
Do you use a specific command to specify maxmem and maxswp units within the lsf.conf, lsf.shared, or other config files?
Or does LSF force return the most practical unit?
Any way to override this?
LSF_UNIT_FOR_LIMITS should work, if you completely drained the cluster of all running, pending, and finished jobs. According to the docs, MB is the default, so I'm surprised.
That said, you can use something like this to transform the results:
$ cat to_mb.awk
function to_mb(s) {
e = index("KMG", substr(s, length(s)))
m = substr(s, 0, length(s) - 1)
return m * 10^((e-2) * 3)
}
{ print $1 " " to_mb($6) " " to_mb($7) }
$ lshosts | tail -n +2 | awk -f to_mb.awk
server1 191900 159700
server2 191900 191200
server3 191900 191200
The to_mb function should also handle 'K' or 'M' units, should those pop up.
If LSF_UNIT_FOR_LIMITS is defined in lsf.conf, lshosts will always print the output as a floating point number, and in some versions of LSF the parameter is defined as 'KB' in lsf.conf upon installation.
Try searching for any definitions of the parameter in lsf.conf and commenting them all out so that the parameter is left undefined, I think in that case it defaults to printing it out as an integer in megabytes.
(Don't ask me why it works this way)

Compare different item in two file and output combined result to new file by using AWK

Greeting!
I have some file in pair taken from two nodes in network, and file has records about TCP segment send/receive time, IP id number, segment type,seq number and so on.
For same TCP flow, it looks like this on sender side:
1420862364.778332 50369 seq 17400:18848
1420862364.780798 50370 seq 18848:20296
1420862364.780810 50371 seq 20296:21744
....
or on receiver side(1 second delay, segment with IP id 50371 lost)
1420862364.778332 50369 seq 17400:18848
1420862364.780798 50370 seq 18848:20296
....
I want to compare IP identification number in two file and output to new one like this:
1420862364.778332 1420862365.778332 50369 seq 17400:18848 o
1420862364.780798 1420862365.780798 50370 seq 18848:20296 o
1420862364.780810 1420862365.780810 50371 seq 20296:21744 x
which has time of arrive on receiver side, and by comparing id field, when same value is not found in receiver sid(packet loss), an x will be added, otherwise o will be there.
I already have code like this,
awk 'ARGIND==1 {w[$2]=$1}
ARGIND==2 {
flag=0;
for(a in w)
if($2==a) {
flag=1;
print $1,w[a],$2,$3,$4;
break;
}
if(!flag)
print $1,"x",$2,$3,$4;
}' file2 file1 >file3
but it doesn't work in Linux, it stops right after I pressed Enter, and leave only empty file.
Shell script contains these code has been through chomd +x.
Please help. My code is not well organized, any new one liner will be appreciated.
Thank you for your time.
ARGIND is gawk-specific btw so check your awk version. – Ed Morton

Resources