Reduce TCP maximum segment size (MSS) in Linux on a socket - linux

In a special application in which our server needs to update firmware of low-on-resource sensor/tracking devices we encountered a problem in which sometimes data is lost in the
remote devices (clients) receiving packets of the new firmware. The connection is TCP/IP over
GPRS network. The devices use SIM900 GSM chip as a network interface.
The problems possibly come because of the device receiving too much data. We tried reducing the
traffic by sending packages more rarely but sometimes the error still occured.
We contacted the local retailer of the SIM900 chip who is also responsible for giving technical support and possibly contacting the chinese manufacturer (simcom) of the chip. They said that at first we should try to reduce the TCP MSS (Maximum Segment Size) of our connection.
In our server I did the following:
static int
create_master_socket(unsigned short master_port) {
static struct sockaddr_in master_address;
int master_socket = socket(AF_INET,SOCK_STREAM,0);
if(!master_socket) {
perror("socket");
throw runtime_error("Failed to create master socket.");
}
int tr=1;
if(setsockopt(master_socket,SOL_SOCKET,SO_REUSEADDR,&tr,sizeof(int))==-1) {
perror("setsockopt");
throw runtime_error("Failed to set SO_REUSEADDR on master socket");
}
master_address.sin_family = AF_INET;
master_address.sin_addr.s_addr = INADDR_ANY;
master_address.sin_port = htons(master_port);
uint16_t tcp_maxseg;
socklen_t tcp_maxseg_len = sizeof(tcp_maxseg);
if(getsockopt(master_socket, IPPROTO_TCP, TCP_MAXSEG, &tcp_maxseg, &tcp_maxseg_len)) {
log_error << "Failed to get TCP_MAXSEG for master socket. Reason: " << errno;
perror("getsockopt");
} else {
log_info << "TCP_MAXSEG: " << tcp_maxseg;
}
tcp_maxseg = 256;
if(setsockopt(master_socket, IPPROTO_TCP, TCP_MAXSEG, &tcp_maxseg, tcp_maxseg_len)) {
log_error << "Failed to set TCP_MAXSEG for master socket. Reason: " << errno;
perror("setsockopt");
} else {
log_info << "TCP_MAXSEG: " << tcp_maxseg;
}
if(getsockopt(master_socket, IPPROTO_TCP, TCP_MAXSEG, &tcp_maxseg, &tcp_maxseg_len)) {
log_error << "Failed to get TCP_MAXSEG for master socket. Reason: " << errno;
perror("getsockopt");
} else {
log_info << "TCP_MAXSEG: " << tcp_maxseg;
}
if(bind(master_socket, (struct sockaddr*)&master_address,
sizeof(master_address))) {
perror("bind");
close(master_socket);
throw runtime_error("Failed to bind master_socket to port");
}
return master_socket;
}
Running the above code results in:
I0807 ... main.cpp:267] TCP_MAXSEG: 536
E0807 ... main.cpp:271] Failed to set TCP_MAXSEG for master socket. Reason: 22 setsockopt: Invalid argument
I0807 ... main.cpp:280] TCP_MAXSEG: 536
As you may see, the problem in the second line of the output: setsockopt returns "Invalid argument".
Why does this happen? I read about some constraints in setting TCP_MAXSEG but I did not encounter any report on such a behaviour as this.
Thanks,
Dennis

In addition to xaxxon's answer, just wanted to note my experience with trying to force my Linux to send only maximum TCP segments of a certain size (lower than what they normally are):
The easiest way I found to do so, was to use iptables:
sudo iptables -A INPUT -p tcp --tcp-flags SYN,RST SYN --destination 1.1.1.1 -j TCPMSS --set-mss 200
This overwrites the remote incoming SYN/ACK packet on an outbound connection, and forces the MSS to a specific value.
Note1: You do not see this in wireshark, since wireshark capture before this happens.
Note 2: Iptables does not allow you to -increase- the MSS, just lower it
Alternatively, I also tried setting the socket option TCP_MAXSEG, like dennis had done. After taking the fix from xaxxon, this also worked.
Note: You should read the MSS value after the connection has been set up. Otherwise it returns the default value, which put me (and dennis) on the wrong track.
Now finally, I also ran into a number of other things:
I ran into TCP-offloading issues, where despite my MSS being set correctly, the frames being sent were still shown by wireshark as too big. You can disable this feature by : sudo ethtool -K eth0 tx off sg off tso off. This took me a long time to figure out.
TCP has lots of fancy things like MTU path discovery, which actually try to dynamically increase the MSS. Fun and cool, but confusing obviously. I did not have issues with it though in my tests
Hope this helps someone trying to do the same thing one day.

Unless otherwise noted, optval is a pointer to an int.
but you're using a u_int16. I don't see anything saying that this parameter isn't an int.
edit: Yeah, here is the source code and you can see:
637 if (optlen < sizeof(int))
638 return -EINVAL;

Related

Qt 5.5 not binding to port

I have some code which worked fine in Qt 5.9 under Windows but is causing issues with Qt 5.5 under Linux.
It's basically contacting an echo server to detect whether a device is on the network and it starts by setting up a UDP sender and receiver, after which it sends a packet and checks that it comes back.
The problematic bit seems to be in binding the receiving socket:
// Create the two sockets.
m_sendSocket = new QUdpSocket(this);
m_recvSocket = new QUdpSocket(this);
// Connect sending socket to other end.
m_sendSocket->connectToHost(QHostAddress(host), 43837);
// Bind receiving socket so it will receive any response.
bool x = m_recvSocket->bind(m_sendSocket->localPort());
std::cout << x << " " << m_sendSocket->localPort() << std::endl;
It creates the two sockets (outgoing and incoming) then attempts to bind the outgoing socket's local port to the incoming socket's receiving port (so the response can be picked up).
This works fine under the Windows environment but the bind is failing under Linux. Since all it returns is a boolean value, I'm at a loss as to why it's failing.
Thinking it may be the default binding mode for the platform, I also tried explicitly setting that so that Linux would match Windows:
bool x = m_recvSocket->bind(m_sendSocket->localPort(), QAbstractSocket::ShareAddress);
But that didn't help.
The local port for the sender socket seems okay, I've seen values like 38349, 58597, and 37433.
Does anyone know why this might be failing, or whether Qt makes more information available somewhere as to why it's failing, or whether there's a better way to do this (that won't fail)?
You can get further information from the socket by accessing m_recvSocket->errorString(). In your case, modify the cout line to be:
std::cout << x
<< " " << m_sendSocket->localPort()
<< " " << m_recvSocket->errorString().toStdString()
<< std::endl;
What you'll see is something along the lines of "address already in use" because, as you realise, Windows and Linux have different default binding modes - the former is a more sharing OS whereas the latter appears to be vindictive and possessive (a).
So the sending socket has effectively locked up the local port it's using when calling connectToHost(), meaning the receiving socket cannot bind to it.
The way to solve this is to have the sending socket bind explicitly with the sharing option before connecting to the host. This can be done with:
m_sendSocket = new QUdpSocket(this);
m_recvSocket = new QUdpSocket(this);
m_sendSocket->bind(0, QAbstractSocket::ShareAddress);
m_sendSocket->connectToHost(QHostAddress(host), 43837);
bool x = m_recvSocket->bind(m_sendSocket->localPort(), QAbstractSocket::ShareAddress);
std::cout << x
<< " " << m_sendSocket->localPort()
<< " " << m_recvSocket->errorString().toStdString()
<< std::endl;
That results in the output:
1 47334 Unknown error
on my system, the error text can be ignored since the return value of bind was true (the initial 1 on the output line).
(a) Pax ducks for cover :-)

tcp keepalive - Protocol not available?

I'm trying to set tcp keepalive but in doing so I'm seeing the error
"Protocol not available"
int rc = setsockopt(s, SOL_SOCKET, TCP_KEEPIDLE, &keepalive_idle, sizeof(keepalive_idle));
if (rc < 0)
printf("error setting keepalive_idle: %s\n", strerror(errno));
I'm able to turn on keepalive, set keepalive interval and count but keepalive idle which is keepalive time is throwing that error and I never see any keepalive packets being transmitted/received either with wireshark and the filter tcp.analysis.keep_alive or with tcpdump
sudo tcpdump -vv "tcp[tcpflags] == tcp-ack and less 1"
Is there a kernel module that needs to be loaded or something? Or are you no longer able to override the global KEEPIDLE time.
By the way the output of
matt#devpc:~/ sysctl net.ipv4.tcp_keepalive_time net.ipv4.tcp_keepalive_probes net.ipv4.tcp_keepalive_intvl
net.ipv4.tcp_keepalive_time = 7200
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_intvl = 75
In an application that I coded, the following works:
setsockopt(*sfd, SOL_SOCKET, SO_KEEPALIVE,(char *)&enable_keepalive, sizeof(enable_keepalive));
setsockopt(*sfd, IPPROTO_TCP, TCP_KEEPCNT, (char *)&num_keepalive_strobes, sizeof(num_keepalive_strobes));
setsockopt(*sfd, IPPROTO_TCP, TCP_KEEPIDLE, (char *)&keepalive_idle_time_secs, sizeof(keepalive_idle_time_secs));
setsockopt(*sfd, IPPROTO_TCP, TCP_KEEPINTVL, (char *)&keepalive_strobe_interval_secs, sizeof(keepalive_strobe_interval_secs));
Try changing SOL_SOCKET to IPPROTO_TCP for TCPKEEPIDLE.
There a very handy lib that can help you, it's called libkeepalive : http://libkeepalive.sourceforge.net/
It can be used with LD_PRELOAD in order to enable and control keep-alive on all TCP sockets. You can also override keep-alive settings with environment variables.
I tried to run a tcp server with it:
KEEPIDLE=5 KEEPINTVL=5 KEEPCNT=100 LD_PRELOAD=/usr/lib/libkeepalive.so nc -l -p 4242
Then I connected a client:
nc 127.0.0.1 4242
And I visualized the traffic with Wireshark: the keep-alive packets began exactly after 5 seconds of inactivity (my system wide setting is 75). Therefore it means that it's possible to override the system settings.
Here is how libkeepalive sets TCP_KEEPIDLE:
if((env = getenv("KEEPIDLE")) && ((optval = atoi(env)) >= 0)) {
setsockopt(s, SOL_TCP, TCP_KEEPIDLE, &optval, sizeof(optval));
}
Looks like they use SOL_TCP instead of SOL_SOCKET.

Linux Serial Port Blocked using termios.h configuration

I'm writing an embedded Linux application that (1) opens a serial connection to another device, (2) sends a known command, (3) checks port for incoming characters (response) until expected response phrase or character is detected, (4) repeats step 2 and 3 until a series of commands are sent and responses received, (5) then closes the port.
My app would go through some cycles of the above sequence and every now and then it would be waiting for a response (reading) when all of a sudden the communication stops and my software faults out because of my built-in timeout logic.
Is there anything in my port configuration that would cause the port to be blocked due to specific byte sent (possibly due to electrical noise)?
Here is how I'm opening my ports (showing configurations via termios.h):
struct termios options;
fd = open("/dev/ttyUSB0", O_RDWR | O_NOCTTY | O_NONBLOCK);
if (fd == -1) {
debug() << "Port open failed!"
return FAIL;
}
debug() << "Port Opened Successful"
fcntl(fd, F_SETFL, 0); // This setting interacts with VMIN and VTIME below
// Get options
tcgetattr(fd, &options);
// Adjust Com port options
options.c_cflag |= (CLOCAL | CREAD); // Program will not "own" port, enable reading on port
options.c_lflag &= ~(ICANON | ECHO | ECHOE | ISIG); // Sets RAW input mode (does not treat input as a line of text with CR/LF ending)
options.c_oflag &= ~ OPOST; // Sets RAW ouput mode (avoids newline mapping to CR+LF characters)
options.c_iflag &= ~(IXON | IXOFF | IXANY); // Turns off SW flow c
options.c_cc[VMIN] = 0;
options.c_cc[VTIME] = 10;
// Set options
tcsetattr(fd, TCSANOW, &options);
//return fd;
return SUCCEED;
I can't figure out why the communication all of a sudden just freezes up and then goes away when I cycle power to my device. Thanks all!
More info - here are my read and write functions:
int Comm::Receive(unsigned char* rBuf)
{
int bytes;
ioctl(fd, FIONREAD, &bytes);
if (bytes >= 1)
{
bytes = read(fd, rBuf, 1);
if (bytes < 0)
return READ_ERR;
return SUCCEED;
}
else
return NO_DATA_AVAILABLE;
}
int Comm::Send(int xCt, unsigned char* xBuf)
{
int bytes;
if (fd == -1)
return FAIL;
bytes = write(fd, xBuf, xCt);
if (bytes != xCt)
return FAIL;
else
return SUCCEED;
}
Welcome to the joys of serial ports...
Thought 1: wrap your read calls with a select ()
Thought 2: Unset the ICANON flag in tcsetattr, and set a VTIME attribute for a deliberate timeout (and then, obviously, handle it)
Thought 3: Nothing about serial comms ever works perfectly.
I also had a similar problem with sending command to devices and reading responses from the device. Please refer to below SOF post and the answer this was working for my problem.
In these cases, We have to care about the protocol which we are going to use for device communication (send and receive). If we can send commands successfully and we didn't receive a response with a noise from device implies there is something wrong in the data packet which was sent to the device. First of all check the protocol specification first and create a byte array for a simple command (like make a beep sound) and send it.
Send data to a barcode scanner over RS232 serial port
I can do something for you If you can post your complete source code with the output.
Enjoy the code. Thanks.

recv with flags MSG_DONTWAIT | MSG_PEEK on TCP socket

I have a TCP stream connection used to exchange messages. This is inside Linux kernel. The consumer thread keeps processing incoming messages. After consuming one message, I want to check if there are more pending messages; in which case I would process them too. My code to achieve this looks like below. krecv is wrapper for sock_recvmsg(), passing value of flags without modification (krecv from ksocket kernel module)
With MSG_DONTWAIT, I am expecting it should not block, but apparently it blocks. With MSG_PEEK, if there is no data to be read, it should just return zero. Is this understanding correct ? Is there a better way to achieve what I need here ? I am guessing this should be a common requirement as message passing across nodes is used frequently.
int recvd = 0;
do {
recvd += krecv(*sockp, (uchar*)msg + recvd, sizeof(my_msg) - recvd, 0);
printk("recvd = %d / %lu\n", recvd, sizeof(my_msg));
} while(recvd < sizeof(my_msg));
BUG_ON(recvd != sizeof(my_msg));
/* For some reason, below line _blocks_ even with no blocking flags */
recvd = krecv(*sockp, (uchar*)tempbuf, sizeof(tempbuf), MSG_PEEK | MSG_DONTWAIT);
if (recvd) {
printk("more data waiting to be read");
more_to_process = true;
} else {
printk("NO more data waiting to be read");
}
You might check buffer's length first :
int bytesAv = 0;
ioctl(m_Socket,FIONREAD,&bytesAv); //m_Socket is the socket client's fd
If there are data in it , then recv with MSG_PEEK should not be blocked ,
If there are no data at all , then no need to MSG_PEEK ,
that might be what you like to do .
This is a very-very old question, but
1. problem persits
2. I faced with it.
At least for me (Ubuntu 19.04 with python 2.7) this MSG_DONTWAIT has no effect, however if I set the timeout to zero (with settimeout function), it works nicely.
This can be done in c with setsockopt function.

Crafting an ICMP packet inside a Linux kernel Module

I'm tring to experiment with the ICMP protocol and have created a kernel-module for linux that analyses ICMP packet ( Processes the packet only if if the ICMP code field is a magic number ) . Now to test this module , i have to create a an ICMP packet and send it to the host where this analysing module is running . In fact it would be nice if i could implement it the kernel itself (as a module ) . I am looking for something like a packetcrafter in kernel , I googled it found a lot of articles explaining the lifetime of a packet , rather than tutorials of creating it . User space packetcrafters would be my last resort, that too those which are highly flexible like where i'll be able to set ICMP code etc . And I'm not wary of kernel panics :-) !!!!! Any packet crafting ideas are welcome .
Sir, I strongly advice you against using the kernel module to build ICMP packets.
You can use user-space raw-sockets to craft ICMP packets, even build the IP-header itself byte by byte.
So you can get as flexible as it can get using that.
Please, take a look at this
ip = (struct iphdr*) packet;
icmp = (struct icmphdr*) (packet + sizeof(struct iphdr));
/*
* here the ip packet is set up except checksum
*/
ip->ihl = 5;
ip->version = 4;
ip->tos = 0;
ip->tot_len = sizeof(struct iphdr) + sizeof(struct icmphdr);
ip->id = htons(random());
ip->ttl = 255;
ip->protocol = IPPROTO_ICMP;
ip->saddr = inet_addr(src_addr);
ip->daddr = inet_addr(dst_addr);
if ((sockfd = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP)) == -1)
{
perror("socket");
exit(EXIT_FAILURE);
}
/*
* IP_HDRINCL must be set on the socket so that
* the kernel does not attempt to automatically add
* a default ip header to the packet
*/
setsockopt(sockfd, IPPROTO_IP, IP_HDRINCL, &optval, sizeof(int));
/*
* here the icmp packet is created
* also the ip checksum is generated
*/
icmp->type = ICMP_ECHO;
icmp->code = 0;
icmp->un.echo.id = 0;
icmp->un.echo.sequence = 0;
icmp->checksum = 0;
icmp-> checksum = in_cksum((unsigned short *)icmp, sizeof(struct icmphdr));
ip->check = in_cksum((unsigned short *)ip, sizeof(struct iphdr));
If this part of code looks flexible enough, then read about raw sockets :D maybe they're the easiest and safest answer to your need.
Please check the following links for further info
http://courses.cs.vt.edu/~cs4254/fall04/slides/raw_6.pdf
http://www.cs.binghamton.edu/~steflik/cs455/rawip.txt
http://cboard.cprogramming.com/networking-device-communication/107801-linux-raw-socket-programming.html a very nice topic, pretty useful imo
You can try libcrafter for packet crafting on user space. Is very easy to use! The library is able to craft or decode packets of most common networks protocols, send them on the wire, capture them and match requests and replies.
For example, the next code craft and send an ICMP packet:
string MyIP = GetMyIP("eth0");
/* Create an IP header */
IP ip_header;
/* Set the Source and Destination IP address */
ip_header.SetSourceIP(MyIP);
ip_header.SetDestinationIP("1.2.3.4");
/* Create an ICMP header */
ICMP icmp_header;
icmp_header.SetType(ICMP::EchoRequest);
icmp_header.SetIdentifier(RNG16());
/* Create a packet... */
Packet packet = ip_header / icmp_header;
packet.Send();
Why you want to craft an ICMP packet on kernel-space? Just for fun? :-p
Linux kernel includes a packet generator tool pktgen for testing the network with pre-configured packets. Source code for this module resides in net/core/pktgen.c

Resources