I am trying to transfer an image using TCP sockets using linux. I have used the code many times to transfer small amounts but as soon as I tried to transfer the image it only transfered the first third. Is it possible that there is a maximum buffer size for tcp sockets in linux? If so how can I increase it? Is there a function that does this programatically?
I would guess that the problem is on the receiving side when you read from the socket. TCP is a stream based protocol with no idea of packets or message boundaries.
This means when you do a read you may get less bytes than you request. If your image is 128k for example you may only get 24k on your first read requiring you to read again to get the rest of the data. The fact that it's an image is irrelevant. Data is data.
For example:
int read_image(int sock, int size, unsigned char *buf) {
int bytes_read = 0, len = 0;
while (bytes_read < size && ((len = recv(sock, buf + bytes_read,size-bytes_read, 0)) > 0)) {
bytes_read += len;
}
if (len == 0 || len < 0) doerror();
return bytes_read;
}
TCP sends the data in pieces, so you're not guaranteed to get it all at once with a single read (although it's guaranteed to stay in the order you send it). You basically have to read multiple times until you get all the data. It also doesn't know how much data you sent on the receiver side. Normally, you send a fixed size "length" field first (always 8 bytes, for example) so you know how much data there is. Then you keep reading and building a buffer until you get that many bytes.
So the sender would look something like this (pseudocode)
int imageLength;
char *imageData;
// set imageLength and imageData
send(&imageLength, sizeof(int));
send(imageData, imageLength);
And the receiver would look like this (pseudocode)
int imageLength;
char *imageData;
guaranteed_read(&imageLength, sizeof(int));
imageData = new char[imageLength];
guaranteed_read(imageData, imageLength);
void guaranteed_read(char* destBuf, int length)
{
int totalRead=0, numRead;
while(totalRead < length)
{
int remaining = length - totalRead;
numRead = read(&destBuf[totalRead], remaining);
if(numRead > 0)
{
totalRead += numRead;
}
else
{
// error reading from socket
}
}
}
Obviously I left off the actual socket descriptor and you need to add a lot of error checking to all of that. It wasn't meant to be complete, more to show the idea.
The maximum size for 1 single IP packet is 65535, which is extremely close to the number you are hitting. I doubt that is a coincidence.
Related
I'm writing a netfilter module, that deeply inspect the packet. However, during tests I found that netfilter module is not receiving the packet in full.
To verify this, I wrote the following code to dump packet retrieved on port 80 and write the result to dmesg buffer:
const struct iphdr *ip_header = ip_hdr(skb);
if (ip_header->protocol == IPPROTO_TCP)
{
const struct tcphdr *tcp_header = tcp_hdr(skb);
if (ntohs(tcp_header->dest) != 80)
{
return NF_ACCEPT;
}
buff = (char *)kzalloc(skb->len * 10, GFP_KERNEL);
if (buff != NULL)
{
int pos = 0, i = 0;
for (i = 0; i < skb->len; i ++)
{
pos += sprintf(buff + pos, "%02X", skb->data[i] & 0xFF);
}
pr_info("(%pI4):%d --> (%pI4):%d, len=%d, data=%s\n",
&ip_header->saddr,
ntohs(tcp_header->source),
&ip_header->daddr,
ntohs(tcp_header->dest),
skb->len,
buff
);
kfree (buff);
}
}
In virtual machine running locally, I can retrieve the full HTTP request; On Alibaba cloud, and some other OpenStack based VPS provider, the packet is cut in the middle.
To verify this, I execute curl http://VPS_IP on another VPS, and I got the following output in dmesg buffer:
[ 1163.370483] (XXXX):5007 --> (XXXX):80, len=237, data=451600ED000040003106E3983D87A950AC11D273138F00505A468086B44CE19E80180804269300000101080A1D07500A000D2D90474554202F20485454502F312E310D0A486F73743A2033392E3130372E32342E37370D0A4163636570743A202A2F2A0D0A557365722D4167656E743A204D012000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000001E798090F5FFFF8C0000007B00000000E0678090F5FFFF823000003E00000040AE798090F5FFFF8C0000003E000000000000000000000000000000000000000000000000000000000000
When decoded, the result is like this
It's totally weird, everything after User-Agent: M is "gone" or zero-ed. Although the skb->len is 237, but half of the packet is missing.
Any ideas? Tried both PRE_ROUTING and LOCAL_IN, no changes.
It appears that sometimes you are getting a linear skb, and sometimes your skb is not linear. In the latter case you are not reading the full data contents of an skb.
If skb->data_len is zero, then your skb is linear and the full data contents of the skb is in skb->data. If skb->data_len is not zero, then your skb is not linear, and skb->data contains just the the first (linear) part of the data. The length of this area is skb->len - skb->data_len. skb_headlen() helper function calculates that for convenience. skb_is_nonlinear() helper function tells in an skb is linear or not.
The rest of the data can be in paged fragments, and in skb fragments, in this order.
skb_shinfo(skb)->nr_frags tells the number of paged fragments. Each paged fragment is described by a data structure in the array of structures skb_shinfo(skb)->frags[0..skb_shinfo(skb)->nr_frags]. skb_frag_size() and skb_frag_address() helper functions help dealing with this data. They accept the address of the structure that describes a paged fragment. There are other useful helper functions depending on your kernel version.
If the total size of data in paged fragments is less than skb->data_len, then the rest of the data is in skb fragments. It's the list of skb which is attached to this skb at skb_shinfo(skb)->frag_list (see skb_walk_frags() in the kernel).
Please note that there may be that there's no data in the linear part and/or there's no data in the paged fragments. You just need to process data piece by piece in the order just described.
I am writing firmware for a data logging device. It reads data from sensors at 20 Hz and writes data to an SD card. However, the time to write data to SD card is not consistent (about 200-300 ms). Thus one solution is writing data to a buffer at a consistent rate (using a timer interrupt), and have a second thread that writes data to the SD card when the buffer is full.
Here is my naive implementation:
#define N 64
char buffer[N];
int count;
ISR() {
if (count < N) {
char a = analogRead(A0);
buffer[count] = a;
count = count + 1;
}
}
void loop() {
if (count == N) {
myFile.open("data.csv", FILE_WRITE);
int i = 0;
for (i = 0; i < N; i++) {
myFile.print(buffer[i]);
}
myFile.close();
count = 0;
}
}
The code has the following problems:
Writing data to the SD card is blocking reading when the buffer is full
It might have a race conditions.
What is the best way to solve this problem? Using a circular buffer, or double buffering? How do I ensure that a race condition does not happen?
You have rather answered your own question; you should use either double buffering or a circular buffer. Double-buffering is probably simpler to implement and appropriate for devices such as an SD card for which block-writes are generally more efficient.
Buffer length selection may need some consideration; generally you would make the buffer the same as the SD sector buffer size (typically 512 bytes), but that may not be practical, and with a sample rate as low as 20 sps, optimising SD write performance is perhaps not an issue.
Another consideration is that you need to match your sample rate to the file-system latency by selecting an appropriate buffer size. In this case the 64 sample buffer buffer will fill in a little more than three seconds, but the block write takes only up-to 300 ms - so you could use a much smaller buffer if required - 8 samples would be sufficient - although be careful, you may have observed latency of 300 ms, but it may be larger when specific boundaries are crossed in the physical flash memory - I have seen significant latency on some cards at 1 Mbyte boundaries - moreover card performance varies significantly between sizes and manufacturers.
An adaptation of your implementation with double-buffering is below. I have reduced the buffer length to 32 samples, but with double-buffering the total is unchanged at 64, but the write lag is reduced to 1.6 seconds.
// Double buffer and its management data/constants
static volatile char buffer[2][32];
static const int BUFFLEN = sizeof(buffer[0]);
static const unsigned char EMPTY = 0xff;
static volatile unsigned char inbuffer = 0;
static volatile unsigned char outbuffer = EMPTY;
ISR()
{
static int count = 0;
// Write to current buffer
char a = analogRead(A0);
buffer[inbuffer][count] = a;
count++ ;
// If buffer full...
if( count >= BUFFLEN )
{
// Signal to loop() that data available (not EMPTY)
outbuffer = inbuffer;
// Toggle input buffer
inbuffer = inbuffer == 0 ? 1 : 0;
count = 0;
}
}
void loop()
{
// If buffer available...
if( outbuffer != EMPTY )
{
// Write buffer
myFile.open("data.csv", FILE_WRITE);
for( int i = 0; i < BUFFLEN; i++)
{
myFile.print(buffer[outbuffer][i]);
}
myFile.close();
// Set the buffer to empty
outbuffer = EMPTY;
}
}
Note the use of volatile and unsigned char for the shared data. It is important that data shared between concurrent execution contexts is accessed explicitly and atomically; access to an int on 8-bit AVR based Arduino requires multiple machine instructions and the interrupt may occur part way through a read/write in loop() and cause an incorrect value to be read.
I'm bit of a newbie but I have an legacy app that reads 64 bytes of AES encrypted data from a device using ttyACM0. I now need to read 128 bytes. Sounded simple; increase the sizes of buffers etc. But no matter what I try, I still can only read 64 bytes. After that it just hangs. I verified the communications in Windows with a terminal and cdc-acm driver. Device does not use flow control. I cant upload code because its proprietary but below are some snippets:
The Intialization:
CACS_RefID::Initialise()
{
int iRet = 1;
struct termios dev_settings;
if(( m_fdRefdev = open("/dev/ttyACM0", O_RDWR))<0)
{
g_dbg->debug("CACS_RefID::Failed to open device\n");
return 0;
}
g_dbg->debug("CACS_RefID::Initialse completed\n");
// Configure the port
tcgetattr(m_fdRefdev, &dev_settings);
cfmakeraw(&dev_settings);
//*tcflush
//tcflush(m_fdRefdev, TCIOFLUSH);
tcsetattr(m_fdRefdev, TCSANOW, &dev_settings);
return iRet;
}
The implementation:
int CACS_RefID::Readport_Refid(int ilen, char* buf)
{
int ierr=0, iret = 0, ictr=0;
fd_set fdrefid;
struct timeval porttime_refrd;
FD_ZERO(&fdrefid);
FD_SET(m_fdRefdev,&fdrefid);
porttime_refrd.tv_sec = 1;
porttime_refrd.tv_usec = 0; //10 Seconds wait time for read port
do
{
iret = select(m_fdRefdev + 1, &fdrefid, NULL, NULL, &porttime_refrd);
switch(iret)
{
case READ_TIMEOUT:
g_dbg->debug("Refid portread: Select timeout:readlen=%d \n",ilen);
ierr = -1;
break;
case READ_ERROR:
g_dbg->debug("Refid portread: Select error:readlen=%d \n",ilen);
ierr = -1;
break;
default:
iret = read(m_fdRefdev, buf, ilen);
g_dbg->debug("Refid portread: Read len(%d):%d\n",ilen,iret);
break;
}
}while((ierr == 0) && (iret<ilen) );
//Flush terminal content at Input and Output after every read completion
// tcflush(m_fdRefdev, TCIOFLUSH);
return ierr;
}
If I initialize every time that I before running the implementation, I get 128 bytes but the data is corrupt after 64 bytes. Even before working on it, I get a lot of READ_ERRORs. Looks like the original author expected the device to block with select() but it doesn't.
Is there some type of limitation on ttyACM0 buffer size in the system? Does baud rate matter with the ttyACM driver? Does read() stop reading after all bytes are read (thinking the first 64 are available, then empty, then more data)?
Pouring thru man pages but I'm stymied. ANY help would be greatly appreciated.
Heres my latest:
int CACS_RefID::Get_GasTest_Result(int ilen)
{
int ierr=0, iret = 0, ictr=0, iread=0;
fd_set fdrefid;
struct timeval porttime_refrd;
porttime_refrd.tv_sec = 5;
porttime_refrd.tv_usec = 0; //10 Seconds wait time for read port
if (Get_GasTest_FirstPass == 0)
{
g_dbg->debug("GasTest_Result_firstPass\n");
memset(strresult, 0, sizeof(strresult)); //SLY clear out result buffer
iread=0;
Get_GasTest_FirstPass = 1;
}
do
{
iread = strlen(strresult);
FD_ZERO(&fdrefid);
FD_SET(m_fdRefdev,&fdrefid);
iret = select(m_fdRefdev + 1, &fdrefid, NULL, NULL, &porttime_refrd);
switch(iret)
{
case READ_TIMEOUT: //0
g_dbg->debug("Get_GasTest_Result: Select timeout\n");
ierr = -1;
break;
case READ_ERROR: //-1
g_dbg->debug("Get_GasTest_Result: Select error=%d %s \n", errno,strerror(errno)) ;
ierr = -1;
break;
}
iret = read(m_fdRefdev, (&strresult[0] + iread), (ilen-iread));
g_dbg->debug("Get_GasTest_Result: ilen=%d,iret=%d,iread=%d \n",ilen,iret,iread);
}while((ierr == 0) && (iread<ilen) );
return ierr;
Note: I am now reading data regardless of select errors and STILL only getting 64bytes. I've contacted my device mfg. Must be something odd going on.
Here is one possible problem with your code; this may not be the one that is causing you to only get 64 bytes but it could explain what you are seeing. Assume that you invoke the function Readport_Refid() with a buffer of 128 bytes. In other words, your invocation was something like:
char buffer[128];
Readport_Refid(128, buffer);
Assume for whatever reason that the first call to select() gets you a return value of 1 (since one bit is set). Your code is only setting one bit so you go off and you read()
iret = read(m_fdRefdev, buf, ilen);
g_dbg->debug("Refid portread: Read len(%d):%d\n",ilen,iret);
break;
iret returns 64 (which means 64 bytes are read) and your program prints a nice message and since ierr is still 0 and iret (64) is less than ilen (128) you go round again and call select().
Assume that you get more data and select() returns 1 again. Then you will go read again on the same buffer with the same ilen and overwrite the first 64 bytes that were read.
At the very least, you should do the following. I have only shown below the changed lines. First add an iread variable and make sure you use it to preserve data that you've already read. Then use iread to determine whether you've read enough or not.
int CACS_RefID::Readport_Refid(int ilen, char* buf)
{
int ierr=0, iret = 0, ictr=0, iread = 0;
[...]
default:
iret = read(m_fdRefdev, buf + iread, ilen - iread);
if (iret > 0)
iread += iret;
g_dbg->debug("Refid portread: Read len(%d):%d\n",ilen,iret);
break;
}
}while((ierr == 0) && (iread<ilen) );
[...]
**** EDITED 2013-08-19 ****
I want to reiterate a comment made by #wildplasser
You should really also be setting FD_SET on each trip around the loop. Great catch.
With respect to your new code, does it work or do you still have a problem?
**** EDITED again 2013-08-19 ****
Getting EINTR is nothing to be worried about. You should just plan on resetting FD_SET and trying again.
I can't say I know why but the fix was to call the initialization code at the beginning of the implementation even though it is called previously. If I call it again, I can read in 128 bytes. If I don't, I can only read up to 64 bytes.
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
I have two program communicating with TCP/IP.
this works fine.
program A opens a TCP socket.
program B connects to this socket.
program B gives data to program A.
everything works very fine.
but when i put a relay between A and B
and just pass the byte stream, something goes wrong.
program C opens two TCP socket(socket1, socket2).
program B connects to this socket(socket1).
program A connects to this socket(socket2).
program C relays TCP strem from B to A with following code.
(this is based on linux)
char buf[BUFSIZE];
while(1){
// recv a packet segment
if( my_recv(socket1, buf, BUFSIZE) <= 0 ){
return 0;
}
if( send(socket2, buf, BUFSIZE, MSG_NOSIGNAL) != BUFSIZE ){
return 0;
}
}
my_recv is a wrapper for recv to guarantee it will recv
request size.
int my_recv(int sd, char* p, unsigned int len){
// recv a packet segment
unsigned int ssize=0;
int d;
while(ssize < len){
if( (d=recv( sd, p+ssize, len - ssize, 0))<=0){
return -1;
}
ssize += d;
}
return ssize;
}
this works fine for first time.
but after few seconds, everything is messed up.
I have debugged program A and B.
there is nothing wrong with them.
they send and recv in correct order but it seems
that relay gives false data...
some advise would be appreciated.
thank you in advance.
You are assuming each read fills the buffer. You have to use the count returned by the recv() call as the length argument to the send() call.
while ((count = recv(fd, buffer, sizeof buffer, 0)) > 0)
send(fd, buffer, count, 0);
I have an application which runs on Linux (2.6.38.8), using libpcap (>1.0) to capture packets streamed at it over Ethernet. My application uses close to 100% CPU and I am unsure whether I am using libpcap as efficiently as possible.
I am battling to find any correlation between the pcap tunables and performace.
Here is my simplified code (error checking etc. omitted):
// init libpcap
pcap_t *p = pcap_create("eth0", my_errbuf);
pcap_set_snaplen(p, 65535);
pcap_set_promisc(p, 0);
pcap_set_timeout(p, 1000);
pcap_set_buffer_size(p, 16<<20); // 16MB
pcap_activate(p);
// filter
struct bpf_program filter;
pcap_compile(p, &filter, "ether dst 00:11:22:33:44:55", 0, 0);
pcap_setfilter(p, &filter);
// do work
while (1) {
int ret = pcap_dispatch(p, -1, my_callback, (unsigned char *) my_args);
if (ret <= 0) {
if (ret == -1) {
printf("pcap_dispatch error: %s\n", pcap_geterr(p));
} else if (ret == -2) {
printf("pcap_dispatch broken loop\n");
} else if (ret == 0) {
printf("pcap_dispatch zero packets read\n");
} else {
printf("pcap_dispatch returned unexpectedly");
}
} else if (ret > 1) {
printf("processed %d packets\n", ret);
}
}
The result when using a timeout of 1000 miliseconds, and buffer size of 2M, 4M and 16M is the same at high data rates (~200 1kB packets/sec): pcap_dispatch consistently returns 2. According to the pcap_dispatch man page, I would expect pcap_dispatch to return either when the buffer is full or the timeout expires. But with a return value of 2, neither of these conditions should be met as only 2kB of data has been read, and only 2/200 seconds have passed.
If I slow down the datarate (~100 1kB packets/sec), pcap_dispatch returns between 2 and 7, so halving the datarate affects how many packets are processed per pcap_dispatch. (I think the more packets the better, as this means less context switching between OS and userspace - is this true?)
The timeout value does not seem to make a difference either.
In all cases, my CPU usage is close to 100%.
I am starting to wonder if I should be trying the PF_RING version of libpcap, but from what I've read on SO and libpcap mailing lists, libpcap > 1.0 does the zero copy stuff anyway, so maybe no point.
Any ideas, pointers greatly appreciated!
G