I'm bit of a newbie but I have an legacy app that reads 64 bytes of AES encrypted data from a device using ttyACM0. I now need to read 128 bytes. Sounded simple; increase the sizes of buffers etc. But no matter what I try, I still can only read 64 bytes. After that it just hangs. I verified the communications in Windows with a terminal and cdc-acm driver. Device does not use flow control. I cant upload code because its proprietary but below are some snippets:
The Intialization:
CACS_RefID::Initialise()
{
int iRet = 1;
struct termios dev_settings;
if(( m_fdRefdev = open("/dev/ttyACM0", O_RDWR))<0)
{
g_dbg->debug("CACS_RefID::Failed to open device\n");
return 0;
}
g_dbg->debug("CACS_RefID::Initialse completed\n");
// Configure the port
tcgetattr(m_fdRefdev, &dev_settings);
cfmakeraw(&dev_settings);
//*tcflush
//tcflush(m_fdRefdev, TCIOFLUSH);
tcsetattr(m_fdRefdev, TCSANOW, &dev_settings);
return iRet;
}
The implementation:
int CACS_RefID::Readport_Refid(int ilen, char* buf)
{
int ierr=0, iret = 0, ictr=0;
fd_set fdrefid;
struct timeval porttime_refrd;
FD_ZERO(&fdrefid);
FD_SET(m_fdRefdev,&fdrefid);
porttime_refrd.tv_sec = 1;
porttime_refrd.tv_usec = 0; //10 Seconds wait time for read port
do
{
iret = select(m_fdRefdev + 1, &fdrefid, NULL, NULL, &porttime_refrd);
switch(iret)
{
case READ_TIMEOUT:
g_dbg->debug("Refid portread: Select timeout:readlen=%d \n",ilen);
ierr = -1;
break;
case READ_ERROR:
g_dbg->debug("Refid portread: Select error:readlen=%d \n",ilen);
ierr = -1;
break;
default:
iret = read(m_fdRefdev, buf, ilen);
g_dbg->debug("Refid portread: Read len(%d):%d\n",ilen,iret);
break;
}
}while((ierr == 0) && (iret<ilen) );
//Flush terminal content at Input and Output after every read completion
// tcflush(m_fdRefdev, TCIOFLUSH);
return ierr;
}
If I initialize every time that I before running the implementation, I get 128 bytes but the data is corrupt after 64 bytes. Even before working on it, I get a lot of READ_ERRORs. Looks like the original author expected the device to block with select() but it doesn't.
Is there some type of limitation on ttyACM0 buffer size in the system? Does baud rate matter with the ttyACM driver? Does read() stop reading after all bytes are read (thinking the first 64 are available, then empty, then more data)?
Pouring thru man pages but I'm stymied. ANY help would be greatly appreciated.
Heres my latest:
int CACS_RefID::Get_GasTest_Result(int ilen)
{
int ierr=0, iret = 0, ictr=0, iread=0;
fd_set fdrefid;
struct timeval porttime_refrd;
porttime_refrd.tv_sec = 5;
porttime_refrd.tv_usec = 0; //10 Seconds wait time for read port
if (Get_GasTest_FirstPass == 0)
{
g_dbg->debug("GasTest_Result_firstPass\n");
memset(strresult, 0, sizeof(strresult)); //SLY clear out result buffer
iread=0;
Get_GasTest_FirstPass = 1;
}
do
{
iread = strlen(strresult);
FD_ZERO(&fdrefid);
FD_SET(m_fdRefdev,&fdrefid);
iret = select(m_fdRefdev + 1, &fdrefid, NULL, NULL, &porttime_refrd);
switch(iret)
{
case READ_TIMEOUT: //0
g_dbg->debug("Get_GasTest_Result: Select timeout\n");
ierr = -1;
break;
case READ_ERROR: //-1
g_dbg->debug("Get_GasTest_Result: Select error=%d %s \n", errno,strerror(errno)) ;
ierr = -1;
break;
}
iret = read(m_fdRefdev, (&strresult[0] + iread), (ilen-iread));
g_dbg->debug("Get_GasTest_Result: ilen=%d,iret=%d,iread=%d \n",ilen,iret,iread);
}while((ierr == 0) && (iread<ilen) );
return ierr;
Note: I am now reading data regardless of select errors and STILL only getting 64bytes. I've contacted my device mfg. Must be something odd going on.
Here is one possible problem with your code; this may not be the one that is causing you to only get 64 bytes but it could explain what you are seeing. Assume that you invoke the function Readport_Refid() with a buffer of 128 bytes. In other words, your invocation was something like:
char buffer[128];
Readport_Refid(128, buffer);
Assume for whatever reason that the first call to select() gets you a return value of 1 (since one bit is set). Your code is only setting one bit so you go off and you read()
iret = read(m_fdRefdev, buf, ilen);
g_dbg->debug("Refid portread: Read len(%d):%d\n",ilen,iret);
break;
iret returns 64 (which means 64 bytes are read) and your program prints a nice message and since ierr is still 0 and iret (64) is less than ilen (128) you go round again and call select().
Assume that you get more data and select() returns 1 again. Then you will go read again on the same buffer with the same ilen and overwrite the first 64 bytes that were read.
At the very least, you should do the following. I have only shown below the changed lines. First add an iread variable and make sure you use it to preserve data that you've already read. Then use iread to determine whether you've read enough or not.
int CACS_RefID::Readport_Refid(int ilen, char* buf)
{
int ierr=0, iret = 0, ictr=0, iread = 0;
[...]
default:
iret = read(m_fdRefdev, buf + iread, ilen - iread);
if (iret > 0)
iread += iret;
g_dbg->debug("Refid portread: Read len(%d):%d\n",ilen,iret);
break;
}
}while((ierr == 0) && (iread<ilen) );
[...]
**** EDITED 2013-08-19 ****
I want to reiterate a comment made by #wildplasser
You should really also be setting FD_SET on each trip around the loop. Great catch.
With respect to your new code, does it work or do you still have a problem?
**** EDITED again 2013-08-19 ****
Getting EINTR is nothing to be worried about. You should just plan on resetting FD_SET and trying again.
I can't say I know why but the fix was to call the initialization code at the beginning of the implementation even though it is called previously. If I call it again, I can read in 128 bytes. If I don't, I can only read up to 64 bytes.
Related
Context
I wrote a Linux device driver in which the functions read and write are implemented. The problem is with the function write, here the portion of the code:
ssize_t LED_01_write(struct file *filp, const char __user *buf, size_t count, loff_t *f_pos)
{
int retval = 0;
PDEBUG(" reading from user space -> wrinting in kernel space\n");
//struct hello_dev *dev = filp->private_data;
if (count > COMMAND_MAX_LENGHT){
printk(KERN_WARNING "[LEO] LED_01: trying to write more than possible. Aborting write\n");
retval = -EFBIG;
goto out;
}
if (down_interruptible(&(LED_01_devices->sem_LED_01))){
printk(KERN_WARNING "[LEO] LED_01: Device was busy. Operation aborted\n");
return -ERESTARTSYS;
}
if (copy_from_user((void*)&(LED_01_devices-> LED_value), buf, count)) {
printk(KERN_WARNING "[LEO] LED_01: can't use copy_from_user. \n");
retval = -EPERM;
goto out_and_Vsem;
}
write_status_to_LED();
PDEBUG(" Value instert: %u \n", LED_01_devices-> LED_value);
out_and_Vsem:
write_times++;
up(&(LED_01_devices->sem_LED_01));
out:
return retval;
}
Question
If I use the module in a C compiled program, it works properly, as expected.
When I execute echo -n 1 > /dev/LED_01 (from the Command LINE), it writes INFINITE times and, even with the Ctrl+C it doesn't stop. I need to reboot.
Here the snipped code of the test function that works properly:
// ON
result = write(fd, (void*) ON_VALUE, 1);
if ( result != 0 ){
printf("Oh dear, something went wrong with write()! %s\n", strerror(errno));
}
else{
printf("write operation executed succesfully (%u)\n",ON_VALUE[0]);
}
Is the problem in the driver or in the way I use echo?
If you need to whole source code, all the file used are stored in this git repository folder
Value returned by the kernel's .write function is interpreted as:
error code, if it is less than zero (<0),
number of bytes written, if it is more than or equal to zero (>=0)
So, for tell user that all bytes has been written, .write function should return its count parameter.
In case of .write function, returning zero has a little sense: every "standard" utility like echo will just call write() function again.
Consider the following code:
file_fd = open(device, O_RDWR);
if (file_fd < 0) {
perror("open");
return -1;
}
printf("File descriptor: %d\n", file_fd);
uint32_t DskSize;
if (ioctl(file_fd, BLKGETSIZE, &DskSize) < 0) {
perror("ioctl");
return -1;
}
printf("File descriptor after: %d\n", file_fd);
This snippet yields this:
File descriptor: 3
File descriptor after: 0
Why does my file descriptor get reset to 0? The program writes the stuff out to stdout instead of my block device.
This should not happen. I expect my file_fd to be non-zero and retain its value.
Looks like you smash your stack.
Since there are only two stack variables file_fd and DskSize and changing DskSize changes file_fd suggests that DiskSize must be unsigned long or size_t (a 64-bit value), not uint32_t.
Looking at BLKGETSIZE implementation confirms that the value type is unsigned long.
You may like to run your applications under valgrind, it reports this kind of errors.
I have an application which runs on Linux (2.6.38.8), using libpcap (>1.0) to capture packets streamed at it over Ethernet. My application uses close to 100% CPU and I am unsure whether I am using libpcap as efficiently as possible.
I am battling to find any correlation between the pcap tunables and performace.
Here is my simplified code (error checking etc. omitted):
// init libpcap
pcap_t *p = pcap_create("eth0", my_errbuf);
pcap_set_snaplen(p, 65535);
pcap_set_promisc(p, 0);
pcap_set_timeout(p, 1000);
pcap_set_buffer_size(p, 16<<20); // 16MB
pcap_activate(p);
// filter
struct bpf_program filter;
pcap_compile(p, &filter, "ether dst 00:11:22:33:44:55", 0, 0);
pcap_setfilter(p, &filter);
// do work
while (1) {
int ret = pcap_dispatch(p, -1, my_callback, (unsigned char *) my_args);
if (ret <= 0) {
if (ret == -1) {
printf("pcap_dispatch error: %s\n", pcap_geterr(p));
} else if (ret == -2) {
printf("pcap_dispatch broken loop\n");
} else if (ret == 0) {
printf("pcap_dispatch zero packets read\n");
} else {
printf("pcap_dispatch returned unexpectedly");
}
} else if (ret > 1) {
printf("processed %d packets\n", ret);
}
}
The result when using a timeout of 1000 miliseconds, and buffer size of 2M, 4M and 16M is the same at high data rates (~200 1kB packets/sec): pcap_dispatch consistently returns 2. According to the pcap_dispatch man page, I would expect pcap_dispatch to return either when the buffer is full or the timeout expires. But with a return value of 2, neither of these conditions should be met as only 2kB of data has been read, and only 2/200 seconds have passed.
If I slow down the datarate (~100 1kB packets/sec), pcap_dispatch returns between 2 and 7, so halving the datarate affects how many packets are processed per pcap_dispatch. (I think the more packets the better, as this means less context switching between OS and userspace - is this true?)
The timeout value does not seem to make a difference either.
In all cases, my CPU usage is close to 100%.
I am starting to wonder if I should be trying the PF_RING version of libpcap, but from what I've read on SO and libpcap mailing lists, libpcap > 1.0 does the zero copy stuff anyway, so maybe no point.
Any ideas, pointers greatly appreciated!
G
In the following code (the read implementation for a char driver), is it possible for MMU TLB entries to change during wait_event_interruptible, such that __put_user causes an exception even though access_ok succeeded?
Is it possible to lock the user buffer such that it remains valid for the duration of the request?
Would repeating the access_ok check after wait_event_interruptible returns make this safe?
ssize_t mydriver_pkt_read( struct file* filp, char __user* const buff, size_t count, loff_t* offp )
{
struct mydriver_pkt_private* priv;
volatile unsigned short* iobase;
unsigned next;
char __user* p = buff;
if (count <= 0) return -EINVAL;
if (!access_ok(VERIFY_WRITE, buff, count)) return -EFAULT;
priv = (struct mydriver_pkt_private*)filp->private_data;
iobase = priv->iobase;
next = priv->retained;
if ((next & PKTBUF_FLAG_NOTEMPTY) == 0) {
next = ioread16(iobase);
if ((next & PKTBUF_FLAG_NOTEMPTY) == 0) { // no data, start blocking read
iowrite16(1, iobase); // enable interrupts
if (wait_event_interruptible(priv->wait_for_ringbuffer, (priv->retained & PKTBUF_FLAG_NOTEMPTY)))
return -ERESTARTSYS;
next = priv->retained;
}
}
while (count > 0) {
__put_user( (char)next, p );
p++;
count--;
next = ioread16(iobase);
if ((next & PKTBUF_FLAG_STARTPKT) || !(next & PKTBUF_FLAG_NOTEMPTY)) {
priv->retained = next;
return (p - buff);
}
}
/* discard remainder of packet */
do {
next = ioread16(iobase);
} while ((next & PKTBUF_FLAG_NOTEMPTY) && !(next & PKTBUF_FLAG_STARTPKT));
priv->retained = next;
return (p - buff);
}
Exclusive open code:
int mydriver_pkt_open( struct inode* inode, struct file* filp )
{
struct mydriver_pkt_private* priv;
priv = container_of(inode->i_cdev, struct mydriver_pkt_private, cdevnode);
if (atomic_cmpxchg(&priv->inuse, 0, 1))
return -EBUSY;
nonseekable_open(inode, filp);
filp->private_data = priv;
return 0;
}
Unless you have the mm_sem semaphore held, page tables can change at any time (by other threads of the same process unmapping pages from a different processor, or by evictions from page reclaim processes). You don't even need to sleep; it can happen even if you have preemption disabled, as long as the TLB shootdown interrupt can arrive. And it can happen even if interrupts are disabled, if you have a SMP machine, as you can, sometimes, see page table updates reflected even without an explicit TLB flush.
access_ok() only checks that the range of addresses does not overlap with kernel space. So it doesn't tell you anything about whether the page table entries allow access - but its result also does not change, even if you block. If access is denied, __put_user() will return -EFAULT, which must be propagated to userspace (ie, error out here with -EFAULT).
Note that the only difference between put_user() and __put_user() is that put_user() performs an access_ok() check as well. So if you're using it in a loop, doing a single access_ok() ahead of time and using __put_user() is probably the right thing to do.
I am trying to transfer an image using TCP sockets using linux. I have used the code many times to transfer small amounts but as soon as I tried to transfer the image it only transfered the first third. Is it possible that there is a maximum buffer size for tcp sockets in linux? If so how can I increase it? Is there a function that does this programatically?
I would guess that the problem is on the receiving side when you read from the socket. TCP is a stream based protocol with no idea of packets or message boundaries.
This means when you do a read you may get less bytes than you request. If your image is 128k for example you may only get 24k on your first read requiring you to read again to get the rest of the data. The fact that it's an image is irrelevant. Data is data.
For example:
int read_image(int sock, int size, unsigned char *buf) {
int bytes_read = 0, len = 0;
while (bytes_read < size && ((len = recv(sock, buf + bytes_read,size-bytes_read, 0)) > 0)) {
bytes_read += len;
}
if (len == 0 || len < 0) doerror();
return bytes_read;
}
TCP sends the data in pieces, so you're not guaranteed to get it all at once with a single read (although it's guaranteed to stay in the order you send it). You basically have to read multiple times until you get all the data. It also doesn't know how much data you sent on the receiver side. Normally, you send a fixed size "length" field first (always 8 bytes, for example) so you know how much data there is. Then you keep reading and building a buffer until you get that many bytes.
So the sender would look something like this (pseudocode)
int imageLength;
char *imageData;
// set imageLength and imageData
send(&imageLength, sizeof(int));
send(imageData, imageLength);
And the receiver would look like this (pseudocode)
int imageLength;
char *imageData;
guaranteed_read(&imageLength, sizeof(int));
imageData = new char[imageLength];
guaranteed_read(imageData, imageLength);
void guaranteed_read(char* destBuf, int length)
{
int totalRead=0, numRead;
while(totalRead < length)
{
int remaining = length - totalRead;
numRead = read(&destBuf[totalRead], remaining);
if(numRead > 0)
{
totalRead += numRead;
}
else
{
// error reading from socket
}
}
}
Obviously I left off the actual socket descriptor and you need to add a lot of error checking to all of that. It wasn't meant to be complete, more to show the idea.
The maximum size for 1 single IP packet is 65535, which is extremely close to the number you are hitting. I doubt that is a coincidence.