Pipe data to thread. Read stuck - linux

I want to trigger a callback when data is written on a file descriptor. For this I have set up a pipe and a reader thread, which reads the pipe. When it has data, the callback is called with the data.
The problem is that the reader is stuck on the read syscall. Destruction order is as follows:
Close write end of pipe (I expected this to trigger a return from blocking read, but apparently it doesn't)
Wait for reader thread to exit
Restore old file descriptor context (If stdout was redirected to the pipe, it no longer is)
Close read end of pipe

When the write end of the pipe is closed, on the read end, the read() system call returns 0 if it is blocking.
Here is an example program creating a reader thread from a pipe. The main program gets the data from stdin thanks to fgets() and write those data into the pipe. On the other side, the thread reads the pipe and triggers the callback passed as parameter. The thread stops when it gets 0 from the read() of the pipe (meaning that the main thread closed the write side):
#include <stdio.h>
#include <errno.h>
#include <pthread.h>
#include <unistd.h>
#include <string.h>
int pfd[2];
void read_cbk(char *data, size_t size)
{
int rc;
printf("CBK triggered, %zu bytes: %s", size, data);
}
void *reader(void *p){
char data[128];
void (* cbk)(char *data, size_t size) = (void (*)(char *, size_t))p;
int rc;
do {
rc = read(pfd[0], data, sizeof(data));
switch(rc) {
case 0: fprintf(stderr, "Thread: rc=0\n");
break;
case -1: fprintf(stderr, "Thread: rc=-1, errno=%d\n", errno);
break;
default: cbk(data, (size_t)rc);
}
} while(rc > 0);
}
int main(){
pthread_t treader;
int rc;
char input[128];
char *p;
pipe(pfd);
pthread_create(&treader, NULL, reader , read_cbk);
do {
// fgets() insert terminating \n and \0 in the buffer
// If EOF (that is to say CTRL-D), fgets() returns NULL
p = fgets(input, sizeof(input), stdin);
if (p != NULL) {
// Send the terminating \0 to the reader to facilitate printf()
rc = write(pfd[1], input, strlen(p) + 1);
}
} while (p);
close(pfd[1]);
pthread_join(treader, NULL);
close(pfd[0]);
}
Example of execution:
$ gcc t.c -o t -lpthread
$ ./t
azerty is not qwerty
CBK triggered, 22 bytes: azerty is not qwerty
string
CBK triggered, 8 bytes: string
# Here I typed CTRL-D to generate an EOF on stdin
Thread: rc=0

I found the problem. For redirection, the following has to be done
Create a pipe. This creates two file descriptors. One for reading, and one for writing.
dup2 so the original file descriptor is an alias to the write end of the pipe. This increments the use count of the write end by one
Thus, before synchronizing, I have to restore the context. This means that the following order is correct:
Close write end of pipe
Restore old file descriptor context
Wait for reader thread to exit
Close read end of pipe
For reference to the question, step 2 and 3 must be reorder in order to avoid deadlock.

Related

Cygwin FIFO vs native Linux FIFO - discrepancy in blocking behaviour?

The code shown is based on an example using named pipes from some tutorial site
server.c
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
#define FIFO_FILE "MYFIFO"
int main()
{
int fd;
char readbuf[80];
int read_bytes;
// mknod(FIFO_FILE, S_IFIFO|0640, 0);
mkfifo(FIFO_FILE, 0777);
while(1) {
fd = open(FIFO_FILE, O_RDONLY);
read_bytes = read(fd, readbuf, sizeof(readbuf));
readbuf[read_bytes] = '\0';
printf("Received string: \"%s\". Length is %d\n", readbuf, (int)strlen(readbuf));
}
return 0;
}
When executing the server in Windows, using Cygwin, then the server enters an undesired loop, repeating the same message. For example, if you write in a shell:
$ ./server
|
then the "server" waits for the client, but when the FIFO is not empty, e.g. writing in a new shell
$ echo "Hello" > MYFIFO
then the server enters an infinite loop, repeating the "Hello"-string
Received string: "Hello". Length is 4
Received string: "Hello". Length is 4
...
Furthermore, new strings written to the fifo doesn't seem to be read by the server. However, in Linux the behaviour is quite different. In Linux, the server prints the string and waits for new data to appear on the fifo. What is the reason for this discrepancy ?
You need to fix your code to remove at least 3 bugs:
You're not doing a close(fd) so you will get a file descriptor leak and eventually be unable to open() new files.
You're not checking the value of fd (if it returns -1 then there was an error).
You're not checking the value of read (if it returns -1 then there was an error)... and your readbuf[read_bytes] = '\0'; will not be doing what you expect as a result.
When you get an error then errno will tell you what went wrong.
These bugs probably explain why you keep getting Hello output (especially the readbuf[read_bytes] problem).

How to make mprotect() to make forward progress after handling pagefaulte exception? [duplicate]

I want to write a signal handler to catch SIGSEGV.
I protect a block of memory for read or write using
char *buffer;
char *p;
char a;
int pagesize = 4096;
mprotect(buffer,pagesize,PROT_NONE)
This protects pagesize bytes of memory starting at buffer against any reads or writes.
Second, I try to read the memory:
p = buffer;
a = *p
This will generate a SIGSEGV, and my handler will be called.
So far so good. My problem is that, once the handler is called, I want to change the access write of the memory by doing
mprotect(buffer,pagesize,PROT_READ);
and continue normal functioning of my code. I do not want to exit the function.
On future writes to the same memory, I want to catch the signal again and modify the write rights and then record that event.
Here is the code:
#include <signal.h>
#include <stdio.h>
#include <malloc.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/mman.h>
#define handle_error(msg) \
do { perror(msg); exit(EXIT_FAILURE); } while (0)
char *buffer;
int flag=0;
static void handler(int sig, siginfo_t *si, void *unused)
{
printf("Got SIGSEGV at address: 0x%lx\n",(long) si->si_addr);
printf("Implements the handler only\n");
flag=1;
//exit(EXIT_FAILURE);
}
int main(int argc, char *argv[])
{
char *p; char a;
int pagesize;
struct sigaction sa;
sa.sa_flags = SA_SIGINFO;
sigemptyset(&sa.sa_mask);
sa.sa_sigaction = handler;
if (sigaction(SIGSEGV, &sa, NULL) == -1)
handle_error("sigaction");
pagesize=4096;
/* Allocate a buffer aligned on a page boundary;
initial protection is PROT_READ | PROT_WRITE */
buffer = memalign(pagesize, 4 * pagesize);
if (buffer == NULL)
handle_error("memalign");
printf("Start of region: 0x%lx\n", (long) buffer);
printf("Start of region: 0x%lx\n", (long) buffer+pagesize);
printf("Start of region: 0x%lx\n", (long) buffer+2*pagesize);
printf("Start of region: 0x%lx\n", (long) buffer+3*pagesize);
//if (mprotect(buffer + pagesize * 0, pagesize,PROT_NONE) == -1)
if (mprotect(buffer + pagesize * 0, pagesize,PROT_NONE) == -1)
handle_error("mprotect");
//for (p = buffer ; ; )
if(flag==0)
{
p = buffer+pagesize/2;
printf("It comes here before reading memory\n");
a = *p; //trying to read the memory
printf("It comes here after reading memory\n");
}
else
{
if (mprotect(buffer + pagesize * 0, pagesize,PROT_READ) == -1)
handle_error("mprotect");
a = *p;
printf("Now i can read the memory\n");
}
/* for (p = buffer;p<=buffer+4*pagesize ;p++ )
{
//a = *(p);
*(p) = 'a';
printf("Writing at address %p\n",p);
}*/
printf("Loop completed\n"); /* Should never happen */
exit(EXIT_SUCCESS);
}
The problem is that only the signal handler runs and I can't return to the main function after catching the signal.
When your signal handler returns (assuming it doesn't call exit or longjmp or something that prevents it from actually returning), the code will continue at the point the signal occurred, reexecuting the same instruction. Since at this point, the memory protection has not been changed, it will just throw the signal again, and you'll be back in your signal handler in an infinite loop.
So to make it work, you have to call mprotect in the signal handler. Unfortunately, as Steven Schansker notes, mprotect is not async-safe, so you can't safely call it from the signal handler. So, as far as POSIX is concerned, you're screwed.
Fortunately on most implementations (all modern UNIX and Linux variants as far as I know), mprotect is a system call, so is safe to call from within a signal handler, so you can do most of what you want. The problem is that if you want to change the protections back after the read, you'll have to do that in the main program after the read.
Another possibility is to do something with the third argument to the signal handler, which points at an OS and arch specific structure that contains info about where the signal occurred. On Linux, this is a ucontext structure, which contains machine-specific info about the $PC address and other register contents where the signal occurred. If you modify this, you change where the signal handler will return to, so you can change the $PC to be just after the faulting instruction so it won't re-execute after the handler returns. This is very tricky to get right (and non-portable too).
edit
The ucontext structure is defined in <ucontext.h>. Within the ucontext the field uc_mcontext contains the machine context, and within that, the array gregs contains the general register context. So in your signal handler:
ucontext *u = (ucontext *)unused;
unsigned char *pc = (unsigned char *)u->uc_mcontext.gregs[REG_RIP];
will give you the pc where the exception occurred. You can read it to figure out what instruction it
was that faulted, and do something different.
As far as the portability of calling mprotect in the signal handler is concerned, any system that follows either the SVID spec or the BSD4 spec should be safe -- they allow calling any system call (anything in section 2 of the manual) in a signal handler.
You've fallen into the trap that all people do when they first try to handle signals. The trap? Thinking that you can actually do anything useful with signal handlers. From a signal handler, you are only allowed to call asynchronous and reentrant-safe library calls.
See this CERT advisory as to why and a list of the POSIX functions that are safe.
Note that printf(), which you are already calling, is not on that list.
Nor is mprotect. You're not allowed to call it from a signal handler. It might work, but I can promise you'll run into problems down the road. Be really careful with signal handlers, they're tricky to get right!
EDIT
Since I'm being a portability douchebag at the moment already, I'll point out that you also shouldn't write to shared (i.e. global) variables without taking the proper precautions.
You can recover from SIGSEGV on linux. Also you can recover from segmentation faults on Windows (you'll see a structured exception instead of a signal). But the POSIX standard doesn't guarantee recovery, so your code will be very non-portable.
Take a look at libsigsegv.
You should not return from the signal handler, as then behavior is undefined. Rather, jump out of it with longjmp.
This is only okay if the signal is generated in an async-signal-safe function. Otherwise, behavior is undefined if the program ever calls another async-signal-unsafe function. Hence, the signal handler should only be established immediately before it is necessary, and disestablished as soon as possible.
In fact, I know of very few uses of a SIGSEGV handler:
use an async-signal-safe backtrace library to log a backtrace, then die.
in a VM such as the JVM or CLR: check if the SIGSEGV occurred in JIT-compiled code. If not, die; if so, then throw a language-specific exception (not a C++ exception), which works because the JIT compiler knew that the trap could happen and generated appropriate frame unwind data.
clone() and exec() a debugger (do not use fork() – that calls callbacks registered by pthread_atfork()).
Finally, note that any action that triggers SIGSEGV is probably UB, as this is accessing invalid memory. However, this would not be the case if the signal was, say, SIGFPE.
There is a compilation problem using ucontext_t or struct ucontext (present in /usr/include/sys/ucontext.h)
http://www.mail-archive.com/arch-general#archlinux.org/msg13853.html

Use select like function on regular disk file

I have a computer wich logs some sensors data into 8 different files.
I developed a software that allows you to copy this data to another computer when you connect the two machines using an rj45 cable.
After retrieving data at my computer, I need to send it line by line of each file using a pseudo serial (using socat).
I created a program which uses nested for loops in order to check if data is ready in all the 8 files, and then extract a line and send it to puttySX.
Problem is CPU usage. A way to reduce it, is using blocking function to know if data is ready be to read or not but is there any function like select on sockets or serial ports but for such files?
If no, what should I do? Thanks
You can take a look at inotify which lets you monitor file system events.
Here is a sample code to get you started (this is not production code):
#include <stdio.h>
#include <stdlib.h>
#include <sys/inotify.h>
#define BUF_LEN (sizeof(struct inotify_event) * 1)
int main(int argc, char *argv[])
{
char *filepath;
int fd, wd;
struct inotify_event *event;
char buf[BUF_LEN];
ssize_t ret;
if (argc != 2)
{
fprintf(stderr, "Usage: ./%s <filepath>\n", argv[0]);
return (EXIT_FAILURE);
}
filepath = argv[1];
/* Initialization */
fd = inotify_init();
if (fd == -1)
{
perror("inotify_init()");
return (EXIT_FAILURE);
}
/* Specify which file to monitor */
wd = inotify_add_watch(fd, filepath, IN_MODIFY);
if (wd == -1)
{
perror("inotify_add_watch");
close(fd);
return (EXIT_FAILURE);
}
/* Wait for that file to be modified, */
/* and print a notification each time it does */
for (;;)
{
ret = read(fd, buf, BUF_LEN);
if (ret < 1)
{
perror("read()");
close(fd);
return (EXIT_FAILURE);
}
event = (struct inotify_event *)buf;
if (event->mask & IN_MODIFY)
printf("File modified!\n");
}
close(fd);
return(EXIT_SUCCESS);
}
So,
I post to answer my question. Thanks to #yoones I found some trick to do this.
When a log file is created, I set a bool on true in a ini file looking like this
[CreatedFiles]
cli1=false
cli2=false
cli3=false
cli4=false
cli5=false
cli6=false
cli7=false
cli8=false
Another program uses inotify to detect creation and modification in the corresponding files. Once there's some change it reads the ini file, process the data and when it finishes to read the data, it deletes the log file and write false in the ini file in the corresponding line.
Since I have to process several log files in the same time, each time I read a line, I verify my ini file to see if I have to start to process another log file as well so I can start multiple process in the same time.
I did a infinite while loop so when all processes are done, the program is back to a select call, waiting for some change and not consuming all CPU's resources.
I'm sorry if I'm not so clear, English is not my native language.
Thanks all for you reply and comments.

Interprocess Communication with pipe and file

i'm using linux as operating system and trying to communicate three processes with pipe and file. It should work with any file put on STDIN.
And pipe works just fine, but second process is unavailable to write one char into file properly or third to read.
Firstly of course i initialize function as semlock and semunlock and opening pipe is also there. I appreciate any help cause i have no clue.
if (!(PID[1] = fork ())) {
int BUF_SIZE = 4096;
char d[BUF_SIZE];
while (fgets (d, BUF_SIZE, stdin) != NULL) {
write (mypipe[1], &d, BUF_SIZE);
}
}
if (!(PID[2] = fork ())) {
int reading_size = 0;
char r;
close (mypipe[1]);
semlock (semid1);
while (reading_size = read (mypipe[0], &r, 1)) {
if ((file = fopen ("proces2.txt", "w")) == NULL) {
warn ("error !!!");
exit (1);
}
fputc (r, file);
fclose (file);
semunlock (semid2);
}
}
if (!(PID[3] = fork ())) {
char x;
semlock (semid2);
do {
if ((plikProces3 = fopen ("proces2.txt", "r")) == NULL) {
warn ("Blad przy otwarciu pliku do odczytu !!!");
exit (1);
}
i = getc (plikProces3);
o = fprintf (stdout, "%c", i);
fclose (plikProces3);
semunlock (semid1);
} while (i != EOF);
}
What makes you think the child runs first? You haven't waited for the child process to finish so can hit EOF reading the file, before the previous child has written. Shouldn't the last fork() call be a wait, so you know the file was written? As it stands you have 4 processes, NOT 3!!
Then you are closing the mypipe[1] in the 2nd child process which as it is a forked copy, does not close the pipe inthe first child. You also are trying to write BUFSIZ characters, so you appear to be trying to write out more characters than were written, try "write (mypipe[1], &d, strlen(d));".
It looks very odd, to have the fopen() & fclose() within the character read/write loop. You really want to re-open & re-write 1 character into the file over and over?
Similarly the process2 file seems to be re-opened so the first character within would be written again and again, if it's non-empty.
There are bound to be other bugs, but that should help you for now.

Linux input device events, how to retrieve initial state

I am using the gpio-keys device driver to handle some buttons in an embedded device running Linux. Applications in user space can just open /dev/input/eventX and read input events in a loop.
My question is how to get the initial states of the buttons. There is an ioctl call (EVIOCGKEY) which can be used for this, however if I first check this and then start to read from /dev/input/eventX, there's no way to guarantee that the state did not change in between.
Any suggestions?
The evdev devices queue events until you read() them, so in most cases opening the device, doing the ioctl() and immediately starting to read events from it should work. If the driver dropped some events from the queue, it sends you a SYN_DROPPED event, so you can detect situations where that happened. The libevdev documentation has some ideas on how one should handle that situation; the way I read it you should simply retry, i.e. drop all pending events, and redo the ioctl() until there are no more SYN_DROPPED events.
I used this code to verify that this approach works:
#include <stdio.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/input.h>
#include <string.h>
#define EVDEV "/dev/input/event9"
int main(int argc, char **argv) {
unsigned char key_states[KEY_MAX/8 + 1];
struct input_event evt;
int fd;
memset(key_states, 0, sizeof(key_states));
fd = open(EVDEV, O_RDWR);
ioctl(fd, EVIOCGKEY(sizeof(key_states)), key_states);
// Create some inconsistency
printf("Type (lots) now to make evdev drop events from the queue\n");
sleep(5);
printf("\n");
while(read(fd, &evt, sizeof(struct input_event)) > 0) {
if(evt.type == EV_SYN && evt.code == SYN_DROPPED) {
printf("Received SYN_DROPPED. Restart.\n");
fsync(fd);
ioctl(fd, EVIOCGKEY(sizeof(key_states)), key_states);
}
else if(evt.type == EV_KEY) {
// Ignore repetitions
if(evt.value > 1) continue;
key_states[evt.code / 8] ^= 1 << (evt.code % 8);
if((key_states[evt.code / 8] >> (evt.code % 8)) & 1 != evt.value) {
printf("Inconsistency detected: Keycode %d is reported as %d, but %d is stored\n", evt.code, evt.value,
(key_states[evt.code / 8] >> (evt.code % 8)) & 1);
}
}
}
}
After starting, the program deliberately waits 5 seconds. Hit some keys in that time to fill the buffer. On my system, I need to enter about 70 characters to trigger a SYN_DROPPED. The EV_KEY handling code checks if the events are consistent with the state reported by the EVIOCGKEY ioctl.

Resources