How do I trigger an EPOLLRDHUP event on my tcp socket using the other thread programatically,
I have added the epoll instance with EPOLLRDHUP event and tried to generate the event, but it modifies the event on that FD , do not trigger it,
I want my first thread which is continuously waiting for event with epoll_wait(), should receive the event from EPOLLRDHUP, as soon as the other thread triggers it, I am not able to get how to trigger that event, I tried using write system call in another thread but that also do not trigger the event on socket FD I guess, poll should come out of blocking loop is my requirement, Please help, Thanks.
You can't generate epoll events on same file descriptor from another thread, EPOLLRDHUP would be generated based on something happening at the other end of the TCP connection.
If you have 1 thread waiting on epoll_wait() and you want to wake that thread up from another thread, you should create a pipe(), have your epoll_wait wait for read events on the reading side of the pipe in addition to any TCP sockets. When you want to wake up your thread, you write a byte on the writing side of the pipe.
(an eventfd could be used instead of the pipe to achieve the same too)
Related
I am doing a task to receive user input in one thread (thread 1) and another thread(thread 2) to send cyclic CAN messages on a linux environment. The flow of the program is as follows,
In main thread, created a socket using socket(AF_INET,SOCK_STREAM,0) for server-client TCP connection
Bind was successfully done.
Using accept(), server-client established
Upon 3) , two threads are created, thread1 and thread2
thread 1 for user input from recv(), http://man7.org/linux/man-pages/man2/recv.2.html
thread 2 created socket for CAN communication using socket and uses timer to send the CAN messages with specified interval, continuously.
In thread2, I have used timer_create to send cyclic messages, which uses write function to write the CAN messages in the CAN socket.
http://man7.org/linux/man-pages/man2/timer_create.2.html
The problem is I have recv in an infinite loop in thread1 waiting for user input and the timer function in thread2 is also continuous. This makes recv to exit with signal interruption EINTR from thread1. I have tried the following,
Created a condition wait using pthread_cond_wait in thread 1, to wait for status update from thread 2. This does not work, as once the timer has started, it does not check for condition variable any more.
Does anyone has similar experience with using socket in two infinite loops for receiving and sending in two threads and controlling it on a condition variable. Any suggestion on how to handle them?
Is it possible to use epoll in one-shot level-triggered mode?
I couldn't find any information on it when I searched; it seems everyone uses edge-triggered mode.
When the EPOLLONESHOT flag is selected and you have pulled an event for a socket, then the socket won't get removed from epoll as many think but its events get disabled. You can enable them again using epoll_ctl / EPOLL_CTL_MOD.
An example case when the EPOLLONESHOT behavior comes handy is when you've read the available data from a socket into a buffer. That buffer would be emptied independently, but until it isn't empty, you have to disable the socket events, even if the socket has additional data. Then after the buffer got used and emptied, you can re-enable the socket.
The difference between the edge- and level-triggered "one shot" behaviors come only when you re-enable the socket. An example:
The socket receives 7K data (for now it's stored in a kernel buffer)
You wait for an input event, then the socket events get disabled due to the EPOLLONESHOT.
You read 4K into an application level buffer.
Later the application buffer gets used and emptied. You re-enable the socket with epoll_ctl / EPOLL_CTL_MOD.
Level-triggered EPOLLONESHOT:
Since 3K data is still present in the kernel buffers, the event is triggered again.
Edge-triggered EPOLLONESHOT:
It won't trigger an event again for the available data. You must test it by reading, and waiting for EAGAIN / EWOULDBLOCK.
If you want epoll to stop listening on a socket the you should use EPOLLONESHOT. If you do use EPOLLONESHOT then you will have to add the socket back to epoll after epoll signals on that socket. Thus EPOLLONESHOT is not EPOLLET. You can use EPOLLONESHOT without EPOLLET, but it might not be efficient. If you use both the flags, then you will have to make use of non blocking sockets and add a socket to epoll only once recv and send return with an EAGAIN error. Please refer to man page for details.
I'm doing select() on a blocking socket with no timeout select(sock+1, &rfd, NULL, NULL, NULL).
This happens in a thread whose objective is to dispatch incoming data. Another surveillance thread is managing a keep alive with the peer and when it detects a dead connection, it would close the socket.
I was expecting select() to return with -1 in that case. It does that on Windows but never on Linux, so the dispatch thread is locked forever when the peer disappear non-gracefully. For completeness, there is pending data to be transmitted on that, I've tried to play with SO_LINGER but that does not change anything.
The problem can be solved by setting a timeout in select() and in that case after close and timeout, select() ultimately exits with -1, but I thought, reading the doc, that select() with no timeout would still exit on close, even when the peer is not responding.
Do I misuse select() or is there a better way to handle half-open sockets ?
Yes you misuse the select. The man select states:
If a file descriptor being monitored by select() is closed in another thread, the result is unspecified. On some UNIX systems, select() unblocks and returns, with an indication that the file descriptor is ready (a subsequent I/O operation will likely fail with an error, unless another the file descriptor reopened between the time select() returned and the I/O operations was performed). On Linux (and some other systems), closing the file descriptor in another thread has no effect on select(). In summary, any application that relies on a particular behavior in this scenario must be considered buggy.
So you cannot close connection from other thread. Unfortunately the poll has the same issue.
EDIT
There are several possible solution and I have not sufficient information about your application. Following changed can be considered:
Use epoll instead of select if you are on linux or other modern polling mechanism if you on another OS. select is quite old function and it was designed in time when threading was not considered seriously.
Establish a communication channel between the select thread and the keep-alive thread. When keep alive thread detects a dead peer then don't close the socket itself but instructs the select thread to do that. Typically it can be done through a local socket. The local socket is added to select descriptor set and when the keep-alive thread writes something to it the select thread wakes up and can take an action.
I have several threads, one of them calls epoll_wait in a loop, others can open connections that need to be epoll'ed by first thread. Is it possible to just add new sockets with epoll_ctl while another thread waits in epoll_wait?
What will happen in the following scenario:
Thread 1 calls epoll_wait.
Thread 2 creates a socket(A) and adds it to epoll instance using epoll_ctl.
Someone sends some data, socket A becomes ready for read() call.
Will epoll_wait return socket A?
Yes, it will. The whole point of an epoll socket is that you don't have to duplicate effort. No snapshotting or use of multiple wait queues is involved.
Under the hood, the epoll socket has its own wait queue. When you block on the epoll socket, you are added to that single wait queue. No state is saved or anything like that. The state is in the epoll socket itself.
I have a thread that is waiting for events received on a FIFO.
Most of events are configuration events send from another thread in the same process.
I would like the thread also to be able to handle interrupt events by having the interrupt handler writing to the FIFO is that possible?
Any other suggestion instead of using FIFO is welcome!
Yes, it can. Using interrupt handlers is one of the newer ways of handling asynchronous I/O.
The more typical way of doing this, is with a select, poll, or Linux epoll command.
These are arguably a bit more preferred, because rather than "interrupting" your code when an event is available - you are able to handle events and return to the "poll" loop to get more events when you're done with the prior event. These mechanisms can wait on a number of different file descriptors at the same time, and return whichever one is available.
An interrupt handler is probably not what you want to do - because you don't necessarily want to be interrupted while processing one event, to know that another is ready. If you did, you'd probably wind up queuing the request anyway for later handling - which is exactly what poll and epoll to to begin with.
If you're under Linux, "epoll" is the most optimum. If your not, (or want POSIX compliance), use "poll". "select" is an "older" method, and doesn't queue requests as well.