I am assigning sched_deadline policy in a thread via the following code:
struct sched_attr attr;
memset(&attr, sizeof(struct sched_attr), 0);
attr.size = sizeof(struct sched_attr);
attr.sched_policy = SCHED_DEADLINE;
attr.sched_runtime = runtime;
attr.sched_deadline = period;
attr.sched_period = period;
int ret = syscall(SYS_sched_setattr, 0, &attr, 0);
if(ret == -1) {
perror("Error in syscall");
pthread_exit(NULL);
}
This works fine, but the program can only run with sudo. Whenever, I execute it normally, it fails. I am expecting this happens because of the syscall. Is there any way to avoid using sudo for this. There are no wrappers available for this syscall of SCHED_DEADLINE yet in Linux.
Related
I have written an LKM in Linux. The ioctl function, my_ioctl is called by a user-level program foo.c. I want to change the scheduling policy of foo.c. Therefore I am doing the following in my my_ioctl function from this link:
struct sched_attr attr;
int ret;
unsigned int flags = 0;
attr.size = sizeof(attr);
attr.sched_flags = 0;
attr.sched_nice = 0;
attr.sched_priority = 0;
/* This creates a 10ms/30ms reservation */
attr.sched_policy = SCHED_DEADLINE;
attr.sched_runtime = 10 * 1000 * 1000;
attr.sched_period = attr.sched_deadline = 30 * 1000 * 1000;
ret = sched_setattr(current->pid, &attr, flags);
if (ret < 0) {
perror("sched_setattr");
exit(-1);
}
The sched_setattr is following:
int sched_setattr(pid_t pid,
const struct sched_attr *attr,
unsigned int flags) {
return syscall(__NR_sched_setattr, pid, attr, flags);
}
I have changed the syscall to sys_mycall because it's LKM. struct sched_attr is also defined in the above mentioned Linux Kernel documentation link. However, I could not change the scheduling policy by this. It throws me error like scheduling policy cannot be changed from kernel space.
I don't understand why this is the case. There is an utility chrt which does the same thing for a process from the user-space; then why is this not possible from a LKM? Or am I missing something?
I'm trying to get libwebsockets running in a multithreaded environment on OS X. I couldn't trigger sending Data from a different thread than the main service thread. On libwebsocket docs it was implied this should be possible (demo code, mailinglist). So I dug into the code and found the problem in the poll() function.
It seems that poll() is behaving differently concerning the struct pollfd that is given as parameter. libwebsockets is relying on the possibility to change the fds.event fields while poll() is active. This is working fine on Linux but is not working on OS X.
I wrote a small test program to demonstrate the behaviour:
#include <unistd.h>
#include <netdb.h>
#include <poll.h>
#include <iostream>
#include <thread>
#define PORT "3490"
struct pollfd fds[1];
bool connected = false;
void main_loop() {
int sockfd, new_fd;
struct addrinfo hints, *servinfo, *p;
socklen_t sin_size;
int yes=1;
char s[INET6_ADDRSTRLEN];
int rv;
memset(&hints, 0, sizeof hints);
hints.ai_family = AF_INET;
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_PASSIVE;
if ((rv = getaddrinfo(NULL, PORT, &hints, &servinfo)) != 0) {
fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(rv));
return;
}
for(p = servinfo; p != NULL; p = p->ai_next) {
if ((sockfd = socket(p->ai_family, p->ai_socktype, p->ai_protocol)) == -1) {
perror("server: socket");
continue;
}
if (setsockopt(sockfd, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(int)) == -1) {
perror("setsockopt");
exit(1);
}
if (bind(sockfd, p->ai_addr, p->ai_addrlen) == -1) {
close(sockfd);
perror("server: bind");
continue;
}
break;
}
freeaddrinfo(servinfo);
if (p == NULL) {
fprintf(stderr, "server: failed to bind\n");
exit(1);
}
if (listen(sockfd, 10) == -1) {
perror("listen");
exit(1);
}
printf("server: waiting for connections...\n");
new_fd = accept(sockfd, NULL, &sin_size);
if (new_fd == -1) {
perror("accept");
return;
}
fds[0].fd = new_fd;
fds[0].events = POLLIN;
connected = true;
printf("event is %i\n", fds[0].events);
int ret = poll(fds, 1, 5000);
printf("event is %i\n", fds[0].events); //expecting 1 on Mac and 5 on Linux
if (send(new_fd, "Hello, world!\n", 14, 0) == -1)
perror("send");
close(new_fd);
close(sockfd);
}
void second_thread()
{
while(connected == false){}
sleep(1);
fds[0].events = POLLIN|POLLOUT;
printf("set event to %i\n", fds[0].events);
}
int main() {
std::thread t1(main_loop);
std::thread t2(second_thread);
t1.join();
t2.join();
return 0;
}
Compile on OS X using clang++ -std=c++11 -stdlib=libc++ -o poll poll.cpp
and on Linux using g++ -std=c++11 -pthread -o poll poll.cpp
The program starts listening on port 3490. If you connect to it (e.g. using netcat localhost 3490) it will poll for input on the main thread and try to change the event flags in the second thread. It will exit after 5 seconds.
The output on OS X:
server: waiting for connections...
event is 1
set event to 5
event is 1
The output on Linux:
server: waiting for connections...
event is 1
set event to 5
event is 5
So my question is: is there any documentation available that explains this behavior? Is it safe what libwebsockets is doing in expecting that it is legal to change fds.events while poll is active? I couldn't find any details about it in the manpages (OS X, Linux).
You seem to say, at first, that you found some documentation that claims that this is supported and defined behavior. I'd be curious to know where you read that, because I am unable to find anything in either the Linux man page for poll(2), nor in the POSIX man page for poll() that documents that a different thread can actually change the values in the event array argument that another thread passed to poll(), and have the different thread's changes actually take effect in the original thread's poll() call, irrespective of any issues relating to memory barriers, and such.
Both man pages appear to be completely silent, to me, on this subject matter. They do not indicate whether this is expected, supported, or defined behavior; or whether this is not a supported or defined behavior.
The proposition that a different thread can modify the parameters to a system call issued by another thread, after -- AFTER -- the other thread has already entered the syscall, seems rather counter-intertuitive to me. If this is supported behavior, I would expect it to be explicitly documented, and I can't find any reference to it in the Linux or the POSIX man pages.
Having said that: even if I limit the scope of my software to Linux, even if I don't need to care about other platforms; given the absence of any documentation of this, and even if my testing showed the Linux kernel implementing poll(2) this way, I would not expect to have any guarantees that some future kernel version will continue to behave this way. I would not be able to rely on this behavior, except on the specific kernel build I tested this with.
So, to answer your question: the only documentation that's authoritative on this topic are the man pages in question. They do not explicitly document this as legal behavior; and although they do not explicitly say that this is illegal behavior either, for the reasons stated above, I would consider this to be unsupported, undefined behavior.
I'm trying to detect when a gpio pin goes from low to high and am having trouble. From what I've read I should be able to configure the pin as input this way:
# echo in > /sys/class/gpio/gpio51/direction
# echo rising > /sys/class/gpio/gpio51/edge
Next I try running a c program that waits for the rising edge using select. The code looks like this (notice I commented out an attempt to just read the file, since reading is supposed to block if you don't set O_NONBLOCK):
#include<stdio.h>
#include<fcntl.h>
#include <sys/select.h>
int main(void) {
int fd = open("/sys/class/gpio/gpio51/value", O_RDONLY & ~O_NONBLOCK);
//int fd = open("/sys/class/gpio/gpio51/value", O_RDONLY | O_NONBLOCK);
//unsigned char buf[2];
//int x = read(fd, &buf, 2);
//printf("%d %d: %s\n", fd, x, buf);
fd_set exceptfds;
int res;
FD_ZERO(&exceptfds);
FD_SET(fd, &exceptfds);
//printf("waiting for %d: %s\n", exceptfds);
res = select(fd+1,
NULL, // readfds - not needed
NULL, // writefds - not needed
&exceptfds,
NULL); // timeout (never)
if (res > 0 && FD_ISSET(fd, &exceptfds)) {
printf("finished\n");
}
return 0;
}
The program exits immediately no matter what the state of the pin (high or low). Can anyone see something wrong with the way I'm doing this?
PS. I have a python library that uses poll() to do just this, and the python works as expected. I pull the pin low, call the python, it blocks, pull the pin high and the code continues. So I don't think it is a problem with the linux gpio driver.
https://bitbucket.org/cswank/gadgets/src/590504d4a30b8a83143e06c44b1c32207339c097/gadgets/io/poller.py?at=master
I figured it out. You must read from the file descriptor before the select call returns. Here is an example that works:
#include<stdio.h>
#include<fcntl.h>
#include <sys/select.h>
#define MAX_BUF 64
int main(void) {
int len;
char *buf[MAX_BUF];
int fd = open("/sys/class/gpio/gpio51/value", O_RDONLY);
fd_set exceptfds;
int res;
FD_ZERO(&exceptfds);
FD_SET(fd, &exceptfds);
len = read(fd, buf, MAX_BUF); //won't work without this read.
res = select(fd+1,
NULL, // readfds - not needed
NULL, // writefds - not needed
&exceptfds,
NULL); // timeout (never)
if (res > 0 && FD_ISSET(fd, &exceptfds)) {
printf("finished\n");
}
return 0;
}
I want to use eventfd as a way to signal simple events between kernelspace and userspace. eventfd will be used as a way to signal and the actual data will be transferred using ioctl.
Before going ahead with implementing this, I wrote a simple program to see how eventfd behaves with select(). It seems that if you use select to wait on an eventfd, it wont return when u write to it in a separate thread. In the code I wrote, the writing thread waits for 5 seconds beginning from program start before writing to the eventfd twice. I would expect the select() to return in the reading thread immediately following this write but this does not happen. The select() returns only after the timeout of 10 seconds and returns zero. Regardless of this return zero, when I try to read the eventfd after 10 seconds, I get the correct value.
I use Ubuntu 12.04.1 (3.2.0-29-generic-pae) i386
Any idea why this is so? It seems to me that select() is not working as it should.
PS: This question is similar to linux - Can't get eventfd to work with epoll together
Is anyone else facing similar issues?
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <stdint.h> //Definition of uint64_t
#include <pthread.h> //One thread writes to fd, other waits on it and then reads it
#include <time.h> //Writing thread uses delay before writing
#include <sys/eventfd.h>
int efd; //Event file descriptor
void * writing_thread_func() {
uint64_t eftd_ctr = 34;
ssize_t s;
printf("\n%s: now running...",__func__);
printf("\n%s: now sleeping for 5 seconds...",__func__);
fflush(stdout); //must call fflush before sleeping to ensure previous printf() is executed
sleep(5);
printf("\n%s: Writing %lld to eventfd...",__func__,eftd_ctr);
s = write(efd, &eftd_ctr, sizeof(uint64_t));
if (s != sizeof(uint64_t)) {
printf("\n%s: eventfd writing error. Exiting...",__func__);
exit(EXIT_FAILURE);
}
eftd_ctr = 99;
printf("\n%s: Writing %lld to eventfd...",__func__,eftd_ctr);
s = write(efd, &eftd_ctr, sizeof(uint64_t));
if (s != sizeof(uint64_t)) {
printf("\n%s: eventfd writing error. Exiting...",__func__);
exit(EXIT_FAILURE);
}
printf("\n%s: thread exiting...",__func__);
pthread_exit(0);
}
void * reading_thread_func() {
ssize_t s;
uint64_t eftd_ctr;
int retval; //for select()
fd_set rfds; //for select()
struct timeval tv; //for select()
printf("\n%s: now running...",__func__);
printf("\n%s: now waiting on select()...",__func__);
//Watch efd
FD_ZERO(&rfds);
FD_SET(efd, &rfds);
//Wait up to 10 seconds
tv.tv_sec = 10;
tv.tv_usec = 0;
retval = select(1, &rfds, NULL, NULL, &tv);
if (retval == -1){
printf("\n%s: select() error. Exiting...",__func__);
exit(EXIT_FAILURE);
} else if (retval > 0) {
printf("\n%s: select() says data is available now. Exiting...",__func__);
printf("\n%s: returned from select(), now executing read()...",__func__);
s = read(efd, &eftd_ctr, sizeof(uint64_t));
if (s != sizeof(uint64_t)){
printf("\n%s: eventfd read error. Exiting...",__func__);
exit(EXIT_FAILURE);
}
printf("\n%s: Returned from read(), value read = %lld",__func__, eftd_ctr);
} else if (retval == 0) {
printf("\n%s: select() says that no data was available even after 10 seconds...",__func__);
printf("\n%s: but lets try reading efd count anyway...",__func__);
s = read(efd, &eftd_ctr, sizeof(uint64_t));
if (s != sizeof(uint64_t)){
printf("\n%s: eventfd read error. Exiting...",__func__);
exit(EXIT_FAILURE);
}
printf("\n%s: Returned from read(), value read = %lld",__func__, eftd_ctr);
exit(EXIT_FAILURE);
}
printf("\n%s: thread exiting...",__func__);
pthread_exit(0);
}
int main() {
pthread_t writing_thread_var, reading_thread_var;
//Create eventfd
efd = eventfd(0,0);
if (efd == -1){
printf("\n%s: Unable to create eventfd! Exiting...",__func__);
exit(EXIT_FAILURE);
}
printf("\n%s: eventfd created. value = %d. Spawning threads...",__func__,efd);
//Create threads
pthread_create(&writing_thread_var, NULL, writing_thread_func, NULL);
pthread_create(&reading_thread_var, NULL, reading_thread_func, NULL);
//Wait for threads to terminate
pthread_join(writing_thread_var, NULL);
pthread_join(reading_thread_var, NULL);
printf("\n%s: closing eventfd. Exiting...",__func__);
close(efd);
exit(EXIT_SUCCESS);
}
So it was a silly mistake:
I changed:
retval = select(1, &rfds, NULL, NULL, &tv);
to:
retval = select(efd+1, &rfds, NULL, NULL, &tv);
and it worked.
Thanks again #Steve-o
I'm porting an application built on top of the ACE Proactor framework. The application runs perfectly for both VxWorks and Windows, but fails to do so on Linux (CentOS 5.5, WindRiver Linux 1.4 & 3.0) with kernel 2.6.X.X - using librt.
I've narrowed the problem down to a very basic issue:
The application begins an asynchronous (via aio_read) read operation on a socket and subsequently begins an asynchronous (via aio_write) write on the very same socket. The read operation cannot be fulfilled yet since the protocol is initialized from the application's end.
- When the socket is in blocking-mode, the write is never reached and the protocol "hangs".
- When using a O_NONBLOCK socket, the write succeeds but the read returns indefinitely with a "EWOULDBLOCK/EAGAIN" error, never to recover (even if the AIO operation is restarted).
I went through multiple forums and could not find a definitive answer to whether this should work (and I'm doing something wrong) or impossible with Linux AIO. Is it possible if I drop the AIO and seek a different implementation (via epoll/poll/select etc.)?
Attached is a sample code to quickly re-produce the problem on a non-blocking socket:
#include <aio.h>
#include <stdio.h>
#include <stdlib.h>
#include <netdb.h>
#include <string.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <assert.h>
#include <errno.h>
#define BUFSIZE (100)
// Global variables
struct aiocb *cblist[2];
int theSocket;
void InitializeAiocbData(struct aiocb* pAiocb, char* pBuffer)
{
bzero( (char *)pAiocb, sizeof(struct aiocb) );
pAiocb->aio_fildes = theSocket;
pAiocb->aio_nbytes = BUFSIZE;
pAiocb->aio_offset = 0;
pAiocb->aio_buf = pBuffer;
}
void IssueReadOperation(struct aiocb* pAiocb, char* pBuffer)
{
InitializeAiocbData(pAiocb, pBuffer);
int ret = aio_read( pAiocb );
assert (ret >= 0);
}
void IssueWriteOperation(struct aiocb* pAiocb, char* pBuffer)
{
InitializeAiocbData(pAiocb, pBuffer);
int ret = aio_write( pAiocb );
assert (ret >= 0);
}
int main()
{
int ret;
int nPort = 11111;
char* szServer = "10.10.9.123";
// Connect to the remote server
theSocket = socket(AF_INET, SOCK_STREAM, 0);
assert (theSocket >= 0);
struct hostent *pServer;
struct sockaddr_in serv_addr;
pServer = gethostbyname(szServer);
bzero((char *) &serv_addr, sizeof(serv_addr));
serv_addr.sin_family = AF_INET;
serv_addr.sin_port = htons(nPort);
bcopy((char *)pServer->h_addr, (char *)&serv_addr.sin_addr.s_addr, pServer->h_length);
assert (connect(theSocket, (const sockaddr*)(&serv_addr), sizeof(serv_addr)) >= 0);
// Set the socket to be non-blocking
int oldFlags = fcntl(theSocket, F_GETFL) ;
int newFlags = oldFlags | O_NONBLOCK;
fcntl(theSocket, F_SETFL, newFlags);
printf("Socket flags: before=%o, after=%o\n", oldFlags, newFlags);
// Construct the AIO callbacks array
struct aiocb my_aiocb1, my_aiocb2;
char* pBuffer = new char[BUFSIZE+1];
bzero( (char *)cblist, sizeof(cblist) );
cblist[0] = &my_aiocb1;
cblist[1] = &my_aiocb2;
// Start the read and write operations on the same socket
IssueReadOperation(&my_aiocb1, pBuffer);
IssueWriteOperation(&my_aiocb2, pBuffer);
// Wait for I/O completion on both operations
int nRound = 1;
printf("\naio_suspend round #%d:\n", nRound++);
ret = aio_suspend( cblist, 2, NULL );
assert (ret == 0);
// Check the error status for the read and write operations
ret = aio_error(&my_aiocb1);
assert (ret == EWOULDBLOCK);
// Get the return code for the read
{
ssize_t retcode = aio_return(&my_aiocb1);
printf("First read operation results: aio_error=%d, aio_return=%d - That's the first EWOULDBLOCK\n", ret, retcode);
}
ret = aio_error(&my_aiocb2);
assert (ret == EINPROGRESS);
printf("Write operation is still \"in progress\"\n");
// Re-issue the read operation
IssueReadOperation(&my_aiocb1, pBuffer);
// Wait for I/O completion on both operations
printf("\naio_suspend round #%d:\n", nRound++);
ret = aio_suspend( cblist, 2, NULL );
assert (ret == 0);
// Check the error status for the read and write operations for the second time
ret = aio_error(&my_aiocb1);
assert (ret == EINPROGRESS);
printf("Second read operation request is suddenly marked as \"in progress\"\n");
ret = aio_error(&my_aiocb2);
assert (ret == 0);
// Get the return code for the write
{
ssize_t retcode = aio_return(&my_aiocb2);
printf("Write operation has completed with results: aio_error=%d, aio_return=%d\n", ret, retcode);
}
// Now try waiting for the read operation to complete - it'll just busy-wait, receiving "EWOULDBLOCK" indefinitely
do
{
printf("\naio_suspend round #%d:\n", nRound++);
ret = aio_suspend( cblist, 1, NULL );
assert (ret == 0);
// Check the error of the read operation and re-issue if needed
ret = aio_error(&my_aiocb1);
if (ret == EWOULDBLOCK)
{
IssueReadOperation(&my_aiocb1, pBuffer);
printf("EWOULDBLOCK again on the read operation!\n");
}
}
while (ret == EWOULDBLOCK);
}
Thanks in advance,
Yotam.
Firstly, O_NONBLOCK and AIO don't mix. AIO will report the asynchronous operation complete when the corresponding read or write wouldn't have blocked - and with O_NONBLOCK, they would never block, so the aio request will always complete immediately (with aio_return() giving EWOULDBLOCK).
Secondly, don't use the same buffer for two simultaneous outstanding aio requests. The buffer should be considered completely offlimits between the time when the aio request was issued and when aio_error() tells you that it has completed.
Thirdly, AIO requests to the same file descriptor are queued, in order to give sensible results. This means that your write won't happen until the read completes - if you need to write the data first, you need to issue the AIOs in the opposite order. The following will work fine, without setting O_NONBLOCK:
struct aiocb my_aiocb1, my_aiocb2;
char pBuffer1[BUFSIZE+1], pBuffer2[BUFSIZE+1] = "Some test message";
const struct aiocb *cblist[2] = { &my_aiocb1, &my_aiocb2 };
// Start the read and write operations on the same socket
IssueWriteOperation(&my_aiocb2, pBuffer2);
IssueReadOperation(&my_aiocb1, pBuffer1);
// Wait for I/O completion on both operations
int nRound = 1;
int aio_status1, aio_status2;
do {
printf("\naio_suspend round #%d:\n", nRound++);
ret = aio_suspend( cblist, 2, NULL );
assert (ret == 0);
// Check the error status for the read and write operations
aio_status1 = aio_error(&my_aiocb1);
if (aio_status1 == EINPROGRESS)
puts("aio1 still in progress.");
else
puts("aio1 completed.");
aio_status2 = aio_error(&my_aiocb2);
if (aio_status2 == EINPROGRESS)
puts("aio2 still in progress.");
else
puts("aio2 completed.");
} while (aio_status1 == EINPROGRESS || aio_status2 == EINPROGRESS);
// Get the return code for the read
ssize_t retcode;
retcode = aio_return(&my_aiocb1);
printf("First operation results: aio_error=%d, aio_return=%d\n", aio_status1, retcode);
retcode = aio_return(&my_aiocb1);
printf("Second operation results: aio_error=%d, aio_return=%d\n", aio_status1, retcode);
Alternatively, if you don't care about reads and writes being ordered with respect to each other, you can use dup() to create two file descriptors for the socket, and use one for reading and the other for writing - each will have its AIO operations queued separately.