Linux Thread priority , behaviour is abnormal - linux

In the below code snippet, I am creating 6 threads. Each with different priorities. The priority is mentioned in global priority array. I am doing a continuous increment of global variables inside each thread based on thread index. I was expecting the count to be higher if thread priority is higher. but my output is not adhering to priority concepts pl. refer to the output order shown below. I am trying this out on Ubuntu 16.04 and Linux kernel 4.10.
O/P,
Thread=0
Thread=3
Thread=2
Thread=5
Thread=1
Thread=4
pid=32155 count=4522138740
pid=32155 count=4509082289
pid=32155 count=4535088439
pid=32155 count=4517943246
pid=32155 count=4522643905
pid=32155 count=4519640181
Code:
#include <stdio.h>
#include <pthread.h>
#define FAILURE -1
#define MAX_THREADS 15
long int global_count[MAX_THREADS];
/* priority of each thread */
long int priority[]={1,20,40,60,80,99};
void clearGlobalCounts()
{
int i=0;
for(i=0;i<MAX_THREADS;i++)
global_count[i]=0;
}
/**
thread parameter is thread index
**/
void funcDoNothing(void *threadArgument)
{
int count=0;
int index = *((int *)threadArgument);
printf("Thread=%d\n",index);
clearGlobalCounts();
while(1)
{
count++;
if(count==100)
{
global_count[index]++;
count=0;
}
}
}
int main()
{
int i=0;
for(int i=0;i<sizeof(priority)/sizeof(long int);i++)
create_thread(funcDoNothing, i,priority[i]);
sleep(3600);
for(i=0;i<sizeof(priority)/sizeof(long int);i++)
{
printf("pid=%d count=%ld\n",getpid(),
global_count[i]);
}
}
create_thread(void *func,int thread_index,int priority)
{
pthread_attr_t attr;
struct sched_param schedParam;
void *pParm=NULL;
int id;
int * index = malloc(sizeof(int));
*index = thread_index;
void *res;
/* Initialize the thread attributes */
if (pthread_attr_init(&attr))
{
printf("Failed to initialize thread attrs\n");
return FAILURE;
}
if(pthread_attr_setschedpolicy(&attr, SCHED_FIFO))
{
printf("Failed to pthread_attr_setschedpolicy\n");
return FAILURE;
}
if (pthread_attr_setschedpolicy(&attr, SCHED_FIFO))
{
printf("Failed to setschedpolicy\n");
return FAILURE;
}
/* Set the capture thread priority */
pthread_attr_getschedparam(&attr, &schedParam);;
schedParam.sched_priority = sched_get_priority_max(SCHED_FIFO) - 1;
schedParam.sched_priority = priority;
if (pthread_attr_setschedparam(&attr, &schedParam))
{
printf("Failed to setschedparam\n");
return FAILURE;
}
pthread_create(&id, &attr, (void *)func, index);
}

The documentation for pthread_attr_setschedparam says:
In order for the parameter setting made by
pthread_attr_setschedparam() to have effect when calling
pthread_create(3), the caller must use pthread_attr_setinheritsched(3)
to set
the inherit-scheduler attribute of the attributes object attr to PTHREAD_EXPLICIT_SCHED.
So you have to call pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED) , for example:
if (pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED) != 0) {
perror("pthread_attr_setinheritsched");
}
pthread_create(&id, &attr, (void *)func, index);
Note: Your code produces a lot of compiler warnings, you need to fix those. You do not want to try to test code which have a lot of undefined behavior - as indicated by some of the warnings. You should probably lower the sleep(3600) to just a few seconds, since when you get your threads running under SCHED_FIFO, they will hog your CPU and the machine appears freezed while they are running.

Related

pthread_kill in multithread'd app causes segmentation fault

I managed to find several references to this problem which suggested pthread_kill was de-referencing the pthread_t structure which was causing some problem however other articles said this is not a problem as long as the pthread_t staructure is created via pthread_create.
Then I found a multi thread example of how to do this correctly :
How to send a signal to a process in C?
However I am still getting seg faults so here is my code example:
static pthread_t GPUthread;
static void GPUsigHandler(int signo)
{
fprintf(stderr, "Queue waking\n");
}
void StartGPUQueue()
{
sigset_t sigmask;
pthread_attr_t attr_obj; /* a thread attribute variable */
struct sigaction action;
/* set up signal mask to block all in main thread */
sigfillset(&sigmask); /* to turn on all bits */
pthread_sigmask(SIG_BLOCK, &sigmask, (sigset_t *)0);
/* set up signal handlers for SIGINT & SIGUSR1 */
action.sa_flags = 0;
action.sa_handler = GPUsigHandler;
sigaction(SIGUSR1, &action, (struct sigaction *)0);
pthread_attr_init(&attr_obj); /* init it to default */
pthread_attr_setdetachstate(&attr_obj, PTHREAD_CREATE_DETACHED);
GPUthread = pthread_create(&GPUthread, &attr_obj, ProcessGPUqueue, NULL);
if (GPUthread != 0)
{
fprintf(stderr, "Cannot start GPU thread\n");
}
}
void ProcessGPUqueue(void *ptr)
{
int sigdummy;
sigset_t sigmask;
sigfillset(&sigmask); /* will unblock all signals */
pthread_sigmask(SIG_UNBLOCK, &sigmask, (sigset_t *)0);
fprintf(stderr, "GPU queue alive\n");
while(queueActive)
{
fprintf(stderr, "Processing GPU queue\n");
while(GPUqueue != NULL)
{
// process stuff
}
sigwait(&sigmask, &sigdummy);
}
}
void QueueGPUrequest(unsigned char cmd, unsigned short p1, unsigned short p2, unsigned short p3, unsigned short p4)
{
// Add request to queue logic ...
fprintf(stderr, "About to Wake GPU queue\n");
pthread_kill(GPUthread, SIGUSR1);// Earth shattering KA-BOOM!!!
}

Trigger multiple pthreads by pthread_cond_broadcast

Since the examples for pthreads with pthread_cond_broadcast wakeup are sparse i wrote one, but are unsure if this is correctly synchronized and the way to do it:
do all threads share the same c and mtx variable?
is it necessary upon pthread_cond_wait return to test if some condition is actually met? in my case every call to broadcast should wake up every thread exactly once, but no-one else should do so. (do i prevent spurious wakeups?)
the program currently does not exit despite async cancel type. also no success with deferred cancellation tried in example code despite pthread_cond_wait being a cancellation point so.
overall does it work like i expect it to.
#include <pthread.h>
#include <iostream>
#include <unistd.h>
struct p_args{
int who;
};
pthread_cond_t c; //share between compilation units
pthread_mutex_t mtx;
void *threadFunc(void *vargs){
//pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS,NULL);
struct p_args * args = (struct p_args *) vargs;
while(true){
//wait for trigger one loop
pthread_mutex_lock(&mtx);
pthread_cond_wait(&c, &mtx);
pthread_mutex_unlock(&mtx);
//should be entangled output showing concurrent execution
std::cout << "t " << args->who << std::endl;
/* expensive work */
}
delete args;
}
int main(int argc, char* argv[])
{
pthread_cond_init(&c, NULL);
pthread_mutex_init(&mtx, NULL);
pthread_t thread_id[2];
struct p_args *args0 = new p_args();
struct p_args *args1 = new p_args();
args0->who = 0;
args1->who = 1;
pthread_create(&thread_id[0], NULL, threadFunc, args0);
pthread_create(&thread_id[1], NULL, threadFunc, args1);
sleep(3);
pthread_mutex_lock(&mtx);
pthread_cond_broadcast(&c);
pthread_mutex_unlock(&mtx);
sleep(3);//test if thread waits
pthread_cancel(thread_id[0]);
pthread_cancel(thread_id[1]);
pthread_join (thread_id[0], NULL);
pthread_join (thread_id[1], NULL);
//could perform cleanup here
return 0;
}
Regarding exiting deferred:
thread_id[0] exits fine and i am stuck in line `pthread_join (thread_id[1], NULL);`, it says (Exiting) but seems stuck on a lock, with debugger:
<br>
[![enter image description here][2]][2]
<br>
EDIT final solution i came up with:
#include <pthread.h>
#include <iostream>
#include <unistd.h>
struct p_args{
int who;
};
pthread_cond_t c;
pthread_mutex_t mtx;
bool doSome[2];
bool exitFlag;
void *threadFunc(void *vargs){
struct p_args * args = (struct p_args *) vargs;
while(true){
//wait for trigger one loop
pthread_mutex_lock(&mtx);
do {
pthread_cond_wait(&c, &mtx);
if(exitFlag) {
std::cout << "return " << args->who << std::endl;
delete args;
pthread_mutex_unlock(&mtx);
return NULL;
}
} while(doSome == false);
doSome[args->who] = false;
pthread_mutex_unlock(&mtx);
std::cout << "t " << args->who << std::endl;
}
}
int main(int argc, char* argv[])
{
pthread_cond_init(&c, NULL);
pthread_mutex_init(&mtx, NULL);
pthread_t thread_id[2];
struct p_args *args0 = new p_args();
struct p_args *args1 = new p_args();
args0->who = 0;
args1->who = 1;
doSome[0] = doSome[1] = true;
exitFlag = false;
pthread_create(&thread_id[0], NULL, threadFunc, args0);
pthread_create(&thread_id[1], NULL, threadFunc, args1);
doSome[0] = doSome[1] = true;
pthread_cond_broadcast(&c);
sleep(3);
doSome[0] = doSome[1] = true;
pthread_cond_broadcast(&c);
sleep(3);
exitFlag = true;
pthread_cond_broadcast(&c);
pthread_join (thread_id[0], NULL);
pthread_join (thread_id[1], NULL);
return 0;
}
do all threads share the same c and mtx variable?
Yes, just like any other global variable. You could print their addresses from each thread to confirm it.
is it necessary upon pthread_cond_wait return to test if some condition is actually met?
Yes, all wait interfaces are subject to spurious wakeups, and you're always responsible for checking your own predicate. See the documentation or a good book.
the program currently does not exit ...
pthread_cancel is uniformly horrible and should never be used. It's really hard to get right. If you want to tell your thread to exit, write a notification mechanism - build it into the existing predicate loop - and signal/broadcast to make sure all threads wake up and realize it's time to die.
Regarding exiting deferred: thread_id[0] exits fine and i am stuck in line pthread_join (thread_id[1], NULL);, it says (Exiting) but seems stuck on a lock
One of the hard things about pthread_cancel is cleanup. If cancellation occurs while you're holding a lock, you need to have used pthread_cleanup_push to emulate cancel-compatible RAII semantics. Otherwise the first thread may (and in this case, did) die with the mutex still locked.
In this case the second thread is trying to exit from pthread_const_wait due to cancellation, but it needs to regain the lock and can't.
The usual form of a condition variable loop is this (and a good reference book should show something similar):
void *thread(void *data)
{
struct Args *args = (struct Args *)data;
/* this lock protects both the exit and work predicates.
* It should probably be part of your argument struct,
* globals are not recommended.
* Error handling omitted for brevity,
* but you should really check the return values.
*/
pthread_mutex_lock(&args->mutex);
while (!exit_predicate(args)) {
while (!work_predicate(args)) {
/* check the return value here too */
pthread_cond_wait(&args->condition, &args->mutex);
}
/* work_predicate() is true and we have the lock */
do_work(args);
}
/* unlock (explicitly) only once.
* If you need to cope with cancellation, you do need
* pthread_cleanup_push/pop instead.
*/
pthread_mutex_unlock(&args->mutex);
return data;
}
where your custom code can just go in bool exit_predicate(struct Args*), bool work_predicate(struct Args*) and void do_work(struct Args*). The loop structure itself rarely needs much alteration.

How to prematurely kill std::async threads before they are finished *without* using a std::atomic_bool?

I have a function that takes a callback, and used it to do work on 10 separate threads. However, it is often the case that not all of the work is needed. For example, if the desired result is obtained on the third thread, it should stop all work being done on of the remaining alive threads.
This answer here suggests that it is not possible unless you have the callback functions take an additional std::atomic_bool argument, that signals whether the function should terminate prematurely.
This solution does not work for me. The workers are spun up inside a base class, and the whole point of this base class is to abstract away details of multithreading. How can I do this? I am anticipating that I will have to ditch std::async for something more involved.
#include <iostream>
#include <future>
#include <vector>
class ABC{
public:
std::vector<std::future<int> > m_results;
ABC() {};
~ABC(){};
virtual int callback(int a) = 0;
void doStuffWithCallBack();
};
void ABC::doStuffWithCallBack(){
// start working
for(int i = 0; i < 10; ++i)
m_results.push_back(std::async(&ABC::callback, this, i));
// analyze results and cancel all threads when you get the 1
for(int j = 0; j < 10; ++j){
double foo = m_results[j].get();
if ( foo == 1){
break; // but threads continue running
}
}
std::cout << m_results[9].get() << " <- this shouldn't have ever been computed\n";
}
class Derived : public ABC {
public:
Derived() : ABC() {};
~Derived() {};
int callback(int a){
std::cout << a << "!\n";
if (a == 3)
return 1;
else
return 0;
};
};
int main(int argc, char **argv)
{
Derived myObj;
myObj.doStuffWithCallBack();
return 0;
}
I'll just say that this should probably not be a part of a 'normal' program, since it could leak resources and/or leave your program in an unstable state, but in the interest of science...
If you have control of the thread loop, and you don't mind using platform features, you could inject an exception into the thread. With posix you can use signals for this, on Windows you would have to use SetThreadContext(). Though the exception will generally unwind the stack and call destructors, your thread may be in a system call or other 'non-exception safe place' when the exception occurs.
Disclaimer: I only have Linux at the moment, so I did not test the Windows code.
#if defined(_WIN32)
# define ITS_WINDOWS
#else
# define ITS_POSIX
#endif
#if defined(ITS_POSIX)
#include <signal.h>
#endif
void throw_exception() throw(std::string())
{
throw std::string();
}
void init_exceptions()
{
volatile int i = 0;
if (i)
throw_exception();
}
bool abort_thread(std::thread &t)
{
#if defined(ITS_WINDOWS)
bool bSuccess = false;
HANDLE h = t.native_handle();
if (INVALID_HANDLE_VALUE == h)
return false;
if (INFINITE == SuspendThread(h))
return false;
CONTEXT ctx;
ctx.ContextFlags = CONTEXT_CONTROL;
if (GetThreadContext(h, &ctx))
{
#if defined( _WIN64 )
ctx.Rip = (DWORD)(DWORD_PTR)throw_exception;
#else
ctx.Eip = (DWORD)(DWORD_PTR)throw_exception;
#endif
bSuccess = SetThreadContext(h, &ctx) ? true : false;
}
ResumeThread(h);
return bSuccess;
#elif defined(ITS_POSIX)
pthread_kill(t.native_handle(), SIGUSR2);
#endif
return false;
}
#if defined(ITS_POSIX)
void worker_thread_sig(int sig)
{
if(SIGUSR2 == sig)
throw std::string();
}
#endif
void init_threads()
{
#if defined(ITS_POSIX)
struct sigaction sa;
sigemptyset(&sa.sa_mask);
sa.sa_flags = 0;
sa.sa_handler = worker_thread_sig;
sigaction(SIGUSR2, &sa, 0);
#endif
}
class tracker
{
public:
tracker() { printf("tracker()\n"); }
~tracker() { printf("~tracker()\n"); }
};
int main(int argc, char *argv[])
{
init_threads();
printf("main: starting thread...\n");
std::thread t([]()
{
try
{
tracker a;
init_exceptions();
printf("thread: started...\n");
std::this_thread::sleep_for(std::chrono::minutes(1000));
printf("thread: stopping...\n");
}
catch(std::string s)
{
printf("thread: exception caught...\n");
}
});
printf("main: sleeping...\n");
std::this_thread::sleep_for(std::chrono::seconds(2));
printf("main: aborting...\n");
abort_thread(t);
printf("main: joining...\n");
t.join();
printf("main: exiting...\n");
return 0;
}
Output:
main: starting thread...
main: sleeping...
tracker()
thread: started...
main: aborting...
main: joining...
~tracker()
thread: exception caught...
main: exiting...

Linux - Control Flow in a linux kernel module

I am learning to write kernel modules and in one of the examples I had to make sure that a thread executed 10 times and exits, so I wrote this according to what I have studied:
#include <linux/module.h>
#include <linux/kthread.h>
struct task_struct *ts;
int flag = 0;
int id = 10;
int function(void *data) {
int n = *(int*)data;
set_current_state(TASK_INTERRUPTIBLE);
schedule_timeout(n*HZ); // after doing this it executed infinitely and i had to reboot
while(!kthread_should_stop()) {
printk(KERN_EMERG "Ding");
}
flag = 1;
return 0;
}
int init_module (void) {
ts = kthread_run(function, (void *)&id, "spawn");
return 0;
}
void cleanup_module(void) {
if (flag==1) { return; }
else { if (ts != NULL) kthread_stop(ts);
}
return;
}
MODULE_LICENSE("GPL");
What I want to know is :
a) How to make thread execute 10 times like a loop
b) How does the control flows in these kind of processes that is if we make it to execute 10 times then does it go back and forth between function and cleanup_module or init_module or what exactly happens?
If you control kthread with kthread_stop, the kthread shouldn't exit until be ing stopped (see also that answer). So, after executing all operations, kthread should wait until stopped.
Kernel already implements kthread_worker mechanism, when kthread just executes works, added to it.
DEFINE_KTHREAD_WORKER(worker);
struct my_work
{
struct kthread_work *work; // 'Base' class
int n;
};
void do_work(struct kthread_work *work)
{
struct my_work* w = container_of(work, struct my_work, work);
printk(KERN_EMERG "Ding %d", w->n);
// And free work struct at the end
kfree(w);
}
int init_module (void) {
int i;
for(i = 0; i < 10; i++)
{
struct my_work* w = kmalloc(sizeof(struct my_work), GFP_KERNEL);
init_kthread_work(&w->work, &do_work);
w->n = i + 1;
queue_kthread_work(&worker, &w->work);
}
ts = kthread_run(&kthread_worker_fn, &worker, "spawn");
return 0;
}
void cleanup_module(void) {
kthread_stop(ts);
}

Differences between POSIX threads on OSX and LINUX?

Can anyone shed light on the reason that when the below code is compiled and run on OSX the 'bartender' thread skips through the sem_wait() in what seems like a random manner and yet when compiled and run on a Linux machine the sem_wait() holds the thread until the relative call to sem_post() is made, as would be expected?
I am currently learning not only POSIX threads but concurrency as a whole so absoutely any comments, tips and insights are warmly welcomed...
Thanks in advance.
#include <stdio.h>
#include <stdlib.h>
#include <semaphore.h>
#include <fcntl.h>
#include <unistd.h>
#include <pthread.h>
#include <errno.h>
//using namespace std;
#define NSTUDENTS 30
#define MAX_SERVINGS 100
void* student(void* ptr);
void get_serving(int id);
void drink_and_think();
void* bartender(void* ptr);
void refill_barrel();
// This shared variable gives the number of servings currently in the barrel
int servings = 10;
// Define here your semaphores and any other shared data
sem_t *mutex_stu;
sem_t *mutex_bar;
int main() {
static const char *semname1 = "Semaphore1";
static const char *semname2 = "Semaphore2";
pthread_t tid;
mutex_stu = sem_open(semname1, O_CREAT, 0777, 0);
if (mutex_stu == SEM_FAILED)
{
fprintf(stderr, "%s\n", "ERROR creating semaphore semname1");
exit(EXIT_FAILURE);
}
mutex_bar = sem_open(semname2, O_CREAT, 0777, 1);
if (mutex_bar == SEM_FAILED)
{
fprintf(stderr, "%s\n", "ERROR creating semaphore semname2");
exit(EXIT_FAILURE);
}
pthread_create(&tid, NULL, bartender, &tid);
for(int i=0; i < NSTUDENTS; ++i) {
pthread_create(&tid, NULL, student, &tid);
}
pthread_join(tid, NULL);
sem_unlink(semname1);
sem_unlink(semname2);
printf("Exiting the program...\n");
}
//Called by a student process. Do not modify this.
void drink_and_think() {
// Sleep time in milliseconds
int st = rand() % 10;
sleep(st);
}
// Called by a student process. Do not modify this.
void get_serving(int id) {
if (servings > 0) {
servings -= 1;
} else {
servings = 0;
}
printf("ID %d got a serving. %d left\n", id, servings);
}
// Called by the bartender process.
void refill_barrel()
{
servings = 1 + rand() % 10;
printf("Barrel refilled up to -> %d\n", servings);
}
//-- Implement a synchronized version of the student
void* student(void* ptr) {
int id = *(int*)ptr;
printf("Started student %d\n", id);
while(1) {
sem_wait(mutex_stu);
if(servings > 0) {
get_serving(id);
} else {
sem_post(mutex_bar);
continue;
}
sem_post(mutex_stu);
drink_and_think();
}
return NULL;
}
//-- Implement a synchronized version of the bartender
void* bartender(void* ptr) {
int id = *(int*)ptr;
printf("Started bartender %d\n", id);
//sleep(5);
while(1) {
sem_wait(mutex_bar);
if(servings <= 0) {
refill_barrel();
} else {
printf("Bar skipped sem_wait()!\n");
}
sem_post(mutex_stu);
}
return NULL;
}
The first time you run the program, you're creating named semaphores with initial values, but since your threads never exit (they're infinite loops), you never get to the sem_unlink calls to delete those semaphores. If you kill the program (with ctrl-C or any other way), the semaphores will still exist in whatever state they are in. So if you run the program again, the sem_open calls will succeed (because you don't use O_EXCL), but they won't reset the semaphore value or state, so they might be in some odd state.
So you should make sure to call sem_unlink when the program STARTS, before calling sem_open. Better yet, don't use named semaphores at all -- use sem_init to initialize a couple of unnamed semaphores instead.

Resources