In Solaris, thr_join documentation states the following:
int thr_join(thread_t thread, thread_t *departed, void
**status);
If the target thread ID is 0, thr_join() finds and returns
the status of a terminated undetached thread in the process.
Is POSIX pthread_join equivalent?
int pthread_join(pthread_t thread, void **status);
suspends processing of the calling thread until the target thread completes
How can I use pthread_join in case of thr_join when I would like to know which child thread have terminated among many.
Is there any other alternative?
In other words, if a parent thread spawns N child threads, how do the parent thread know by polling or something else which thread has exited / terminated?
Is POSIX pthread_join equivalent?
Yes, it's equivalent. Well, close enough. You can see the differences in the implementation:
int
thr_join(thread_t tid, thread_t *departed, void **status)
{
int error = _thrp_join(tid, departed, status, 1);
return ((error == EINVAL)? ESRCH : error);
}
/*
* pthread_join() differs from Solaris thr_join():
* It does not return the departed thread's id
* and hence does not have a "departed" argument.
* It returns EINVAL if tid refers to a detached thread.
*/
#pragma weak _pthread_join = pthread_join
int
pthread_join(pthread_t tid, void **status)
{
return ((tid == 0)? ESRCH : _thrp_join(tid, NULL, status, 1));
}
They're even implemented using the same internal function.
But you don't want to use Solaris threads. Just use POSIX threads.
Related
I'm reading Advanced Programming in the Unix Environment 3rd Edn, ยง7.3, Process Termination, the following statement make me confused:
There are eight ways for a process to terminate. Normal termination
occurs in five ways:
Return from main
Calling exit
Calling _exit or _Exit
Return of the last thread from its start routine (Section 11.5)
Calling pthread_exit (Section 11.5) from the last thread
for
Return of the last thread from its start routine (Section 11.5)
Calling pthread_exit (Section 11.5) from the last thread
I don't think a process will terminate if it is not returned form main function even though the last thread in this process is terminated, am I right? If not, why 4 and 5 are right?
The main thread is one of the threads. For example, in
void *start(void *arg) {
sleep(1);
pthread_exit(0);
}
int main() {
pthread_t t;
pthread_create(&t, 0, start, 0);
pthread_exit(0);
}
the main thread exits immediately, but the process continues running until the last thread has exited. This is true the other way around,
void *start(void *arg) {
pthread_exit(0);
}
int main() {
pthread_t t;
pthread_create(&t, 0, start, 0);
sleep(1);
pthread_exit(0);
}
where the main thread is the last one left.
The main question is: How we can wait for a thread in Linux kernel to complete? I have seen a few post concerned about proper way of handling threads in Linux kernel but i'm not sure how we can wait for a single thread in the main thread to be completed (suppose we need the thread[3] be done then proceed):
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/sched.h>
#include <linux/kthread.h>
#include <linux/slab.h>
void *func(void *arg) {
// doing something
return NULL;
}
int init_module(void) {
struct task_struct* thread[5];
int i;
for(i=0; i<5; i++) {
thread[i] = kthread_run(func, (void*) arg, "Creating thread");
}
return 0;
}
void cleanup_module(void) {
printk("cleaning up!\n");
}
AFAIK there is no equivalent of pthread_join() in kernel. Also, I feel like your pattern (of starting bunch of threads and waiting only for one of them) is not really common in kernel. That being said, there kernel does have few synchronization mechanism that may be used to accomplish your goal.
Note that those mechanisms will not guarantee that the thread finished, they will only let main thread know that they finished doing the work they were supposed to do. It may still take some time to really stop this tread and free all resources.
Semaphores
You can create a locked semaphore, then call down in your main thread. This will put it to sleep. Then you will up this semaphore inside of your thread just before exiting. Something like:
struct semaphore sem;
int func(void *arg) {
struct semaphore *sem = (struct semaphore*)arg; // you could use global instead
// do something
up(sem);
return 0;
}
int init_module(void) {
// some initialization
init_MUTEX_LOCKED(&sem);
kthread_run(&func, (void*) &sem, "Creating thread");
down(&sem); // this will block until thread runs up()
}
This should work but is not the most optimal solution. I mention this as it's a known pattern that is also used in userspace. Semaphores in kernel are designed for cases where it's mostly available and this case has high contention. So a similar mechanism optimized for this case was created.
Completions
You can declare completions using:
struct completion comp;
init_completion(&comp);
or:
DECLARE_COMPLETION(comp);
Then you can use wait_for_completion(&comp); instead of down() to wait in main thread and complete(&comp); instead of up() in your thread.
Here's the full example:
DECLARE_COMPLETION(comp);
struct my_data {
int id;
struct completion *comp;
};
int func(void *arg) {
struct my_data *data = (struct my_data*)arg;
// doing something
if (data->id == 3)
complete(data->comp);
return 0;
}
int init_module(void) {
struct my_data *data[] = kmalloc(sizeof(struct my_data)*N, GFP_KERNEL);
// some initialization
for (int i=0; i<N; i++) {
data[i]->comp = ∁
data[i]->id = i;
kthread_run(func, (void*) data[i], "my_thread%d", i);
}
wait_for_completion(&comp); // this will block until some thread runs complete()
}
Multiple threads
I don't really see why you would start 5 identical threads and only want to wait for 3rd one but of course you could send different data to each thread, with a field describing it's id, and then call up or complete only if this id equals 3. That's shown in the completion example. There are other ways to do this, this is just one of them.
Word of caution
Go read some more about those mechanisms before using any of them. There are some important details I did not write about here. Also those examples are simplified and not tested, they are here just to show the overall idea.
kthread_stop() is a kernel's way for wait thread to end.
Aside from waiting, kthread_stop() also sets should_stop flag for waited thread and wake up it, if needed. It is usefull for threads which repeat some actions infinitely.
As for single-shot tasks, it is usually simpler to use works for them, instead of kthreads.
EDIT:
Note: kthread_stop() can be called only when kthread(task_struct) structure is not freed.
Either thread function should return only after it found kthread_should_stop() return true, or get_task_struct() should be called before start thread (and put_task_struct() should be called after kthread_stop()).
I am not sure if my question is correct, but I have the following example, where the main thread creates two additional threads.
Since I am not using join command at the end of the main, it will continue execution and in the same time, the two created threads will work in parallel. But since the main is terminated before they finish their execution, I am getting the following output:
terminate called without an active exception
Aborted (core dumped)
Here's the code:
#include <iostream> // std::cout
#include <thread> // std::thread
#include <chrono>
void foo()
{
std::chrono::milliseconds dura( 2000 );
std::this_thread::sleep_for( dura );
std::cout << "Waited for 2Sec\n";
}
void bar(int x)
{
std::chrono::milliseconds dura( 4000 );
std::this_thread::sleep_for( dura );
std::cout << "Waited for 4Sec\n";
}
int main()
{
std::thread first (foo);
std::thread second (bar,0);
return 0;
}
So my question is how to keep these two threads working even if the main thread terminated?
I am asking this because in my main program, I have an event handler ,and for each event I create a corresponding thread. But the main problem when the handler creates a new thread, the handler will continue execution. Until it is destroyed which will cause also the newly created thread to be destroyed. So my question is how to keep the thread alive in this case?
Also if I use a join it will convert back to serialization.
void ho_commit_indication_handler(message &msg, const boost::system::error_code &ec)
{
.....
}
void event_handler(message &msg, const boost::system::error_code &ec)
{
if (ec)
{
log_(0, __FUNCTION__, " error: ", ec.message());
return;
}
switch (msg.mid())
{
case n2n_ho_commit:
{
boost::thread thrd(&ho_commit_indication_handler, boost::ref(msg), boost::ref(ec));
}
break
}
};
Thanks a lot.
Keeping the threads alive is a bad idea, because it causes a call to std::terminate. You should definitively join the threads:
int main()
{
std::thread first (foo);
std::thread second (bar, 0);
first.join();
second.join();
}
An alternative is to detach the threads. However you still need to assert that the main thread lives longer (by e.g. using a mutex / condition_variable).
This excerpt from the C++11 standard is relevant here:
15.5.1 The std::terminate() function [except.terminate]
1 In some situations exception handling must be abandoned for less subtle error
handling techniques. [ Note: These situations are:
[...]
-- when the destructor or the copy assignment operator is invoked on an
object of type std::thread that refers to a joinable thread
Hence, you have to call either join or detach on threads before scope exit.
Concerning your edit: You have to store the threads in a list (or similar) and wait for every one of them before main is done. A better idea would be to use a thread pool (because this limits the total number of threads created).
Using pthreads in linux 2.6.30 I am trying to send a single signal which will cause multiple threads to begin execution. The broadcast seems to only be received by one thread. I have tried both pthread_cond_signal and pthread cond_broadcast and both seem to have the same behavior. For the mutex in pthread_cond_wait, I have tried both common mutexes and separate (local) mutexes with no apparent difference.
worker_thread(void *p)
{
// setup stuff here
printf("Thread %d ready for action \n", p->thread_no);
pthread_cond_wait(p->cond_var, p->mutex);
printf("Thread %d off to work \n", p->thread_no);
// work stuff
}
dispatch_thread(void *p)
{
// setup stuff
printf("Wakeup, everyone ");
pthread_cond_broadcast(p->cond_var);
printf("everyone should be working \n");
// more stuff
}
main()
{
pthread_cond_init(cond_var);
for (i=0; i!=num_cores; i++) {
pthread_create(worker_thread...);
}
pthread_create(dispatch_thread...);
}
Output:
Thread 0 ready for action
Thread 1 ready for action
Thread 2 ready for action
Thread 3 ready for action
Wakeup, everyone
everyone should be working
Thread 0 off to work
What's a good way to send signals to all the threads?
First off, you should have the mutex locked at the point where you call pthread_cond_wait(). It's generally a good idea to hold the mutex when you call pthread_cond_broadcast(), as well.
Second off, you should loop calling pthread_cond_wait() while the wait condition is true. Spurious wakeups can happen, and you must be able to handle them.
Finally, your actual problem: you are signaling all threads, but some of them aren't waiting yet when the signal is sent. Your main thread and dispatch thread are racing your worker threads: if the main thread can launch the dispatch thread, and the dispatch thread can grab the mutex and broadcast on it before the worker threads can, then those worker threads will never wake up.
You need a synchronization point prior to signaling where you wait to signal till all threads are known to be waiting for the signal. That, or you can keep signaling till you know all threads have been woken up.
In this case, you could use the mutex to protect a count of sleeping threads. Each thread grabs the mutex and increments the count. If the count matches the count of worker threads, then it's the last thread to increment the count and so signals on another condition variable sharing the same mutex to the sleeping dispatch thread that all threads are ready. The thread then waits on the original condition, which causes it release the mutex.
If the dispatch thread wasn't sleeping yet when the last worker thread signals on that condition, it will find that the count already matches the desired count and not bother waiting, but immediately broadcast on the shared condition to wake workers, who are now guaranteed to all be sleeping.
Anyway, here's some working source code that fleshes out your sample code and includes my solution:
#include <stdio.h>
#include <pthread.h>
#include <err.h>
static const int num_cores = 8;
struct sync {
pthread_mutex_t *mutex;
pthread_cond_t *cond_var;
int thread_no;
};
static int sleeping_count = 0;
static pthread_cond_t all_sleeping_cond = PTHREAD_COND_INITIALIZER;
void *
worker_thread(void *p_)
{
struct sync *p = p_;
// setup stuff here
pthread_mutex_lock(p->mutex);
printf("Thread %d ready for action \n", p->thread_no);
sleeping_count += 1;
if (sleeping_count >= num_cores) {
/* Last worker to go to sleep. */
pthread_cond_signal(&all_sleeping_cond);
}
int err = pthread_cond_wait(p->cond_var, p->mutex);
if (err) warnc(err, "pthread_cond_wait");
printf("Thread %d off to work \n", p->thread_no);
pthread_mutex_unlock(p->mutex);
// work stuff
return NULL;
}
void *
dispatch_thread(void *p_)
{
struct sync *p = p_;
// setup stuff
pthread_mutex_lock(p->mutex);
while (sleeping_count < num_cores) {
pthread_cond_wait(&all_sleeping_cond, p->mutex);
}
printf("Wakeup, everyone ");
int err = pthread_cond_broadcast(p->cond_var);
if (err) warnc(err, "pthread_cond_broadcast");
printf("everyone should be working \n");
pthread_mutex_unlock(p->mutex);
// more stuff
return NULL;
}
int
main(void)
{
pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t cond_var = PTHREAD_COND_INITIALIZER;
pthread_t worker[num_cores];
struct sync info[num_cores];
for (int i = 0; i < num_cores; i++) {
struct sync *p = &info[i];
p->mutex = &mutex;
p->cond_var = &cond_var;
p->thread_no = i;
pthread_create(&worker[i], NULL, worker_thread, p);
}
pthread_t dispatcher;
struct sync p = {&mutex, &cond_var, num_cores};
pthread_create(&dispatcher, NULL, dispatch_thread, &p);
pthread_exit(NULL);
/* not reached */
return 0;
}
I'm trying to write a simple thread pool program in pthread. However, it seems that pthread_cond_signal doesn't block, which creates a problem. For example, let's say I have a "producer-consumer" program:
pthread_cond_t my_cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t my_cond_m = PTHREAD_MUTEX_INITIALIZER;
void * liberator(void * arg)
{
// XXX make sure he is ready to be freed
sleep(1);
pthread_mutex_lock(&my_cond_m);
pthread_cond_signal(&my_cond);
pthread_mutex_unlock(&my_cond_m);
return NULL;
}
int main()
{
pthread_t t1;
pthread_create(&t1, NULL, liberator, NULL);
// XXX Don't take too long to get ready. Otherwise I'll miss
// the wake up call forever
//sleep(3);
pthread_mutex_lock(&my_cond_m);
pthread_cond_wait(&my_cond, &my_cond_m);
pthread_mutex_unlock(&my_cond_m);
pthread_join(t1, NULL);
return 0;
}
As described in the two XXX marks, if I take away the sleep calls, then main() may stall because it has missed the wake up call from liberator(). Of course, sleep isn't a very robust way to ensure that either.
In real life situation, this would be a worker thread telling the manager thread that it is ready for work, or the manager thread announcing that new work is available.
How would you do this reliably in pthread?
Elaboration
#Borealid's answer kind of works, but his explanation of the problem could be better. I suggest anyone looking at this question to read the discussion in the comments to understand what's going on.
In particular, I myself would amend his answer and code example like this, to make this clearer. (Since Borealid's original answer, while compiled and worked, confused me a lot)
// In main
pthread_mutex_lock(&my_cond_m);
// If the flag is not set, it means liberator has not
// been run yet. I'll wait for him through pthread's signaling
// mechanism
// If it _is_ set, it means liberator has been run. I'll simply
// skip waiting since I've already synchronized. I don't need to
// use pthread's signaling mechanism
if(!flag) pthread_cond_wait(&my_cond, &my_cond_m);
pthread_mutex_unlock(&my_cond_m);
// In liberator thread
pthread_mutex_lock(&my_cond_m);
// Signal anyone who's sleeping. If no one is sleeping yet,
// they should check this flag which indicates I have already
// sent the signal. This is needed because pthread's signals
// is not like a message queue -- a sent signal is lost if
// nobody's waiting for a condition when it's sent.
// You can think of this flag as a "persistent" signal
flag = 1;
pthread_cond_signal(&my_cond);
pthread_mutex_unlock(&my_cond_m);
Use a synchronization variable.
In main:
pthread_mutex_lock(&my_cond_m);
while (!flag) {
pthread_cond_wait(&my_cond, &my_cond_m);
}
pthread_mutex_unlock(&my_cond_m);
In the thread:
pthread_mutex_lock(&my_cond_m);
flag = 1;
pthread_cond_broadcast(&my_cond);
pthread_mutex_unlock(&my_cond_m);
For a producer-consumer problem, this would be the consumer sleeping when the buffer is empty, and the producer sleeping when it is full. Remember to acquire the lock before accessing the global variable.
I found out the solution here. For me, the tricky bit to understand the problem is that:
Producers and consumers must be able to communicate both ways. Either way is not enough.
This two-way communication can be packed into one pthread condition.
To illustrate, the blog post mentioned above demonstrated that this is actually meaningful and desirable behavior:
pthread_mutex_lock(&cond_mutex);
pthread_cond_broadcast(&cond):
pthread_cond_wait(&cond, &cond_mutex);
pthread_mutex_unlock(&cond_mutex);
The idea is that if both the producers and consumers employ this logic, it will be safe for either of them to be sleeping first, since the each will be able to wake the other role up. Put it in another way, in a typical producer-consumer sceanrio -- if a consumer needs to sleep, it's because a producer needs to wake up, and vice versa. Packing this logic in a single pthread condition makes sense.
Of course, the above code has the unintended behavior that a worker thread will also wake up another sleeping worker thread when it actually just wants to wake the producer. This can be solved by a simple variable check as #Borealid suggested:
while(!work_available) pthread_cond_wait(&cond, &cond_mutex);
Upon a worker broadcast, all worker threads will be awaken, but one-by-one (because of the implicit mutex locking in pthread_cond_wait). Since one of the worker threads will consume the work (setting work_available back to false), when other worker threads awake and actually get to work, the work will be unavailable so the worker will sleep again.
Here's some commented code I tested, for anyone interested:
// gcc -Wall -pthread threads.c -lpthread
#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
#include <assert.h>
pthread_cond_t my_cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t my_cond_m = PTHREAD_MUTEX_INITIALIZER;
int * next_work = NULL;
int all_work_done = 0;
void * worker(void * arg)
{
int * my_work = NULL;
while(!all_work_done)
{
pthread_mutex_lock(&my_cond_m);
if(next_work == NULL)
{
// Signal producer to give work
pthread_cond_broadcast(&my_cond);
// Wait for work to arrive
// It is wrapped in a while loop because the condition
// might be triggered by another worker thread intended
// to wake up the producer
while(!next_work && !all_work_done)
pthread_cond_wait(&my_cond, &my_cond_m);
}
// Work has arrived, cache it locally so producer can
// put in next work ASAP
my_work = next_work;
next_work = NULL;
pthread_mutex_unlock(&my_cond_m);
if(my_work)
{
printf("Worker %d consuming work: %d\n", (int)(pthread_self() % 100), *my_work);
free(my_work);
}
}
return NULL;
}
int * create_work()
{
int * ret = (int *)malloc(sizeof(int));
assert(ret);
*ret = rand() % 100;
return ret;
}
void * producer(void * arg)
{
int i;
for(i = 0; i < 10; i++)
{
pthread_mutex_lock(&my_cond_m);
while(next_work != NULL)
{
// There's still work, signal a worker to pick it up
pthread_cond_broadcast(&my_cond);
// Wait for work to be picked up
pthread_cond_wait(&my_cond, &my_cond_m);
}
// No work is available now, let's put work on the queue
next_work = create_work();
printf("Producer: Created work %d\n", *next_work);
pthread_mutex_unlock(&my_cond_m);
}
// Some workers might still be waiting, release them
pthread_cond_broadcast(&my_cond);
all_work_done = 1;
return NULL;
}
int main()
{
pthread_t t1, t2, t3, t4;
pthread_create(&t1, NULL, worker, NULL);
pthread_create(&t2, NULL, worker, NULL);
pthread_create(&t3, NULL, worker, NULL);
pthread_create(&t4, NULL, worker, NULL);
producer(NULL);
pthread_join(t1, NULL);
pthread_join(t2, NULL);
pthread_join(t3, NULL);
pthread_join(t4, NULL);
return 0;
}