Finding out which threads own a pthread_rwlock_t - multithreading

How can I see (on linux) which threads own a pthread_rwlock_t (or std::shared_mutex) ?
For a regular mutex there's Is it possible to determine the thread holding a mutex? but how to do this for a r/w lock?

Your good question has a few problems with a complete answer:
It is dependent upon which OS kernel you are running, and possibly even which version.
It is dependent upon which libc/libpthread you are using, and possibly even which compiler.
Presuming you can untangle a specific configuration above, we are looking at two different outcomes: there is a single writer, or a set of readers who currently cause pthread_rw_lock() to block for some callers.
In contrast, a mutex has only ever one owner and only that owner can release it.
So a first test in finding out whether your system has a findable owner set is to see if it actually records the ownership. A quick hack to this would be have one thread acquire the rwlock for read or write, then signal a second one to release it. If the release fails, your implementation properly records it, and you have a chance; if not, your implementation likely implements rwlocks with mutex + condvar + counter; so it has no idea of ownership.
So, some code:
#include <stdio.h>
#include <pthread.h>
pthread_mutex_t lock;
pthread_cond_t cv;
pthread_rwlock_t rwl;
int ready;
void *rlse(void *_)
{
int *t = _;
pthread_mutex_lock(&lock);
while (ready == 0) {
pthread_cond_wait(&cv, &lock);
}
ready = 0;
pthread_cond_signal(&cv);
pthread_mutex_unlock(&lock);
*t = pthread_rwlock_unlock(&rwl);
return 0;
}
void *get(void *_)
{
int *t = _;
*t = (*t) ? pthread_rwlock_wrlock(&rwl) : pthread_rwlock_rdlock(&rwl);
if (*t == 0) {
pthread_mutex_lock(&lock);
ready = 1;
pthread_cond_signal(&cv);
while (ready == 1) {
pthread_cond_wait(&cv, &lock);
}
pthread_mutex_unlock(&lock);
}
return 0;
}
int main()
{
pthread_t acq, rel;
int v0, v1;
int i;
for (i = 0; i < 2; i++) {
pthread_rwlock_init(&rwl, 0);
pthread_mutex_init(&lock, 0);
pthread_cond_init(&cv, 0);
v0 = i;
pthread_create(&acq, 0, get, &v0);
pthread_create(&rel, 0, rlse, &v1);
pthread_join(acq, 0);
pthread_join(rel, 0);
printf("%s: %d %d\n", i ? "write" : "read", v0, v1);
}
return 0;
}
which we run as:
u18:src $ cc rw.c -lpthread -o rw
u18:src $ ./rw
read: 0 0
write: 0 0
This is telling us that in either case (rdlock, wrlock), a thread different from the calling thread can release the rwlock, thus it fundamentally has no owner.
With a little less sense of discovery, we could have found out a bit by reading the manpage for pthread_rwlock_unlock, which states that this condition is undefined behavior, which is the great cop-out.
Posix establishes a base, not a limit, so it is possible your implementation can support this sort of ownership. I program like the one above is a good investigative tool; if it turns up something like ENOTOWNER; but EINVAL would be pretty non-committal.
The innards of glic's rwlock (sysdeps/htl/bits/types/struct___pthread_rwlock.h):
struct __pthread_rwlock
{
__pthread_spinlock_t __held;
__pthread_spinlock_t __lock;
int __readers;
struct __pthread *__readerqueue;
struct __pthread *__writerqueue;
struct __pthread_rwlockattr *__attr;
void *__data;
};
confirms our suspicion; at first I was hopeful, with the queues and all, but a little diving through the code revealed them to be the waiting lists, not the owner lists.
The above was run on ubuntu 18.04; linux 4.15.0-112-generic; gcc 7.5.0; glibc 2.27 libpthread 2.27.

Related

Danger of using std::sort with vector of shared_ptr in multi-threaded world

This is simplified version(C++11) of issue I am facing when I upgraded an app to multithreaded world. Essentially I have vector of shared_ptr and I am doing std::sort on it. When multiple threads try to sort it, I can understand, its dangerous as while sorting, first time, iterators may have to move around. But, here, I already have a sorted vector . Now calling, std::sort on it shouldn't impose any trouble(that's what I thought as nothing needs to move) but it's crashing, randomly.( now why I call std::sort on a sorted container, actually, in original code, data is unsorted, but that doesn't matter for end result it seems). Here is sample code
#include <iostream>
#include <thread>
#include <vector>
#include <boost/shared_ptr.hpp>
const int MAX = 4;
#define LOOP_COUNT 200
struct Container {
int priority;
Container(int priority_)
: priority( priority_)
{}
};
struct StrategySorter {
int operator()( const boost::shared_ptr<Container>& v1_,
const boost::shared_ptr<Container>& v2_ )
{
return v1_->priority > v2_->priority;
}
};
std::vector<boost::shared_ptr<Container>> _creators;
void func() {
for(int i=0; i < LOOP_COUNT; ++i) {
std::sort( _creators.begin(), _creators.end(), StrategySorter() );
}
}
int main()
{
int priority[] = {100, 245, 312, 423, 597, 656, 732 };
size_t size = sizeof(priority)/sizeof(int);
for(int i=0; i < size; ++i)
{
_creators.push_back(boost::shared_ptr<Container>(new Container(priority[i])));
}
std::thread t[MAX];
for(int i=0;i < MAX; i++)
{
t[i] = std::thread(func);
}
for(int i=0;i < MAX; i++)
{
t[i].join();
}
}
Error :
../boost_1_56_0/include/boost/smart_ptr/shared_ptr.hpp:648: typename boost::detail::sp_member_access::type boost::shared_ptr::operator->() const [with T = Container; typename boost::detail::sp_member_access::type = Container*]: Assertion `px != 0' failed.
Having raw pointers doesn't crash it, so it's specific to shared_ptr.
Protecting std::sort with mutex is preventing crash.
I am not able to understand why this scenario should result into inconsistent behavior.
When more than one thread accesses the same data without synchronisation and at least one of them is doing a modifying operation, it is a race condition and as such, Undefined Behaviour. Anything can happen.
std::sort requires mutable iterators to operate, so it is by definition a modifying operation, therefore applying it concurrently to overlapping ranges withough synchronisation is a race condition (and thus UB).
There is no guarantee that a sort that ends up not moving elements will not write.
It could want to move pivots around, sort some stuff backwards in an intermediate stage, or even call swap(a,a) without a self-check optimization (as the check might be more expensive than the swap).
In any case, an operation that if it doesn't do nothing is UB is a horrible operation to invoke.
Here is a sort guaranteed to do nothing if nothing is to be done:
template<class C, class Cmp>
void my_sort( C& c, Cmp cmp ) {
using std::begin; using std::end;
if (std::is_sorted( begin(c), end(c), cmp ))
return;
std::sort( begin(c), end(c), cmp );
}
but I wouldn't use it.

How to join a thread in Linux kernel?

The main question is: How we can wait for a thread in Linux kernel to complete? I have seen a few post concerned about proper way of handling threads in Linux kernel but i'm not sure how we can wait for a single thread in the main thread to be completed (suppose we need the thread[3] be done then proceed):
#include <linux/kernel.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <linux/sched.h>
#include <linux/kthread.h>
#include <linux/slab.h>
void *func(void *arg) {
// doing something
return NULL;
}
int init_module(void) {
struct task_struct* thread[5];
int i;
for(i=0; i<5; i++) {
thread[i] = kthread_run(func, (void*) arg, "Creating thread");
}
return 0;
}
void cleanup_module(void) {
printk("cleaning up!\n");
}
AFAIK there is no equivalent of pthread_join() in kernel. Also, I feel like your pattern (of starting bunch of threads and waiting only for one of them) is not really common in kernel. That being said, there kernel does have few synchronization mechanism that may be used to accomplish your goal.
Note that those mechanisms will not guarantee that the thread finished, they will only let main thread know that they finished doing the work they were supposed to do. It may still take some time to really stop this tread and free all resources.
Semaphores
You can create a locked semaphore, then call down in your main thread. This will put it to sleep. Then you will up this semaphore inside of your thread just before exiting. Something like:
struct semaphore sem;
int func(void *arg) {
struct semaphore *sem = (struct semaphore*)arg; // you could use global instead
// do something
up(sem);
return 0;
}
int init_module(void) {
// some initialization
init_MUTEX_LOCKED(&sem);
kthread_run(&func, (void*) &sem, "Creating thread");
down(&sem); // this will block until thread runs up()
}
This should work but is not the most optimal solution. I mention this as it's a known pattern that is also used in userspace. Semaphores in kernel are designed for cases where it's mostly available and this case has high contention. So a similar mechanism optimized for this case was created.
Completions
You can declare completions using:
struct completion comp;
init_completion(&comp);
or:
DECLARE_COMPLETION(comp);
Then you can use wait_for_completion(&comp); instead of down() to wait in main thread and complete(&comp); instead of up() in your thread.
Here's the full example:
DECLARE_COMPLETION(comp);
struct my_data {
int id;
struct completion *comp;
};
int func(void *arg) {
struct my_data *data = (struct my_data*)arg;
// doing something
if (data->id == 3)
complete(data->comp);
return 0;
}
int init_module(void) {
struct my_data *data[] = kmalloc(sizeof(struct my_data)*N, GFP_KERNEL);
// some initialization
for (int i=0; i<N; i++) {
data[i]->comp = &comp;
data[i]->id = i;
kthread_run(func, (void*) data[i], "my_thread%d", i);
}
wait_for_completion(&comp); // this will block until some thread runs complete()
}
Multiple threads
I don't really see why you would start 5 identical threads and only want to wait for 3rd one but of course you could send different data to each thread, with a field describing it's id, and then call up or complete only if this id equals 3. That's shown in the completion example. There are other ways to do this, this is just one of them.
Word of caution
Go read some more about those mechanisms before using any of them. There are some important details I did not write about here. Also those examples are simplified and not tested, they are here just to show the overall idea.
kthread_stop() is a kernel's way for wait thread to end.
Aside from waiting, kthread_stop() also sets should_stop flag for waited thread and wake up it, if needed. It is usefull for threads which repeat some actions infinitely.
As for single-shot tasks, it is usually simpler to use works for them, instead of kthreads.
EDIT:
Note: kthread_stop() can be called only when kthread(task_struct) structure is not freed.
Either thread function should return only after it found kthread_should_stop() return true, or get_task_struct() should be called before start thread (and put_task_struct() should be called after kthread_stop()).

File segment/section/record locks in Linux threads

I have a multi-threaded process where a file is shared (read and written) by multiple threads. Is there any way a thread can lock one file segment so that other threads cannot access it?
I have tried fcntl(fd, F_SETLKW, &flock), but this lock only works for processes, not threads (a lock is shared between all threads in an process).
Yes - but not with the same mechanism. You'll have to use something like pthread mutexes, and keep track of the bookkeeping yourself.
Possible outline for how to make this work
Wait on and claim a process-level mutex over a bookkeeping structure
make sure no other threads within your process are trying to use that segment
mark yourself as using the file segment
Release the process-level mutex
Grab fnctl lock for process (if necessary)
Do your writing
Release fnctl lock to allow other processes to use the segment (if necessary)
Wait again on process-levelbookkeeping structure mutex (may not be necessary if you can mark it unused atomically)
mark segment as unused within your process.
Release process-level mutex
No. The region-locking feature you're asking about has surprising semantics and it is not being further developed because it is controlled by POSIX. (In fact, it is Kirk McKusick's preferred example of what's wrong with POSIX.) If there is a non-POSIX byte-range lock facility in Linux, I can't find it.
There is discussion of the problems of POSIX byte-range locking in a multithreaded world here: http://www.samba.org/samba/news/articles/low_point/tale_two_stds_os2.html.
However, if you're concerned only with threads within one process, you can build your own region-locking using semaphores. For example:
#include <stdbool.h>
#include <pthread.h>
#include <sys/types.h>
// A record indicating an active lock.
struct threadlock {
int fd; // or -1 for unused entries.
off_t start;
off_t length;
};
// A table of all active locks (and the unused entries).
static struct threadlock all_locks[100];
// Mutex housekeeping.
static pthread_mutex_t mutex;
static pthread_cond_t some_lock_released;
static pthread_once_t once_control = PTHREAD_ONCE_INIT;
static void threadlock_init(void) {
for (int i = 0; i < sizeof(all_locks)/sizeof(all_locks[0]); ++i)
all_locks[i].fd = -1;
pthread_mutex_init(&mutex, (pthread_mutexattr_t *)0);
pthread_cond_init(&some_lock_released, (pthread_condattr_t *)0);
}
// True iff the given region overlaps one that is already locked.
static bool region_overlaps_lock(int fd, off_t start, off_t length) {
for (int i = 0; i < sizeof(all_locks)/sizeof(all_locks[0]); ++i) {
const struct threadlock *t = &all_locks[i];
if (t->fd == fd &&
t->start < start + length &&
start < t->start + t->length)
return true;
}
return false;
}
// Returns a pointer to an unused entry, or NULL if there isn't one.
static struct threadlock *find_unused_entry(void) {
for (int i = 0; i < sizeof(all_locks)/sizeof(all_locks[0]); ++i) {
if (-1 == all_locks[i].fd)
return &all_locks[i];
}
return 0;
}
// True iff the lock table is full.
static inline bool too_many_locks(void) {
return 0 == find_unused_entry();
}
// Wait until no thread has a lock for the given region
// [start, start+end) of the given file descriptor, and then lock
// the region. Keep the return value for threadunlock.
// Warning: if you open two file descriptors on the same file
// (including hard links to the same file), this function will fail
// to notice that they're the same file, and it will happily hand out
// two locks for the same region.
struct threadlock *threadlock(int fd, off_t start, off_t length) {
pthread_once(&once_control, &threadlock_init);
pthread_mutex_lock(&mutex);
while (region_overlaps_lock(fd, start, length) || too_many_locks())
pthread_cond_wait(&some_lock_released, &mutex);
struct threadlock *newlock = find_unused_entry();
newlock->fd = fd;
newlock->start = start;
newlock->length = length;
pthread_mutex_unlock(&mutex);
return newlock;
}
// Unlocks a region locked by threadlock.
void threadunlock(struct threadlock *what_threadlock_returned) {
pthread_mutex_lock(&mutex);
what_threadlock_returned->fd = -1;
pthread_cond_broadcast(&some_lock_released);
pthread_mutex_unlock(&mutex);
}
Caution: the code compiles but I haven't tested it even a little.
If you don't need file locks between different processes, avoid the file locks (which are one of the worst designed parts of the POSIX API) and just use mutexes or other shared memory concurrency primitives.
There are 2 ways you can do it:
Use Mutex to get a record's lock in a thread within the same process. Once the lock is acquired, any other thread in the process, mapping the file that tries to acquire the lock is blocked until the lock is released.(Preferable and only most straightforward solution available in Linux).
Semaphores and mutexes on a shared memory or a memory mapped file.

How to determine if code is running in signal-handler context?

I just found out that someone is calling - from a signal handler - a definitely not async-signal-safe function that I wrote.
So, now I'm curious: how to circumvent this situation from happening again? I'd like to be able to easily determine if my code is running in signal handler context (language is C, but wouldn't the solution apply to any language?):
int myfunc( void ) {
if( in_signal_handler_context() ) { return(-1) }
// rest of function goes here
return( 0 );
}
This is under Linux.
Hope this isn't an easy answer, or else I'll feel like an idiot.
Apparently, newer Linux/x86 (probably since some 2.6.x kernel) calls signal handlers from the vdso. You could use this fact to inflict the following horrible hack upon the unsuspecting world:
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <string.h>
#include <signal.h>
#include <unistd.h>
uintmax_t vdso_start = 0;
uintmax_t vdso_end = 0; /* actually, next byte */
int check_stack_for_vdso(uint32_t *esp, size_t len)
{
size_t i;
for (i = 0; i < len; i++, esp++)
if (*esp >= vdso_start && *esp < vdso_end)
return 1;
return 0;
}
void handler(int signo)
{
uint32_t *esp;
__asm__ __volatile__ ("mov %%esp, %0" : "=r"(esp));
/* XXX only for demonstration, don't call printf from a signal handler */
printf("handler: check_stack_for_vdso() = %d\n", check_stack_for_vdso(esp, 20));
}
void parse_maps()
{
FILE *maps;
char buf[256];
char path[7];
uintmax_t start, end, offset, inode;
char r, w, x, p;
unsigned major, minor;
maps = fopen("/proc/self/maps", "rt");
if (maps == NULL)
return;
while (!feof(maps) && !ferror(maps)) {
if (fgets(buf, 256, maps) != NULL) {
if (sscanf(buf, "%jx-%jx %c%c%c%c %jx %u:%u %ju %6s",
&start, &end, &r, &w, &x, &p, &offset,
&major, &minor, &inode, path) == 11) {
if (!strcmp(path, "[vdso]")) {
vdso_start = start;
vdso_end = end;
break;
}
}
}
}
fclose(maps);
printf("[vdso] at %jx-%jx\n", vdso_start, vdso_end);
}
int main()
{
struct sigaction sa;
uint32_t *esp;
parse_maps();
memset(&sa, 0, sizeof(struct sigaction));
sa.sa_handler = handler;
sa.sa_flags = SA_RESTART;
if (sigaction(SIGUSR1, &sa, NULL) < 0) {
perror("sigaction");
exit(1);
}
__asm__ __volatile__ ("mov %%esp, %0" : "=r"(esp));
printf("before kill: check_stack_for_vdso() = %d\n", check_stack_for_vdso(esp, 20));
kill(getpid(), SIGUSR1);
__asm__ __volatile__ ("mov %%esp, %0" : "=r"(esp));
printf("after kill: check_stack_for_vdso() = %d\n", check_stack_for_vdso(esp, 20));
return 0;
}
SCNR.
If we can assume your application doesn't manually block signals using sigprocmask() or pthread_sigmask(), then this is pretty simple: get your current thread ID (tid). Open /proc/tid/status and get the values for SigBlk and SigCgt. AND those two values. If the result of that AND is non-zero, then that thread is currently running from inside a signal handler. I've tested this myself and it works.
There are two proper ways to deal with this:
Have your co-workers stop doing the wrong thing. Good luck pulling this off with the boss, though...
Make your function re-entrant and async-safe. If necessary, provide a function with a different signature (e.g. using the widely-used *_r naming convention) with the additional arguments that are necessary for state preservation.
As for the non-proper way to do this, on Linux with GNU libc you can use backtrace() and friends to go through the caller list of your function. It's not easy to get right, safe or portable, but it might do for a while:
/*
* *** Warning ***
*
* Black, fragile and unportable magic ahead
*
* Do not use this, lest the daemons of hell be unleashed upon you
*/
int in_signal_handler_context() {
int i, n;
void *bt[1000];
char **bts = NULL;
n = backtrace(bt, 1000);
bts = backtrace_symbols(bt, n);
for (i = 0; i < n; ++i)
printf("%i - %s\n", i, bts[i]);
/* Have a look at the caller chain */
for (i = 0; i < n; ++i) {
/* Far more checks are needed here to avoid misfires */
if (strstr(bts[i], "(__libc_start_main+") != NULL)
return 0;
if (strstr(bts[i], "libc.so.6(+") != NULL)
return 1;
}
return 0;
}
void unsafe() {
if (in_signal_handler_context())
printf("John, you know you are an idiot, right?\n");
}
In my opinion, it might just be better to quit rather than be forced to write code like this.
You could work out something using sigaltstack. Set up an alternative signal stack, get the stack pointer in some async-safe way, if within the alternative stack go on, otherwise abort().
I guess you need to do the following. This is a complex solution, which combines the best practices not only from coding, but from software engineering as well!
Persuade your boss that naming convention on signal handlers is a good thing. Propose, for example, a Hungarian notation, and tell that it was used in Microsoft with great success.
So, all signal handlers will start with sighnd, like sighndInterrupt.
Your function that detects signal handling context would do the following:
Get the backtrace().
Look if any of the functions in it begin with sighnd.... If it does, then congratulations, you're inside a signal handler!
Otherwise, you're not.
Try to avoid working with Jimmy in the same company. "There can be only one", you know.
for code optimized at -O2 or better (istr) have found need to add -fno-omit-frame-pointer
else gcc will optimize out the stack context information

In pthread, how to reliably pass signal to another thread?

I'm trying to write a simple thread pool program in pthread. However, it seems that pthread_cond_signal doesn't block, which creates a problem. For example, let's say I have a "producer-consumer" program:
pthread_cond_t my_cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t my_cond_m = PTHREAD_MUTEX_INITIALIZER;
void * liberator(void * arg)
{
// XXX make sure he is ready to be freed
sleep(1);
pthread_mutex_lock(&my_cond_m);
pthread_cond_signal(&my_cond);
pthread_mutex_unlock(&my_cond_m);
return NULL;
}
int main()
{
pthread_t t1;
pthread_create(&t1, NULL, liberator, NULL);
// XXX Don't take too long to get ready. Otherwise I'll miss
// the wake up call forever
//sleep(3);
pthread_mutex_lock(&my_cond_m);
pthread_cond_wait(&my_cond, &my_cond_m);
pthread_mutex_unlock(&my_cond_m);
pthread_join(t1, NULL);
return 0;
}
As described in the two XXX marks, if I take away the sleep calls, then main() may stall because it has missed the wake up call from liberator(). Of course, sleep isn't a very robust way to ensure that either.
In real life situation, this would be a worker thread telling the manager thread that it is ready for work, or the manager thread announcing that new work is available.
How would you do this reliably in pthread?
Elaboration
#Borealid's answer kind of works, but his explanation of the problem could be better. I suggest anyone looking at this question to read the discussion in the comments to understand what's going on.
In particular, I myself would amend his answer and code example like this, to make this clearer. (Since Borealid's original answer, while compiled and worked, confused me a lot)
// In main
pthread_mutex_lock(&my_cond_m);
// If the flag is not set, it means liberator has not
// been run yet. I'll wait for him through pthread's signaling
// mechanism
// If it _is_ set, it means liberator has been run. I'll simply
// skip waiting since I've already synchronized. I don't need to
// use pthread's signaling mechanism
if(!flag) pthread_cond_wait(&my_cond, &my_cond_m);
pthread_mutex_unlock(&my_cond_m);
// In liberator thread
pthread_mutex_lock(&my_cond_m);
// Signal anyone who's sleeping. If no one is sleeping yet,
// they should check this flag which indicates I have already
// sent the signal. This is needed because pthread's signals
// is not like a message queue -- a sent signal is lost if
// nobody's waiting for a condition when it's sent.
// You can think of this flag as a "persistent" signal
flag = 1;
pthread_cond_signal(&my_cond);
pthread_mutex_unlock(&my_cond_m);
Use a synchronization variable.
In main:
pthread_mutex_lock(&my_cond_m);
while (!flag) {
pthread_cond_wait(&my_cond, &my_cond_m);
}
pthread_mutex_unlock(&my_cond_m);
In the thread:
pthread_mutex_lock(&my_cond_m);
flag = 1;
pthread_cond_broadcast(&my_cond);
pthread_mutex_unlock(&my_cond_m);
For a producer-consumer problem, this would be the consumer sleeping when the buffer is empty, and the producer sleeping when it is full. Remember to acquire the lock before accessing the global variable.
I found out the solution here. For me, the tricky bit to understand the problem is that:
Producers and consumers must be able to communicate both ways. Either way is not enough.
This two-way communication can be packed into one pthread condition.
To illustrate, the blog post mentioned above demonstrated that this is actually meaningful and desirable behavior:
pthread_mutex_lock(&cond_mutex);
pthread_cond_broadcast(&cond):
pthread_cond_wait(&cond, &cond_mutex);
pthread_mutex_unlock(&cond_mutex);
The idea is that if both the producers and consumers employ this logic, it will be safe for either of them to be sleeping first, since the each will be able to wake the other role up. Put it in another way, in a typical producer-consumer sceanrio -- if a consumer needs to sleep, it's because a producer needs to wake up, and vice versa. Packing this logic in a single pthread condition makes sense.
Of course, the above code has the unintended behavior that a worker thread will also wake up another sleeping worker thread when it actually just wants to wake the producer. This can be solved by a simple variable check as #Borealid suggested:
while(!work_available) pthread_cond_wait(&cond, &cond_mutex);
Upon a worker broadcast, all worker threads will be awaken, but one-by-one (because of the implicit mutex locking in pthread_cond_wait). Since one of the worker threads will consume the work (setting work_available back to false), when other worker threads awake and actually get to work, the work will be unavailable so the worker will sleep again.
Here's some commented code I tested, for anyone interested:
// gcc -Wall -pthread threads.c -lpthread
#include <stdio.h>
#include <pthread.h>
#include <stdlib.h>
#include <assert.h>
pthread_cond_t my_cond = PTHREAD_COND_INITIALIZER;
pthread_mutex_t my_cond_m = PTHREAD_MUTEX_INITIALIZER;
int * next_work = NULL;
int all_work_done = 0;
void * worker(void * arg)
{
int * my_work = NULL;
while(!all_work_done)
{
pthread_mutex_lock(&my_cond_m);
if(next_work == NULL)
{
// Signal producer to give work
pthread_cond_broadcast(&my_cond);
// Wait for work to arrive
// It is wrapped in a while loop because the condition
// might be triggered by another worker thread intended
// to wake up the producer
while(!next_work && !all_work_done)
pthread_cond_wait(&my_cond, &my_cond_m);
}
// Work has arrived, cache it locally so producer can
// put in next work ASAP
my_work = next_work;
next_work = NULL;
pthread_mutex_unlock(&my_cond_m);
if(my_work)
{
printf("Worker %d consuming work: %d\n", (int)(pthread_self() % 100), *my_work);
free(my_work);
}
}
return NULL;
}
int * create_work()
{
int * ret = (int *)malloc(sizeof(int));
assert(ret);
*ret = rand() % 100;
return ret;
}
void * producer(void * arg)
{
int i;
for(i = 0; i < 10; i++)
{
pthread_mutex_lock(&my_cond_m);
while(next_work != NULL)
{
// There's still work, signal a worker to pick it up
pthread_cond_broadcast(&my_cond);
// Wait for work to be picked up
pthread_cond_wait(&my_cond, &my_cond_m);
}
// No work is available now, let's put work on the queue
next_work = create_work();
printf("Producer: Created work %d\n", *next_work);
pthread_mutex_unlock(&my_cond_m);
}
// Some workers might still be waiting, release them
pthread_cond_broadcast(&my_cond);
all_work_done = 1;
return NULL;
}
int main()
{
pthread_t t1, t2, t3, t4;
pthread_create(&t1, NULL, worker, NULL);
pthread_create(&t2, NULL, worker, NULL);
pthread_create(&t3, NULL, worker, NULL);
pthread_create(&t4, NULL, worker, NULL);
producer(NULL);
pthread_join(t1, NULL);
pthread_join(t2, NULL);
pthread_join(t3, NULL);
pthread_join(t4, NULL);
return 0;
}

Resources