I have some questions about pthreads in linux:
Is it the case that pthread_t is it a datatype similar to int and char indicating we are defining a thread?
If so, how much size does it take? 2 bytes or 4 bytes?
Does the compiler allocate memory to pthread_t thread1 immediately after that statement or does it wait until it a pthread_create() call?
How does one set the thread attributes, and what is their typical use?
Can one only pass more than one argument in the pthread_create() call? If so, how?
I have lots of things on my mind like this. Please also feel free to suggest any good sites or documents to read.
Answering the questions one by one, though not necessarily in the same order:
Is pthread_t a data type similar to int or char, indicating we are defining a thread ? Does the compiler allocate memory to pthread_t thread1 immediately after that sentence or does it wait until it finds the pthread_create() call
pthread_t is a type similar to int and it's created when you define it, not when you call pthread_create. In the snippet:
pthread_t tid;
int x = pthread_create (&tid, blah, blah, blah);
it's the first line that creates the variable, although it doesn't hold anything useful until the return from pthread_create.
How much size does a pthread_t take, 2 bytes or 4 bytes?
You shouldn't care how much space it takes, any more than you should care how much space is taken by a FILE structure. You should just use the structure as intended. If you really want to know, then sizeof is your friend.
Any good information about how to set the thread attributes?
If you want to use anything other than default attributes, you have to create an attributes variable first and then pass that to the pthread_create call.
Can we only pass one argument in the pthread_create function to the function? Can't we send 2 or 3 arguments in the pthread_create() function to the called thread?
While you're only allowed to pass one extra parameter to the thread , there's nothing stopping you from making this one parameter a pointer to a structure holding a hundred different things.
If you're looking for information on how to use pthreads, there's plenty of stuff at the end of a Google search but I still prefer the dead-tree version myself:
how much size does it take
pthread_t uses sizeof pthread_t bytes.
and we can only pass one argument in the pthread_create to the function not more than one? cant we send 2 or 3 arguments in the pthread_create() function to the called thread?
All you need is one argument. All you get is one argument. It's a void * so you can pass a pointer to whatever you want. Such as a structure containing multiple values.
i have lots of things on my mind like this suggest any good sites or documents to read
Have a look at the pthread man pages, online or in your shell of choice (man pthread, man pthread_create, etc.). I started out reading some basic lecture slides (here's the sequel).
pthread_t could be any number of bytes. It could be a char, an int, a pointer, or a struct... But you neither need to know nor to care. If you need the size for allocation purposes, you use sizeof(pthread_t). The only type of variable you can assign it to is another pthread_t.
The compiler may or may not allocate the resources associated with the thread when you define a pthread_t. Again, you do not need to know nor to care, because you are required to call pthread_join (or pthread_detach) on any thread you create. As long as you follow the rules, the system will make sure it does not leak memory (or any other resource).
Attributes are admittedly a bit clumsy. They are held in an pthread_attr_t object, which again could be represented as an integer, pointer, or entire struct. You have to initialize it with pthread_attr_init and destroy it with pthread_attr_destroy. Between those two, you use various pthread_attr_... calls to set or clear attributes, and then you can pass it as part of one or more pthread_create calls to set the attributes of the new threads.
Different implementations can and will handle all of these things differently.
LLNL has a decent set of introductory information.
Look into pthread.h file to get more information. On my system, pthread_t is defined as an unsigned long int. But I guess this is platform dependent, since it is defined into bits/pthreadtype.h.
Related
I am unable to find a function to convert a thread id (pid_t) into a pthread_t which would allow me to call pthread_cancel() or pthread_kill().
Even if pthreads doesn't provide one is there a Linux specific function?
I don't think such a function exists but I would be happy to be corrected.
Background
I am well aware that it is usually preferable to have threads manage their own lifetimes via condition variables and the like.
This use is for testing purposes. I am trying to find a way to test how an application behaves when one of its threads 'dies'. So I'm really looking for a way to kill a thread. Using syscall(tgkill()) kills the process, so instead I provided a means for a tester to give the process the id of the thread to kill. I now need to turn that id into a pthread_t so that I can then:
use pthread_kill(tid,0) to check for its existence followed by
calling pthread_kill() or pthread_cancel() as appropriate.
This is probably taking testing to an unnecessary extreme. If I really want to do that some kind of mock pthreads implementation might be better.
Indeed if you really want robust isolation you are typically better off using processes rather than threads.
I don't think such a function exists but I would be happy to be corrected.
As a workaround I can create a table mapping &pthread_t to pid_t and ensure that I always invoke pthread_create() via a wrapper that adds an entry to this table. This works very well and allows me to convert an OS thread id to a pthread_t which I can then terminate using pthread_cancel(). Here is a snippet of the mechanism:
typedef void* (*threadFunc)(void*);
static void* start_thread(void* arg)
{
threadFunc threadRoutine = routine_to_start;
record_thread_start(pthread_self(),syscall(SYS_gettid));
routine_to_start = NULL; //let creating thread know its safe to continue
return threadRoutine(arg);
}
Sensible conversion requires there to be a 1:1 mapping between pthread_t and pid_t tid, which is the case with NPTL, but hasn't always been the case, and won't be the case on every pthread platform. That said...
Two options:
A) override the actual pthread_create, using LD_PRELOAD and dlsym, and keep track of each pthread_t and their corresponding pid_t there. To get the thread pid_t you can either take advantage of the pthread private headers to de-opaque the pthread_t and access the pid_t inside there, or if you want to stick to documented APIs pthread_sigqueue each pthread_t thread as it is created and have a sigaction signal handler call gettid and pass you back the thread pid_t, with appropriate synchronisation between your new pthread_create and the signal handler[1].
B) You can read the all of the thread pid_t from /proc/<process pid_t>/task/. Then use the SYS_rt_tgsigqueueinfo[2] syscall to implement a new function thread_sigqueue, a pid_t variant of pthread_sigqueue so that you can signal the pid_t thread, and from the sigaction signal handler call pthread_self passing out the value with suitable synchronization, etc.
Notes:
1 - I think it's worth writing 2 executeOnThread variants (one for pthread_t and one for pid_t style thread ids) that take a std::function<void()> (for C++), or a void(*)(void*) function pointer and void* parameter (for C), and SIGUSR1 that thread to execute the passed function in a sigaction that you also setup to perform relevant synchronization with the calling thread. It's handy to be able to use the thread-dependent APIs like pthread_self, gettid, backtrace, getrusage, etc. without devising a custom execution scheme each time.
2 - SYS_rt_tgsigqueueinfo is a low level syscall meant for implementing sigqueue/pthread_sigqueue, rather than application use, but is still a documented API, and we're using it to implement another variant of sigqueue, so fair game IMHO.
Given - Thread id of a thread.
Requirement - Set Linux priority of the thread id.
Constraint - Cant use setpriority()
I have tried to use below
pthread_setschedprio(pthread_t thread, int prio);
pthread_setschedparam(pthread_t thread, int policy,
const struct sched_param *param);
Both the above APIs use pthread_t as an argument. I am not able to construct (typecast) pthread_t from thread id. I understand converting this is not possible due to different types.
Is there a way to still accomplish this ?
Some aspects of the pthread_setschedprio interface are available for plain thread IDs with the sched_setparam function (declared in <thread.h>). The sched_setparam manual page says that the process is affected (which is the POSIX-mandated behavior), but on Linux, it's actually the thread of that ID.
Keep in mind that calling sched_setparam directly may break the behavior expected from PI mutexes and other synchronization primitives because the direct call does not perform the additional bookkeeping performed by the pthread_* functions.
In C++/C#/Java we can start the thread with function that accepts some arguments. In WinAPI we start the thread with function that accepts only void*.
How many arguments are passed really to real windows threads? Maybe many arguments are turned into void* that points on some struct?
At the core of most threading APIs is a function pointer to execute and a void* parameter that lets you provide some data to the executing function. The void* typically points to some object instance that the thread function then casts into the known object type to use. This, however, is ripe for programmer error.
The higher level APIs that you mention (std::thread in C++, Thread in Java, etc) are doing this under the hood and providing you convenient, type-safe APIs to ensure you can't mess it up.
I know there is a system call pthread_equal in Linux for comparing 2 thread ids. But why cannot one directly compare 2 thread ids using '==' operator?
From the pthread_equal man page on Linux:
The pthread_equal() function is necessary because thread IDs should
be considered opaque: there is no portable way for applications to
directly compare two pthread_t values.
It might be a struct. It might be a pointer. It might be a pointer to a struct held somewhere. == might, or might not, return true for all cases it should return true and vice versa.
So you are provided with an accessor that is guaranteed to return the correct result, no matter the implementation.
I have checked the definition of pthread_t with the gcc compiler which I am using, there it is defined as "typedef unsigned long int pthread_t;". But the definition of pthread_t is implementation dependent. While porting the same code to different platforms(hardware), if we compare directly the pthread id's it will be a problem because different platforms may implement pthread_t in different ways(implementaion dependent, like structure, or type other that unsigned long int). So to make our code portable across different platforms we are using pthread_t as an opaque type.
I came across the function InterlockedExchange and was wondering when I should use this function. In my opinion, setting a 32 Bit value on an x86 processor should always be atomic?
In the case where I want to use the function, the new value does not depend on the old value (it is not an increment operation).
Could you provide an example where this method is mandatory (I'm not looking for InterlockedCompareExchange)
InterlockedExchange is both a write and a read -- it returns the previous value.
This is necessary to ensure another thread didn't write a different value just after you did. For example, say you're trying to increment a variable. You can read the value, add 1, then set the new value with InterlockedExchange. The value returned by InterlockedExchange must match the value you originally read, otherwise another thread probably incremented it at the same time, and you need to loop around and try again.
As well as writing the new value, InterlockedExchange also reads and returns the previous value; this whole operation is atomic. This is useful for lock-free algorithms.
(Incidentally, 32-bit writes are not guaranteed to be atomic. Consider the case where the write is unaligned and straddles a cache boundary, for instance.)
In a multi-processor or multi-core machine each core has it's own cache - so each core has each own potentially different "view" of what the content of the system memory is.
Thread synchronization mechanisms take care of synchronizing between cores, for more information look at http://blogs.msdn.com/oldnewthing/archive/2008/10/03/8969397.aspx or google for acquire and release semantics
Setting a 32-bit value is atomic, but only if you're setting a literal.
b = a is 2 operations:
mov eax,dword ptr [a]
mov dword ptr [b],eax
Theoretically there could be some interruption between the first and second operation.
Writing a value is never atomic by default. When you write a value to a variable, several machine instructions are generated. With modern, preemptive OSes, the OS might switch to another thread between the individual operations of the write.
This is even more a problem on multi-processor machines, where several threads could be executing at the same time, and trying to write to a single memory location simultaneously.
Interlocked operations avoid this by using specialized instructions to make the write (x86 has dedicated instructions for this kind of situation), which do the read-modify-write in one instruction. These instructions also lock the memory bus of all processors, to ensure that no other executing thread could be writing to the value at the same time.
InterlockedExchange makes sure that the change of a variable and the return of its original value are not interrupted by other threads.
So, if 'i' is an int, these calls (taken individually) do not need InterlockedExchange around 'i':
a = i;
i = 9;
i = a;
i = a + 9;
a = i + 9;
if(0 == i)
None of these statements rely upon BOTH the initial AND final values of 'i'. But these following calls DO need InterlockedExchange around 'i':
a = i++; //a = InterlockedExchange(&i, i + 1);
Without it, two threads running through this same code might get the same value of 'i' assigned to 'a' or 'a' may unexpectedly skip two or more numbers.
if(0 == i++) //if(0 == InterlockedExchange(&i, i + 1))
Two threads may both execute the code that is only supposed to happen once.
etc.
wow, so many conflicting answers. Hard to sift through who's right, who's wrong, and what information is misleading.
I'm unsure of the answer too, given the above half-answers, but I think it works like this, I may be wrong, and it will be interesting to find out if I am:
32-bit read & writes ARE atomic, but depending on your code, that may not mean much.
don't worry about non-aligned read/writes. ALL 32-bit writes to a 32-bit variable have to be aligned or the machine page-faults.
don't worry about a write wrapping around the end of a cached page, that can't happen.
If you need to write-then-read on one thread, and you're writing on another thread, then you need to use InterlockedExchange. If you're simply reading the value on one thread, and writing it on another, then you don't need to use it, but those values may be wiggly because of multithreading.