I have to implement a wrapper function that serves as pthread_self() to get a pthread ID but I've been searching and haven´t found which syscall does this. Reading another post from Stack O. I know clone() is used to create threads, also that I can trace the syscalls with ptrace() but before tracing it by hand...could someone knows which syscall is?
There are 3 different IDs for a linux process thread: pid, pthread id, and tid.
The 'pid' is global and equivalent to the parent process id, and is easily obtained by 'getpid()'. This value is unique, but only for the duration of an the active process assigned the given id. This value may be 'recycled' for a new process after a process is terminated and new ones are spawned. This value is the same across all threads, within a process. This value is what you'll see in top, and htop, 'ps -ef', and pidstat.
The 'pthread id' is reported by pthread_create() and phtread_self(). This is value is unique only within the process, and only for the duration of the assign thread. This value may be 'recycled' as threads are terminated and spawned. This value is not unique across the system, nor across threads that have been terminated and started. This value is NOT visible outside of a program. This value is opaque and may be a pointer or structure depending on the platform.
The 'tid' Thread id is reported by gettid(). This was introduced to Linux 2.4, and does not appear to be available on other platforms. This value is unique within the process and across the system. This value is reported by top and htop, and 'pidstat -t'. I'm not 100% certain, but suspect this value can be 'recycled' as processes are terminated and spawned. This is the value that appears in the Linux tools 'top','htop', 'pidstat -t', and 'ps -efL', when shown threads.
Documentation for gettid: linux.die.net/gettid
You can obtain 'gettid()' through:
#include <sys/types.h>
#include <sys/syscall.h>
#include <pthread.h>
My CentOS 6.5 is not properly setup and missing the gettid prototype, though the documentation says it should be present through the above #includes. Here is a macro that mimics 'gettid':
#ifndef gettid
// equivalent to: pid_t gettid(void)
#define gettid() syscall(SYS_gettid)
#endif
Be aware that since this is a syscall(), you'll gain efficiency by caching the result and avoiding repeatedly using the syscall().
How about syscall 0xe0, gettid()?
gettid() returns the caller's thread ID (TID). In a single-threaded process, the thread ID is equal to the process ID (PID, as returned by getpid(2)). In a multithreaded process, all threads have the same PID, but each one has a unique TID. For further details, see the discussion of CLONE_THREAD in clone(2).
In glibc, pthread_self() does not do system calls, but returns a pointer to a struct pthread, located in the TSD segment.
This might be helpfull.
UINT32 tid= syscall(SYS_gettid);
Related
On Linux ps -Lf will display a thread ID in the column LWP and number of thread in NLWP column. Any single threaded process will have PID and LWP values the same.
What should pthread_self() return on a single threaded application? Initially I was expecting that its value should be the same as a process ID, executing this call, but results were different. Then I read man pthread_self and man gettid and learned that the value returned by pthread_self() is not the same as gettid() result.
So can I even trust pthread_self() output executed in a non-threaded environment (process)?
pthread_self is defined to return the calling thread's ID regardless of whether the program
is multi-threaded or a single-threaded.
As you found, the return value of pthread_self() isn't same as the LWP in Linux (gettid) and as such it doesn't have any meaning outside of the process; pthread_t is
an opaque type. Related: The thread ID returned by pthread_self() is not the same thing as the kernel thread ID returned by a call to gettid(2)
Its utility is very limited as there's not much practical use for pthread_t in a single-threaded program. You can use in pthread_setschedparam for example.
But if you are asking whether returns any valid value in single-threaded program, then the answer is yes.
I'm trying to find a way to uniquely identify threads in a multi-process environment. I have a server that keeps track of the different processes connecting to it, some of which are multi-threaded and some of which are not. To identify the threads from multi-threaded connections I'm using the thread ID as a unique identifier (there will be a maximum of 1 multi-threaded process connected at any given time). My question is: is it possible the thread ID of one of these threads could be the same as the process ID of another processes running on the system?
Thanks in advance for the help!
The TID (as returned by the sys_gettid() system call) is unique across all threads on the system1, and for a single-threaded process the PID and TID are equal. This means that a TID will never clash with a PID from another process.
1. With the caveat that if PID namespaces are in use, TIDs and PIDs are only unique within the same PID namespace.
According to the man page of pthreads the thread ID is unique within the creating process, so yes another thread or process could have the same ID. However, If it's unique within a process and a process ID is unique in the system then maybe you can use a combination of the two as a unique identifier.
Each of the threads in a process has a unique thread identifier
(stored in the type pthread_t). This identifier is returned to the
caller of pthread_create(3), and a thread can obtain its own thread
identifier using pthread_self(3). Thread IDs are only guaranteed to
be unique within a process.
While the pthread ID might not be unique, in a implementation where threads map to tasks, the task id (as seen in /proc/PID/task) will in fact be unique system wide, and have a form similar to an actual PID.
Well, I came across the same problem just now and here is my program for validation.
#include <pthread.h>
#include <stdio.h>
int main() {
printf("%lu\n", pthread_self());
}
clang -pthread test.c && strace ./a.out
Part of the output is as follows.
...
arch_prctl(ARCH_SET_FS, 0x7f53259be740) = 0
...
write(1, "139995089987392\n", 16139995089987392
) = 16
...
Then we know 0x7f53259be740 equals to 139995089987392 and the second argument of arch_prctl should be within the process address space(man arch_prctl). That is to say, the thread ID is a virtual address in fact. So if you use pthread_self() to identify threads in a multi-process environment, collisions may happen though it is a small chance.
pthread_equal(id1,id2)
It will compare the ID's of two threads and will return 0 if they are the same and a non zero number if they are different.
What is the difference between PID and TID?
The standard answer would be that PID is for processes while TID is for threads. However, I have seen that some commands use them interchangeably. For example, htop has a column for PIDs, in which PIDs for threads of the same process are shown (with different values). So when does a PID represent a thread or a process?
It is complicated: pid is process identifier; tid is thread identifier.
But as it happens, the kernel doesn't make a real distinction between them: threads are just like processes but they share some things (memory, fds...) with other instances of the same group.
So, a tid is actually the identifier of the schedulable object in the kernel (thread), while the pid is the identifier of the group of schedulable objects that share memory and fds (process).
But to make things more interesting, when a process has only one thread (the initial situation and in the good old times the only one) the pid and the tid are always the same. So any function that works with a tid will automatically work with a pid.
It is worth noting that many functions/system calls/command line utilities documented to work with pid actually use tids. But if the effect is process-wide you will simply not notice the difference.
Actually, each thread in a Linux process is Light Weight Process (LWP). So, people may call thread as a process... But there is surely a difference.
Each thread in a process has a different thread ID (TID) and share the same process ID (PID).
If you are working with pthread library functions, then these functions don't use these TIDs because these are kernel/OS level thread IDs.
Just to add to other answers, according to man gettid:
The thread ID returned by this call is not the same thing as a POSIX thread ID (i.e., the opaque value returned by pthread_self(3)).
So there are two different things one could mean by TID!
pid and tid are the same except when a process is created with a call to clone with CLONE_THREAD (per the man pages of gettid). In this case, you get a unique thread id but all threads belonging to the same thread group share the same process id.
However, I also recall reading (though I cant find the source) that the values returned from getpid may be cached.
[UPDATE]
See the NOTES section here for a discussion on the effects of caching pids.
For a class I'm taking I've been doing some work directly with the clone() system call in Linux. I got curious about how it actually worked and started doing some digging. What is confusing me is that it seems to rely on some of the same underpinnings as fork() functionality (they call the same do_fork() function albeit with different arguments). On one hand, this makes sense to me as a thread is really just a light-weight process but I was always under the impression that there were some significant differences between the way a thread was created an the way a process was created. I did some digging into the implementation of do_fork() and subsequently copy_process() (which do_fork() calls) but I haven't been able to convince myself I understand what's going on.
So, to the guru's out there, am I missing something or is this actually how it works? Are there flags that basically tell the OS just how much to copy as well as what instruction to begin execution of the new task at (I'm thinking the answer has to be yes, but I'm just not sure how they translate)?
Below is the code I'm looking at, perhaps you could explain how the arguments that are passed in control whether a light-weight or heavy-weight process is created.
asmlinkage int sys_fork(struct pt_regs *regs){
#ifdef CONFIG_MMU
return do_fork(SIGCHLD, regs->ARM_sp, regs, 0, NULL, NULL);
#else
/* can not support in nommu mode */
return(-EINVAL);
#endif
}
asmlinkage int sys_clone(unsigned long clone_flags, unsigned long newsp,
int __user *parent_tidptr, int tls_val,
int __user *child_tidptr, struct pt_regs *regs)
{
if (!newsp)
newsp = regs->ARM_sp;
return do_fork(clone_flags, newsp, regs, 0, parent_tidptr, child_tidptr);
}
Thanks!
Actually, at the conceptual level, the Linux kernel doesn't know anything about processes or threads, it only knows about "tasks".
A Linux task can be a process, a thread or something in between. (Incidentally, this means that the strange children that vfork() creates fit perfectly well into the Linux "task" paradigm).
Now, tasks can share several things, see all the CLONE_* flags in the manpage for clone(2). (Not all these flags can be described as sharing, some specify more complex behaviours).
Or new tasks can choose to have their own copies of the respective resources. And since 2.6.16, they can do so after having been started, see unshare(2).
For instance, the only difference between a vfork() and a fork() call, is that vfork() has CLONE_VM and CLONE_VFORK set. CLONE_VM makes it share its parent's memory (the same way threads share memory), while CLONE_VFORK makes the parent block until the child releases its memory mappings (by calling execve() or _exit()).
Note that Linux is not the only OS to generalize processes and threads in this manner. Plan 9 has rfork().
Nothing in the clone manpage suggests that it's "lightweight".
The critical difference is that fork creates a new address space, while clone optionally shares the address space between the parent and child, as well as file handles and so forth.
This shared address space enables lightweight IPC later on, but the process itself is not slimmer.
I understand that the difference between all the three clone,fork and vfork is in the flags because finally all the three calls the do_fork() in kernel
fork()-->C_lib-->sys_fork()-->do_fork()
vfork()-->C_lib-->sys_vfork()-->do_fork()
clone()-->C_lib-->sys_clone()-->do_fork()
The difference between the fork and vfork is that vfork guarantees that child will execute first and parent will block until child calls exit or exec. vfork passes extra flag that is CLONE_VM, this flag ask kernel not to duplicate the page table, the reason is simple the child will either do exit or exec, if child exits nothing would be done, if child does the exec the page table will definitely be changed. i hope the fork and vfork flags are clear now at kernel level.
Now lets look at the clone flags
The main usage of clone is to implement thread, where the memory space shared other then stack. Along with same parameter as fork and vfork the clone also takes the function pointer as parameter, which is called as soon as the child process is created.
In a graduate class, we've had to use semaphores to accomplish work with threads.
We were directed to use sem_init along with a bunch of other sem_* procedure but we were not given much information about the details of each of these sem_* methods.
The prototype (and header file) of sem_init is the following:
#include <semaphore.h>
int sem_init(sem_t *sem, int pshared, unsigned int value);
but I don't understand what the pshared value is used for. According to opengroup.org:
If the pshared argument has a non-zero
value, then the semaphore is shared
between processes; in this case, any
process that can access the semaphore
sem can use sem for performing
sem_wait(), sem_trywait(), sem_post(),
and sem_destroy() operations.
but I guess I don't understand the difference between say 1,2, 10, 25, 50000, etc. I think it is saying that if the value is 0 then the semaphore is not shared. (But then, what is the point?)
How do I appropriately use this pshared parameter?
The GLIBC version of sem_init (what you get if you man sem_init on Linux) has this to say:
"The pshared argument indicates whether this semaphore is to be
shared between the threads of a process, or between processes."
So pshared is a boolean value: in practice meaningful values passed to it are false (0) and true (1), though any non-0 value will be treated as true. If you pass it 0 you will get a semaphore that can be accessed by other threads in the same process -- essentially an in-process lock. You can use this as a mutex, or you can use it more generally for the resource-counting properties of a semaphore. Arguably if pthreads supported a semaphore API you wouldn't need this feature of sem_init, but semaphores in Unix precede pthreads by quite a bit of time.
It would be better if the boolean was some kind of enumeration (e.g. SEM_PROCESS_PRIVATE vs SEM_PROCESS_SHARED), because then you wouldn't have had this question, but POSIX semaphores are a fairly old API as these things go.
I would say that there is no significant difference between the value s 1, 2, 5 and so on with respect to the shared parameter. Probably it is written that way because when the API was first created, C did not have boolean types.
The pshared argument indicates whether this semaphore is to be shared between the threads of a process, or between processes.
If pshared has the value 0, then the semaphore is shared between the threads of a process, and should be located at some address that is visible to all threads (e.g., a global variable, or a variable allocated dynamically on the heap).
If pshared is nonzero, then the semaphore is shared between processes, and should be located in a region of shared memory (see shm_open(3), mmap(2), and shmget(2)). (Since a child created by fork(2) inherits its parent's memory mappings, it can also access the semaphore.) Any process that can access the shared memory region can operate on the semaphore using sem_post(3), sem_wait(3), etc.
The pshared argument indicates whether this semaphore is to be shared between the threads of a process, or between processes.If pshared has the value 0, then the semaphore is shared between the threads of a process, and should be located at some address that is visible to all threads.If pshared is nonzero, then the semaphore is shared betweenprocesses, and should be located in a region of shared memory.