What is the difference between PID and TID?
The standard answer would be that PID is for processes while TID is for threads. However, I have seen that some commands use them interchangeably. For example, htop has a column for PIDs, in which PIDs for threads of the same process are shown (with different values). So when does a PID represent a thread or a process?
It is complicated: pid is process identifier; tid is thread identifier.
But as it happens, the kernel doesn't make a real distinction between them: threads are just like processes but they share some things (memory, fds...) with other instances of the same group.
So, a tid is actually the identifier of the schedulable object in the kernel (thread), while the pid is the identifier of the group of schedulable objects that share memory and fds (process).
But to make things more interesting, when a process has only one thread (the initial situation and in the good old times the only one) the pid and the tid are always the same. So any function that works with a tid will automatically work with a pid.
It is worth noting that many functions/system calls/command line utilities documented to work with pid actually use tids. But if the effect is process-wide you will simply not notice the difference.
Actually, each thread in a Linux process is Light Weight Process (LWP). So, people may call thread as a process... But there is surely a difference.
Each thread in a process has a different thread ID (TID) and share the same process ID (PID).
If you are working with pthread library functions, then these functions don't use these TIDs because these are kernel/OS level thread IDs.
Just to add to other answers, according to man gettid:
The thread ID returned by this call is not the same thing as a POSIX thread ID (i.e., the opaque value returned by pthread_self(3)).
So there are two different things one could mean by TID!
pid and tid are the same except when a process is created with a call to clone with CLONE_THREAD (per the man pages of gettid). In this case, you get a unique thread id but all threads belonging to the same thread group share the same process id.
However, I also recall reading (though I cant find the source) that the values returned from getpid may be cached.
[UPDATE]
See the NOTES section here for a discussion on the effects of caching pids.
Related
I am reading pthread man and seeing following:
With NPTL, all of the threads in a process are placed in the same
thread group; all members of a thread group share the same PID.
My current architecture is running on NPTL 2.17 and when I run htop that is showing threads I see that all PIDs are unique. But why? I am expecting some of them (e.g. chrome) sharing same PID with each other?
See man gettid:
gettid() returns the caller's thread ID (TID). In a single-threaded
process, the thread ID is equal to the process ID (PID, as returned
by getpid(2)). In a multithreaded process, all threads have the same
PID, but each one has a unique TID. For further details, see the
discussion of CLONE_THREAD in clone(2).
What htop shows is TID, not PID. You can toggle display of the threads on/off with H key.
You can also enable PPID column in htop and that shows the PID / TID of the main thread for threads.
Google's documentation for Chromium (which probably operates similarly to Chrome when it comes to these concepts) states that they use a "multi-process architecture". Your quote from pthread's man page states that all of the threads in a single process are placed under the same PID, which would not apply to Chrome's architecture.
Because kernel-level threads are no more than processes with the (nearly) same address space.
This was "solved" by the linux kernel development by renaming them the processes to "threads", the "pid"-s to "tid"-s, and the old processes became "thread groups".
However, the sad truth is that if you create the thread on Linux (clone()), it will create a process - only using the (nearly) same memory segments.
That means 1:1 thread model. It means that all the threads are actually kernel-level threads, meaning that they are essentially processes in the same address space.
Some other alternatives would be:
1:M thread model. It means that the kernel doesn't know about threads, it is the task of the user-space libraries to make an "in-process multitasking" to run appearantly multi-threaded.
N:M thread model. This is best, unfortunately some opinion favorize still 1:1. It would mean that we have both user- and kernel-level threads and some optimization algorithm decides, what to run and where.
Once Linux had an N:M model (ngpt), but it was removed on a yet another fallback. It was that Linux kernel calls are inherently synchronous (blocking). Resulting that some kernel-cooperation had been needed even for user-space synchronization. Nobody wanted to do that.
So is it.
P.s. to create a well-performant app, you should actually avoid to create a lot of threads at once. You need to use a thread pool with well-thought locking protocols. If you don't minimize the usage of the thread creations/joins, your app will be slow and ineffective, it doesn't matter if it is N:M or not.
The Linux kernel does have the concept of POSIX pids (explorable in /proc/*) but it calls them thread group ids in the kernel source and it refers to its internal thread ids as pids (explorable in /proc/*/task/*).
I believe this is rooted in Linux's original treatment of threads as "just processes" that happen to share address spaces and a bunch of other stuff with each other.
Your user tool is likely propagating this perhaps confusing Linux kernel terminology.
in Linux threads are called light weight processes. Whether process or thread, they are implemented by task_struct data structure.
1> So, in that sense how kernel distinguishes between thread and process?
2> when context switching happens, how do threads get less overhead in context switching? because prior to this thread, another thread from another process may be running. So kernel should load all resources even if resources are shared between threads of a processes.
how kernel distinguishes between thread and process.
From http://www.kernel.org/doc/ols/2002/ols2002-pages-330-337.pdf and from Linux - Threads and Process
This was addressed during the 2.4 development cycle with the
addition of a concept called a ’thread group’. There is a linked
list of all tasks that are part of the thread group, and there is an
ID that represents the group, called the tgid. This ID is actually the
pid of the first task in the group (pid is the task ID assigned with a
Linux task), similar to the way sessions and process groups work. This
feature is enabled via a flag to clone().
and
In the kernel, each thread has it's own ID, called a PID (although it would possibly make more sense to call this a TID, or thread ID) and they also have a TGID (thread group ID) which is the PID of the thread that started the whole process.
Simplistically, when a new process is created, it appears as a thread
where both the PID and TGID are the same (new) number.
When a thread starts another thread, that started thread gets its own
PID (so the scheduler can schedule it independently) but it inherits
the TGID from the original thread.
So a main thread is a thread with the same PID and TGID and this PID is a process PID. A thread (but not a main thread) has different PID but the same TID.
Inside kernel, each process and threads have a unique id (even threads of same process) which is stored in pid variable and threads of same process also share a common id which is stored in tgid variable, and is returned to user when getpid() is invoked therefore allowing kernel to distinguish them as different entities which are schedulable in themselves.
When a thread is preempted by another thread of same process, since various segments such as .text, .bss, .data, file descriptors etc. are shared and hence allows a fast context switch compared to when different processes are context switched, or when threads of different processes are context switched.
It seems you mixed some concepts together, implemented by the same data structure does not mean they run in the same way.
you can read
what-is-the-difference-between-a-process-and-a-thread to clarify your comprehension about process and thread firstly.
My understanding is that threads and processes are really the same entity on Linux, the difference being in what memory is shared between them. I'm finding that it's...difficult to ensure that child processes are properly cleaned up without explicit communication between the parent and child. I'd like to be able to run sub-processes with a similar mental model as threads, in that they're cleaned up automatically when the parent exits, but with the memory safety that processes provide. How does Linux manage to clean up threads automatically, and can that same mechanism be used for child processes?
After reading the Linux source, I think I have the answer. Tasks are differentiated by their task ID and thread group ID. getpid() actually returns the thread group ID of the tasks, which is the same for all tasks in the group. This lets the kernel have a single notion of schedulable task which can be used to implement threading.
Since glibc 2.3, exit() actually invokes the exit_group syscall, rather than just the exit syscall. This syscall kills all the tasks in a thread group rather than just the calling task. It does this by sending a SIGKILL to all the tasks with the same thread ID.
I'm trying to find a way to uniquely identify threads in a multi-process environment. I have a server that keeps track of the different processes connecting to it, some of which are multi-threaded and some of which are not. To identify the threads from multi-threaded connections I'm using the thread ID as a unique identifier (there will be a maximum of 1 multi-threaded process connected at any given time). My question is: is it possible the thread ID of one of these threads could be the same as the process ID of another processes running on the system?
Thanks in advance for the help!
The TID (as returned by the sys_gettid() system call) is unique across all threads on the system1, and for a single-threaded process the PID and TID are equal. This means that a TID will never clash with a PID from another process.
1. With the caveat that if PID namespaces are in use, TIDs and PIDs are only unique within the same PID namespace.
According to the man page of pthreads the thread ID is unique within the creating process, so yes another thread or process could have the same ID. However, If it's unique within a process and a process ID is unique in the system then maybe you can use a combination of the two as a unique identifier.
Each of the threads in a process has a unique thread identifier
(stored in the type pthread_t). This identifier is returned to the
caller of pthread_create(3), and a thread can obtain its own thread
identifier using pthread_self(3). Thread IDs are only guaranteed to
be unique within a process.
While the pthread ID might not be unique, in a implementation where threads map to tasks, the task id (as seen in /proc/PID/task) will in fact be unique system wide, and have a form similar to an actual PID.
Well, I came across the same problem just now and here is my program for validation.
#include <pthread.h>
#include <stdio.h>
int main() {
printf("%lu\n", pthread_self());
}
clang -pthread test.c && strace ./a.out
Part of the output is as follows.
...
arch_prctl(ARCH_SET_FS, 0x7f53259be740) = 0
...
write(1, "139995089987392\n", 16139995089987392
) = 16
...
Then we know 0x7f53259be740 equals to 139995089987392 and the second argument of arch_prctl should be within the process address space(man arch_prctl). That is to say, the thread ID is a virtual address in fact. So if you use pthread_self() to identify threads in a multi-process environment, collisions may happen though it is a small chance.
pthread_equal(id1,id2)
It will compare the ID's of two threads and will return 0 if they are the same and a non zero number if they are different.
Just a quick question, if I clone a process, the PID of the cloned process is the same, yes ? fork() creates a child process where the PID differs, but everything else is the same. Vfork() creates a child process with the same PID. Exec works to change a process currently in execution to something else.
Am I correct in all of these statements ?
Not quite. If you clone a process via fork/exec, or vfork/exec, you will get a new process id. fork() will give you the new process with a new process id, and exec() replaces that process with a new process, but maintaining the process id.
From here:
The vfork() function differs from
fork() only in that the child process
can share code and data with the
calling process (parent process). This
speeds cloning activity significantly
at a risk to the integrity of the
parent process if vfork() is misused.
Neither fork() nor vfork() keep the same PID although clone() can in one scenario (*a). They are all different ways to achieve roughly the same end, the creation of a distinct child.
clone() is like fork() but there are many things shared by the two processes and this is often used to enable threading.
vfork() is a variant of clone in which the parent is halted until the child process exits or executes another program. It's more efficient in those cases since it doesn't involve copying page tables and such. Basically, everything is shared between the two processes for as long as it takes the child to load another program.
Contrast that last option with the normal copy-on-write where memory itself is shared (until one of the processes writes to it) but the page tables that reference that memory are copied. In other words, vfork() is even more efficient than copy-on-write, at least for the fork-followed-by-immediate-exec use case.
But, in most cases, the child has a different process ID to the parent.
*a Things become tricky when you clone() with CLONE_THREAD. At that stage, the processes still have different identifiers but what constitutes the PID begins to blur. At the deepest level, the Linux scheduler doesn't care about processes, it schedules threads.
A thread has a thread ID (TID) and a thread group ID (TGID). The TGID is what you get from getpid().
When a thread is cloned without CLONE_THREAD, it's given a new TID and it also has its TGID set to that value (i.e., a brand new PID).
With CLONE_THREAD, it's given a new TID but the TGID (hence the reported process ID) remains the same as the parent so they really have the same PID. However, they can distinguish themselves by getting the TID from gettid().
There's quite a bit of trickery going on there with regard to parent process IDs and delivery of signals (both to the threads within a group and the SIGCHLD to the parent), all which can be examined from the clone() man page.
It deserves some explanation. And it's simple as rain.
Consider this. A program has to do some things at the same time. Say, your program is printing "hello world!", each second, until somebody enters "hello, Mike", then, each second, it prints that string, waiting for John to change that in the future.
How do you write this the standard way? In your program, that basically prints "hello," you must create another branch that is waiting for user input.
You create two processes, one outputting those strings, and another one, waiting the user input. And, the only way to create a new process in UNIX was calling the system call fork(), like this:
ret = fork();
if(ret > 0) /* parent, continue waiting */
else /* child */
This scheme posed numerous problems. The user enters "Mike" but you have no simple way to pass that string to the parent process so that it'd be able to print that, because +each+ process has its own view of memory that isn't shared with the child.
When the processes are created by fork(), each one receives a copy of the memory existing at that moment, and if that memory really changes later, the mapping that was identical for those memory segments will be chaged at once (it's called a copy-on-write mechanism).
Another thingies to share between the child and the parent are, for example, opened file descriptors, descriptors of the shared memory, input/outpue stuff, etc., that also wouldn't survive after fork().
So. The very fork() call had to be alleviated, to include shared memory/signals etc. But how? This was the idea behind clone(). That call takes a flag indicating what exatly would you share with the child. For example, the memory, the signal handlers, etc. And if you call this with flag=0, this will be identical to fork(), up to the args they take. And when POSIX pthreads are created, that flag will reflect the attributes you have indicated in pthread_attr.
From the kernel point of view, there's no difference between the processes created such way, and no special semantics to differentiate the "processess". The kernel does not even know, what that "thread" is, it creates a new process, but it simply combines it as belogning to that process group that had the parent who called it, taking care what that process may do. So, you have different procesess (that share the same pid) combined in a process group each assigned with a different "TID" (that starts from PID of the parent).
Care to explain that clone() does exactly that. You may pass this whaterver you need (as the matter of fact, the old vfork() call will do). Are you going to share memory? Hanlers? You may tune everything, just be sure you don't clash with the pthreads library written right away around this very call.
An important thing, the kernel vesion is quite outrageous, it expects just 2 out of 4 parameters to be passed, the user stack, and options.
Since PID is an unique identifier for a process, there's no way to have two distinct process with the same PID.
Threads (which have the same visible 'pid') are implemented with the clone() call. When the flag CLONE_THREAD is supplied then the new process (a 'thread') share the Thread Group Identifier (TGID) with its creator process. getpid actually returns the TGID.
See the clone manpage for more details.
In summary the real PID, as seen by the kernel is always different. The visible PID is the same for threads.