in Linux threads are called light weight processes. Whether process or thread, they are implemented by task_struct data structure.
1> So, in that sense how kernel distinguishes between thread and process?
2> when context switching happens, how do threads get less overhead in context switching? because prior to this thread, another thread from another process may be running. So kernel should load all resources even if resources are shared between threads of a processes.
how kernel distinguishes between thread and process.
From http://www.kernel.org/doc/ols/2002/ols2002-pages-330-337.pdf and from Linux - Threads and Process
This was addressed during the 2.4 development cycle with the
addition of a concept called a ’thread group’. There is a linked
list of all tasks that are part of the thread group, and there is an
ID that represents the group, called the tgid. This ID is actually the
pid of the first task in the group (pid is the task ID assigned with a
Linux task), similar to the way sessions and process groups work. This
feature is enabled via a flag to clone().
and
In the kernel, each thread has it's own ID, called a PID (although it would possibly make more sense to call this a TID, or thread ID) and they also have a TGID (thread group ID) which is the PID of the thread that started the whole process.
Simplistically, when a new process is created, it appears as a thread
where both the PID and TGID are the same (new) number.
When a thread starts another thread, that started thread gets its own
PID (so the scheduler can schedule it independently) but it inherits
the TGID from the original thread.
So a main thread is a thread with the same PID and TGID and this PID is a process PID. A thread (but not a main thread) has different PID but the same TID.
Inside kernel, each process and threads have a unique id (even threads of same process) which is stored in pid variable and threads of same process also share a common id which is stored in tgid variable, and is returned to user when getpid() is invoked therefore allowing kernel to distinguish them as different entities which are schedulable in themselves.
When a thread is preempted by another thread of same process, since various segments such as .text, .bss, .data, file descriptors etc. are shared and hence allows a fast context switch compared to when different processes are context switched, or when threads of different processes are context switched.
It seems you mixed some concepts together, implemented by the same data structure does not mean they run in the same way.
you can read
what-is-the-difference-between-a-process-and-a-thread to clarify your comprehension about process and thread firstly.
Related
I was reading about thread implementation in linux. It seems that linux infact models threads as "special" processes (using clone system call) passing appropriate flags to it so that address space is shared.
Multiple threads that spawned this way would have separate pid (process id - logical thread id) but same tgid (thread group id - logical process id). How would thread spawning be quicker than process spawning (since threads are modelled as process)?
How would operations like TLB flushes work? Would context switch between (logical) threads still be be cheap?
My understanding is that threads and processes are really the same entity on Linux, the difference being in what memory is shared between them. I'm finding that it's...difficult to ensure that child processes are properly cleaned up without explicit communication between the parent and child. I'd like to be able to run sub-processes with a similar mental model as threads, in that they're cleaned up automatically when the parent exits, but with the memory safety that processes provide. How does Linux manage to clean up threads automatically, and can that same mechanism be used for child processes?
After reading the Linux source, I think I have the answer. Tasks are differentiated by their task ID and thread group ID. getpid() actually returns the thread group ID of the tasks, which is the same for all tasks in the group. This lets the kernel have a single notion of schedulable task which can be used to implement threading.
Since glibc 2.3, exit() actually invokes the exit_group syscall, rather than just the exit syscall. This syscall kills all the tasks in a thread group rather than just the calling task. It does this by sending a SIGKILL to all the tasks with the same thread ID.
I have a question regarding linux kernel scheduling.
We know that, usually, linux maintains the current largest pid. If we want to start a new process, the kernel will use that largest id. So, if we kill and restart a new process, the process id are not sequential. Linux will use the largest id until it hits a limit.
But my question is how linux decides thread ID.
Say, process A and B are running. Process A crashes but process B is spawning new threads. Will process B just reuse that old tid belonging to process A, or, process B will also use the largest id as tid. Which case is more often? Do we have documents?
Thanks.
The kernel sets a maximum number of process/thread ids and simply recycles identifiers when the threads are garbage collected. So if process B spawns enough threads it will eventually reclaim thread ids from process A assuming it has been properly destroyed
Edit: Here are some links that can provide you with more specific answers
Difference between pid and tid
https://stackoverflow.com/a/8787888/5768168
"what is the value range of thread and process id?"
what is the value range of thread and process id?
"Linux PID recycling"
https://stackoverflow.com/a/11323428/5768168
"Process identifer"
https://en.wikipedia.org/wiki/Process_identifier#Unix-like
"The Linux kernel: Processes"
https://www.win.tue.nl/~aeb/linux/lk/lk-10.html
It sounds like you need to run your threads in with a PTHREAD_CREATE_JOINABLE attribute passed to pthread_create(), then have one reaper thread in your process dedicated to using pthread_join() or pthread_tryjoin() to wait for terminated threads. Rather than having an outside process trying to sort it out, have your process record the PID/TID pair after pthread_create() succeeds and have the reaper thread remove the pair when it detects the thread has terminated.
I typically combined that with a main thread that did nothing but spawn the thread-creation and reaper threads, then wait for a termination signal and terminate the thread-creator and reaper. The thread-creator stops immediately when signaled, the reaper stops when no more unterminated threads are running, the main thread terminates when both the thread-creator and reaper threads can be pthread_join()'d. Since the main thread's so simple it's unlikely to crash, which means most crashes in work threads simply deliver them to the reaper. If you want absolute certainty, your outside process should be the one to start your main process, then it can use wait() or it's siblings to monitor whether the main process has terminated (normally or by crashing).
What is the difference between PID and TID?
The standard answer would be that PID is for processes while TID is for threads. However, I have seen that some commands use them interchangeably. For example, htop has a column for PIDs, in which PIDs for threads of the same process are shown (with different values). So when does a PID represent a thread or a process?
It is complicated: pid is process identifier; tid is thread identifier.
But as it happens, the kernel doesn't make a real distinction between them: threads are just like processes but they share some things (memory, fds...) with other instances of the same group.
So, a tid is actually the identifier of the schedulable object in the kernel (thread), while the pid is the identifier of the group of schedulable objects that share memory and fds (process).
But to make things more interesting, when a process has only one thread (the initial situation and in the good old times the only one) the pid and the tid are always the same. So any function that works with a tid will automatically work with a pid.
It is worth noting that many functions/system calls/command line utilities documented to work with pid actually use tids. But if the effect is process-wide you will simply not notice the difference.
Actually, each thread in a Linux process is Light Weight Process (LWP). So, people may call thread as a process... But there is surely a difference.
Each thread in a process has a different thread ID (TID) and share the same process ID (PID).
If you are working with pthread library functions, then these functions don't use these TIDs because these are kernel/OS level thread IDs.
Just to add to other answers, according to man gettid:
The thread ID returned by this call is not the same thing as a POSIX thread ID (i.e., the opaque value returned by pthread_self(3)).
So there are two different things one could mean by TID!
pid and tid are the same except when a process is created with a call to clone with CLONE_THREAD (per the man pages of gettid). In this case, you get a unique thread id but all threads belonging to the same thread group share the same process id.
However, I also recall reading (though I cant find the source) that the values returned from getpid may be cached.
[UPDATE]
See the NOTES section here for a discussion on the effects of caching pids.
I have couple of questions on threads. Could you please clarify.
Suppose process with one or multiple threads. If the process is prempted/suspended, does the threads also get preempted or does the threads continue to run?
When the suspended process rescheduled, does the process threads also gets scheduled? If the process has process has multiple threads, which threads will be rescheduled and on what basis?
if the thread in the process is running and recieves a signal(say Cntrl-C) and the default action of the signal is to terminate a process, does the running thread terminates or the parent process will also terminate? What happens to the threads if the running process terminates because of some signal?
If the thread does fork fallowed exec, does the exece'd program overlays the address space of parent process or the running thread? If it overlays the parent process what happens to threads, their data, locks they are holding and how they get scheduled once the exec'd process terminates.
Suppose process has multiple threads, how does the threads get scheduled. If one of the thread blocks on some I/O, how other threads gets scheduled. Does the threads scheduled with the parent process is running?
While the thread is running what the current kernel variable points(parent process task_stuct or threads stack_struct?
If the process with the thread is running, when the thread starts does the parent
process gets preempted and how each threads gets scheduled?
If the process running on CPU creates multiple threads, does the threads created by the parent process schedule on another CPU on multiprocessor system?
Thanks,
Ganesh
First, I should clear up some terminology that you appear to be confused about. In POSIX, a "process" is a single address space plus at least one thread of control, identified by a process ID (PID). A thread is an individually-scheduled execution context within a process.
All processes start life with just one thread, and all processes have at least one thread. Now, onto the questions:
Suppose process with one or multiple threads. If the process is prempted/suspended, does the threads also get preempted or does the threads continue to run?
Threads are scheduled independently. If a thread blocks on a function like connect(), then other threads within the process can still be scheduled.
It is also possible to request that every thread in a process be suspended, for example by sending SIGSTOP to the process.
When the suspended process rescheduled, does the process threads also gets scheduled? If the process has process has multiple threads, which threads will be rescheduled and on what basis?
This only makes sense in the context that an explicit request was made to stop the entire process. If you send the process SIGCONT to restart the process, then any of the threads which are not blocked can run. If more threads are runnable than there are processors available to run them, then it is unspecified which one(s) run first.
If the thread in the process is running and recieves a signal(say Cntrl-C) and the default action of the signal is to terminate a process, does the running thread terminates or the parent process will also terminate? What happens to the threads if the running process terminates because of some signal?
If a thread recieves a signal like SIGINT or SIGSEGV whose action is to terminate the process, then the entire process is terminated. This means that every thread in the process is unceremoniously killed.
If the thread does fork followed by exec, does the exece'd program overlays the address space of parent process or the running thread? If it overlays the parent process what happens to threads, their data, locks they are holding and how they get scheduled once the exec'd process terminates.
The fork() call creates a new process by duplicating the address space of the original process, and duplicating just the single thread that called fork() within that new address space.
If that thread in the new process calls execve(), it will replace the new, duplicated address space with the exec'd program. The original process, and all its threads, continue running normally.
Suppose process has multiple threads, how does the threads get scheduled. If one of the thread blocks on some I/O, how other threads gets scheduled. Does the threads scheduled with the parent process is running?
The threads are scheduled independently. Any of the threads that are not blocked can run.
While the thread is running what the current kernel variable points(parent process task_stuct or threads stack_struct?
Each thread has its own task_struct within the kernel. What userspace calls a "thread" is called a "process" in kernel space. Thus current always points at the task_struct corresponding to the currently executing thread (in the userspace sense of the word).
If the process with [a second] thread is running, when the thread starts does the parent process gets preempted and how each threads gets scheduled?
Presumably you mean "the process's main thread" rather than "parent process" here. As before, the threads are scheduled independently. It's unspecified whether one runs before the other - and if you have multiple CPUs, both might run simultaneously.
If the process running on CPU creates multiple threads, does the threads created by the parent process schedule on another CPU on multiprocessor system?
That's really up to the kernel, but the threads are certainly allowed to execute on other CPUs.
Depends. If a thread is preempted because the OS scheduler decides to give CPU time to some other thread, then other threads in the process will continue running. If the process is suspended (i.e. it gets the SIGSTP signal) then AFAIK all the threads will be suspended.
When a suspended process is woken up, all the threads are marked as waiting or blocked (if they are waiting e.g. on a mutex). Then the scheduler at some points run them. There is no guarantee about any specific order the threads are run after waking up the process.
The process will terminate, and with it the threads as well.
When you fork you get a new address space, so there is no "overlay". Note that fork() and the exec() family affect the entire process, not only the thread from which they where called. When you call fork() in a multi-threaded process, the child gets a copy of that process, but with only the calling thread. Then if you call exec() in one or both of the processes (presumably only in the child process, but that's up to you), then the process which calls exec() (and with it, all its threads) is replaced by the exec()'ed program.
The thread scheduling order is decided by the OS scheduler, there is no guarantee given about any particular order.
From the kernel perspective a process is an address space with one or more threads (and some other gunk). There is no concept of threads that somehow exist without a process.
There is no such thing as a process without a single thread. A "plain process" is just a process with a single thread.
Probably yes. This is determined by the OS scheduler. Note that there are API's and tools (numactl) that one can use to force some thread(s) to run on a specific CPU core.
Assuming your questions are about POSIX threads, then
1a. A process that's preempted by the O/S will have all its threads preempted.
1b. The O/S will suspend all the threads of a process that is sent a SIGSTOP.
The O/S will resume all thread of a suspended process that is sent a SIGCONT.
By default, a SIGINT will terminate all the threads in a process.
If a thread calls fork(), then all its threads are duplicated. If it then call one of the exec() functions, then all the duplicated threads disappear.
POSIX allows for user-selection of the thread scheduling algorithm.
I don't understand the question.
I don't understand the question.
How threads are mapped to CPU-s is implementation-dependent. Many implementations will try to distribute threads amongst the available CPU-s to improve performance.
The Linux kernel doesn't distinguish between threads and processes. As far as kernel is concerned, a thread is simply another process which happens to share address space with other processes. (You would call the set of "processes" (i.e. threads) which share a single address space a "process".)
So POSIX threads are scheduled exactly as full-blown processes would be. There is no difference in scheduling whether you have one process with five threads, or five separate processes.
There are kernel calls that provide fine grained control over what is shared between processes. The POSIX threads API wraps over them.