I was wondering if threads created via the pthreads library are actually kernel-level threads or user-space threads that have nothing to do with the kernel? I have heard mutually exclusive opinions about this, so I want to know the truth.
Before Linux 2.6 they were essentially userspace threads, separate processes which were glued together, because the kernel had no real thread support. Edit: There was some limited support for kernel level threads (a clone() function) before 2.6, but it wasn't used with posix threads, only with an alternative thread library called linuxthreads.
Since the arrival of NPTL (Native Posix Thread Library) the are kernel threads.
Threads created by pthread_create() on Linux have always been kernel-level threads. LinuxThreads didn't comply fully with POSIX (threads in a same process had different pid's, signal handling was different than what POSIX requires, ...), so NPTL was created to fix those issues.
There are libraries that implement user-level threads (e.g: GNU pth, the p is for Portable), but they don't use the POSIX thread API.
Related
Before the futex system calls existed in Linux, what underlying system calls were used by threading libraries like pthreads to block/sleep a thread and to subsequently wake those threads from userland?
For example, if a thread tries to acquire a mutex, the userland implementation will block the thread (perhaps after a short spinning interval), but I can't find the syscalls that are used for this (other than futex which are a relatively recent creation).
Before futex and current implementation of pthreads for Linux, the NPTL (require kernel 2.6 and newer), there were two other threading libraries with POSIX Thread API for Linux: linuxthreads and NGPT (which was based on Gnu Pth. LinuxThreads was the only widely used libpthread for years (and it can still be used in some strange & unmaintained micro-libc to work on 2.4; other micro-libc variants may have own builtin implementation of pthread-like API on top of futex+clone). And Gnu Pth is not thread library, it is single process thread with user-level "thread" switching.
You should know that there are several Threading Models when we check does the kernel knows about some or all of user threads (how many CPU cores can be used with adding threads to the program; what is the cost of having the thread / how many threads may be started). Models are named as M:N where M is userspace thread number and N is thread number schedulable by OS kernel:
"1:1" ''kernel-level threading'' - every userspace thread is schedulable by OS kernel. This is implemented in Linuxthreads, NPTL and many modern OS.
"N:1" ''user-level threading'' - userspace threads are planned by the userspace, they all are invisible to the kernel, it only schedules one process (and it may use only 1 CPU core). Gnu Pth (GNU Portable Threads) is example of it, and there are many other implementations for some computer architectures.
"M:N" ''hybrid threading'' - there are some entities visible and schedulable by OS kernel, but there may be more user-space threads in them. And sometimes user-space threads will migrate between kernel-visible threads.
With 1:1 model there are many classic sleep mechanisms/APIs in Unix like select/poll and signals and other variants of IPC APIs. As I remember, Linuxthreads used separate processes for every thread (with fully shared memory) and there was special manager "thread" (process) to emulate some POSIX thread features. Wikipedia says that SIGUSR1/SIGUSR2 were used in Linuxthreads for some internal communication between threads, same says IBM "The synchronization of primitives is achieved by means of signals. For example, threads block until awoken by signals.". Check also the project FAQ http://pauillac.inria.fr/~xleroy/linuxthreads/faq.html#H.4 "With LinuxThreads, I can no longer use the signals SIGUSR1 and SIGUSR2 in my programs! Why?"
LinuxThreads needs two signals for its internal operation. One is used to suspend and restart threads blocked on mutex, condition or semaphore operations. The other is used for thread cancellation.
On ``old'' kernels (2.0 and early 2.1 kernels), there are only 32 signals available and the kernel reserves all of them but two: SIGUSR1 and SIGUSR2. So, LinuxThreads has no choice but use those two signals.
With "N:1" model thread may call some blocking syscall and block everything (some libraries may convert some blocking syscalls into async, or use some SIGALRM or SIGVTALRM magic); or it may call some (very) special internal threading function which will do user-space thread switching by rewriting machine state register (like switch_to in linux kernel, save IP/SP and other regs, restore IP/SP and regs of other thread). So, kernel does not wake any user thread directly from userland, it just schedules whole process; and user space scheduler implement thread synchronization logic (or just calls sched_yield or select when there is no threads to work).
With M:N model things are very complicated... Don't know much about NGPT... There is one paragraph about NGPT in POSIX Threads and the Linux Kernel, Dave McCracken, OLS2002,330 page 5
There is a new pthread library under development called NGPT. This library is based on the GNU Pth library, which is an M:1 library. NGPT extends Pth by using multiple Linux tasks, thus creating an M:N library. It attempts to preserve Pth’s pthread compatibility while also using multiple Linux tasks for concurrency, but this effort is hampered by the underlying differences in the Linux threading model. The NGPT library at present uses non-blocking wrappers around blocking system calls to avoid
blocking in the kernel.
Some papers and posts: POSIX Threads and the Linux Kernel, Dave McCracken, OLS2002,330, LWN post about NPTL 0.1
The futex system call is used extensively in all synchronization
primitives and other places which need some kind of
synchronization. The futex mechanism is generic enough to support
the standard POSIX synchronization mechanisms with very little
effort. ... Futexes also allow the implementation of inter-process
synchronization primitives, a sorely missed feature in the old
LinuxThreads implementation (Hi jbj!).
NPTL design pdf:
5.5 Synchronization Primitives
The implementation of the synchronization primitives such as mutexes, read-write
locks, conditional variables, semaphores, and barriers requires some form of kernel
support. Busy waiting is not an option since threads can have different priorities (beside wasting CPU cycles). The same argument rules out the exclusive use of sched yield. Signals were the only viable solution for the old implementation. Threads would block in the kernel until woken by a signal. This method has severe drawbacks in terms of speed and reliability caused by spurious wakeups and derogation of the quality of the signal handling in the application.
Fortunately some new functionality was added to the kernel to implement all kinds
of synchronization primitives: futexes [Futex]. The underlying principle is simple but
powerful enough to be adaptable to all kinds of uses. Callers can block in the kernel
and be woken either explicitly, as a result of an interrupt, or after a timeout.
Futex stands for "fast userspace mutex." It's simply an abstraction over mutexes which is considered faster and more convenient than traditional mutex mechanisms because it implements the wait system for you. Before and after futex(), threads were put to sleep and awoken via a change in their process state. The process states are:
Running state
Sleeping state
Un-interruptible sleeping state (i.e. blocking for a syscall like read() or write()
Defunct/zombie state
When a thread is suspended, it is put into (interruptible) 'sleep' state. Later, it can be woken via the wake_up() function, which operates on its task structure within the kernel. As far as I can tell, wake_up is a kernel function, not a syscall. The kernel doesn't need a syscall to wake or sleep a task; it (or a process) simply changes the task structure to reflect the state of the process. When the Linux scheduler next deals with that process, it treats it according to its state (again, the states are listed above).
Short story: futex() implements a wait system for you. Without it, you need a data structure that's accessible from the main thread and from the sleeping thread in order to wake up a sleeping thread. All of this is done with userland code. The only thing you might need from the kernel is a mutex--the specifics of which do include locking mechanisms and mutex datastructures, but don't inherently wake or sleep the thread. The syscalls you're looking for don't exist. Essentially, most of what you're talking about can be achieved from userspace, without a syscall, by manually keeping track of data conditions that determine whether and when to sleep or wake a thread.
My CS professor told to the class that the OS has no idea that an application has launched threads. Is this true?
It depends on the type of thread. Threads implemented purely at user-level would be unknown to the operating system. This can be done with signals and setjmp and longjmp (see www.gnu.org/s/pth/rse-pmt.ps for details). Alternatively, if you are talking about something such as Linux pthreads, which only implements of subset of the pthreads specification, specifically the part that involves create new threads of execution that the kernel are aware of and schedules then the kernel is aware.
If you want to see more detail about how the kernel is aware you can look at the the clone system call. This system call can be used to create a new thread of execution that shares the address space of the calling process.
Also in the case of user-space implemented threading you will not get true parallelism, in the sense that two threads will be executing at the exact same time on different cores/hardware threads, because the operating system, which does the scheduling, does not know about the multiple threads.
It depends upon the operating system. Older operating system had no threads. Programming libraries would implement threads (e.g., Ada tasks) with timers. The library included a thread scheduler.
It is increasingly common now for operating systems to schedule threads for execution. There, the OS is aware of threads.
I'm just curious to if there is any difference to how threads work in the different flavors of Unix.
In particular Linux, Solaris and the original Unix
Any thoughts?
The original Unix does not use threads for multiprogramming. An article from 1987 comparing the Mach and Unix kernel states that "neither Unix System V nor 4.3 BSD provide a way to manage more than one thread of control within an address space".
Modern Unix clones adhere to the POSIX Threads specification. The POSIX Threads standard only appeared in 1995. The underlying implementation in Linux is based on the clone() and futex() system calls, which are implemented by the kernel. The clone() system call creates a new lightweight process that can share a memory space with its child. So, for example, the pthread_create() library call internally calls the clone() system call. The futex() system call implements a synchronization primitive that can be used for creating larger synchronization operations, like mutexes, semaphores, etc. So, for example, pthread_mutex_lock() will call futex() internally.
Using POSIX terminology, Linux implementation model is a "kernel-thread model", also known as 1:1 model, where 1 kernel lightweight process is used for 1 user visible thread.
Linux also had different threading implementations over time. More description about threading support in Linux in pthreads(7), clone(2) and futex(2) manual pages.
Other implementations may be completely different and still implement the POSIX threads API. FreeBSD used a M:N implementation, or "hybrid model", where an user visible thread could be managed either by the kernel, or by a user space library. See the kse(2) manual page for a complete description. I have once traced the current 1:1 FreeBSD threading implementation to an article in kerneltrap:
http://web.archive.org/web/20110512021159/http://kerneltrap.org/node/624
OpenSolaris doesn't seem to describe its own threading implementation on man pages. I could find instead a document comparing the OpenSolaris kernel with Linux 2.6 and Windows Vista. There, a main difference between the Linux and OpenSolaris implementation is that in Linux, each process is a lightweight process, unifying threads and processes, while in OpenSolaris one process contains multiple lightweight processes, in practice separating in the kernel the process from the kernel thread.
References:
http://en.wikipedia.org/wiki/Thread_(computing)
http://en.wikipedia.org/wiki/POSIX_Threads
http://repository.cmu.edu/cgi/viewcontent.cgi?article=2728&context=compsci
http://man7.org/linux/man-pages/man7/pthreads.7.html
http://man7.org/linux/man-pages/man2/clone.2.html
http://man7.org/linux/man-pages/man2/futex.2.html
http://pubs.opengroup.org/onlinepubs/7908799/xsh/threads.html
http://www.freebsd.org/cgi/man.cgi?query=kse&apropos=0&sektion=0&manpath=FreeBSD+7.2-RELEASE&format=html
http://web.archive.org/web/20110512021159/http://kerneltrap.org/node/624
http://www.unix.com/man-page/opensolaris/5/threads/
http://www.infoq.com/articles/kernel-comparison-unix-zhu
I'm getting confused on the idea of "pthread" and "thread". I know pthread is short form for POSIX thread, which is a type of standardized thread used in UNIX. But people often use "thread" to refer a thread. Are pthread and thread equivalent? Or pthread is only the name for threads used in UNIX?
Thanks in advance.
Threads are a generic concept. Wikipedia defines it as:
In computer science, a thread of execution is the smallest sequence of programmed instructions that can be managed independently by an operating system scheduler. A thread is a light-weight process.
Pthreads or POSIX threads are one implementation of that concept used with C program on Unix. Most modern languages have their own implementation of threads. From that web page:
Pthreads are defined as a set of C language programming types and procedure calls, implemented with a pthread.h header/include file and a thread library - though this library may be part of another library, such as libc, in some implementations.
To add to Gray,
Pthread is POSIX complaint which means you can use it across most of the UNIX operating systems.
No need to rewrite them for each of Unix (Linxux,FreeBSD, etc) and the behavior would be same across all of them .
Pthreads refers to the POSIX standard (IEEE 1003.1c) defining an API for thread creation and synchronization. This is a specification for thread behavior, not an implementation. Operating-system designers may implement the specification in any way they wish. Numerous systems implement the Pthreads specification; most are UNIX-type systems, including Linux, Mac OS X, and Solaris. Although Windows doesn’t support Pthreads natively, some third- party implementations for Windows are available.
Three main thread libraries are in use today: POSIX Pthreads, Windows, and Java. Pthreads, the threads extension of the POSIX standard, may be provided as either a user-level or a kernel-level library. The Windows thread library is a kernel-level library available on Windows systems. The Java thread API allows threads to be created and managed directly in Java programs. However, because in most instances the JVM is running on top of a host operating system, the Java thread API is generally implemented using a thread library available on the host system. This means that on Windows systems, Java threads are typically implemented using the Windows API; UNIX and Linux systems often use Pthreads
I have gone through mostly all the questions on here regarding the topic of Pthreads in Linux but there is a basic doubt which remains unresolved for me:
It is mentioned in various responses that when we create a POSIX thread on Linux, there is a 1:1 mapping between user thread and kernel thread.
My doubt is when we use pthread_create() in Linux, is there 1 user thread and a corresponding unique kernel thread created implicitly (i.e. a total of 2 threads, one of which is invisible to the user)?
OR
There only one kernel thread created and there is nothing anymore in newer Linux kernels called a user thread?
The NPTL (Native POSIX Thread Library) and the older LinuxThreads both use a 1:1 model, where each threads (or process) created by the user corresponds with one schedulable entity in the kernel.
However, you maybe confused by user-level threads, or fibers, those are threads of execution created via calls like makecontext() and swapcontext() that have N:1 model, the kernel doesn't know about user-level threads and their scheduling is done in user-space.
pthread_create() internally calls fork(), and vfork()/fork() internally calls clone(). So in most of the case it is 1:1 mapping.