mprotect() is used to protect memory pages, for example, making pages read-only. It sets this protection for the whole process, that is, if a page is read-only, no thread can write to that page. Is there a way to protect pages in different ways for different threads? For example, 1 thread can write to page P, and all other threads in my program can only read from P.
If you create a thread using CLONE_VM flag in the "clone" system call (this is what you would normally call a thread) then the MMU settings are the same as for the parent thread.
This means that write access is possible for both threads.
If you do not use CLONE_VM flags then the both threads do not have shared memory at all!
(pthread_create() sets the CLONE_VM flag internally).
It would be possible to do what you want - however it would be very difficult:
Allocate all memory blocks using shared memory functions (e.g. shmget()) instead of standard functions (e.g. malloc()).
If a new thread is created use "clone()" directly instead of "pthread_create()" with the CLONE_VM flag not set.
The shared memory is shared between the threads and the threads created by "normal" memory allocation functions (e.g. malloc()) is not shared between the threads. The same is true for mmap() mapped memory.
When a new thread is created such memory blocks (created by malloc or mmap) are copied so that both threads have their own copy of this memory block at the same address. If one thread writes to this address then the other thread will not see the change.
Allocating more "shared" memory is rather tricky. It is easy if the memory only should be shared between the allocating thread and child threads that are not created, yet. it is difficult to share memory between already-running threads or between threads that are (indirect) children of different already-running threads.
The threads do not have shared stack memory so they cannot access each other's stack.
Global and "static" variables are not shared by default - to make them "shared" between the threads some tricky programming is required.
With newer Intel CPUs you can use Memory Protection Keys [1] for different access settings per thread within a process. On Linux, run lscpu and check for the pku and ospke flags.
The example on the man page [2] is a bit outdated in the sense that the corresponding system calls do not need to be invoked manually anymore. Instead, glibc provides the following API calls:
pkey_alloc() to allocate a new key (16 are available)
pkey_set() to set the permission for a given key
pkey_mprotect() to apply the key to a given memory region
pkey_free() to free a key
Since the register that maintains the permission bits per protection key is thread-local, different protection settings are possible per thread. Protection key settings can only further lock down the general settings and do not affect instruction fetch.
[1] https://www.kernel.org/doc/Documentation/x86/protection-keys.txt
[2] http://man7.org/linux/man-pages/man7/pkeys.7.html
Related
I am using the libloading crate to load a dynamic library that I need to use in multiple threads.
let lib = Library::new("lib.dylib").unwrap();
Do I load the library on each thread, or is there a way to inject/share the library into a thread when the thread is started?
The shared libraries are loaded into a process. It shouldn't matter what thread you've used to load the library, the code should equally be available to all the threads once the library has been loaded. This is because the POSIX threads share the same memory space, namely the memory space of the process.
When a CPU jumps to a memory address belonging to a loaded library, the memory is already there, mapped by the operating system into the process memory space. Threads has no influence whatsoever on the availability of the said memory address.
Keep in mind though, that according to the library documentation you might want to avoid calling Library::new from multiple threads simultaneously.
As I got to know that apart from data segment and code segment threads also share heap segment
What resources are shared between threads??
then if I create a variable dynamically using malloc() or calloc() inside the thread then does that variable would be accessible to all the other threads of the same process?
Theoretically, if you know the memory address. Yes, heap allocated variables should be accessible from any thread within the same process.
{malloc, calloc, realloc, free, posix_memalign} of glibc-2.2+ are thread safe
http://linux.derkeiler.com/Newsgroups/comp.os.linux.development.apps/2005-07/0323.html
Original post
Generally, malloc/new/free/delete on multi threaded systems are thread safe, so this should be no problem - and allocating in one thread , deallocating in another is a quite common thing to do.
As threads are an implementation feature, it certainly is implementation dependant though - e.g. some systems require you to link with a multi threaded runtime library.
And this
Besides also answered in: the link you posted
Threads differ from traditional multitasking operating system
processes in that:
processes are typically independent, while threads exist as subsets of
a process processes carry considerable state information, whereas
multiple threads within a process share state as well as memory and
other resources processes have separate address spaces, whereas
threads share their address space processes interact only through
system-provided inter-process communication mechanisms. Context
switching between threads in the same process is typically faster than
context switching between processes.
So, yes it is.
I currently intend to implement threading using clone() and a question is, if I have all threads using the same memory space, with each function I call in a given thread, will each thread using a different part of memory when the same function is called, or do I do have todo something to ensure this happens?
Each thread will be using the same memory map overall but a different, separate thread-local stack for function calls. When different threads have called the same function (which itself lives at the same executable memory location), any local variables will not be shared, because they are allocated on the stack upon entry into the function / as needed, and each thread has its own stack by default.
References to any static/global memory (i.e., anything not allocated on the thread-local stack, such as globals or references to the heap or mmap memory regions passed to / visible to a thread after calling clone, and in general, the full memory map and process context itself) will, of course, be shared and subject to the usual host of multithreading issues (i.e., synchronization of shared state).
Note that you have to setup this thread-local stack space yourself before calling clone. From the manpage:
The child_stack argument specifies the location of the stack used by
the child process. Since the child and calling process may share
memory, it is not possible for the child process to execute in the
same stack as the calling process. The calling process must therefore
set up memory space for the child stack and pass a pointer to this
space to clone().
Stacks grow downward on all processors that run Linux (except the HP
PA processors), so child_stack usually points to the topmost address
of the memory space set up for the child stack.
The child_stack parameter is the second argument to clone. It's the responsibility of the caller (i.e., the parent thread) to ensure that each child thread created via clone receives a separate and non-overlapping chunk of memory for its stack.
Note that allocating and setting-up this thread-local stack memory region is not at all simple. Ensure that your allocations are page-aligned (start address is on a 4K boundary), a multiple of the page size (4K), amply-sized (if you only have a few threads, 2MB is safe), and ideally contains a "guard" section following the usable space. The stack guard is some number of pages with no access privileges-- no reading, no writing-- following the main stack region to guard the rest of the virtual memory address space should a thread dynamically exceed its stack size (e.g., with a bunch of recursion or functions with very large temporary buffers as local variables) and try to continue to grow into the stack guard region, which will fail-early as the thread will be served a SIGSEGV right away rather than insidiously corrupting. The stack guard is technically optional. You should probably be using mmap to allocate your stacks, although posix_memalign would do as well.
All that said, I've got to ask: why try to implement threading with clone to start? There are some very challenging problems here, and the POSIX threading library has solved them (in a portable way as well). If it's the fine-grained control of clone you want, then checkout the pthread_attr_* functions; they pretty much cover every non-obscure use case (such as allowing you to allocate your own stack if you like-- from the previous discussion, I'd think you wouldn't). The very performant, general Linux implementation, amongst other things, fully wraps clone and a large variety of other heinous system calls relevant to threading-- many of which do not even have C library wrappers and must be called via syscall. It all depends upon what you want to do.
After thinking about the the whole concept of shared memory , a question came up:
can two processes share the same shared memory segment ? can two threads share the same shared memory ?
After thinking about it a little more clearly , I'm almost positive that two processes can share the same shared memory segment , where the first is the father and the second is the son , that was created with a fork() , but what about two threads ?
Thanks
can two processes share the same shared memory segment?
Yes and no. Typically with modern operating systems, when another process is forked from the first, they share the same memory space with a copy-on-write set on all pages. Any updates made to any of the read-write memory pages causes a copy to be made for the page so there will be two copies and the memory page will no longer be shared between the parent and child process. This means that only read-only pages or pages that have not been written to will be shared.
If a process has not been forked from another then they typically do not share any memory. One exception is if you are running two instances of the same program then they may share code and maybe even static data segments but no other pages will be shared. Another is how some operating systems allow applications to share the code pages for dynamic libraries that are loaded by multiple applications.
There are also specific memory-map calls to share the same memory segment. The call designates whether the map is read-only or read-write. How to do this is very OS dependent.
can two threads share the same shared memory?
Certainly. Typically all of the memory inside of a multi-threaded process is "shared" by all of the threads except for some relatively small stack spaces which are per-thread. That is usually the definition of threads in that they all are running within the same memory space.
Threads also have the added complexity of having cached memory segments in high speed memory tied to the processor/core. This cached memory is not shared and updates to memory pages are flushed into central storage depending on synchronization operations.
In general, a major point of processes is to prevent memory being shared! Inter-process comms via a shared memory segment is certainly possible on the most common OS, but the mechanisms are not there by default. Failing to set up, and manage, the shared area correctly will likely result in a segFault/AV if you're lucky and UB if not.
Threads belonging to the same process, however, do not have such hardware memory-management protection can pretty much share whatever they like, the obvious downside being that they can corrupt pretty much whatever they like. I've never actually found this to be a huge problem, esp. with modern OO languages that tend to 'structure' pointers as object instances, (Java, C#, Delphi).
Yes, two processes can both attach to a shared memory segment. A shared memory segment wouldn't be much use if that were not true, as that is the basic idea behind a shared memory segment - that's why it's one of several forms of IPC (inter-Process communication).
Two threads in the same process could also both attach to a shared memory segment, but given that they already share the entire address space of the process they are part of, there likely isn't much point (although someone will probably see that as a challenge to come up with a more-or-less valid use case for doing so).
In general terms, each process occupies a memory space isolated from all others in order to avoid unwanted interactions (including those which would represent security issues). However, there is usually a means for processes to share portions of memory. Sometimes this is done to reduce RAM footprint ("installed files" in VAX/VMS is/was one such example). It can also be a very efficient way for co-operating processes to communicate. How that sharing is implemented/structured/managed (e.g. parent/child) depends on the features provided by the specific operating system and design choices implemented in the application code.
Within a process, each thread has access to exactly the same memory space as all other threads of the same process. The only thing a thread has unique to itself is "execution context", part of which is its stack (although nothing prevents one thread from accessing or manipulating the stack "belonging to" another thread of the same process).
I was studying about virtual memory management from galvin ,I am unable to understand this statement :
In addition to separating logical memory from physical memory ,virtual
memory allows files and memory to be shared by two or more processes
through page sharing .This leads to the following benefits
Virtual memory can allow pages to be shared during process creation with fork() system call,thus speeding up process creation .
How can pages be shared with fork()? Please clarify.
I believe the text is referring to the copy-on-write optimisation done for fork().
Basically a fork() clones a process, duplicating its entire memory.
This can take a long time, especially for processes which use a lot of money. Moreover, it's very common for a fork() to be immediately followed by an exec(), rendering the previous copy pointless.
Instead of doing all of that work for every fork() modern Unixes create the new process, but don't copy all of the memory. They just point the virtual memory pages for both the original process and the new one to the same physical pages.
This greatly reduces the cost of a fork(), in terms of reduced copies and reduced memory usage.
The downside is that whenever either the fork()ed process or the original process write to a page the write raises an exception (because the physical pages are marked read-only) and the page is copied after all.
Fortunately it turns out this doesn't happen all that often.
pages may be shared by fork if two processes having same or different virtual address page share same physical memory frame. they have same entry of frame number in their page table