I am using Apache/2.4.41 (Ubuntu), with mod_perl. Apache uses multiple threads, and I want to share read/WRITE data between threads. In "Practical mod_perl" Stas Bekman describes how to share read-only data between threads, but says nothing about how to share WRITABLE data between threads, so that one thread can modify data that all threads see.
How can this be done, and what mechanisms are available -- such as locks or semaphores -- to coordinate shared data access between threads?
Suggestions received via the modperl mailing list:
Use the IPC::Sharable module to access memory that can be shared by multiple processes (and threads). (Credit: Jacques Deguest)
Use a cache server (Redis/Varnish/Elasticache/etc. (Credit: Brad Van Sickle)
Reconsider my approach, because mod_perl's copy-on-write architecture is a bad fit for this. (Credit: Brad Van Sickle)
Related
I want to write a "embedded control system", on Linux
In order to make it easy to update in the future, i think multi-process is better than muti-thread
so, this system maybe separate into 3 program
"process", read some input data from others and do some calculation, then save the result to shared memory
"display", read instant data from shared memory and choose some data to display on UI(written by Qt)
"database", read instant data from shared memory and save them in a period, data will be saved in binary files at first, and maybe use sqlite instead in the future
and nore, maybe i will add a web server to read the instant data and show it through the browser
here is the question:
is multi-process really better multi-thread?
If use multi-process, how about use shared memory, is there any disadvantage.
is multi-process really better multi-thread?
Depends on what you want to do. Multiprocessing enforces a strict separation between system components, allowing for various parts to run with different credentials. It does require a more complicated communication mechanism than multithreading and incurs some overhead.
If use multi-process, how about use shared memory, is there any disadvantage.
The main disadvantage, compared to the obvious alternative of using sockets, is that it limits your entire system to running on a single host. No distributed computing.
You're already using Qt, which is suitable for embedded systems. Great.
It has good APIs for TCP sockets and *nix sockets, and good data streaming protocols. Unless you demand absolutely scorching performance, I would err on the side of secure, separate processes.
The upside of three processes is that the protected address space prevents unwanted communications between the processes.
The upside of three processes is that the protected address space hinders wanted communications between the processes.
Inter-process communication is slower and more difficult to manage than inter-thread comms.
I'm just starting to learn Python and have heard about the GIL and how it prevents "real" multithreading (by which I mean, allowing multiple threads to run on different cores simultaneously).
Now, hypothetically, were the GIL to be removed wouldn't each thread (now running on a different core) require a separate instance of the VM in order to execute? Does the JVM have the same issue?
If so, is there any benefit to using threads in programs that are interpreted/execute on a VM vs separate processes (aside from the performance gain in using POSIX threads vs Processes - although I believe in Linux the difference isn't that great)? Because having to have a separate instance of the VM for each thread seems like a lot of overhead.
Thanks.
No, there's no requirement for separate "instances" of the VM. Why would there be? The problem with the GIL is it's one data structure which needs to be shared across all threads, and can't be safely accessed by multiple threads without locking. Basically the solution is to try to avoid that sort of thing :)
Many VMs will have some per-thread or per-core data structures (e.g. some JVMs can allocate memory from a thread-local heap, making the allocation really quick) but that's not the same as a whole separate VM instance per thread or core.
The problem with the GIL is just one of granularity. The GIL is a global lock that protects all the data structures of the Python interpreter, a solution would be to use finer-grained locks to protect those same data structures.
The situation is similar to that of the Linux kernel BKL (Big Kernel Lock), which was introduced in 2.0 to enable SMP. The BKL has been gradually substituted by finer-grained locks, and has finally went away very recently (2.6.39?).
Is it simply because they only need a stack and storage for registers so they are cheap to create ?
Is the fact that threads can share common data, i.e they don't need to use interprocess communication a factor here ? Would this result in less need for protection ?
Or do threads take advantage of multiprocessors better than processes ?
Who says it is? On some operating systems there is little difference. Are you thinking of Windows where threads are much lighter weight than processes?
I suspect you would learn more by consulting this Stack Overflow question.
If we speak of heavy-weight threads (Windows Threads for example), a Process has Threads, and it has at least one thread (the main thread), so clearly it's heavier or at least not-lighter :-) (the sum is always >= the part)
There are many "tables" that a Process must have (the open file table, the table that show how the memory is mapped (LDT, Local Descriptor Table)...). If you create a process all these tables have to be initialized. If you create a thread they don't (because the thread uses the ones of its process). And then a new process has to load again all the DLL, check them for remapping...
From the windows perspective, a process can take longer to create if it is loading many DLLs and also moving them around in memory due to conflicts in the base address. Then see all of the other reasons listed in the link from David Heffernan's answer.
Process switch requires change in CS/DS registers. Change in the value of these registers require fetching a new descriptor from global descriptor table which is actually expensive process in terms of CPU time.
I've heard that under linux on multicore server it would be impossible to reach top performance when you have just 1 process but multiple threads because Linux have some limitations on the IO, so that 1 process with 8 threads on 8-core server might be slower than 8 processes.
Any comments? Are there other limitation which might slow the applications?
The applications is a network C++ application, serving 100s of clients, with some disk IO.
Update: I am concerned that there are some more IO-related issues other than the locking I implement myself... Aren't there any issues doing simultanious network/disk IO in several threads?
Drawbacks of Threads
Threads:
Serialize on memory operations. That is the kernel, and in turn the MMU must service operations such as mmap() that perform page allocations.
Share the same file descriptor table. There is locking involved making changes and performing lookups in this table, which stores stuff like file offsets, and other flags. Every system call made that uses this table such as open(), accept(), fcntl() must lock it to translate fd to internal file handle, and when make changes.
Share some scheduling attributes. Processes are constantly evaluated to determine the load they're putting on the system, and scheduled accordingly. Lots of threads implies a higher CPU load, which the scheduler typically dislikes, and it will increase the response time on events for that process (such as reading incoming data on a socket).
May share some writable memory. Any memory being written to by multiple threads (especially slow if it requires fancy locking), will generate all kinds of cache contention and convoying issues. For example heap operations such as malloc() and free() operate on a global data structure (that can to some degree be worked around). There are other global structures also.
Share credentials, this might be an issue for service-type processes.
Share signal handling, these will interrupt the entire process while they're handled.
Processes or Threads?
If you want to make debugging easier, use threads.
If you are on Windows, use threads. (Processes are extremely heavyweight in Windows).
If stability is a huge concern, try to use processes. (One SIGSEGV/PIPE is all it takes...).
If threads aren't available, use processes. (Not so common now, but it did happen).
If your threads share resources that can't be use from multiple processes, use threads. (Or provide an IPC mechanism to allow communicating with the "owner" thread of the resource).
If you use resources that are only available on a one-per-process basis (and you one per context), obviously use processes.
If your processing contexts share absolutely nothing (such as a socket server that spawns and forgets connections as it accept()s them), and CPU is a bottleneck, use processes and single-threaded runtimes (which are devoid of all kinds of intense locking such as on the heap and other places).
One of the biggest differences between threads and processes is this: Threads use software constructs to protect data structures, processes use hardware (which is significantly faster).
Links
pthreads(7)
About Processes and Threads (MSDN)
Threads vs. Processes
it really should make no difference but is probably about design.
A multi process app may have to do less locking but may use more memory. Sharing data between processes may be harder.
On the other hand multi process can be more robust. You can call exit() and quit the child safely mostly without affecting others.
It depends how dependent the clients are. I usually recommend the simplest solution.
I have two processes which access to the same physical memory(GPIO data addr).
So how can I have synchronized between these apps?
I understand that we have some kind of locking mechanism such as mutex and semaphore, so which method is the fastest?
Thank for your help,
-nm
Mutexes and semaphores are generally considered to be concurrency solutions in the same address space -- meaning that different parts of the same program will lock their access to a resource using one of these contraptions.
When you're dealing with separate processes, the standard way to do this on Linux is to create something in /var/lock, like /var/lock/myapp.lock, and place your PID followed by a newline in it. Then other processes will check for its existence, and if you're crafty check the PID to make sure it's still alive, too.
If you need real-time access to the area, skip the filesystem and the processes will have to communicate via IPC (LET_ME_KNOW_WHEN_DONE, OKAY_IM_DONE, you get the idea), or -- better -- write a process whose sole purpose is to read and write to the GPIO memory, and your other programs communicate with it via IPC (probably the best approach).
mutex means mutual exclusion -- a semaphore is just a variable used to determine if the resource is in use. In windows, there is a Mutex object that can be created to protect a shared resource.
The issue is what language are you using? What OS (I am assuming linux). Most languages provide support for multi-threading and mutual exclusion, and you should use the built-in constructs.
For example, using C on Linux, you might want to
include semaphore.h
and look up the calls for sem_init, sem_wait etc.