i have a problem.
my progrem creates a some number of processes in the system (Windows).
They all must write some data in ONE file on a disk.
so, i need to synchronize them... but don't know...
exactly: my program calls SetWindowsHook and injects one dll in many processes. and they all need to write some data to one file
The synchronisation object that works across processes is a mutex.
Windows have a lock foreach file, so if one process is writting in a file windows wont let another write. Mutex is what you want, protect the code where you are writting into the file with one.
Single System As David mentioned, you can use a mutex to accomplish this task. In Windows, this is done by using named mutexes and (if you want) named semaphores to do this.
The function CreateMutex can be used to both create the mutex (by the first process) and open it by the other processes. Supply the same lpName value in all processes. Use WaitForSingleObject to gain ownership of the mutex. And use ReleaseMutex to give up ownership.
An example of creating a named mutex can be found here.
If use a named semaphore, you can accomplish the same thing by giving the semaphore an initial count of 1. Use CreateSemaphore to create and open it, WaitForSingleObject to gain ownership (same as with a mutex) and ReleaseSemaphore to give up ownwership.
Multiple Systems The above approach assumes that the processes are all running on the same system. If the processes are running on different systems, then you may need to use the idea mentioned by DVD. You can lock portions of a file and use that as the synchronization method. For example, you could, by convention, lock 1 byte at some offset (it can even be past the end of the file) as a type of semaphore. Using this mechanism, though, may mean you need to implement some kind of efficient wait depending on the functions you use. If you use CreateFile and LockFileEx, you can have the function do a blocked wait by not specifying LOCKFILE_FAIL_IMMEDIATELY in the call.
The answer to your problem is to implement Thread synchronization.
If you are using C#, you can put a lock{} statement over your file writing code.
For other languages, you must use a Monitor or Mutex class to synchronize.
Use Stream.Synchronized()
See http://msdn.microsoft.com/en-us/library/system.io.stream.synchronized.aspx. This method only works for C# though
I recently had to do almost this exact thing (for logging to a single file from a dll injected into multiple processes).
Use _fsopen() to open the file in each process and then use a mutex for synchronization to ensure that only one process at a time is ever writing to the file.
Related
I have looked online and done some searching through stackoverflow and the internet about locks and I just seem to get a general understanding that when a lock is active another thread cannot use it??
I have multiple shared objects which are being read/written constantly throughout the script and I'm still not 100% sure how the locking function really works? When do you need to use it, when do you not need to use it and is it worth creating individual locks for each shared variable/object?
When a thread calls a lock does that mean other threads will only pause at that particular part of the script where the lock was originally called or does it somehow acknowledge to stop reading/writing any variables within the acquire/release function call throughout the entire script?
If I have multiple locks specifically for each shared variable/object and one lock function is called, does this effect the rest of the locks too?
I think to summerise, I'm struggling to understand the "in-depth" version of locking, only being able to find a general overview amongst previous explanations online.
I'm working on a multithreaded application where multiple threads may want exclusive access to the same file. I'm looking for a way of serializing these operations. I was planning to use flock, lockf, or fcntl locking. However it appears that with these methods an attempt to lock a file by a second thread when a first thread already owns the lock will be granted, because the two threads are in the same process. This is according to the manpages for flock and fnctl (and I guess in linux lockf is implemented with fnctl). Also supported by this other question. So, are there other ways of locking a file in linux which works at a thread-level instead of a process-level?
Some alternatives that I came up with which I do not like are:
1) Use a lockfile (xxx.lock) opened with O_CREAT | O_EXCL flags. This call will succeed only in one thread if there is contention. The problem with this is that then other threads have to spin on the call until they achieve the lock, meaning that I have to _yield() or sleep() which makes me think this is not a great option.
2) Keep a mutex'ed list of all open files. When a thread wants to open/close a file it has to lock the list first. When opening a file, it searches the list to see if it's open. This sounds particularly inefficient because it requires a significant amount of work even if the file is not owned yet.
Are there other ways of doing this?
Edit:
I just discovered this text in my system's manpages which isn't in the online man pages:
If a process uses open(2) (or similar) to obtain more than one descriptor for the same file, these descriptors are treated independently by flock(). An attempt to lock the file using one of these file descriptors may be denied by a lock that the calling process has already placed via another descriptor.
I'm not happy about the words "may be denied", I'd prefer "will be denied" but I guess it's time to test that.
Two Linux processes open and mmap the same /dev/shm/ shared memory file and use it as common memory. Question: what is the simplest and best way for one process to "wake up" the other process to notify that it should look in the memory?
For example, can one process cause a poll() event for the other process's file descriptor?
The solution doesn't need to be portable but I would like it to be simple.
That's why POSIX has condition variables.
Define a shared POSIX condition variable and its associated mutex in the shared memory region.
Then have one thread wait on the condition variable and the other signal the condition variable event when it wants the other thread to look in the memory.
There's a lot of material on the web on condition variables.
Here is one pretty good short one: https://computing.llnl.gov/tutorials/pthreads/#ConditionVariables
You may please consider using a semaphore (POSIX named semaphore) also to solve this.
One simple example, using shared Memory (In the example it is in System V, but you can use it with POSIX too) and POSIX semaphore is in the link ,
How can 2 processes talk to each other without pipe()?
I was wondering whether it would ever make sense to use a mutex or semaphore when there is only one thread?.
Thanks for your help.
I design thread protection into my components because they are reusable and scalable components intended to work in any environment I can realistically anticipate. Many times they are initially used in a single thread environment. Often times the scope of the implementation expands to include more threads. Then I don't have to chase down resources to protect from the new access scenarios.
Mutex can make sense, since Mutex can be used for system wide sharing, instead of internal process-wide sharing. For example, you can use a Mutex to prevent an application from being started twice.
This may be a bit out there but lets say you are writing a recursive function and you want each level to register with a separate resource. This way you can keep the responsibility of cleaning up the resource in one place (The resource pool).
Sounds like a trick question. Technically, yes. A named mutex can be used to synch multiple processes containing a single thread in each.
You can use system-wide semaphores (and even mutexes) to do inter-process communication.
You can signal from a single-threaded process to another single-threaded process by acquire()/release()-ing on a named semaphore, for example.
In case the environment supports system interrupts it adds non-linear behaviour.
Semaphore can be used in order to sleep in main thread until interrupt triggers.
If one of my processes open a file, let's say for reading only, does the O.S guarantee that no other process will write on it as I'm reading, maybe
leaving the reading process with first part of the old file version, and second part of the newer file version, making data integrity questionable?
I am not talking about pipes which have no seek, but on regular files, with seek option (at least when opened with only one process).
No, other processes can change the file contents as you are reading it. Try running "man fcntl" and ignore the section on "advisory" locks; those are "optional" locks that processes only have to pay attention to if they want. Instead, look for the (alas, non-POSIX) "mandatory" locks. Those are the ones that will protect you from other programs. Try a read lock.
No, if you open a file, other processes can write to it, unless you use a lock.
On Linux, you can add an advisory lock on a file with:
#include <sys/file.h>
...
flock(file_descriptor,LOCK_EX); // apply an advisory exclusive lock
Any process which can open the file for writing, may write to it. Writes can happen concurrently with your own writes, resulting in (potentially) indeterminate states.
It is your responsibility as an application writer to ensure that Bad Things don't happen. In my opinion mandatory locking is not a good idea.
A better idea is not to grant write access to processes which you don't want to write to the file.
If several processes open a file, they will have independent file pointers, so they can seek() and not affect one another.
If a file is opened by a threaded program (or a task which shares its file descriptors with another, more generally), the file pointer is also shared, so you need to use another method to access the file to avoid race conditions causing chaos - normally pread, pwrite, or the scatter/gather functions readv and writev.