I am trying to find a way to do 1-to-many communication between processes in Linux. I found that linux has named pipes, but those won't work with multiple readers (I found on other SO answers that once a reader reads some data, others don't get it); could someones clarify this - I see mixed answers on SO - whether its possible or not.
Also, from my understanding, with sockets, multiple clients can connect to a server, but in my case, I need a process to send out data to multiple processes (kind-of reverse) - like a broadcast.
Could anyone suggest what options are available ?
Edit : Can shared memory be used some how ?
Named pipes indeed are just like a pipe, i. e. they have one input and one output. So you cannot use a single (named) pipe to send information to more than one other process.
But there is nothing against the idea of using one pipe for each other process and filling all pipes with the same information. This will work when using named pipes, (unnamed) pipes, and sockets. But it will in all cases mean that the sender keeps track of all the receivers and sends the information to each of them separately.
If you want to keep the sender uninformed about the list of receivers, then the only way I can think of right now is to use a plain file. Your sender can write it, and the receivers all can read it.
You can use shared memory with read-write semaphores to exchange data from one process to many process. The read-write semaphores can be used to synchronize the data between the processes.
The following is the pseudo code to synchronize communication between one writer thread and 20 reader threads. Write is blocked till all the reader threads read the shared memory.
if it is a writer thread
{
Take read-write mutex lock
increment writers++;
Release read-write mutex lock
Sem wait for the node-is-empty
Access the shared memory and fill the empty node
Take read-write mutext lock
Decrement writers--;
Release read-write mutex lock
Sem post for the node-is-filled
}
else /* want to read */
{
Take read-write mutex lock
if (writers > 0 || readers == 20) //reader default value is 20 reader threads
{
Release read-write mutex lock
sem wait for the node-is-filled
Take read-write mutex lock
}
Release read write mutex lock
printf("Shared Memory: %d\n", memory);
Take read-write mutex lock
readers--;
if (readers == 0) //wait for all the threads to read the buffer
sem post for the node-is-empty
Release read-write mutex lock
}
Related
According to this link, How do I recover a semaphore when the process that decremented it to zero crashes? it seems that there is no robust semaphore inter-process, and the author finally chose filelock, which is guaranteed to be released properly by the system-level or kernel-level control.
But I also found robust mutex provided by pthread https://man7.org/linux/man-pages/man3/pthread_mutexattr_setrobust.3.html why there is no something like robust semaphore?
And an extra question: what robust alternatives we have regarding IPC synchronization? filelock seems to be the best one. I think providing such mechanism is not that difficult from system or kernel level,since they do implement fielock. then why they don't provide some other approaches?
When you use a mutex, it can be acquired by at most one thread at a time. Therefore, once the mutex has been acquired, the owner can write its process ID or thread ID (depending on the system) into the mutex, and future users can detect whether the owner is still alive or not.
However, a semaphore is ultimately a counter. It is possible that different threads may increment or decrement the counter. There isn't intrinsically one resource that is being shared; there could instead be multiple resources.
For example, if we're trying to limit ourselves to a certain number of outgoing connections (say, 8), then we could create a semaphore with that value and allow threads to acquire it (wait) to make a connection, and then increment it (post) when they're done. If we never want to make more than 8 connections at once, the semaphore will never block; we'll have acquired it successfully each time, even though there's no exclusion.
In such a situation, there isn't going to be space inside the semaphore to store every process's thread ID. Using memory allocation is tricky because that code needs to be synchronized independently, and even if that could be solved, it means that a semaphore value would have at least O(N) performance when acquiring the semaphore. I work on a production system that uses hundreds of threads, so you can imagine the performance problems if we had such a semaphore design.
There are other solutions which you can use when you need robustness, such as file locking or a robust mutex in a shared memory segment, but none of them have the same properties as a semaphore. Therefore, any discussion of what primitives should be used instead depends on the particular needs of the situation (which should probably be placed in a new question).
I read one of the differences between semaphore and mutex is in case of mutex the process/thread (which ever is having the lock) can only release the lock. But in the case of the semaphore any other process can release the semaphore. My doubt arises when a process that does not have the semaphore with it can release the semaphore. What is the use of having a semaphore?
Let's say I have two processes A and B. Assume process A is having a semaphore with it and executing some critical task. Now let us say process B sends a signal to release the semaphore. In this scenario, will process A release the semaphore even if it is executing some critical task?
You are making half-sense. It is not about ownership. Partner-release in semaphores (and mutexes) is usable, for instance, in my favorite interview question of thread ping-pong. As a matter of fact, I have specifically tried to partner-release a mutex on 3 implementations available to me at a time (Linux/Solaris/AIX) and partner-release did work for mutexes as expected - i.e. mutex was successsfully released and threads blocking on it resumed execution. However, this is, of course, prohibited by Posix.
I think you might be confused on the whole set of differences between a semaphore and a mutex. A mutex provides mutual exclusion. A semaphore counts until it reaches a level where it starts excluding. A semaphore that counted to one would give similar semantics to a mutex though.
A good example would be a television set. Only so many people can watch the same television set, so protecting it with a semaphore would make sense. Anyone can stop watching the television. The remote control for the television can only be operated by one person at a time though, so you could protect it with a mutex.
Some reading...
https://en.wikipedia.org/wiki/Mutual_exclusion
https://en.wikipedia.org/wiki/Semaphore_%28programming%29
"Let's say I have two processes A and B. Assume process A is having a semaphore with it and executing some critical task. Now let us say process B sends a signal to release the semaphore. In this scenario, will process A release the semaphore even if it is executing some critical task?"
One key point to note here is the role of OS kernel. Process B can't send a signal to Process A 'to release the semaphore'. What it can do is request the kernel to give it access to the resource. Process A had requested the kernel and the kernel granted it access to the resource.
Now process A, after it finishes its job, will let the kernel know that it is done with the resource and then kernel grants access to B.
"My doubt arises when a process that does not have the semaphore with it can release the semaphore. What is the use of having a semaphore?"
The key difference between a mutex and a semaphore is, a semaphore serializes access to multiple instances of a resource. Mutex does the same when there is one instance of the resource.
A count is maintained by kernel in case of semaphore and mutex is a special case where the count is 1.
Consider the processes as customers waiting in line at a bank.
The use of semaphore is analogous to the case where there are multiple tellers serving the customers. Usage of mutex is analogous to the case where there is just one teller.
Say there are processes A, B and C that need concurrent access to a resource (lock, file or a data structure in memory, etc.). Further suppose there are 2 instances of the resource. So at most two processes can be granted access at a time.
Process A requests access to an instance of the resource following the required semantics. This request to the kernel involves data structures to identify the resource and maximum number of instances as 2. kernel creates the semaphore with a count of 2, grants A access to the resource and decrements the count to 1, because now only one other process can get access.
Now process B requests access to the resource by following the same semantics. Kernel grants it access and decrements the count to 0.
Now process C requests access, but kernel keeps it in waiting state, because count is 0 and no more than 2 processes can get concurrent access.
Process A is done with the resource and lets kernel know. Kernel notices this and grants access to process C that has been waiting.
In case of mutex, kernel grants access to the resource only one process at a time.
A normal binary semaphore is basically used for synchronization. However, the mutex is for exclusive access to a resource. A mutex is a special variant of semaphore that allows only one locker at a time and with more stringency on ownership than a normal semaphore such as the mutex should be released only by the thread that acquired it. Also, please note that in case of pthreads, fast mutex may not check for this error related to ownership, whereas the error checking mutex shall return error.
For the query related to 2 process A and B, the Process A shall intimate via kernel that it is done with its critical work so that the resource can be made available for waiting processes like B.
You could find some related information in this link too :
When should we use mutex and when should we use semaphore
There is no such thing as "having" a semaphore. Semaphores don't have ownership like mutexes do. The code you describe would simply be buggy. Mutexes won't work if your code is buggy either.
Consider the most classic example of a semaphore -- allowing one train at a time on a section of track. You could implement this with a mutex if the train is a thread. The train would lock the track mutex before going on the track and unlock it after leaving the track.
But what if the train itself is multi-threaded? Which thread should own the track?
And what if the signalling devices are the threads, not the train? Here, the signalling device that detects the train entering the track has to lock the track while the signalling device that detects the train leaving the track has to unlock it.
Mutexes are suitable for cases where there is something that is owned by a particular thread for a short period of time. That thread can "own" the mutex. Semaphores are useful for cases where there is no thread to own anything or nothing for the thread to own.
I'm writing a basic UNIX program that involves processes sending messages to each other. My idea to synchronize the processes is to simply have an array of flags to indicate whether or not a process has reached a certain point in the code.
For example, I want all the processes to wait until they've all been created. I also want them to wait until they've all finished sending messages to each other before they begin reading their pipes.
I'm aware that a process performs a copy-on-write operation when it writes to a previously defined variable.
What I'm wondering is, if I make an array of flags, will the pointer to that array be copied, or will the entire array be copied (thus making my idea useless).
I'd also like any tips on inter-process communication and process synchronization.
EDIT: The processes are writing to each other process' pipe. Each process will send the following information:
typedef struct MessageCDT{
pid_t destination;
pid_t source;
int num;
} Message;
So, just the source of the message and some random number. Then each process will print out the message to stdout: Something along the lines of "process 20 received 5724244 from process 3".
Unix processes have independent address spaces. This means that the memory in one is totally separate from the memory in another. When you call fork(), you get a new copy of the process. Immediately on return from fork(), the only thing different between the two processes is fork()'s return value. All of the data in the two processes are the same, but they are copies. Updating memory in one cannot be known by the other, unless you take steps to share the memory.
There are many choices for interprocess communication (IPC) in Unix, including shared memory, semaphores, pipes (named and unnamed), sockets, message queues and signals. If you Google these things you will find lots to read.
In your particular case, trying to make several processes wait until they all reach a certain point, I might use a semaphore or shared memory, depending on whether there is some master process that started them all or not.
If there is a master process that launches the others, then the master could setup the semaphore with a count equal to the number of processes to synchronize and then launch them. Each child could then decrement the semaphore value and wait for the semaphore value to reach zero.
If there is no master process, then I might create a shared memory segment that contains a count of processes and a flag for each process. But when you have two or more processes using shared memory, then you also need some kind of locking mechanism (probably a semaphore again) to ensure that two processes do not try to update the shared memory simultaneously.
Keep in mind that reading a pipe that nobody is writing to will block the reader until data appears. I don't know what your processes do, but perhaps that is synchronization enough? One other thing to consider if you have multiple processes writing to a given pipe, their data may become interleaved if the writes are larger than PIPE_BUF. The value and location of this macro are system dependent.
-Kevin
The entire array of flags will seem to be copied. It will not actually be copied until one process or another writes to it of course. But that's an implementation detail and transparent to the individual processes. As far as each process is concerned, they each get a copy of the array.
There are ways to make this not happen. You can use mmap with the MAP_SHARED option for the memory used for your flags. Then each sub-process will share the same region of memory. There's also Posix shared memory (which I, BTW, think is an awful hack). To find out about Posix shared memory, look at the shm_overview(7) man page.
But using memory in this way isn't really a good idea. On multi-core systems it's not always the case that when one process (or thread) writes to an area of shared memory that all other processes will see the value written right away. Frequently the value will hang out for awhile in the L2 cache and not be immediately flushed.
If you want to communicate using shared memory, you will have to used mutexes or the C++11 atomic operations to ensure that writes are properly seen by the other processes.
I am going to implement a program where one parent process reads a text file and feeds the data he's reading into a shared memory buffer that's going to be read by some children processes. All this work will be mediated by semaphores. Let's assume the parent is going to read one character at a time from the file and the shared memory buffer contains 5 slots.
At first, I thought of only having 2 semaphores:
writeSemaphore, initialized to 5, is the semaphore that tells whether the writer is allowed to write to the buffer. when it finally goes down to 0, the parent process will be blocked until one of the children unlocks it (after having read some block).
readSemaphore, initialized to 0, is the semaphore that tells whether any of the readers is allowed to read from the buffer.
But now that I think of it, this wouldn't prevent me from having 2 consumers accessing the the shared memory at the same time. I must prevent it. So I introduced a third semaphore:
allowedToRead that is either 1 or 0, that allows or blocks access to the children processes.
Here is pseudo code for both children and parent:
Child:
while (something) {
wait(readSemaphore)
wait(allowedToRead)
<<read from shared memory>>
post(allowedToRead)
post(writeSemaphore)
}
Parent:
while (something) {
wait(writeSemaphore)
<<writes to shared memory>>
post(allowedToRead)
}
Is my reasoning correct?
Thanks
Khachik is half right. He's may be all right, but his description isn't as clear as it could be.
Firstly, where you have the parent posting allowedToRead you probably mean for it to post readSemaphore.
Secondly your code allows the parent to write at the same time as a child is reading. You say you have 5 slots. If the parent writes to a different slot than the child is reading then this is ok I suppose, but how does the child determine where to read? Is it using the same variables as the parent is using to determine where to write? You probably need some extra protection there. After all I assume the different children are all reading different slots, so if you need to prevent them treading one ach other's toes you'll need to do the same for the parent too.
Thirdly, I'd have used a mutex instead of a semaphore for allowedToRead.
Fourthly, what determines which child reads which data or is it meant to be first come first served like pigs at a slop bucket?
If the shared memory has 5 independant slots, then I'd be inclined to add a "next read" and "next write" variable. Protect those two variables with a mutex in both producer and consumers, and then use the semaphores just to block/trigger reading and writing as you are already doing. If it weren't a school exercise, you could do better using a single condition variable attached to the mutex I mentioned. When it gets signalled the parent checks if he can write and the children check if they can read. When either a read or a write occurs, signal the condition variable globally to wake everybody up to check their conditions. This has the advantage that if you have independant buffer slots then you can safely and happily have multiple consumers consuming at the same time.
No.
the writer should release readSemaphore when it write one unit of information;
the writer should acquire allowedToRead lock (0,1 semaphore is a lock/mutex) before writing to shared memory to prevent race conditions.
To simplify: consider two functions read_shared_memory, write_shared_memory, which are to read and write from/to the shared memory respectively and both acquiring/releasing the same lock before reading/writing.
The producer acquires write semaphore, calls the write function, releases the read semaphore.
The consumer acquire read semaphore, calls the read function, releases the the write semaphore.
Sure this can be implemented without read/write functions, they are just to simplify using atomic access to the shared memory. A critical section can be implemented inside produce/consume loops without additional functions.
Wikipedia describes it in more scientific way :)
I am using an application where a lower level application always invokes a callback RecData(char *buf) when it receives data.
In the callback I am creating two threads and pass the consumer and producer function to these created threads respectively.
My code:
void RecData (char * buf)
{
CreateThread(NULL,0,producer_queue,(void *)buf,0,NULL);
CreateThread(NULL,0,consumer_queue,NULL,0,NULL);
}
The above works when I receive one data at a time. If I receive say 5 data almost at the same time then producer_queue should first put all the data in queue and then consumer_queue should start retrieving the data but here as soon as producer_queue puts the first data in queue, consumer_queue retrieves it.
What you want to do, I believe, is control access to the queue. You'll want to look at using a mutex to control reading from the queue.
When you recieve data, you will lock the mutex, then enqueue data. When you are done queing the data, then release the lock.
When reading from the queue, you will see if the mutex is locked. If you are writing data to the queue, you won't be able to start reading, until your producer thread has completed writing all of it's data and release the lock. If you actually lock the mutex, then you prevent your writer thread from writing while you are reading data.
This approach could introduce potential deadlocks. If your writer thread dies prior to releasing the lock, then your reader thread will not be able to continue (then again your thread dying may just trigger an error state).
I hope this makes sense.
Use the concept of condition variables. The probelm you have is the most common one in multi-threaded programming world. Just using mutexes doesn't help the situation. Always remember that mutexes are for locking & condition variables are for waiting. The later is always safer and almost certain when a thread should start consuming from a shared queue.
Check out the below link on how you can create a condition variable on your own on windows:
http://www.cs.wustl.edu/~schmidt/win32-cv-1.html
If you are using windows vista, the below msdn example may help you:
http://msdn.microsoft.com/en-us/library/ms686903(VS.85).aspx
In all cases use the logic as shown in Schmidt's website as it looks more portable (oh yes portable on different versions of windows atleast). Schmidt's implemention gives you the standard POSIX api feel which is the widely used standard on most modern UNIX/LINUX systems.