In our project we are using Ignite semaphore to lock access to resources.
Resources are uniquely identified by UUID and we are using 1 Ignite semaphore per resource.
We don't close() the semaphores because it is not obvious to know when a resource is not used. I tried to use getQueueLength() and hasQueueThreads() to determine if I could close() but it doesn't work well.
Obviously the number of these semaphores will keep growing.
Does ignite remove automatically "old" semaphores (which haven't been updated for a given period of time)?
If not, what would be the better option here to cleanup these "leaking" semaphores
Thanks !
Yes, you need to close() semaphore explicitly to remove it from cache.
Then you can use removed() to see if semaphore was already closed and removed.
Related
According to this link, How do I recover a semaphore when the process that decremented it to zero crashes? it seems that there is no robust semaphore inter-process, and the author finally chose filelock, which is guaranteed to be released properly by the system-level or kernel-level control.
But I also found robust mutex provided by pthread https://man7.org/linux/man-pages/man3/pthread_mutexattr_setrobust.3.html why there is no something like robust semaphore?
And an extra question: what robust alternatives we have regarding IPC synchronization? filelock seems to be the best one. I think providing such mechanism is not that difficult from system or kernel level,since they do implement fielock. then why they don't provide some other approaches?
When you use a mutex, it can be acquired by at most one thread at a time. Therefore, once the mutex has been acquired, the owner can write its process ID or thread ID (depending on the system) into the mutex, and future users can detect whether the owner is still alive or not.
However, a semaphore is ultimately a counter. It is possible that different threads may increment or decrement the counter. There isn't intrinsically one resource that is being shared; there could instead be multiple resources.
For example, if we're trying to limit ourselves to a certain number of outgoing connections (say, 8), then we could create a semaphore with that value and allow threads to acquire it (wait) to make a connection, and then increment it (post) when they're done. If we never want to make more than 8 connections at once, the semaphore will never block; we'll have acquired it successfully each time, even though there's no exclusion.
In such a situation, there isn't going to be space inside the semaphore to store every process's thread ID. Using memory allocation is tricky because that code needs to be synchronized independently, and even if that could be solved, it means that a semaphore value would have at least O(N) performance when acquiring the semaphore. I work on a production system that uses hundreds of threads, so you can imagine the performance problems if we had such a semaphore design.
There are other solutions which you can use when you need robustness, such as file locking or a robust mutex in a shared memory segment, but none of them have the same properties as a semaphore. Therefore, any discussion of what primitives should be used instead depends on the particular needs of the situation (which should probably be placed in a new question).
I have a program that requires several processes access a shared resource. This shared resource does not exist when they all start, so one of them needs to create it. Once the shared resource is created, important infrastructure is installed for use later. There is, however, a possibility that if the "creator" process is scheduled out before it can install the infrastructure in the shared resource, that other processes will try to use the uninitialized data (leading to undefined behavior).
In order to control this, I've created a named semaphore (sem_t *sem_init). Any process that is not the creator "downs" or "waits" on this zero-initialized semaphore. When the creator process has finished setup, it "up's" or "posts" the semaphore, releasing the processes. However, there remains one problem. I do not know exactly how many processes are waiting on it.
In order to solve this problem, I have the following options:
I create a counting semaphore. Each process "up's" or "posts" on this semaphore before blocking on the initialization semaphore. This way, I can know how many processes to release.
I just "post" on the initialization semaphore until it is the maximum allowed value.
I don't like these "solutions" though. For one, I am limited by the maximum size of a semaphore when it comes to the number of processes that I can count. It also seems like "posting" so many times would incur a nasty overhead. My question then, is whether there is any way in which I can instruct a semaphore to release all blocked processes, without me having to do any explicit bookkeeping on my end. I'd also not like to be constrained by the maximum value of a semaphore.
Something like: sem_releaseAll (sem_t *sem_p); would be ideal.
Note: I would greatly prefer a Linux-native solution.
I was wondering whether it would ever make sense to use a mutex or semaphore when there is only one thread?.
Thanks for your help.
I design thread protection into my components because they are reusable and scalable components intended to work in any environment I can realistically anticipate. Many times they are initially used in a single thread environment. Often times the scope of the implementation expands to include more threads. Then I don't have to chase down resources to protect from the new access scenarios.
Mutex can make sense, since Mutex can be used for system wide sharing, instead of internal process-wide sharing. For example, you can use a Mutex to prevent an application from being started twice.
This may be a bit out there but lets say you are writing a recursive function and you want each level to register with a separate resource. This way you can keep the responsibility of cleaning up the resource in one place (The resource pool).
Sounds like a trick question. Technically, yes. A named mutex can be used to synch multiple processes containing a single thread in each.
You can use system-wide semaphores (and even mutexes) to do inter-process communication.
You can signal from a single-threaded process to another single-threaded process by acquire()/release()-ing on a named semaphore, for example.
In case the environment supports system interrupts it adds non-linear behaviour.
Semaphore can be used in order to sleep in main thread until interrupt triggers.
This may sound like a stupid question, but if one locks a resource in a multi-threaded app, then the operation that happens on the resource, is that done atomically?
I.E.: can the processor be interrupted or can a context switch occur while that resource has a lock on it? If it does, then nothing else can access this resource until it's scheduled back in to finish off it's process. Sounds like an expensive operation.
The processor can very definitely still switch to another thread, yes. Indeed, in most modern computers there can be multiple threads running simultaneously anyway. The locking just makes sure that no other thread can acquire the same lock, so you can make sure that an operation on that resource is atomic in terms of that resource. Code using other resources can operate completely independently.
You should usually lock for short operations wherever possible. You can also choose the granularity of locks... for example, if you have two independent variables in a shared object, you could use two separate locks to protect access to those variables. That will potentially provide better concurrency - but at the same time, more locks means more complexity and more potential for deadlock. There's always a balancing act when it comes to concurrency.
You're exactly right. That's one reason why it's so important to lock for short period of time. However, this isn't as bad as it sounds because no other thread that's waiting on the lock will get scheduled until the thread holding the lock releases it.
Yes, a context switch can definitely occur.
This is exactly why when accessing a shared resource it is important to lock it from another thread as well. When thread A has the lock, thread B cannot access the code locked.
For example if two threads run the following code:
1. lock(l);
2. -- change shared resource S here --
3. unlock(l);
A context switch can occur after step 1, but the other thread cannot hold the lock at that time, and therefore, cannot change the shared resource. If access to the shared resource on one of the threads is done without a lock - bad things can happen!
Regarding the wastefulness, yes, it is a wasteful method. This is why there are methods that try to avoid locks altogether. These methods are called lock-free, and some of them are based on strong locking services such as CAS (Compare-And-Swap) or others.
No, it's not really expensive. There are typically only two possibilities:
1) The system has other things it can do: In this case, the system is still doing useful work with all available cores.
2) The system doesn't have anything else to do: In this case, the thread that holds the lock will be scheduled. A sane system won't leave a core unused while there's a ready-to-run thread that's not scheduled.
So, how can it be expensive? If there's nothing else for the system to do that doesn't require acquiring that lock (or not enough other things to occupy all cores) and the thread holding the lock is not ready-to-run. So that's the case you have to avoid, and the context switch or pre-empt issue doesn't matter (since the thread would be ready-to-run).
I've got a queue resource that is shared across multiple producers and multiple consumers. All are independent processes; no one process "owns" the queue.
By nature of the implementation access to the queue must be controlled and only one process must be allowed to push or pop at any given moment.
I figured using a POSIX named semaphore would be the right solution, however a few of the details are bothering me. (This is a Linux-only implementation, btw.)
When (if ever) should I do a sem_unlink? Is there any reason to actually remove the queue?
I'm concerned about a process dying while holding the queue semaphore locked. Is there any good way around this? I can do a timed wait when trying to get the lock, but if the timeout expires I've now got a race condition.
Is there a better solution for a simple binary lock like this? Perhaps a lockfile using fcntl and/or exclusive opens?
File locks have the benefit of unlocking in the event of of unexpected process death. I think that they best suit your scenario.
I can imagine using semaphores when I need the more complex semantics they support (they do more than support the mutex usage you have in mind) but if I do use them I need some way to perform housekeeping in the event of untimely death. I observe that Lotus Notes on Windows has a "ZapNotes" houskeeper that tidies in what I assume are similar "shouldn't happen" scenarios.