I have a producer and a consumer. Producer fills its internal queue with objects, consumer takes these objects one by one. I want to synchronize the cosumer with the producer, so that the consumer blocks when there are no objects ready, and I want to synchronize the producer with itself, so that it stops producing when the queue is full (and starts again when there’s space). How do I do that? I was able to solve a simpler case without the queue using NSConditionalLock, but with the queue the problem looks more complex.
You might consider using a pair of NSOperationQueues or dispatch queues. Have your production operations (in the producer queue) send messages, on the main thread if necessary, to an object that adds consumption operations to the consumer queue.
I ended up using two semaphores, objectsReady and bufferFreeSlots:
#implementation Producer
- (id) getNextObject {
[objectsReady wait];
id anObject = [[buffer objectAtIndex:0] retain];
[buffer removeObjectAtIndex:0];
[bufferFreeSlots signal];
return [anObject autorelease];
}
- (void) decodeLoop {
while (1) {
[bufferFreeSlots wait];
[buffer push:[self produceAnObject]];
[objectsReady signal];
}
}
#end
The bufferFreeSlots is initialized to the maximum queue size. So far it seems to work, but God knows this is an act of faith, not a solid confidence.
Related
One way would be to lock and then check the status of first shared queue, push data if space available, or ignore if not, and then unlock.
Then check the status of second shared queue, push data if space available, or ignore if not, and then unlock.
So, on and so forth.
Here we'll be constantly locking and unlocking to see the status of a shared queue and then act accordingly.
Questions:
What are the drawbacks of this method? Of course time will be spent in locking and unlocking. Is that it?
What are the other ways to achieve the same effect without the current method's drawbacks?
Lock contention is very expensive because it requires a context switch - see the LMAX Disruptor for a more in-depth explanation, in particular the performance results page; the Disruptor is a lock-free bounded queue that exhibits less latency than a bounded queue that uses locks.
One way to reduce lock contention is to have your producers check the queues in a different order from each other, for example instead of each producer checking Queue1, then Queue2, ... and finally QueueN, each producer would repeatedly generate a random number between [1, N] and then check Queue[Rand(N)]. A more complex solution would be to maintain a set of queues sorted according to their available space (e.g. in Java this would be a ConcurrentSkipListSet), then have each producer remove the queue from the head of the set (i.e. the queue with the most available space that is not being simultaneously accessed by another producer), add an element, and insert the queue back into the set; a simpler solution along the same vein is to maintain an unbounded unsorted queue of queues and to have a producer remove and check the queue from the head of the queue of queues and to then insert the queue back into the tail of the queue of queues, which would ensure that only one producer is able to check a queue at any given point of time.
Another solution is to reduce and ideally eliminate the number of locks - it's difficult to write lock-free algorithms but it's also potentially very rewarding as demonstrated by the performance of LMAX's lock-free queue. In lieu of replacing your locked bounded queues with LMAX's lock-free bounded queues, another solution is to replace your locked bounded queues with lock-free unbounded queues (e.g. Java's ConcurrentLinkedQueue; lock-free unbounded queues are much more likely to be in your language's standard library than lock-free bounded queues) and to place conservative lock-free guards on these queues. For example, using Java's AtomicInteger for the guards:
public class BoundedQueue<T> {
private ConcurrentLinkedQueue<T> queue = new ConcurrentLinkedQueue<>();
private AtomicInteger bound = new AtomicInteger(0);
private final int maxSize;
public BoundedQueue(int maxSize) {
this.maxSize = maxSize;
}
public T poll() {
T retVal = queue.poll();
if(retVal != null) {
bound.decrementAndGet();
}
return retVal;
}
public boolean offer(T t) {
if(t == null) throw new NullPointerException();
int boundSize = bound.get();
for(int retryCount = 0; retryCount < 3 && boundSize < maxSize; retryCount++) {
if(bound.compareAndSet(boundSize, boundSize + 1)) {
return queue.offer(t);
}
boundSize = bound.get();
}
return false;
}
}
poll() will return the element from the head of the queue, decrementing bound if the head element isn't null i.e. if the queue isn't empty. offer(T t) attempts to increment the size of bound without exceeding maxSize, if this succeeds then it puts the element at the tail of the queue, else if this fails three times then the method returns false. This is a conservative guard because it is possible for offer to fail even if the queue isn't full, e.g. if an element is removed after boundSize = bound.get() sets boundSize to maxSize, or if the bound.compareAndSet(expected, newVal) method happens to fail three times due to multiple consumers calling poll().
Really, you are making too many locks/unlocks here. The solution is to make the same check twice:
check if space is available, if not, continue
lock
check if space is available AGAIN
... go on as you did before.
This way you will lock if you needn't to do it only in very rare cases.
I have first seen the solution in the book "Professional Java EE Design Patterns" (Yener, Theedom)
Edit.
About spreading the start queue numbers among threads.
Notice, that without any special organization these threads are waiting for queues only the first time. The next time the necessary timeshift will be already created by simply waiting. Of course, we can create the timeshift ourselves, spreading the start numbers among threads. And that simple spread by equal shift will be more effective that a random one.
Is there ever any reason to add blocks to a serial dispatch queue asynchronously as opposed to synchronously?
As I understand it a serial dispatch queue only starts executing the next task in the queue once the preceding task has completed executing. If this is the case, I can't see what you would you gain by submitting some blocks asynchronously - the act of submission may not block the thread (since it returns straight-away), but the task won't be executed until the last task finishes, so it seems to me that you don't really gain anything.
This question has been prompted by the following code - taken from a book chapter on design patterns. To prevent the underlying data array from being modified simultaneously by two separate threads, all modification tasks are added to a serial dispatch queue. But note that returnToPool adds tasks to this queue asynchronously, whereas getFromPool adds its tasks synchronously.
class Pool<T> {
private var data = [T]();
// Create a serial dispath queue
private let arrayQ = dispatch_queue_create("arrayQ", DISPATCH_QUEUE_SERIAL);
private let semaphore:dispatch_semaphore_t;
init(items:[T]) {
data.reserveCapacity(data.count);
for item in items {
data.append(item);
}
semaphore = dispatch_semaphore_create(items.count);
}
func getFromPool() -> T? {
var result:T?;
if (dispatch_semaphore_wait(semaphore, DISPATCH_TIME_FOREVER) == 0) {
dispatch_sync(arrayQ, {() in
result = self.data.removeAtIndex(0);
})
}
return result;
}
func returnToPool(item:T) {
dispatch_async(arrayQ, {() in
self.data.append(item);
dispatch_semaphore_signal(self.semaphore);
});
}
}
Because there's no need to make the caller of returnToPool() block. It could perhaps continue on doing other useful work.
The thread which called returnToPool() is presumably not just working with this pool. It presumably has other stuff it could be doing. That stuff could be done simultaneously with the work in the asynchronously-submitted task.
Typical modern computers have multiple CPU cores, so a design like this improves the chances that CPU cores are utilized efficiently and useful work is completed sooner. The question isn't whether tasks submitted to the serial queue operate simultaneously — they can't because of the nature of serial queues — it's whether other work can be done simultaneously.
Yes, there are reasons why you'd add tasks to serial queue asynchronously. It's actually extremely common.
The most common example would be when you're doing something in the background and want to update the UI. You'll often dispatch that UI update asynchronously back to the main queue (which is a serial queue). That way the background thread doesn't have to wait for the main thread to perform its UI update, but rather it can carry on processing in the background.
Another common example is as you've demonstrated, when using a GCD queue to synchronize interaction with some object. If you're dealing with immutable objects, you can dispatch these updates asynchronously to this synchronization queue (i.e. why have the current thread wait, but rather instead let it carry on). You'll do reads synchronously (because you're obviously going to wait until you get the synchronized value back), but writes can be done asynchronously.
(You actually see this latter example frequently implemented with the "reader-writer" pattern and a custom concurrent queue, where reads are performed synchronously on concurrent queue with dispatch_sync, but writes are performed asynchronously with barrier with dispatch_barrier_async. But the idea is equally applicable to serial queues, too.)
The choice of synchronous v asynchronous dispatch has nothing to do with whether the destination queue is serial or concurrent. It's simply a question of whether you have to block the current queue until that other one finishes its task or not.
Regarding your code sample code, that is correct. The getFromPool should dispatch synchronously (because you have to wait for the synchronization queue to actually return the value), but returnToPool can safely dispatch asynchronously. Obviously, I'm wary of seeing code waiting for semaphores if that might be called from the main thread (so make sure you don't call getFromPool from the main thread!), but with that one caveat, this code should achieve the desired purpose, offering reasonably efficient synchronization of this pool object, but with a getFromPool that will block if the pool is empty until something is added to the pool.
I have a class that wraps a data model and is accessed/modified by multiple threads. I need to make sure modification to data model is synchronized. I am using a dispatch_queue_create(..., DISPATCH_QUEUE_SERIAL). This is working really well for my needs.
Most of the methods on my class internally call "dispatch_async(queue, ^{...});". There are a few places where I need to return a snapshot result. This is a simplified example of how that looks:
- (NSArray*) getSomeData {
__block NSArray* result = nil;
dispatch_sync(queue, ^{
... Do Stuff ...
result = blah.blah;
}
return result;
}
Now, lets assume that 5 "async tasks" are queued and one is currently executing. Now a "sync" task is scheduled. When will the "sync task" execute?
What I would like to have happen is "sync task" is executed ahead of any pending "async tasks". Is this what happens by default? If not is there a way to priority queue the "sync task"?
BTW,
I know I can set an overall queue priority but that is not what this question is about. For me queue priority normal is just fine. I just want my synchronous tasks to happen before any pending asynchronous tasks.
There's not a generic setting for "perform sync tasks first" or for setting relative priority between enqueued blocks in a single queue. To recap what may be obvious, a serial queue is going to work like a queue: first in, first out. That said, it's pretty easy to conceive of how you might achieve this effect using multiple queues and targeting. For example:
realQueue = dispatch_queue_create(NULL, DISPATCH_QUEUE_SERIAL);
asyncOpsQueue = dispatch_queue_create(NULL, DISPATCH_QUEUE_SERIAL);
dispatch_set_target_queue(asyncOpsQueue, realQueue);
for (NSUInteger i = 0; i < 10; i++)
{
dispatch_async(asyncOpsQueue, ^{
NSLog(#"Doing async work block %#", #(i));
sleep(1);
});
}
// Then whenever you have high priority sync work to do, stop the async
// queue, do your work, and then restart it.
dispatch_suspend(asyncOpsQueue);
dispatch_sync(realQueue, ^{
NSLog(#"Doing sync work block");
});
dispatch_resume(asyncOpsQueue);
One thing to know is that an executing block effectively can't be canceled/suspended/terminated (from the outside) once it's begun. So any async enqueued block that's in flight has to run to completion before your sync block can start, but this arrangement of targeting allows you to pause the flow of async blocks and inject your sync block. Note that it also doesn't matter that you're doing a sync block. It could, just as easily, be an async block of high priority, but in that case you would probably want to move the dispatch_resume into the block itself.
In most of my interviews, I've been asked about web services and multithreading. I've done neither, so I decided to learn more about Web Services and Multithreading using Grand Central Dispatch.
For web services, the way that I understand it is that you need to fetch the data using a class such as NSURLConnection. basically setup a new NSURL, then a connection, then a request. You also need to make use of the API's methods such as didConnect, didReceiveData, and didFailLoadWithError. After you receive the data, which is generally in JSON or XML format and stored as an NSData object, you can store it and parse through it. There are multiple ways to parse through it, such as by using SBJSON or NSXMLParser. You can then do with it what you need.
For multithreading, Grand Central Dispatch is a c-style way of multithreading. Basically, you use it when you need to do heavy hauling away from the main thread to avoid the app freezing. You can dispatch synchronously or asynchronously. Asynchronously means that the method on the main thread will continue executing, synchronously means that it will not. You never need to use GCD alongside with NSURLConnection, because NSURLConnection already does its work in the background then calls upon delegates in the main thread. But, for saving and unzipping files, you should use GCD. When you call dispatch_async, you pass in a dispatch queue. You can use either a serial queue or a concurrent queue. A serial queue will execute tasks in the queue one at a time, in the order that they arrived. It is the default setting. With concurrently queues, tasks executed concurrently might be executed at the same time.
My first question is, do I have a proper understanding of these two concepts? I know that there is a lot to learn about GCD, but I just want to make sure that I have the basic ideas correct. Also, with GCD, why would someone ever want to dispatch synchronously, wouldn't that defeat the purpose of multithreading?
The only reason to dispatch synchronously is to prevent the current code from continuing until the critical section finishes.
For example, if you wanted to get some value from the shared resource and use it right away, you would need to dispatch synchronously. If the current code does not need to wait for the critical section to complete, or if it can simply submit additional follow-up tasks to the same serial queue, submitting asynchronously is generally preferred.
You can make synchronous request and dispatch it by using dispatch_async or dispatch_sync call. It will totally run in background.
-(void)requestSomething:(NSString *)url
{
NSString *queue_id = #"queue_identifier";
dispatch_queue_t queue = dispatch_queue_create([queue_id UTF8String], 0);
dispatch_queue_t main = dispatch_get_main_queue();
dispatch_async(queue, ^{
NSURLRequest *theRequest = [NSURLRequest requestWithURL:[NSURL URLWithString:url]];
NSError *serviceError = nil;
NSURLResponse *serviceResponse = nil;
NSData *dataResponse = [NSURLConnection sendSynchronousRequest:theRequest returningResponse:&serviceResponse error:&serviceError];
if(serviceError)
{
dispatch_sync(main, ^{
// Do UI work like removing indicator or show user an alert with description of error using serviceError object.
return;
});
}
else
{
// Use dataResponse object and parse it as this part of code will not executed on main thread.
dispatch_sync(main, ^{
// Do UI work like updating table-view or labels using parsed data or removing indicator
});
}
});
// If your project is not developed under ARC mechanism, add following line
dispatch_release(queue);
}
Once upon a time, I remembered this stuff by heart. Over time, my understanding has diluted and I mean to refresh it.
As I recall, any so called single threaded application has two threads:
a) the primary thread that has a pointer to the main or DllMain entry points; and
b) For applications that have some UI, a UI thread, a.k.a the secondary thread, on which the WndProc runs, i.e. the thread that executes the WndProc that recieves messages that Windows posts to it. In short, the thread that executes the Windows message loop.
For UI apps, the primary thread is in a blocking state waiting for messages from Windows. When it recieves them, it queues them up and dispatches them to the message loop (WndProc) and the UI thread gets kick started.
As per my understanding, the primary thread, which is in a blocking state, is this:
C++
while(getmessage(/* args &msg, etc. */))
{
translatemessage(&msg, 0, 0);
dispatchmessage(&msg, 0, 0);
}
C# or VB.NET WinForms apps:
Application.Run( new System.Windows.Forms() );
Is this what they call the Dispatcher?
My questions are:
a) Is my above understanding correct?
b) What in the name of hell is the Dispatcher?
c) Point me to a resource where I can get a better understanding of threads from a Windows/Win32 perspective and then tie it up with high level languages like C#. Petzold is sparing in his discussion on the subject in his epic work.
Although I believe I have it somewhat right, a confirmation will be relieving.
It depends on what you consider the primary thread. Most UI frameworks will have an event handler thread that sits mostly idle, waiting for low level events. When an event occurs this thread gets a lock on the event queue, and adds the events there. This is hardly what I'd consider the primary thread, though.
In general a dispatcher takes some events and, based on their content or type sends (dispatches, if you will) them to another chunk of code (often in another thread, but not always). In this sense the event handler thread itself is a simple dispatcher. On the other end of the queue, the framework typically provides another dispatcher that will take events off of the queue. For instance, sending mouse events to mouse listeners, keyboard events to keyboard listeners etc.
Edit:
A simple dispatcher may look like this:
class Event{
public:
EventType type; //Probably an enum
String data; //Event data
};
class Dispatcher{
public:
...
dispatch(Event event)
{
switch(event.type)
{
case FooEvent:
foo(event.data);
break;
...
}
};
Most people I've met use "dispatcher" to describe something that's more than just a simple passthrough. In this case, it performs different actions based on a type variable which is consistent with most of the dispatchers I've seen. Often the switch is replaced with polymorphism, but switch makes it clearer what's going on for an example.