Given an instance of the producer-consumer problem, where several producers send messages to a single consumer: What techniques do you recommend to avoid starvation of producers, when some of the messages arrive "at the same time" to the consumer. Until now I am considering:
Choosing "non-deterministically" by sampling some probability distribution (not sure how, considering that a different number of messages are arrived at different time stamps).
Using some counters and put a producer to sleep for a while after it has send n messages.
If you can have a priority queue I think each producer can have a message sent counter. And the queue will order based on the messageSent number and the date, such that a message should be sent before another message if its sent number is less then the other message.
In Java
class Message { //or you can implement Comparable<Message>
final Date created = new Date();
final int messageNumber;
public Message(int m ){this.messageNumber = m;}
}
BlockingQueue<Message> queue = new PriorityBlockingQueue<Message>(new Comparator(){
public int compare(Message m1, Message m2){
if(m1.messageNumber < m2.messageNumber) return 1;
if(m2.messageNumber < m1.messageNumber) return -1;
if(m1.messageNumber == m2.messageNumber) return m1.created.compareTo(m2.created);
}
});
class Provider{
int currentMessage = 0;
void send(){
queue.offer(new Message(currentMessage++));
}
}
So if Producer 1 adds 5 elements to the queue (first) and Producer 2 adds 1, the queue will have
P1: 5
P1: 4
P1: 3
P1: 2
P2: 1
P1: 1
One of the simplest and best way is to process the messages on the order of their arrival (a simple FIFO list would do the trick). it doesn't matter even if multiple messages come at the same time. By this way, none of the producers will be starved.
One thing I would make sure is the consumer consumes the messages more rapidly than the producers producing the messages. If not it might end up in producers waiting for the consumer and there won't be any advantage on having multiple producers for a single consumer.
Related
I am a beginner in the spin. I am trying that the model runs the two receiving processes (function called consumer in the model) alternatively, ie. (consumer 1, consumer 2, consumer 1, consumer 2,...). But when I run this code, my output for 2 consumer processes are showing randomly. Can someone help me?
This is my code I am struggling with.
mtype = {P, C};
mtype turn = P;
chan ch1 = [1] of {bit};
byte current_consumer = 1;
byte previous_consumer;
active [2] proctype Producer()
{`
bit a = 0;
do
:: atomic {
turn == P ->
ch1 ! a;
printf("The producer %d --> sent %d!\n", _pid, a);
a = 1 - a;
turn = C;
}
od
}
active [2] proctype Consumer()
{
bit b;
do
:: atomic{
turn == C ->
current_consumer = _pid;
ch1 ? b;
printf("The consumer %d --> received %d!\n\n", _pid, b);
assert(current_consumer == _pid);
turn = P;
}
od
}
Sample out is as photo
First of all, let me draw your attention to this excerpt of atomic's documentation:
If any statement within the atomic sequence blocks, atomicity is lost, and other processes are then allowed to start executing statements. When the blocked statement becomes executable again, the execution of the atomic sequence can be resumed at any time, but not necessarily immediately. Before the process can resume the atomic execution of the remainder of the sequence, the process must first compete with all other active processes in the system to regain control, that is, it must first be scheduled for execution.
In your model, this is currently not causing any problem because ch1 is a buffered channel (i.e. it has size >= 1). However, any small change in the model could break this invariant.
From the comments, I understand that your goal is to alternate consumers, but you don't really care which producer is sending the data.
To be honest, your model already contains two examples of how processes can alternate with one another:
The Producer/Consumers alternate one another via turn, by assigning a different value each time
The Producer/Consumers alternate one another also via ch1, since this has size 1
However, both approaches are alternating Producer/Consumers rather than Consumers themselves.
One approach I like is message filtering with eval (see docs): each Consumer knows its own id, waits for a token with its own id in a separate channel, and only when that is available it starts doing some work.
byte current_consumer;
chan prod2cons = [1] of { bit };
chan cons = [1] of { byte };
proctype Producer(byte id; byte total)
{
bit a = 0;
do
:: true ->
// atomic is only for printing purposes
atomic {
prod2cons ! a;
printf("The producer %d --> sent %d\n", id, a);
}
a = 1 - a;
od
}
proctype Consumer(byte id; byte total)
{
bit b;
do
:: cons?eval(id) ->
current_consumer = id;
atomic {
prod2cons ? b;
printf("The consumer %d --> received %d\n\n", id, b);
}
assert(current_consumer == id);
// yield turn to the next Consumer
cons ! ((id + 1) % total)
od
}
init {
run Producer(0, 2);
run Producer(1, 2);
run Consumer(0, 2);
run Consumer(1, 2);
// First consumer is 0
cons!0;
}
This model, briefly:
Producers/Consumers alternate via prod2cons, a channel of size 1. This enforces the following behavior: after some producers created a message some consumer must consume it.
Consumers alternate via cons, a channel of size 1 containing a token value indicating which consumer is currently allowed to perform some work. All consumers peek on the contents of cons, but only the one with a matching id is allowed to consume the token and move on. At the end of its turn, the consumer creates a new token with the next id in the chain. Consumers alternate in a round robin fashion.
The output is:
The producer 0 --> sent 0
The consumer 1 --> received 0
The producer 1 --> sent 1
The consumer 0 --> received 1
The producer 1 --> sent 0
The consumer 1 --> received 0
...
The producer 0 --> sent 0
The consumer 1 --> received 0
The producer 0 --> sent 1
The consumer 0 --> received 1
The producer 0 --> sent 0
The consumer 1 --> received 0
The producer 0 --> sent 1
The consumer 0 --> received 1
Notice that producers do not necessarily alternate with one another, whereas consumers do -- as requested.
I have a function that boils down to:
while(doWork)
{
config = generateConfigurationForTesting();
result = executeWork(config);
doWork = isDone(result);
}
How can I rewrite this for efficient asynchronous execution, assuming all functions are thread safe, independent of previous iterations, and probably require more iterations than the maximum number of allowable threads ?
The problem here is we don't know how many iterations are required in advance so we can't make a dispatch_group or use dispatch_apply.
This is my first attempt, but it looks a bit ugly to me because of arbitrarily chosen values and sleeping;
int thread_count = 0;
bool doWork = true;
int max_threads = 20; // arbitrarily chosen number
dispatch_queue_t queue =
dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
while(doWork)
{
if(thread_count < max_threads)
{
dispatch_async(queue, ^{ Config myconfig = generateConfigurationForTesting();
Result myresult = executeWork();
dispatch_async(queue, checkResult(myresult)); });
thread_count++;
}
else
usleep(100); // don't consume too much CPU
}
void checkResult(Result value)
{
if(value == good) doWork = false;
thread_count--;
}
Based on your description, it looks like generateConfigurationForTesting is some kind of randomization technique or otherwise a generator which can make a near-infinite number of configuration (hence your comment that you don't know ahead of time how many iterations you will need). With that as an assumption, you are basically stuck with the model that you've created, since your executor needs to be limited by some reasonable assumptions about the queue and you don't want to over-generate, as that would just extend the length of the run after you have succeeded in finding value ==good measurements.
I would suggest you consider using a queue (or OSAtomicIncrement* and OSAtomicDecrement*) to protect access to thread_count and doWork. As it stands, the thread_count increment and decrement will happen in two different queues (main_queue for the main thread and the default queue for the background task) and thus could simultaneously increment and decrement the thread count. This could lead to an undercount (which would cause more threads to be created than you expect) or an overcount (which would cause you to never complete your task).
Another option to making this look a little nicer would be to have checkResult add new elements into the queue if value!=good. This way, you load up the initial elements of the queue using dispatch_apply( 20, queue, ^{ ... }) and you don't need the thread_count at all. The first 20 will be added using dispatch_apply (or an amount that dispatch_apply feels is appropriate for your configuration) and then each time checkResult is called you can either set doWork=false or add another operation to queue.
dispatch_apply() works for this, just pass ncpu as the number of iterations (apply never uses more than ncpu worker threads) and keep each instance of your worker block running for as long as there is more work to do (i.e. loop back to generateConfigurationForTesting() unless !doWork).
How can I limit number of threads that are being executed at the same time?
Here is sample of my algorithm:
for(i = 0; i < 100000; i++) {
Thread.start {
// Do some work
}
}
I would like to make sure that once number of threads in my application hits 100, algorithm will pause/wait until number of threads in the app goes below 100.
Currently "some work" takes some time to do and I end up with few thousands of threads in my app. Eventually it runs out of threads and "some work" crashes. I would like to fix it by limiting number of pools that it can use at one time.
Please let me know how to solve my issue.
I believe you are looking for a ThreadPoolExecutor in the Java Concurrency API. The idea here is that you can define a maximum number of threads in a pool and then instead of starting new Threads with a Runnable, just let the ThreadPoolExecutor take care of managing the upper limit for Threads.
Start here: http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html
import java.util.concurrent.*;
import java.util.*;
def queue = new ArrayBlockingQueue<Runnable>( 50000 )
def tPool = new ThreadPoolExecutor(5, 500, 20, TimeUnit.SECONDS, queue);
for(i = 0; i < 5000; i++) {
tPool.execute {
println "Blah"
}
}
Parameters for the ThreadBlockingQueue constructor: corePoolSize (5), this is the # of threads to create and to maintain if the system is idle, maxPoolSize (500) max number of threads to create, 3rd and 4th argument states that the pool should keep idle threads around for at least 20 seconds, and the queue argument is a blocking queue that stores queued tasks.
What you'll want to play around with is the queue sizes and also how to handle rejected tasks. If you need to execute 100k tasks, you'll either have to have a queue that can hold 100k tasks, or you'll have to have a strategy for handling a rejected tasks.
I have just started learning multi-threading. I have written a simple application. The application creates three threads. Two threads write and one thread reads. The writer threads write to separate location in a global array. The writer thread after incrementing the value in the array notifies the reader. The reader thread then decrements that value in the array and waits again for the writer threads to update their corresponding value in the array. The code for the application is pasted below.
What I see is that the writer(Producer) threads get more time slice than the reader(Consumer) thread. I think I am doing something wrong. If the output of the application is redirected to a file, then it can be observed that there are more consecutive messages from the Producers and the messages from the Consumer occur infrequently. What I was expecting was that, when a Producer updates its data, the Consumer immediately processes it i.e. after every Producer message there should be a Consumer message printed.
Thanks and regards,
~Plug
#include <stdio.h>
#include <pthread.h>
const long g_lProducerCount = 2; /*Number of Producers*/
long g_lProducerIds[2]; /*Producer IDs = 0, 1...*/
long g_lDataArray[2]; /*Data[0] for Producer 0, Data[1] for Producer 1...*/
/*Producer ID that updated the Data. -1 = No update*/
long g_lChangedProducerId = -1;
pthread_cond_t g_CondVar = PTHREAD_COND_INITIALIZER;
pthread_mutex_t g_Mutex = PTHREAD_MUTEX_INITIALIZER;
pthread_t g_iThreadIds[3]; /*3 = 2 Producers + 1 Consumer*/
unsigned char g_bExit = 0; /*Exit application? 0 = No*/
void* Producer(void *pvData)
{
long lProducerId = *(long*)pvData; /*ID of this Producer*/
while(0 == g_bExit) {
pthread_mutex_lock(&g_Mutex);
/*Tell the Consumer who's Data is updated*/
g_lChangedProducerId = lProducerId;
/*Update the Data i.e. Increment*/
++g_lDataArray[lProducerId];
printf("Producer: Data[%ld] = %ld\n",
lProducerId, g_lDataArray[lProducerId]);
pthread_cond_signal(&g_CondVar);
pthread_mutex_unlock(&g_Mutex);
}
pthread_exit(NULL);
}
void* Consumer(void *pvData)
{
while(0 == g_bExit) {
pthread_mutex_lock(&g_Mutex);
/*Wait until one of the Producers update it's Data*/
while(-1 == g_lChangedProducerId) {
pthread_cond_wait(&g_CondVar, &g_Mutex);
}
/*Revert the update done by the Producer*/
--g_lDataArray[g_lChangedProducerId];
printf("Consumer: Data[%ld] = %ld\n",
g_lChangedProducerId, g_lDataArray[g_lChangedProducerId]);
g_lChangedProducerId = -1; /*Reset for next update*/
pthread_mutex_unlock(&g_Mutex);
}
pthread_exit(NULL);
}
void CreateProducers()
{
long i;
pthread_attr_t attr;
pthread_attr_init(&attr);
for(i = 0; i < g_lProducerCount; ++i) {
g_lProducerIds[i] = i;
pthread_create(&g_iThreadIds[i + 1], &attr,
Producer, &g_lProducerIds[i]);
}
pthread_attr_destroy(&attr);
}
void CreateConsumer()
{
pthread_attr_t attr;
pthread_attr_init(&attr);
pthread_create(&g_iThreadIds[0], &attr, Consumer, NULL);
pthread_attr_destroy(&attr);
}
void WaitCompletion()
{
long i;
for(i = 0; i < g_lProducerCount + 1; ++i) {
pthread_join(g_iThreadIds[i], NULL);
}
}
int main()
{
CreateProducers();
CreateConsumer();
getchar();
g_bExit = 1;
WaitCompletion();
return 0;
}
You would have to clarify what is it exactly that you want to achieve. For now the producers only increment an integer and the consumer decrements the value. This is not a very useful activity ;) I understand that this is only a test app, but still it is not clear enough what's the purpose of this processing, what are the constraints and so on.
The producers produce some 'items'. The outcome of this production is represented as an integer value. 0 means no items, 1 means there is a pending item, that consumer can take. Is that right? Now, is it possible for the producer to produce several items before any of them gets consumed (incrementing the array cell to a value higher than 1)? Or does he have to wait for the last item to be consumed before the next one can be put into the storage? Is the storage limited or unlimited? If it is limited then is the limit shared among all the producers or is it defined per producer?
What I was expecting was that, when a Producer updates its data,
the Consumer immediately processes it i.e. after every Producer
message there should be a Consumer message printed.
Though it's not really clear what you want to achieve I will hold on to that quote and assume the following: there is a limit of 1 item per producer and the producer has to wait for the consumer to empty the storage before a new item can be put in the cell i.e. the only allowed values in the g_lDataArray are 0 and 1.
To allow maximum concurrency between threads you will need a conditional variable/mutex pair for each cell of g_lDataArray (for each producer). You will also need a queue of updates that is a list of producers that have submitted their work and a conditional variable/mutex pair to guard it, this will replace g_lChangedProducerId which can only hold one value at a time.
Every time a producer wants to put an item into the storage it has to acquire the respective lock, check if the storage is empty (g_lDataArray[lProducerId] == 0), if not wait on the condition variable and then, increment the cell, release the held lock, acquire the consumer lock, add his id to the update queue, notify the consumer, release the consumer lock. Of course if the producer would perform any real computations producing some real item, this work should be performed out of the scope of any lock, before the attempt to put the item in the storage.
In pseudo code this looks like this:
// do some computations
item = compute();
lock (mutexes[producerId]) {
while (storage[producerId] != 0)
wait(condVars[producerId]);
storage[producerId] = item;
}
lock (consumerMutex) {
queue.push(producerId);
signal(consumerCondVar);
}
The consumer should act as follows: acquire his lock, check if there are any pending updates to process, if not wait on the condition variable, take one update out of the queue (that is the number of the updating producer), acquire the lock for producer who's update is going to be processed, decrement the cell, notify the producer, release the producer's lock, release his lock, finally process the update.
lock (consumerMutex) {
while (queue.isEmpty())
wait(consumerCondVar);
producerId = queue.pop();
lock (mutexex[producerId]) {
item = storage[producerId];
storage[producerId] = 0;
signal(condVars[producerId]);
}
}
//process the update
process(item);
Hope this answer is what you needed.
The problem may be that all producers change g_lChangedProducerId, so the value written by one producer may be overwritten by another producer before the consumer sees it.
This means that the consumer effectively doesn't see that the first producer has produced some output.
Well,when you producer produced, it may wake up the ProThread or ConThread.
And If it waked up the ProThread,the producer produced again,and the ConThread didn't consume immediately after data is produced.
That's what you don't want to see.
All you need is to make sure that when it produced,it won't wake the ProThread up.
Here's one kind of solution for this
void* Producer(void *pvData)
{
........
//wait untill consumer consume its number
while(-1!=g_lChangedProducerId)
pthread_cond_wait(&g_CondVar,&g_Mutex);
//here to inform the consumer it produced the data
g_lChangedProducerId = lProducerId;
........
}
void* Consumer(void *pvData)
{
g_lChangedProducerId = -1;
**//wake up the producer when it consume
pthread_cond_signal(&g_CondVar);**
pthread_mutex_unlock(&g_Mutex);
}
Something I don't get about the classical algorithm for the Producer-Consumer problem (from Wikipedia:)
semaphore mutex = 1
semaphore fillCount = 0
semaphore emptyCount = BUFFER_SIZE
procedure producer() {
while (true) {
item = produceItem()
down(emptyCount)
down(mutex)
putItemIntoBuffer(item)
up(mutex)
up(fillCount)
}
up(fillCount) //the consumer may not finish before the producer.
}
procedure consumer() {
while (true) {
down(fillCount)
down(mutex)
item = removeItemFromBuffer()
up(mutex)
up(emptyCount)
consumeItem(item)
}
}
I note that both producers and consumers lock 'mutex' prior to messing with the buffer, and unlock it thereafter. If that is the case, i.e. only a single thread is accessing the buffer at any given moment, I don't really see how the above algo differs from a simple solution that entails only putting a guarding mutex over the buffer:
semaphore mutex = 1
procedure producer() {
while (true) {
item = produceItem()
flag = true
while (flag) {
down(mutex)
if (bufferNotFull()) {
putItemIntoBuffer(item)
flag = false
}
up(mutex)
}
}
}
procedure consumer() {
while (true) {
flag = true
while (flag) {
down(mutex)
if (bufferNotEmpty()) {
item = removeItemFromBuffer()
flag = false
}
up(mutex)
}
consumeItem(item)
}
}
Only thing I can think of that necessitates using the 'fillCount' and 'emptyCount' semaphores is scheduling.
Maybe the first algo is for making sure that in a state where 5 consumers are waiting on an empty buffer (zero 'fillCount'), it is assured that when a new producer comes along, it will go past its "down(emptyCount)" statement quickly and get the 'mutex' quickly.
(whereas in the other solution the consumers will needlessly get the 'mutex' only to relinquish it until the new producer gets it and inserts an item).
Am I right? Am I missing something?
If there are no messages in the buffer, the consumer will down the mutex, check the buffer, find that it's empty, up the mutex, loop back around and immediately repeat the process. In simple terms, consumers and producers are stuck in busy loops that chew up 100% of a CPU core. This is not just a theoretical problem, either. You may well find that your computer's fan starts to spin every time you run your program.
The hole concept of the producer/consumer pattern is that the shared resource (buffer) is only accessed if certain criteria are met. And to not utilize unnecessary CPU cycles in order to make sure they are met.
Consumer:
Wait if there is nothing to consume (=> empty buffer).
Producer:
Wait if there is not enough space in the buffer.
And for both its very important to note:
Wait != Spin wait
There's more than just the locking of the buffer going on.
The fillCount semaphore in the first program is to pause the consumer(s) when there's nothing left to consume. Without it, you're constantly polling the buffer to see if there's anything to get, and that's quite wasteful.
Likewise, the emptyCount semaphore pauses the producer when the buffer is full.
There is a starving issue if more than one consumer is involved. In the latter example consumer checks for an item in a loop. But by the time he get's to try again some other consumer may "steal" his item. And next time again, and again. That is not nice.