I am writing a client for a networked application and I would like to seperate receiving and processing the messages to different threads.
This is my solution at the moment:
Mutex mutex;
Queue queue;
recv()
{
while(true)
{
messages = receive_some_messages();
mutex.lock();
queue.add(messages);
mutex.unlock();
process.notify();
}
}
proc()
{
while(true)
{
block_until_notify();
Queue to_process;
mutex.lock();
to_process.add( queue.take_all() );
mutex.unlock();
foreach(message in to_process)
{
process_message(message);
}
}
}
This has a race condition however:
recv receives a lot of messages and puts them in queue.
recv notifices proc.
proc takes all messages from queue and starts working
recv receives some more messages and puts them in queue
recv notifies proc, but as proc is still working this does nothing.
proc completes its iteration.
proc blocks - there are still unprocessed messages in queue
I can think of several methods of fixing it, however none are favorable.
Solution 1
I could adapt sync to keep the lock on the mutex during the processing:
proc()
{
while(true)
{
block_until_notify();
Queue to_process;
mutex.lock();
to_process.add( queue.take_all() );
foreach(message in to_process)
{
process_message(message);
}
mutex.unlock();
}
}
But this would mean the threads run exclusively: either recv or proc is active, but not both.
Solution 2
I could remove the block and notify.
recv()
{
while(true)
{
messages = receive_messages();
mutex.lock();
queue.add(messages);
mutex.unlock();
}
}
proc()
{
while(true)
{
Queue to_process;
mutex.lock();
to_process.add( queue.take_all() );
mutex.unlock();
foreach(message in to_process)
{
process_message(message);
}
}
}
But this means that the proc will run in a busy-wait loop, only blocking when recv is adding messages to queue.
The question
I would like a solution where proc and recv do not run exclusively and without busy-waiting.
Does anybody have any idea on what I could do?
I think you can get by if the consumer checks for an empty queue after the queue is drained.
proc()
{
while(true)
{
Queue to_process;
mutex.lock();
if (queue.empty()) {
mutex.unlock();
block_until_notify();
mutex.lock();
}
to_process.add( queue.take_all() );
mutex.unlock();
foreach(message in to_process)
{
process_message(message);
}
}
}
I believe that this fixes the race condition you mentioned.
Your block_until_notify() function is probably a condition variable. The way to do this is to change that function so that you lock the mutex, check that the queue is empty, then wait for notification. If the queue is not empty then you proceed with processing it. After processing the queue you return to the block_until_notify function and repeat the process, again checking if the queue is empty before blocking.
If you aren't using a condition variable of some kind then I'd suggest using a semaphore. On a Windows system you'd call ReleaseSemaphore every time messages were added to the queue. The receiver would call WaitForSingleObject on the semaphore handle. That would be done in a loop and the loop would continue to repeat, even if the queue is empty, until the wait blocks.
Related
Along with the main thread, i have one more thread that receives data to write them in a file.
std::queue<std::vector<int>> dataQueue;
std::mutex mutex;
void setData(const std::vector<int>& data) {
std::lock_guard<std::mutex> lock(mutex);
dataQueue.push(data);
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store) {
std::lock_guard<std::mutex> lock(mutex);
while (!dataQueue.empty()) {
std::vector<int>& data= dataQueue.front();
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
setData is used by the main thread and write is actually the writing thread. I use std::lock_quard to avoid memory conflict but when locking on the writing thread, it slows down the main thread as it has to wait for the Queue to be unlocked. But i guess i can avoid this as the threads never act on the same element of the queue at the same time.
So i would like to do it lock-free but i don't really understand how i should implement it. I mean, how can i do it without locking anything ? moreover, if the writing thread is faster than the main thread, the queue might be empty most of the time, so it should somehow waits for new data instead of looping infinitly to check for non empty queue.
EDIT: I changed simple std::lock_guard by std::cond_variable so that it could wait when the queue is empty. But the main thread can still be blocked as , when cvQeue.wait(.) is resolved, it reacquire the lock. moreover, what if the main thread does cvQueue.notify_one() but the writing thread is not waiting ?
std::queue<std::vector<int>> dataQueue;
std::mutex mutex;
std::condition_variable cvQueue;
void setData(const std::vector<int>& data) {
std::unique_lock<std::mutex> lock(mutex);
dataQueue.push(data);
cvQueue.notify_one();
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store) {
std::lock_guard<std::mutex> lock(mutex);
while (!dataQueue.empty()) {
std::unique_lock<std::mutex> lock(mutex);
cvQueue.wait(lock);
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
If you only have two threads, than you could use a lock-free single-producer-single-consumer (SPSC) queue.
A bounded version can be found here: https://github.com/rigtor/SPSCQueue
Dmitry Vyukov presented an unbounded version here: http://www.1024cores.net/home/lock-free-algorithms/queues/unbounded-spsc-queue (You should note though, that this code should be adapted to use atomics.)
Regarding a blocking pop operation - this is something that lock-free data structures do not provide since such an operation is obviously not lock-free. However, it should be relatively straight forward to adapt the linked implementations in such a way, that a push operation notifies a condition variable if the queue was empty before the push.
i guess i have something that met my needs. I did a LockFreeQueue that uses std::atomic. I can thus manage the state of the head/tail of the queue atomically.
template<typename T>
class LockFreeQueue {
public:
void push(const T& newElement) {
fifo.push(newElement);
tail.fetch_add(1);
cvQueue.notify_one();
}
void pop() {
size_t oldTail = tail.load();
size_t oldHead = head.load();
if (oldTail == oldHead) {
return;
}
fifo.pop();
head.store(++oldHead);
}
bool isEmpty() {
return head.load() == tail.load();
}
T& getFront() {
return fifo.front();
}
void waitForNewElements() {
if (tail.load() == head.load()) {
std::mutex m;
std::unique_lock<std::mutex> lock(m);
cvQueue.wait_for(lock, std::chrono::milliseconds(TIMEOUT_VALUE));
}
}
private:
std::queue<T> fifo;
std::atomic<size_t> head = { 0 };
std::atomic<size_t> tail = { 0 };
std::condition_variable cvQueue;
};
LockFreeQueue<std::vector<int>> dataQueue;
std::atomic<bool> store(true);
void setData(const std::vector<int>& data) {
dataQueue.push(data);
// do other things
}
void write(const std::string& fileName) {
std::unique_ptr<std::ostream> ofs = std::unique_ptr<std::ostream>(new zstr::ofstream(fileName));
while (store.load()) {
dataQueue.waitForNewElements();
while (!dataQueue.isEmpty()) {
std::vector<int>& data= dataQueue.getFront();
ofs->write(reinterpret_cast<char*>(data.data()), sizeof(data[0])*data.size());
dataQueue.pop();
}
}
}
}
I still have one lock in waitForNewElements but it is not locking the whole process as it is waiting for things to do. But the big improvement is that the producer can push while the consumer pop. It is only forbidden when LockFreQueue::tail and LockFreeQueue::head are the same. Meaning that the queue is empty and it enters the waiting state.
The thing that i'm not very satisfied at is cvQueue.wait_for(lock, TIMEOUT_VALUE). I wanted to do a simple cvQueue.wait(lock), but the problem is that when it comes to end the thread, I do store.store(false) in the main thread. So if the writing thread is waiting it will never end without a timeout. So, I set a big enough timeout so that most of the time the condition_variable is resolved by the lock, and when the thread ends it is resolved by the timeout.
If you feel that something must be wrong or must be improved, feel free to comment.
I need a dispatch loop in C++/CX that takes tasks as input and dispatches them in a queue on a different thread. I managed a simple implementation that consists of the following:
void Threading::DispatchQueue::Add(TaskType task)
{
// The scope causes the lock to be released before WakeByAddressSingle.
// This mutex is used for synchronizing queue operations, so it shoudln't
// be needed for WakeByAddressSingle.
{
std::lock_guard<std::mutex> lock(_queue_mux);
GetQueue().push(task);
}
WakeByAddressSingle(&CompareAddress);
}
void Threading::DispatchQueue::Runner()
{
while (true) {
TaskType task = DequeueTask();
if (task != nullptr) {
task();
}
else {
Log::LogMessage(this->GetType()->FullName, Level::Debug, "While is running!");
// The scope guarantees that the lock is released before the thread is
// put to sleep. It's noticeable that the thread must not sleep while
// the lock is acquired, or otherwise the Add(TaskType) method can never
// acquire the lock, meaning that the thread will never wake up.
{
std::lock_guard<std::mutex> lock(_queue_mux);
if (!GetQueue().empty()) {
continue;
}
}
WaitOnAddress(&CompareAddress, &UndesiredValue, sizeof(CompareAddress), -1);
}
}
}
Notice that queue operations on the two methods are synchronized by DequeueTask, as follows:
Threading::DispatchQueue::TaskType Threading::DispatchQueue::DequeueTask()
{
std::lock_guard<std::mutex> lock(_queue_mux);
QueueType & queue = GetQueue();
if (queue.empty()) {
return nullptr;
}
TaskType task = GetQueue().front();
GetQueue().pop();
return task;
}
This implementation has been working fine in most situations except when Add calls WakeByAddressSingle before the thread is put to wait. Take the following example:
The Runner acquires the mutex and checks whether the queue is empty. Lets assume it is. At this point, the block of code terminates and the lock is relinquished.
At this moment Add kicks in, acquires the lock, pushes a task to the queue, and calls WakeByAddressSingle. Notice that the Runner thread is not sleeping yet.
The Runner thread gets CPU time again and goes to sleep with WaitOnAddress.
As the task was already added and WakeByAddressSingle called, the thread never awakes until another task is added.
The problem is that I cannot acquire locks (I think?) before WaitOnAddress because if the thread goes to sleep with the lock then the lock will never be released. Is there some sort of paradigm that can be used in this situation? How do you synchronize WakeByAddressSingle and WaitOnAddress?
I have a server listening to some port, and I create several detached threads.
Not only the server it self will run forever, but also the detached threads will run forever.
//pseudocode
void t1_func()
{
for(;;)
{
if(notified from server)
dosomething();
}
}
thread t1(t1_func);
thread t2(...);
for(;;)
{
// read from accepted socket
string msg = socket.read_some(...);
//notify thread 1 and thread 2;
}
Since I am new to multithreading, I don't know how to implement such nofity in server, and check the nofity in detached threads.
Any helpful tips will be appreciated.
The easiest way to do this is with std::condition_variable.
std::condition_variable will wait until another thread calls either notify_one or notify_all on it and only then will it wake up.
Here is your t1_func implemented using condition variables:
std::condition_variable t1_cond;
void t1_func()
{
//wait requires a std::unique_lock
std::mutex mtx;
std::unique_lock<std::mutex> lock{ mtx };
while(true)
{
t1_cond.wait(lock);
doSomething();
}
}
The wait method takes a std::unique_lock but the lock doesn't have to be shared to notify the thread. When you want to wake up the worker thread from the main thread you would call notify_one or notify_all like this:
t1_cond.notify_one();
If you want to have the thread wake up after a certain amount of time you could use wait_for instead of wait.
I'm trying to create a thread and let it run until my main signals it to start, which I think is done with SetEvent. But the code in the thread is never executed. Below is the bare code I have stripped down of (I think) unrelated functions. Is the algorithm correct ?
Here is what I thought it did :
When in the main, the thread is created, which means it'll run in the background. When the event is set (SetEvent), the thread picks it up at WaitForSingleObject and then execute the code in the thread, right ?
HANDLE hThread;
HANDLE Event;
DWORD Thread()
{
while(1)
{
wait = WaitForSingleObject(Event, INFINITE)
//This is where I want to execute something
}
}
int _tmain()
{
DWORD dw;
int i;
Event = CreateEvent(NULL,false,false,NULL);
hThread = CreateThread(NULL,0,Thread,EventA,0,NULL);
while(1)
{
if (condition is correct)
{
SetEvent(Event);
}
CloseHandle(Thread);
CloseHandle(Event);
}
return 0;
}
Thanks for having read.
Move CloseHandle lines out of the while loop.
I want to use two queues where the 1st queue will push the data to the queue and the 2nd thread will remove the data from the queue.
Can somebody help me plz on implementing this in VC++?
I am new to the threads and queue.
This is the producer / consumer problem, here is one implementation of it in c++
Here is a few pointers and some sample code:
std::queue<int> data; // the queue
boost::mutex access; // a mutex for synchronising access to the queue
boost::condition cond; // a condition variable for communicating the queue state
bool empty()
{
return data.empty();
}
void thread1() // the consumer thread
{
while (true)
{
boost::mutex::scoped_lock lock(access);
cond.wait(lock, empty);
while (!empty())
{
int datum=data.top();
data.pop();
// do something with the data
}
}
}
void thread2() // the producer thread
{
while (true)
{
boost::mutex::scoped_lock lock(access);
data.push_back(1); // guaranteed random by a fair dice roll
cond.notify_one();
}
}
int main()
{
boost::thread t1(thread1);
boost::thread t2(thread2);
t1.join();
t2.join();
return 0;
}
I used a queue from the STL and threads, condition variables and mutexes from the Boost library. The stuff in boost is pretty much what is going to be in the standard C++ library in a few years so it is good to be familiar with it.
I only typed the code in, I have no way to test it. You'll probably find bugs. This is a good thing, threaded code is difficult and the best way to get good at it is lots of practice.