Platform: Windows 7 Professional 64 bit
Compiler: VS2010 Express
Boost: Version 1.49
Plugin System: OBSE 20 (for the Oblivion game by Bethesda)
I have a class based upon the async udp examples. I run the io service itself as a thread. Here is the code for the class:
// udp buffer queues
extern concurrent_queue<udp_packet> udp_input_queue; // input from external processes
extern concurrent_queue<udp_packet> udp_output_queue; // output to external processes
using boost::asio::ip::udp;
class udp_server
{
public:
udp_server(boost::asio::io_service& io_service, short port)
: io_service_(io_service),
socket_(io_service_, udp::endpoint(boost::asio::ip::address_v4::from_string(current_address), port))//, // udp::v4()
{
// start udp receive
socket_.async_receive_from(
boost::asio::buffer(recv_buf), sender_endpoint_,
boost::bind(&udp_server::handle_receive_from, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
send_timer_ = NULL;
}
~udp_server(){
io_service_.stop();
if(send_timer_){
send_timer_->cancel();
delete send_timer_;
}
}
void start(){
// start send timer
send_timer_ = new boost::asio::deadline_timer(io_service_, boost::posix_time::milliseconds(500));
send_timer_restart();
}
void handle_send_to(const boost::system::error_code& error, size_t bytes_recvd);
void handle_receive_from(const boost::system::error_code& error, size_t bytes_recvd);
//void handle_send_timer(const boost::system::error_code& error);
void handle_send_timer();
void send_timer_restart();
void stop()
{
io_service_.stop();
}
private:
boost::asio::io_service& io_service_;
udp::socket socket_;
udp::endpoint sender_endpoint_;
std::vector<udp::endpoint> clientList;
//std::auto_ptr<boost::asio::io_service::work> busy_work;
udp_buffer recv_buf;
boost::asio::deadline_timer* send_timer_;
};
Now I instantiate the class and thread like this:
udp_server *udp_server_ptr=NULL;
boost::asio::deadline_timer* dlineTimer=NULL;
static void PluginInit_PostLoadCallback()
{
_MESSAGE("NetworkPipe: PluginInit_PostLoadCallback called");
if(!g_Interface->isEditor)
{
_MESSAGE("NetworkPipe: Starting UDP");
udp_server_ptr = new udp_server(io_service, current_port);
//dlineTimer = new boost::asio::deadline_timer(io_service);
udp_thread = new boost::thread(boost::bind(&boost::asio::io_service::run, &io_service));
//
_MESSAGE("NetworkPipe: UDP Started");
NetworkPipeEnable = true;
}
else
{
_MESSAGE("NetworkPipe: Running in editor, not starting UDP");
}
}
Now notice that dlineTimer is commented out above. If I enable that it ceases to function. The only way I can get the dlineTimer to function with this io service is to create it during the udp_server::handle_receive_from call. I think this is because it is running inside the other thread. So for some reason the deadline_timer object does not like being created outside the thread it needs to run inside.
Now, in order to communicate to the main thread I use concurrent_queue objects. So these allow me to send messages in and out of the thread pretty simply. I could theoretically run the dlineTimer inside its own thread and use the output queue to manage its activity. However, I like the simplicity of having is in the same thread as the udp_server. For instance the udp_server object keeps track of clients in a vector. When the deadline_timer expires I cycle through the known clients and send them messages. Then I restart the timer. This makes my response independent of the udp packets that are sent to the server. So when packets arrive they are put on a queue for another part of the process. Then later data is placed on the output queue and the deadline_timer processes those responses and sends them to the appropriate clients.
So my main question is:
How do I more cleanly create the deadline_timer object using the same thread and same io_service as the udp_server object?
Okay, I was thinking about this really stupidly.
First the deadline_timer needs to be completely inside the thread I want it to time in. That means it needs to be created inside the thread.
Second I need to define the function called in the thread loop and not set it to the io_service::run function. So I made it the udp_server::start function. Inside the start call I create my deadline_timer.
So here is the class:
class udp_server
{
public:
udp_server(boost::asio::io_service& io_service, short port)
: io_service_(io_service),
socket_(io_service_, udp::endpoint(boost::asio::ip::address_v4::from_string(current_address), port))//, // udp::v4()
{
// start udp receive
socket_.async_receive_from(
boost::asio::buffer(recv_buf), sender_endpoint_,
boost::bind(&udp_server::handle_receive_from, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
send_timer_ = NULL;
}
~udp_server(){
io_service_.stop();
if(send_timer_){
send_timer_->cancel();
delete send_timer_;
}
}
void start();
void startSendTimer();
void handle_send_to(const boost::system::error_code& error, size_t bytes_recvd);
void handle_receive_from(const boost::system::error_code& error, size_t bytes_recvd);
void handle_send_timer();
void send_timer_restart();
void stop()
{
io_service_.stop();
}
private:
boost::asio::io_service& io_service_;
udp::socket socket_;
udp::endpoint sender_endpoint_;
std::vector<udp::endpoint> clientList;
udp_buffer recv_buf;
boost::asio::deadline_timer* send_timer_;
};
Here are the relevant functions:
void udp_server::start(){
// startup timer
startSendTimer();
// run ioservice
io_service_.run();
}
void udp_server::startSendTimer(){
// start send timer
if(!send_timer_)
send_timer_ = new boost::asio::deadline_timer(io_service_, boost::posix_time::milliseconds(500));
send_timer_restart();
}
void udp_server::send_timer_restart(){
if(send_timer_){
// restart send timer
send_timer_->expires_from_now(boost::posix_time::milliseconds(500));
send_timer_->async_wait(boost::bind(&udp_server::handle_send_timer, this));
}
}
void udp_server::handle_send_timer(){
for(std::vector<udp::endpoint>::iterator itr = clientList.begin(); itr != clientList.end(); ++itr){
socket_.async_send_to(
boost::asio::buffer("heart beat", strlen("heart beat")), *itr,
boost::bind(&udp_server::handle_send_to, this,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred));
}
send_timer_restart();
}
So I was thinking about this all wrong in the first place. I need to define my starting point of where the thread begins execution. The I can create the objects that need to reside in that thread inside the thread.
The udp_server is now started like this:
static void PluginInit_PostLoadCallback()
{
_MESSAGE("NetworkPipe: PluginInit_PostLoadCallback called");
if(!g_Interface->isEditor)
{
_MESSAGE("NetworkPipe: Starting UDP");
udp_server_ptr = new udp_server(io_service, current_port);
udp_thread = new boost::thread(boost::bind(&udp_server::start, udp_server_ptr));
_MESSAGE("NetworkPipe: UDP Started");
NetworkPipeEnable = true;
}
else
{
_MESSAGE("NetworkPipe: Running in editor, not starting UDP");
}
}
The deadline_timer creation occurs within the udp_thread now. Creating the deadline_timer object in the main thread would cause the program to fail to load properly.
Related
Intro
I am working on a multithreaded cross platform logging tool which purposes logging to the console and/or a file. However the problem I have right now is only related to the console logging.
Problem breakdown
The way the logger works is by adding a string to a Queue of strings. In the "LogToConsole" thread there is a wait method that waits until there is a string added to the queue. When that happens it should get notified and print, then pop and unlock.
Used variables
class Logger
{
public:
friend void LogToConsole(Logger* logger);
private:
std::atomic_flag _ThreadIsRunning { ATOMIC_FLAG_INIT };
std::thread _LogThread;
std::queue<std::string> _LogBuffer;
std::map<std::thread::id, std::string> _ThreadName;
std::mutex _LogMutex;
std::mutex _AppendLock;
std::condition_variable _LogLock;
...
The place I add data to the buffer
template<LogSeverity severity>
void Logger::Log(std::stringstream& message)
{
std::stringstream log;
log << _ThreadName[std::this_thread::get_id()] << ":\t";
log << message.str();
std::unique_lock<std::mutex> logAppendLock(_AppendLock);
_LogBuffer.push(log.str());
logAppendLock.unlock();
_LogLock.notify_one();
}
template<LogSeverity severity>
void Logger::Log(std::string message)
{
std::stringstream log;
log << message.c_str();
this->Log<severity>(log);
}
The thread that runs in a separate loop (notice however that this method is not part of the logger class):
void LogToConsole(Logger* logger)
{
do
{
std::unique_lock<std::mutex> lock(logger->_LogMutex);
logger->_LogLock.wait(lock);
std::printf("%s", logger->_LogBuffer.back().c_str());
logger->_LogBuffer.pop();
lock.unlock();
} while (logger->_ThreadIsRunning.test_and_set() || !logger->_LogBuffer.empty());
}
Place of thread creation
Logger::Logger() : _LogThread(), _LogBuffer(), _ThreadName(), _LogMutex()
{
_ThreadIsRunning.test_and_set();
_LogThread = std::thread(LogToConsole, this);
}
Test casing
std::shared_ptr<Logger> engineLogger = std::make_shared<Logger>();
engineLogger->SetThreadName("EngineLogger");
std::shared_ptr<Logger> coreLogger = std::make_shared<Logger>();
coreLogger->SetThreadName("CoreLogger");
while(true)
{
engineLogger->Log<LOG_INFO>("LOG\n");
coreLogger->Log<LOG_WARNING>("WARNING\n");
}
The code seems to be working threadsafe, no dataraces etc. but it crashes after a 5-10~ seconds. I have searched if there are people that are having a similar problem, this does not seem to be the case.
I am however not very experienced with concurrent programming, thus do not now how to handle these kind of problems very easily.
Hope someone can solve the problem or give me some advice to prevent the problem.
In while (logger->_ThreadIsRunning.test_and_set() || !logger->_LogBuffer.empty()); you access logger without a mutex, so that is a data race. Also _LogBuffer.push(log.str()); is accessed with only _AppendLock locked and logger->_LogBuffer.pop() is accessed with only _LogMutex locked, so that is another data race.
Data races are UB and a possible reason for the crash.
I have a timer that will create a new thread and wait for the timer to expire before calling the notify function. It works correctly during the first execution, but when the timer is started a second time, an exception is thrown trying to create the new thread. The debug output shows that the previous thread has exited before attempting to create the new thread.
Timer.hpp:
class TestTimer
{
private:
std::atomic<bool> active;
int timer_duration;
std::thread thread;
std::mutex mtx;
std::condition_variable cv;
void timer_func();
public:
TestTimer() : active(false) {};
~TestTimer() {
Stop();
}
TestTimer(const TestTimer&) = delete; /* Remove the copy constructor */
TestTimer(TestTimer&&) = delete; /* Remove the move constructor */
TestTimer& operator=(const TestTimer&) & = delete; /* Remove the copy assignment operator */
TestTimer& operator=(TestTimer&&) & = delete; /* Remove the move assignment operator */
bool IsActive();
void StartOnce(int TimerDurationInMS);
void Stop();
virtual void Notify() = 0;
};
Timer.cpp:
void TestTimer::timer_func()
{
auto expire_time = std::chrono::steady_clock::now() + std::chrono::milliseconds(timer_duration);
std::unique_lock<std::mutex> lock{ mtx };
while (active.load())
{
if (cv.wait_until(lock, expire_time) == std::cv_status::timeout)
{
lock.unlock();
Notify();
Stop();
lock.lock();
}
}
}
bool TestTimer::IsActive()
{
return active.load();
}
void TestTimer::StartOnce(int TimerDurationInMS)
{
if (!active.load())
{
if (thread.joinable())
{
thread.join();
}
timer_duration = TimerDurationInMS;
active.store(true);
thread = std::thread(&TestTimer::timer_func, this);
}
else
{
Stop();
StartOnce(TimerDurationInMS);
}
}
void TestTimer::Stop()
{
if (active.load())
{
std::lock_guard<std::mutex> _{ mtx };
active.store(false);
cv.notify_one();
}
}
The error is being thrown from my code block here:
thread = std::thread(&TestTimer::timer_func, this);
during the second execution.
Specifically, the error is being thrown from the move_thread function: _Thr = _Other._Thr;
thread& _Move_thread(thread& _Other)
{ // move from _Other
if (joinable())
_XSTD terminate();
_Thr = _Other._Thr;
_Thr_set_null(_Other._Thr);
return (*this);
}
_Thrd_t _Thr;
};
And this is the exception: Unhandled exception at 0x76ED550B (ucrtbase.dll) in Sandbox.exe: Fatal program exit requested.
Stack trace:
thread::move_thread(std::thread &_Other)
thread::operator=(std::thread &&_Other)
TestTimer::StartOnce(int TimerDurationInMS)
If it's just a test
Make sure the thread handler is empty or joined when calling the destructor.
Make everything that can be accessed from multiple threads thread safe (specifically, reading the active flag). Simply making it an std::atomic_flag should do.
It does seem like you are killing a thread handle pointing to a live thread, but hard to say without seeing the whole application.
If not a test
...then generally, when need a single timer, recurreing or not, you can just go away with scheduling an alarm() signal into itself. You remain perfectly single threaded and don't even need to link with the pthread library. Example here.
And when expecting to need more timers and stay up for a bit it is worth to drop an instance of boost::asio::io_service (or asio::io_service if you need a boost-free header-only version) into your application which has mature production-ready timers support. Example here.
You create the TestTimer and run it the first time via TestTimer::StartOnce, where you create a thread (at the line, which later throws the exception). When the thread finishes, it sets active = false; in timer_func.
Then you call TestTimer::StartOnce a second time. As active == false, Stop() is not called on the current thread, and you proceed to creating a new thread in thread = std::thread(&TestTimer::timer_func, this);.
And then comes the big but:
You have not joined the first thread before creating the second one. And that's why it throws an exception.
I am working on a c++ (11) project and on the main thread, I need to check the value of two variables. The value of the two variables will be set by other threads through two different callbacks. I am using two condition variables to notify changes of those two variables. Because in c++, locks are needed for condition variables, I am not sure if I should use the same mutex for the two condition variables or I should use two mutex's to minimize exclusive execution. Somehow, I feel one mutex should be sufficient because on one thread(the main thread in this case) the code will be executed sequentially anyway. The code on the main thread that checks (wait for) the value of the two variables wont be interleaved anyway. Let me know if you need me to write code to illustrate the problem. I can prepare that. Thanks.
Update, add code:
#include <mutex>
class SomeEventObserver {
public:
virtual void handleEventA() = 0;
virtual void handleEventB() = 0;
};
class Client : public SomeEventObserver {
public:
Client() {
m_shouldQuit = false;
m_hasEventAHappened = false;
m_hasEventBHappened = false;
}
// will be callbed by some other thread (for exampe, thread 10)
virtual void handleEventA() override {
{
std::lock_guard<std::mutex> lock(m_mutexForA);
m_hasEventAHappened = true;
}
m_condVarEventForA.notify_all();
}
// will be called by some other thread (for exampe, thread 11)
virtual void handleEventB() override {
{
std::lock_guard<std::mutex> lock(m_mutexForB);
m_hasEventBHappened = true;
}
m_condVarEventForB.notify_all();
}
// here waitForA and waitForB are in the main thread, they are executed sequentially
// so I am wondering if I can use just one mutex to simplify the code
void run() {
waitForA();
waitForB();
}
void doShutDown() {
m_shouldQuit = true;
}
private:
void waitForA() {
std::unique_lock<std::mutex> lock(m_mutexForA);
m_condVarEventForA.wait(lock, [this]{ return m_hasEventAHappened; });
}
void waitForB() {
std::unique_lock<std::mutex> lock(m_mutexForB);
m_condVarEventForB.wait(lock, [this]{ return m_hasEventBHappened; });
}
// I am wondering if I can use just one mutex
std::condition_variable m_condVarEventForA;
std::condition_variable m_condVarEventForB;
std::mutex m_mutexForA;
std::mutex m_mutexForB;
bool m_hasEventAHappened;
bool m_hasEventBHappened;
};
int main(int argc, char* argv[]) {
Client client;
client.run();
}
Platform: Qt 4.8.2 (built from source using MinGW x64), Win 7
I'm using QRunnable to separate long running tasks from main GUI thread, and I sometimes experience random crash/strange behavior with no traceable errors. Please help to provide suggestions on what/how to debug. Thanks
Runner class: multi-inherit for signal/slot connection to main window
class MyRunner : public QObject, public QRunnable
{
Q_OBJECT
Q_SIGNALS:
void feedbackLog(QString text);
void finished();
public:
explicit MyRunner(/* some args */) { /* some initialization */ }
void run() {
stopped_ = false;
for (int run = 0; run < SOME_COUNT; run++) {
Q_EMIT feedbackLog("Resetting everything ...");
if (stopped_) return;
/* start the daily interaction until the epidemic is over */
do
{
if (stopped_) return;
lengthySubTaskA();
if (stopped_) return;
lengthySubTaskB();
if (stopped_) return;
lengthySubTaskC();
}
while (conditionNotReached());
} // end: for(run)
stopped_ = true;
Q_EMIT finished();
}
bool isStopped() { return stopped_; }
public Q_SLOTS:
void stop() {
Q_EMIT feedbackLog("Cancel ...");
stopped_ = true;
}
private:
bool stopped_;
/** other class members follow */
};
MainWindow segment
void MainWindow::callMyRunnable() {
runner_ = new MyRunner(/* args */); // runner_ is class member */
runner_->setAutoDelete(true); // (a)
progress_dialog_ = new QProgressDialog("Running", "Cancel", 0, 0, this);
progress_dialog_->setAttribute(Qt::WA_DeleteOnClose);
connect(runner_, SIGNAL(feedbackLog(QString)), SLOT(logMessage(QString)));
connect(runner_, SIGNAL(finished()), SLOT(endLengthyJob()));
connect(runner_, SIGNAL(finished()), progress_dialog_, SLOT(close()));
connect(progress_dialog_, SIGNAL(canceled()), runner_, SLOT(stop()));
connect(progress_dialog_, SIGNAL(canceled()), SLOT(endLengthyJob()));
QThreadPool::globalInstance()->start(runner_);
progress_dialog_->show();
/* flu_runner_->deleteLater(); */ // (b)
}
void MainWindow::closeEvent(QCloseEvent *e) {
if (runner_ && !runner_->isStopped()) runner_->stop(); // (c)
if (QThreadPool::globalInstance()->activeThreadCount())
{
/* display a dialog to notify user about shutdown */
Dialog::WaitDialog dlg(5 * 1000);
dlg.exec();
}
event->accept();
} // end_event(MainWindow::closeEvent)
1) If I disable autodelete at (a), should I add statement (b) to prevent leaking? Or any better way to handle the leaking?
2) Sometimes, when the Runnable task completed, I cannot close the app normally (neither [x] button nor Alt-F4 work), and I have to kill the app from within QtCreator (I'm debugging, right?). What would be possible cause to this?
[Edit]: 3) For un-closable mainwindow, sometimes it happens after I cancelled the task handled by MyRunner class, will this be a possible cause?
[Edit]: I added qDebug() statement around (c), and found that it stop at (c) and refuse to proceed to display the waiting dialog in case if the [x] button is not responding.
Thanks.
The problem is solved by changing the 'close-strategy' of the mainwindow: remove 'auto-killing' and force user to wait for spawned thread. Variables runner_ and progress_dialog_ are now localized to the function callMyRunnable()
I created two threads, and use mutex to synchronize them.
In the mainwindow program(which i regard as the main thread) in which the other thread is created, I have to use mutex in at least two functions, because one is a slot to accept signals from UI when user selects a menu and configure the data, and there is also a timer which runs out 1 time per sec and triggers a slot function which reads the data.
My program often crashes even i use mutex. In 'main thread' there are different functions which have mutex's lock and unlock operations, one of the functions is a slot linked to the timer. Also the other thread continuously writes the data.
I am so confused, why ?
(:) I really need a better phone to edit my question before this time :) )
My code:
In thread:
class Background : public QThread
{
Q_OBJECT
public:
void Background::run(void)
{
initFile();
while(1)
{
Mutex->lock();
msleep(40);
rcv(); //writes map here
Mutex->unlock();
}
}
...
}
In thread's rcv():
void Background::rcv()
{
DEVMAP::iterator dev_r;
for(dev_r= DevMap.begin(); dev_r!= DevMap.end(); dev_r++)//DevMap is a refrence to the dev_map in mainwindow.
{
... ....//writes the map
}
}
In mainwindow:
void MainWindow::initTimer()
{
refreshTimer = new QTimer(this);
connect(refreshTimer, SIGNAL(timeout()), this, SLOT(refreshLogDisplay()));
refreshTimer->start(1000);
}
void MainWindow::refreshLogDisplay()
{
//MUTEX
mutex->lock();
......//read the map
//MUTEX
mutex->unlock();
}
In the thread's construction:
Background(DEVMap& map,...,QMutex* mutex):DevMap(map)...,Mutex(mutex){}
In mainwindow which creates the thread:
void MainWindow::initThread()
{
mutex = new QMutex;
back = new Background(dev_map,..., mutex);
back->start();
}
And:
void MainWindow::on_Create_triggered()//this function is a slot triggered by a menu item in the MainWindow UI
{
......//get information from a dialog
//MUTEX
mutex->lock();
BitState* bitState = new BitState(string((const char *)dlg->getName().toLocal8Bit()),
string((const char *)dlg->getNO().toLocal8Bit()),
dlg->getRevPortNo().toInt(), dlg->getSndPortNo().toInt());
dev_map.insert(DEVMAP::value_type (string((const char *)dlg->getPIN().toLocal8Bit()), *bitState));
//writes map here
//MUTEX
mutex->unlock();
}
You can use mutex in any thread. It was designed for this purposes. But you should not create dead locks, for instance if you do 'nested' calls of the 'lock'.
Good:
mutex->lock();
//code
mutex->unlock();
//code
mutex->lock();
//code
mutex->unlock();
Bad:
mutex->lock();
//code
mutex->lock(); //dead lock
//code
mutex->unlock();
//code
mutex->unlock();
Be accurate when using locks in functions:
void foo()
{
mutex->lock();
//code
mutex->unlock();
}
mutex->lock();
foo(); //dead lock
mutex->unlock()
Also you need to lock as less code as possible. Placing sleep() inside the lock is not
not a good idea as far other threads will wait while it's sleeping.
Not good:
while(1)
{
Mutex->lock();
msleep(40);
rcv();
Mutex->unlock();
}
Better:
while(1)
{
msleep(40);
Mutex->lock();
rcv();
Mutex->unlock();
}