I have code that I am trying to guard using boost locking mechanism. The problem is that RecomputeStuff can not only be called from RemoveStuff, but it can also be called from another thread. My question is, using these boost locking mechanisms, what is the correct way to fix RecomputeStuff? The way it is now it deadlocks.
#include <boost/thread.hpp>
boost::shared_mutex values_mutex;
int globaldata;
class A
{
public:
void RecomputeStuff();
void RemoveStuff();
private:
std::vector<std::string> data;
};
//Note, RecomputeStuff just reads from std::vector<std::string> data, but it also writes to other global stuff that RemoveStuff also writes to.
void A::RecomputeStuff()
{
boost::upgrade_lock<boost::shared_mutex> lock(values_mutex);
boost::upgrade_to_unique_lock<boost::shared_mutex> unique_lock(lock);
// this function reads std::vector<std::string> data
// but also modifies `globaldata` that RemoveStuff also modifies.
}
void A::RemoveStuff()
{
boost::upgrade_lock<boost::shared_mutex> lock(values_mutex);
boost::upgrade_to_unique_lock<boost::shared_mutex> unique_lock(lock);
//here, remove stuff from std::vector<std::string> data
//...then call RecomputeStuff
RecomputeStuff();
// modify `globaldata`
}
A solution is to move the not locked code of the A::RecomputeStuff method to a separate one and call it from the A::RemoveStuff and A::RecomputeStuff. See the code below
boost::shared_mutex values_mutex;
int globaldata;
class A
{
private:
void RecomputeStuffUnsafe();
public:
void RecomputeStuff();
void RemoveStuff();
private:
std::vector<std::string> data;
};
void A::RecomputeStuffUnsafe()
{
// this function reads std::vector<std::string> data
// but also modifies `globaldata` that RemoveStuff also modifies.
}
//Note, RecomputeStuff just reads from std::vector<std::string> data, but it also writes to other global stuff that RemoveStuff also writes to.
void A::RecomputeStuff()
{
boost::upgrade_lock<boost::shared_mutex> lock(values_mutex);
boost::upgrade_to_unique_lock<boost::shared_mutex> unique_lock(lock);
RecomputeStuffUnsafe();
}
void A::RemoveStuff()
{
boost::upgrade_lock<boost::shared_mutex> lock(values_mutex);
boost::upgrade_to_unique_lock<boost::shared_mutex> unique_lock(lock);
//here, remove stuff from std::vector<std::string> data
//...then call RecomputeStuff
RecomputeStuffUnsafe();
// modify `globaldata`
}
Edit #00:
Also upgrade_lock has a constructor which accepts the try_to_lock_t tag. It looks like what you are asking for.
Related
I'm currently trying to make a software that downloads a lot of files from Google Drive. Downloading is currently not a problem.
Nevertheless, I encounter an issue when launching 500+ simultaneous downloads. I use a slightly modified version of this tutorial : https://wiki.qt.io/Download_Data_from_URL.
Here is the .h file :
class FileDownloader : public QObject
{
Q_OBJECT
public:
explicit FileDownloader(QUrl url, QObject *parent = 0, int number = 0);
QByteArray downloadedData() const;
void launchNewDownload(QUrl url);
QByteArray m_DownloadedData;
QNetworkReply* reply;
static QNetworkAccessManager *m_WebCtrl;
signals:
void downloaded();
private slots:
void fileDownloaded(QNetworkReply* pReply);
};
And here is the .cpp file :
QNetworkAccessManager* FileDownloader::m_WebCtrl = nullptr;
FileDownloader::FileDownloader(QUrl url, QObject *parent) :
QObject(parent)
{
if (m_WebCtrl == nullptr) {
m_WebCtrl = new QNetworkAccessManager(this);
}
connect(m_WebCtrl, SIGNAL (finished(QNetworkReply*)),this, SLOT (fileDownloaded(QNetworkReply*)));
launchNewDownload(url);
}
void FileDownloader::launchNewDownload(QUrl url) {
QNetworkRequest request(url);
this->reply = m_WebCtrl->get(request);
}
void FileDownloader::fileDownloaded(QNetworkReply* pReply) {
m_DownloadedData = pReply->readAll();
//emit a signal
pReply->deleteLater();
emit downloaded();
}
QByteArray FileDownloader::downloadedData() const {
return m_DownloadedData;
}
The issue is "QThread::start: Failed to create thread ()" when reaching about the 500th download. I tried to limit the number of downloads which run at the same time - but I always get the same issue. Besides, I tried to delete every downloader when finishing its task - it did nothing else than crashing the program ;)
I think that it is coming from the number of threads allowed for an only process, but I'm not able to solve it !
Does anyone have an idea that could help me ?
Thank you !
QNetworkAccessManager::finished signal is documented to be emitted whenever a pending network reply is finished.
This means that if the QNetworkAccessManager is used to run multiple requests at a time (and this is perfectly normal usage). finished signal will be emitted once for every request. Since you have a shared instance of QNetworkAccessManager between your FileDownloader objects, the finished signal gets emitted for every get call you have made. So, all the FileDownloader objects get a finished signal as soon as the first FileDownloader finishes downloading.
Instead of using QNetworkAccessManager::finished, you can use QNetworkReply::finished to avoid mixing up signals. Here is an example implementation:
#include <QtNetwork>
#include <QtWidgets>
class FileDownloader : public QObject
{
Q_OBJECT
//using constructor injection instead of a static QNetworkAccessManager pointer
//This allows to share the same QNetworkAccessManager
//object with other classes utilizing network access
QNetworkAccessManager* m_nam;
QNetworkReply* m_reply;
QByteArray m_downloadedData;
public:
explicit FileDownloader(QUrl imageUrl, QNetworkAccessManager* nam,
QObject* parent= nullptr)
:QObject(parent), m_nam(nam)
{
QNetworkRequest request(imageUrl);
m_reply = m_nam->get(request);
connect(m_reply, &QNetworkReply::finished, this, &FileDownloader::fileDownloaded);
}
~FileDownloader() = default;
QByteArray downloadedData()const{return m_downloadedData;}
signals:
void downloaded();
private slots:
void fileDownloaded(){
m_downloadedData= m_reply->readAll();
m_reply->deleteLater();
emit downloaded();
}
};
//sample usage
int main(int argc, char* argv[]){
QApplication a(argc, argv);
QNetworkAccessManager nam;
FileDownloader fileDownloader(QUrl("http://i.imgur.com/Rt8fqpt.png"), &nam);
QLabel label;
label.setAlignment(Qt::AlignCenter);
label.setText("Downloading. . .");
label.setMinimumSize(640, 480);
label.show();
QObject::connect(&fileDownloader, &FileDownloader::downloaded, [&]{
QPixmap pixmap;
pixmap.loadFromData(fileDownloader.downloadedData());
label.setPixmap(pixmap);
});
return a.exec();
}
#include "main.moc"
If you are using this method to download large files, consider having a look at this question.
One solution could be to uses a QThreadPool. You simply feed it tasks (QRunnable) and it will handle the number of threads and the task queue for you.
However in your case this is not perfect because you will be limiting the number of simultaneous downloads to the number of threads created by QThreadPool (generally the number of CPU core you have).
To overcome this you will have to handle the QThread yourself and not use QThreadPool. You would create a small number of thread (see QThread::idealThreadCount()) and run multiple FileDownloader on each QThread.
I have one thread, producer and class consumer. Consumer is registered in QML, and producer is connected with signals to consumer. Producer send data to consumer, so consumer can update the model and GUI.
Code looks something like this:
Main function:
int main(int argc, char ** argv)
{
QGuiApplication app(argc, argv);
Producer producer;
Consumer consumer;
/* Connect signals between producer and consumer */
...
...
QQuickView view;
/* Set root context */
QQmlContext *ctxt = view.rootContext();
producer.start();
ctxt->setContextProperty("consumer", &consumer);
/*Connect signals between consumer and QML*/
...
view.show();
return app.exec();
}
Producer:
class Producer : public QThread
{
Q_OBJECT
protected:
void run()
{
while(true) {
if (someFlag == true)
{
// do some work
// emit signal with data to consumer
}
}
}
signals:
// Signals for sending data
};
The question is how to properly stop thread Producer when exit in application is pressed?
Instead of while(true) check for a "should end" condition, e.g. isInterruptionRequested().
Then, in main(), before returning, you tell the thread to stop, e.g. with QThread::requestInterruption and then wait on the thread
view.show();
const int ret = app.exec();
producer.requestInterruption();
producer.wait();
return ret;
I have a suggestion:
My way of doing is, Register the windows close event and do any sort of cleaning activities in it.
pseudo code below. try to fit it in your application.
Close filter event:
class CloseEventFilter : public QObject
{
Q_OBJECT
public:
CloseEventFilter(QObject *parent) : QObject(parent) {}
protected:
bool eventFilter(QObject *obj, QEvent *event)
{
if (event->type() == QEvent::Close)
{
// Destroy your thread here.
producer->exit();//Make your thread available
//... may be global....producer is derived from QThread.
}
return QObject::eventFilter(obj, event)
}
}
};
Registering to windows
//Registering window
QGuiApplication *app = new QGuiApplication(argc,argv);
QWindow* qtwindow = new QWindow();
CloseEventFilter *closeFilter = new CloseEventFilter(window);
qtwindow >installEventFilter(closeFilter);
Why thread exit funtion (from documentation)
void QThread::exit(int returnCode = 0)
Tells the thread's event loop to exit with a return code.
After calling this function, the thread leaves the event loop and returns from the call to QEventLoop::exec(). The QEventLoop::exec() function returns returnCode.
By convention, a returnCode of 0 means success, any non-zero value indicates an error.
Note that unlike the C library function of the same name, this function does return to the caller -- it is event processing that stops.
No QEventLoops will be started anymore in this thread until QThread::exec() has been called again. If the eventloop in QThread::exec() is not running then the next call to QThread::exec() will also return immediately.
See also quit() and QEventLoop.
The while (true) if (someFlag) loop will peg the CPU at 100% when there's no work to be done. You don't want that!
Alas, there's no need to make the Producer be a QThread. It can be a plain old QObject. There are different ways of doing it.
You can use a zero-duration timer to keep the event loop active in the thread:
class Producer : public QObject
{
Q_OBJECT
QBasicTimer m_timer;
void timerEvent(QTimerEvent * ev) override {
if (ev->timerId() == m_timer.timerId())
doWorkChunk();
}
void doWorkChunk() {
...
emit hasData(...);
}
public:
Producer(QObject * parent = nullptr) : QObject{parent} {
connect(this, &Producer::start, this, [this]{ m_timer.start(this, 0); });
connect(this, &Producer::stop, this, [this]{ m_timer.stop(); });
}
Q_SIGNAL void start();
Q_SIGNAL void stop();
Q_SIGNAL void hasData(const DataType &);
};
Instead of setting/clearing the flag, call start() and stop(). Then you can move the producer to any thread and it will work just fine, doing the right thing on exit (wrapping things up cleanly):
class Thread : public QThread {
using QThread::run;
public:
~Thread() { quit(); wait(); }
};
int main(int argc, char ** argv) {
QCoreApplication app{argc, argv};
Producer producer;
Thread thread; // must come after producer above, the order has meaning!
producer.moveToThread(&thread);
thread.start();
return app.exec();
}
You can use the Qt Concurrent framework, too:
class Producer : public QObject {
{
Q_OBJECT
volatile bool m_active = false;
void doWorkChunk() {
...
emit hasData();
}
void doWork() {
while (m_active) doWorkChunk();
}
public:
using QObject::QObject;
~Producer() { stop(); }
void start() {
m_active = true;
QtConcurrent::run(this, &Producer::doWork);
}
void stop() {
m_active = false;
}
Q_SIGNAL void hasData(const DataType &);
};
I am using c++ builder XE8, and I am just a beginner. I don't know anything about Delphi. To listen for incoming data on server and client both end, I inserted Thread Object from Flie->New->Other->C++ builder Files (as I saw in a video demo of threading), and named it TSocketThreard.
On the client end I assign the client socket to a TIdIOHandlerSocket var (fsock) and used the following code to listen continuously for imcoming data.
len is int, s is String var and lbox is a TListBox.
void __fastcall TSocketThread::Execute()
{
FreeOnTerminate = True;
while(1){
if (Form1->fsock != NULL && Form1->fsock->Connected()) {
if(Form1->fsock->CheckForDataOnSource(10))
Form1->len = Form1->fsock->ReadLongInt();
Form1->s = Form1->fsock->ReadString(Form1->len);
Form1->lbox->Items->Add(Form1->s);
}
}
}
I tried to start it from client's onConnect event and from a button(I use to connect the client to server) also. But both times its got freeze.
On the server end I tried to OnExecute event on the same way. I mean, at first assigned socket to a var, put the code like above on a thread's(which I inserted like before) execute method, and started the thread's start() method from onExecute event.For the server I want to make it work on asynchronous mode (to handle at least few hundreds clients).
I can feel I am doing it the wrong way, but its really hard to find a good example in c++. Please some one show me the right way.
Indy servers are already multi-threaded, you don't need to create your own reading threads. The TIdTCPServer::OnExecute event runs in a thread, and each client has its own thread. Just use the socket that the event gives you, don't use your own variable.
void __fastcall TMyServerForm::IdTCPServer1Execute(TIdContext *AContext)
{
// use AContext->Connection->IOHandler as needed...
}
On the client side, your reading thread is doing things that are not thread-safe, which can lead to UI freezing, amongst other problems. Try something more like this instead:
class TSocketThread : public TThread
{
private:
TIdIOHandler *fIOHandler;
String fString;
void __fastcall UpdateListBox();
protected:
virtual void __fastcall Execute();
public
__fastcall TSocketThread(TIdIOHandler *AIOHandler);
};
__fastcall TSocketThread::TSocketThread(TIdIOHandler *AIOHandler)
: TThread(false), fIOHandler(AIOHandler)
{
}
void __fastcall TSocketThread::Execute()
{
while (!Terminated)
{
int len = fIOHandler->ReadLongInt();
fString = fIOHandler->ReadString(len);
Synchronize(&UpdateListBox);
}
}
void __fastcall TSocketThread::UpdateListBox()
{
Form1->lbox->Items->Add(fString);
}
TSocketThread *Thread = NULL;
void __fastcall TMyClientForm::IdTCPClient1Connected(TObject *Sender)
{
Thread = new TSocketThread(IdTCPClient1->IOHandler);
}
void __fastcall TMyClientForm::IdTCPClient1Disconnected(TObject *Sender)
{
if (!Thread) return;
Thread->Terminate();
if (!IsCurrentThread(Thread))
{
Thread->WaitFor();
delete Thread;
Thread = NULL;
}
}
I can not get the QSoundEffect to play in a separate thread. Could you please tell me why is the sound played only by the first code snippet and not by the second?
//main.cpp
#include <QCoreApplication>
#include "SoundThread.h"
int main(int argc, char *argv[])
{
QCoreApplication a(argc, argv);
// #1
QSoundEffect alarmSound;
alarmSound.setSource(QUrl::fromLocalFile(":/sound"));
alarmSound.play();
/* #2
SoundThread thread;
thread.start();
thread.wait();
*/
return a.exec();
}
and
//SoundThread.h
#ifndef SOUNDTHREAD_H
#define SOUNDTHREAD_H
#include <QThread>
#include <QtMultimedia/QSoundEffect>
class SoundThread : public QThread
{
Q_OBJECT
private:
void run()
{
QSoundEffect alarmSound;
alarmSound.setSource(QUrl::fromLocalFile(":/sound"));
alarmSound.play();
while(true){}
}
};
#endif // SOUNDTHREAD_H
From the Qt documentation on QThread: -
By default, run() starts the event loop by calling exec()
Since you've inherited from QThread, you now have a run function which doesn't call exec(). Therefore, the event loop is not running and is most likely required for playing the sound effect.
Calling exec() should be substituted for the while(true){} as exec() will wait until exit() is called.
Doing it properly, with moving an object to the thread, based on "How to Really Truly Use QThreads..."
class Worker : public QObject
{
Q_OBJECT
public:
Worker();
~Worker();
public slots:
void PlaySoundEffect();
signals:
void finished();
void error(QString err);
private:
// store the sound effect, so we can reuse it multiple times
QSoundEffect* m_pAlarmSound;
private slots:
};
Worker::Worker()
{
m_pAlarmSound = new QSoundEffect;
m_pAlarmSound.setSource(QUrl::fromLocalFile(":/sound"));
}
Worker::~Worker()
{
delete m_pAlarmSound;
m_pAlarmSound = nullptr; // C++ 11
}
void Worker::PlaySoundEffect()
{
m_pAlarmSound->play();
}
// setup the worker and move it to another thread...
MainWindow::MainWindow
{
QThread* thread = new QThread;
Worker* worker = new Worker();
worker->moveToThread(thread);
connect(worker, SIGNAL(error(QString)), this, SLOT(errorString(QString)));
connect(thread, SIGNAL(started()), worker, SLOT(PlaySoundEffect()));
connect(worker, SIGNAL(finished()), thread, SLOT(quit()));
connect(worker, SIGNAL(finished()), worker, SLOT(deleteLater()));
connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
thread->start();
// We can also connect a signal of an object in the main thread to the PlaySoundEffect slot
// Assuming MainWindow has declared a signal void Alert();
connect(this, &MainWindow::Alert, worker, &Worker::PlaySoundEffect);
// Then play the sound when we want: -
emit Alert();
}
While this seems like a lot of effort, there are many advantages of doing it this way. If, for example, you have a lot of sound effects, the method of inheriting QThread means that you're creating a thread per sound effect, which isn't ideal.
We could easily extend the above Worker object to hold a list of sound effects and play the one we want, by passing an enum into the PlaySoundEffect slot. As this thread is constantly running, playing sounds will incur less delay; it takes time and resources to create a thread at run-time.
You enter to an infinite loop at the end of your run function which causes to block the thread and consequently the QSoundEffect not working. It should be like:
void run()
{
QSoundEffect alarmSound;
alarmSound.setSource(QUrl::fromLocalFile(":/sound"));
alarmSound.play();
exec();
}
Replace while(true){} by QThread::run();, which will launch an internal event loop and will wait (sleep) for events, such as timer events and asynchronous signals calling slots, which is probably what is happening internally in QSoundEffect: When you call QSoundEffect::play() Some event (probably signals/slots) are queued in the event queue from within QThread, but nothing is processing the event queue. Remember: you are overriding virtual void run() and the original implementation was calling QThread::exec() for you. It is always a good idea to always call your super classes virtual function whenever you override them, as long as they are not pure virtual.
void run()
{
QSoundEffect alarmSound;
alarmSound.setSource(QUrl::fromLocalFile(":/sound"));
alarmSound.play();
QThread::run();
}
Some people suggested that calling QThread::exec() would do the trick. They may be right. I have to check the implementation of QThread to confirm that it is the only thing called in QThread::run() implementation. I personally think (by experience) that it is always safer to call your superclasse's virtual function in case there are other things called (other than QThread::exec() for this particular case).
Another option would be to move your QSoundEffect instance onto the thread and use signals and slots default auto-connection behaviour type to switch threads.
SoundPlayer.h:
#ifndef SOUNDPLAYER_H_
#define SOUNDPLAYER_H_
#include <QObject>
#include <QThread>
#include <QSoundEffect>
class SoundPlayer : public QObject
{
Q_OBJECT
public:
SoundPlayer();
signals:
void play();
private:
QThread m_thread;
QSoundEffect m_alarmSound;
};
#endif
SoundPlayer.cpp :
#include "SoundPlayer.h"
SoundPlayer()
{
m_alarmSound.setSource(QUrl::fromLocalFile(":/sound"));
m_alarmSound.moveToThread(&m_thread);
connect(this, SIGNAL(play()), &m_alarmSound, SLOT(play()));
m_thread.start(); // QThread::exec() will be called for you, making the thread wait for events
}
And then calling the play() signal would start playing in the correct thread.
SoundPlayer player;
emit player.play(); // m_alarmSound.play() will be called by m_thread
I am using thread wrapper which checks if function which updates VCL (which also has some arguments) was called from main thread or not and then executes in within the context of Main thread always.
It works but I want to make it simpler. The problem is that I have to repeat this code in every function which needs VCL synchronization which is prone to errors. Is there a way to make this wrapper simpler and more re-useable? Note that this particular wrapper only uses one parameter but there can be any number of parameters which are copied to TLocalArgs and passed on.
Current code:
boost::scoped_ptr<TIdThreadComponent> WorkerThread;
...
void TForm1::SetMemoMessage(UnicodeString Msg)
{
// Check which thread called function, main thread or worker thread
if (GetCurrentThreadId() != System::MainThreadID)
{
struct TLocalArgs
{
TForm1 *Form;
UnicodeString Msg;
void __fastcall SetMemoMessage() // Same name as main function to make it easier to maintain
{
// We are in main thread now, safe to call message update directly
Form->SetMemoMessage(Msg);
}
};
// We are in worker thread, wrap into Synchronize
TLocalArgs Args = { this, Msg };
WorkerThread->Synchronize(&Args.SetMemoMessage);
return;
}
// MAIN THREAD CODE is very simple compared to wrapper above
Memo1->Text = Msg;
}
TThread::Synchronize() checks MainThreadID internally for you and calls the specified procedure directly if Synchronize() is called from the main thread. So just call Synchronize() unconditionally and let it handle the details. Synchronize() also has overloaded static versions available so you don't even need a TThread pointer to call it.
Try this:
void TForm1::SetMemoMessage(UnicodeString Msg)
{
struct TLocalArgs
{
UnicodeString Msg;
void __fastcall SetMemoMessage()
{
Form1->Memo1->Text = Msg;
}
};
TLocalArgs Args;
Args.Msg = Msg;
TThread::Synchronize(NULL, &Args.SetMemoMessage);
}