I have a basic producer/consumer application where some workers go and perform a task on a shared problem. It's not trivial enough to use a concurrent for or something similar. I've gotten used to using Qt for threading because it makes it very easy to do things like this:
for(int i = 0; i < nworkers; i++){
Worker* myworker = new Worker();
QThread* thread = new QThread;
myworker->attachToThread(thread);
myworker->doSomething(someArgument);
myworker->doSomethingElse(someOtherArgument);
myworker->run();
}
I'd like to try and switch to using std::thread partially for the learning exercise and partially so that I don't need to link in Qt. So my question is, is there a straightforward way to spin off an object in a thread and then call functions from it? The model above works very well in this case. To be honest for my application I can get away with using std::async and passing everything as arguments to the worker function, but I was curious if there were a way to write the above code in terms of the standard library instead.
I guess I can do something like
void foo(someArguments){
Worker* myworker = new Worker();
myworker->doSomething(someArguments);
myworker->run();
}
and thread that, but then everything is encapsulated inside foo. With Qt I'd have access to that threaded object anywhere it's in scope.
I understand the basics of running a function in another thread with the standard library, but I wondered if what I want is something that's only possible with the help of a larger framework?
You want to run class method as a thread function?
std::thread t(&Worker::run, myworker, arg1, arg2, ...);
t.join();
or maybe make Worker method
Worker::runT() {
doSomething();
doMore();
std::thread t(&Worker::run, this);
}
you could put .join() somewhere, or in destructor
Related
I've found some evil bug in my user-level threads library.
My scheduler is actually a singleton class that initializes a signal timer this way:
sigAlarm_ is a member field of the scheduler, and its of type struct sigaction.
This is the related part of the scheduler initialization:
sigAlarm_.sa_handler = timerHandlerGlobal; // Assign the first field of sigAlarm (sa_handler) as needed, others zeroed
if (sigaction(SIGVTALRM, &sigAlarm_, nullptr) != 0) { uthreadSystemError("sigaction"); }
Now, this timerHandlerGlobal is a static function, and not a member function of the scheduler, as C++ doesn't permit passing function members this way.
Now, when I terminate the main thread of the library (which actually runs the scheduler), I'm invoking std::exit(1) which cleans the resources up.
When I'm running my tests with ASan (Address Sanitizer), in some executions, it gets into the timerHandlerGlobal while the scheduler is already nullptr!
Now, I've been already two days on that, inspecting what's the cause.
Now I see that if I'm adding this ugly condition, no problem appears with ASAN:
void timerHandlerGlobal(int signo)
{
if (scheduler_manager)
{
scheduler_manager->timerHandler(signo);
}
}
But, why is after std::exit(1) invoked by the scheduler, the sigaction.sa_handler (which is timerHandlerGlobal), is still running?
Please tell me you know why it is, I just want to omit this awful condition.
I am using multithreading in my QT program. I need to pass data to the worker object that lives in the worker thread from the main gui thread. I created a setData function in a QObject subclass to pass all the necessary data from the main gui thread. However I verified the function is called from the main thread by looking at QThread::currentThreadId() in the setData function. Even though the worker object function is called from the main thread does this ensure that the worker thread still has its own copy of the data as is required for a reentrant class? Keep in mind this is happening before the worker thread is started.
Also if basic data types are used in a class without dynamic memory and no static global variables is that class reentrant as long as all of its other member data is reentrant? (it's got reentrant data members like qstrings, qlists etc in addition the the basic ints bools etc)
Thanks for the help
Edited new content:
My main question was simply is it appropriate to call a QObject subclass method living in another thread from the main gui thread in order to pass my data to the worker thread to be worked on (in my case custom classes containing backup job information for long-pending file scans and copies for data backup). The data pass all happens before the thread is started so there's no danger of both threads modifying the data at once (I think but I'm no multithreading expert...) It sounds like the way to do this from your post is to use a signal from the main thread to a slot in the worker thread to pass the data. I have confirmed my data backup jobs are reentrant so all I need to do is assure that the worker thread works on its own instances of these classes. Also the transfer of data currently done by calling the QObject subclass method is done before the worker thread starts - does this prevent race conditions and is it safe?
Also here under the section "Accessing QObject Subclasses from Other Threads" it looks a little dangerous to use slots in the QObject subclass...
OK here's the code I've been busy recently...
Edited With Code:
void Replicator::advancedAllBackup()
{
updateStatus("<font color = \"green\">Starting All Advanced Backups</font>");
startBackup();
worker = new Worker;
worker->moveToThread(workerThread);
setupWorker(normal);
QList<BackupJob> jobList;
for (int backupCount = 0; backupCount < advancedJobs.size(); backupCount++)
jobList << advancedJobs[backupCount];
worker->setData(jobList);
workerThread->start();
}
The startBackup function sets some booleans and updates the gui.
the setupWorker function connects all signals and slots for the worker thread and worker object.
the setData function sets the worker job list data to that of the backend and is called before the thread starts so there is no concurrency.
Then we start the thread and it does its work.
And here's the worker code:
void setData(QList<BackupJob> jobs) { this->jobs = jobs; }
So my question is: is this safe?
There are some misconceptions in your question.
Reentrancy and multithreading are orthogonal concepts. Single-threaded code can be easily forced to cope with reentrancy - and is as soon as you reenter the event loop (thus you shouldn't).
The question you are asking, with correction, is thus: Are the class's methods thread-safe if the data members support multithreaded access? The answer is yes. But it's a mostly useless answer, because you're mistaken that the data types you use support such access. They most likely don't!
In fact, you're very unlikely to use multithread-safe data types unless you explicitly seek them out. POD types aren't, most of the C++ standard types aren't, most Qt types aren't either. Just so that there are no misunderstandings: a QString is not multithread-safe data type! The following code is has undefined behavior (it'll crash, burn and send an email to your spouse that appears to be from an illicit lover):
QString str{"Foo"};
for (int i = 0; i < 1000; ++i)
QtConcurrent::run([&]{ str.append("bar"); });
The follow up questions could be:
Are my data members supporting multithreaded access? I thought they did.
No, they aren't unless you show code that proves otherwise.
Do I even need to support multithreaded access?
Maybe. But it's much easier to avoid the need for it entirely.
The likely source of your confusion in relation to Qt types is their implicit sharing semantics. Thankfully, their relation to multithreading is rather simple to express:
Any instance of a Qt implicitly shared class can be accessed from any one thread at a given time. Corollary: you need one instance per thread. Copy your object, and use each copy in its own thread - that's perfectly safe. These instances may share data initially, and Qt will make sure that any copy-on-writes are done thread-safely for you.
Sidebar: If you use iterators or internal pointers to data on non-const instances, you must forcibly detach() the object before constructing the iterators/pointers. The problem with iterators is that they become invalidated when an object's data is detached, and detaching can happen in any thread where the instance is non-const - so at least one thread will end up with invalid iterators. I won't talk any more of this, the takeaway is that implicitly shared data types are tricky to implement and use safely. With C++11, there's no need for implicit sharing anymore: they were a workaround for the lack of move semantics in C++98.
What does it mean, then? It means this:
// Unsafe: str1 potentially accessed from two threads at once
QString str1{"foo"};
QtConcurrent::run([&]{ str1.apppend("bar"); });
str1.append("baz");
// Safe: each instance is accessed from one thread only
QString str1{"foo"};
QString str2{str1};
QtConcurrent::run([&]{ str1.apppend("bar"); });
str2.append("baz");
The original code can be fixed thus:
QString str{"Foo"};
for (int i = 0; i < 1000; ++i)
QtConcurrent::run([=]() mutable { str.append("bar"); });
This isn't to say that this code is very useful: the modified data is lost when the functor is destructed within the worker thread. But it serves to illustrate how to deal with Qt value types and multithreading. Here's why it works: copies of str are taken when each instance of the functor is constructed. This functor is then passed to a worker thread to execute, where its copy of the string is appended to. The copy initially shares data with the str instance in the originating thread, but QString will thread-safely duplicate the data. You could write out the functor explicitly to make it clear what happens:
QString str{"Foo"};
struct Functor {
QString str;
Functor(const QString & str) : str{str} {}
void operator()() {
str.append("bar");
}
};
for (int i = 0; i < 1000; ++i)
QtConcurrent::run(Functor(str));
How do we deal with passing data using Qt types in and out of a worker object? All communication with the object, when it is in the worker thread, must be done via signals/slots. Qt will automatically copy the data for us in a thread-safe manner so that each instance of a value is ever only accessed in one thread only. E.g.:
class ImageSource : public QObject {
QImage render() {
QImage image{...};
QPainter p{image};
...
return image;
}
public:
Q_SIGNAL newImage(const QImage & image);
void makeImage() {
QtConcurrent::run([this]{
emit newImage(render());
});
}
};
int main(int argc, char ** argv) {
QApplication app...;
ImageSource source;
QLabel label;
label.show();
connect(source, &ImageSource::newImage, &label, [&](const QImage & img){
label.setPixmap(QPixmap::fromImage(img));
});
source.makeImage();
return app.exec();
}
The connection between the source's signal and the label's thread context is automatic. The signal happens to be emitted in a worker thread in the default thread pool. At the time of signal emission, the source and target threads are compared, and if different, the functor will be wrapped in an event, the event posted the label, and the label's QObject::event will run the functor that sets the pixmap. This is all thread-safe and leverages Qt to make it almost effortless. The target thread context &label is critically important: without it, the functor would run in the worker thread, not the UI thread.
Note that we didn't even have to move the object to a worker thread: in fact, moving a QObject to a worker thread should be avoided unless the object does need to react to events and does more than merely generate a piece of data. You'd typically want to move e.g. objects that deal with communications, or complex application controllers that are abstracted from their UI. Mere generation of data can be usually done using QtConcurrent::run using a signal to abstract away the thread-safety magic of extracting the data from the worker thread to another thread.
In order to use Qt's mechanisms for passing data between threads with queues, you cannot call the object's function directly. You need to either use the signal/slot mechanism, or you can use the QMetaObject::invokeMethod call:
QMetaObject::invokeMethod(myObject, "mySlotFunction",
Qt::QueuedConnection,
Q_ARG(int, 42));
This will only work if both the sending and receiving objects have event queues running - i.e. a main or QThread based thread.
For the other part of your question, see the Qt docs section on reentrancy:
http://doc.qt.io/qt-4.8/threads-reentrancy.html#reentrant
Many Qt classes are reentrant, but they are not made thread-safe,
because making them thread-safe would incur the extra overhead of
repeatedly locking and unlocking a QMutex. For example, QString is
reentrant but not thread-safe. You can safely access different
instances of QString from multiple threads simultaneously, but you
can't safely access the same instance of QString from multiple threads
simultaneously (unless you protect the accesses yourself with a
QMutex).
I have the same situation like this: stop thread started by qtconcurrent::run
I need to close child thread (started with QtConcurrent::run) on closeEvent in QMainWindow.
But my function in child thread use code from *.dll: I can`t use loop because all that I do - is calling the external dll like
QFuture<void> = QtConcurrent::run(obj->useDllfunc_with_longTermJob());
And when I close the app with x-button my gui is closed, but second thread with_longTermJob() still worked and when is finished I have an error.
I know some decisions for this:
using other functions like map() or something else with
QFuture.cancel/stop functionality, not QtConcurrent::run().But I need only one function call. run() is what I need.
or use QThread instead Concurrent.But it`s not good for me.
What method more simple and better and how can I implement this? Is there a method that I don`t listed?
Could you provide small code sample for decision. Thx!
QtConcurrent::run isn't a problem here. You must have means of stopping the dllFuncWithLongTermJob. If you don't have such means, then the API you're using is broken, and you're out of luck. There's nothing you can do that'd be generally safe. Forcibly terminating a thread can leave the heap in an inconsistent state, etc. - if you need to terminate a thread, you need to immediately abort the application.
Hopefully, you can call something like stopLongTermJob that sets some flag that interrupts the dllFuncWithLongTermJob.
Then:
auto obj = new Worker;
auto objFuture = QtConcurrent::run([=]{obj->dllFuncWithLongTermJob();});
To interrupt:
obj->stopLongTermJob(); // must be thread-safe, sets a flag
objFuture.waitForFinished();
I have an issue with Microsoft Active Accessibility and threads in Qt. Code for example below:
CComPtr<IAccessible> _pAccMain;
HWND _hWnd = ...; // Handle of some window
HRESULT hr0 = ::AccessibleObjectFromWindow(_hWnd,
OBJID_CLIENT,
IID_IAccessible,
(void**)(&_pAccMain));
long childCount = 0;
HRESULT hr1 = _pAccMain->get_accChildCount(&childCount);
It works fine and in the main-thread these functions returns success for hr0 and hr1 and I got a proper success data for _pAccMain and childCount. But when I create a new thread and try to use the code in that new thread I got again success for hr0 and hr1, but I got different data in _pAccMain and childCount. I.e. the same code have more than one behavior in different threads.
Why another thread have another behavior in that example?
How can I fix it behavior?
Can I fix it?
There is no promise that AccessibleObjectFromWindow calls return the same interface pointer, so _pAccMain values don't have to be pointer-equal
Additionally, calling from different threads (apartments) might have marshaling involved, in which case you might get a proxy interface, not real object interface; it is behavior by design that those are different pointers
By mentioning different childCount you should have mentioned if worker thread get you zero, or otherwise what exactly is different in child enumeration
There is no free COM pointer passing between threads in COM, what your question suggests you are doing; you can only do this with MTA threads, and otherwise you have to marshal/unmarshal pointers to get a valid pointer in another thread
It seems like Qt quietly initialized COM in each new thread and after that CoInitializeEx with any COINIT can't do anything with it. But if you call in new QThread CoUnitialize before CoInitializeEx all will be ok, it is works for me.
I'm having a tough problem with invoking a native function using JNI from a thread.
The native function is legacy code that performs a computation-intensive task. Since I'd like not to freeze the rest of the program, the computation should be performed in a background thread. EventBus is used to send the calculation result back to the main program.
Basically it should be quite simple, something like this:
public class CalculationEngine {
private CalculationEngine(){}
public static void calculateInBackground(final Parameters parameters) {
new Thread(new Runnable() {
public void run() {
// Someone might change the parameters while our thread is running, so:
final Parameters clonedParameters = parameters.clone();
Results results = new Results();
natCalc(clonedParameters, results);
EventBus.publish("Results", results);
}
}).start();
}
public static void calculateNormally(final Parameters parameters) {
Results results = new Results();
natCalc(parameters, results);
EventBus.publish("Results", results);
}
private static native synchronized void
natCalc(Parameters parameters, Results results);
}
Now, the calculateNormally method, which blocks the main program, works fine, but the calculateInBackground method, which just constructs a background thread to do the same thing, causes various crashes in the native code when it's invoked consecutively. By consecutively I mean that it's called again only after the previous thread has finished and returned the result. Note that the native code is marked synchronized to ensure that only one instance of it can be running at a time.
My question is, how on earth can the native code behave differently depending on whether it's invoked from the main thread, or from some other thread? It's like the native code were keeping "state", and not really quitting, when it's called from within a thread other than the main thread. Is there a way to "clean" or "flush" a thread after it's finished? There must be something in JNI & Threads that I simply don't know.
Thanks for any hints!
I figured out a working solution, after googling and finding the phrase "I've found JNI to be very buggy when called from seperate threads... So make sure only one thread ever calls your native code!". It seems to be true; the solution is to keep a persistent, "reusable" thread around - I used Executors.newSingleThreadExecutor() - and to call the native code only from that thread. It works.
So the difference from JNI point of view was not between main thread vs. some other thread, but in using different threads in consecutive calls. Note that in the problematic code a new thread was constructed each time. It should work that way, but it doesn't. (And no, I'm not caching JNIEnv pointer.)
Whether it's a JNI bug, bug in the native code, something in the interaction between them and OS or whatever, would be interesting to know. But sometimes you just have no chance to debug 10000+ lines of existing code in detail, however, you're happy to get it to work. Here's working version of the example code, let's call this a workaround:
public class CalculationEngine {
private CalculationEngine(){}
private static Parameters parameters;
private static ExecutorService executor = Executors.newSingleThreadExecutor();
private static Runnable analysis = new Runnable() {
public synchronized void run() {
Results results = new Results();
natCalc(parameters, results);
EventBus.publish("Results", results);
}
};
public static synchronized void
calculateInBackground(final Parameters parameters) {
CalculationEngine.parameters = parameters.clone();
executor.submit(analysis);
}
private static native synchronized void
natCalc(Parameters parameters, Results results);
}
My advice on using JNI is DON'T if you can possibly avoid it. The chances are that it will cause stability issues for you. Here are some possible alternatives:
Recode the native library in Java.
Write a wrapper command for the native library in C / C++ / whatever and run it using java.lang.Process and friends
Turn the native library into a daemon and access it using Sockets.
While you've got an answer, I don't think too much has been provided as to possible root cause. Here's a few possibilities, but there are others. Note, these apply to Windows.
There's an apartment threaded COM object involved. Apartment threaded COM objects, which are the only type VB can create, can only be used on the thread that creates them.
Security features, like impersonation, are often thread isolated. If the initialization code modified the context of the thread, future calls that expect the context to be in place will fail.
Thread specific memory storage is a technique within some applications to support multi-threadedness (Java also has such a feature).
Here there is a good documentation about it: Section 8.1 JNI and Threads.
http://java.sun.com/docs/books/jni/download/jni.pdf