I am learning how to use PyQt5 and there are quite a few points that elude me.
I have started implementing QThreads to replace the threads I have been using in my UI as I realised that mixing threads and QThreads could possibly lead to issues later and have started using pyqtSignal simultaneously
So far I have seen that the pyqtSygnal needs to be implemented on a class level to be able to work ( putting it in a class constructor does not work )
In the architecture I use currently, I have a pyqtSignal that is instantiated in the main thread and is then used by all of the child threads. This is due to having one class that is responsible for my logs of all the program.
I am unsure if this is a good implementation or not.
Here are my questions :
are pyqtSignals thread safe ?I know that they use a queue system to be thread safe but is the emit() method itself thread safe ? My understanding of them is a bit limited
Do I need to protect my pyqtSignals with locks ?
Can I have multiple different signals emitting to the same slot without having any issues ?
The signals are one of the few elements of the QObjects that are thread-safe so it is not necessary to protect the data. So you can have different types of signals connected to the same slot.
Since the signals are thread-safe then Qt recommended to communicate QObjects that live in different threads.
Related
The SDL documentation for threading states:
NOTE: You should not expect to be able to create a window, render, or receive events on any thread other than the main one.
The glfw documentation for glfwCreateWindow states:
Thread safety: This function must only be called from the main thread.
I have read about issues regarding the glut library from people who have tried to run the windowing functions on a second thread.
I could go on with these examples, but I think you get the point I'm trying to make. A lot of cross-platform libraries don't allow you to create a window on a background thread.
Now, two of the libraries I mentioned are designed with OpenGL in mind, and I get that OpenGL is not designed for multithreading and you shouldn't do rendering on multiple threads. That's fine. The thing that I don't understand is why the rendering thread (the single thread that does all the rendering) has to be the main one of the application.
As far as I know, neither Windows nor Linux nor MacOS impose any restrictions on which threads can create windows. I do know that windows have affinity to the thread that creates them (only that thread can receive input for them, etc.); but still that thread does not need to be the main one.
So, I have three questions:
Why do these libraries impose such restrictions? Is it because there is some obscure operating system that mandates that all windows be created on the main thread, and so all operating systems have to pay the price? (Or did I get it wrong?)
Why do we have this imposition that you should not do UI on a background thread? What do threads have to do with windowing, anyways? Is it not a bad abstraction to tie your logic to a specific thread?
If this is what we have and can't get rid of it, how do I overcome this limitation? Do I make a ThreadManager class and yield the main thread to it so it can schedule what needs to be done in the main thread and what can be done in a background thread?
It would be amazing if someone could shed some light on this topic. All the advice I see thrown around is to just do input and UI both on the main thread. But that's just an arbitrary restriction if there isn't a technical reason why it isn't possible to do otherwise.
PS: Please note that I am looking for a cross platform solution. If it can't be found, I'll stick to doing UI on the main thread.
While I'm not quite up to date on the latest releases of MacOS/iOS, as of 2020 Apple UIKit and AppKit were not thread safe. Only one thread can safely change UI objects, and unless you go to a lot of trouble that's going to be the main thread. Even if you do go to all the trouble of closing the window manager connection etc etc you're still going to end up with one thread only doing UI. So the limitation still applies on at least one major system.
While it's possibly unsafe to directly modify the contents of a window from any other thread, you can do software rendering to an offscreen bitmap image from any thread you like, taking as long as you like. Then hand the finished image over to the main thread for rendering. (The possibly is why cross platform toolkits disallow/tell you not to. Sometimes it might work, but you can't say why, or even that it will keep working.)
With Vulkan and DirectX 12 (and I think but am not sure Metal) you can render from multiple threads. Woohoo! Of course now you have to figure out how to do all the coordination and locking and cross-synching without making the whole thing slower than single threaded, but at least you have the option to try.
Adding to the excellent answer by Matt, with Qt programs you can use invokeMethod and postEvent to have background threads update the UI safely.
It's highly unlikely that any of these frameworks actually care about which thread is the 'main thread', i.e., the one that called the entry point to your code. The real restriction is that you have to do all your UI work on the thread that initialized the framework, i.e., the one that called SDL_Init in your case. You will usually do this in your main thread. Why not?
Multithreaded code is difficult to write and difficult to understand, and in UI work, introducing multithreading makes it difficult to reason about when things happen. A UI is a very stateful thing, and when you're writing UI code, you usually need to have a very good idea about what has happened already and what will happen next -- those things are often undefined when multithreading is involved. Also, users are slow, so multithreading the UI is not really necessary for performance in normal cases. Because of all this, making a UI framework thread-safe isn't usually considered beneficial. (multithreading compute-intensive parts of your rendering pipeline is a different thing)
Single-threaded UI frameworks have a dispatcher of some sort that you can use to enqueue activities that should happen on the main thread when it next has time. In SDL, you use SDL_PushEvent for this. You can call that from any thread.
I read that mutex is a semaphore with value 1 (binary semaphore) used to enforce mutual exclusion.
I read this link
Semaphore vs. Monitors - what's the difference?
which says that monitor helps in achieving mutual exclusion.
Can someone tell me the difference between mutex and monitor as both help achieve the same thing (Mutual Exclusion)?
Since you haven't specified which OS or language/library you are talking about, let me answer in a generic way.
Conceptually they are the same. But usually they are implemented slightly differently
Monitor
Usually, the implementation of monitors is faster/light-weight, since it is designed for multi-threaded synchronization within the same process. Also, usually, it is provided by a framework/library itself (as opposed to requesting the OS).
Mutex
Usually, mutexes are provided by the OS kernel and libraries/frameworks simply provide an interface to invoke it. This makes them heavy-weight/slower, but they work across threads on different processes. OS might also provide features to access the mutex by name for easy sharing between instances of separate executables (as opposed to using a handle that can be used by fork only).
Monitor are different then Mutex but they can be considered similar in a sense that Monitor are build on top of Mutex. See depiction of monitor in an image at the bottom, for clarity.
Monitor is a synchronization construct that allows threads to have both mutual exclusion (using locks) and cooperation i.e. the ability to make threads wait for certain condition to be true (using wait-set).
In other words, along with data that implements a lock, every Java object is logically associated with data that implements a wait-set. Whereas locks help threads to work independently on shared data without interfering with one another, wait-sets help threads to cooperate with one another to work together towards a common goal e.g. all waiting threads will be moved to this wait-set and all will be notified once lock is released. This wait-set helps in building monitors with additional help of lock (mutex).
I you want, you can see my answer here, which may or may not be relevant to this question.
You can find another relevant discussion here
Semaphore vs. Monitors - what's the difference?
Unfortunately the textbook definitions does not always correspond to how different platforms and languages use the terms. So to get precise answers you have to specify the platform and context. But in general:
A mutex is a lock which can only be owned by a single thread at a time. The lock doesn't in itself protect anything, but code can check for ownership of a mutex to ensure that some section of code is only executed by a single thread at a time. If a thread wants to acquire a mutex lock the thread is blocked until it becomes available.
In Java terminology a monitor is a mutex lock which is implicitly associated with an object. When the synchronized keyword is applied to classes or methods an implicit mutex lock is created around the code, which ensures that only one thread at a time can execute it. This is called a monitor lock or just a monitor.
So in Java a monitor is not a specific object, rather any object has a monitor lock available which is invoked with the synchronized keyword.
The synchronized keyword can also be used on a block of code, in which case the object to lock on is explicit specified. Here it gets a bit weird because you can use the monitor of one object to lock access to another object.
In computer science textbooks you may meet a different kind of monitor, the Brinch-Hansen or Hoare-monitor, which is a class or module which is implicitly thread-safe (like a synchronized class in Java) and which have multiple conditions threads can wait/signal on. This is a higher-level concept than the Java monitor.
C#/.NET has monitors similar to Java, but also have a Mutex class in the standard library - which is different from the mutex lock used in the monitor. The monitor lock only exist inside a single process, while the Mutex-lock is machine wide. So a monitor lock is appropriate for making objects and data-structures thread safe, but not for providing system-wide exclusive access to say a file or device.
So bottom line: These terms can mean different things, so if you want a more specific answer you should specify a specific platform.
I wonder whether I could use QEventLoop (QProcess?) to parallelize multiple calls to same function with Qt. What is precisely the difference with QtConcurrent or QThread? What is a process and an event loop more precisely? I read that QCoreApplication must exec() as early as possible in main() method, so that I wonder why it is different from main Thread.
could you point as some efficient reference to processes and thread with Qt? I came through the official doc and those things remain unclear.
Thanks and regards.
Process and thread are not Qt-specific concepts. You can search for "process vs. thread" anywhere for that distinction to be explained. For instance: What resources are shared between threads?
Though related concepts, spawning a new process is a more "heavyweight" form of parallelism than spawning a new thread within your existing process. Processes are protected from each other by default, while threads of execution within a process can read and write each other's memory directly. The protection you get from spawning processes comes at a greater run-time cost...and since independent processes can't read each other's memory, you have to share data between them using methods of inter-process communication.
Odds are that you want threads, because they're simpler to use in a case where one is writing all the code in a program. Given all the complexities in multithreaded programming, I'd suggest looking at a good book or reading some websites to start with. See: What are some good resources for learning threaded programming?
But if you want to dive in and just get a feel for how threading in Qt looks, you can spend time looking at the examples:
http://qt-project.org/doc/qt-4.8/examples-threadandconcurrent.html
QtConcurrent is an abstraction library that makes it easier to implement some kinds of parallel programming patterns. It's built on top of the QThread abstractions, and there's nothing it can do that you couldn't code yourself by writing to QThread directly. But it might make your code easier to write and less prone to errors.
As for an event loop...that is merely a generic term for how any given thread of execution in your program waits for work items to process, processes them, and can decide when it is no longer needed. If a thread's job were merely to start up, do some math, and exit...then it wouldn't need an event loop. But starting and stopping a thread takes time and churns resources. So typically threads live for longer periods of time, and have an event loop that knows how to wait for events it needs to respond to.
If you build on top of QtConcurrent, you won't have to worry about an event loop in your worker threads because they are managed automatically in a thread pool. The word count example is pretty simple to see:
http://qt-project.org/doc/qt-4.8/qtconcurrent-wordcount-main-cpp.html
How I can determine which thread is waiting for more time?
My requirement is, in a synchronized methods, when one thread finishes its work, I want to allow the thread which is waiting for the longest time. I hope my question make sense.
All depends on which language and/or environment you are using. So far as I know there's no intrinsic support for this in Java, if multiple threads are waiting to enter a synchronized method then the system will pick an arbitrary one to run when entry is possible.
If instead you use Java's wait() / notify() then you control which threads are notified and so can build your own priority mechanism, for example you could have a simple queue to which each thread adds itself before its wait() then you just take the top item from the queue and notify that thread.
You should not and almost certainly do not need to do this.
The threading environment will schedule threads for you.
If the software design is such that this appears to be a problem, then the design is incorrect for a pre-emptive threading environment.
What you may want to be doing is something more like managing and prioritizing units of work, where you for example service work in the order that it arrives.
In other words, the order of work processing should not in your design depend on which thread runs, but rather, on your design of how work is handed out to threads.
#djna Java doesn't let you choose which thread to notify. If 10 threads are in the queue any one of them can be notified.
This can be done by using the lock/condition interfaces in concurrent package.
Here you can associate each of these threads with a condition and then take out an item from that queue and signal the condition that is mapped with that thread/task.
I'm trying to write a library to separate all the disk activity out into its own thread, but the documentation doesn't really care about such things.
What I want to accomplish is that aside from startup, all disk activity is asynchronous, and for that, I need to wrap every class that accesses the disk. Here's what I found so far:
QtCore:
QFile
QTemporaryFile
QDir
QFileInfo
QFileSystemWatcher
QDirIterator
QSettings
QtGui:
QFileDialog
QFileSystemModel
QDirModel
I'm sure there are more.
I have a couple of points -
First, when you do this, remember that all GUI objects are based on QWidget, have run in the start-up thread. See http://doc.trolltech.com/4.6/threads-qobject.html which talks about threading. The quote is "Although QObject is reentrant, the GUI classes, notably QWidget and all its subclasses, are not reentrant. They can only be used from the main thread. As noted earlier, QCoreApplication::exec() must also be called from that thread".
This also means that if you need to display information from one of these wrapper classes on the screen, you need to be careful about ownership of objects when you pass information back to the GUI thread. Particularly, anything that is based on QObject.
Second, starting threads carries a run-time cost. So I would suggest that you structure your design to minimize the number of times this wrapper thread class is created and destroyed.
Overall an interesting approach to files. This is one that I'm going to consider for my current application. It may solve some problems I'm having.