suppose I use the QGLWidget's paintGL() method to draw into the widget using OpenGL. After the Qt called the paintGL() method, it automatically triggers a buffer swap. In OpenGL, this buffer swap usually blocks the calling thread until the frame rendering to the background buffer is completed, right? I wonder which Qt thread calls the paintGL as well as the buffer swap. Is it the main Qt UI thread? If it is, wouldn't that mean that the block during the buffer swap also blocks the whole UI? I could not find any information about this process in general..
Thanks
I don't use the QGLWidget very often, but consider that yes, if swapBuffers() is synchronous the Qt GUI thread is stuck. This means that during that operation you'll be unable to process events.
Anyway, if you're experiencing difficulties while doing this, consider reading this article which manage to allow multithreaded OpenGL to overcome this difficulty.
Even better, this article explains well the situation and introduces the new multithreading OpenGL capabilities in Qt 4.8, which is now in release candidate.
In OpenGL, this buffer swap usually blocks the calling thread until the frame rendering to the background buffer is completed, right?
It depends on how it is implemented. Which means that it varies from hardware to hardware and driver to driver.
If it is, wouldn't that mean that the block during the buffer swap also blocks the whole UI?
Even if it does block, it will only do so for 1/60th of a second. Maybe 1/30th if your game is slowing down. If you're really slow, 1/15th. The at most one keypress or mouse action that the user gives will still be in the message queue.
The issue with blocking isn't about the UI. It will be responsive enough for the user to not notice. But if you have strict timings (such as you might for a game), I would suggest avoiding paintGL at all. You should be rendering when you want to, not when Qt tells you to.
Related
The SDL documentation for threading states:
NOTE: You should not expect to be able to create a window, render, or receive events on any thread other than the main one.
The glfw documentation for glfwCreateWindow states:
Thread safety: This function must only be called from the main thread.
I have read about issues regarding the glut library from people who have tried to run the windowing functions on a second thread.
I could go on with these examples, but I think you get the point I'm trying to make. A lot of cross-platform libraries don't allow you to create a window on a background thread.
Now, two of the libraries I mentioned are designed with OpenGL in mind, and I get that OpenGL is not designed for multithreading and you shouldn't do rendering on multiple threads. That's fine. The thing that I don't understand is why the rendering thread (the single thread that does all the rendering) has to be the main one of the application.
As far as I know, neither Windows nor Linux nor MacOS impose any restrictions on which threads can create windows. I do know that windows have affinity to the thread that creates them (only that thread can receive input for them, etc.); but still that thread does not need to be the main one.
So, I have three questions:
Why do these libraries impose such restrictions? Is it because there is some obscure operating system that mandates that all windows be created on the main thread, and so all operating systems have to pay the price? (Or did I get it wrong?)
Why do we have this imposition that you should not do UI on a background thread? What do threads have to do with windowing, anyways? Is it not a bad abstraction to tie your logic to a specific thread?
If this is what we have and can't get rid of it, how do I overcome this limitation? Do I make a ThreadManager class and yield the main thread to it so it can schedule what needs to be done in the main thread and what can be done in a background thread?
It would be amazing if someone could shed some light on this topic. All the advice I see thrown around is to just do input and UI both on the main thread. But that's just an arbitrary restriction if there isn't a technical reason why it isn't possible to do otherwise.
PS: Please note that I am looking for a cross platform solution. If it can't be found, I'll stick to doing UI on the main thread.
While I'm not quite up to date on the latest releases of MacOS/iOS, as of 2020 Apple UIKit and AppKit were not thread safe. Only one thread can safely change UI objects, and unless you go to a lot of trouble that's going to be the main thread. Even if you do go to all the trouble of closing the window manager connection etc etc you're still going to end up with one thread only doing UI. So the limitation still applies on at least one major system.
While it's possibly unsafe to directly modify the contents of a window from any other thread, you can do software rendering to an offscreen bitmap image from any thread you like, taking as long as you like. Then hand the finished image over to the main thread for rendering. (The possibly is why cross platform toolkits disallow/tell you not to. Sometimes it might work, but you can't say why, or even that it will keep working.)
With Vulkan and DirectX 12 (and I think but am not sure Metal) you can render from multiple threads. Woohoo! Of course now you have to figure out how to do all the coordination and locking and cross-synching without making the whole thing slower than single threaded, but at least you have the option to try.
Adding to the excellent answer by Matt, with Qt programs you can use invokeMethod and postEvent to have background threads update the UI safely.
It's highly unlikely that any of these frameworks actually care about which thread is the 'main thread', i.e., the one that called the entry point to your code. The real restriction is that you have to do all your UI work on the thread that initialized the framework, i.e., the one that called SDL_Init in your case. You will usually do this in your main thread. Why not?
Multithreaded code is difficult to write and difficult to understand, and in UI work, introducing multithreading makes it difficult to reason about when things happen. A UI is a very stateful thing, and when you're writing UI code, you usually need to have a very good idea about what has happened already and what will happen next -- those things are often undefined when multithreading is involved. Also, users are slow, so multithreading the UI is not really necessary for performance in normal cases. Because of all this, making a UI framework thread-safe isn't usually considered beneficial. (multithreading compute-intensive parts of your rendering pipeline is a different thing)
Single-threaded UI frameworks have a dispatcher of some sort that you can use to enqueue activities that should happen on the main thread when it next has time. In SDL, you use SDL_PushEvent for this. You can call that from any thread.
Is there anything wrong with creating a window in a separate thread, which will also contain the message loop, then creating an OpenGL Context in another thread?
You should be able to get it to work, if you're careful. See the parallel opengl faq.
Q: Why does my OpenGL application crash/not work when
I am rendering from another thread?
A: The OpenGL context is thread-specific. You have to
make it current in the thread using glXMakeCurrent,
wglMakeCurrent or aglSetCurrentContext, depending on
your operating system.
What you want to do is perfectly possible. Even better, OpenGL contexts can migrate between threads and even be used with multiple windows as long as their pixel format is compatible. The one constraint is, that a OpenGL context can be bound in only one thread at a time and that only a unbound context can be bound.
So you could even create the window and the context in one thread, then unbind the context, create another thread and re-bind the context to the window in the secondary thread. No problem there.
The only thing you must be aware of is, that OpenGL itself doesn't like to be multithreaded. The API itself is more or less thread safe, as only one context can be bound to a thread at a time. But all the bookkeeping required if OpenGL operations spawn over several threads may trigger nasty driver bugs and also has a certain performance hit.
I have a Delphi 6 application that receives and processes an image stream from an external camera. I have the code on a background thread since it is CPU heavy and I don't want it interfering with with the user interface code that runs on the main thread. I want to update a rectangular area on a form or frame with the TBitmaps I create from the camera's JPEG frames that are received at a rate of 25 frames per second.
I want to know what method will give me the best performance and what Windows API calls or Delphi calls to use to do it. I would guess I should not use a TImage or TPicture or similar VCL component because they run on the main thread and I'm pretty sure trying to get anything done via a Synchronize() call is going to be inefficient and has the potential to slow down the threads involved. I would also want a technique that provides a smooth video display like double buffered controls do without any "striping" effects. Also, any tips on proper Canvas locking or device context management, etc. would be appreciated, especially tips on avoiding common mistakes in freeing resources.
Of course, a link to a good code sample that does what I need would be great.
AFAIK TBitmap are thread-safe, if you work only on its canvas. Synchronize is needed if you send GDI messages and need to refresh the screen, but from my experiment, using TBitmap.Canvas is just a wrapper around thread-safe Windows API. If you process the bitmap with pixel arithmetics (using e.g. Scanline), one unique bitmap per thread, you can do it on background.
But I suspect using TBitmap is not the most efficient way. Give a try to http://graphics32.org or http://aggpas.org which are very fast way to work on bitmaps.
If you can, as imajoosy proposed, the best way to process your input stream is to use direct X streaming process abilities.
For thread-safe process, if each thread is about to consume 100% of its core (which is very likely for image process), it is generally assumed that you shall better create NumberOfCPU-1 threads for your processing. For instance, you could create a pool of threads, then let those consume the bitmaps from the input stream.
I am trying to come up with a synchronization model for the following scenario:
I have a GUI thread that is responsible for CPU intensive animation + blocking I/O. The GUI thread retrieves images from the network (puts them in a shared buffer) , these images are processed (CPU intensive operation..done by a worker thread) and then these images are animated ( again CPU intensive..done by the GUI thread).
The processing of images is done by a worker thread..it retrieves images from the shared buffer processes them and puts them in an output buffer.
There is only once CPU and the GUI thread should not get scheduled out while it is animating the images (the animation has to be really smooth). This means that the work thread should get the CPU only when the GUI thread is waiting for I/O operation to complete.
How do i go about achieving this? This looks like a classic producer consumer problem...but i am not quite sure how i can guarantee that the animation will be as smooth as possible ( i am open to using more threads).
I would like to use QThreads (Qt framework) for platform independence but i can consider pthreads for more control ( as currently we are only aiming for linux).
Any ideas?
EDIT:
i guess the problems boils down to one thing..how do i ensure that the animation thread is not interrupted while it is animating the images ( the animation runs when the user goes from one page to the other..all the images in the new page are animated before shown in their proper place..this is a small operation but it must be really smooth).The worker thread can only run when the animation is over..
Just thinking out loud here, but it sounds like you have two compute-intensive tasks, animation and processing, and you want animation to always have priority over processing. If that is correct then maybe instead of having these tasks in separate threads you could have a single thread that handles both animation and processing.
For instance, the thread could have two task-queues, one for animation jobs and one for processing jobs, and it only starts a job from the processing queue when the animation queue is empty. But, this will only work well if each individual processing job is relatively small and/or interruptible at arbitrary positions (otherwise animation jobs will get delayed, which is not what you want).
The first big question is: Do I really need threads? Qt 's event system and network objects make it easy to not having the technical burden of threads and all the snags that comes with it.
Have a look at alternative ways to address issues here and here. These techniques great if you are sticking to pure Qt code and do not depend on a 3rd party library. If you must use a 3rd party lib that does blocking calls then sure, you can use threads.
Here is an example of a consumer producer.
Also have a look at Advanced Qt Programming: Creating Great Software with C++ and Qt 4
My advice is to start without threads and see how it fares. You can always refactor to threads after. So, best is to design your objects/architecture without too much coupling.
If you want you can post some code to give more context.
I don't want multiple windows, each with its own UI thread, nor events raised on a single UI thread, not background workers and notifications, none of that Invoke, BeginInvoke stuff either.
I'm interested in a platform that allows multiple threads to update the same window in a safe manner. Something like first thread creates three buttons, the second thread another five, and they both can access them,change their properties and delete them without any unwanted consequences.
I want safe multi-threaded access to the UI without Invoking, a platform where the UI objects can be accessed directly from any thread without raising errors like "The object can only be accessed from the thread that created it". To let me do the synchronizing if I have to, not prevent me from cross-tread accessing the UI in a direct manner.
I'm gonna get down voted but ... Go Go Gadget Soapbox.
Multi threaded GUI are not possible in the general case. It has been attempted time and time again and it never comes out well. It is not a coincidence that all of the major windowing frameworks follow the single threaded ui model. They weren't copying each other, it's just that the constraints of the problem lead them to the same answer. Many people smarter than you or i have tried to solve this.
It might be possible to implement a multi-thread ui for a particular project. I'm only saying that it can't be done in the general case. That means it's unlikely you'll find a framework to do what you want.
The gist of the problem is this. Envision the gui components as a chain (in reality it's more like a tree, but a chain is simple to describe). The button connects to the frame, connects to the box, connects to the window. There are two source of events for a gui the system/OS and the user. The system/OS event originate at the bottom of the chain (the windowing system), the user event originate at the top of the chain (the button). Both of these events must move through the gui chain. If two threads are pushing these events simultaneously they must be mutex protected. However, there is no known algorithm for concurrently traversing a double linked list in both directions. It is prone to dead lock. GUI experts tried and tried to figure out ways to get around the deadlocking problem, and eventually arrived at the solution we use today called Model/View/Controller, aka one thread runs the UI.
You could make a thread-safe Producer/Consumer queue of delegates.
Any thread that wants to update a UI component would create a delegate encapsulating the operations to be performed, and add it to the queue.
The UI thread (assuming all components were created on the same thread) would then periodically pull an item from the queue, and execute the delegate.
I don't believe a platform like that exists per se
There is nothing stopping you from saying taking .Net and creating all new controls which are thread safe and can work like that(or maybe just the subset of what you need) which shouldn't be an extremely large job(though definitely no small job) because you can just derive from the base controls and override any thread-unsafe methods or properties.
The real question though is why? It would definitely be slower because of all the locking. Say your in one thread that is doing something with the UI, well it has to lock the window it's working on else it could be changed without it knowing by the other thread. So with all the locking, you will spend most of your drawing time and such waiting on locks and (expensive) context switches from threads. You could maybe make it async, but that just doesn't seem safe(and probably isn't) because controls that you supposedly just created may or may not exist and would be about like
Panel p=new Panel();
Button b=new Button();
WaitForControlsCreated(); //waits until the current control queue is cleared
p.Controls.Add(b);
which is probably just as slow..
So the real question here is why? The only "good" way of doing it is just having an invoke abstracted away so that it appears you can add controls from a non-UI thread.
I think you are misunderstanding how threads really work and what it takes to actually make an object thread safe
Accept that any code updating the GUI has to be on the GUI thread.
Learn to use BeginInvoke().
On Windows, Window handles have thread affinity. This is a limitation of the Window manager. It's a bad idea to have multiple threads accessing the same window on Windows.
I'm surprised to see these answers.
Only the higher level language frameworks like C# have thread restrictions on GUI elements.
Windows, at the SDK layer, is 100% application controlled and there are no restrictions on threads except at insignificant nitty gritty level. For example if multiple threads want to write to a window, you need to lock on a mutex, get the device context, draw, then release the context, then unlock the mutex. Getting and releasing a device context for a moment of drawing needs to be on the same thread... but those are typically within 10 lines of code from each other.
There isn't even a dedicated thread that windows messages come down on, whatever thread calls "DispatchMessage()" is the thread the WINPROC will be called on.
Another minor thread restriction is that you can only "PeekMessage" or "GetMessage" a window that was created on the current thread. But really this is very minor, and how many message pumps do you need anyway.
Drawing is completely disconnected from threads in Windows, just mutex your DC's for drawing. You can draw anytime, from anywhere, not just on a WM_PAINT message.
BeOS / Haiku OS
Based on my guessing of your requirement, you want a single Windows Form and having ways to execute certain routines asynchronously (like multi-threading), yes?
Typically (for the case of .NET WinForms) Control.Invoke / Control.BeginInvoke is used to a certain effect what I think you want.
Here's an interesting article which might help: http://www.yoda.arachsys.com/csharp/threads/winforms.shtml