Is there anything wrong with creating a window in a separate thread, which will also contain the message loop, then creating an OpenGL Context in another thread?
You should be able to get it to work, if you're careful. See the parallel opengl faq.
Q: Why does my OpenGL application crash/not work when
I am rendering from another thread?
A: The OpenGL context is thread-specific. You have to
make it current in the thread using glXMakeCurrent,
wglMakeCurrent or aglSetCurrentContext, depending on
your operating system.
What you want to do is perfectly possible. Even better, OpenGL contexts can migrate between threads and even be used with multiple windows as long as their pixel format is compatible. The one constraint is, that a OpenGL context can be bound in only one thread at a time and that only a unbound context can be bound.
So you could even create the window and the context in one thread, then unbind the context, create another thread and re-bind the context to the window in the secondary thread. No problem there.
The only thing you must be aware of is, that OpenGL itself doesn't like to be multithreaded. The API itself is more or less thread safe, as only one context can be bound to a thread at a time. But all the bookkeeping required if OpenGL operations spawn over several threads may trigger nasty driver bugs and also has a certain performance hit.
Related
The SDL documentation for threading states:
NOTE: You should not expect to be able to create a window, render, or receive events on any thread other than the main one.
The glfw documentation for glfwCreateWindow states:
Thread safety: This function must only be called from the main thread.
I have read about issues regarding the glut library from people who have tried to run the windowing functions on a second thread.
I could go on with these examples, but I think you get the point I'm trying to make. A lot of cross-platform libraries don't allow you to create a window on a background thread.
Now, two of the libraries I mentioned are designed with OpenGL in mind, and I get that OpenGL is not designed for multithreading and you shouldn't do rendering on multiple threads. That's fine. The thing that I don't understand is why the rendering thread (the single thread that does all the rendering) has to be the main one of the application.
As far as I know, neither Windows nor Linux nor MacOS impose any restrictions on which threads can create windows. I do know that windows have affinity to the thread that creates them (only that thread can receive input for them, etc.); but still that thread does not need to be the main one.
So, I have three questions:
Why do these libraries impose such restrictions? Is it because there is some obscure operating system that mandates that all windows be created on the main thread, and so all operating systems have to pay the price? (Or did I get it wrong?)
Why do we have this imposition that you should not do UI on a background thread? What do threads have to do with windowing, anyways? Is it not a bad abstraction to tie your logic to a specific thread?
If this is what we have and can't get rid of it, how do I overcome this limitation? Do I make a ThreadManager class and yield the main thread to it so it can schedule what needs to be done in the main thread and what can be done in a background thread?
It would be amazing if someone could shed some light on this topic. All the advice I see thrown around is to just do input and UI both on the main thread. But that's just an arbitrary restriction if there isn't a technical reason why it isn't possible to do otherwise.
PS: Please note that I am looking for a cross platform solution. If it can't be found, I'll stick to doing UI on the main thread.
While I'm not quite up to date on the latest releases of MacOS/iOS, as of 2020 Apple UIKit and AppKit were not thread safe. Only one thread can safely change UI objects, and unless you go to a lot of trouble that's going to be the main thread. Even if you do go to all the trouble of closing the window manager connection etc etc you're still going to end up with one thread only doing UI. So the limitation still applies on at least one major system.
While it's possibly unsafe to directly modify the contents of a window from any other thread, you can do software rendering to an offscreen bitmap image from any thread you like, taking as long as you like. Then hand the finished image over to the main thread for rendering. (The possibly is why cross platform toolkits disallow/tell you not to. Sometimes it might work, but you can't say why, or even that it will keep working.)
With Vulkan and DirectX 12 (and I think but am not sure Metal) you can render from multiple threads. Woohoo! Of course now you have to figure out how to do all the coordination and locking and cross-synching without making the whole thing slower than single threaded, but at least you have the option to try.
Adding to the excellent answer by Matt, with Qt programs you can use invokeMethod and postEvent to have background threads update the UI safely.
It's highly unlikely that any of these frameworks actually care about which thread is the 'main thread', i.e., the one that called the entry point to your code. The real restriction is that you have to do all your UI work on the thread that initialized the framework, i.e., the one that called SDL_Init in your case. You will usually do this in your main thread. Why not?
Multithreaded code is difficult to write and difficult to understand, and in UI work, introducing multithreading makes it difficult to reason about when things happen. A UI is a very stateful thing, and when you're writing UI code, you usually need to have a very good idea about what has happened already and what will happen next -- those things are often undefined when multithreading is involved. Also, users are slow, so multithreading the UI is not really necessary for performance in normal cases. Because of all this, making a UI framework thread-safe isn't usually considered beneficial. (multithreading compute-intensive parts of your rendering pipeline is a different thing)
Single-threaded UI frameworks have a dispatcher of some sort that you can use to enqueue activities that should happen on the main thread when it next has time. In SDL, you use SDL_PushEvent for this. You can call that from any thread.
I have several QQuickFramebufferObjects between which I want to share some GL objects (shaders and VBOs mainly). My initial plan was:
Create a class SharedGLData to hold the shared object
Instantiate this class on the stack in C++'s main()
Pass a pointer the object to QML via ctx->assignRootProperty or something
Pass the object as a property to the QML items of type SharedGLData
Access the pointer from C++
But that means I'd be creating GL objects on the main thread and later access them on the render thread. I'm pretty sure that's forbidden, for example see here where it says:
QOpenGLContext can be moved to a different thread with moveToThread(). Do not call makeCurrent() from a different thread than the one to which the QOpenGLContext object belongs.
Is it ok to follow my initial plan or is there a way to create shared GL objects on the render thread?
A possible hack would be to put the shared stuff into a singleton that gets initialized on first use, and make my first use be directly from the rendering code. But that's a hack.
Another idea is to call moveToThread on the QQFBO's GL context to move it to the main thread, instantiate SharedGLData, then move the GL context back to the render thread. But I don't have a pointer to the render thread...
Clarification after the answer I got: By "render thread" I mean the thread that Qt SceneGraph silently creates to do all the rendering. It's not a thread that I'm creating!
If you want multiple threads sharing OpenGL objects,
Create a QOpenGLContext. Usually, the first one you create should be the one belonging to the window (which will actually draw to the screen).
Create a second QOpenGLContext, but call setShareContext before calling create.
You now have two OpenGL contexts which share objects (shaders, VBOs, etc) and you can now use these contexts simultaneously from different threads.
But... this often not an ideal experience. In many cases, using OpenGL simultaneously from different threads will be no faster than using it from one thread, or it will be slower, or it will be buggier. You are at the mercy of the OpenGL implementation.
A Different Way
It sounds like your goal is to load assets (shaders, textures, vertex data) in a background thread while your main thread continues to render. There is a more straightforward way of doing this that does not involve creating multiple contexts at all.
Simply map OpenGL buffers into memory in the rendering thread, and then pass the pointer to the background loader thread. The loader thread is free to write data into the buffer while the render thread continues to make OpenGL calls. When the loader thread is done, it signals the main thread, which does the appropriate synchronization and calls glTexImage2D or whatever. These days, you can even keep a single buffer persistently mapped, but the traditional two-buffer method also works quite well.
Under this scheme, your rendering thread does not have to do any IO, and your background thread does not have to make any OpenGL calls at all.
You can't use this to compile shaders in the background but c'est la vie.
OpenGL functions are only supposed to be called from the thread in which the OpenGL context is current. Does this limit apply to updating data using glMapBuffer/glMapBufferRange, i.e. can I map (a region of) a buffer and then read from / write to that region in another thread? Assuming, of course, that the mapping (and unmapping) functions are called from the rendering thread.
Before answering the main question, let's cover some misinformation:
I know that you're supposed to only call OpenGL functions in the thread that created the OpenGL context.
This is not true. You must call OpenGL functions only on the thread where the context is current. You can make an OpenGL context current in a different thread (which will automatically make it not current in the previous one. OpenGL contexts can only be current in one thread at the same time). And you can create multiple contexts which share objects. Each such context can be current in a different thread.
Now to the issue. Yes, you are perfectly free to use the mapped pointer however you wish from another thread. Though, as you said, you must use appropriate synchronization mechanisms to let the original thread know that you've finished.
In Apple's documentation, I read this:
1 — "Shared contexts share all texture objects, display lists, vertex programs, fragment programs, and buffer objects created before and after sharing is initiated."
2 — "Contexts that are on different threads can share object resources. For example, it is acceptable for one context in one thread to modify a texture, and a second context in a second thread to modify the same texture. The shared object handling provided by the Apple APIs automatically protects against thread errors."
So I expected to be able to create my buffer objects once, then use them to render simultaneously on multiple contexts. However if I do that, I get crashes on my NVIDIA GeForce GT 650M with backtraces like this:
Crashed Thread: 10 Dispatch queue: com.apple.root.default-qos
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: EXC_I386_GPFLT
…
Thread 10 Crashed:: Dispatch queue: com.apple.root.default-qos
0 GLEngine 0x00007fff924111d7 gleLookupHashObject + 51
1 GLEngine 0x00007fff925019a9 gleBindBufferObject + 52
2 GLEngine 0x00007fff9243c035 glBindBuffer_Exec + 127
I've posted my complete code at https://gist.github.com/jlstrecker/9df10ef177c2a49bae3e. At the top, there's #define SHARE_BUFFERS — when commented out it works just fine, but uncommented it crashes.
I'm not looking to debate whether I should be using OpenGL 2.1 — it's a requirement of other software I'm interfacing with. Nor am I looking to debate whether I should use GLUT — my example code just uses that since it's included on Mac and doesn't have any external dependencies. Nor am I looking for feedback on performance/optimization.
I'd just like to know if I can expect to be able to simultaneously render from a single shared buffer object on multiple contexts — and if so, why my code is crashing.
We also ran into the 'gleLookupHashObject' crash and made a small repro-case (very similar to yours) which was posted in an 'incident' to Apple support. After investigation, an Apple DTS engineer came back with the following info, quoting:
"It came to my attention that glFlush() is being called on both the main thread and also a secondary thread that binds position data. This would indeed introduce issues and, while subtle, actually does indicate that the constraints we place on threads and GL contexts aren’t being fully respected.
At this point it behoves you to either further investigate your implementation to ensure that such situations are avoided or, better yet, extend your implementation with explicit synchronization mechanisms (such as what we offer with GCD). "
So if you run into this crash you will need to do explicit synchronization on the application side (pending a fix on the driver-side).
Summary of relevant snippets related to "OpenGL, Contexts and Threading" from the official Apple Documentation:
[0] Section: "Use Multiple OpenGL Contexts"
If your application has multiple scenes that can be rendered in parallel, you can use a context for each scene you need to render. Create one context for each scene and assign each context to an operation or task. Because each task has its own context, all can submit rendering commands in parallel.
https://developer.apple.com/library/mac/documentation/GraphicsImaging/Conceptual/OpenGL-MacProgGuide/opengl_threading/opengl_threading.html#//apple_ref/doc/uid/TP40001987-CH409-SW6
[1] Section: Guidelines for Threading OpenGL Applications
(a) Use only one thread per context. OpenGL commands for a specific context are not thread safe. You should never have more than one thread accessing a single context simultaneously.
(b) Contexts that are on different threads can share object resources. For example, it is acceptable for one context in one thread to modify a texture, and a second context in a second thread to modify the same texture. The shared object handling provided by the Apple APIs automatically protects against thread errors. And, your application is following the "one thread per context" guideline.
https://developer.apple.com/library/mac/documentation/GraphicsImaging/Conceptual/OpenGL-MacProgGuide/opengl_threading/opengl_threading.html
[2] OpenGL Restricts Each Context to a Single Thread
Each thread in an OS X process has a single current OpenGL rendering context. Every time your application calls an OpenGL function, OpenGL implicitly looks up the context associated with the current thread and modifies the state or objects associated with that context.
OpenGL is not reentrant. If you modify the same context from multiple threads simultaneously, the results are unpredictable. Your application might crash or it might render improperly. If for some reason you decide to set more than one thread to target the same context, then you must synchronize threads by placing a mutex around all OpenGL calls to the context, such as gl* and CGL*. OpenGL commands that blockâsuch as fence commandsâdo not synchronize threads.
https://developer.apple.com/library/mac/documentation/GraphicsImaging/Conceptual/OpenGL-MacProgGuide/opengl_threading/opengl_threading.html
suppose I use the QGLWidget's paintGL() method to draw into the widget using OpenGL. After the Qt called the paintGL() method, it automatically triggers a buffer swap. In OpenGL, this buffer swap usually blocks the calling thread until the frame rendering to the background buffer is completed, right? I wonder which Qt thread calls the paintGL as well as the buffer swap. Is it the main Qt UI thread? If it is, wouldn't that mean that the block during the buffer swap also blocks the whole UI? I could not find any information about this process in general..
Thanks
I don't use the QGLWidget very often, but consider that yes, if swapBuffers() is synchronous the Qt GUI thread is stuck. This means that during that operation you'll be unable to process events.
Anyway, if you're experiencing difficulties while doing this, consider reading this article which manage to allow multithreaded OpenGL to overcome this difficulty.
Even better, this article explains well the situation and introduces the new multithreading OpenGL capabilities in Qt 4.8, which is now in release candidate.
In OpenGL, this buffer swap usually blocks the calling thread until the frame rendering to the background buffer is completed, right?
It depends on how it is implemented. Which means that it varies from hardware to hardware and driver to driver.
If it is, wouldn't that mean that the block during the buffer swap also blocks the whole UI?
Even if it does block, it will only do so for 1/60th of a second. Maybe 1/30th if your game is slowing down. If you're really slow, 1/15th. The at most one keypress or mouse action that the user gives will still be in the message queue.
The issue with blocking isn't about the UI. It will be responsive enough for the user to not notice. But if you have strict timings (such as you might for a game), I would suggest avoiding paintGL at all. You should be rendering when you want to, not when Qt tells you to.