Cuda has cuCtxPopCurrent() and cuCtxPushCurrent() for popping cuda context from the pthread that created it and push it another pthread that wants to use it.
By default, that CUDA context can be accessed only from the CPU thread that created it. If you want to access the CUDA context from other threads, you have to call cuCtxPopCurrent() to pop it from the thread that created it. The context then can be pushed onto any other CPU thread's current context stack, and subsequent CUDA calls would reference that context.
I could not find something similar for openCL.
OpenCL contexts and command queues have no thread association. There is no such thing as 'current context' for a thread. You have to explicitly supply the context / queue handle to each API call.
Also, There is no special issue in accessing an OpenCL context or command queue from multiple threads. They are thread safe.
Related
If
CPU has mode (privilege level) (Added because not all processors have privilege levels according to here)
CPU is multi-core (CISC / x86-64 instruction set)
scheduling is round robin scheduling
thread is kernel managed thread
OS is windows if necessary
I want to know simplified core execution flow in the point of thread context switch and CPU mode switch per time slice.
My understanding is as follows. Please correct me if I'm wrong.
In case of the thread is kernel managed user mode thread not involving interrupt, or anything that requires kernel mode,
The thread context switch occurs.
The core executing thread switches to kernel mode because context switch can only occur in kernel mode according to here, here and here unless the thread is user managed thread.
The core executing thread switches to user mode.
The core executes sequence of instructions located in user space.
Time slice expires.
Repeat 1.
Closest related diagram I could find is below.
Even a little clue to answer will be sincerely appreciated.
You said it yourself:
context switch can only occur in kernel mode
So the CPU must enter kernel mode before there can be a context switch. That can happen in either one of two ways in most operating systems:
The user-mode code makes a system call, or
An interrupt occurs.
If the thread enters kernel mode by making a system call, then there could be a context switch if the syscall causes the thread to no longer be runnable (e.g., a sleep() call), or there could be a context switch if the syscall causes some higher-priority thread to become runnable. (e.g., the syscall releases a mutex that the higher priority thread was awaiting.)
If the thread enters kernel mode because of an interrupt, then there could be a context switch because the interrupt handler made some higher-priority thread runnable (e.g., if the other thread was awaiting data from the disk), or there could be a context switch because it was a timer interrupt, and the current thread's time slice has expired.
The mechanism of context switching may be different on different hardware platforms. Here's how it could happen on some hypothetical CPU:
The current thread (threadA) enters sheduler code which chooses some other thread (threadB) as the next to run on the current CPU.
It calls some switchContext(threadB) function.
The switchContext function copies values from the stack pointer register, and from other live registers into the current thread (threadA)'s saved context area.*
It then sets the "current thread" pointer to point to threadB's saved context area, and it restores threadB's context by copying all the same things in reverse.**
Finally, the switchContext function returns... IN threadB,... at exactly the place where threadB last called it.
Eventually, threadB returns from the interrupt or system call to application code running in user-mode.
* The author of switchContext may have to be careful, may have to do some tricky things, in order to save the entire context without trashing it. E.g., it had better not use any register that needs saving before it has actually saved it somewhere.
** The trickiest part is when restoring the stack pointer register. As soon as that happens, "the" stack suddenly is threadB's stack instead of threadA's stack.
So two questions here really. First, (and yes, I have searched this already, but wanted clarification), what is the difference between a user thread and a kernel thread? Is it simply that one is generated by a user program and the other by an OS, with the latter having access to privileged instructions? Are they conceptually the same or are there actual differences in the threads themselves?
Second, and the real problem of my question is: the book I am using says that "a relationship must exist between user threads and kernel threads," going on to list the different models of such a relationship. But the book fails to clearly explain why a user thread must always be mapped to a specific kernel thread. Why is this?
A kernel thread is a thread object maintained by the operating system. It is an actual thread that is capable of being scheduled and executed by the processor. Typically, kernel threads are heavyweight objects with permissions settings, priorities, etc. The kernel thread scheduler is in charge of scheduling kernel threads.
User programs can make their own thread schedulers too. They can make their own "threads" and simulate context-switches to switch between them. However, these threads aren't kernel threads. Each user thread can't actually run on its own, and the only way for a user thread to run is if a kernel thread is actually told to execute the code contained in a user thread. That said, user threads have major advantages over kernel threads. They can be a lot more lightweight, since they don't necessarily need to have their own priorities, can be managed by a single process (which might have better info about what threads need to run when), and don't create lots of kernel objects for purposes of security and locking.
The reason that user threads have to be associated with kernel threads is that by itself a user thread is just a bunch of data in a user program. Kernel threads are the real threads in the system, so for a user thread to make progress the user program has to have its scheduler take a user thread and then run it on a kernel thread. The mapping between user threads and kernel threads doesn't have to be one-to-one (1 : 1); you can have multiple user threads share the same kernel thread (only one of those user threads runs at a time), and you can have a single user thread which is rotated across different kernel threads in a 1 : n mapping.
I think a real world example will clear the confusion, so let’s see how things are done in Linux.
First of all Linux doesn’t differentiate between process and thread, entity that can be scheduled is called task in Linux and represented by task_struct. So whenever you execute a fork() system call, a new task_struct is created which holds data (or pointer) associated with new task.
So in Linux world a kernel thread means a task_struct object.
Because scheduler only knows about these entities which can be assigned to different CPU’s (logical or physical). In other words if you want Linux scheduler to schedule your process you must create a task_struct.
User thread is something that is supported and managed outside of kernel by some execution environment (EE from now on) such as JVM. These EE’s will provide you with some functions to create new threads.
But why a user thread must always be mapped to a specific kernel thread.
Let’s say you created some threads using your EE. eventually they must be executed by the CPU and from above explanation we know that the thread must have a task_struct in order to be assigned to some CPU. That is why the mapping must exist. It’s the duty of your EE to create task_structs.
If your EE uses many to one model then it will create only one task_struct for all the threads and it will schedule all these threads onto that task_struct. Think of it as there is one CPU (task_struct) and many processes (threads created in EE), your operating system (the EE) will multiplex these processes on that single CPU.
If it uses one to one model than there will be one task_struct for every thread created in EE. So when you create a new thread in your EE, corresponding task_struct gets created in the kernel.
Windows does things differentlly ( process and thread is different ) but general idea stays the same that is kernel thread is the entity that CPU scheduler considers for assignment hence user threads must be mapped to corresponding kernel threads (if you want CPU to execute them).
I have a cuda kernel which works fine when called from a single CPU threads. However when the same is called from multiple CPU threads (~100), most of the kernel seems not be executed at all as the results comes out to be all zeros.Can someone please guide me how to resolve this problem?
In the current version of kernel I am using a cudadevicesynchronize() at the end of kernel call. Will adding a sync command before cudaMalloc() and kernel call be of any help in this case?
There is another thing which need some clarification. i.e. If two CPU threads executes the same cudaMalloc() command, will the later overwrite the former in GPU memory or will they create their own memory?
Thanks in advance for your help
Usually one CPU thread can be used for calling a CUDA kernel. However, since CUDA 4.0, multiple CPU threads can share context. You can use cuCtxSetCurrent to tie the context of the kernel to the current thread. More information about this API function can be found here.
Another workaround for this is to create a GPU worker thread that holds the context and pass any CUDA request to that thread.
Regarding your other question, without setting the context for the proper thread, I remember that cudaMalloc would not even execute (I work with JCuda so the behavior may be a little different). But if the context is currently set to the calling kernel, the memories will not be overwritten.
So two questions here really. First, (and yes, I have searched this already, but wanted clarification), what is the difference between a user thread and a kernel thread? Is it simply that one is generated by a user program and the other by an OS, with the latter having access to privileged instructions? Are they conceptually the same or are there actual differences in the threads themselves?
Second, and the real problem of my question is: the book I am using says that "a relationship must exist between user threads and kernel threads," going on to list the different models of such a relationship. But the book fails to clearly explain why a user thread must always be mapped to a specific kernel thread. Why is this?
A kernel thread is a thread object maintained by the operating system. It is an actual thread that is capable of being scheduled and executed by the processor. Typically, kernel threads are heavyweight objects with permissions settings, priorities, etc. The kernel thread scheduler is in charge of scheduling kernel threads.
User programs can make their own thread schedulers too. They can make their own "threads" and simulate context-switches to switch between them. However, these threads aren't kernel threads. Each user thread can't actually run on its own, and the only way for a user thread to run is if a kernel thread is actually told to execute the code contained in a user thread. That said, user threads have major advantages over kernel threads. They can be a lot more lightweight, since they don't necessarily need to have their own priorities, can be managed by a single process (which might have better info about what threads need to run when), and don't create lots of kernel objects for purposes of security and locking.
The reason that user threads have to be associated with kernel threads is that by itself a user thread is just a bunch of data in a user program. Kernel threads are the real threads in the system, so for a user thread to make progress the user program has to have its scheduler take a user thread and then run it on a kernel thread. The mapping between user threads and kernel threads doesn't have to be one-to-one (1 : 1); you can have multiple user threads share the same kernel thread (only one of those user threads runs at a time), and you can have a single user thread which is rotated across different kernel threads in a 1 : n mapping.
I think a real world example will clear the confusion, so let’s see how things are done in Linux.
First of all Linux doesn’t differentiate between process and thread, entity that can be scheduled is called task in Linux and represented by task_struct. So whenever you execute a fork() system call, a new task_struct is created which holds data (or pointer) associated with new task.
So in Linux world a kernel thread means a task_struct object.
Because scheduler only knows about these entities which can be assigned to different CPU’s (logical or physical). In other words if you want Linux scheduler to schedule your process you must create a task_struct.
User thread is something that is supported and managed outside of kernel by some execution environment (EE from now on) such as JVM. These EE’s will provide you with some functions to create new threads.
But why a user thread must always be mapped to a specific kernel thread.
Let’s say you created some threads using your EE. eventually they must be executed by the CPU and from above explanation we know that the thread must have a task_struct in order to be assigned to some CPU. That is why the mapping must exist. It’s the duty of your EE to create task_structs.
If your EE uses many to one model then it will create only one task_struct for all the threads and it will schedule all these threads onto that task_struct. Think of it as there is one CPU (task_struct) and many processes (threads created in EE), your operating system (the EE) will multiplex these processes on that single CPU.
If it uses one to one model than there will be one task_struct for every thread created in EE. So when you create a new thread in your EE, corresponding task_struct gets created in the kernel.
Windows does things differentlly ( process and thread is different ) but general idea stays the same that is kernel thread is the entity that CPU scheduler considers for assignment hence user threads must be mapped to corresponding kernel threads (if you want CPU to execute them).
Is there anything wrong with creating a window in a separate thread, which will also contain the message loop, then creating an OpenGL Context in another thread?
You should be able to get it to work, if you're careful. See the parallel opengl faq.
Q: Why does my OpenGL application crash/not work when
I am rendering from another thread?
A: The OpenGL context is thread-specific. You have to
make it current in the thread using glXMakeCurrent,
wglMakeCurrent or aglSetCurrentContext, depending on
your operating system.
What you want to do is perfectly possible. Even better, OpenGL contexts can migrate between threads and even be used with multiple windows as long as their pixel format is compatible. The one constraint is, that a OpenGL context can be bound in only one thread at a time and that only a unbound context can be bound.
So you could even create the window and the context in one thread, then unbind the context, create another thread and re-bind the context to the window in the secondary thread. No problem there.
The only thing you must be aware of is, that OpenGL itself doesn't like to be multithreaded. The API itself is more or less thread safe, as only one context can be bound to a thread at a time. But all the bookkeeping required if OpenGL operations spawn over several threads may trigger nasty driver bugs and also has a certain performance hit.