I'm currently learning pthreads and am struggling to understand the relationship between thread priority and policy. What I know so far:
The thread priority is an integer that indicates priority. The higher this number, the higher priority the thread is treated by the OS.
The thread policy determines how the thread is executed among processes with the shared priority number. SCHED_RR and SCHED_FIFO are real-time policies that continuously execute unless an explicit "sleep" command is issued. Thus a programmer must very carefully write code when using these policies. SCHED_OTHER is a round robin policy that is not executed in real-time.
However, let's say I have the following scenarios (assume each thread does not use a "sleep" command).
Thread 1: priority = 0, policy = SCHED_OTHER
Thread 2: priority = 1, policy = SCHED_OTHER
// would thread 1 run at all?
Thread 1: priority = 0, policy = SCHED_RR
Thread 2: priority = 1, policy = SCHED_RR
// would thread 1 run at all?
I'm confused as to whether or not the the thread policy affects the thread priority, or if the thread priority always trumps the policy.
Edit: Found a web page that cleared up most of my confusion: https://computing.llnl.gov/tutorials/pthreads/man/sched_setscheduler.txt
Documentation for SCHED_RR says that it is the same as SCHED_FIFO except in certain cases when two or more threads have the same static priority.
Documentation for SCHED_FIFO makes it clear that if a thread with higher static priority is ready-to-run but not running, and if one or more threads with lower static priority are running, then one of the lower priority threads will be preempted in favor of the higher priority thread.
would thread 1 run at all [in the SCHED_RR case]?
That depends. What is thread 0 doing? How many CPUs does the system have? If those were the only two threads on a system that had only one CPU, then thread 1 would be allowed to run whenever thread 0 did not want to run.
Generally speaking, when you use static priorities, you want the highest priority threads to do the least amount of work. A high priority thread should spend most of its time waiting for some event. Then when the event happens, the thread should promptly acknowledge it, and then possibly signal a lower-priority thread if some kind of follow-up computation is required.
would thread 1 run at all [in the SCHED_OTHER case]?
As mentioned in my comment, if you're talking about static priorities (i.e., as set by the sched_setattr() system call, then the question is meaningless because threads that are scheduled under the SCHED_OTHER policy are all required to have the same static priority--zero.
Related
Apologies, if it is not the right place for this question.
I am working on an RTOS (RTX), my question is if a High Priority Thread is waiting for a Thread Flag (says the flag is X_High) (currently is in a blocked state) and a Low Priority Thread is running, and this low priority thread sets "X_High" thread flag.
Would context switching from low priority to High Thread take place immediately or low priority thread will keep on running till a blocking statement (Delay or waiting for thread flag)?
The guarantee of a pre-emptive priority bases real-time scheduler is that the highest priority thread that is ready to run, will run (outside if interrupt processing).
The scheduling decision is made whenever a scheduling call is made in a normal thread and in exit from the interrupt context. osEventFlagsSet is a scheduling call. If any event set by it causes a higher priority thread to become ready, the calling thread is pre-empted immediately (before the function returns).
It depends on the specific RTOS you're using, and how it is configured. For example, FreeRTOS can be configured to use either cooperative scheduling or preemptive scheduling. When using preemptive scheduling, task switching takes place as soon as the flag is set (semaphore is given). But when using cooperative scheduling, task switching takes place when the currently running task is blocked, or yields voluntarily.
I am testing my multithreaded application in Linux RT multicore machine.
However during testing, we are observing that scheduling (created with SCHED_FIFO scheduling policy ) in Linux RT is not happening according to the SCHED_FIFO policy.
We could see in multiple places that the higher priority thread execution is getting preempted by a lower priority thread.
Based on some research we did on the internet, we found that the following kernel parameters need to be changed from
/proc/sys/kernel/sched_rt_period_us containing 1000000
/proc/sys/kernel/sched_rt_runtime_us containing 950000
to
/proc/sys/kernel/sched_rt_period_us containing 1000000
/proc/sys/kernel/sched_rt_runtime_us containing 1000000
or
/proc/sys/kernel/sched_rt_period_us containing -1
/proc/sys/kernel/sched_rt_runtime_us containing -1
We tried doing both but still we are facing the problem sometimes. We are facing the issue even when higher priority thread is not suspended by any system call.
It would be great if you could let us know if you are aware of such problems in Linux RT scheduling and/or have any solutions to make the Linux RT scheduling deterministic based on priority.
There are no printfs or any system calls in the higher priority thread but still the higher priority thread is getting preempted by the lower priority thread.
Also I have made sure all the threads in the process are running on a single core using taskset command.
There could be two reasons:
CPU throttling: the scheduler is designed to reserve some CPU time to non-RT tasks; you have already disabled it by acting on the /proc/sys/kernel/ entries
blocking: your high-priority task is blocking either
on some synchronization mechanism (e.g., mutex, semaphore) or
on some blocking call (e.g., malloc, printf, read, write, etc.)
In RR scheduling policy what will happen if a low priority thread locks a mutex and is removed by the scheduler because another high priority thread is waiting?
Will it also releases the lock held by low priority thread?
For example consider 3 threads running in a process with priorities 10,20 and 30 in RR scheduling policy.
Now at a given point of time low priority thread 1 locks mutex and is still doing execution mean while the high priority thread pops in and also waits on mutex held by thread 1. Now thread 2 comes in to picture which also needs the same mutex locked by thread 1.
As far as I know as per scheduling algorithm the threads sleeping or waiting for mutex,semaphore etc are removed and the other ones, even having low priority are allowed to execute. Is this correct? If so, in the above example ultimately high priority threads wait for the completion of low priority thread which doesn't make any sense.
Is this how the system works if at all threads are designed like I said above?
or
the thread priority should be set in such a way that high priority one's will not depend on low priority one's mutexe's ?
Also can anyone please explain me how scheduling works at process level? How do we set priority for a process?
Typically, scheduling and locks are unrelated in any other aspect than a "waiting thread is not scheduled until it's finished waiting". It would be rather daft to have a MUTEX that "stops other thread from accessing my data" but it only works if the other thread has the same or lower priority than the current thread.
The phenomena of "a low priority holds a lock that a high priority thread 'needs'" is called priority inversion, and it's a well known scenario in computer theory.
There are some schemes that "temporarily increase priority of a lock-holding thread until it releases the lock to the highest priority of the waiting threads" (or the priority of the first waiting thread if it's higher than the current thread, or some other variation on that theme). This is done to combat priority inversion - but it has other drawbacks too, so it's not implemented in all OS's/schedulers (after all, it affects other threads than the one that is waiting too).
Edit:
The point of a mutex (or other similar locks) is that it prevents two threads from accessing the same resources at once. For example, say we want to update five different variables with some pretty lengthy processing (complicated math, fetching data from a serial port or a network drive, or some such), but if we only do two of the variables, some other process using these would get an invalid result, then we clearly can't "let go" of the lock.
The high priority thread simply has to wait for all five variables to be updated and the low priority lock.
There is no simple workaround that the application can do to "fix" this problem - don't hold the locks more than necessary of course [and it may be that we can actually fix the problem described above, by performing the lengthy processing OUTSIDE of the lock, and only do the final "store it in the 5 variables" with the lock on. This would reduce the potential time that the high priority thread has to wait, but if the system is really busy, it won't really fix the problem.
There are whole PhD thesis written on this subject, and I'm not a specialist on "how to write a scheduler" - I have a fair idea how one works, just like I know how the engine in my car works - but if someone gave me a bunch of suitable basic shapes of steel and aluminium, and the required tools/workspace and told me to build an engine, I doubt it would work great...Same with a scheduler - I know what the parts are called, but not how to build one.
If a high priority thread needs to wait on a low priority thread (mutex semaphore etc), the low priority thread is temporarily elevated to the same priority as the high priority thread.
The high priority thread is not going to have the lock for which it is requesting until the low priority thread will unlock it.
To avoid this we can use semaphore where any other thread can initiate to unlock but in mutex it is not possible.
SCHED_FIFO and SCHED_RR are both meant for real-time uses. I am aware that SCHED_RR can be preempted by time slicing. But say if I have one thread set to SCHED_FIFO, and another set to SCHED_RR, if both threads are ready to run, are they scheduled purely by priority? What if they have same priority?
Conceptually, there is a list of runnable processes associated with each static priority level. These lists can contain both SCHED_FIFO and SCHED_RR processes - the two scheduling policies share the same set of static priorities.
When selecting a process to run, the scheduler takes the process at the head of the non-empty list with the highest static priority, regardless of the scheduling policy of that process.
The scheduling policies affect how the processes move within those lists. For SCHED_FIFO, once a process reaches the head of the list for a given priority it will stay there until it blocks or yields. For SCHED_RR, a runnable process that has exceeded its maximum time quantum will be moved to the end of the list for its static priority.
This morning I read about Linux real time scheduling. As per the book 'Linux system programming by Robert Love', there are two main scheduling there. One is SCHED_FIFO, fifo and the second is SCHED_RR, the round robin. And I understood how a fifo and a rr algorithm works. But as we have the system call,
sched_setscheduler (pid_t pid, int policy, const struct sched_parem *sp)
we can explicitly set the scheduling policy for our process. So in some case, two process running by root, can have different scheduling policy. As one process having SCHED_FIFO and another having SCHED_RR and with same priority. In that case, which process will be selected first? the FIFO classed process or the RR classed process? Why?
Consider this case. There are three process A,B,C. All are having same priority. A and B are RR classed processes and C is FIFO classed one. A and B are runnable (so both are running alternatively in some time intervel). And currently A is running. Now C becomes runnable. In this case, whether
1. A will preempt for C, or
2. A will run until its timeslice goes zero and let C run. Or
3. A will run until its timeslice goes zero and let B run.
a) here after B runs till its timeslice becomes zero and let C run or
b) after B runs till its timeslice becomes zero and let A run again (then C will starve untill A and B finishes)
In realtime scheduling, FIFO and RR do not have exactly the same meaning they have in non-realtime scheduling. Processes are always selected in a FIFO- manner, however, the time quantum for SCHED_FIFO is not limited unlike the time quantum for SCHED_RR.
SCHED_FIFO processes do not preempt SCHED_RR processes of the same priority.
sched_setscheduler(2) - Linux man page
...
"A process's scheduling policy determines where it will be inserted into the list of processes with equal static priority and how it will move inside this list. All scheduling is preemptive: if a process with a higher static priority becomes ready to run, the currently running process will be preempted and returned to the wait list for its static priority level. The scheduling policy only determines the ordering within the list of runnable processes with equal static priority."
...
"A SCHED_FIFO process runs until either it is blocked by an I/O request, it is preempted by a higher priority process, or it calls sched_yield(2)."
...
"When a SCHED_FIFO process becomes runnable, it will be inserted at the end of the list for its priority."
...
"SCHED_RR: Round Robin scheduling
SCHED_RR is a simple enhancement of SCHED_FIFO. Everything described above for SCHED_FIFO also applies to SCHED_RR, except that each process is only allowed to run for a maximum time quantum. If a SCHED_RR process has been running for a time period equal to or longer than the time quantum, it will be put at the end of the list for its priority. A SCHED_RR process that has been preempted by a higher priority process and subsequently resumes execution as a running process will complete the unexpired portion of its round robin time quantum."
man sched_setscheduler explains these scheduling policies in detail.
In this particular case because the two real-time processes have the same priority none of them will preempt the other. A SCHED_FIFO process runs until it blocks itself, SCHED_RR process runs until it blocks itself or its time quantum expires.
According to the man page, I think 1 is the answer. A, B are RR policy, C is FIFO policy. Since RR is also an enhancement FIFO, all of them are FIFO class.
Since all of them have the same priority, and man page say " A call to sched_setscheduler() or sched_setparam(2) will put the SCHED_FIFO (or SCHED_RR) process identified by pid at the start of the list if it was runnable. As a consequence, it may preempt the currently running process if it has the same priority. (POSIX.1-2001 specifies that the process should go to the end of the list.)"
Once calling sched_setscheduler to set the policy of C as FIFO, C will preempt A.
My understanding of the two different classes is that a process SCHED_FIFO is never pre-empted by the kernel. Even if another "SCHED_FIFO" class process is waiting its turn...
While SCHED_RR policy shares the cpu ressources a little bit more. The scheduler will let the SCHED_RR process run for a quanta of time, then pre-empt it only to let turn another SCHED_RR process. That is exactly Round Robin.
SCHED_FIFO is "stronger" in the sense that if a SCHED_FIFO process never yield() to the kernel or invoke a system call on a single core device, then all your other Real time processes may never run.