Scheduling policy and priority using pthread does not make any difference - linux

Question

I am running a simple multi-threaded program using pthread. Considering using real-scheduler (SCHED_FIFO policy), lower priority threads won't be able to run until higher priority ones are finished. But, when I run two versions of this program (the only difference is priority 99->1) at the same time, they finish at almost the same time. I even changed the policy to SCHED_OTHER but still no difference.
# include <stdio.h>
# include <string.h>
# include <pthread.h>
# include <stdlib.h>
# include <unistd.h>
# include <math.h>
# define NUM_THREADS 128
pthread_t tid[NUM_THREADS];
int indexes[NUM_THREADS];
void* dummyThread(void *arg)
{
unsigned long i = 0;
pthread_t id = pthread_self();
float a, b = 5, c = 8;
printf("Thread %d started.\n", *(int*)arg + 1);
for(i = 0; i < 10000000; i++)
a = sin(b) + sqrt(b);
printf("Thread %d finished.\n", *(int*)arg + 1);
return NULL;
}
int main(void)
{
int i = 0;
pthread_attr_t attr;
struct sched_param schedParam;
struct timespec start, finish;
double elapsed;
pthread_attr_init(&attr);
pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
schedParam.sched_priority = 1;
pthread_attr_setschedparam(&attr, &schedParam);
pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
clock_gettime(CLOCK_MONOTONIC, &start);
for (i = 0 ; i < NUM_THREADS; i++)
{
indexes[i] = i;
if (!pthread_create((void*)&tid[i], &attr, &dummyThread, &indexes[i]))
printf("Thread %d created successfully.\n", i + 1);
else
printf("Failed to create Thread %d.\n", i + 1);
}
for (i = 0 ; i < NUM_THREADS; i++)
pthread_join(tid[i], NULL);
clock_gettime(CLOCK_MONOTONIC, &finish);
elapsed = (finish.tv_sec - start.tv_sec);
elapsed += (finish.tv_nsec - start.tv_nsec) / 1000000000.0;
printf("%lf\n", elapsed);
return 0;
}
Edit 1: Updated my code by adding pthread_attr_setschedparam and error checking. I don't get any errors when running it without sudo, and changing priority or scheduling policy still does not change the result.
Edit 2: I noticed that when I create threads with different priorities within the same process it works well. In the following code for threads with even index I assign priority 1 while for threads with an odd index I assign priority 99. It works well and odd threads finish first before even threads.
# include <stdio.h>
# include <string.h>
# include <pthread.h>
# include <stdlib.h>
# include <unistd.h>
# include <math.h>
# define NUM_THREADS 128
pthread_t tid[NUM_THREADS];
int indexes[NUM_THREADS];
void* dummyThread(void *arg)
{
unsigned long i = 0;
pthread_t id = pthread_self();
float a, b = 5, c = 8;
printf("Thread %d started.\n", *(int*)arg);
for(i = 0; i < 10000000; i++)
a = sin(b) + sqrt(b);
printf("Thread %d finished.\n", *(int*)arg);
return NULL;
}
int main(void)
{
int i = 0;
pthread_attr_t attr;
struct sched_param schedParam;
struct timespec start, finish;
double elapsed;
clock_gettime(CLOCK_MONOTONIC, &start);
for (i = 0 ; i < NUM_THREADS; i++)
{
indexes[i] = i;
pthread_attr_init(&attr);
pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
schedParam.sched_priority = i % 2 == 0 ? 1 : 99;
pthread_attr_setschedparam(&attr, &schedParam);
pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
if (!pthread_create((void*)&tid[i], &attr, &dummyThread, &indexes[i]))
printf("Thread %d created successfully.\n", i);
else
printf("Failed to create Thread %d.\n", i);
}
for (i = 0 ; i < NUM_THREADS; i++)
pthread_join(tid[i], NULL);
clock_gettime(CLOCK_MONOTONIC, &finish);
elapsed = (finish.tv_sec - start.tv_sec);
elapsed += (finish.tv_nsec - start.tv_nsec) / 1000000000.0;
printf("%lf\n", elapsed);
return 0;
}
Since threads from different processes are all sent to the same scheduler in the Kernel, I don't know why it does not work with different processes.

Answer 1

From man pthread_attr_setschedpolicy
In order for the policy setting made by
pthread_attr_setschedpolicy() to have effect when calling
pthread_create(3), the caller must use
pthread_attr_setinheritsched(3) to set the inherit-scheduler
attribute of the attributes object attr to
PTHREAD_EXPLICIT_SCHED.
You've neglected to do that, so your SCHED_FIFO didn't have any effect.
As soon as I add pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED); call, pthread_create() starts failing with EPERM (since only root can create SCHED_FIFO threads).