Can two hrtimer callbacks in linux kernel run at the same time? - linux

As stated in https://lwn.net/Articles/308545/ hrtimer callbacks run in hard interrupt context with irqs disabled.
But what about a SMP?
Can a second callback for another hrtimer run on another core,
while a first callback is allready running or do they exclude each other on all cores, so that no locking is needed between them?
edit:
When a handler for a "regular" hardware IRQ (let's call it X) is running on a core, all IRQs are disabled on only that core, but IRQ X is disabled on the whole system, so two handlers for X never run concurrently.
How do hrtimer interrupts behave in this regard?
Do they all share the same quasi IRQ, or is there one IRQ per hrtimer?
edit:
Did some experiments with two timers A and B:
// starting timer A to fire as fast as possible...
A_ktime = ktime_set(0, 1); // 1 NS
hrtimer_start( &A, A_ktime, HRTIMER_MODE_REL );
// starting timer B to fire 10 us later
B_ktime = ktime_set(0, 10000); // 10 us
hrtimer_start( &B, B_ktime, HRTIMER_MODE_REL );
Put some printks into the callbacks and a huge delay into the one for timer A
// fired after 1 NS
enum hrtimer_restart A(struct hrtimer *timer)
{
printk("timer A: %lu\n",jiffies);
int i;
for(i=0;i<10000;i++){ // delay 10 seconds (1000 jiffies with HZ 100)
udelay(1000);
}
printk("end timer A: %lu\n",jiffies);
return HRTIMER_NORESTART;
}
// fired after 10 us
enum hrtimer_restart B(struct hrtimer *timer)
{
printk("timer B: %lu\n",jiffies);
return HRTIMER_NORESTART;
}
Result was reproducible something like
[ 6.217393] timer A: 4294937914
[ 16.220352] end timer A: 4294938914
[ 16.224059] timer B: 4294938915
1000 jiffies after start of timer A,
when timer B was setup to fire after less than one jiffie after it.
When driving this further and increasing the delay to 70 seconds,
I got 7000 jiffies between start of timer A callback and timer B callback.
[ 6.218258] timer A: 4294937914
[ 76.220058] end timer A: 4294944914
[ 76.224192] timer B: 4294944915
edit:
Locking is probably required, because hrtimers
just get enqueued an any CPU. If two of them are enqueued on the same, it might happen, that they delay each other, but there is no guarantee.
from hrtimer.h:
* On SMP it is possible to have a "callback function running and enqueued"
* status. It happens for example when a posix timer expired and the callback
* queued a signal. Between dropping the lock which protects the posix timer
* and reacquiring the base lock of the hrtimer, another CPU can deliver the
* signal and rearm the timer.

Related

Does wake_up cause a race condition?

I was looking at the wake_up function here from the linux kernel code
https://elixir.bootlin.com/linux/latest/source/kernel/sched/wait.c#L154
It's line 154
/**
* __wake_up - wake up threads blocked on a waitqueue.
* #wq_head: the waitqueue
* #mode: which threads
* #nr_exclusive: how many wake-one or wake-many threads to wake up
* #key: is directly passed to the wakeup function
*
* If this function wakes up a task, it executes a full memory barrier before
* accessing the task state.
*/
void __wake_up(struct wait_queue_head *wq_head, unsigned int mode,
int nr_exclusive, void *key)
{
__wake_up_common_lock(wq_head, mode, nr_exclusive, 0, key);
}
If it's waking up all the threads, couldn't this cause a race condition? Let's say all the threads are waiting for the same data structure or something, so once the wake_up is called, aren't all the threads racing for the same thing?

will the timer started by "setitimer()" be restarted?

Each task_struct in linux kernel has a field named "real timer", which is a struct hrtimer (high resolution timer). When we set a timer using setitimer, it sets the "real timer" in the process to be expired by the given value. When it is expired, the function named it_real_fn is called. Here is the source code in Linux kernel 2.6.39.4:
/*
* The timer is automagically restarted, when interval != 0
*/
enum hrtimer_restart it_real_fn(struct hrtimer *timer)
{
struct signal_struct *sig =
container_of(timer, struct signal_struct, real_timer);
trace_itimer_expire(ITIMER_REAL, sig->leader_pid, 0);
kill_pid_info(SIGALRM, SEND_SIG_PRIV, sig->leader_pid);
return HRTIMER_NORESTART;
}
I found that it returns HRTIMER_NORESTART, which means it shall not be restarted. However, if we assign an interval value when calling setitimer, meaning that we want to trigger the timer at every interval, where shall the "real timer" be restarted?
You are correct in saying that the it_real_fn function will not restart the real-time timer but it is not the function that is called on timer expiration when using setitimer.
The setitimer function is part of the POSIX timers and their Linux implementation is in posix-timers.c, In this file, the function posix_timer_fn, which may return both HRTIMER_RESTART and HRTIMER_NORESTART is defined and assigned to the it_real_fn of the struct hrtimer (the code which sets this function as the timer callback is in common_timer_set, which is called by the POSIX timer initialization).

Multithreaded Environment - Signal Handling in c++ in unix-like environment (freeBSD and linux)

I wrote a network packet listener program and I have 2 threads. Both runs forever but one of them sleeps 30 sec other sleeps 90 sec. In main function, I use sigaction function and after installed signal handler, I created these 2 threads. After creation of threads, main function calls pcaploop function, which is infinite loop. Basic structure of my program:
(I use pseudo syntax)
signalHandler()
only sets a flag (exitState = true)
thread1()
{
while 1
{
sleep 30 sec
check exit state, if so exit(0);
do smth;
}
}
thread2()
{
while 1
{
sleep 90 sec
check exit state, if so exit(0);
do smth;
}
}
main()
{
necassary snytax for sigaction ;
sigaction( SIGINT, &act, NULL );
sigaction( SIGUSR1, &act, NULL );
create thread1;
create thread2;
pcaploop(..., processPacket,...); // infinite loop, calls callback function (processPacket) everytime a packet comes.
join threads;
return 0;
}
processPacket()
{
check exitState, if true exit(0);
do smth;
}
And here is my question. When I press CTRL-C program does not terminate. If the program run less than 6-7 hours, when I press CTRL-C, program terminates. If the program run 1 night, at least 10 hours or more, I cannot terminate the program. Actually, signal handler is not called.
What could be the problem? Which thread does catch the signal?
Basically it would be better to remove all pseudo code you put in your example, and leave the minimum working code, what exactly you have.
From what I can see so far from your example, is that the error handling of sigaction's is missing.
Try to perform checks against errors in your code.
I am writing this for those who had faced with this problem. My problem was about synchronization of threads. After i got handle synchronization problem, the program now, can handle the signals. My advice is check the synchronization again and make sure that it works correctly.
I am sorry for late answer.
Edited :
I have also used sigaction for signal handling
and I have change my global bool variable whit this definition :
static volatile sig_atomic_t exitFlag = 0;
This flag has been used for checking whether the signal received or not.

Is there a timed signal similar to pthread_cond_timedwait?

I have created many threads all waiting for there own condition. Each thread when runs signals its next condition and again goes into wait state.
However, I want that the currently running thread should signal its next condition after some specified period of time (very short period). How to achieve that?
void *threadA(void *t)
{
while(i<100)
{
pthread_mutex_lock(&mutex1);
while (state != my_id )
{
pthread_cond_wait(&cond[my_id], &mutex1);
}
// processing + busy wait
/* Set state to i+1 and wake up thread i+1 */
pthread_mutex_lock(&mutex1);
state = (my_id + 1) % NTHREADS;//usleep(1);
// (Here I don't want this sleep. I want that this thread completes it processing and signals next thread a bit later.)
/*nanosleep(&zero, NULL);*/
pthread_cond_signal(&cond[(my_id + 1) % NTHREADS]); // Send signal to Thread (i+1) to awake
pthread_mutex_unlock(&mutex1);
i++;
}
Signalling a condition does nothing if there is nothing waiting on the condition. So, if pthread 'x' signals condition 'cx' and then waits on it, it will wait for a very long time... unless some other thread also signals 'cx' !
I'm not really sure I understand what you mean by the pthread signalling its "next condition", but it occurs to me that there is not much difference between waiting to signal a waiting thread and the thread sleeping after it is signalled ?

CPU contention (wait time) for a process in Linux

How can I check how long a process spends waiting for the CPU in a Linux box?
For example, in a loaded system I want to check how long a SQL*Loader (sqlldr) process waits.
It would be useful if there is a command line tool to do this.
I've quickly slapped this together. It prints out the smallest and largest "interferences" from task switching...
#include <sys/time.h>
#include <stdio.h>
double seconds()
{
timeval t;
gettimeofday(&t, NULL);
return t.tv_sec + t.tv_usec / 1000000.0;
}
int main()
{
double min = 999999999, max = 0;
while (true)
{
double c = -(seconds() - seconds());
if (c < min)
{
min = c;
printf("%f\n", c);
fflush(stdout);
}
if (c > max)
{
max = c;
printf("%f\n", c);
fflush(stdout);
}
}
return 0;
}
Here's how you should go about measuring it. Have a number of processes, greater than the number of your processors * cores * threading capability wait (block) on an event that will wake them up all at the same time. One such event is a multicast network packet. Use an instrumentation library like PAPI (or one more suited to your needs) to measure the differences in real and virtual "wakeup" time between your processes. From several iterations of the experiment you can get an estimate of the CPU contention time for your processes. Obviously, it's not going to be at all accurate for multicore processors, but maybe it'll help you.
Cheers.
I had this problem some time back. I ended up using getrusage :
You can get detailed help at :
http://www.opengroup.org/onlinepubs/009695399/functions/getrusage.html
getrusage populates the rusage struct.
Measuring Wait Time with getrusage
You can call getrusage at the beginning of your code and then again call it at the end, or at some appropriate point during execution. You have then initial_rusage and final_rusage. The user-time spent by your process is indicated by rusage->ru_utime.tv_sec and system-time spent by the process is indicated by rusage->ru_stime.tv_sec.
Thus the total user-time spent by the process will be:
user_time = final_rusage.ru_utime.tv_sec - initial_rusage.ru_utime.tv_sec
The total system-time spent by the process will be:
system_time = final_rusage.ru_stime.tv_sec - initial_rusage.ru_stime.tv_sec
If total_time is the time elapsed between the two calls of getrusage then the wait time will be
wait_time = total_time - (user_time + system_time)
Hope this helps

Resources