tickless kernel , isolcpus,nohz_full,and rcu_nocbs - linux

Question

I have add "isolcpus=3 nohz_full=3 rcu_nocbs=3" in grub.conf in
RedHat 7.1 , kernel: linux 3.10.0-229 kernel and according to http://www.breakage.org/2013/11/15/nohz_fullgodmode/
I also execute the following command :
cat /sys/bus/workqueue/devices/writeback/cpumask
f
echo 1 > /sys/bus/workqueue/devices/writeback/cpumask
cat /sys/bus/workqueue/devices/writeback/numa
1
echo 0 > /sys/bus/workqueue/devices/writeback/numa
The box has only 4 cpu cores , I run the following shell :
watch -d 'cat /proc/interrupts'
look like work perfect , only cpu0 Local timer interrupts has 2000 per 2 secs,
the else cpu 1 to cpu 3 has less than 10 per 2 secs .
and then I test the following source :
void *Thread2(void *param)
{
pthread_detach(pthread_self());
while( 1 ){
sleep( 100000 ) ;
}
}
void *Thread1(void *param)
{
pthread_detach(pthread_self());
while( 1 ){
;
}
}
int main(int argc, char** argv)
{
pthread_t tid ;
pthread_create(&tid , NULL, Thread1, (void*)(long)3);
pthread_create(&tid , NULL, Thread2, (void*)(long)3);
while( 1 )
sleep( 5 ) ;
}
and run it by :
taskset -c 3 ./x1.exe
watch the output in :
watch -d 'cat /proc/interrupts'
this time , cpu 3 get 10~30 Local timer interrupts per 2 secs , look fine,
then I try to run 2 thread1 by :
pthread_create(&tid , NULL, Thread1, (void*)(long)3);
pthread_create(&tid , NULL, Thread1, (void*)(long)3);
then again run it :
taskset -c 3 ./x1.exe
then I watch the core 3 has the same Local timer interrupts with core 0 ,
it is 2000 interrupts per 2 secs .
May I ask , why 2 very busy thread1 will cause core 3 has
much more timer interrupts ?! what cause this happened ?!
and how to modify it if it can be ?!

Answer 1

In the second case, Kernel needs to schedule 2 cpu bound tasks on core 3 and the dynamic ticks configuration is applicable only when there is exactly one runnable task.
I thought SCHED_FIFO would stop these interrupts (and so I started answering), but that isn't yet implemented as per https://www.kernel.org/doc/Documentation/timers/NO_HZ.txt
There is no way to change this behaviour except scheduling threads on different CPUs. You can always hack the kernel to achieve what you need.