Lowest latency notification method between process under Linux - linux

Question

I'm looking for the lowest latency IPC that allow to put one process to sleep and allow other process to wake it.
I'm looking for the lowest latency method. Some possible methods so far:
Writing a byte to a pipe and reading it from it.
Writing a byte to a socket and reading it from it.
Sending a signal (kill) and waiting for it (sigwait)
Using sem_post/sem_wait
Any other better ideas?
Any solution that is Linux specific is fine as well.

Answer 1

Generally... There is almost no difference between the OS methods.
Setup:
Two processes with affinity to two different CPUs.
One process sleeps (nanosleep) for N microseconds measures current time
and then notifies other process.
Other process wakes measures current time and compares it to the client's time.
Average, standard deviation, median and percentile 95 is calculated over 1K samples after warm-up on 100 notifications.
OS: Linux 2.6.35 x86_64
CPU: Intel i5 M460
Results:
Semaphore (sem_wait/sem_post - kernel - futex):
sleep us mean median %95
1 4.98 ±18.7 3.78 5.04
10 4.14 ±14.8 3.54 4.00
100 20.60 ±29.4 22.96 26.96
1000 49.42 ±37.6 30.62 78.75
10000 63.20 ±22.0 68.38 84.38
Signal (kill/sigwait)
sleep us mean median %95
1 4.69 ±3.8 4.21 5.39
10 5.91 ±14.8 4.19 7.45
100 23.90 ±17.7 23.41 35.90
1000 47.38 ±28.0 35.27 81.16
10000 60.80 ±19.9 68.50 82.36
Pipe (pipe + write/read)
sleep us mean median %95
1 3.75 ±5.9 3.46 4.45
10 4.42 ±3.5 3.84 5.18
100 23.32 ±25.6 24.17 38.05
1000 51.17 ±35.3 46.34 74.75
10000 64.69 ±31.0 67.95 86.80
Socket (socketpair +write/read)
sleep us mean median %95
1 6.07 ±3.2 5.55 6.78
10 7.00 ±7.1 5.51 8.50
100 27.57 ±14.1 28.39 50.86
1000 56.75 ±25.7 50.82 88.74
10000 73.89 ±16.8 77.54 88.46
As a reference busy waiting:
sleep us mean median %95
1 0.17 ±0.5 0.13 0.23
10 0.15 ±0.3 0.13 0.19
100 0.17 ±0.3 0.16 0.21
1000 0.22 ±0.1 0.18 0.35
10000 0.38 ±0.3 0.30 0.78

Answer 2

Using the same code provided by #Artyom, but in a more morden hardware.
CPU: i9-9900k, closing C/S/P-state and set scaling policy as performance which keeps cores running at the maximum frequency(~5GHz).
OS: Preempt-RT patched Linux with kernel5.0.21, providing better real-time performance.
CPU affinity: two processes apartly running in two isolated cores, which keep away from irrelevant processes and interrupts AMSP.
Results:
Semaphore (sem_wait/sem_post - kernel - futex):
sleep us mean minimum median %99
1 1.75 ±0.1 1.60 1.74 1.82
10 1.76 ±0.0 1.61 1.75 1.83
100 2.12 ±0.3 1.59 2.24 2.42
1000 2.46 ±0.3 1.75 2.47 2.56
10000 2.45 ±0.1 2.11 2.44 2.54
Signal (kill/sigwait)
sleep us mean minimum median %99
1 2.15 ±0.2 2.00 2.13 2.22
10 2.12 ±0.2 1.93 2.11 2.19
100 2.56 ±0.3 2.00 2.67 2.88
1000 2.90 ±0.3 2.17 2.90 3.01
10000 2.94 ±0.5 2.66 2.89 3.03
Pipe (pipe + write/read)
sleep us mean minimum median %99
1 2.05 ±0.2 1.88 2.03 2.15
10 2.06 ±0.3 1.89 2.04 2.17
100 2.54 ±0.4 1.88 2.63 2.87
1000 2.98 ±0.3 2.27 2.98 3.09
10000 2.98 ±0.3 2.69 2.96 3.07
Socket (socketpair +write/read)
sleep us mean minimum median %99
1 3.11 ±0.4 2.85 3.09 3.22
10 3.14 ±0.1 2.92 3.14 3.25
100 3.66 ±0.5 2.92 3.74 4.01
1000 4.03 ±0.4 3.28 4.03 4.17
10000 3.99 ±0.4 3.64 3.96 4.10
As a reference busy waiting:
sleep us mean minimum median %99
1 0.07 ±0.1 0.06 0.07 0.07
10 0.07 ±0.1 0.06 0.07 0.07
100 0.07 ±0.0 0.06 0.07 0.08
1000 0.09 ±0.1 0.07 0.08 0.09
10000 0.09 ±0.1 0.07 0.09 0.09