Record instruction addresses of page faults with perf - linux

Question

I would like to get addresses of the instructions which leads to major page faults using perf.
I have a simple program:
#include <time.h>
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdint.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/mman.h>
int main(int argc, char* argv[]) {
int fd = open("path to large file several Gb", O_RDONLY);
struct stat st;
fstat(fd, &st);
void* ptr = mmap(NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
const uint8_t* data = (const uint8_t*) ptr;
srand(time(NULL));
size_t i1 = ((double) rand() / RAND_MAX) * st.st_size;
size_t i2 = ((double) rand() / RAND_MAX) * st.st_size;
size_t i3 = ((double) rand() / RAND_MAX) * st.st_size;
printf("%x[%lu], %x[%lu], %x[%lu]\n", data[i1], i1, data[i2], i2, data[i3], i3);
munmap(ptr, st.st_size);
close(fd);
return 0;
}
I compile it using gcc -g -O0 main.c
and run perf record -e major-faults -g -d ./a.out
Next I open the resulting report using perf report -g
The report says that there are 3 major page faults (it's correct),
but I can't understand addresses of the instructions which leads to the page faults.
The report is below:
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 3 of event 'major-faults'
# Event count (approx.): 3
#
# Children Self Command Shared Object Symbol
# ........ ........ ....... ................ ......................
#
100.00% 0.00% a.out libc-2.23.so [.] __libc_start_main
|
---__libc_start_main
main
100.00% 100.00% a.out a.out [.] main
|
---0x33e258d4c544155
__libc_start_main
main
100.00% 0.00% a.out [unknown] [.] 0x033e258d4c544155
|
---0x33e258d4c544155
__libc_start_main
main
a.out doesn't contain an address 0x33e258d4c544155 or something which ends with 155.
The question is how to get instruction addresses which leads to page faults?

Answer 1

For some reason I cannot reproduce your example, i.e. I'm not getting any samples with the major-faults event. But I can explain with a different example.
The pref report output is misleading, it doesn't the three events, it shows the three stack levels. It's much easier to understand by using perf script - which shows the actual events (including their stacks). The entries look like this (repeated for each sample):
a.out 22107 14721.378764: 10000000 cycles:u:
5653c1afb134 main+0x1b (/tmp/a.out)
7f58bb1eeee3 __libc_start_main+0xf3 (/usr/lib/libc-2.29.so)
49564100002cdb3d [unknown] ([unknown])
Now you see the function stack with the virtual instruction address, nearest symbol and offset from the symbol. If you want to fiddle with the addresses yourself, you can run perf script --show-mmap-events, which tells you:
a.out 22107 14721.372233: PERF_RECORD_MMAP2 22107/22107: [0x5653c1afb000(0x1000) # 0x1000 00:2b 463469 624179165]: r-xp /tmp/a.out
^ Base ^ size ^ offset ^ file
Then you can do the math for 0x5653c1afb134 by subtracting the base 0x5653c1afb000 and adding the offset 0x1000 - you get the address of the instruction or return address within the file.
You also see that 0x49564100002cdb3d is not mapped, could not be resolved - it's just garbage from the frame-pointer based stack unwinding. You can ignore it. You can also use --call-graph dwarf or --call-graph lbr which seem to show more sensible stack origins.