When I wanted to run perf under WSL, I met the follow question:
WARNING: perf not found for kernel 4.4.0-18362
You may need to install the following packages for this specific kernel:
linux-tools-4.4.0-18362-Microsoft
linux-cloud-tools-4.4.0-18362-Microsoft
You may also want to install one of the following packages to keep up to date:
linux-tools-Microsoft
linux-cloud-tools-Microsoft
But I can't find packages called linux-tools-4.4.0-18362-Microsoft or linux-cloud-tools-4.4.0-18362-Microsoft. I guess the package names are generated automatically.
I also tried to use perf in docker container. However, docker container use the same kernel as the hosts.
Is there any method to run perf under WSL?
I heard that perf can be used in WSL2. But after I upgraded to WSL2, it shows the similar error message:
WARNING: perf not found for kernel 4.19.84-microsoft
You may need to install the following packages for this specific
kernel:
linux-tools-4.19.84-microsoft-standard
linux-cloud-tools-4.19.84-microsoft-standard
You may also want to install one of the following packages to keep
up to date:
linux-tools-standard
linux-cloud-tools-standard
WARNING: perf not found for kernel 4.19.84-microsoft
Because WSL2 uses custom Linux kernel. Its source code can be found here
microsoft/WSL2-Linux-Kernel. We have to compile perf tools from it.
Procedure
Install required build packages. If you are using Ubuntu in WSL2 this is the
required command:
sudo apt install build-essential flex bison libssl-dev libelf-dev
Clone the WSL2 Linux kernel repository:
git clone --depth=1 https://github.com/microsoft/WSL2-Linux-Kernel.git
Go to perf folder and compile it:
cd WSL2-Linux-Kernel/tools/perf
make
perf executable file will be in that folder.
You can install linux-tools-generic.
apt install linux-tools-generic
Then run perf using the install path /usr/lib/linux-tools/<linux-version>-generic/perf.
Some tools, like flamegraph, will use environment variable PERF as the perf path.
PERF=/usr/lib/linux-tools/<linux-version>-generic/perf flamegraph -- my_program
The accepted answer works. However, some features are missing.
In order to get useful and demangled information, I had to install the following libs and then run make again.
libbabeltrace-dev
libunwind-dev
libdw-dev
binutils-dev
libiberty-dev
I'm not sure if all of them are necessary. However, those are adequate for cargo-flamegraph (my usecase) to work.
You could install some generic version of perf, rather than the WSL2 version, like:
sudo apt install linux-tools-5.4.0-126-generic linux-tools-common
And then when you run perf, it will error out, like:
$ perf
WARNING: perf not found for kernel 5.10.16.3-microsoft
You may need to install the following packages for this specific kernel:
linux-tools-5.10.16.3-microsoft-standard-WSL2
linux-cloud-tools-5.10.16.3-microsoft-standard-WSL2
This is because script /usr/bin/perf always trying to get the perf binary from uname -r
$ grep uname `which perf`
full_version=`uname -r`
We could replace /usr/bin/perf with the actual perf :
mv /usr/bin/perf /usr/bin/perf.bk && ln -s /usr/lib/linux-tools/5.4.0-126-generic/perf /usr/bin/perf
and then:
$ perf stat ls 1>/dev/null
Performance counter stats for 'ls':
1.79 msec task-clock:u # 0.827 CPUs utilized
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
112 page-faults:u # 0.063 M/sec
<not supported> cycles:u
<not supported> instructions:u
<not supported> branches:u
<not supported> branch-misses:u
0.002158900 seconds time elapsed
0.002182000 seconds user
0.000000000 seconds sys
I think it is expected the hardware/cache counters are not available on WSL2
If you follow the accepted answer, make sure you read the complains the make command prints at the start, as it might be missing some headers and disables functionality.
For me it disabled tui, gtk and demangling to name a few features.
Related
I am trying to use the perf tool of Linux on my WSL2 using Windows10 machine. I have done every step of the accepted answer here:Is there any method to run perf under WSL?
When I ran "make" comment, I got a warning:
Warning: Kernel ABI header at 'tools/include/uapi/linux/stat.h'
differs from latest version at 'include/uapi/linux/stat.h'
But the perf executable was still created.
However when I try to use perf like this:
sudo perf record -g myexe myargs
I received this error:
WARNING: perf not found for kernel 5.10.16.3-microsoft
You may need to install the following packages for this specific
kernel:
linux-tools-5.10.16.3-microsoft-standard-WSL2
linux-cloud-tools-5.10.16.3-microsoft-standard-WSL2
Then I tried to run this:
sudo apt install linux-tools-5.10.16.3-microsoft-standard-WSL2
However that did not work either, I got this:
Reading package lists... Done Building dependency tree Reading state
information... Done E: Unable to locate package
linux-tools-5.10.16.3-microsoft-standard-WSL2 E: Couldn't find any
package by glob 'linux-tools-5.10.16.3-microsoft-standard-WSL2'
What should I do now?
I need to run real time applications on Ubuntu RT Linux and was reading about ways to make linux act as RT system and I learned two ways to do it
preemptive_rt kernel patching
enabling CONFIG_RT_GROUP_SCHED flag in the kernel.
I've already tried my hands on 1st method Install RT Linux patch for Ubuntu
However, apart from uname -r showing #1 SMP PREEMPT RT I've no other proof that it is actually a RT system and hence want to try the 2nd method. Enable CONFIG_RT_GROUP_SCHED flag in the kernel and see its performance.
I read we can confirm if the kernel already has the flag by following command:
# zcat /proc/config.gz | grep RT_GROUP
CONFIG_RT_GROUP_SCHED=y
However, my system doesn't even have the config.gz file in proc, so I believe my kernel does not have this enabled.
I'm relatively new to linux kernels so this might be naive but how can I enable this in the kernel?
Step 1
Download linux kernel from https://www.kernel.org/pub/linux/kernel/. For the purpose of this PoC we downloaded linux-4.16.18.tar.gz kernel from above link.
Step 2
Unzip the kernel
$ tar -xzvf linux-4.16.18.tar.gz
Step 3
Move to kernel source directory
$ cd linux-4.16.18
Step 4
Install kernel build dependencies
$ sudo apt install git build-essential kernel-package fakeroot libncurses5-dev libssl-dev ccache bison flex
Step 5
Run kernel configuration
$make menuconfig
Step 6
Go to General setup ─> Control Group Support ─> CPU controller ─> Group scheduling for SCHED_RR/FIFO configuration as shown below:
Go to General setup ─> Kernel .config support and enable access to .config through /proc/config.gz
Step 7
Compile the kernel
$ make -j20
Make modules & install
$ sudo make modules_install -j20
$ sudo make install -j20
Step 8
Open the grub.cfg file to verify if kernel is installed
$ vim /boot/grub/grub.cfg
Look for the menuentry with menuentry 'Ubuntu, with Linux linux-4.16.18'
If it's not your default kernel then change the GRUB_DEFAULT=0 value to your kernel
Step 9
Reboot your system
sudo reboot
Step 10
Verify the system by the following command:
# zcat /proc/config.gz | grep RT_GROUP
CONFIG_RT_GROUP_SCHED=y
I knew that "perf" tool requires installation correspond to specific linux kernel versions. And I knew that all docker images run with the same linux kernel version, no matter linux distribution version.
I'm using ubuntu 16.04 inside docker and already installed linux-tools-common, start perf tell me I still lack some packages:'
# perf
/usr/bin/perf: line 32: lsb_release: command not found
WARNING: perf not found for kernel 4.9.12
You may need to install the following packages for this specific kernel:
linux-tools-4.9.12-moby
linux-cloud-tools-4.9.12-moby
You may also want to install one of the following packages to keep up to date:
linux-tools-moby
linux-cloud-tools-moby
Then I tried to install what I missed:
root#xxxxxx:/# apt-get install linux-tools-4.9.12-moby linux-cloud-tools-4.9.12-moby
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package linux-tools-4.9.12-moby
E: Couldn't find any package by glob 'linux-tools-4.9.12-moby'
E: Couldn't find any package by regex 'linux-tools-4.9.12-moby'
E: Unable to locate package linux-cloud-tools-4.9.12-moby
E: Couldn't find any package by glob 'linux-cloud-tools-4.9.12-moby'
E: Couldn't find any package by regex 'linux-cloud-tools-4.9.12-moby'
root#b2543b6e985d:/# apt-get install linux-tools-moby linux-cloud-tools-moby
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package linux-tools-moby
E: Unable to locate package linux-cloud-tools-moby
(1) So weird, but how to solve this installation problem?
(2) Plus: I'm still confused, if there's no strict match between "perf" and "kernel", how could "perf" work well inside docker of linux?
I knew that "perf" tool requires installation correspond to specific linux kernel versions. And I knew that all docker images run with the same linux kernel version, no matter linux distribution version.
Perf tool from other version of kernel still can be used (the syscalls in perf_event subsystem have good design and are compatible with older/newer tools). So, you can just find any perf binary (not the /usr/bin/perf script) anywhere, check its library depends with (ldd ..path_to_perf/perf) and copy perf inside Docker (and install libs).
Usage of hardware events, system-wide or kernel profiling may be limited in the docker, so try
perf -e cycles:u ./program (hardware counter only for user-space),
perf -e task-clock ./program (software timer for kernel and user-space),
perf -e task-clock:u ./program (software timer for user-space only),
I'm trying to build Caffe on Ubuntu 14.04 x64 in VirtualBox with openblas in CPU_ONLY mode.(Enviroment install script , Makefile.config )
Also I'm not compiling OpenBlas, but install it via apt-get like sudo apt-get -y install libopenblas-dev, can it be reason of the problem?
After I set any of this variables, there is no speed improvement and in htop I see only one CPU utilisation.
export OPENBLAS_NUM_THREADS=4
export GOTO_NUM_THREADS=4
export OMP_NUM_THREADS=4
How to check if Caffe use several threads / CPUs?
UPDATE:
I tried caffe binary on MNIST example and it utilise 400% of CPU.
1 thread
I0520 15:58:09.749832 12424 caffe.cpp:178] Use CPU.
...
I0520 16:06:14.553506 12424 caffe.cpp:222] Optimization Done.
~8 min
4 threads
I0520 16:06:44.634735 12446 caffe.cpp:178] Use CPU.
...
I0520 16:13:15.904394 12446 caffe.cpp:222] Optimization Done.
~6.5 min
ps -T -p <PID> gives me:
export OPENBLAS_NUM_THREADS=1
6 threads
export OPENBLAS_NUM_THREADS=4
9 threads
Seems openblas works, but it depends on network architecture?
Also seems Caffe also use BLAS for conv layers.
I'm using the intel branch on CentOS 7.3 and able to see 2500% CPU usage on my Broadwell when training caffenet example. The following is the steps I used to build the tools:
git clone https://github.com/BVLC/caffe.git caffe_intel
cd caffe_intel
git branch -r
git checkout intel
cp Makefile.config.example Makefile.config
# mkl will be automatically download once make is run
# Edit Makefile.config in the following lines
PYTHON_LIB := /usr/lib64
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib64 /usr/lib
yum install cmake # will be used to build mkl as a sub-routine of make
make -j32 all
make pycaffe
make distribute
cd distribute/lib
ln -s ../../external/mkldnn/install/lib/libmkldnn.so .
And put caffe_intel/distribute/bin in $PATH, caffe_intel/distribute/lib in $LD_LIBRARY_PATH. Also, enable mkl library by adding the following line at the beginning of prototxt file(s)
engine: "MKL2017"
I'm compiling erlang otp from source like following:
./configure --with-dynamic-trace=systemtap
But it reports error:
checking for dtrace... no
configure: error: No dtrace utility found.
I think systemtap is for linux usage and it is different from dtrace, am I right?
After installed dtrace Now the error is:
dtrace: failed to compile script emulator/beam/erlang_dtrace.d: Preprocessor not found
configure: error: Could not precompile erlang_dtrace.d: dtrace -h failed
configure: error: /root/otp/erts/configure failed for erts
Which distro are you using? I'd forget using DTRACE, it will be a bit harder to make it to work than Systemtap (at least it was for me). I'm using Systemtap and it works perfectly.
If you are not using Fedora/RH you should patch your kernel with UTRACE or use a kernel >= 3.5 with UPBOBES enabled.
To verify if your kernel has UTRACE/UPROBE enabled run this:
grep UTRACE /boot/config-$(uname -r)
or
grep PROBES /boot/config-$(uname -r)
For example, in my machine (Ubuntu 12.04 with Mainline kernel 3.5):
$ grep UTRACE /boot/config-$(uname -r)
$
$ grep UPROBES /boot/config-$(uname -r)
CONFIG_ARCH_SUPPORTS_UPROBES=y
CONFIG_UPROBES=y
You should also have systemtap installed. The version shipped with Ubuntu (my case) is quite old, so I compiled it from source (traditional configure/make/make install). You can get it here:
http://sourceware.org/systemtap/ftp/releases/
I'm using version 1.8.
Then try to recompile it again. It should work.
Regards
After installing systemtap(sudo apt-get install systemtap), exiting or logging out from the current shell session then logging in may help.
Also, you may try using kerl.