I have a pc with ubuntu 14.04,the configuration is:
CPU:Intel® Core™ i7-4790 CPU # 3.60GHz × 8
Memory:16GB
A server with ubuntu-14.04-server,the configuration is:
CPU:Intel(R) Xeon(R) CPU E5-2603 v2 # 1.80GHz x 4
Memory:32GB
And I was trying to make Android by the following command:
make -j
Then the computer halt...
So how to specify the value of 'j' to ensure making fastest?
I suppose the value should be the numbers of cpu's processors...
make -j launches as many jobs in parallel as the build dependencies allow - in the kernel build case that's way too many for most consumer class machines, the system becomes barely responsive and eventually runs out of PIDs or memory.
You need to add a number to that -j. Which brings you to this Q&A:
https://unix.stackexchange.com/questions/208568/how-to-determine-the-maximum-number-to-pass-to-make-j-option
If you want to go into fine tuning you might find the BuildIn tool useful (disclaimer - I'm the author): https://apartsw-buildin.appspot.com/
Related
hope someone knows the answer to this...
I have a code that compiles perfectly well with openMP (it uses libsharp). However, I am finding it impossible to make the M1 Pro chip use all the 8 or 10 cores I have.
I am setting the threads variable correctly as export OMP_NUM_THREADS=10 such that the code correctly identifies it's supposed to be running with 10 threads (see image below showing a print-screen from my activity monitor):
Activity Monitor Print Screen
Print screen is showing that the code is compiled for Apple Silicon, uses 10 threads but not much of the CPU available.
Does anyone know how to properly compile/set the number of threads such that all the cores will be used?
This is trivial in x86 architectures.
Not really an answer, but long for a comment...
If both LLVM and GCC behave the same then it's not an OpenMP runtime issue. (And your monitor output shows that the correct number of threads have been created). I'm also not certain that it's really an Arm issue.
Are you comparing with an Apple x86 machine (so running the same operating system), or with a Linux x86 system?
The scheduling decisions of the two OSes are likely different, and (for instance) MacOS has no interface to bind threads to logicalCPUs.
As well as that, there's the issue of having some fast and some slow cores. That could mean that statically scheduled loops are inefficient.
I'm also confused by the fact that you arm to show multiple instances of your code running at the same time, so you are explicitly causing over-subscription of the logicalCPUs...
I'm trying to achieve this on a Raspberry Pi 4:
Less then 10 seconds boot time
Custom UI ( build with python/Gtk for example )
I already tried editing the /boot/config.txt with
# Disable the rainbow splash screen
disable_splash=1
# Disable bluetooth
dtoverlay=pi3-disable-bt
#Disable Wifi
dtoverlay=pi3-disable-wifi
dtoverlay=sdtweak,overclock_50=100
# Set the bootloader delay to 0 seconds. The default is 1s if not specified.
boot_delay=0
# Overclock the raspberry pi. This voids its warranty. Make sure you have a good power supply.
force_turbo=1
Also I have an A2 class 10 SdCard and I have disabled almost every service that I don't need.
My best results are 14 seconds boot time until my app started, but I need to make it faster....
I would like to achieve something like this in terms of speed, but using GTK, not QT:
https://www.youtube.com/watch?v=TTcP3xeLrEY
I am also currently trying to reach fast boot time with my Raspberry Pi 4, but I cannot breach the wall of 10s without radical solutions such as buildroot or unikernel.
What do you need on your system ? On Raspbian you can use those two commands to verify what is slowing down the boot process :
systemd-analyze critical-chain
and
systemd-analyze blame
Critical chain will show in red the services blocking the boot.
On my setup I could desactivate all those services :
hciuart #GPIOs
nmbd
smbd
wpa_supplicant
systemd-timesyncd
rpi-eeprom-update.service
raspi-config.service
networking
Just use sudo systemctl disable SERVICE , I guess there are others that are not critical for your application.
Also, in the cmdline.txt file in /boot you can add quiet fastboot , it will decrease boot time further
GTK is a toolkit coded in C and callable from C code.
In practice, Python can be 10x slower than equivalent C code.
Did you consider starting Xorg with a minimal window manager and a single client coded in C? Without a single line of Python? Of course, read more about X windows systems protocols and architecture then about EWMH.
So, remove the Python interpreter on your RaspBerryPi (probably /usr/bin/python) and replace every old Python script by an equivalent ELF executable coded in C or in C++ and compiled with optimizations (so gcc -O3 with GCC). See also linuxfromscratch.org for inspiration and guidance.
I would like to achieve something like this in terms of speed, but using GTK
Of course, you want to use a GCC cross-compiler.
If that still does not work (after weeks of effort), replace your hardware by something more powerful.
It would be easier for you to install a Linux distribution such as Debian on your development laptop or desktop. Because most of what you would learn on your development computer (e.g. Advanced Linux Programming) can be applied to the RaspBerrry Pi. And cross-building on a Debian laptop for a Raspbian RaspBerryPi is really easy. Focus on learning low-level things starting from their C code. E.g. code your GUI directly with Xlib (no GTK, no Qt) in C. The same C code is very likely to be easily portable -if you write it carefully- from x86/Debian to your Raspberry Pi.
You could read the From powerup to to bash prompt howto; most of that document is relevant for a RaspBerryPi running RaspBian.
You could even avoid starting any Xorg or other display server (e.g. WayLand) on your RaspBerryPi. That certainly would make it boot faster.
You could replace your /sbin/init program by your own one coded in C entirely. That process starts within less than a second, and forks all other processes. Your custom GUI could be just Xorg with your single X11 client coded in C.
Observe that you could have the source code (millions of source lines) of all the code running on RaspBian. Then download it, study it, and optimize it. Of course this could take decades.
First, this is a poorly formed question. You've provided no details on what your boot process is or what OS/configuration you're running with for people to help.
Regardless, you appear primarily focused on a short boot time. So since your question is generic... here is a generic answer: there is nothing magical about the boot process. There is no config file with a fast_boot=1 option that gets you up in 0 seconds instead of 15. You want to boot faster? Do less.
Throw out slow crap like Python
Remove unnecessary application loading from your init system (sysvinit/systemd/whatever)
Remove unnecessary driver loading from the kernel startup. Start by trimming down your device tree to only the hardware you really need to initialize.
Optimize your bootloader (u-boot?) to only initialize the hardware you really need (obviously turn off any prompt and timer it may implement)
That's how you boot faster.
I have core i5 with 8gb RAM.
I have VMware workstation 10.0.1 installed on my machine.
I have fedora 20 Desktop Edition installed on VMware as guest OS.
I am working on Linux kernel source code v 3.14.1. I am developing an I/O scheduler for Linux kernel. After any modifications in code every time it takes around 1 hour and 30 minutes for compiling and installing the whole kernel code to see the changes.
Compilation and Installation commands:
make menuconfig,
make,
make modules,
make modules_install,
make install
So my question is it possible to reduce 1 hour and 30 minutes time into only 10 to 15 minutes?
Do not do make menuconfig for every change you make to the sources, because it will trigger a full compilation of everything, no matter how trivial your change is. This is only needed when the configuration option of the kernel changes, and that should sheldom happen during your development.
Just do:
make
or if you prefer the parallel compilation:
make -j4
or whatever number of concurrent tasks you fancy.
Then the make install, etc. may be needed for deploying the recently built binaries, of course.
Another trick is to configure the kernel to the minimum needed for your tests. I've found that for many tasks a UML compilation (User Mode Linux) is the fastest. You may also find useful make localmodconfig instead of make menuconfig to start with.
Use make parallel build with -j option
Compile for the target architecture only, since otherwise make will build the kernel for every listed architecture.
i.e. for eg instead of running:
make
run:
make ARCH=<your architecture> -jN
where N is the no of cores on your machine (cat /proc/cpuinfo lists the no of cores). For eg, for i386 target and host machine with 4 cores (output of cat /proc/cpuinfo):
make ARCH=i386 -j4
Similarly you can run the other make targets (modules, modules_install, install) with -jN flag.
Note: make does a check of the files modified and compiles only those files which have been modified so only the initial build should take time, subsequent builds will be faster.
make -j will make use of all available CPUs.
You do not need to run make menuconfig again every time you make a change — it is only needed once to create the kernel .config file. (Or possibly again if you edit Kconfig files to add or modify configuration options, but this certainly shouldn't be happening often.)
So long as your .config is left alone, running make should only recompile files that you changed. There are a few files that must be compiled every time, but the vast majority are not.
ccache should be able to dramatically speed up your compile times. It speeds up recompilation by caching previous compilations and detecting when the same compilation is being done again. Your first compilation with ccache will be slower since it needs to populate the cache, but subsequent builds should be much faster.
If you don't want to fuss with ccache configurations you can just run it like so to compile the kernel:
ccache make
Perhaps in addition to the previous suggestions, while using ccache, you might want to unset CONFIG_GCC_PLUGINS (if it was set) otherwise you may get a lot of cache misses, as seen in this example.
Perhaps in addition to the previous suggestions, using ccache software (https://ccache.samba.org/) and a compilation directory on SSD drive should drastically decrease the compilation time.
If you have suffitient RAM and you wont be using your machine while the kernel is being built u can spawn a large number of concurrent jobs. But make sure your RAM is sufficient otherwise your system will hang and crash.
Use this command:
sudo make -j 4 && sudo make modules_install -j 4 && sudo make install -j 4
Where 4 is the number of cores I have alloted to working on this process.
Credits
Simple trick. If you don't use your own machine or have another one, you can log out completely and switch to a TTY terminal using CTRL + ALT + F*. Everything is much much faster.
I am doing project on Pandaboard using Embedded Linux (UBUNTU 12.10 Server Prebuild image) to optimize boot time. I need techniques or tools through which I can find boot time and techniques to optimize the boot time. If anyone can help.
Just remove application which is not required from /etc/init.d/rc file also put echo after every process initialization and check which process is taking much time for starting,
if you find application which is taking more time then debug that application and so on.
There is program that can be helpful to know the approximate boot-up time. Check this link
Time Stamp.
First of all the best you have to do is to compile yourself your own made kernel, get the source on the internet and do a make xconfig and then unselected everythin you don't need.
In a second time create your own root filesystem using Buildroot and make xconfig to select/unselect everything you need or not.
Hope this help.
I had the same problem and do that way, now it's clearly not the same ;)
EDIT: Everything you need will be here
to analyze the boot process, you can use Bootchart2, its available on github: https://github.com/mmeeks/bootchart
or Bootchart, from the Ubuntu packages:
sudo apt-get update
sudo apt-get install bootchart pybootchartgui
There are broadly 3 areas where you can reduce boot time
Bootloader:
Modify the linker script to initialize only the required h/w. Also, if you are using an SD card to boot, merge kernel and bootloader image to save time.
Kernel:
Remove unwanted modules from kernel config. Also try using compressed and uncompressed image. If your CPU is good enough to handle it go compressed image and check uncompression time required for different compression types.
Filesystem:
FS size can be significantly reduced by removing the unwanted bins and libs. Check for dependencies and use only the one's that are required.
For more techniques and information on tools that help in measuring the boot time please refer to the following link.
Refer to Training Material
The basic rule is: the fastest code is code that never gets loaded and
run, so remove everything you don't need:
in U-Boot: don't load and run the full U-Boot at all; use FALCON
mode and have the SPL load the Linux kernel and DTB directly
in Linux: remove all drivers and other stuff you don't really need;
load all drivers that are not essential for your core application as
modules - and load them after your application was started. If you
take this serious, you may even want to start only one CPU core
initially (and start the remaining ones after your application is
running).
in user space: minimize the size of the root file system. throuw
out anything you don't need; configure tools (like busybox) to
contain only the really needed functionality; use efficient code
(for example, link against musl libc instead of glibc) etc.
What can be acchieved by combining all these measures can be seen in
this video - and yes, the complete code for this optimization is
available here.
Optimizing embedded Linux Boot process , needs modifications in three level of embedded Linux design.
Note: you will need the source codes of bootloader and kernel
Boot : the first step in optimizing and reducing boot time of board is optimizing boot loader. first you should know what is your bootloader is. If your bootloader is an opensource bootloader like u-boot than you have the opportunity to modify and optimize it. In u-boot we have a procedure that we can skip unnecessary system check and just upload kernel image to ram and start. the documentation and instruction for this is available in u-boot website. by doing this you will save about 4 ~ 5 second in boot.
Kernel : for having a quicker kernel , you should optimize kernel in many sections. for editing you can use on of Linux config menu. I always use a low graphic menu. it need some dependency you can use it by this command:
$ make menuconfig
our goal for Linux kernel is to have smaller kernel image and less module to load in boot. first change the algorithm of compression from gzip to LZO. the point of this action is gzip algorithm will take much time to extract kernel. by using LZO we have a quicker kernel decompression process. the second , disable any unnecessary driver or module that you don’t have it on your board or you don’t use it any more. by doing this , you will lose some device access and cannot use them in Linux but you will have two positive points: less Ram usage , quicker boot time.
but please remind that some driver are necessary for Linux and by disabling them you will lose some of main features (for example if you disable I2C driver in Linux you will no longer have a HDMI interface) that you need or in worst case you will have a boot problem (such as boot-loop). The third is to disable some of unusable filesystem to reduce kernel size and boot time. The Fourth is to remove some of compression algorithm to have smaller kernel image.
the last thing , If you are using a u-boot bootloader create a uImage instead of zImage. the following steps , are general and main actions , for having quicker boot as 1 second after power attach you should change more option.
after two base layer modifications, now we should optimize boot process in user-space (root file system). depend on witch system are you using , we have different changes to do. in abstract root file system of Linux that have necessary package and system to boot Linux we should use systemd instead of Unix systemv , because systemd have a multi-task init. system and it is faster , after that is udev that you should modify some of loading modules. if you have a graphical user-interface , we can use an easy trick to have a big boot time reduction by initing GUI first and load other module after loading GUI.
if you do all of following tasks , you can have quick boot time and fast system to work with.
I'm a little stuck here.
The idea is that I'd like to get a file of every instruction run by a program during it's execution. I'd like to do it with just the executable in hand (no source) and be able to determine what operation is occuring on what address when.
For example, I'd like to be able to run it on Google Chrome, Firefox, etc.
I want to use this for a performance prediction system I'm working on. I figure if I'm able to obtain each instruction that is executed in order it is executed on system 1, I can attempt to simulate/model the run time of an identical program being run on system 2, because I'll be able to predict(although I know not with 100% accuracy) L1/L2 cache-misses, L1/L2 cache-hits, TLB hits/misses, page faults, time taken on floating point multiplication operations, etc.
I'd like to try to do this on two different systems:
System 1: Ubuntu 10.10 on Intel Core 2 Duo CPU
System 2: Ubuntu 12.04 on system with 2x AMD Sixteen Core Opteron model 6274
(I can definitely change the OS's as neccessary, but would prefer to stay with Ubuntu, if possible)
Is this possible / how could I go about doing it? I know with debuggers, you can use them to step through everything, but I don't have the source available.
I think, you can use qemu (or even bochs) or valgrind to monitor every executed instruction. They are x86 binary translation tools (excluding bochs - which is an interpreter of x86 code). There is a valgrind tool called cachegrind (+ kcachegrind gui), which is ready to emulate cache by instrumenting every memory access and simulating some L1/L2 cache model (sizes may be configured via command line options).
To get deeper (into pipeline) you may want to look on free ptlsim (http://www.ptlsim.org/)