Is there any efficient way (maybe by abusing the gcc preprocessor?) to get a set of stripped kernel sources where all code not needed according to .config is left out?
Well got some steps into a solution.
First, one can obtain the used compiler commands by
make KBUILD_VERBOSE=1 | tee build.log
grep '^ gcc' build.log
For now, I select only one gcc command line for further steps. For example the build of kernel/kmod.c, it looks like:
gcc <LIST OF MANY OPTIONS> -c -o kernel/kmod.o kernel/kmod.c
I now remove the option -c, -o ... and add -E, thus disabling compilation and writing preprocessor output to the screen. Further I add -fdirectives-only to prevent macro expansion and -undef to remove the GNU defined macro definitions. -nostdinc to remove the standard c headers is already added by the kernel makefile.
Now includes are still included and thus expanded on the preprocessor output. Thus I pipe the input file through grep removing them: grep -v '#include' kernel/kmod.c. Now only one include is left: autoconf.h is included by the Makefile's command line. This is great as it actually defines the macros used by #ifdef CONFIG_... to select the active kernel code.
The only thing left is to filter out the preprocessor comments and the remaining #defines from autoconf.h by means of grep -v '^#'.
The whole pipe looks like:
grep -v '#include' kernel/kmod.c | gcc -E -fdirectives-only -undef <ORIGINAL KERNEL BUILD GCC OPTIONS WITHOUT -c AND -o ...> - |grep -v '^#'
and the result is a filtered version of kernel/kmod.c containing the code that is actually build into kmod.o.
Questions remain: How to do that for the whole source tree? Are there files that are actually build but never used and stripped at linking?
Kernel Minimization Script :
A project inspired by this question and providing an easy answer...
It contains a Python script that generate a minimized sources code during build time. The new minimized source tree will only contain used sources. (project page)
Info :
The script is tested working with the kernel v4.14.x, however building the kernel one more time from those generated minimized sources require to copy make files and Kconfig files etc... at least we could easily isolate only used source for investigations and development
Usage :
cd /kernel/sources
make
wget https://github.com/Hitachi-India-Pvt-Ltd-RD/minimization/raw/master/minimize.py
export PATH=$PATH:`pwd`
make C=2 CHECK=minimize.py CF="-mindir ../path-to-minimized-source-tree/"
Note & Reminder :
If we are building within and against the targeted machine, we also have the make localmodconfig command that shrink the current config file with only the currently used modules, if used before "Minimization" it will generate further more stripped sources
Compile everything and use atime to find out which files were not used. It might not be very accurate but it's probably worth a try.
Related
Long story short: I worked on relative rpath linking with this script (that uses automake, autoconf, libtool).
The problem is that the final rpath/runpath entry in the binary executable or so file still has the absolute path:
it turned out libtool is configured by default like this with hardcode_libdir_flag_spec to include any -L value if it's set in LDFLAGS
The only question remains: how and at which point (what's the proper way) can I set other libtool variables, like hardcode_minus_L. (I've searched for it on the net, but I couldn't find anything.)
I tried to do the following:
after configure is called I tried to replace the value of the variable with sed in libtool file (in the proper directory): it worked but when make is called it overwrote the whole libtool file again (it was regenerated)
Note, that 2 binary files are effected by this, entry for rpath/runpath with objdump -p:
libcurl.so : RUNPATH /home/user1/lib/rtorrent-0.9.7-1.5.3/lib:$ORIGIN/../lib
rtorrent : RUNPATH $ORIGIN/../lib:/home/user1/lib/rtorrent-0.9.7-1.5.3/lib
Thanks
I don't know if modifying the generated libtool script is the best approach to solve your problem. But if you go this way, you need to make the approach robust by executing your sed command within AC_CONFIG_COMMANDS.
The libtool script is generated during config.status as an configuration command (AC_CONFIG_COMMANDS: https://www.gnu.org/software/autoconf/manual/autoconf.html#Configuration-Commands) .
config.status: executing libtool commands
You can modify this generated file by adding another AC_CONFIG_COMMANDS.
We use following to alter the prefer_static_libs variable:
AC_CONFIG_COMMANDS([libtool-fix-linker-preference],
[${SED} -e '1,/prefer_static_libs=/ s/prefer_static_libs=.*$/prefer_static_libs=yes/' \
libtool > libtool.fix && mv libtool.fix libtool])
You need to trigger your AC_CONFIG_COMMANDS after LT_INIT.
configure/config.status reports the execution:
config.status: executing libtool commands
config.status: executing libtool-fix-linker-preference commands
Hope that helps,
Christian
It turned out it's fairly easy to modify these variables in configure.ac, no need for sed - after fiddling around and taking a look into the generated scripts. The only thing can be confusing that these variables can be applied to
tags defined in the given project.
E.g. to change hardcode_libdir_flag_spec to an empty value in rtorrent project (means it will break compilation), you would insert into configure.ac:
_LT_TAGVAR(hardcode_libdir_flag_spec, )=""
_LT_TAGVAR(hardcode_libdir_flag_spec, CXX)=""
_LT_TAGVAR(hardcode_minus_L, )=yes
_LT_TAGVAR(hardcode_minus_L, CXX)=yes
The 2nd parameter is the tag or default tag if it's empty.
I'm running through the "first kernel patch" tutorial on kernel newbies http://kernelnewbies.org/FirstKernelPatch
While running through the tutorial, i've had absolutely no issues what so ever until now, I am at a point where I am setting up my kernel configuration. I've followed the tutorial exactly as shown but the following command:
cp /boot/config-'uname -r'* .config
leaves me with the following error message in the terminal:
cp: cannot stat '/boot/config-uname -r*': No such file or directory
Is there a way I can generate this file without going through the effort of looking for it in the finder? I'd rather not go through the thousands of files there are in a kernel, it could take me forever.
It seems like your tutorial has a quotation error. instead of ' you should be using ` (backtick)
cp /boot/config-`uname -r`* .config
What it does is execute the command uname -r and place the stdout of the command in place of the command. I'd suggest using $(command) instead of `command` since it's more obvious what is going on.
cp /boot/config-$(uname -r)* .config
First things first .. You're using simple quotes which is wrong, the command is meant to use backticks (`) -- they will include the output of the command inside them:
> uname -r
3.16.1-ck1
> echo /boot/config-`uname -r`
/boot/config-3.16.1-ck1
So this could already solve your problem.
If this file isn't present on your system, you have some alternatives:
If you have the source the running kernel is built from, the kernel config is the file .config there.
Although most packaging/installation systems copy the kernel config to /boot/config-`uname -r`, some just copy it to /boot/config (without version suffix)
The kernel can be built to serve it's config in /proc/config.gz (gzip compressed)
If really neither of these succeed, you're out of luck and your only option is get hold of the source package your kernel is built from.
I have few header files in /my/path/to/file folder. I know how to include these files in new C program but everytime I need to type full path to header file before including it. Can I set some path variable in linux such that it automatically looks for header files ?
You could create a makefile. A minimal example would be:
INC_PATH=/my/path/to/file
CFLAGS=-I$(INC_PATH)
all:
gcc $(CFLAGS) -o prog src1.c src2.c
From here you could improve this makefile in many ways. The most important, probably, would be to state compilation dependencies (so only modified files are recompiled).
As a reference, here you have a link to the GNU make documentation.
If you do not want to use makefiles, you can always set an environment variable to make it easier to type the compilation command:
export MY_INC_PATH=/my/path/to/file
Then you could compile your program like:
gcc -I${MY_INC_PATH} -o prog src1.c src2.c ...
You may want to define MY_INC_PATH variable in the file .bashrc, or probably better, create a file in a handy place containing the variable definition. Then, you could use source to set that variable in the current shell:
source env.sh
I think, however, that using a makefile is a much preferable approach.
there is a similar question and likely better solved (if you are interested in a permanent solution): https://stackoverflow.com/a/558819/1408096
Try setting C_INCLUDE_PATH (for C header files) or CPLUS_INCLUDE_PATH (for C++ header files).
Kudos:jcrossley3
I'm not in Linux right now and I can't be bothered to reboot to check if everything's right, but have you tried making symbolic links? For example, if you are on Ubuntu:
$ cd /usr/include
$ sudo ln -s /my/path/to/file mystuff
So then when you want to include stuf, you can use:
#include <mystuff/SpamFlavours.h>
I have noticed nmake.exe limits its inference-rule search to one missing file. I find no mention of the problem on the Web. Am I missing something?
$ cat Makefile
.SUFFIXES: .a .b .d .e
all: abc.e
.a.b:
copy $** $#
.b.d:
copy $** $#
.d.e:
copy $** $#
$ touch abc.a
$ nmake
NMAKE : fatal error U1073: don't know how to make 'abc.e'
Stop.
$ nmake -n abc.a
'abc.a' is up-to-date
$ nmake -n abc.b
copy abc.a abc.b
$ nmake -n abc.d
NMAKE : fatal error U1073: don't know how to make 'abc.d'
Stop.
This same Makefile produces tbe following with GNU make:
$ make -n
copy abc* abc.b
copy abc* abc.d
copy abc* abc.e
rm abc.b abc.d
Of course, the $** macro and copy command aren't as useful in with GNU make. ;-)
Does your version of nmake.exe handle this any better? Is there a magic switch? Or is it really as broken as it seems?
The problem here is tracking multi-step operations in your build process. Your source files produce intermediate files of some sort which in turn produce the final build output. In a bad universe, you might make changes to a source file and then your final binaries could still be built from stale versions of the intermediate files. Obviously, that would be bad.
GNU make takes the approach of modeling the entire dependency tree at once, and tracing the modified files all the way through to the output. This is great if make is the only build tool you use. This doesn't work well if you have very large projects, so you need to make them in a specific order. This doesn't work well if 'make' doesn't support some tools of your build process, so you would need to run make multiple times anyway.
nmake.exe takes the approach of doing the simplest possible thing: Only doing one pass at a time. It assumes it will be part of a larger tool chain. So if you have multi-pass dependencies, you will need multiple passes of nmake. If your build process requires more than 3 passes, you are probably doing A Bad Thing and you should fix your process. And for crying out loud, if you need multiple passes, just write a script to do it.
What is the usage of the -I and -L flags in a makefile?
These are typically part of the linker command line, and are either supplied directly in a target action, or more commonly assigned to a make variable that will be expanded to form link command. In that case:
-L is the path to the directories containing the libraries. A search path for libraries.
-l is the name of the library you want to link to.
For instance, if you want to link to the library ~/libs/libabc.a you'd add:
-L$(HOME)/libs -labc
To take advantage of the default implicit rule for linking, add these flags to the variable LDFLAGS, as in
LDFLAGS+=-L$(HOME)/libs -labc
It's a good habit to separate LDFLAGS and LIBS, for example
# LDFLAGS contains flags passed to the compiler for use during linking
LDFLAGS = -Wl,--hash-style=both
# LIBS contains libraries to link with
LIBS = -L$(HOME)/libs -labc
program: a.o b.o c.o
$(CC) $(LDFLAGS) $^ $(LIBS) -o $#
# or if you really want to call ld directly,
# $(LD) $(LDFLAGS:-Wl,%=%) $^ $(LIBS) -o $#
Even if it may work otherwise, the -l... directives are supposed to go after the objects that reference those symbols. Some optimizations (-Wl,--as-needed is the most obvious) will fail if linking is done in the wrong order.
To really grok a makefile, you need to also have a good understanding of the command lines for all of the components of your project's toolchain. Options like -I and -L are not understood by make itself. Rather, make is attempting to create a command line that will execute a tool to transform a prerequisite file into a target file.
Often, that is a C or C++ source file being compiled to an object file, and eventually linked to get an executable file.
In that case, you need to see the manual for your compiler, and especially the bits related to the command line options it understands.
All that said in generic terms, those specific options are pretty standard among compilers and linkers. -I adds a directory to the list of places searched by the compiler for a file named on a #include line, and -L adds a directory to the list of places searched by the linker for a library named with the -l option.
The bottom line is that the "language" of a makefile is a combination of the syntax of the makefile itself, your shell as known to make (usually /bin/sh or something similar), common shell commands (such as rm, cp, install, etc.), and the commands specific to your compiler and linker (e.g. typing gcc -v --help at your shell prompt will give you a nearly complete (and extremely long) list of the options understood by gcc as one starting point).
One thing to note is that these are the options passed to the compiler/linker.
So you should be looking at the compiler man pages/documentation to know their role.