My main goal is that I want to understand how parameters works in ./configure. So, I decided to install ncurses and vim in $HOME/local/, so, as to not affect my system.
while configuring vim, I looked into --with-local-dir and includedir. I want to know what is the difference between them.
When I looked into vim-git-INSTALL-script, it said:
Unix: LOCAL HEADERS AND LIBRARIES NOT IN /usr/local
Sometimes it is necessary to search different path than /usr/local for locally
installed headers (/usr/local/include) and libraries (/usr/local/lib).
To search /stranger/include and /stranger/lib for locally installed
headers and libraries, use:
./configure --with-local-dir=/stranger
I thought that was the purpose of includedir. I would appreciate if someone can tell me where I went wrong. Thank you!
Conventionally (and per documentation), includedir is used for the directory where the current program would install header files. That's not the same as the places where the configure script might have to tell the compiler to find the header files needed to compile the program.
I'm using linux3.3 and recently while building busybox for a new command, found the added busybox source file uses linux kernel headers.
So I looked up the internet and did 'make headers_install ARCH=.. CROSS_COMPILE=.. INSTALL_HDR_PATH=..' to extract headers usable for the user space program.
Then I used the new header files instead of files under sparc-snake-linux/sys-include.
But I had to copy over some missing files from the sys-include to the new header directories and had to copy some missing definitions from the sys-include files to the corresponing file in the new header files.(somewhere on the internet I read this 'make headers_install' was not upgraded after linux2.6 or so)
Is this what I am supposed to do? (why are there some missing files? I geuss it's because the 'make headers_install' is not well-maintained and doesn't work well for versions later than 2.6? Am I correct?)
Using this method, I have removed tens of 'undefined' errors but now I see some definitions conflict between files from sparc-snake-linux/sys-include (of course new cleaned and beefed-up version) and sparc-snake-linux/include. What version should be used?
And if I succeed compilation(by fixing header problems), do I have to build the glibc again with this new header files? (I'm afraid it's so. I'm using glibc for busybox)
any help would be deeply appreciated.
Thanks
Chan
ADD : I've extracted the new header files using above command and built busybox with new added command(route and other IP related functions). It works fine and the reason it didn't work was I had the variable KERNEL defined for busybox which should not be done(because busybox is not kernel code, but user program).
I've extracted the new header files using above command and built busybox with new added command(route and other IP related functions). It works fine and the reason it didn't work was I stupidly had the variable _KERNEL_ defined for a while for busybox which should not be done(because busybox is not kernel code, but user program).
when you use
echo "" | arch-abc-linux-gcc -o /tmp/tmp.o -v -x c -
you can see what the standard include path is. If the cross compiler is for compiling application on linux (like the one above), it will have linux system header path in the standard include path. Replace that with the new extracted header path. What I did was to use -nostdinc option and provide include path explicitly.
I want to add some debug info or printf in the random.c in order to look deeply into the Linux random number generator. The entropy in /dev/random and /dev/urandom are both generated by random.c. My questions are:
1. Where I can find the random.c file in Linux 2.6.32?
2. What is the best way to add my modification of random source code into the kernel? Is it OK to just compile random.c and load it as loadable kernel module? Or do I have to recompile and install the kernel to make the new random.c with debug msg somehow take effect? The key point is to make sure that only one copy of random number generator is running in the kernel.
Thank you. Any kind of suggestion is highly appreciated.
random.c is linked directly into the kernel, it isn't built as a module, so you can't just recompile it alone and load it into your kernel, you need to recompile the whole new kernel.
To build the kernel, make sure you have the usual development tools installed: gcc, GNU make, etc. Some distros provide a "build-essentials" or "Development Tools" or similar metapackage that include all of the usual development tools for building the core system packages.
How you build your kernel depends on whether you have any distribution specific patches that are needed to use your system, or if you want to ensure that you use your distro's packaging system to install the kernel. If so, you should probably follow your distro's instructions for building the kernel. For example, Ubuntu's instructions, Arch's instructioins, Fedora's instructions, CentOS instructions (likely similar on RHEL 6, Red Hat doesn't provide documentation as they don't support building custom kernels), SuSE instructions.
Otherwise, if you don't mind configuring and installing your kernel manually, you can do it by hand. The following instructions should cover most distros reasonable well, but be sure to check your distro docs in case there are any distro-specific gotchas.
Download the appropriate tarball from kernel.org and decompress it somewhere. Or if you prefer, check it out using Git. Since you reference 2.6.32, I've included the latest version of 2.6.32 in the below instructions.
$ curl -O https://www.kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32/linux-2.6.32.61.tar.xz
$ xzcat linux-2.6.32.61.tar.xz | tar xvf -
$ cd linux-2.6.32.61
# or...
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
$ cd linux
$ git checkout -b my-branch v2.6.32.61
Now you need to do to configure it, build it, and install it. Greg Kroah-Hartmann, a leading kernel developer and stable kernel maintainer, has a free book on the subject. I'd recommend reading his book, but if you want a quick rundown, I'll summarize the highlights.
There are several ways to configure it. A good way to start is to just copy your current config in, and then run make menuconfig or make xcconfig to get a curses or graphical kernel configuration utility that allows you to easily browse and choose the right options (as there may be new options in the new kernel that you are building). Many distros install the config for a given kernel in /boot/config or /boot/config-version corresponding to the kernel version. Copy that into your source tree as .config, and then run make menuconfig or make xconfig:
$ cp /boot/config .config
$ make xconfig
After configuring it, I'd recommend adding something to the EXTRAVERSION definition in the Makefile. The contents of that are tacked on to the version, to help distinguish your modified kernel from the upstream one. I'd recommend setting it to help keep track of which is your modified kernel.
Once it's configured, just build it like anything else. I recommend using -j to run a parallel build if you have multiple cores.
$ make -j8
Now it's built, and you can install it. On most systems, the following works; if not, check out Greg's book or check your distro's documentation:
$ sudo make modules_install
$ sudo make install
And finally you have to add it to your bootloader (on some systems, make install may do this, on some it may not). Depending on whether you use Lilo, Grub, or Grub2, you may need to edit /etc/lilo.conf (followed by running sudo lilo to install the changes), /boot/grub/menu.lst, or /boot/grub/custom.cfg (followed by sudo grub-mkconfig -o /boot/grub/grub.cfg to install the changes). See the relevant documentation for the given bootloader for more details. Generally you want to copy an existing entry and duplicate it, updating it to point to your new kernel. Make sure you leave the existing entries, so you will be able to boot back into your old kernel if this doesn't work.
Now reboot, select your new kernel, and hope your system boots. Woo! You've built your own kernel.
Now that you've made sure you can do that successfully without modifications, you can make your change. You are going to want to modify drivers/char/random.c. To print out debugging statements, use printk(). It works mostly like printf(), though it's not exactly the same so check out the documentation before using it. After you modify, rebuild, and reinstall your new kernel, and reboot into it, you can see the messages printed out with printk() statements using the dmesg command.
For more information, check out Greg's book that I linked to above, the kernel README, and HOWTO, browse around the kernel's Documentation directory, and various other docs.
If you look at the Makefile for it, char driver is not meant to be compiled as a module (random.o is included as obj-y in drivers/char/Makefile).
You can read more about how to kbuild (kernel build) system works from: https://www.kernel.org/doc/Documentation/kbuild/makefiles.txt
Particularly section --- 3.1 Goal definitions touches this topic.
Generally you can search for files in kernel sources from source cross references (called LXR's). One is for example provided in http://lxr.free-electrons.com/
Indeed, you can add your modifications to the drivers/char/random.c, and recompile the char driver. After that you will have to rebuild the kernel, so that it will link also your new random.o to the kernel. And then you will have to boot that kernel, that process will depend on your distribution.
Most distributions have good/decent instructions around how to recompile/boot your own kernel.
I've got a problem with shared libraries and gcc. At first I couldn't run my compiled program because I was getting the following error: gcc error while loading shared libraries.
I did some searching and found that this is because the shared library cannot be found. However I had already identified that the shared library is in /usr/local/lib, which AFAICT is a commonly used directory for shared libraries and should work from the get go.
I read that you can set LD_LIBRARY_PATH, which worked for me. However I do not wish to set this each time I want to run my program.
Further searching suggested editing ld.so.conf. When I looked in this it had the following:
include /etc/ld.so.conf.d/*.conf
Looking in the ld.so.conf.d directory shows me a range of files, including libc.conf. Inside this file is the following:
/usr/local/lib
So my question is, why do I need to manually set LD_LIBRARY_PATH when the ld.so.conf appears to use the libc.conf which includes /usr/local/lib?
Is there something that I'm missing here that must be configured first? Is there an option at compile time that I'm missing?
I should note that to compile, I had to specify the path to the library, I don't know if this is a symptom of my problem or normal behaviour.
I should also note that this is a concern for me for when I deploy my software on other systems. I would have thought that I should be able to put the .so in the appropriate place and install my program without messing with ld.so.conf.
I hope this is the proper forum for this question, I read the FAQ and I think it's ok.
Cheers.
You should run ldconfig (as root) after every change of the directories configured via /etc/ld.so.conf or under /etc/ld.so.conf.d/, in particular in your case after every update inside /usr/local/lib (e.g. after every addition or update of some shared libraries there).
I'm developing kernel modules right now, and the build times are starting to get under my skin. As a side effect I'm taking way too many "coffee" breaks during builds.
So I was looking for a way to build only the stuffs I need for my platform. Chapter 7 and 8 of "linux kernel in a nutshell" gave a good detail of how to do that by hand. Its a good read : http://www.kroah.com/lkn/
But Although I understand the stuffs, this is still a lot of tweaks to make that work.
2.6.32 and later kernels added a new target make localmodconfig. which scans through lsmod and change the .config appropriately. So I thought I found my "automation". But this perl script has some problem too.
This thread describes the problems : https://bbs.archlinux.org/viewtopic.php?pid=845113
There was also a proposed solution which apparently worked for others , is to run the script directly instead of using make's target.
Although for me, make localmodconfig does not work at all. its because of the following :
make clean
make mrproper
cp /boo/config-'uname -r' .config
make localmodconfig
and it halts with
vboxguest config not found!!
nf_defrag_ipv6 config not found!!
vboxsf config not found!!
vboxvideo config not found!!
The thing is my kernel development environment is inside virtualbox. These vbox modules were installed when I chose to install "virtualbox guest addtion".
And the netfilter module might be a Distribution specific module(Lot of netfilter modules are not part of the mainline kernel, so its not a shock to me), which is not included in mainline kernel.
Now the workaround this obviously unloading these module and trying again. But I'm thinking if there is patch for the streamline_config.pl which will enable the user to exclude certain modules if s/he wants. Problem is I have zero knowledge about perl and I like it that way.
So my problems in nutshell
Patching streamline_config.pl so I can give a list of module name as argument which it will exclude from processing the config file.
The script is located at kernel.org
EDIT: Removed the stuff about perl script not running. As mugen kenichi pointed out (How dumb I can be?). But still make localmodconfig does not work because of not having some modules code under source tree. patching streamline_config.pl still valid requirement.
Anyone else trying to build a minimum kernel image also looking for reducing build time, should do the following:
1) copy distribution kernel config in your source tree. It can be done with either command given below:
$zcat /proc/config.gz > .config
or
$cp /boot/config-'uname -r' .config
2) Use localmodconfig target.
$make localmodconfig
It will use lsmod to find which modules are loaded at this moment. Then it will search through distribution's .config to enable them and disable others.
Its important to know that it does not always work flawlessly. So you should tweak your config further using make menuconfig. You will see some modules are still marked to be built which is in reality unnecessary for your system.
Sometimes out of tree modules may cause make localmodconfig to fail. If that's the case you can work around that issue in two ways :
a) unload the out of tree modules and try make localmodconfig again.
b) Run the perl script directly:
$chmod +x script/kconfig/streamline_config.pl
$perl script/kconfig/streamline_config.pl > .config
3) Install ccache[1]. It will improve your build time dramatically. It caches objects. So it will reduce subsequent builds.
Its possible that ccache is included in the distribution's repository so that you can install it through apt-get or yum. In CentOS its available in EPEL repo.[2]
4) Give as many cores as possible for the build job
$make -j8 CC="ccache gcc"
My results are :
real 3m10.871s
user 4m36.949s
sys 1m52.656s
[1] http://ccache.samba.org/
[2] http://fedoraproject.org/wiki/EPEL