Linux kernel is written for compiling with gcc and uses a lot of small and ugly gcc-hacks.
Which compilers can compile linux kernel except gcc?
The one, which can, is the Intel Compiler. What minimal version of it is needed for kernel compiling?
There also was a Tiny C compiler, but it was able to compile only reduced and specially edited version of the kernel.
Is there other compilers capable of building kernel?
An outdatet information: you need to patch the kernel in order to compile using the Intel CC
Download Linux kernel patch for Intel® Compiler
See also Is it possible to compile Linux kernel with something other than gcc for further links and information
On of the most recent sources :http://forums.fedoraforum.org/showthread.php?p=1328718
There is ongoing process of committing LLVMLinux patches into vanilla kernel (2013-2014).
The LLVMLinux is project by The Linux Foundation: http://llvm.linuxfoundation.org/ to enable vanilla kernel to be built with LLVM. Lot of patches are prepared by Behan Webster, who is LLVMLinux project lead.
There is LWN article about the project from May 2013
https://lwn.net/Articles/549203/ "LFCS: The LLVMLinux project"
Current status of LLVMLinux project is tracked at page http://llvm.linuxfoundation.org/index.php/Bugs#Linux_Kernel_Issues
Things (basically gcc-isms) already eliminated from kernel:
* Expicit Registers Variables (non-C99)
* VLAIS (non C99-compliant undocumented GCC feature "Variable length arrays in structs") like struct S { int array[N];} or even struct S { int array[N]; int array_usb_gadget[M]; } where N and M are non-constant function argument
* Nested Functions (Ada feature ported into C by GCC/Gnat developers; not allowed in C99)
* Some gcc/gas magic like special segments, or macro
Things to be done:
* Usage of __builtin_constant_p builtin to implement scary magic like BUILD_BUG_ON(!__builtin_constant_p(offset));
The good news about LLVMLinux are that after its patches kernel not only becomes buildable with LLVM+clang, but also easier to build by other non-GCC compilers, because the project kills much not C99 code like VLAIS, created by usb gadget author, by netfilter hackers, and by crypto subsystem hackers; also nested functions are killed.
In short, you cannot, because the kernel code was written to take advantage of the gcc's compiler semantics...and between the kernel and the compiled code, the relationship is a very strong one, i.e. must be compiled with gcc...Since gcc uses 'ELF' (Embedded Linking Format) object files, the kernel must be built using the object code format. Unless you can hack it up to work with another compiler - it may well compile but may not work, as the compilers under Windows produces PE code, there could be unexpected results, meaning the kernel may not boot at all!
Related
I am trying to package some native libraries for inclusion into a java natives .jar. Right now, we are targeting 32-bit and 64-bit linux and windows, with macosx upcoming (which would yield a total of 6 variations). In addition, we have some naming problems which would be resolved if we could roll up several small libraries into one big one.
My goal is to convert
my_library.so dependencyA-55.so dependencyB-50.so
into
my_library_without_dependencies.so
I have full (C and C++) sources for dependencyA and dependencyB; however, I would much rather not to meddle in their compilation, as it is quite complex (ffmpeg). I am trying to pull this off using gcc 4.6 (ubuntu 12.04 64-bit), and the solution, if found, should ideally work for 64-bit and 32-bit linux, and 64-bit and 32-bit windows architectures (cross-compiling via mingw32).
Is there any magic combination of linker options that would cause GCC to subsume the dependencies into a single final shared library?. I have looked intently at the linker options without success, and related SO questions do not address this use-case.
Its not possible.
Shared objects are already a product of linker and in the form of ready to execute.
Instead you can create static libraries as "dependencyA.a" and "dependencyB.a"
( as you have source code ) and use "--whole-archive" linker switch while creating "my_library.so"
how can I obtain runtime information about which version of kernel is running from inside linux kernel module code (kernel mode)?
By convention, Linux kernel module loading mechanism doesn't allow loading modules that were not compiled against the running kernel, so the "running kernel" you are referring to is most likely is already known at kernel module compilation time.
For retrieving the version string constant, older versions require you to include <linux/version.h>, others <linux/utsrelease.h>, and newer ones <generated/utsrelease.h>. If you really want to get more information at run-time, then utsname() function from linux/utsname.h is the most standard run-time interface.
The implementation of the virtual /proc/version procfs node uses utsname()->release.
If you want to condition the code based on kernel version in compile time, you can use a preprocessor block such as:
#if LINUX_VERSION_CODE <= KERNEL_VERSION(2,6,16)
...
#else
...
#endif
It allows you to compare against major/minor versions.
You can only safely build a module for any one kernel version at a time. This means that asking from a module at runtime is redundant.
You can find this out at build time, by looking at the value of UTS_RELEASE in recent kernels this is in <generated/utsrelease.h> amongst other ways of doing this.
Why can't I build a kernel module for any version?
Because the kernel module API is unstable by design as explained in the kernel tree at: Documentation/stable_api_nonsense.txt. The summary reads:
Executive Summary
-----------------
You think you want a stable kernel interface, but you really do not, and
you don't even know it. What you want is a stable running driver, and
you get that only if your driver is in the main kernel tree. You also
get lots of other good benefits if your driver is in the main kernel
tree, all of which has made Linux into such a strong, stable, and mature
operating system which is the reason you are using it in the first
place.
See also: How to build a Linux kernel module so that it is compatible with all kernel releases?
How to do it at compile time was asked at: Is there a macro definition to check the Linux kernel version?
Going thru the ARMv8 manual, I have the following questions to help understand the big picture.
Can legacy 32 bit app. (ARMv7 or earlier) run as is on the ARMv8 OS?
If the legacy applications need to be rebuilt for ARMv8 and assuming that I rebuild the application as 32 bit (Aarch32), does this need 32 bit OS underlying support? (It is interesting to know how the addressing mechanism works here.)
Please provide references wherever possible.
PS: I am targeting Linux OS with Aarch64 support (3.7 and later)
Aarch64 platform may run 32bit ARM but this compatibility is optional.
To run AArch32 binaries you need all libraries application would use in 32bit versions. Same as with i686 binaries on x86-64 systems.
There is also a Linux arm64 CONFIG_COMPAT at: https://github.com/torvalds/linux/blob/v4.17/arch/arm64/Kconfig#L1274 which says:
This option enables support for a 32-bit EL0 running under a 64-bit
kernel at EL1. AArch32-specific components such as system calls,
the user helper functions, VFP support and the ptrace interface are
handled appropriately by the kernel.
which will likely be required, and an ARM employee mentioned on this thread: https://community.arm.com/processors/f/discussions/5535/running-armv7-binaries-on-armv8 that userland instructions are basically the same with some exceptions:
For something like a Linux application, then yes. ARMv8-A includes AArch32, which provides backwards compatibility with ARMv7-A. There are some limitations, such as the SWP instruction no longer being supported. But these are types of things that applications are unlikely to be using (and were deprecated in ARMv7).
For baremetal, you have all the usual problems of using a binary from one platform on another. So you are going to need to do some degree of porting in most cases.
I then tried it for myself with this QEMU full system setup but my attempt failed: I compiled a C hello world with the armv7 compiler as:
arm-linux-gcc -static hello_world.c
and put the built file into the aarch64 target, but when I tried to run it it failed with:
a.out: line 1: syntax error: unexpected word (expecting ")")
even though /proc/config.gz says that CONFIG_COMPAT is set.
It seems that the Linux kernel is not identifying it as an ELF file but rather falling back to /bin/sh, I get the same error if I do:
sh /mnt/9p/a.out
is trying to use the shell binfmt instead of ELF.
In particular, I know that the Linux kernel can choose between archs from the binfmt signature because qemu-user does so: https://unix.stackexchange.com/questions/41889/how-can-i-chroot-into-a-filesystem-with-a-different-architechture
My question is somewhat weird but I will do my best to explain.
Looking at the languages the linux kernel has, I got C and assembly even though I read a text that said [quote] Second iteration of Unix is written completely in C [/quote]
I thought that was misleading but when I said that kernel has assembly code I got 2 questions of the start
What assembly files are in the kernel and what's their use?
Assembly is architecture dependant so how can linux be installed on more than one CPU architecture
And if linux kernel is truly written completely in C than how can it get GCC needed for compiling?
I did a complete find / -name *.s
and just got one assembly file (asm-offset.s) somewhere in the /usr/src/linux-headers-`uname -r/
Somehow I don't think that is helping with the GCC working, so how can linux work without assembly or if it uses assembly where is it and how can it be stable when it depends on the arch.
Thanks in advance
1. Why assembly is used?
Because there are certain things then can be done only in assembly and because assembly results in a faster code. For eg, "you can get access to unusual programming modes of your processor (e.g. 16 bit mode to interface startup, firmware, or legacy code on Intel PCs)".
Read here for more reasons.
2. What assembly file are used?
From: https://www.kernel.org/doc/Documentation/arm/README
"The initial entry into the kernel is via head.S, which uses machine
independent code. The machine is selected by the value of 'r1' on
entry, which must be kept unique."
From https://www.ibm.com/developerworks/library/l-linuxboot/
"When the bzImage (for an i386 image) is invoked, you begin at ./arch/i386/boot/head.S in the start assembly routine (see Figure 3 for the major flow). This routine does some basic hardware setup and invokes the startup_32 routine in ./arch/i386/boot/compressed/head.S. This routine sets up a basic environment (stack, etc.) and clears the Block Started by Symbol (BSS). The kernel is then decompressed through a call to a C function called decompress_kernel (located in ./arch/i386/boot/compressed/misc.c). When the kernel is decompressed into memory, it is called. This is yet another startup_32 function, but this function is in ./arch/i386/kernel/head.S."
Apart from these assembly files, lot of linux kernel code has usage of inline assembly.
3. Architecture dependence?
And you are right about it being architecture dependent, that's why the linux kernel code is ported to different architecture.
Linux porting guide
List of supported arch
Things written mainly in assembly in Linux:
Boot code: boots up the machine and sets it up in a state in which it can start executing C code (e.g: on some processors you may need to manually initialize caches and TLBs, on x86 you have to switch to protected mode, ...)
Interrupts/Exceptions/Traps entry points/returns: there you need to do very processor-specific things, e.g: saving registers and reenabling interrupts, and eventually restoring registers and properly returning to user mode. Some exceptions may be handled entirely in assembly.
Instruction emulation: some CPU models may not support certain instructions, may not support unaligned data access, or may not have an FPU. An option is using emulation when getting the corresponding exception.
VDSO: the VDSO is a virtual library that the kernel maps into userspace. It allows e.g: selecting the optimal syscall sequence for the current CPU (on x86 use sysenter/syscall instead of int 0x80 if available), and implementing certain system calls without requiring a context switch (e.g: gettimeofday()).
Atomic operations and locks: Maybe in a future some of these could be written using C11 support for atomic operations.
Copying memory from/to user mode: Besides using an optimized copy, these check for out-of-bounds access.
Optimized routines: the kernel has optimized version of some routines, e.g: crypto routines, memset, clear_page, csum_copy (checksum and copy to another place IP data in one pass), ...
Support for suspend/resume and other ACPI/EFI/firmware thingies
BPF JIT: newer kernels include a JIT compiler for BPF expressions (used for example by tcpdump, secmode mode 2, ...)
...
To support different architectures, Linux has assembly code (re-)written for each architecture it supports (and sometimes, there are several implementations of some code for different platforms using the same CPU architecture). Just look at all the subdirectories under arch/
Assembly is needed for a couple of reasons.
There are many instructions that are needed for the operation of an operating system that have no C equivalent, at least on most processors. A good example on Intel x86/64 processors is the iret instruciton, which returns from hardware/software interrupts. These interrupts are key to handling hardware events (like a keyboard press) and system calls from programs on older processors.
A computer does not start up in a state that is immediately ready for execution of C code. For an Intel example, when execution gets to the startup routine the processor may not be in 32-bit mode (or 64-bit mode), and the stack required by C also may not be ready. There are some other features present in some processors (like paging) which need to be turned on from assembly as well.
However, most of the Linux kernel is written in C, which interfaces with some platform specific C/assembly code through standardized interfaces. By separating the parts in this way, most of the logic of the Linux kernel can be shared between platforms. The build system simply compiles the platform independent and dependent parts together for specific platforms, which results in different executable kernel files for different platforms (and kernel configurations for that matter).
Assembly code in the kernel is generally used for low-level hardware interaction that can't be done directly from C. They're like a platform- specific foundation that's used by higher-level parts of the kernel that are written in C.
The kernel source tree contains assembly code for a variety of systems. When you compile a kernel for a particular type of system (such as an x86 PC), only the appropriate assembly code for that platform is included in the build process.
Linux is not the second version of Unix (or Unix in general). It is Unix compatible, but Unix and Linux have separate histories and, in terms of code base (of their kernels), are completely separate. Linus Torvald's idea was to write an open source Unix.
Some of the lower level things like some of the architecture dependent parts of memory management are done in assembly. The old (but still available) Linux kernel API for x86, int 0x80, is implemented in assembly. There are probably other places in the kernel that are implemented in assembly, but I don't know any others.
When you compile the kernel, you select an architecture to target. Depending on the target, the right assembly files for that architecture are included in the build.
The reason you don't find anything is because you're searching the headers, not the sources. Download a tar ball from kernel.org and search that.
Thanks to every one,
This is the question asked in one of the interview i faced.
I have a Linux device driver which was compiled in Linux kernel version 2.6.I would like to port the same driver in a Linux PC which has kernel 3.X without compiling in new versions.
Is it possible ? If it is possible please let me know how. If it is not possible please let me know why not ?
Thanks & Regards
Siva
No you cannot port module which is compiled for one version to other version.
The reason is as follows
Modules are strongly tied to the data structures and function prototypes defined in a particular kernel version;
the interface seen by a module can change significantly from one kernel version to
the next. This is especially true of development kernels, of course
The kernel does not just assume that a given module has been built against the
proper kernel version. One of the steps in the build process is to link your module
against a file (called vermagic.o) from the current kernel tree; this object contains a
fair amount of information about the kernel the module was built for, including the
target kernel version, compiler version, and the settings of a number of important
configuration variables. When an attempt is made to load a module, this information
can be tested for compatibility with the running kernel. If things don’t match,
the module is not loaded; instead, you see something like:
# insmod hello.ko
Error inserting './hello.ko': -1 Invalid module format
A look in the system log file (/var/log/messages or whatever your system is configured
to use) will reveal the specific problem that caused the module to fail to load.
Kernel interfaces often change between releases. If you are writing a module that is
intended to work with multiple versions of the kernel (especially if it must work
across major releases), you likely have to make use of macros and #ifdef constructs
to make your code build properly.
now it's not possible:
usually, a "driver" is a binary kernel-module
porting will involve code-changes to the kernel module. if you change the code, you need to compile it, in order to get a binary.
since kernel modules run in kernel space, it is crucial that they are robust. since parts of the kernel-API change every now and then, trying to use a module compiled for kernel-X with another kernel-Y, might either not load because of missing symbols (if you are lucky) or lead to a kernel panic because semantics have changed.
btw, all this is not really related to 2.6.x vs 3.y, but holds true for any kernel version
but then: of course in theory it is possible to "write" a kernel-module as binary code in your favourite hex-editor, without resorting to compilers and such. this would allow you to "port" a driver from one kernel to another without recompilation. i guess this is not for humans though...