I'm cross-compiling an application for aarch64 on my x86 Ubuntu Bionic system, and I have problems with glibc version mismatch. My cross-compile toolchain was using v2.27, while the system that is to run the application has v2.24. I thought that it might be due to my toolchain having a too high version, so I decided to downgrade.
After removing all previous cross-compilation installs, I installed gcc-4.8-aarch64-linux-gnu (as I had successfully cross-compiled the application with this version on a different host system), thinking that it would install an older aarch64 version of glibc to /usr/aarch64-linux-gnu/lib/. However, again, v2.27 was installed (I verified that this directory didn't exist before installing the new cross-compilation toolchain).
So my question is twofold:
What determines which aarch64 version of glibc is installed on my system when installing gcc-4.8-aarch64-linux-gnu? Is it directly tied to my own system's x86 version of glibc?
Is there a correct way to install the aarch64 version of glibc v2.24 (or lower) on my system?
I concur with your hypothesis. After battling similar symptoms for 40 hours straight, I've discovered this confirmation:
https://packages.ubuntu.com/impish/gcc-10-aarch64-linux-gnu
https://packages.debian.org/bullseye/gcc-aarch64-linux-gnu
Note that Ubuntu 21.10 (Impish) and Debian 11 (Bullseye) have packages for a gcc 10 cross compiler. Be wary of the very confusing fact the Ubuntu's default package is actually gcc 11, but Debian 11's default is gcc 10. The similar version numbers of Debian and gcc are a coincidence. Also ignore for now the fact that Ubuntu's package is gcc 10.3.0 and Debian's is gcc 10.2.1.
Focus instead on the recommendations and dependencies of each package. Ultimately the Ubuntu package calls up libc >= 2.34, while the Debian package calls up libc >= 2.28.
Sure enough, when I cross-compile from Impish on x86 for Bullseye on aarch64 (despite having a complete SYSROOT for the target), I get this at runtime:
/lib/aarch64-linux-gnu/libc.so.6: version 'GLIBC_2.34' not found
But your question remains, is there any tie between the host libc and that used by the cross-compiler? The answer is a definite maybe.
See this excellent answer and links for an overview of a cross-compiler. The take-away:
You don't just cross-compile glibc, you need to cross-compile an entire toolchain. Toolchain components are ALWAYS: ld + gcc + libc + gdb.
So the C library is an integral part of the cross-compiler.
What shenanigans then, are going on when you install gcc-aarch64-linux-gnu? It's just a compiler - only one of the four parts of a toolchain.
Well apparently there's some flexibility. Technically, a cross-compiler can be naked. That's typically only useful when you're compiling an operating system, rather than an executable that runs on an operating system. So you can construct special toolchains for special purposes.
But for the standard purpose (cross compiling for Linux on another architecture) you want a typical toolchain. Which is where the package's dependencies and recommendations come in. A gcc is always in want of an ld which is always in want of a libc, and the ménage à trois is intimate. In fact, gcc is built with libc using ld in a complex do-si-do. See this example from a great guide by Preshing on Programming:
It's possible to force separation and link to other libraries, but it's not easy.
For example, the linker you use has a set of default search directories that are baked in. From the fine manual:
The default set of paths searched (without being specified with -L) depends on which emulation mode ld is using, and in some cases also on how it was configured.
And it gets more intwined. By default, gcc will call on a dynamic linker whose location is hard-coded. For a cross-compiler, it might be something like /lib/ld-linux-aarch64.so.1. Not only that, the executable may also end up with the hardcoded path, as its program interpreter.
Again, if you're careful you can tear apart the toolchain and override things. But not only is it tricky to enforce, particularly if you have a complex build, the multitude of combinations of options and paths means there are also often bugs. So your host environment can easily leak into your cross-compiling toolchain.
So in summary, cross-compiling requires a toolchain. While pulling a cross-compiler from a package manager seems like an easy and legitimate thing to do, it comes with a lot of implicit baggage. You can either carefully follow the package dependencies to check what version you're getting, or use one of the many dedicated toolchain environments, such as crosstool-NG.
Related
I have dependencies in my code that requires libc. When building (cargo build --release) on Ubuntu 20.04 (glibc 2.31) the resulting executable doesn't run on CentOS 7 (glibc 2.17). It throws an error saying it requires GLIBC 2.18.
When build the same code on CentOS 7 the resulting executable runs on CentOS 7 and Ubuntu 20.04.
Is there a way to control which GLIBC version is required to build this version on Ubuntu 20.04 too?
If your project does not depend on any native libraries, then probably the easiest way would be to use the x86_64-unknown-linux-musl target.
This target statically links against MUSL Libc rather than dynamically linking against the system's libc. As a result it produces completely static binaries which should run on a wide range of systems.
To install this target:
rustup target add x86_64-unknown-linux-musl
To build your project using this target:
cargo build --target x86_64-unknown-linux-musl
See the edition guide for more details.
If you are using any non-rust libraries it becomes more difficult, because they may be dynamically linked and may in turn depend on the system libc. In that case you would either need to statically link the external libraries (assuming that is even possible, and that the libraries you are using will work with MUSL libc), or make different builds for each platform you want to target.
If you end up having to make different builds for each platform, a docker container would be the easiest way to achieve that.
Try cross.
Install it globally:
cargo install cross
Then build your project with it:
cross build --target x86_64-unknown-linux-gnu --release
cross take the same arguments as cargo but you have to specify a target explicitly. Also, the build directory is always target/{TARGET}/(debug|release), not target/(debug|release)
cross uses docker images prebuilt for different target architectures but nothing stops you from "cross-compiling" against the host architecture. The glibc version in these docker images should be conservative enough. If it isn't, you can always configure cross to use a custom image.
In general, you need to build binaries for a given OS on that OS, or at the very least build on the oldest OS you intend to support.
glibc uses symbol versioning to preserve the behavior of older programs while adding support for new functionality. For example, a newer version of pthread_mutex_lock may support lock elision, while the old one would not. You're seeing this error because when you link against libc, you link against the default version of the symbol if a version isn't explicitly specified, and in at least one case, the version you linked against is from glibc 2.18. Changing this would require recompiling libstd (and the libc crate, if you're using it) with custom changes to pick the old versioned symbols, which is a lot of work for little gain.
If your only dependency is glibc, then it might be sufficient to just compile on CentOS 7. However, if you depend on other libraries, like OpenSSL, then those just aren't compatible across OS versions because their SONAMEs differ, and there's no way around that. So that's why generally you want to build different binaries per OS.
Glibc 2.10(or any >2.10) with compile flag PER_THREAD and ATOMIC_FASTBINS behaves totally different then glibc 2.10 without those flags.
If my Linux is using glibc 2.10 I still don't know the exact version because it doesn't say anything about compilation flags. Ubuntu may use those flags in theirs glibc and Debian not?
How to list used compilation parameters, having glibc shared library file?
You won't find this information in /lib/libc.so.6. Though, if you're running Debian or Ubuntu you can still grab the source package (apt-get source libc6) and have a look at debian/rules file.
You can also write a quick test that checks glibc behavior and conclude if it has been compiled with these flags or not.
I am using Debian/MIPS+QEMU to build MIPS ports of PortFusion (a TCP tunneling solution). The resulting binaries are linked against GNU libc. Thus, they cannot be just copied over and used on vanilla OpenWrt which ships with uclibc instead of eglibc (which seems binary-compatible with GNU libc).
Is there a way to link Haskell/GHC binaries on Debian/MIPS against uclibc instead of eglibc?
Can OpenWrt's using uclibc be really the reason why PortFusion binaries copied over from Debian fail to run with -ash: binary not found or can this message be due to something entirely else?
See https://github.com/corsis/PortFusion/wiki/MIPS-Builds for details on which haskell-platform, Linux kernel and CPU emulation are used.
The current head of OpenWrt's GIT repository is failing at make when I attempt building custom OpenWrt images that use eglibc instead.
Is there a way to link Haskell/GHC binaries on Debian/MIPS against
uclibc instead of eglibc?
No. You need to to rebuild Haskell/GHC from sources using uclibc-based GCC cross-compiler.
Can OpenWrt's using uclibc be really the reason
Yes. Also, you can try to use ldd on your MIPS pplatform to check what library is missing. I'm sure it will be some of libc-related libraries.
How can I install gcc on a system that have not any c compiler?
this system is a linux base firewall and have not any c compiler.
I guess you a have an appliance running Linux and shell-access, but neither a package manager nor a compiler is installed.
So, you need to cross-compile gcc and the whole toolchain (at least binutils) - this is quite simple, because the ./configure scripts of gcc, binutils, gdb etc. support cross-compiling with the --target= option. So all you have to do is to find out the target architecure (uname helps) and then download, unpack the gcc sources on a linux-host and run ./configure --target=$YOUR_TARGET.
With this, you now can build a cross-compiler gcc - this still runs on your host, but produces binaries for your target (firewall appliances).
This may already be sufficient for you, a typical desktop PC is much faster than a typical appliance, so it may make sense to compile everything you need on the Desktop PC with the cross-compiler and cross-binutils.
But if you really wish to do so, you can now also use your cross-compiler to compile a gcc running on your target (set this as --host= option) and compiling for your target (set this as --target option).
You can find details about allowed host/targets and examples in the gcc documentation: http://gcc.gnu.org/install/specific.html.
It depends on the distribution, if it's based on debian or some other of the big ones you can install gcc through apt-get or similar tool.
If it's a more basic system you need to compile gcc yourself on another computer and copy it over. It will be easiest if you have another computer with the same architecture (i386, arm or x86_64 for example).
I think that you might want to compile it statically also, so that you don't have dependencies on external libraries.
How do you plan to get all the source code needed for GCC loaded onto your machine? Could you mount the ISO image onto this machine and install from there?
Since you are using Endian Firewall, see "Building a development box" at the following link:
http://alumnus.caltech.edu/~igormt/endian/tips.html
If it's a debian based distribution, you can use
sudo apt-get install gcc
Note: maybe you must change "gcc" by a specific version of the debian package.
Cheers,
I want to avoid problems with compiling my code on amd64, yet I don't have a 64-bit CPU available and have no hopes of getting upgrade to my machine any time soon. I have no dreams of testing the code (although that should theoretically be possible using qemu-system) but I'd like to at least compile the code using gcc -m64.
Basic idea works:
CFLAGS=-m64 CXXFLAGS=-m64 ./configure --host x86_64-debian-linux
However, the code depends on some libraries which I typically install from Debian packages, such as libsdl1.2-dev, libgmp3-dev and such. Obviously, getting 64-bit versions of packages installed alongside of 32-bit versions is not a one-liner.
What would be your practices for installing the 64-bit packages? Where would you put them, how would you get them there and how would you use them?
To repeat, I don't have 64-bit CPU and cannot afford getting a new machine.
I have already set up amd64-libs-dev to give some basic push to gcc's -m64.
Attempted so far:
Setting up a 64-bit chroot jail with debootstrap in order to simplify installation of 64-bit development packages for libraries. Failed since finishing the setup (and installing anything afterwards!) requires 64-bit CPU.
Installing gcc-multilib and g++-multilib. This appears to do nothing beside depending on libc6-dev-amd64 which I already installed through amd64-libs-dev.
If you're using debian, before you can use gcc -m64, you need to install gcc-multilib and g++-multilib. This will also install all files needed to link and create a 64bit binary.
You don't have to have a 64bit capable CPU for this either.
Then you can call GCC as follows:
$ gcc -m64 source.c -o source
As for external libraries, debian takes care of that if you have multilib installed. I have a 32bit machine that compiles 64bit code for another machine and links a handful of libraries (libpng, libz for example). Works great and the executable run (debian to debian).
You want to look into the dchroot package to set up a simple chroot(8) environment -- that way you can compile real amd64 binaries in a real 64-bit setting with proper libraries and dependencies. This surely works the other way (i.e. I am using i386 chroots on amd64 hosts) but I don't see why it shouldn't work the other way if your cpu supports amd64.
Edit: Now that you stress that you do not have a amd64-capable cpu, it gets a little trickier. "In theory" you could just rebuild gcc from source as a cross-compiler. In practice, that may be too much work. Maybe you can just get another headless box for a few dollars and install amd64 on that?
check out this fine article that describes how to easily create a 32bit chroot, where you can install all the 32bit tools (gcc and libs)
Doesn't Debian distinguish between lib32 and lib64 directories? In that case, you can just grab the packages and force them to install, regardless of architecture.
If that does not work (or would hose your system!) I would set up a chroot environment and apt-get the 64-bit libraries into there.
Check out pbuilder, It can create build environments for many architectures, some instructions here
Try cross compiling SDL, gmp and other libraries yourself. Or manually extract the files you need from the Debain packages.