We do coding & building in one Linux machine, & deploy to our cluster with hundreds of cores. For now both type of machines are Redhat EL 5.7, with the default GCC 4.1.2 installed.
Recently we realized the latest GCCs (e.g. 4.8.1) got extensive optimizations for arithmetic calculations, including the usage of MPFR/MPC etc. Because our programs are very floating point calculation intensive, we hope to rebuild our programs in latest GCCs to achieve the boost.
Here are the current linking details for a typical program built by us:
linux-gate.so.1 => (0x007e0000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x009bb000)
libc.so.6 => /lib/libc.so.6 (0x00581000)
/lib/ld-linux.so.2 (0x0034c000)
Because it's unlikely for us to upgrade OS, or install new stuffs in cluster. (tightly controlled). So questions are:
1, for developing, possible for us to install the latest GCC on existing machine? (We tried and found lot's of dependencies needed). And possible to link to the older libs?
2, for deployment, possible to deploy to our cluster without installing new softwares? For these MPFR etc, can we just deploy the so files instead of installing RPMs in the target cluster nodes?
Thanks a lot for any help.
You need to install the required dependencies (and their required versions) to build GCC 4.8; notice that MPFR, CLOOG etc are only needed to the compiler (so are needed at compilation time, not at run time of your compiled program), so you don't need to install them to deploy your compiled program. Don't link the compiler to older versions of required dependencies.
The gcc-4.8 source tarball has a contrib/download_prerequisites script which could be helpful.
If building GCC 4.8 from source tarball, don't forget to build outside of the source tree, and to follow the install instruction.
You may want to link your program with the -static-libgcc option, or even to ..../configure the compiler with --disable-shared and --program-suffix=-4.8 configure options
(with that program-suffix option, you'll run your new GCC as gcc-4.8 and it will get installed, unless you configure some --prefix, in /usr/local/bin/ by default; this won't interfere with the system gcc; if you don't have [root] write access to /usr/local you should configure your own --prefix).
BTW, you might consider perhaps customizing your GCC 4.8, e.g. thru plugins or better yet using MELT.
Related
I have dependencies in my code that requires libc. When building (cargo build --release) on Ubuntu 20.04 (glibc 2.31) the resulting executable doesn't run on CentOS 7 (glibc 2.17). It throws an error saying it requires GLIBC 2.18.
When build the same code on CentOS 7 the resulting executable runs on CentOS 7 and Ubuntu 20.04.
Is there a way to control which GLIBC version is required to build this version on Ubuntu 20.04 too?
If your project does not depend on any native libraries, then probably the easiest way would be to use the x86_64-unknown-linux-musl target.
This target statically links against MUSL Libc rather than dynamically linking against the system's libc. As a result it produces completely static binaries which should run on a wide range of systems.
To install this target:
rustup target add x86_64-unknown-linux-musl
To build your project using this target:
cargo build --target x86_64-unknown-linux-musl
See the edition guide for more details.
If you are using any non-rust libraries it becomes more difficult, because they may be dynamically linked and may in turn depend on the system libc. In that case you would either need to statically link the external libraries (assuming that is even possible, and that the libraries you are using will work with MUSL libc), or make different builds for each platform you want to target.
If you end up having to make different builds for each platform, a docker container would be the easiest way to achieve that.
Try cross.
Install it globally:
cargo install cross
Then build your project with it:
cross build --target x86_64-unknown-linux-gnu --release
cross take the same arguments as cargo but you have to specify a target explicitly. Also, the build directory is always target/{TARGET}/(debug|release), not target/(debug|release)
cross uses docker images prebuilt for different target architectures but nothing stops you from "cross-compiling" against the host architecture. The glibc version in these docker images should be conservative enough. If it isn't, you can always configure cross to use a custom image.
In general, you need to build binaries for a given OS on that OS, or at the very least build on the oldest OS you intend to support.
glibc uses symbol versioning to preserve the behavior of older programs while adding support for new functionality. For example, a newer version of pthread_mutex_lock may support lock elision, while the old one would not. You're seeing this error because when you link against libc, you link against the default version of the symbol if a version isn't explicitly specified, and in at least one case, the version you linked against is from glibc 2.18. Changing this would require recompiling libstd (and the libc crate, if you're using it) with custom changes to pick the old versioned symbols, which is a lot of work for little gain.
If your only dependency is glibc, then it might be sufficient to just compile on CentOS 7. However, if you depend on other libraries, like OpenSSL, then those just aren't compatible across OS versions because their SONAMEs differ, and there's no way around that. So that's why generally you want to build different binaries per OS.
I installed gcc-9.3.0 in linux following the simple steps in https://gcc.gnu.org/wiki/InstallingGCC
The installation was successful. I thought this process would install gmp and mpfr along the way. But I cannot find the gmp or mpfr libraries.
Should I install them when I do ./configure?
There are two different ways you can provide GCC with prerequisite
external host libraries like GMP and
MPFR:
Before configuring, run ./contrib/download_prerequisites from the
toplevel GCC source directory. It will download the sources from the
respective project repositories. That's all you have to do; GCC will
configure and build these libraries before running configure for itself
or its target libraries.
configure with --with-gmp etc. to point to the place where the
respective host library had been installed.
Either approach has its ups and down:
The first approach needs internet connection and build times go up.
However the increase of build time is negligible to the build time
needed for the rest of the compiler and its target libraries.
On the other hand, you will get the right version and build options of
either host lib as preferred by GCC, irrespective of which lib version
might already have been installed on the host. And this configuration
is much more convenient and self-contained when you are cross-compiling
GCC (because it's about host libraries, and you thus have to install
the prerequisite libraries on the host; installing it on build is
pointless). And it's more robust / self-contained if you are
distributing the built compiler: If the intended host does not have GMP
etc installed, then you'll have additional work to do on that host.
The second approach is more complicated because you have to build /
install the prerequisites on the host; correct version, correct
configure flags etc.
On the other hand, you only have to build the prerequisites once for
each host.
When you are configuring / rebuilding the compiler more than once, for
example when you are doing GCC development, then it's a bit faster cycle.
In the first case, the lib versions are independent of the host versions
of the libraries; GCC will actually not care whether the libs are
present on the host because it is using it's own "copy". The libraries
will be linked statically into the executable, hence you won't find them
anywhere (Maybe GCC has the option to do shared in-tree builds for the
libs, I don't know).
I am very much preferring the 1st approach because it is self-contained
and easier, in particular because I am frequently building GCC as
canadian cross, i.e. build ≠ host ≠ target ≠ build.
I'm trying to install gcc4.9 on a SUSE system without an internet connection. I compiled gcc on an Ubuntu machine and installed it into a prefix, then copied the prefix folder to the SUSE machine. When I tried to run it gcc complained about not finding GLIBC_2_14, so I downloaded an rpm for libc6 online and included it into the prefix folders. my LD_LIBRARY_PATH includes prefix/lib and prefix/lib64. When I try to run any program now (ls, cp, cat, etc) I get the error error while loading shared libraries: /home/***/prefix/lib64/libc.so.6: unexpected reloc type 0x25.
Is there any way I can fix this so that I can get gcc4.9 up and running on this system?
As an alternative, is it possible to build gcc staticaly so that I don't have to worry about linking at all when I transfer it between computers?
my LD_LIBRARY_PATH includes prefix/lib and prefix/lib64
See this answer for explanation of why this can't work.
Is there any way I can fix this so that I can get gcc4.9 up and running on this system?
Your best bet is to install whatever GCC package comes with the SuSE system, then use that GCC to configure and install gcc-4.9 on it.
If for some reason you can't do that, this answer has some of the ways in which you can build gcc-4.9 on a newer system and have it still work on an older one.
is it possible to build gcc staticaly so that I don't have to worry about linking at all when I transfer it between computers?
Contrary to popular belief, fully-static binaries are generally less portable then dynamic ones on Linux.
I'm trying to build a portable version of gcc 4.8.2. (for only C/C++ languages) The end result is have gcc installed into a specific application directory, eg, /opt/gcc-4.8.2 so that I can copy that directory from one computer to another (all computers are either intel corei5 or corei7, running recent Linux versions, eg, Ubuntu 12, Suse 10/11, Centos 5 & 6).
So far I'm able to build gcc ok, using --prefix to have the gcc outputs placed in a single directory (which can then be later copied to the other hosts). I configured & built gcc's dependencies (gmp, mpfr, mpc, isl) to have --disable-shared, so I can be sure that the final gcc, when copied to other hosts, won't complain about missing libraries or symbols.
I have a question with cloog. I configured gcc with --with-cloog (to pick up my locally built cloog, which I built along with the other gcc dependencies). However, what I don't know, is whether I also need to copy the cloog libraries and binary to each host I copy gcc to?
Also, how can I test gcc & cloog interaction? Is there a simple C file example and/or gcc command line that can be used to test whether gcc is successfully making use of cloog?
Additionally, are there any other considerations when trying to build a gcc which I then want to run on other hosts?
It depends if cloog is installed as a shared library libcloog-isl.so.* or as a static one libcloog.a ; use
ldd $(gcc-4.8 -print-file-name=cc1)
to find out. Of course you need to install all the shared libraries dependencies. If libcloog*so appears in the output of above ldd command, it is a shared library. Otherwise a static one.
You could set the LD_LIBRARY_PATH, or add the directory containing libcloog-isl.so.* (e.g. /usr/local/lib/ or /opt/lib/ etc...) to /etc/ld.so.conf (then run ldconfig)
I am not entirely sure your gcc build can run on every platform you mentioned. There might be libc* dependencies. See this. And perhaps also binutils dependencies (notably for gcc-4.8 -flto compilations).
To test gcc just compile with optimizations (e.g. gcc-4.8 -Wall -O3) some non-trivial file.
Cheers,
I want to avoid problems with compiling my code on amd64, yet I don't have a 64-bit CPU available and have no hopes of getting upgrade to my machine any time soon. I have no dreams of testing the code (although that should theoretically be possible using qemu-system) but I'd like to at least compile the code using gcc -m64.
Basic idea works:
CFLAGS=-m64 CXXFLAGS=-m64 ./configure --host x86_64-debian-linux
However, the code depends on some libraries which I typically install from Debian packages, such as libsdl1.2-dev, libgmp3-dev and such. Obviously, getting 64-bit versions of packages installed alongside of 32-bit versions is not a one-liner.
What would be your practices for installing the 64-bit packages? Where would you put them, how would you get them there and how would you use them?
To repeat, I don't have 64-bit CPU and cannot afford getting a new machine.
I have already set up amd64-libs-dev to give some basic push to gcc's -m64.
Attempted so far:
Setting up a 64-bit chroot jail with debootstrap in order to simplify installation of 64-bit development packages for libraries. Failed since finishing the setup (and installing anything afterwards!) requires 64-bit CPU.
Installing gcc-multilib and g++-multilib. This appears to do nothing beside depending on libc6-dev-amd64 which I already installed through amd64-libs-dev.
If you're using debian, before you can use gcc -m64, you need to install gcc-multilib and g++-multilib. This will also install all files needed to link and create a 64bit binary.
You don't have to have a 64bit capable CPU for this either.
Then you can call GCC as follows:
$ gcc -m64 source.c -o source
As for external libraries, debian takes care of that if you have multilib installed. I have a 32bit machine that compiles 64bit code for another machine and links a handful of libraries (libpng, libz for example). Works great and the executable run (debian to debian).
You want to look into the dchroot package to set up a simple chroot(8) environment -- that way you can compile real amd64 binaries in a real 64-bit setting with proper libraries and dependencies. This surely works the other way (i.e. I am using i386 chroots on amd64 hosts) but I don't see why it shouldn't work the other way if your cpu supports amd64.
Edit: Now that you stress that you do not have a amd64-capable cpu, it gets a little trickier. "In theory" you could just rebuild gcc from source as a cross-compiler. In practice, that may be too much work. Maybe you can just get another headless box for a few dollars and install amd64 on that?
check out this fine article that describes how to easily create a 32bit chroot, where you can install all the 32bit tools (gcc and libs)
Doesn't Debian distinguish between lib32 and lib64 directories? In that case, you can just grab the packages and force them to install, regardless of architecture.
If that does not work (or would hose your system!) I would set up a chroot environment and apt-get the 64-bit libraries into there.
Check out pbuilder, It can create build environments for many architectures, some instructions here
Try cross compiling SDL, gmp and other libraries yourself. Or manually extract the files you need from the Debain packages.