I need to run relatively new package on not-so-new RHEL 5.6.
I have 3rd party library (lib3rdparty.so) which is compiled against glibc 2.6 while RHEL 5.6 have only 2.5 installed. But in the library there is only a couple of references to sched_getcpu##GLIBC_2.6. I've checked it like this
readelf -s lib3rdparty.so | egrep "#GLIBC_2.[6-9]"
to find references to something newer than GLIBC_2.5 which is installed. The output is
0 FUNC GLOBAL DEFAULT UND sched_getcpu#GLIBC_2.6 (62)
0 FUNC GLOBAL DEFAULT UND sched_getcpu##GLIBC_2.6
So, I have only one function from GLIBC_2.6. Now I want to make library think, that I have this function. For that purpose I forged small library (libcheat.so) like it mentioned here. Now I have libcheat.so file which, if run through readelf, will show this string:
10 FUNC GLOBAL DEFAULT 11 sched_getcpu##GLIBC_2.6
With this library I managed to succesfully build executable which is dynamically linked with lib3rdparty.so. Without this library I can't build anything, because ld can't find reference to sched_getcpu.
But the problem is with running this file: when I try to run it I have a following error:
./hello_world: version `GLIBC_2.6' not found (required by ./lib3rdparty.so)
So, I believe there is one last step to make it work, but I don't know what to do. I've tried to use /etc/ld.conf.preload and exporting LD_LIBRARY_PATH so it would point to my library to load before others. But it won't run. Tried to run it through strace but have no meaningful output.
Any ideas?
I'm not sure if the workaround you picked up is the best way to go. Why not supplying a newer glibc for the application? You can try to pass it over using the LD_PRELOAD environment variable. For best binary compatibility, you can obtain a glibc binary from a newer distribution version, or rebuild a newer glibc version for RHEL 5.6.
Also, try to fiddle with the LD_DEBUG environment variable to see what the dynamic linker does.
Ok, I believe I should mention "last step" which in my case included patching of binary 3rd party libraries.
Using objdump I found an address where lib3rdparty.so searches for GLIBC_2.6
Using hexedit I replaced this address with the one where GLIBC_2.5 was exported (you should search for bytes in reversed order). Also I replaced symbol with 2.5.
Using objdump I've checked that all references now using GLIBC_2.5
I needed to rebuild libcheat.so as sched_getcpu now was referencing GLIBC_2.5, so I used sched_getcpu##GLIBC_2.5 as assembler inline
After that I put libcheat.so near lib3rdparty.so, set LD_LIBRARY_PATH to . and my binary started to work. I'm not sure that this stuff will work totally correct, but if you stumbled into this kind of problem, this question and answer might be of some help.
Related
I'm very new to Yesod and I'm having trouble building Yesod statically
so I can deploy to Heroku.
I have changed the default .cabal file to reflect static compilation
if flag(production)
cpp-options: -DPRODUCTION
ghc-options: -Wall -threaded -O2 -static -optl-static
else
ghc-options: -Wall -threaded -O0
And it no longer builds. I get a whole bunch of warnings and then a
slew of undefined references like this:
Linking dist/build/personal-website/personal-website ...
/usr/lib/ghc-7.0.3/libHSrts_thr.a(Linker.thr_o): In function
`internal_dlopen':
Linker.c:(.text+0x407): warning: Using 'dlopen' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/unix-2.4.2.0/libHSunix-2.4.2.0.a(HsUnix.o): In
function `__hsunix_getpwent':
HsUnix.c:(.text+0xa1): warning: Using 'getpwent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/unix-2.4.2.0/libHSunix-2.4.2.0.a(HsUnix.o): In
function `__hsunix_getpwnam_r':
HsUnix.c:(.text+0xb1): warning: Using 'getpwnam_r' in statically
linked applications requires at runtime the shared libraries from the
glibc version used for linking
/usr/lib/libpq.a(thread.o): In function `pqGetpwuid':
(.text+0x15): warning: Using 'getpwuid_r' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/libpq.a(ip.o): In function `pg_getaddrinfo_all':
(.text+0x31): warning: Using 'getaddrinfo' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__63.o): In function `sD3z_info':
(.text+0xe4): warning: Using 'gethostbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__164.o): In function `sFKc_info':
(.text+0x12d): warning: Using 'getprotobyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__155.o): In function `sFDs_info':
(.text+0x4c): warning: Using 'getservbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/libpq.a(fe-misc.o): In function `pqSocketCheck':
(.text+0xa2d): undefined reference to `SSL_pending'
/usr/lib/libpq.a(fe-secure.o): In function `SSLerrmessage':
(.text+0x31): undefined reference to `ERR_get_error'
/usr/lib/libpq.a(fe-secure.o): In function `SSLerrmessage':
(.text+0x41): undefined reference to `ERR_reason_error_string'
/usr/lib/libpq.a(fe-secure.o): In function `initialize_SSL':
(.text+0x2f8): undefined reference to `SSL_check_private_key'
/usr/lib/libpq.a(fe-secure.o): In function `initialize_SSL':
(.text+0x3c0): undefined reference to `SSL_CTX_load_verify_locations'
(... snip ...)
If I just compile with just -static and without -optl-static
everything builds fine but the application crashes when it tries to
start on Heroku.
2011-12-28T01:20:51+00:00 heroku[web.1]: Starting process with command
`./dist/build/personal-website/personal-website -p 41083`
2011-12-28T01:20:51+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: error while loading shared libraries: libgmp.so.10:
cannot open shared object file: No such file or directory
2011-12-28T01:20:52+00:00 heroku[web.1]: State changed from starting
to crashed
I tried adding libgmp.so.10 to the LD_LIBRARY_PATH as suggested in here
and then got the following error:
2011-12-28T01:31:23+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: /lib/libc.so.6: version `GLIBC_2.14' not found
(required by ./dist/build/personal-website/personal-website)
2011-12-28T01:31:23+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: /lib/libc.so.6: version `GLIBC_2.14' not found
(required by /app/dist/build/personal-website/libgmp.so.10)
2011-12-28T01:31:25+00:00 heroku[web.1]: State changed from starting
to crashed
2011-12-28T01:31:25+00:00 heroku[web.1]: Process exited
It seems that the version of libc that I'm compiling against is
different. I tried also adding libc to the batch of libraries the
same way I did for libgmp but this results in a segmentation fault
when the application starts on the Heroku side.
Everything works fine on my PC. I'm running 64bit archlinux with ghc
7.0.3. The blog post on the official Yesod blog looked pretty easy
but I'm stumped at this point. Anyone have any ideas? If there's a way to get this thing working without building statically I'm open to that too.
EDIT
Per Employed Russians answer I did the following to fix this.
First created a new directory lib under the project directory and copied the missing shared libraries into it. You can get this information by running ldd path/to/executable and heroku run ldd path/to/executable and comparing the output.
I then did heroku config:add LD_LIBRARY_PATH=./lib so when the application is started the dynamic linker will look for libraries in the new lib directory.
Finally I created an ubuntu 11.10 virtual machine and built and deployed to Heroku from there, this has an old enough glibc that it works on the Heroku host.
Edit:
I've since written a tutorial on the Yesod wiki
I have no idea what Yesod is, but I know exactly what each of your other errors means.
First, you should not try to link statically. The warning you get is exactly right: if you link statically, and use one of the routines for which you are getting the warning, then you must arrange to run on a system with exactly the same version of libc.so.6 as the one you used at build time.
Contrary to popular belief, static linking produces less, not more, portable executables on Linux.
Your other (static) link errors are caused by missing libopenssl.a at link time.
But let's assume that you are going to go the "sane" route, and use dynamic linking.
For dynamic linking, Linux (and most other UNIXes) support backward compatibility: an old binary continues to work on newer systems. But they don't support forward compatibility (a binary built on a newer system will generally not run on an older one).
But that's what you are trying to do: you built on a system with glibc-2.14 (or newer), and you are running on a system with glibc-2.13 (or older).
The other thing you need to know is that glibc is composed of some 200+ binaries that must all match exactly. Two key binaries are /lib/ld-linux.so and /lib/libc.so.6 (but there are many more: libpthread.so.0, libnsl.so.1, etc. etc). If some of these binaries came from different versions of glibc, you usually get a crash. And that is exactly what you got, when you tried to place your glibc-2.14 libc.so.6 on the LD_LIBRARY_PATH -- it no longer matches the system /lib/ld-linux.
So what are the solutions? There are several possibilities (in increasing difficulty):
You could copy ld-2.14.so (the target of /lib/ld-linux symlink) to the target system, and invoke it explicitly:
/path/to/ld-2.14.so --library-path <whatever> /path/to/your/executable
This generally works, but can confuse an application that looks at argv[0], and breaks for applications that re-exec themselves.
You could build on an older system.
You could use appgcc (this option has disappeared, see this for description of what it used to be).
You could set up a chroot environment matching the target system, and build inside that chroot.
You could build yourself a Linux-to-olderLinux crosscompiler
You have several issues.
You should not build production binaries on bleeding edge distributions. The libraries on the production system will not be forward compatible.
You should not link glibc statically - it will always at runtime try to load additional libraries. For example cpu-based assembly. That is what your first warnings are about.
The last linker errors look like they are related to a missing openssl library on the command line.
But all in all - downgrade your distribution.
I had similar problems launching to Heroku (which uses glibc-2.11) where I had an application that required glibc-2.14, but I did not have access to the source and could not re-build it. I tried many things and nothing worked.
My workaround was to launch the service on Amazon Elastic Beanstalk and just provide an API interface.
I found the information provided useful as well, I think the various descriptions miss a critical issue I also ran into while forcing an updated version of Vagrant to start working again.
It's the dependency references internal to something like complicated installs, like Yesod to Heroku. Those interanl refences need to be preserved.
This is the script I wrote to make problems go away (at least, hopefully, for a little while):
#!/bin/bash
cd $HOME/
GLIBC_VERSION="2.17"
GLIBC_PREFIX="/usr/glibc/"
VAGRANT_VERSION="2.2.19"
# Install the basic build system utilities.
yum groupinstall -y "Development tools"
yum install -y curl patchelf
# Grab the tarball with the GNU libc source code.
curl -Lfo glibc-${GLIBC_VERSION}.tar.gz "https://ftp.gnu.org/gnu/glibc/glibc-${GLIBC_VERSION}.tar.gz"
echo "a3b2086d5414e602b4b3d5a8792213feb3be664ffc1efe783a829818d3fca37a glibc-${GLIBC_VERSION}.tar.gz" | sha256sum -c || exit 1
# Extract the secrets and get ready to rumble.
tar xzvf glibc-${GLIBC_VERSION}.tar.gz
# The configure script requrires an independent build directory.
mkdir -p glibc-build && cd glibc-build
# Configure glibc with a GLIBC_PREFIX so it doesn't conflict with distro libc files..
../glibc-${GLIBC_VERSION}/configure --prefix="${GLIBC_PREFIX}" --libdir="${GLIBC_PREFIX}/lib" \
--libexecdir="${GLIBC_PREFIX}/lib" --enable-multi-arch
# Compile and then install GNU libc.
make -j8 && make install
# Download and install Vagrant.
curl -Lfo vagrant_${VAGRANT_VERSION}_x86_64.rpm "https://releases.hashicorp.com/vagrant/${VAGRANT_VERSION}/vagrant_${VAGRANT_VERSION}_x86_64.rpm"
echo "990e8d2159032915f21c0f1ccdcbca1a394f7937e06e43dc1dabe605d208dc20 vagrant_${VAGRANT_VERSION}_x86_64.rpm" | sha256sum -c || exit 1
yum install -y vagrant_${VAGRANT_VERSION}_x86_64.rpm
# Patch the binaries and shared libraries inside the Vagrant directory, so they use the new version of GNU libc.
(find /opt/vagrant/ -type f -exec file {} \; )| grep "dynamically linked" | awk -F':' '{print $1}' | while read FILE ; do
patchelf --set-rpath /opt/vagrant/embedded/lib:/opt/vagrant/embedded/lib64:/usr/glibc/lib:/usr/lib64:/lib64:/lib --set-interpreter /usr/glibc/lib/ld-linux-x86-64.so.2 "${FILE}"
done
The script should be pretty easy to understand, and adapt easily to whatever MacGuffin you want to make work, provied you understand it.
The only tricky part is the rpath you pass to patchelf. Upi need to make sure you preserve the search paths, and precedence your software requires. Or you end up fixing one problem only to create another equally frustrating roadblock.
P.S. Don't forget the update the hashes for any file you down. In particular, you need to compile/install a different version of GNU libc, you will need to update that hash to match the version you want to use.
I'm working in a project that uses a number of external libraries. These libraries are included using Conan. The project is primarily written for Unix, but it also need to compile and run on Windows.
My current problem is that Windows defaults fopen() to be O_TEXT, while Unix expects this to be O_BINARY. I have a fix that works for my own code, simply include binmode.obj when linking to change this default to O_BINARY.
The problem is that this does not affect my third party libraries. Googling for this didn't turn up much, most suggestions seems to be based on where you are creating your own package and want flags added, rather than how to add flags when using other's packages.
What I have tried so far:
Make binmode.obj come before libraries, in case the linking order matters. Made no difference.
Added --env 'CL=link binmode.obj' to conan install, but this flag did not end up as part of the compile flags nor link flags.
Any suggestions for what I could try?
EDIT: I was wrong about "CL" taking no effect. This was caused by confusing output. But I did observe that CL seems to be applied for both compiler and linker, which makes it somewhat challenging what flags to give. Using "/link" prefix makes it work with compiler, but does not work with linker.
EDIT 2: More confusions... I didn't realize that the syntax of the CL value was: "<compile flags> /link <link flags>". It affected compile, but not link, however. So this environment variable apparently can't be used to make Conan add a linker flag for autotools based packages.
Hi Mats L welcome to our community!
I once had a similar problem and what I end up doing was quite hacky but quite simple also:
On my conan profile located at ~/.conan/profiles/default or any other profile actually I added an enviromental variable as such:
CXX=/usr/bin/clang++ -fms-compatibility. This one made me compile all the c++ sources with this flag (that can understand windows specific code).
So in your case you can run which c++ to find the location of your compiler
and edit the CXX environmental variable in the conan profile you use your final file will probably look like :
[settings]
os=Macos
os_build=Macos
arch=x86_64
arch_build=x86_64
compiler=clang
compiler.version=11
compiler.libcxx=libc++
build_type=Release
[options]
[build_requires]
[env]
CXX=c++ --required_flag
Some additional notes: You might also want this flag set on your CC enviromental variable .
It's preferable to not change the default profile but copy it (lets say on a file named default_edited_copy) and then invoke the other profile with conan ... --profile default_edited_copy
In Windows, the dynamic loader always looks for modules in the path of the loaded executable first, making it possible to have private libraries without affecting system libraries.
The dynamic loader on Linux only looks for libraries in a fixed path, in the sense that it is independent on the chosen binary. I needed GCC 5 for its overflow checked arithmetic functions, but since the C++ ABI changed between 4.9 and 5, some applications became unstable and recompiling them solved the issue. While waiting for my distro [kubuntu] to upgrade the default compiler, is it possible to have newly compiled application linking to the new runtime, while packaged application still links to the old library, either by static linkage, or something that mimics the Windows behavior?
One way of emulating it would be to create a wrapper script
#!/bin/bash
LD_LIBRARY_PATH=$(dirname $(which your_file)) your_file
And after the linking step copy the affected library but it is sort of a hack.
You can use rpath.
Let's say your "new ABI" shared libraries are in /usr/local/newapi-libs.
gcc -L/usr/local/newapi-libs
-Wl,-rpath,/usr/local/newapi-libs
program.cpp -o program -lsomething`
The -rpath option of the linker is the runtime counterpart to -L. When a program compiled this way is run, the linker will first look in /usr/local/newapi-libs before searching the system library paths.
More information here and here.
You can emulate the Windows behavior of looking in the executable's directory by specifying -Wl,-rpath,.
[edit] added missing -L parameter and dashes before rpath.
I am trying to use libfann version 2.0.1 instead of the newest version 2.2.0, but could not figure out how to do so. Any thoughts on how to do that?
normally that works perfectly:
gcc fann_calculator.c -o run_fann_calculator -lfann -lm
where fann_calculator.c contains a program that calls a neural network.
Thanks
It depends upon where the two libraries sit. If they are installed in the same directory (e.g. both installed in /usr/lib/) you'll probably get the youngest one.
I suggest to carefully read the ld.so(8) and ldd(1) man pages. You certainly can trace what library is loaded (with e.g. the LD_DEBUG envirnonment variable). Don't forget to re-run ldconfig appropriately after library installation.
You could also play some LD_LIBRARY_PATH trick; for instance, set it to $HOME/lib:/usr/lib and install appropriate symlinks in your $HOME/lib/ to the precise library you want. For instance, you might do
ln -s /usr/lib/libfann.so.2.0.1 $HOME/lib/libfann.so.2
export LD_LIBRARY_PATH=$HOME/lib:/usr/lib:/lib
then check with ldd run_fann_calculator that you get the expected [version of the] libfann library.
Don't forget to read the Program Library Howto. You might want to pass appropriate flags to ld such as -rpath. You may need to pass them using gcc, perhaps with Gcc Link Options such as -Wl
Is there some way to use ld.so.preload and cover both 32bit and 64bit binaries?
If I list both the 32bit and 64bit versions of the fault handler in ld.so.preload then the loader always complains that one of them fails to preload for whatever command I run. Not exactly earth shaking since the error is more a warning but I could certainly do without the printout.
Instead of specifying an absolute path I tried specifying simply "segv_handler.so" in the hopes that the loader would choose the lib in the arch appropriate path (a 32bit version is in /lib and a 64bit version is in /lib64).
Not likely apparently.
Is there a way to setup ld.so.preload to be architecturally aware? Or if not is there some way to turn off the error message?
This works:
put library under /path/lib for 32bit one, and put the 64bit one under /path/lib64
and they should have the same name
put the following line in /etc/ld.so.preload:
/path/$LIB/libname.so
$LIB will get the value "lib" (for 32bit) or "lib64" (for 64bit) automatically.
There's no reason to try to use ld.so.preload like this. By default ld is smart enough to know that if you're running a 64bit app to only lookup 64bit libs, and same with 32bit.
Case in point, if you have
/lib64/libawesome.so
/lib/libawesome.so
And you try
gcc -lawesome -o funtime funtime.c
It'll choose whatever the default that gcc wants to build, ld will skip libraries of incorrect bit size for that build.
gcc -m64 -lawesome -o funtime funtime.c will pick the 64bit one
gcc -m32 -lawesome -o funtime funetime.c will pick the 32bit one.
This presumes that /etc/ld.so.conf lists /lib and /lib64 by default..
Sadly, I think the answer might be "Don't do that."
From glibc, elf/rtld.c:
There usually is no ld.so.preload file, it should only be used for emergencies and testing. So the open call etc should usually fail. Using access() on a non-existing file is faster than using open(). So we do this first. If it succeeds we do almost twice the work but this does not matter, since it is not for production use.
You can provide 32 and 64 bit library using special expansion keys in the path name.
For instance you can use /lib/$PLATFORM/mylib.so and create /lib/i386/mylib.so and /lib/x86_64/mylib.so. Linked will choose the correct one for your executable.