I am climbing the learning curve of Yocto (Honister) trying to add an third party executable ELF file to my Linux image.
My current issue is that the ELF requires a couple of shared libraries e.g. libpam.so. I have created a recipe that builds these libraries from source code and puts then in the correct place in the file system, but they are not being added to the shared library cache (e.g. not listed when you run /sbin/ldconfig -p | grep pam on the resulting system).
I can correct this manually on a running system via /sbin/ldconfig /path/to/library but I assume I am missing a something in my do_install() step or something.
My reading of the various docs suggest that the bitbake build process should automatically sort this out
On a side note the library version of Linux-PAM is is 1.5.2 but the automake Makefile.am file specifies a version number of 0.85.1
libpam_la_LDFLAGS = -no-undefined -version-info 85:1:85
Is there a way I can access this 0.85.1 version number from an environment variable similar to ${PV} (which returns 1.5.2)
Thanks
Related
When attempting to generate a large static library in a Fortran/C makefile-based project with devtoolset-11, the 'ar' step fails with:
ar: first/object/in/alphabetical/order.o: file format not recognized
I'm using a devtoolset-11 installation contained in a Podman image based on Red Hat Enterprise Linux Server release 7.9 (Maipo). Devtoolset-11 is activated with: "source /opt/rh/devtoolset-11/enable", and "which ar" prints "/opt/rh/devtoolset-11/root/usr/bin/ar". "ar --version" is "GNU ar version 2.36.1-1.el7.1".
Where it gets interesting is if I rename devtoolset's 'ar' and create a link "ln -s /usr/bin/ar /opt/rh/devtoolset-11/root/usr/bin/ar", then the build completes without issue and the library is generated!
I've also tried using devtoolset's 'ar' to manually generate a library using one of the object files, and that also works, so it doesn't seem that the object is actually being compiled with an incompatible ELF format for 'ar'. I've also tried building test programs with the gcc/gfortran in devtoolset-11 and manually creating the library with devtoolset's 'ar' and this also works, it only fails within the context of my makefile project.
Any idea as to what could be going on here and why I can complete the build with the system default 'ar' but not the one in devtoolset-11? Appreciate any insight.
I'm very new to Yesod and I'm having trouble building Yesod statically
so I can deploy to Heroku.
I have changed the default .cabal file to reflect static compilation
if flag(production)
cpp-options: -DPRODUCTION
ghc-options: -Wall -threaded -O2 -static -optl-static
else
ghc-options: -Wall -threaded -O0
And it no longer builds. I get a whole bunch of warnings and then a
slew of undefined references like this:
Linking dist/build/personal-website/personal-website ...
/usr/lib/ghc-7.0.3/libHSrts_thr.a(Linker.thr_o): In function
`internal_dlopen':
Linker.c:(.text+0x407): warning: Using 'dlopen' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/unix-2.4.2.0/libHSunix-2.4.2.0.a(HsUnix.o): In
function `__hsunix_getpwent':
HsUnix.c:(.text+0xa1): warning: Using 'getpwent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/unix-2.4.2.0/libHSunix-2.4.2.0.a(HsUnix.o): In
function `__hsunix_getpwnam_r':
HsUnix.c:(.text+0xb1): warning: Using 'getpwnam_r' in statically
linked applications requires at runtime the shared libraries from the
glibc version used for linking
/usr/lib/libpq.a(thread.o): In function `pqGetpwuid':
(.text+0x15): warning: Using 'getpwuid_r' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/libpq.a(ip.o): In function `pg_getaddrinfo_all':
(.text+0x31): warning: Using 'getaddrinfo' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__63.o): In function `sD3z_info':
(.text+0xe4): warning: Using 'gethostbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__164.o): In function `sFKc_info':
(.text+0x12d): warning: Using 'getprotobyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__155.o): In function `sFDs_info':
(.text+0x4c): warning: Using 'getservbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/libpq.a(fe-misc.o): In function `pqSocketCheck':
(.text+0xa2d): undefined reference to `SSL_pending'
/usr/lib/libpq.a(fe-secure.o): In function `SSLerrmessage':
(.text+0x31): undefined reference to `ERR_get_error'
/usr/lib/libpq.a(fe-secure.o): In function `SSLerrmessage':
(.text+0x41): undefined reference to `ERR_reason_error_string'
/usr/lib/libpq.a(fe-secure.o): In function `initialize_SSL':
(.text+0x2f8): undefined reference to `SSL_check_private_key'
/usr/lib/libpq.a(fe-secure.o): In function `initialize_SSL':
(.text+0x3c0): undefined reference to `SSL_CTX_load_verify_locations'
(... snip ...)
If I just compile with just -static and without -optl-static
everything builds fine but the application crashes when it tries to
start on Heroku.
2011-12-28T01:20:51+00:00 heroku[web.1]: Starting process with command
`./dist/build/personal-website/personal-website -p 41083`
2011-12-28T01:20:51+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: error while loading shared libraries: libgmp.so.10:
cannot open shared object file: No such file or directory
2011-12-28T01:20:52+00:00 heroku[web.1]: State changed from starting
to crashed
I tried adding libgmp.so.10 to the LD_LIBRARY_PATH as suggested in here
and then got the following error:
2011-12-28T01:31:23+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: /lib/libc.so.6: version `GLIBC_2.14' not found
(required by ./dist/build/personal-website/personal-website)
2011-12-28T01:31:23+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: /lib/libc.so.6: version `GLIBC_2.14' not found
(required by /app/dist/build/personal-website/libgmp.so.10)
2011-12-28T01:31:25+00:00 heroku[web.1]: State changed from starting
to crashed
2011-12-28T01:31:25+00:00 heroku[web.1]: Process exited
It seems that the version of libc that I'm compiling against is
different. I tried also adding libc to the batch of libraries the
same way I did for libgmp but this results in a segmentation fault
when the application starts on the Heroku side.
Everything works fine on my PC. I'm running 64bit archlinux with ghc
7.0.3. The blog post on the official Yesod blog looked pretty easy
but I'm stumped at this point. Anyone have any ideas? If there's a way to get this thing working without building statically I'm open to that too.
EDIT
Per Employed Russians answer I did the following to fix this.
First created a new directory lib under the project directory and copied the missing shared libraries into it. You can get this information by running ldd path/to/executable and heroku run ldd path/to/executable and comparing the output.
I then did heroku config:add LD_LIBRARY_PATH=./lib so when the application is started the dynamic linker will look for libraries in the new lib directory.
Finally I created an ubuntu 11.10 virtual machine and built and deployed to Heroku from there, this has an old enough glibc that it works on the Heroku host.
Edit:
I've since written a tutorial on the Yesod wiki
I have no idea what Yesod is, but I know exactly what each of your other errors means.
First, you should not try to link statically. The warning you get is exactly right: if you link statically, and use one of the routines for which you are getting the warning, then you must arrange to run on a system with exactly the same version of libc.so.6 as the one you used at build time.
Contrary to popular belief, static linking produces less, not more, portable executables on Linux.
Your other (static) link errors are caused by missing libopenssl.a at link time.
But let's assume that you are going to go the "sane" route, and use dynamic linking.
For dynamic linking, Linux (and most other UNIXes) support backward compatibility: an old binary continues to work on newer systems. But they don't support forward compatibility (a binary built on a newer system will generally not run on an older one).
But that's what you are trying to do: you built on a system with glibc-2.14 (or newer), and you are running on a system with glibc-2.13 (or older).
The other thing you need to know is that glibc is composed of some 200+ binaries that must all match exactly. Two key binaries are /lib/ld-linux.so and /lib/libc.so.6 (but there are many more: libpthread.so.0, libnsl.so.1, etc. etc). If some of these binaries came from different versions of glibc, you usually get a crash. And that is exactly what you got, when you tried to place your glibc-2.14 libc.so.6 on the LD_LIBRARY_PATH -- it no longer matches the system /lib/ld-linux.
So what are the solutions? There are several possibilities (in increasing difficulty):
You could copy ld-2.14.so (the target of /lib/ld-linux symlink) to the target system, and invoke it explicitly:
/path/to/ld-2.14.so --library-path <whatever> /path/to/your/executable
This generally works, but can confuse an application that looks at argv[0], and breaks for applications that re-exec themselves.
You could build on an older system.
You could use appgcc (this option has disappeared, see this for description of what it used to be).
You could set up a chroot environment matching the target system, and build inside that chroot.
You could build yourself a Linux-to-olderLinux crosscompiler
You have several issues.
You should not build production binaries on bleeding edge distributions. The libraries on the production system will not be forward compatible.
You should not link glibc statically - it will always at runtime try to load additional libraries. For example cpu-based assembly. That is what your first warnings are about.
The last linker errors look like they are related to a missing openssl library on the command line.
But all in all - downgrade your distribution.
I had similar problems launching to Heroku (which uses glibc-2.11) where I had an application that required glibc-2.14, but I did not have access to the source and could not re-build it. I tried many things and nothing worked.
My workaround was to launch the service on Amazon Elastic Beanstalk and just provide an API interface.
I found the information provided useful as well, I think the various descriptions miss a critical issue I also ran into while forcing an updated version of Vagrant to start working again.
It's the dependency references internal to something like complicated installs, like Yesod to Heroku. Those interanl refences need to be preserved.
This is the script I wrote to make problems go away (at least, hopefully, for a little while):
#!/bin/bash
cd $HOME/
GLIBC_VERSION="2.17"
GLIBC_PREFIX="/usr/glibc/"
VAGRANT_VERSION="2.2.19"
# Install the basic build system utilities.
yum groupinstall -y "Development tools"
yum install -y curl patchelf
# Grab the tarball with the GNU libc source code.
curl -Lfo glibc-${GLIBC_VERSION}.tar.gz "https://ftp.gnu.org/gnu/glibc/glibc-${GLIBC_VERSION}.tar.gz"
echo "a3b2086d5414e602b4b3d5a8792213feb3be664ffc1efe783a829818d3fca37a glibc-${GLIBC_VERSION}.tar.gz" | sha256sum -c || exit 1
# Extract the secrets and get ready to rumble.
tar xzvf glibc-${GLIBC_VERSION}.tar.gz
# The configure script requrires an independent build directory.
mkdir -p glibc-build && cd glibc-build
# Configure glibc with a GLIBC_PREFIX so it doesn't conflict with distro libc files..
../glibc-${GLIBC_VERSION}/configure --prefix="${GLIBC_PREFIX}" --libdir="${GLIBC_PREFIX}/lib" \
--libexecdir="${GLIBC_PREFIX}/lib" --enable-multi-arch
# Compile and then install GNU libc.
make -j8 && make install
# Download and install Vagrant.
curl -Lfo vagrant_${VAGRANT_VERSION}_x86_64.rpm "https://releases.hashicorp.com/vagrant/${VAGRANT_VERSION}/vagrant_${VAGRANT_VERSION}_x86_64.rpm"
echo "990e8d2159032915f21c0f1ccdcbca1a394f7937e06e43dc1dabe605d208dc20 vagrant_${VAGRANT_VERSION}_x86_64.rpm" | sha256sum -c || exit 1
yum install -y vagrant_${VAGRANT_VERSION}_x86_64.rpm
# Patch the binaries and shared libraries inside the Vagrant directory, so they use the new version of GNU libc.
(find /opt/vagrant/ -type f -exec file {} \; )| grep "dynamically linked" | awk -F':' '{print $1}' | while read FILE ; do
patchelf --set-rpath /opt/vagrant/embedded/lib:/opt/vagrant/embedded/lib64:/usr/glibc/lib:/usr/lib64:/lib64:/lib --set-interpreter /usr/glibc/lib/ld-linux-x86-64.so.2 "${FILE}"
done
The script should be pretty easy to understand, and adapt easily to whatever MacGuffin you want to make work, provied you understand it.
The only tricky part is the rpath you pass to patchelf. Upi need to make sure you preserve the search paths, and precedence your software requires. Or you end up fixing one problem only to create another equally frustrating roadblock.
P.S. Don't forget the update the hashes for any file you down. In particular, you need to compile/install a different version of GNU libc, you will need to update that hash to match the version you want to use.
I created a shared library, libsslab_core.so.1.0.0 using gcc with proper options. I am pretty sure that the shared library works because I already linked it to another source code (I explicitly tells a compiler, gcc, the location of the library using -l option of the compiler).
After testing the library works, I tried to integrate the library into my Linux machine. I went to the /etc/ld.so.conf.d/ and added a file, sslab.conf. In the file I just typed the absolute path of the library, /opt/lib/sslab. Next, I executed ldconfig as a root to update the cache file of ldconfig. And I checked if the system finds the newly added library by typing ldconfig -p | grep libsslab. My Linux machine found the library, so I thought everything is finished.
However, when I try to compile a source code using the library, it gives me the following error:
/usr/bin/ld: cannot find -lsslab_core
When I move the library to /usr/local/lib, update the content of sslab.conf, and execute ldconfig as a root. I can use the shared library without any problems.
Do you have any ideas about the problem that I've come across on Ubuntu 12.04?
For your information, I refer to a document in TLDP to generate my own shared library. Here is the link: http://www.tldp.org/HOWTO/Program-Library-HOWTO/
I need to parse the loadable parts of an ELF executable for a simple loader. Found the nice "libelf by Example" tutorial (by J. Koshy) that gives a nice overview on the structure of an ELF. It then lead me to the ELF Toolchain project, which implements, among other things, the libelf library of ELF handling functions.
The nice thing about using this library is that it encapsulates the variations in ELF formats and makes the code more portable and future proof. However, I found out that in order to be able to build the package on Ubuntu, I needed to install a great amount of prerequisites.
I tried to build only the libelf part by running pmake in the libelf directory, but got a build error complaining on missing header file - which is apparently automatically generated by the global make process and is architecture dependent.
My question is - how can I build just the libelf part of ELF Toolchain?
You can build select parts of the Elftoolchain source tree by commenting out the appropriate SUBDIR lines in the top-level Makefile.
For building libelf, you should be able to get by with the following two directories:
% grep ^SUBDIR Makefile | head -2
SUBDIR += common
SUBDIR += libelf
Note: On Ubuntu, you could also use GNU libelf, which offers a different implementation of the same API, and for which there appears to be ready-made packages.
My build process consists of Qt's qmake Makefile generator and the typical make utility bundled with linux.
My application consists of a few shared libraries and the main application is linked against them.
How can I apply the typical linux versioning scheme on my libraries? (Use version 2 -> link against foo.so.2 that points to foo.so.2.y.z with an ldconfig generated link).
The answer doesn't have to be specific for my build process.
Your library should be named libfoo.so.2.y.z, with symlinks of libfoo.so.2 and libfoo.so both pointing to that. The library should be created using -soname libfoo.so.2 in the linker command line (or -Wl,-soname,libfoo.so.2 on the gcc command line).
Hope that helps!