I am building an RPM which is essentially just packaging a set of vendor provided .so binaries so that they are available for our internal application, which is also installed via RPM.
One of the libraries comes with multiple versions - libexample_base.so and libexample_debug.so (among others). I am currently trying to package these so that they are all included in the RPM (so that developers can switch between them if needed), but picks libexample_base.so as the default version by creating a symlink during %install, which then gets packaged as a file in the RPM.
%install
[... Copy the files from the tarball to the buildroot ...]
pushd %{buildroot}%{_libdir}
ln -sf libexample_base.so libexample.so
popd
This works great... except for one problem. It's using automatic dependency generation, and while it's providing all of the shared objects for which is has actual files, it's not providing libexample.so, despite the symlink being in %files and installing properly. Unfortunately, the vendor libraries do not provide SONAME entries, and as they are binary blobs I can't readily add them, so RPM is depending on the actual file names. All of the downstream RPMs require libexample.so, and since this RPM is not listing it as a requires, they are refusing to install due to missing dependencies, even though the do actually work (ldconfig can find libexample.so without issue).
Any ideas on how to prompt rpmbuild to parse the symlink as a provides?
After some further research, I have determined that what I am trying to do is not possible, as well as why. The core problem comes down to rpmbuild's behavior, which was built to handle correctly generated shared objects with SONAME entries. It explicitly does not list symlinks to shared objects that end in .so as provides because with normal behavior, those point to a versioned shared object (.so.#.#), and your application is supposed to depend on those versioned objects - the symlink is meant to be included in the devel package just to give your linker a way to find the latest one.
I'm running into a second case that is used as a backup for when things aren't done correctly - for both RPM and GCC, when no SONAME is present, it uses the file name. When I run GCC, the filename is libexample.so (the symlink to the real one), and it has no SONAME, so it just configures itself to link against libexample.so; rpmbuild sees this and sets that as a requires for the application RPM. However, since rpmbuild is explicitly excluding that name from the library rpm since it looks like a devel link, there is no way to reconcile the two rpms.
The solution? For now, I'm using my workaround, just making a copy of the file - when there's a physical file with that name, it works. The correct way is to fix the shared objects, although I'm going to need to go to the upstream vendor to get that done.
Related
I'm zipping a pre-built (no source/object files) binary application for distribution. The binary application requires a couple of libraries not included by default. The only way I seem to be able to get the application to start on the end-user is by including a run.sh that sets the library path to the current directory:
export LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH
./MyApp.out
However, I'd really like to allow the user to just unzip the zip and doubleclick MyApp.out (without the shell script). Can I edit MyApp.out to search the current directory for the library? I've done something similar on OSX using install_name_tool, but that tool isn't available here.
You want to set the rpath. See this answer. So link using
gcc yourobjects*.o -L/some/lib/dir/ -lsome -Wl,-rpath,.
But you might want even to use -Wl,-rpath,$PWD or perhaps -Wl,-rpath,'$ORIGIN'. See this.
You could also (and this should work for a pre-built executable) configure your /etc/ld.so.conf by adding a line there with an absolute path (of the directory containing the lib), then running ldconfig -v ... See ldconfig(8)
I would suggest adding /usr/local/lib into /etc/ld.so.conf and making a symlink from /usr/local/lib/libfoo.so to e.g. $HOME/libfoo.so etc... (then run ldconfig ...). I don't think adding a user specific directory to /etc/ld.so.conf is reasonable ...
PS. What you really want is to package your application (e.g. as a *.deb package for Debian or Ubuntu, or an *.rpm for Fedora or Redhat). Package management systems handle dependencies!
In short: This question is basically about telling Linux to load the development version of the .so file for executables in the dev directory and the installed .so file for others.
In long: Imagine a shared library, let's call it libasdf.so. And imagine the following directories:
/home/user/asdf/lib: libasdf.so
/home/user/asdf/test: ... perform_test
/opt/asdf/lib: libasdf.so
/home/user/jkl: ... use_asdf
In other words, you have a development directory for your library (/home/user/asdf) and you have an installed copy of its previous stable version (/opt/asdf) and some other programs using it (/home/user/jkl).
My question is, how can I tell Linux, to load /home/user/asdf/lib/libasdf.so when executing /home/user/asdf/test/perform_test and to load /opt/asdf/lib/libasdf.so when executing /home/user/jkl/use_asdf? Note that, even though I specify the directory by -L during link, Linux uses other methods (for example /ect/ld.so.conf and $LD_LIBRARY_PATH) to find the .so file.
The reason I need such a thing is that, of course the executables in the development directory need to link with the latest version of the library, while the other programs, would want to use the stable version.
Putting ../lib in the library path doesn't seem like a secure idea, not to mention not completely correct since you can't run the test from a different directory.
One solution I thought about is to have perform_test link with libasdf-dev.so and upon install, copy libasdf-dev.so as libasdf.so and have others link with that. This solution has one problem though. Imagine the following additional directory:
/home/user/asdf/tool: ... use_asdf_too
Which gets installed to:
/opt/asdf/bin: use_asdf_too
In my solution, it is unknown what use_asdf_too should be linked against. If linked against libasdf.so, it wouldn't work properly if invoked from the dev directory and if linked against libasdf-dev.so, it wouldn't work properly if invoked from the installed location.
What can I do? How is this managed by other people?
Installed shared objects usually don't just end with ".so". Usually they also include their soname, such as libadsf.so.42.1. The .so file for development is typically a symlink to a fully-versioned filename. The linker will look for the .so file and resolve it to the full filename, and the loader will then load the fully-versioned library instead.
I've got a problem with shared libraries and gcc. At first I couldn't run my compiled program because I was getting the following error: gcc error while loading shared libraries.
I did some searching and found that this is because the shared library cannot be found. However I had already identified that the shared library is in /usr/local/lib, which AFAICT is a commonly used directory for shared libraries and should work from the get go.
I read that you can set LD_LIBRARY_PATH, which worked for me. However I do not wish to set this each time I want to run my program.
Further searching suggested editing ld.so.conf. When I looked in this it had the following:
include /etc/ld.so.conf.d/*.conf
Looking in the ld.so.conf.d directory shows me a range of files, including libc.conf. Inside this file is the following:
/usr/local/lib
So my question is, why do I need to manually set LD_LIBRARY_PATH when the ld.so.conf appears to use the libc.conf which includes /usr/local/lib?
Is there something that I'm missing here that must be configured first? Is there an option at compile time that I'm missing?
I should note that to compile, I had to specify the path to the library, I don't know if this is a symptom of my problem or normal behaviour.
I should also note that this is a concern for me for when I deploy my software on other systems. I would have thought that I should be able to put the .so in the appropriate place and install my program without messing with ld.so.conf.
I hope this is the proper forum for this question, I read the FAQ and I think it's ok.
Cheers.
You should run ldconfig (as root) after every change of the directories configured via /etc/ld.so.conf or under /etc/ld.so.conf.d/, in particular in your case after every update inside /usr/local/lib (e.g. after every addition or update of some shared libraries there).
I have an application that relies on Qt, GDCM, and VTK, with the main build environment being Qt. All of these libraries are cross-platform and compile on Windows, Mac, and Linux. I need to deploy the application to Linux after deploying on Windows. The versions of vtk and gdcm I'm using are trunk versions from git (about a month old), more recent than what I can get apt-get on Ubuntu 11.04, which is my current (and only) Linux deployment target.
What is the accepted method for deploying an application that relies on these kinds of libraries?
Should I be statically linking here, to avoid LD_LIBRARY_PATH? I see conflicting reports on LD_LIBRARY_PATH; tutorials like this one suggest that it's the 'right way' to modify the library path to use shared libraries through system reboots. Others suggest that I should never set LD_LIBRARY_PATH. In the default version of GDCM, the installation already puts libraries into the /usr/local/lib directory, so those libraries get seen when I run ldd <my program>. VTK, on the other hand, puts its libraries into /usr/local/lib/vtk-5.9, which is not part of the LD_LIBRARY_PATH on most user's machines, and so is not found unless some change is made to the system. Copying the VTK files into '/usr/local/lib' does not allow 'ldd' to see the files.
So, how can I make my application see VTK to use the libraries?
On windows, deploying the dlls is very straightforward, because I can just include them in the installer, and the application finds them because they are in the local directory. That approach does not work in Linux, so I was going to have the users install Qt, GDCM, and VTK from whatever appropriate source and use the default locations, and then have the application point to those default locations. However, since VTK is putting things into a non-standard location, should I also expect users to modify LD_LIBRARY_PATH? Should I include the specific versions of the libraries that I want and then figure out how to make the executable look in the local directory for those libraries and ignore the ones it finds in the library path?
Every "serious" commercial application I have ever seen uses LD_LIBRARY_PATH. They invariably include a shell script that looks something like this:
#!/bin/sh
here="${0%/*}" # or you can use `dirname "$0"`
LD_LIBRARY_PATH="$here"/lib:"$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
exec "$0".bin "$#"
They name this script something like .wrapper and create a directory tree that looks like this:
.wrapper
lib/ (directory full of .so files)
app1 -> .wrapper (symlink)
app1.bin (executable)
app2 -> .wrapper (symlink)
app2.bin (executable)
Now you can copy this whole tree to wherever you want, and you can run "/path/to/tree/app1" or "/path/to/tree/app2 --with --some --arguments" and it will work. So will putting /path/to/tree in your PATH.
Incidentally, this is also how Firefox and Chrome do it, more or less.
Whoever told you not to use LD_LIBRARY_PATH is full of it, IMHO.
Which system libraries you want to put in lib depends on which Linux versions you want to officially support.
Do not even think about static linking. The glibc developers do not like it, they do not care about supporting it, and they somehow manage to break it a little harder with every release.
Good luck.
In general, you're best off depending on the 'normal' versions of the libraries for whatever distribution you're targetting (and saying you don't support dists that don't support recent enough versions of the lib), but if you REALLY need to depend on a bleeding edge version of some shared lib, you can link your app with -Wl,-rpath,'$ORIGIN' and then install a copy of the exact version you want in the same directory as your executable.
Note that if you use make, you'll need $$ in the makefile to get a single $ into the argument that is actually sent to the linker. The single qutoes are needed so the shell doesn't munge things...
Well, there are two options for deploying Linux application.
The correct way:
make a package for your app and for the libraries, if they are so special, that they can't be installed from standard repositories
There are two major package formats. RPM and DEB.
The easy way:
make a self-extract file that will install the "windows way" into /opt.
You can have libraries in the same directory as the executable, it's just not the preferred way.
I am using libcurl for my utility and its working very well till now for all Linux platforms. I downloaded, unzipped and simply followed the instructions given without any changes. My product uses the libcurl.so file and is linked dynamically. The .so file is bundled along with our product. Recently there were issues in Suse wherein we found that Libcurl is bundled by default and there was a conflict in installation.
To avoid this issue we tried renaming the libcurl.so into libother_curl.so but it did not work and my binaries still show libcurl.so as a dependency through ldd. I had since learnt that the ELF format of linux shared objects specifies the file name hardcoded as SO file name in the headers.(I could verify the same with objdump -p).
Now my question is what is the simplest way to go? How do I build a libcurl with a different name? My original process involves running configure with the following switches
./configure --without-ssl --disable-ldap --disable-telnet --disable-POP3 --disable-IMAP --disable-RTSP --disable-SMTP --disable-TFTP --disable-dict --disable-gopher --disable-debug --enable-nonblocking --enable-thread --disable-cookies --disable-crypto-auth --disable-ipv6 --disable-proxy --enable-hidden-symbols --without-libidn --without-zlib
Make
Then pick the generated files from /lib/.libs
Are there any Configure Switches available wherein I can specify the target file name? Any specific Makefile I could change?
I tried changing in what I thought could be obvious locations but either could not generate the libs or were generated with the same name.
Any help is much appreciated.
I got the answer from the curl forums(Thanks Dan). Basically we have to use the makefile.am as a starting point to go through a list of files and change the library name "libxxx_curl".
$find . -name Makefile.am |xargs sed -i 's/libcurl(.la)/libxxx_curl\1/g'
$buildconf
$configure
$make
I lot of commercial applications bundle their particular library versions in a non standard path and then tweak environment variable LD_LIBRARY_PATH in a launch script so to avoid conflict. IMHO it is better than trying to change the target name.