Installation and maintenance of multiple versions of OpenCV (applicable to any other 3rd party library as well) - linux

I have been trying to do build and use OpenCV 2.3.0 on my Fedora15 Lovelock 64bit machine.
Background:
First, on my 64bit Fedora15, OpenCV2.2.0 seems to be in the locations namely
/usr/share/opencv
/usr/doc
/usr/lib64 &
/usr/bin
I do not find the include files though (in /usr/include). This means that the development package was n t installed. My package manager does not list the development packages when i try to Add/remove software.
I have a need to create applications, some of which just link to 2.2 and others which link to 2.3.O of the OpenCV library.So, I thought the best solution would be to have a separate location for 3rd party libraries that i use for my development . So I created a directory in /local named soft and created an OpenCV directory. A directory structure like this one.
/local/soft/
OpenCV/
OpenCV2.2.0/
source-files
build
OpenCV2.3.0/
source-files
build
installation
share/opencv
doc
include
lib
Now, i tried building OpenCV2.3.0 and i succeeded. I configure CMake to use CMAKE_INSTALL_PREFIX to the directory named "installation" (see above), instead of the default /usr/local/. Clean. huh?
I tried building and installing OpenCv 2.2.0 in the same way. Alas 2.2.0 complains something during the build. So i thought i ll link to the already existing version in the standard locations. BUT, when i try to install the dev packages for 2.2 using my package manager,the development files for x86_64 are not found :-) which means i dont have the headers to link to the libraries in the standard location.
I cant build my executable since linker ld would not find the OpenCV that i have installed in the non-standard location.(although i point it to the exact location using the -L and -l options with gcc in my Eclipse).
Question 1: Am i doing the right thing in maintaining installations in non-standard locations? Is /usr/ the standard location where the package manager will always do the installation?
Question2 : What is the right way of linking to these libraries installed in non-standard locations? Why would not ld recognize my .so files in the lib folder?
sudo g++ logpolar.cpp -o logpolar.o -I /local/soft/OpenCV/opencv2.3.1/installation/include/ -l/local/soft/OpenCV/opencv2.3.1/build/lib/libopencv_core.so
But ld canot find -l/local/soft/OpenCV/opencv2.3.1/build/lib/libopencv_core.so
I checked the lib folder and there sure is a beautiful symbolic link to libopencv_core.so.2.3

The standard approach is to use /usr/local directory structure that already has predefined paths like /usr/local/bin, /usr/local/sbin, /usr/local/include, /usr/local/lib.
You put your software here and everything will JustWork(TM). Every Linux distro (incl. Fedora) is set up so it will load programs (libraries, headers) from this libraries.
If you would use GNU toolchain (autoconf, automake => autotools) you would be fine. With CMake you probably need to setup paths for /usr/local/include and /usr/local/lib.
On the other hand this approach wont let you use multiple versions. You can only have one. The one in /usr/local overrides the system one (installed in /usr/bin) because these paths goes first.
You can keep your approach, it is nothing incorrect. We usually put such a software in the /opt folder, so you would go for /opt/opencv/X.Y where X.Y are the version numbers.
Q2: Read the gcc man page and search for the -L option. You need something like:
gcc ... -I/opt/opencv/2.0/include -lsystem_lib -L/opt/opencv/2.0/lib -lopencv ... ...
Do not forget to set LD_LIBRARY_PATH when running programs in multiple versions to properly load correct version:
LD_LIBRARY_PATH=/opt/opencv/2.0/lib /opt/opencv/2.0/bin/opencv

Related

How to make Cmake globally available

I just installed Cmake from git clone wget http://www.cmake.org/files/v2.8/cmake-2.8.3.tar.gz in a new folder on a Linux server. The compilation worked but cmake command is not recognized from other paths. Should I copy the entire contents of cmake-2.8.0 folder to usr/local/bin? Or is the contents of bin folder that need to be copied?
Thanks
On Linux and other Unix-based systems, a common arrangement is to install packages to /opt and add relevant entries to the PATH environment variable to make them available. This is intended for packages not provided by the native package manager or distribution. By choosing an appropriate directory structure, this can be done in a way which also allows different versions to be installed simultaneously and the user can pick which one they want by adding the relevant directory to the PATH.
For the specific case of CMake asked about in the question, you can use a directory structure like /opt/cmake/<version> and then add the relevant /opt/cmake/<version>/bin directory to your PATH (e.g. /opt/cmake/3.8.2/bin for the 3.8.2 CMake release). You can even just download the official pre-built CMake tarballs, unpack them and move the top level directory into the /opt/cmake area as the particular version you downloaded. I've used this successfully on Linux, MacOS and Solaris, as I'm sure have many others.
Note that once you've run CMake on a particular source tree, the cmake executable doesn't need to be on the PATH any more. If cmake needs to be re-run, the build will do so itself and it records the full path to the cmake executable in its own cache, so the PATH isn't even consulted (this is essential in ensuring the same version of CMake continues to be used for all builds regardless of the PATH, since PATH can change between login sessions, etc.). You would only need cmake on your PATH if you intend to invoke cmake manually or for the first time you run it on a source tree, but in both of these cases you can always just use the full path to the cmake executable if you preferred.
I should also add that the entire set of files provided in the CMake package are required, not just the bin directory. CMake makes extensive use of files in its other directories, such as the various modules it comes with. If you are building CMake from source, you may want to build the package target so you get a relocatable tarball or similar which will contain everything that should be included when you provide a CMake package on your system.
After the build, use 'sudo make install'. This will make sure the correct libraries and binaries are copied to their proper places.
Usually this will install the binary to /usr/local/bin.
Make sure the PATH variable has this included.
sudo make install did not copy to /usr/local/bin/ for some reason, so I copied the content of CMAKE /bin. to usr/local/bin an it worked.
cp –a bin/. /usr/local/bin/

Can I avoid exporting LD_LIBRARY_PATH by hardcoding library paths in the executable?

I'm zipping a pre-built (no source/object files) binary application for distribution. The binary application requires a couple of libraries not included by default. The only way I seem to be able to get the application to start on the end-user is by including a run.sh that sets the library path to the current directory:
export LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH
./MyApp.out
However, I'd really like to allow the user to just unzip the zip and doubleclick MyApp.out (without the shell script). Can I edit MyApp.out to search the current directory for the library? I've done something similar on OSX using install_name_tool, but that tool isn't available here.
You want to set the rpath. See this answer. So link using
gcc yourobjects*.o -L/some/lib/dir/ -lsome -Wl,-rpath,.
But you might want even to use -Wl,-rpath,$PWD or perhaps -Wl,-rpath,'$ORIGIN'. See this.
You could also (and this should work for a pre-built executable) configure your /etc/ld.so.conf by adding a line there with an absolute path (of the directory containing the lib), then running ldconfig -v ... See ldconfig(8)
I would suggest adding /usr/local/lib into /etc/ld.so.conf and making a symlink from /usr/local/lib/libfoo.so to e.g. $HOME/libfoo.so etc... (then run ldconfig ...). I don't think adding a user specific directory to /etc/ld.so.conf is reasonable ...
PS. What you really want is to package your application (e.g. as a *.deb package for Debian or Ubuntu, or an *.rpm for Fedora or Redhat). Package management systems handle dependencies!

What's the accepted method for deploying a linux application that relies on shared libraries?

I have an application that relies on Qt, GDCM, and VTK, with the main build environment being Qt. All of these libraries are cross-platform and compile on Windows, Mac, and Linux. I need to deploy the application to Linux after deploying on Windows. The versions of vtk and gdcm I'm using are trunk versions from git (about a month old), more recent than what I can get apt-get on Ubuntu 11.04, which is my current (and only) Linux deployment target.
What is the accepted method for deploying an application that relies on these kinds of libraries?
Should I be statically linking here, to avoid LD_LIBRARY_PATH? I see conflicting reports on LD_LIBRARY_PATH; tutorials like this one suggest that it's the 'right way' to modify the library path to use shared libraries through system reboots. Others suggest that I should never set LD_LIBRARY_PATH. In the default version of GDCM, the installation already puts libraries into the /usr/local/lib directory, so those libraries get seen when I run ldd <my program>. VTK, on the other hand, puts its libraries into /usr/local/lib/vtk-5.9, which is not part of the LD_LIBRARY_PATH on most user's machines, and so is not found unless some change is made to the system. Copying the VTK files into '/usr/local/lib' does not allow 'ldd' to see the files.
So, how can I make my application see VTK to use the libraries?
On windows, deploying the dlls is very straightforward, because I can just include them in the installer, and the application finds them because they are in the local directory. That approach does not work in Linux, so I was going to have the users install Qt, GDCM, and VTK from whatever appropriate source and use the default locations, and then have the application point to those default locations. However, since VTK is putting things into a non-standard location, should I also expect users to modify LD_LIBRARY_PATH? Should I include the specific versions of the libraries that I want and then figure out how to make the executable look in the local directory for those libraries and ignore the ones it finds in the library path?
Every "serious" commercial application I have ever seen uses LD_LIBRARY_PATH. They invariably include a shell script that looks something like this:
#!/bin/sh
here="${0%/*}" # or you can use `dirname "$0"`
LD_LIBRARY_PATH="$here"/lib:"$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
exec "$0".bin "$#"
They name this script something like .wrapper and create a directory tree that looks like this:
.wrapper
lib/ (directory full of .so files)
app1 -> .wrapper (symlink)
app1.bin (executable)
app2 -> .wrapper (symlink)
app2.bin (executable)
Now you can copy this whole tree to wherever you want, and you can run "/path/to/tree/app1" or "/path/to/tree/app2 --with --some --arguments" and it will work. So will putting /path/to/tree in your PATH.
Incidentally, this is also how Firefox and Chrome do it, more or less.
Whoever told you not to use LD_LIBRARY_PATH is full of it, IMHO.
Which system libraries you want to put in lib depends on which Linux versions you want to officially support.
Do not even think about static linking. The glibc developers do not like it, they do not care about supporting it, and they somehow manage to break it a little harder with every release.
Good luck.
In general, you're best off depending on the 'normal' versions of the libraries for whatever distribution you're targetting (and saying you don't support dists that don't support recent enough versions of the lib), but if you REALLY need to depend on a bleeding edge version of some shared lib, you can link your app with -Wl,-rpath,'$ORIGIN' and then install a copy of the exact version you want in the same directory as your executable.
Note that if you use make, you'll need $$ in the makefile to get a single $ into the argument that is actually sent to the linker. The single qutoes are needed so the shell doesn't munge things...
Well, there are two options for deploying Linux application.
The correct way:
make a package for your app and for the libraries, if they are so special, that they can't be installed from standard repositories
There are two major package formats. RPM and DEB.
The easy way:
make a self-extract file that will install the "windows way" into /opt.
You can have libraries in the same directory as the executable, it's just not the preferred way.

How to link shared libraries in local directory, OSX vs Linux

I have some shared/dynamic libraries installed in a sandbox directory. I'm building some applications which link agains the libraries. I'm running into what appears to be a difference between OSX and Linux in this regard and I'm not sure what the (best) solution is.
On OSX the location of library itself is recorded into the library, so that if your applications links against it, the executable knows where to look for the library at runtime. This works like expected with my sandbox, because the executable looks there instead of system wide install paths.
On Linux I can't get this to work. Apparently the library location is not present in the library itself. As I understand it you have to add the folders which contain libraries to /etc/ld.so.conf and regenerate the ld cache by running ldconfig.
This doesn't seem to do the trick for me because my libraries are located inside a users home directory. It looks like ldconfig doesn't like that, which makes sense actually.
How can I solve this? I don't want to move the libraries out of my sandbox.
On Linux, run your program with the environment variable LD_LIBRARY_PATH set to your sandbox dir.
(I remember having used a flag -R to include library paths in the binary, but either it has been removed from gcc or it was only available on BSD systems.)
On Linux you should set LD_RUN_PATH to your sandbox dir. This is better than setting LD_LIBRARY_PATH because you're telling the linker where the library is at link time, rather than telling the shared library loader at run time.
See: Link

Install multiple versions of a package

I want to install multiple versions of a package (say libX) from src. The package (libX) uses Autotools to build, so follows the ./configure , make, make install convention. The one installed by default goes to /usr/local/bin and /usr/local/lib and I want to install another version of this in /home/user/libX .
The other problem is that libX is a dependency for another package (say libY) which also uses autotools. How to I make libY point to the version installed in /home/user/libX ? There could be also a possibility that its a system package like ffmpeg and I want to use the latest svn version for my src code and hence build it from src. What do i do in that case ? What is the best practice in this case so that I do not break the system libraries?
I'm using Ubuntu 10.04 and Opensuse 10.3.
You can usually pass the --prefix option to configure to tell it to install the library in a different place. So for a personal version, you can usually run it as:
./configure --prefix=$HOME/usr/libX
and it will install in $HOME/usr/libX/bin, $HOME/usr/libX/lib, $HOME/usr/libX/etc and so on.
If you are building libY from source, the configure script usually uses the pkg-config tool to find out where a package is stored. libX should have included a .pc file in the directory $HOME/usr/libX/lib/pkgconfig which tells configure where to look for headers and library files. You will need to tell the pkg-config tool to look in your directory first.
This is done by setting the PKG_CONFIG_PATH to include your directory first.
When configuring libY, try
PKG_CONFIG_PATH=$HOME/usr/libX/lib/pkgconfig:/usr/local/lib/pkgconfig ./configure
man pkg-config should give details.

Resources