When I install a program using apt-get install, it tells me which dependency libraries also need to be installed.
For example, Nginx requires libgd3 (3d graphics library) to be installed.
When Nginx needs to call code in libgd3 how does it know where the file is and
How does it actually go about it? I assume it must load it into the Nginx process heap and then use some kind of function table to make calls?
I am not sure how this process works, thanks.
The libraries are in standard path usually /usr/lib which the linker searches during linking. The dynamic libraries are called shared objects in linux having the extension .so .
Check this link to know more about Linux Libraries:
http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html
Usually through the environment variable LD_LIBRARY_PATH which is set of directories where libraries should be searched for first
See what it is set to :
env | grep LD_LIBRARY_PATH
You can update the new location to search temporarily by
export $LD_LIBRARY_PATH = $LD_LIBRARY_PATH:/new/dir/to/look
There is every chance that this env variable is not present in your distro. So you can try the below
1) Add library directories to /etc/ld.so.conf or
2) add it to the library cache by using ldconfig
Please read more here
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html
Once a program is installed in Linux, sometimes I find out that it is easier to put in a different location. In general, what is the significance of the location of the files of an installed program on Linux?
Often the advice on the internet is to add the (wrong or inconvenient) paths to environment variables. I'd much rather move the files to locations where they are automatically found by commands and programs.
One recent example is site-packages of Python. My Python did not appear to check the PYTHONPATH variable, moving the libraries there to the Python2.7/ directory worked well.
Now Ia m facing the same issue with OpenCV.
I also wonder why Linux installation does not prompt (like Windows) for the desired installation directory and why, so often, things wind up in places where they don't work?
In general, programs are installed in /usr/bin (for binaries) and /usr/lib, or a specific path to that specific linux distro, so that any program that you install that uses a specific library/program will search in that path for it. If you install a program in a different path, let's say /home/user/program, it will be installed locally and other programs won't be able by default to access it.
You can install any program wherever you want. However, it is good use to use the repo and install them in the general path.
I don't know how you install programs, but I use apt-get and dpkg on Ubuntu. You can also install some python modules this way.
Generally you are supposed to use the package system provided by your distro (IMHO).
If you do not use packages then you are on your own.
About PYTHONPATH. Did you add it to your .bashrc and made sure that it was set in the terminal you are using?
Also please see:
http://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard
Consider these 4 pieces of software:
COBRA 2.05
LibSBML 5.10
MATLAB R2013a (Also known as 8.1, 64-bit; MATLAB no longer supports 32-bit Linux anyway)
A 64-bit Linux OS (such as Ubuntu 14.04 or the latest Mint but not restricted to them)
Intro
The COBRA toolbox is an optimization suite that runs on top of MATLAB aimed at the development of MATLAB code for metabolic network modelling. Such a "network" is a system of equations that can have a very large number of equations and variables (such as thousands). Therefore, routines to read and write those large models according to some format specification are a must-have, and COBRA uses the standard SBML for that.
Problem
Unlike the Windows versions, the Linux binary packages do not integrate well out-of-the-box: to begin with, the pre-compiled Linux binary of libSBML (open-source) available for download does not come with MATLAB support. If one tries to use the pre-compiled libSBML, COBRA won't find the "MATLAB bindings" and therefore won't be able to, for example, read and write SBML XML files from the disk in a m-script.
The question
What needs to be done to make COBRA 2.05 running on top of MATLAB R2013a under Linux (Ubuntu 14.04 or the latest Mint, but this is not likely distro-specific) able to read and write SBML XML files? In other words, what needs to be done system-wide to make COBRA pass its own testSBML test?
This is how I got it working and what I could learn from all the hassle regarding how my Linux box works. I hope I am not forgetting/missing/mistaking anything.
1. MATLAB
1.1. Install MATLAB
Install it in its default location (you will need root access for this), don't be stubborn like I tried to be. Why:
1.1.1. Integration
It is very likely you will want to install some other software that uses the MATLAB framework at some point, and in the real world software doesn't always find other software even if you know how to tell it where to look for.
1.1.2. Make your life easier
Even though it seems like a good idea to install a big software in a place where you have lots of available space and that you can use in multiple machines (specially in Linux, which doesn't have that abomination called Registry, and has symbolic links), that would only perhaps be a good idea - apart from item 1.1.1 - if that place is a partition with a Linux filesystem, since at some point, some executable/script will need execution permission, and mounting the entire partition with execution permission for all its files is rather unsafe. Therefore, do not put MATLAB in an NTFS partition of an external HD; perhaps creating a Linux partition in the external HD just for Linux-specific stuff could work for this matter, but how much hassle is that?
1.2. Setup MATLAB so people and other software can launch it
Even though I have seen somewhere that the MATLAB installer eventually shows an option to create symbolic links in the system path for convenience, it didn't in my case. But that is OK, since I would have to replace the symbolic link /usr/local/bin/matlab by the following shell-script (same path, same filename) anyway:
#!/bin/sh
export LD_LIBRARY_PATH=/usr/local/lib:/usr/lib/x86_64-linux-gnu:$LD_LIBRARY_PATH
exec /usr/local/MATLAB/R2013a/bin/matlab $*
OBS: That LD_LIBRARY_PATH is needed for MATLAB to find SBML bindings later and to be able to use them. Also, if you install some third-party solver such as TOMLAB, you will most likely need to add some more paths in this little launcher script.
OBS 2: In my case, the installation script didn't automatically create any launchers or shortcuts, but I have found an iconless and extension-less Matlab 8.01 file and a matlab icon as a png file, and that first file was a template .desktop file that I could edit to fit my needs and put in /usr/share/applications so the Unity Dash would find it. The contents of this Matlab.desktop file are:
[Desktop Entry]
Type=Application
Icon=/usr/local/MATLAB/R2013a/Matlab.png
Name=Matlab 8.01
Comment="Start Matlab 8.01"
Exec=/usr/local/MATLAB/R2013a/bin/matlab -desktop
Categories=Development;
Name=Matlab 8.01
GenericName=Matlab 8.01
Comment="Start Matlab 8.01"
2. libSBML
2.1. Install libSBML
libSBML is provided by a deb package specific for Ubuntu (and for CentOS), and also versions for several flavours of Windows and MacOSX (their home page: http://sbml.org/Software/libSBML). Guess which is the only platform whose binaries weren't compiled with MATLAB support? Linux, of course. That means you will need to compile from source (and that the deb package is therefore useless to you). To compile:
2.1.1. Install dependencies
The dependency libxml2-dev (from software manager or from a terminal):
sudo apt-get install libxml2-dev
2.1.2. Save yourself some time in the future
Usually, one would do configure, make and then make install. But this is not recommended for the same reason as installing anything that doesn't come in a pretty little package: you will loose control of which files went where, and will need to keep the source-code to be able to run make uninstall if you need to uninstall it later. So, install checkinstall and use it to replace the step make install, since checkinstall creates a package for your system that can be later uninstalled or reinstalled just as any regular packaged software (from software manager or from a terminal):
sudo apt-get install checkinstall
2.1.3. Configure the compiling-process
Get LibSBML source code and extract it to some folder. From a terminal, navigate to that folder and configure the compilation:
./configure --with-matlab
OBS: with the with-matlab flag, the configure script will fail it it cannot find an executable whose filename is matlab. If it fails, it outputs that the matlab file could not be found, but the test it performs is actually both for the existence of the file and whether it is executable. So, if the file is in an NTFS partition, configure will fail even if it finds the file, but will tell you the file couldn't be found. You can enforce it to look for the executable in /path/to/matlab/root by passing (it will look for a bin folder inside that path, and for the executable inside that bin folder):
./configure --with-matlab=/path/to/matlab/root
OBS: This will install libSBML in the default location: /usr/local/lib. Again, it is a good idea to just let it install in its default location, but if you need to change it, you can pass the path with the flag: --prefix=/your/installation/path
OBS 2: You might ask why libSBML needs to find and execute matlab to be compiled with support for it: it needs to fire up MATLAB later to build MEX-files (compiled MATLAB code), so I would speculate you wouldn't be able to install libSBML after all if your MATLAB has no compiler to generate MEX-files.
2.1.4. Build and install libSBML
make
checkinstall
VERY IMPORTANT OBS:
I) checkinstall asks for confirmation of the metadata of the package it is about to create. In my case, the string for the version field came by default as "Source" (without the quotes), which caused checkinstall to fail because dpkg (the system tool that actually builds the deb file) failed complaining the version number must start with, well, a number. So, save yourself some time and make sure the string in the version field starts with a number (i.e. "5.10", without the quotes obviously)
II) checkinstall asks if you want to exclude from the future package files that the make install command would put in your home folder and says it is a good idea to exclude them. LibSBML has a test.xml file that it needs to be in the $HOME folder later, or else it won't let you integrate with MATLAB. And even though it tells you a test.xml is missing, it doesn't tell you where that file should be or if that file was something that came with the library. I only noticed it because checkinstall had found a $HOME/test.xml reference earlier (that I excluded from the package, of course) and I had found that odd. So, save yourself some time and exclude $HOME/test.xml from the package generated by checkinstall, and then search for test.xml inside the source-code folder and copy it to $HOME as soon as libSBML finishes being installed by checkinstall.
2.2. Integrate libSBML to MATLAB
Fire up MATLAB, navigate to where the SBML MATLAB-bindings were installed in step 2.1.5 (in my case: /usr/local/lib) and run the file installSBML.m that should be there.
2.2.1. Shared libraries problems
In my case, I had an error due to an old unresolved issue: libstdc++.so.6 not having a reference to GLIBCXX_3.4.15. Turns out that MATLAB was trying to use a libstdc++.so.6.0.13 (libstdc++.so.6 was a symbolic link pointing to this file) that came with it in /usr/local/MATLAB/R2013a/sys/os/glnxa64, which indeed didn't have that reference (one could verify that by issuing:
strings /usr/local/MATLAB/R2013a/sys/os/glnxa64/libstdc++.so.6.0.13 | grep GLIBC
). My system has a libstdc++.so.6.0.19 located in /usr/lib/x86_64-linux-gnu that has that reference, so I enforced MATLAB to use 6.0.19 one by setting the LD_LIBRARY_PATH properly (refer to step 1.2) and also by renaming the libstdc++.so.6 that came with MATLAB to something else so it would not find it and would keep looking until it found my system's. A friend of mine running Linux Mint didn't need to rename anything: for him, setting the LD_LIBRARY_PATH was enough.
2.2.2. Other problems
installSBML.m will fail if it doesn't find that $HOME/test.xml file mentioned in step 2.1.5, regardless of whether the library actually works. It assumes that if it could not test itself using a file that it assumes to be in $HOME, the user shouldn't have the option to install it anyway.
3. COBRA / SBML toolbox
3.1. Setup COBRA
In MATLAB, navigate to <YOUR_COBRA_ROOT_FOLDER_HERE>/external/toolboxes/SBMLToolbox-4.1.0/toolbox and run the file install.m there. You should have all set so it finds the MATLAB-bindings you set up in step 2.2.
3.2. MATLAB setpaths problems
I had to manually edit the file /usr/local/MATLAB/R2013a/toolbox/local/pathdef.m as root to include the folder /usr/local/lib (where libSBML and its MATLAB-bindings are) to make that setting persistent. Every time I restarted MATLAB, its setpath had gone back to the default, no matter if I started MATLAB as root when setting its setpath via the MATLAB GUI.
3.3. Test
Now you have hopefully connected all the dots. Try it: in MATLAB, navigate to <YOUR_COBRA_ROOT_FOLDER_HERE> and issue:
initCobraToolbox
testAll
If you haven't got any third-party solvers installed and configured, it should pass 14 of the 19 tests, including the SBML test (testSBML). Now you can load SBML files into MATLAB and play with them.
I also needed to add a symbolic link from /usr/local/lib/libsbml.so.5 to the MATLAB sys folder by:
sudo ln -s /usr/local/lib/libsbml.so.5 /usr/local/MATLAB/R2014a/sys/os/glnxa64/
This finally made the installation possible.
I installed using Cmake. To do this it is necessary to find the FindMatlab.cmake in the source package and insert the MATLAB path manually!
.............
elseif(EXISTS "/Applications/MATLAB_R2008a.app/")
set(MATLAB_ROOT_PATH "/Applications/MATLAB_R2008a.app/")
endif()
else()
if (EXISTS "/usr/local/MATLAB/R2014a/")
set(MATLAB_ROOT_PATH "/usr/local/MATLAB/R2014a/")
endif()
endif()
..........
FYI, to resolve the shared library issue at point 2.2.1 I needed to install the package matlab-support (in Ubuntu repositories)
I'm zipping a pre-built (no source/object files) binary application for distribution. The binary application requires a couple of libraries not included by default. The only way I seem to be able to get the application to start on the end-user is by including a run.sh that sets the library path to the current directory:
export LD_LIBRARY_PATH=./:$LD_LIBRARY_PATH
./MyApp.out
However, I'd really like to allow the user to just unzip the zip and doubleclick MyApp.out (without the shell script). Can I edit MyApp.out to search the current directory for the library? I've done something similar on OSX using install_name_tool, but that tool isn't available here.
You want to set the rpath. See this answer. So link using
gcc yourobjects*.o -L/some/lib/dir/ -lsome -Wl,-rpath,.
But you might want even to use -Wl,-rpath,$PWD or perhaps -Wl,-rpath,'$ORIGIN'. See this.
You could also (and this should work for a pre-built executable) configure your /etc/ld.so.conf by adding a line there with an absolute path (of the directory containing the lib), then running ldconfig -v ... See ldconfig(8)
I would suggest adding /usr/local/lib into /etc/ld.so.conf and making a symlink from /usr/local/lib/libfoo.so to e.g. $HOME/libfoo.so etc... (then run ldconfig ...). I don't think adding a user specific directory to /etc/ld.so.conf is reasonable ...
PS. What you really want is to package your application (e.g. as a *.deb package for Debian or Ubuntu, or an *.rpm for Fedora or Redhat). Package management systems handle dependencies!
I have been trying to do build and use OpenCV 2.3.0 on my Fedora15 Lovelock 64bit machine.
Background:
First, on my 64bit Fedora15, OpenCV2.2.0 seems to be in the locations namely
/usr/share/opencv
/usr/doc
/usr/lib64 &
/usr/bin
I do not find the include files though (in /usr/include). This means that the development package was n t installed. My package manager does not list the development packages when i try to Add/remove software.
I have a need to create applications, some of which just link to 2.2 and others which link to 2.3.O of the OpenCV library.So, I thought the best solution would be to have a separate location for 3rd party libraries that i use for my development . So I created a directory in /local named soft and created an OpenCV directory. A directory structure like this one.
/local/soft/
OpenCV/
OpenCV2.2.0/
source-files
build
OpenCV2.3.0/
source-files
build
installation
share/opencv
doc
include
lib
Now, i tried building OpenCV2.3.0 and i succeeded. I configure CMake to use CMAKE_INSTALL_PREFIX to the directory named "installation" (see above), instead of the default /usr/local/. Clean. huh?
I tried building and installing OpenCv 2.2.0 in the same way. Alas 2.2.0 complains something during the build. So i thought i ll link to the already existing version in the standard locations. BUT, when i try to install the dev packages for 2.2 using my package manager,the development files for x86_64 are not found :-) which means i dont have the headers to link to the libraries in the standard location.
I cant build my executable since linker ld would not find the OpenCV that i have installed in the non-standard location.(although i point it to the exact location using the -L and -l options with gcc in my Eclipse).
Question 1: Am i doing the right thing in maintaining installations in non-standard locations? Is /usr/ the standard location where the package manager will always do the installation?
Question2 : What is the right way of linking to these libraries installed in non-standard locations? Why would not ld recognize my .so files in the lib folder?
sudo g++ logpolar.cpp -o logpolar.o -I /local/soft/OpenCV/opencv2.3.1/installation/include/ -l/local/soft/OpenCV/opencv2.3.1/build/lib/libopencv_core.so
But ld canot find -l/local/soft/OpenCV/opencv2.3.1/build/lib/libopencv_core.so
I checked the lib folder and there sure is a beautiful symbolic link to libopencv_core.so.2.3
The standard approach is to use /usr/local directory structure that already has predefined paths like /usr/local/bin, /usr/local/sbin, /usr/local/include, /usr/local/lib.
You put your software here and everything will JustWork(TM). Every Linux distro (incl. Fedora) is set up so it will load programs (libraries, headers) from this libraries.
If you would use GNU toolchain (autoconf, automake => autotools) you would be fine. With CMake you probably need to setup paths for /usr/local/include and /usr/local/lib.
On the other hand this approach wont let you use multiple versions. You can only have one. The one in /usr/local overrides the system one (installed in /usr/bin) because these paths goes first.
You can keep your approach, it is nothing incorrect. We usually put such a software in the /opt folder, so you would go for /opt/opencv/X.Y where X.Y are the version numbers.
Q2: Read the gcc man page and search for the -L option. You need something like:
gcc ... -I/opt/opencv/2.0/include -lsystem_lib -L/opt/opencv/2.0/lib -lopencv ... ...
Do not forget to set LD_LIBRARY_PATH when running programs in multiple versions to properly load correct version:
LD_LIBRARY_PATH=/opt/opencv/2.0/lib /opt/opencv/2.0/bin/opencv