I'm working on a simple plugin architecture for my app. A plugin is a shared object with a known "get interface" function. I access the .so via dlopen/dlsym etc. Fairly standard.
Some of my plugins have extra .so files that need to be loaded too. These files can be shipped with a plugin fine, but I need the app to know where/how to load them.
So my question is: how to I add a particular plugin's directory to the list of places to look for libraries? I can't set LD_LIBRARY_PATH before starting the app, as the plugin will be loaded some time later. Alternatively, do I need each plugin to explicitly provide the list of dependencies in some kind of manifest file, so that I may load them with dlopen() first? Thanks.
Edit: to clarify, the plugins' directories are not known prior to starting the app, so I cannot set the LD_LIBRARY_PATH at that time.
According to this linux evaluates LD_LIBRARY_PATH before starting the app, so it is unaware of any changes you make at runtime.
Related
I know that normally linux search for dynamic libraries in specific folders in the system, like /lib, /usr/lib/, /usr/local/lib. And if doesn't find the shared library that is needed by the running program, then it will show an error.
My question is : Why doesn't linux implement the feature to look for libraries in the current folder. Lets say, after searching for the shared library inside all the folders mentioned above, it will search for the shared library inside the current folder before saying that the library wasn't found.
Isn't this feature simple to implement ?? or am I missing something ??
Adding . to LD_LIBRARY_PATH is generally not recommended as it introduces security risk and also makes program behavior less predictable for the end user. If you absolutely want to go down this path and want to avoid explicit setting of LD_LIBRARY_PATH, you can
link with -Wl,-rpath -Wl,'$ORIGIN'
not call your app directly but rather through a wrapper shell script which would set proper LD_LIBRARY_PATH and then run the app
The feature you are asking is not that hard to implement, but it would be considered a security risk. You can specify a shared library with the help of the LD_LIBRARY_PATH environment variable.
You can find more here:
For a website of mine I'm trying to make wkhtmltopdf(Link) work. The website is hosted on a shared hosting, which are a bit troublesome when using libraries not installed.
After a few tries with multiple version of the library(some where supposed to be static but I still got error about shared library not being found) I ended up contacting the provider, who told me that it would work if I have a static version of the library.
Problem is, my linux knowledge is very limited.
If I understand correctly, a static library would be a version of wkhtmltopdf, one single file, including all dependencies ?
As the official site mention are the followings : zlib, fontconfig, freetype, X11 libs (libX11, libXext, libXrender)
Second question is, could you point me to where I could find a step by step guide to build such library ? as my research are unsuccessful so far..
my linux knowledge is very limited
I will assume you are more or less familiar with windows dlls, which are similar to linux .sos (shared objects).
A shared object can be shared (hence the name) between different programs. In most cases, when the executable is loaded, the library is loaded in memory too. You can see such dependencies with ldd.
A static library (or statically linked library, or static executable, or whatever) is a library that is embedded in the executable at compile time. To statically link your library, you need to rebuild your executable, and link with a .a static library file, which is similar to .lib files on windows (with the visual studio compiler, at least, IIRC).
This can be troublesome and time consuming. That's why I advise you to take another route:
On windows, .dll files that share the same folder as the executable are given a higher preference than the one on the path (IIRC). On Linux (and generally UNIX), this is regarded as a security flaw, as someone could easily drop a rogue .so file and alter the program's behavior. You can however control this behavior with two environment variables: LD_LIBRARY_PATH and LD_PRELOAD. The second one is a bit more powerful, and is just some kind of "dll" injection. The first one, however, controls the path in which .so files will be searched.
So, I advise you to look for the required dependencies with ldd, and do it once again on your server if you can. Look for every missing .so file. You could do so by issuing the command ldd wkhtmltopdf | grep not found.
Once you have this list of missing libraries, bundle them together and send them on your server (be aware that they can have some dependencies too). You can probably find them on a local Linux installation of matching architecture, but I encourage you to try to match the distribution with the one of your provider.
Then, issue the wkhtmltopdf call after setting the LD_LIBRARY_PATH environment variable. You can do it like so:
LD_LIBRARY_PATH='/home/me/my_libs':$LD_LIBRARY_PATH /home/me/programs/wkhtmltopdf
Note that I append the old LD_LIBRARY_PATH variable at the end. It is rarely set out of the box, but at least you shouldn't have any problem if you do it this way.
To answer your comment: it is indeed a bit like modifying the PATH on windows (just to make this clear once again: on Linux, you have the same PATH environment variable, but it only works for executables' search path; so we're changing another LD_LIBRARY_PATH environment variable to specify the libraries search path).
Please note that in the above example, I didn't change it system-wide, but only for calling wkhtmltopdf. On windows, there are multiple ways to change the PATH environment variable. You can open the dedicated gui, which will change the path variable in the registry. But you can also override it locally in a command prompt or batch script. This is exactly what I did here.
Once LD_LIBRARY_PATH is exported, it will be used for every program you call, so it might be dangerous to set it system wide, if you have some incompatibilities. Moreover, whatever you try, you won't be able to set it system-wide if you don't have root access. So, you will at most affect only your programs.
As a final note, you might pull a lot of dependencies with this project, since it is Qt-based. If you want to rebuild it statically, you have to build Qt first with -static. Next time, you might be interested in some containerization technology, such as docker (or even appimages/flatpack/snap), which is designed to work around this kind of problems.
For further reading on dynamic link libraries on Linux, you might be interested in this resource or similar.
I am working on bindings for a cpp library.
To do this I wrote a capi / wrapper for the library and compiled that to a shared lib (.so file).
My question is, how do I then use and integrate this file into cargo without forcing the user to install it? Currently I build the cpp via a Makefile called from the build variable in Cargo.toml, but I am unsure what to do with the compiled lib.
For testing, I can either use rpath or LD_LIBRARY_PATH to point the executable to the right location, but this will not work when distributing a library.
How are people managing this?
First of all, determine whether you really need a shared library. It's not clear from your question, but if you compiled your own wrapper into a shared library, that's probably unnecessary - you can compile your code into a static library and link it directly into your executable.
Moreover, you can try to link that third-party library statically too. I don't think this should be hard. And yes, you need to use build command in the manifest to do all of this now.
However, if you still need to use a shared library and you don't want the end user to install it herself (which is strange, because that's the point of shared libraries), you have to distribute it manually. For example, you can write a makefile which assembles an archive which your users may extract and use. For your program to find the library correctly you will either have the user to install this archive into the system root directory (e.g. /usr on linux; then this shared library will be located automatically) or you will have to write small shell script wrapper around your executable which will locate the shared library and set appropriate LD_LIBRARY_PATH.
I'd go for the first path. Usually all major platforms provide means to create installation packages (deb/rpm/pkg.tar.xz/whatever on Linux, brew on Mac, windows installer on Windows, though on Windows you can just put your shared library in the same directory as the executable and it will work). You just have to create packages for the platform your users work on, so your program will be installed in correct directories and your shared library will be resolved automatically.
For the purposes of prototyping a new set of shared libraries in a development sandbox (to which I don't have root access), I'd like to know how to execute a binary while completely overriding the contents of /etc/ld.so.cache, so that none of the system libraries get loaded.
How can this be done? I have looked at mechanisms like setting the LD_LIBRARY_PATH environment variable or launching the program wrapped inside /lib/ld-linux.so, but these methods all seem to supplement the loading of libraries from /etc/ld.so.cache, but not override it completely.
Help?
Short of providing your own loader, there is no way to ignore the library path cache.
There doesn't seem to be a good way to do this directly on your host system, but you could use a chroot or a container (Docker/LXC/etc) to isolate your application and ignore the host's ld.so.cache simply by not mounting it into the chroot/container.
I am developing cross-platform Qt application.
It is freeware though not open-source. Therefore I want to distribute it as a compiled binary.
On windows there is no problem, I pack my compiled exe along with MinGW's and Qt's DLLs and everything goes great.
But on Linux there is a problem because the user may have shared libraries in his/her system very different from mine.
Qt deployment guide suggests two methods: static linking and using shared libraries.
The first produces huge executable and also require static versions of many libraries which Qt depends on, i.e. I'll have to rebuild all of them from scratches. The second method is based on reconfiguring dynamic linker right before the application startup and seems a bit tricky to me.
Can anyone share his/her experience in distributing Qt applications under Linux? What method should I use? What problems may I confront with? Are there any other methods to get this job done?
Shared libraries is the way to go, but you can avoid using LD_LIBRARY_PATH (which involves running the application using a launcher shell script, etc) building your binary with the -rpath compiler flag, pointing to there you store your libraries.
For example, I store my libraries either next to my binary or in a directory called "mylib" next to my binary. To use this on my QMake file, I add this line in the .pro file:
QMAKE_LFLAGS += -Wl,-rpath,\\$\$ORIGIN/lib/:\\$\$ORIGIN/../mylib/
And I can run my binaries with my local libraries overriding any system library, and with no need for a launcher script.
You can also distribute Qt shared libraries on Linux. Then, get your software to load those instead of the system default ones. Shared libraries can be over-ridden using the LD_LIBRARY_PATH environment variable. This is probably the simplest solution for you. You can always change this in a wrapper script for your executable.
Alternatively, just specify the minimum library version that your users need to have installed on the system.
When we distribute Qt apps on Linux (or really any apps that use shared libraries) we ship a directory tree which contains the actual executable and associated wrapper script at the top with sub-directories containing the shared libraries and any other necessary resources that you don't want to link in.
The advantage of doing this is that you can have the wrapper script setup everything you need for running the application without having to worry about having the user set environment variables, install to a specific location, etc. If done correctly, this also allows you to not have to worry about from where you are calling the application because it can always find the resources.
We actually take this tree structure even further by placing all the executable and shared libraries in platform/architecture sub-directories so that the wrapper script can determine the local architecture and call the appropriate executable for that platform and set the environment variables to find the appropriate shared libraries. We found this setup to be particularly helpful when distributing for multiple different linux versions that share a common file system.
All this being said, we do still prefer to build statically when possible, Qt apps are no exception. You can definitely build with Qt statically and you shouldn't have to go build a lot of additional dependencies as krbyrd noted in his response.
sybreon's answer is exactly what I have done. You can either always add your libraries to LD_LIBRARY_PATH or you can do something a bit more fancy:
Setup your shipped Qt libraries one per directory. Write a shell script, have it run ldd on the executable and grep for 'not found', for each of those libraries, add the appropriate directory to a list (let's call it $LDD). After you have them all, run the binary with LD_LIBRARY_PATH set to it's previous value plus $LDD.
Finally a comment about "I'll have to rebuild all of them from scratches". No, you won't have to. If you have the dev packages for those libraries, you should have .a files, you can statically link against these.
Not an answer as such (sybreon covered that), but please note that you are not allowed to distribute your binary if it is statically linked against Qt, unless you have bought a commercial license, otherwise your entire binary falls under the GPL (or you're in violation of Qt's license.)
If you have a commercial license, never mind.
If you don't have a commercial license, you have two options:
Link dynamically against Qt v4.5.0 or newer (the LGPL versions - you may not use the previous versions except in open source apps), or
Open your source code.
The probably easiest way to create a Qt application package on Linux is probably linuxdeployqt. It collects all required files and lets you build an AppImage which runs on most Linux distributions.
Make sure you build the application on the oldest still-supported Ubuntu LTS release so your AppImage can be listed on AppImageHub.
You can look into QtCreator folder and use it as an example. It has qt.conf and qtcreator.sh files in QtCreator/bin.
lib/qtcreator is the folder with all needed Qt *.so libraries. Relative path is set inside qtcreator.sh, which should be renamed to you-app-name.sh
imports,plugins,qml are inside bin directory. Path to them is set in qt.conf file. This is needed for QML applications deployment.
This article has information on the topic. I will try it myself:
http://labs.trolltech.com/blogs/2009/06/02/deploying-a-browser-on-gnulinux/
In a few words:
Configure Qt with -platform linux-lsb-g++
Linking should be done
with –lsb-use-default-linker
Package everything and deploy (will
need a few tweaks here but I haven't yet tried it sorry)