I'm trying to log the calls made by an app prior to a crash, including libc calls. I've used the -finstrument-functions support in gcc with my own libs but I can't build glibc with this instrumentation.
I added -finstrument-functions to libc_extra_cflags in libc/configure but the build fails with "undefined reference to __libc_multiple_libcs" when linking ld.so.
Just running CFLAGS=-finstrument-functions ./configure doesn't work because the configure tests fail since they don't define __cyg_profile_func_enter/_exit.
I'm currently trying to figure out how to add instrumentation per module (stdlib, io, string, etc) and looking through libc/foo/Makefile's it should be possible using e.g. CFLAGS_stdlib = -finstrument-functions but the flag doesn't show up in gcc commands.
Is there a way to add per-module flags to the glibc build, or is glibc known not to work with -finstrument-functions ?
I'm trying to log the calls made by an app prior to a crash, including libc calls.
You can use ltrace to trace calls made by the application to any shared library, including GLIBC.
is glibc known not to work with -finstrument-functions
Pretty much.
If you think about it, what is your __cyg_profile_func_enter going to do? It can't call into libc, or you'll end up with infinite recursion. It's possible to use direct system calls, but it's far from trivial.
Related
I have a libsomething.a file which is a static library with all dependencies included.
I need to be able to import this in Python as it is a Python C library. According to this, it is not possible to use a static library as a CPython library.
How can I take my .a file and make it a .so, keeping all static dependencies baked in?
Background: I am using Crowbar to build a CPython shared library which can be called from Python in AWS Lambda. Until now, it has worked flawlessly, but as soon as I added in dependencies which require OpenSSL, I get linker problems when running the code in Lambda.
The issue here is that the Amazon Linux image that is used to execute code has an ancient OpenSSL version. I have recreated the runtime environment, but the issue is that the old version of OpenSSL no longer exists in Amazon's yum repository. This means that installing openssl-devel pulls down OpenSSL 1.0.2k, where in the runtime the version of OpenSSL provided is 1.0.1.
This results in linking failing at runtime in Lambda. Therefore, I need a way to build a (mostly) statically linked shared library. The only shared libraries I want my SO to link from are libc and the kernel, with everything else statically compiled in.
In the Lambda execution environment, LD_LIBRARY_PATH is set to /usr/lib64:/lib64:./lib, so anything in the lib folder will be loaded, but only as a last result, and if I link against OpenSSL, I get the wrong version every time.
In Rust, I have the option of producing liblambda.a or liblambda.so, a static or a shared library. I'm assuming that producing a *.a and then converting into a shared library only linking to glibc and kernel dependencies.
No, you cannot do that conversion from static library to shared one (at least not in practice). Read How To Write Shared Libraries by Drepper.
One of the main reason is that shared libraries want (that it nearly need) to have position independent code (which static libraries usually don't have).
However, on Linux most libraries are free software. So why don't you recompile your library from source code into a shared library?
(you might perhaps recompile that specific version of OpenSSL from source)
Usually one rpm depends on many other packages or libs. This is not easy for massive deployment without internet access.
Since yum can automatically resolve dependencies. Is it possible to build a portable executable? So that we can copy it to other machines with the same OS.
If you want a known collection of RPMs to install, yum offers a downloadonly plugin. With that, you should be able to collect all the associated RPMs in one shot to install what you wanted on a disconnected machine.
The general way to build a binary without runtime library dependencies is to build it to be static, ie. using the -static argument to gcc, which links in static versions of the libraries required such that they're included in the resulting executable. This doesn't bundle in any data file dependencies or external executables (ie. libexec-style helpers), but simpler applications often don't need them.
For more complex needs (where data files are involved, or elements of the dependency chain can't be linked in for one reason or another), consider using AppImageKit -- which bundles an application and its dependency chain into a runnable ISO. See docs/links at PortableLinuxApps.org.
In neither of these cases does rpm or yum have anything to do with it. It's certainly possible to build an RPM that packages static executables, but that's a matter of changing the %build section of the spec file such that it passes -static to gcc, not of doing anything RPM-specific.
To be clear, by the way -- there are compelling reasons why we don't use static libraries all the time!
Using shared libraries means that applying a security update to a library only means replacing the library itself, not recompiling all applications using it.
Using shared libraries is more memory-efficient, since the single shared copy of the library in memory can be used by multiple applications.
Using shared libraries means your executables don't need to include full copies of all the libraries they use, making them much smaller.
I am working on bindings for a cpp library.
To do this I wrote a capi / wrapper for the library and compiled that to a shared lib (.so file).
My question is, how do I then use and integrate this file into cargo without forcing the user to install it? Currently I build the cpp via a Makefile called from the build variable in Cargo.toml, but I am unsure what to do with the compiled lib.
For testing, I can either use rpath or LD_LIBRARY_PATH to point the executable to the right location, but this will not work when distributing a library.
How are people managing this?
First of all, determine whether you really need a shared library. It's not clear from your question, but if you compiled your own wrapper into a shared library, that's probably unnecessary - you can compile your code into a static library and link it directly into your executable.
Moreover, you can try to link that third-party library statically too. I don't think this should be hard. And yes, you need to use build command in the manifest to do all of this now.
However, if you still need to use a shared library and you don't want the end user to install it herself (which is strange, because that's the point of shared libraries), you have to distribute it manually. For example, you can write a makefile which assembles an archive which your users may extract and use. For your program to find the library correctly you will either have the user to install this archive into the system root directory (e.g. /usr on linux; then this shared library will be located automatically) or you will have to write small shell script wrapper around your executable which will locate the shared library and set appropriate LD_LIBRARY_PATH.
I'd go for the first path. Usually all major platforms provide means to create installation packages (deb/rpm/pkg.tar.xz/whatever on Linux, brew on Mac, windows installer on Windows, though on Windows you can just put your shared library in the same directory as the executable and it will work). You just have to create packages for the platform your users work on, so your program will be installed in correct directories and your shared library will be resolved automatically.
I'm trying to compile a statically linked binary with GCC and I'm getting warning messages like:
warning: Using 'getpwnam_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
I don't even know what getwnam_r does, but I assume it's getting called from inside some higher level API. I receive a similar message for gethostbyname.
Why would it not be possible to just statically link these functions in like every other function?
Function calls that need access to NSS or iconv need access will open other libs dynamically, since NSS needs plugins to work (the helper modules like pam_unix.so). When the NSS system dlopens these modules, there will be two conflicting versions of glibc - the one your program brought with it (statically compiled in), and the one dlopen()ed by NSS dependencies. Shit will happen.
This is why you can't build static programs using getpwnam_r and a few other functions.
AFAIK, it's not impossible to fully statically link an application.
The problem would be incompatibility with newer library versions which might be completely different. Say for example printf(). You can statically link it, but what if in a future that printf() implementation changes radically and this new implementation is not backward-compatible? Your appliction would be broken.
Please someone correct me if I'm wrong here.
I am developing cross-platform Qt application.
It is freeware though not open-source. Therefore I want to distribute it as a compiled binary.
On windows there is no problem, I pack my compiled exe along with MinGW's and Qt's DLLs and everything goes great.
But on Linux there is a problem because the user may have shared libraries in his/her system very different from mine.
Qt deployment guide suggests two methods: static linking and using shared libraries.
The first produces huge executable and also require static versions of many libraries which Qt depends on, i.e. I'll have to rebuild all of them from scratches. The second method is based on reconfiguring dynamic linker right before the application startup and seems a bit tricky to me.
Can anyone share his/her experience in distributing Qt applications under Linux? What method should I use? What problems may I confront with? Are there any other methods to get this job done?
Shared libraries is the way to go, but you can avoid using LD_LIBRARY_PATH (which involves running the application using a launcher shell script, etc) building your binary with the -rpath compiler flag, pointing to there you store your libraries.
For example, I store my libraries either next to my binary or in a directory called "mylib" next to my binary. To use this on my QMake file, I add this line in the .pro file:
QMAKE_LFLAGS += -Wl,-rpath,\\$\$ORIGIN/lib/:\\$\$ORIGIN/../mylib/
And I can run my binaries with my local libraries overriding any system library, and with no need for a launcher script.
You can also distribute Qt shared libraries on Linux. Then, get your software to load those instead of the system default ones. Shared libraries can be over-ridden using the LD_LIBRARY_PATH environment variable. This is probably the simplest solution for you. You can always change this in a wrapper script for your executable.
Alternatively, just specify the minimum library version that your users need to have installed on the system.
When we distribute Qt apps on Linux (or really any apps that use shared libraries) we ship a directory tree which contains the actual executable and associated wrapper script at the top with sub-directories containing the shared libraries and any other necessary resources that you don't want to link in.
The advantage of doing this is that you can have the wrapper script setup everything you need for running the application without having to worry about having the user set environment variables, install to a specific location, etc. If done correctly, this also allows you to not have to worry about from where you are calling the application because it can always find the resources.
We actually take this tree structure even further by placing all the executable and shared libraries in platform/architecture sub-directories so that the wrapper script can determine the local architecture and call the appropriate executable for that platform and set the environment variables to find the appropriate shared libraries. We found this setup to be particularly helpful when distributing for multiple different linux versions that share a common file system.
All this being said, we do still prefer to build statically when possible, Qt apps are no exception. You can definitely build with Qt statically and you shouldn't have to go build a lot of additional dependencies as krbyrd noted in his response.
sybreon's answer is exactly what I have done. You can either always add your libraries to LD_LIBRARY_PATH or you can do something a bit more fancy:
Setup your shipped Qt libraries one per directory. Write a shell script, have it run ldd on the executable and grep for 'not found', for each of those libraries, add the appropriate directory to a list (let's call it $LDD). After you have them all, run the binary with LD_LIBRARY_PATH set to it's previous value plus $LDD.
Finally a comment about "I'll have to rebuild all of them from scratches". No, you won't have to. If you have the dev packages for those libraries, you should have .a files, you can statically link against these.
Not an answer as such (sybreon covered that), but please note that you are not allowed to distribute your binary if it is statically linked against Qt, unless you have bought a commercial license, otherwise your entire binary falls under the GPL (or you're in violation of Qt's license.)
If you have a commercial license, never mind.
If you don't have a commercial license, you have two options:
Link dynamically against Qt v4.5.0 or newer (the LGPL versions - you may not use the previous versions except in open source apps), or
Open your source code.
The probably easiest way to create a Qt application package on Linux is probably linuxdeployqt. It collects all required files and lets you build an AppImage which runs on most Linux distributions.
Make sure you build the application on the oldest still-supported Ubuntu LTS release so your AppImage can be listed on AppImageHub.
You can look into QtCreator folder and use it as an example. It has qt.conf and qtcreator.sh files in QtCreator/bin.
lib/qtcreator is the folder with all needed Qt *.so libraries. Relative path is set inside qtcreator.sh, which should be renamed to you-app-name.sh
imports,plugins,qml are inside bin directory. Path to them is set in qt.conf file. This is needed for QML applications deployment.
This article has information on the topic. I will try it myself:
http://labs.trolltech.com/blogs/2009/06/02/deploying-a-browser-on-gnulinux/
In a few words:
Configure Qt with -platform linux-lsb-g++
Linking should be done
with –lsb-use-default-linker
Package everything and deploy (will
need a few tweaks here but I haven't yet tried it sorry)