How to build libcurl-gnutls.so from source code - linux

Where can I find the source code and instructions for building libcurl-gnutls.so?
I'm working on a project that needs libcurl-gnutls.so. I am required to build it from source code - I am not allowed to simply install it with "apt-get install libcurl". Unfortunately, my google-fu is failing me and I can't find a source code repository or instructions to build libcurl-gnutls.so anywhere.
Here's what I have found:
Linux-from-scratch has well-documented instructions for building libcurl.so, here: https://www.linuxfromscratch.org/blfs/view/svn/basicnet/curl.html. That lets me build libcurl.so with gnutls, but not libcurl-gnutls.so.
The curl website (curl.se), has detailed instructions on its various options here: https://curl.se/docs/install.html. Those show me how to build libcurl with gnutls, but the end product is still libcurl.so, not libcurl-gnutls.so.
When I run ldd -r on my project, it identifies the functions it needs (curl_easy_init, curl_easy_setopt, curl_easy_perform, and curl_easy_cleanup). I can find those symbols in both libcurl.so and a pre-built libcurl-gnutls.so. This leads me to suspect that libcurl-gnutls.so is libcurl.so, published under a different name. However, renaming libcurl.so to libcurl-gnutls.so is not sufficient to meet the dependency requirements. I could try altering the libcurl project to set its name and version to libcurl-gnutls (not that I know how to do it - I would poke around until I figure it out), but I don't know how appropriate that would be.
I found one other question on Stack Overflow about libcurl-gnutls (How to create lib curl-gnutls.so.4), but the answers to that are to install a pre-built version via apt-get install, which I am not allowed to do.

libcurl-gnutls.so actually just comes from a cURL built with gnutls support. You can find the repository here: https://github.com/curl/curl
Check out the docs/INSTALL.md. It has all the information you need to build cURL, specifically the part about Building from git.
Here is a snippet you might need:
./configure --with-openssl [--with-gnutls --with-wolfssl]
make
make test (optional)
make install

Here's a complete answer cobbled together from everyone's input (Thanks especially to Knud Larsen and Wassim Dhif):
libcurl-gnutls.so is just libcurl.so built with gnutls support. Archives for the project are here: https://curl.se/download
Change the SONAME that libcurl.so is built with by editing ltmain.sh to change:
if test -n "$soname_spec";
then eval soname=\"$soname_spec\"
To:
if test -n "$soname_spec"; then
soname=libcurl-gnutls.so.4
I'm sure there's a more elegant way to do it, but this works and I need to move on.
Alternatively, you can modify the the SONAME after libcurl.so is built with:
patchelf –set-soname libcurl-gnutls.so.4 libcurl.so
Check your client (the program or shared library that requires libcurl-gnutls.so as a dependency), to see if it requires version information.
For instance, when I run objdump -p myprogram, I get this:
Version References:
required from libcurl-gnutls.so.4:
0x0b103d23 0x00 14 CURL_GNUTLS_3
To build libcurl-gnutls.so with this version information:
2a. Set the version information to version 3:
In lib/libcurl.vers.in change:
CURL_#CURL_LT_SHLIB_VERSIONED_FLAVOUR#4
To:
CURL_#CURL_LT_SHLIB_VERSIONED_FLAVOUR#3
2b. Use --enable-versioned-symbols when configuring the libcurl project. This adds the required version information.
./configure --with-gnutls --enable-versioned-symbols [other arguments as needed]
The final product may be named libcurl.so, but can be renamed. It will have its SONAME set to libcurl-gnutls.so.4 and will have the required version information.

Related

How to build a .so for export to another machine? [duplicate]

I'm very new to Yesod and I'm having trouble building Yesod statically
so I can deploy to Heroku.
I have changed the default .cabal file to reflect static compilation
if flag(production)
cpp-options: -DPRODUCTION
ghc-options: -Wall -threaded -O2 -static -optl-static
else
ghc-options: -Wall -threaded -O0
And it no longer builds. I get a whole bunch of warnings and then a
slew of undefined references like this:
Linking dist/build/personal-website/personal-website ...
/usr/lib/ghc-7.0.3/libHSrts_thr.a(Linker.thr_o): In function
`internal_dlopen':
Linker.c:(.text+0x407): warning: Using 'dlopen' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/unix-2.4.2.0/libHSunix-2.4.2.0.a(HsUnix.o): In
function `__hsunix_getpwent':
HsUnix.c:(.text+0xa1): warning: Using 'getpwent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/unix-2.4.2.0/libHSunix-2.4.2.0.a(HsUnix.o): In
function `__hsunix_getpwnam_r':
HsUnix.c:(.text+0xb1): warning: Using 'getpwnam_r' in statically
linked applications requires at runtime the shared libraries from the
glibc version used for linking
/usr/lib/libpq.a(thread.o): In function `pqGetpwuid':
(.text+0x15): warning: Using 'getpwuid_r' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/libpq.a(ip.o): In function `pg_getaddrinfo_all':
(.text+0x31): warning: Using 'getaddrinfo' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__63.o): In function `sD3z_info':
(.text+0xe4): warning: Using 'gethostbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__164.o): In function `sFKc_info':
(.text+0x12d): warning: Using 'getprotobyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__155.o): In function `sFDs_info':
(.text+0x4c): warning: Using 'getservbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/libpq.a(fe-misc.o): In function `pqSocketCheck':
(.text+0xa2d): undefined reference to `SSL_pending'
/usr/lib/libpq.a(fe-secure.o): In function `SSLerrmessage':
(.text+0x31): undefined reference to `ERR_get_error'
/usr/lib/libpq.a(fe-secure.o): In function `SSLerrmessage':
(.text+0x41): undefined reference to `ERR_reason_error_string'
/usr/lib/libpq.a(fe-secure.o): In function `initialize_SSL':
(.text+0x2f8): undefined reference to `SSL_check_private_key'
/usr/lib/libpq.a(fe-secure.o): In function `initialize_SSL':
(.text+0x3c0): undefined reference to `SSL_CTX_load_verify_locations'
(... snip ...)
If I just compile with just -static and without -optl-static
everything builds fine but the application crashes when it tries to
start on Heroku.
2011-12-28T01:20:51+00:00 heroku[web.1]: Starting process with command
`./dist/build/personal-website/personal-website -p 41083`
2011-12-28T01:20:51+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: error while loading shared libraries: libgmp.so.10:
cannot open shared object file: No such file or directory
2011-12-28T01:20:52+00:00 heroku[web.1]: State changed from starting
to crashed
I tried adding libgmp.so.10 to the LD_LIBRARY_PATH as suggested in here
and then got the following error:
2011-12-28T01:31:23+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: /lib/libc.so.6: version `GLIBC_2.14' not found
(required by ./dist/build/personal-website/personal-website)
2011-12-28T01:31:23+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: /lib/libc.so.6: version `GLIBC_2.14' not found
(required by /app/dist/build/personal-website/libgmp.so.10)
2011-12-28T01:31:25+00:00 heroku[web.1]: State changed from starting
to crashed
2011-12-28T01:31:25+00:00 heroku[web.1]: Process exited
It seems that the version of libc that I'm compiling against is
different. I tried also adding libc to the batch of libraries the
same way I did for libgmp but this results in a segmentation fault
when the application starts on the Heroku side.
Everything works fine on my PC. I'm running 64bit archlinux with ghc
7.0.3. The blog post on the official Yesod blog looked pretty easy
but I'm stumped at this point. Anyone have any ideas? If there's a way to get this thing working without building statically I'm open to that too.
EDIT
Per Employed Russians answer I did the following to fix this.
First created a new directory lib under the project directory and copied the missing shared libraries into it. You can get this information by running ldd path/to/executable and heroku run ldd path/to/executable and comparing the output.
I then did heroku config:add LD_LIBRARY_PATH=./lib so when the application is started the dynamic linker will look for libraries in the new lib directory.
Finally I created an ubuntu 11.10 virtual machine and built and deployed to Heroku from there, this has an old enough glibc that it works on the Heroku host.
Edit:
I've since written a tutorial on the Yesod wiki
I have no idea what Yesod is, but I know exactly what each of your other errors means.
First, you should not try to link statically. The warning you get is exactly right: if you link statically, and use one of the routines for which you are getting the warning, then you must arrange to run on a system with exactly the same version of libc.so.6 as the one you used at build time.
Contrary to popular belief, static linking produces less, not more, portable executables on Linux.
Your other (static) link errors are caused by missing libopenssl.a at link time.
But let's assume that you are going to go the "sane" route, and use dynamic linking.
For dynamic linking, Linux (and most other UNIXes) support backward compatibility: an old binary continues to work on newer systems. But they don't support forward compatibility (a binary built on a newer system will generally not run on an older one).
But that's what you are trying to do: you built on a system with glibc-2.14 (or newer), and you are running on a system with glibc-2.13 (or older).
The other thing you need to know is that glibc is composed of some 200+ binaries that must all match exactly. Two key binaries are /lib/ld-linux.so and /lib/libc.so.6 (but there are many more: libpthread.so.0, libnsl.so.1, etc. etc). If some of these binaries came from different versions of glibc, you usually get a crash. And that is exactly what you got, when you tried to place your glibc-2.14 libc.so.6 on the LD_LIBRARY_PATH -- it no longer matches the system /lib/ld-linux.
So what are the solutions? There are several possibilities (in increasing difficulty):
You could copy ld-2.14.so (the target of /lib/ld-linux symlink) to the target system, and invoke it explicitly:
/path/to/ld-2.14.so --library-path <whatever> /path/to/your/executable
This generally works, but can confuse an application that looks at argv[0], and breaks for applications that re-exec themselves.
You could build on an older system.
You could use appgcc (this option has disappeared, see this for description of what it used to be).
You could set up a chroot environment matching the target system, and build inside that chroot.
You could build yourself a Linux-to-olderLinux crosscompiler
You have several issues.
You should not build production binaries on bleeding edge distributions. The libraries on the production system will not be forward compatible.
You should not link glibc statically - it will always at runtime try to load additional libraries. For example cpu-based assembly. That is what your first warnings are about.
The last linker errors look like they are related to a missing openssl library on the command line.
But all in all - downgrade your distribution.
I had similar problems launching to Heroku (which uses glibc-2.11) where I had an application that required glibc-2.14, but I did not have access to the source and could not re-build it. I tried many things and nothing worked.
My workaround was to launch the service on Amazon Elastic Beanstalk and just provide an API interface.
I found the information provided useful as well, I think the various descriptions miss a critical issue I also ran into while forcing an updated version of Vagrant to start working again.
It's the dependency references internal to something like complicated installs, like Yesod to Heroku. Those interanl refences need to be preserved.
This is the script I wrote to make problems go away (at least, hopefully, for a little while):
#!/bin/bash
cd $HOME/
GLIBC_VERSION="2.17"
GLIBC_PREFIX="/usr/glibc/"
VAGRANT_VERSION="2.2.19"
# Install the basic build system utilities.
yum groupinstall -y "Development tools"
yum install -y curl patchelf
# Grab the tarball with the GNU libc source code.
curl -Lfo glibc-${GLIBC_VERSION}.tar.gz "https://ftp.gnu.org/gnu/glibc/glibc-${GLIBC_VERSION}.tar.gz"
echo "a3b2086d5414e602b4b3d5a8792213feb3be664ffc1efe783a829818d3fca37a glibc-${GLIBC_VERSION}.tar.gz" | sha256sum -c || exit 1
# Extract the secrets and get ready to rumble.
tar xzvf glibc-${GLIBC_VERSION}.tar.gz
# The configure script requrires an independent build directory.
mkdir -p glibc-build && cd glibc-build
# Configure glibc with a GLIBC_PREFIX so it doesn't conflict with distro libc files..
../glibc-${GLIBC_VERSION}/configure --prefix="${GLIBC_PREFIX}" --libdir="${GLIBC_PREFIX}/lib" \
--libexecdir="${GLIBC_PREFIX}/lib" --enable-multi-arch
# Compile and then install GNU libc.
make -j8 && make install
# Download and install Vagrant.
curl -Lfo vagrant_${VAGRANT_VERSION}_x86_64.rpm "https://releases.hashicorp.com/vagrant/${VAGRANT_VERSION}/vagrant_${VAGRANT_VERSION}_x86_64.rpm"
echo "990e8d2159032915f21c0f1ccdcbca1a394f7937e06e43dc1dabe605d208dc20 vagrant_${VAGRANT_VERSION}_x86_64.rpm" | sha256sum -c || exit 1
yum install -y vagrant_${VAGRANT_VERSION}_x86_64.rpm
# Patch the binaries and shared libraries inside the Vagrant directory, so they use the new version of GNU libc.
(find /opt/vagrant/ -type f -exec file {} \; )| grep "dynamically linked" | awk -F':' '{print $1}' | while read FILE ; do
patchelf --set-rpath /opt/vagrant/embedded/lib:/opt/vagrant/embedded/lib64:/usr/glibc/lib:/usr/lib64:/lib64:/lib --set-interpreter /usr/glibc/lib/ld-linux-x86-64.so.2 "${FILE}"
done
The script should be pretty easy to understand, and adapt easily to whatever MacGuffin you want to make work, provied you understand it.
The only tricky part is the rpath you pass to patchelf. Upi need to make sure you preserve the search paths, and precedence your software requires. Or you end up fixing one problem only to create another equally frustrating roadblock.
P.S. Don't forget the update the hashes for any file you down. In particular, you need to compile/install a different version of GNU libc, you will need to update that hash to match the version you want to use.

How to find out path of Boost library for manual package build and installation?

I am using Windows subsystem for linux for Ubuntu and I am very new to this. I am trying to build the program (AutoDock Vina) from its source code provided on GitHub - GitHub Link.
In its installation procedure - provided here, it's mentioned that - To compile the binary (you might need to customize the Makefile by setting the paths to the Boost library)
Makefile has the following code written in it -
BASE=/usr/local
BOOST_VERSION=
BOOST_INCLUDE = $(BASE)/include
C_PLATFORM=-static -pthread
GPP=/usr/bin/g++
C_OPTIONS= -O3 -DNDEBUG -std=c++11
BOOST_LIB_VERSION=
include ../../makefile_common
I had previously installed Boost and SWIG using apt-get command as mentioned in the manual.
sudo apt-get install libboost-all-dev swig
When I tried to find out location of Boost library, I found that at following two places boost related files are there -
/usr/lib/x86_64-linux-gnu/ and
/lib/x86_64-linux-gnu/ (these two folders have many files having names starting with libboost)
/usr/include/boost/ (this appears to be main folder for boost)
But, I tried many combinations to the BASE value in Makefile, but in all cases, after building the files, I see the executable named Vina (as expected) is built in the folder but when I try to run the same on terminal, I get command not found error.
Please help me with this - how to find out Boost related values for Makefiles?
Edit 1 - After reading the comment from #MadScientist, I have realized it's not a problem of Boost library but the issue I am facing is because of PATH not set. So I will follow his comment and proceed.

arm-none-eabi-objdump: error while loading shared libraries: libdebuginfod.so.1: cannot open shared object file

If you have an answer for this, or further information, I'd welcome it. I'm following advice from here, to offer some unsolicited help by posting this question then an answer I've already found for it.
I have a bare-metal ARM board for which I'm building a cross-toolchain, from sources for GNU binutils, gcc and gdb, and for SourceWare's Newlib. I got those four working and cross-built a DoNothing.c into an ELF file - but I couldn't disassemble it with this:
$ arm-none-eabi-objdump -S DoNothing.elf
The error was:
$ arm-none-eabi-objdump: error while loading shared libraries: libdebuginfod.so.1: cannot open shared object file: No such file or directory
I'll follow up with a solution.
The error was correct - my system didn't have libdebuginfod.so.1 installed - but I have another cross-binutils, installed from binary for a different target, and its objdump -S works fine on the same host. Why would one build of objdump complain about missing that shared library, when clearly not all builds of objdump need it?
First I tried rebuilding cross binutils, specifying --without-debuginfod as a configure option. No change, which seems odd: surely that should build tools that not only don't use debuginfod but which don't depend on it in any way. (If someone can answer that, or point out what I've misunderstood, it may help people.)
Next I figured debuginfod was inescapable (for my cross-tools built from source at least), so I'd install it to get rid of the error. It's a component of the elfutils package, but installing the latest elfutils available for my Ubuntu 20.04 system didn't bring libdebuginfod.so.1 with it.
I found a later one, for Arch Linux, whose package contents suggested it would - but its package format doesn't match Ubuntu's and installing it was going to involve a lot of work. Instead I opted to build it from the Arch Linux source package. However, running ./configure on that gave a couple of infuriatingly similar errors:
configure: checking libdebuginfod dependencies, --disable-libdebuginfod or --enable-libdebuginfo=dummy to skip
...
configure: error: dependencies not found, use --disable-libdebuginfod to disable or --enable-libdebuginfod=dummy to build a (bootstrap) dummy library.
No combination of those suggestions would allow configure for elfutils-0.182 to run to completion.
The problem of course was my own lack of understanding. The solution came from the Linux From Scratch project: what worked was to issue configure with both of the suggested options, like this:
$ ./configure --prefix=/usr \
--disable-debuginfod \
--enable-libdebuginfod=dummy \
--libdir=/lib
That gave a clean configure; make worked first time, as did make check and then sudo make install which of course installed libdebuginfod.so.1 as required. I then had an arm-none-eabi-objdump which disassembles cross-compiled ELF files without complaining.

What's the difference between the various kernel-debug*.rpm files created after rpmbuild?

I build Fedora Linux kernel (rpmbuild -bb --target=`uname -m` kernel.spec) according to this guide, and now I have among others:
kernel-debug-debuginfo-3.18.6-100.myself.fc20.x86_64.rpm (~360MB)
kernel-debuginfo-3.18.6-100.myself.fc20.x86_64.rpm (~360MB)
kernel-debuginfo-common-x86_64-3.18.6-100.myself.fc20.x86_64.rpm (~51MB)
kernel-debug-3.18.6-100.myself.fc20.x86_64.rpm (~67MB)
kernel-debug-devel-3.18.6-100.pae_sw_tal.fc20.x86_64.rpm (~10MB)
some others...
What's the difference between them and which one should I install if I want a development (debug-able build) version?
Basically you can find following information on rpmfind.net
just put kernel-debug-debuginfo in the search and it will tell you what it is.
Similarly for other packages.
for e.g.
kernel-debug-debuginfo-3.18.6-100.myself.fc20.x86_64.rpm
This package provides debug information for package kernel-debug. This is required to use SystemTap with kernel-debug-3.18.6-100.myself.fc20.x86_64.rpm
And I think you need kernel-debug-3.18.6-100.myself.fc20.x86_64.rpm (~67MB)
look at the options of rpmbuild for creating less files. Options like --without devel --without debuginfo

linking libraries under Linux

I experienced a (for me) strange behaviour today: Using QMake with the PkgConfig-options etc. I was able to link the opencv libraries, but then I switched to CMake using PkgConfig. Once I tried to build my software, the linker complained that it was not able to find the library libcvaux, which pkg-config returns asked to deliver the libraries for opencv (pkg-config --libs opencv).
In /usr/lib I found a libcvaux.so.{version}, but no "plain" entry libcvaux.so. So what I did was to create a symlink, and now it works.
Now I wonder why it worked before. Is there something to pass ld an option saying "use the newest version, and you get the version by looking at the numbers behind the so suffix"? Or is it more some kind of bug that the maintainers of the opencv package forgot to add this symlink? Because e.g. libcv or libhighgui have such symbolic links.
Thank you!
From the ldconfig manpage:
ldconfig checks the header and file
names of the libraries it encounters
when determining which versions should
have their links updated.
Maybe an earlier ldconfig run deleted the link.

Resources