CMake: How to use an imported .so.1.2.3 in transitive dependencies - linux

I have to import a libft4222.so.1.2.3 into a CMake project on Linux. There are no symlinks without version information like a plain libmyimportedlibrary.so. I am writing a FindLibFT4222.cmake to make the library usable in my project.
In my project, a shared library uses libft4222 like this:
if (LIBFT4222_FOUND)
if (NOT TARGET LibFT4222::LibFT4222)
add_library(LibFT4222::LibFT4222 UNKNOWN IMPORTED)
set_target_properties(LibFT4222::LibFT4222
PROPERTIES IMPORTED_LOCATION ${LibFT4222_LIBS})
target_include_directories(LibFT4222::LibFT4222
INTERFACE
${LibFT4222_INCLUDE_DIR})
endif()
endif()
LibFT4222_LIBS points to the imported location of the libft4222.so.1.2.3:
/path/to/libft4222.so.1.2.3
I am adding this library as a dependency to a shared library A in my project.
Finally, I have an executable depending on A.
When I link the executable, the linker reports an error that it cannot find libft4222.so
When I add libft4222 directly to the executable, it works.
The error message I get:
/usr/lib/gcc-cross/arm-linux-gnueabihf/9/../../../../arm-linux-gnueabihf/bin/ld: warning: libft4222.so, needed by libSomeOtherLib.so, not found (try using -rpath or -rpath-link)
How do I tell CMake to correctly handle libft4222.so.1.2.3 and not to omit .1.2.3?
Update 1: I looked at the SONAME property of libft4222.so.1.2.3:
$ arm-linux-gnueabihf-objdump -p libft4222.so.1.4.4.44 | grep SONAME
SONAME libft4222.so
So I guess creating the symlinks is unavoidable.

Related

CMake Ubuntu set soname for shared object

How to set the SONAME for a shared library with CMake in Ubuntu?
With help of How to add linker flag for libraries with CMake? I have created a CMakeLists.txt:
project(mylib VERSION 1.2.3)
set(src_files_mylib
Client.cpp
Server.cpp
)
set(hdr_files_mylib
Client.h
Server.h
)
add_library(mylib SHARED ${src_files_mylib} ${hdr_files_mylib})
set_target_properties(mylib PROPERTIES PREFIX "")
set_target_properties(mylib PROPERTIES SUFFIX "")
set_target_properties(mylib PROPERTIES OUTPUT_NAME "mylib.so.${PROJECT_VERSION}")
add_link_options("LINKER: -l,soname,mylib.so.${PROJECT_VERSION_MAJOR}")
Using CMake 3.16 and out of source build. It produces the library named mylib.so.1.2.3 but it does not seem to have an SONAME in it. When executing ldconfig -n . in the library dir, it does not produce a link.
set(CMAKE_SHARED_LINKER_FLAGS "-Wl,soname,mylib.so.${PROJECT_VERSION_MAJOR}")
instead of
add_link_options("LINKER: -l,soname,mylib.so.${PROJECT_VERSION_MAJOR}")
behaves the same to me.
When writing the Makefile manually, the target is built correctly with this (going ldconfig -n . like above produces a link):
(...)
MYLIB_SO=mylib.so.$(VERSION_MAJOR).$(VERSION_MINOR).$(VERSION_PATCH)
MYLIB_SONAME=mylib.so.$(VERSION_MAJOR)
(...)
$(MYLIB_SO): $(OBJ_MYLIB)
$(CXX) $(SO_FLAGS) -Wl,-soname,$(MYLIB_SONAME) -o $(MYLIB_SO) $(DEBUGFLAGS) $(OBJ_MYLIB)
if libraries is called SoundTouch
project(SoundTouch VERSION 2.3.0 LANGUAGES CXX)
add_library(SoundTouch [SHARED]
sources/animation.cpp
sources/buffers.cpp
[...]
)
set_target_properties(SoundTouch PROPERTIES VERSION ${CMAKE_PROJECT_VERSION}
SOVERSION ${PROJECT_VERSION_MAJOR} )

"undefined reference" error from static lib of external parquet project

I want to link expernal 'parquet' project ( https://github.com/apache/arrow/tree/master/cpp ) as part of my current project on Linux.
For this purposes I ran cmake of parquet with such parameters
cd build_parquet
cmake -DCMAKE_BUILD_TYPE=Release -DARROW_PARQUET=ON \
-DBoost_NO_BOOST_CMAKE=TRUE -DBoost_NO_SYSTEM_PATHS=TRUE -DBOOST_ROOT=${BOOST_BUILD_DIR}/include -DBOOST_LIBRARYDIR=${BOOST_BUILD_DIR}/lib/boost -DARROW_BOOST_USE_SHARED=OFF -DBOOST_INCLUDEDIR=${BOOST_BUILD_DIR}/include/boost ..
cmake --build . --config Release
// There are a lot of dependencies except boost, but only boost required to be installed on system, since other could be downloaded and installed by cmake script
Project successfully compiled. I got executable which could launch, generated static libs libarrow.a, libparquet.a, shared libraries libarrow.so, libparquet.so
In my main project I want to use such libraries and I use such commands in cmake to find them
find_path(PARQUET_INCLUDE_DIR NAMES arrow/api.h PATHS ${PARQUET_DIR}/src)
find_library(PARQUET_LIBRARY_RELEASE NAMES parquet.a
PATHS build_parquet/release/Release/ )
find_library(ARROW_LIBRARY_RELEASE NAMES arrow.a
PATHS build_parquet/release/Release/ )
set(PARQUET_LIBRARIES_RELEASE ${PARQUET_LIBRARY_RELEASE} ${ARROW_LIBRARY_RELEASE} )
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Parquet DEFAULT_MSG PARQUET_INCLUDE_DIR
${PARQUET_LIBRARIES_RELEASE } )
That's work okay, libraries and includes are found.
Then I link this libraries to my project
target_link_libraries(${myExe} ${PARQUET_LIBRARIES_RELEASE} ${mySomeOtherLibraries} )
after this I got enormous amount of linker errors such that
libparquet.a(column_writer.cc.o): In function `apache::thrift::transport::TMemoryBuffer::~TMemoryBuffer()':
column_writer.cc:(.text._ZN6apache6thrift9transport13TMemoryBufferD0Ev[_ZN6apache6thrift9transport13TMemoryBufferD5Ev]+0x3): undefined reference to `vtable for apache::thrift::transport::TMemoryBuffer'
.....
so that's what I don't understand much, why lib compiled well in parquet project itself but has a lot of unresolved now, when I use it to link in my own project? Moreover I compiled project for windows and when I did the same things, but with arrow.lib and parquet.lib (instead of libparquet.a and libarrow.a ) things worked fine! I was needed only to put arrow.dll, parquet.dll to executables to run project. But in Linux I've already crashed my head
So, why it doesn't work, what I should do to finally link the project with library ?
Update
I found the problem, I had to link libraries with adding .so files (not only .a files) like this
find_library(PARQUET_LIBRARY_RELEASE NAMES parquet.so parquet.a
PATHS build_parquet/release/Release/ )
find_library(ARROW_LIBRARY_RELEASE NAMES arrow.so arrow.a
PATHS build_parquet/release/Release/ )
set(PARQUET_LIBRARIES_RELEASE ${PARQUET_LIBRARY_RELEASE} ${ARROW_LIBRARY_RELEASE} )
Project is builded. So now the question is, why I need to add .so files to linker (in Windows only static .lib are enough), is it always a case when I build project in Linux ? Is order of linkage important ( .so files first and .a files next ? )
As Uwe wrote in a comment, the https://github.com/apache/parquet-cpp repository is deprecated, and the Parquet C++ library is being developed as part of the Apache Arrow C++ codebase https://github.com/apache/arrow/tree/master/cpp. Can you try building based on that, and if you have trouble can you post on the dev#arrow.apache.org mailing list?
You have succeed to build the project when link with the shared (.so) libraries instead of the static (.a) ones.
(The command find_library actually looks for one library, which name is listed in NAMES option. In your case it found .so library because its name comes before the .a one).
Actually, both shared and static parquet libraries contain the same set of symbols, and both sets are insufficient for link. The difference is that the shared library contains information, where to find remaining symbols (in the thrift library in your case), but the static library doesn't.
For correctly link with the static libraries, you need to list dependent libraries manually.
On Windows .lib file may mean either a static library, or a import file for the shared (.dll) one. It seems that you link with the dynamic one (it has no lib prefix), which succeed like on Linux.

Understanding `Makevars` for linking to external C library in R package

I am working on a package which includes C code from third-party library (SUNDIALS). The package compiles and works (i.e., is able to solve a test ODE) with the following Makevars file performing static linking
CXX=clang++
PKG_CPPFLAGS = -I../inst/include
PKG_LDFLAGS = /usr/local/lib
PKG_LIBS= $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) $(PKG_LDFLAGS)/libsundials_cvode.a $(PKG_LDFLAGS)/libsundials_nvecserial.a
However, a slightly modified version (based on the example in R-Exts, i.e. -
PKG_LIBS = -L$(XML_DIR)/lib -lxml2) of Makevars (below) fails
CXX=clang++
PKG_CPPFLAGS = -I../inst/include
PKG_LDFLAGS = /usr/local/lib
PKG_LIBS= $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) -L$(PKG_LDFLAGS) -lsundials_cvode -lsundials_nvecserial -lm
fails with the following error message.
Error: package or namespace load failed for ‘Rcppsbmod’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so':
dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so, 6): Library not loaded: libsundials_cvode.3.dylib
Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod/libs/Rcppsbmod.so
Reason: image not found
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod’
* restoring previous ‘/Library/Frameworks/R.framework/Versions/3.4/Resources/library/Rcppsbmod’
Exited with status 1.
I am not sure why it is looking for the libraries in another location when I am specifying PKG_LDFLAGS as /usr/local/lib.
As an aside, the test example which comes which the SUNDIALS package compiles and works with the following command
gcc -Wall cvRoberts_dns.c -o cvRoberts_dns.exe -I/usr/local/include -L/usr/local/lib/ -lsundials_cvode -lsundials_nvecserial -lm
So, I know that the library is installed properly and correct files (for linking) are available at /usr/local/lib location.
The entire package source code can be found at - https://github.com/sn248/Rcppsbmod
Any help or guidance will be highly appreciated!
System-wide dynamic linking, as in your second use case which fails, requires the cooperation of the dynamic linker on your system.
That means after build and copyring the libraries to /usr/local/lib you must typically run sudo ldconfig to update the linker cache.
You can check if the libraries are know by grep'ing through the output of ldconfig -p. On my system, no sundials:
edd#rob:~$ ldconfig -p | grep sundials
edd#rob:~$
Relatedly you can (locally) use different directories by declaring them in
/etc/ld.so.conf.d/somefile.conf -- but that is of course not portable and would not help you with a package designated for CRAN.
The use of static libraries you build as part of your package as in your first example would work as it does not require any system assistance. It just takes longer the build the libraries each time.
I am fighting with similar issues, c.f. Runtime linking R-extension on MacOS. My current workaround is to set the rpath at compile time. In your case that would mean:
CXX=clang++
PKG_CPPFLAGS = -I../inst/include
PKG_LDFLAGS = /usr/local/lib
PKG_LIBS= $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) -L$(PKG_LDFLAGS) -lsundials_cvode -lsundials_nvecserial -lm -Wl,-rpath,$(PKG_LDFLAGS)
However, this does not fix your problems. Comparing the error messages I see one difference: In your case the library libsundials_cvode.3.dylib is not found, while in my case it is #rpath/libaf.3.dylib. This means that the library you installed identifies itself as libsundials_cvode.3.dylib. You can check this with
$ otool -L /usr/local/lib/libsundials_cvode.3.dylib
/usr/local/lib/libsundials_cvode.3.dylib:
/usr/local/opt/sundials/lib/libsundials_cvode.3.dylib (compatibility version 3.0.0, current version 3.1.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.0.0)
In your case the second output line should not contain an absolute path but only the basename of the library. My installation uses brew, which typically uses absolute paths as library names. In some trivial tests I had no problem with linking an R extension with these libraries.
I see several possibilities:
Try SUNDIAL from brew.
Adjust the library path in your installed libraries with
install_name_tool -id /usr/local/lib/libsundials_cvode.3.dylib /usr/local/lib/libsundials_cvode.3.dylib
to use absolute paths.
Adjust the library path in your installed libraries with
install_name_tool -id '#rpath/libsundials_cvode.3.dylib' /usr/local/lib/libsundials_cvode.3.dylib
and set rpath as above.
Adjust the name of the library your R extension is looking for with this addition to Makevars
all: $(SHLIB)
#if command -v install_name_tool; then install_name_tool -change libsundials_cvode.3.dylib /usr/local/lib/libsundials_cvode.3.dylib $(SHLIB); fi

Using CMake to find symbols in libraries

I am trying to use CMake to find if an external library depends on another library.
Example: the HDF5 library can be optionally linked with zlib
In Linux, this can be found with
readelf -Ws /usr/local/lib/libhdf5.so | grep inflateEnd
54: 0000000000000000 0 FUNC GLOBAL DEFAULT UND inflateEnd
With CMake it seems this can be found with CheckLibraryExists
https://cmake.org/cmake/help/v3.0/module/CheckLibraryExists.html
In Cmake script
set(CMAKE_REQUIRED_LIBRARIES ${HDF5_LIBRARY})
check_library_exists(${HDF5_LIBRARY} inflateEnd "" NEED_ZLIB)
message(${NEED_ZLIB})
if (NEED_ZLIB)
message("-- ZLIB library is needed...")
else()
message("-- ZLIB library is not needed...")
endif()
output is not found
-- Looking for inflateEnd in /usr/local/lib/libhdf5.so - not found
-- ZLIB library is not needed...
because of this I did the Cmake version of using readelf, that finds the symbol
but, still would like to know why the above Cmake script fails :-)
the working version is
set(have_read_symbols "no")
find_program(read_symbols_prg readelf)
if (${read_symbols_prg} MATCHES "readelf")
set(have_read_symbols "yes")
message("-- Readelf program found: " ${read_symbols_prg})
execute_process(COMMAND ${read_symbols_prg} -Ws ${HDF5_LIBRARY} OUTPUT_FILE ${CMAKE_CURRENT_BINARY_DIR}/symbols.txt)
endif()
if (${have_read_symbols} MATCHES "yes")
message("-- Detecting if HDF5 library ${HDF5_LIBRARY} needs the ZLIB library...")
file(STRINGS ${CMAKE_CURRENT_BINARY_DIR}/symbols.txt NEED_ZLIB REGEX "inflateEnd")
if (NEED_ZLIB)
message("${color_blue}-- ZLIB library is needed...${color_reset}")
else()
message("-- ZLIB library is not needed...")
endif(NEED_ZLIB)
endif()
that finds the symbol
-- Detecting if HDF5 library /usr/local/lib/libhdf5.so needs the ZLIB library...
-- ZLIB library is needed...
Ok, The Cmake documentation for checkLibraryExists() is short on details on how this is accomplished, they just say "Check if the function exists".
checkLibraryExists() tries to link a test program, and depending on success or not, the output variable parameter is set to value 1 or empty.
In this case, the symbol to look must be one that is defined in libhdf5.so only with using the zlib library. There is one and only one symbol for this case, a structure named "H5Z_DEFLATE" that is added to the library depending on a #ifdef for the zlib case.
So, this does the trick
check_library_exists(${HDF5_LIBRARY} H5Z_DEFLATE "" NEED_ZLIB)
However for the Windows case using Visual Studio this is error prone, because for check_library_exists() to detect it, Visual Studio must set an option as additional dependencies the zlib library, and this is not a requirement for the library to build successfully. So, if this option is set, check_library_exists detects the dependency, if not set, it does not.

Getting "undefined reference to" when using the lib for ARM, but not when compiling it

For one of my Qt Embedded projects I'm using a external Qt lib called SMTPEmail. This lib needs to be compiled before being included into a project, something that I managed to do successfully both for Qt 4.8 ARM and for Desktop.
The problem I'm getting is that when I include the headers into my project and include the library in the .pro, the linker gives me
(path_to_libs)/libSMTPEmail.so: undefined reference to `QSslSocket::connectToHostEncrypted(QString const&, unsigned short, QFlags<QIODevice::OpenModeFlag>)'
(path_to_libs)/libSMTPEmail.so: undefined reference to `QSslSocket::QSslSocket(QObject*)'
collect2: ld returned 1 exit status
make: *** [re8k_interface-tgt] Error 1
but only for compiling for ARM. IOW compiling the lib for both ARM and Desktop goes OK, compiling the project for Desktop using the lib goes OK but compiling it for ARM using the lib goes wrong.
Following this forum thread I suspected this could be due to missing the link to the library file of openssl (the project points to different lib folders when compiling for different environments). So I searched for all "openssl" related files inside the compiler for ARM (arm-arago-linux-gnueabi) and included in the same folder where the .so is located; same error. I then suspected the lib itself had other dependencies which were not in the path_to_libs, so I did a readelf -d libSMTPEmail.so and later in the .so.1 and readelf did return some lib dependencies that were not inside the same folder of the library. I then copied all such dependencies to the folder and got no success either.
So what could be happening? All dependencies known by me were put in place and I still get the error only for the situation where the lib is included by another project compiled for ARM.
You need to point your QMake where your libs and header file is in your .pro file;
So find where your library is assume /usr/local/include then ;
INCLUDEPATH += /usr/local/include
Add which libs you will use;
LIBS += -lSMTPEmail
You can check my answer here;
Two things stand out for me in your question:
1.
undefined reference to
This error message means that there was an error in the linking step of compilation. This occurs when you include a header to a function/class/variable but don't have the definition included in your own sources, or you do not link in a static library that does.
Searching for dependencies in libraries that are already compiled (.dll or .so) is too late, the compiler is looking for a static link, not a dynamic link.
2.
compiling the lib for both ARM and Desktop goes OK, compiling the
project for Desktop using the lib goes OK but compiling it for ARM
using the lib goes wrong.
This suggests that you are using conditional compilation in your .pro file that does a "both" compilation, a "desktop only" compilation and a "ARM only" compilation. If this is correct, you need to examine your compilation instructions for your "ARM only" compilation.
The error message itself refers to you using two functions from the QSslStock class. These are part of the QtNetwork module so you should have the following in your .pro file in order for the necessary links to be formed.
Qt += network

Resources