Building and using shared libraries with bjam - linux

Using bjam on ubuntu, I am building a c++ shared library and trying to use it in an executable. I have to build as shared since it wont link as static (lots of undefined references arise). Thats fine.
Two related problems:
1) Using a heirarchy of Jamfiles, my exe project (testServerHub) has a dependency on the shared library (pythonManager). Here's the Jamfile for the exe:
echo "Compiling serverHub//test" ;
# declare project name
project serverHub//testServerHub
: build-dir ../_gcc/intermediate
;
# build unit-test using these source files, dependent libraries and settings
exe testServerHub
: # Source
..\\..\\..\\common\\0_8_1\\test\\runner.cpp
successfulTest.cpp
# Dependent libraries by path and project name
../controller/pythonManager//pythonManager
/boost//unit_test_framework
: # Settings
<link>shared
;
install ..\\bin : testServerHub ;
And here's my lib Jamfile:
echo "Compiling serverHub/controller//pythonManager" ;
# declare project name
project serverHub/controller//pythonManager
: requirements
<define>URTH_SERVERHUB
: build-dir ../../_gcc/intermediate
;
# build library using these source files and settings
lib pythonManager
: ../../../../common/0_8_1/controller/pythonManager/pythonManager.cpp
../../../../common/0_8_1/controller/pythonManager/cppInterfaceBase.cpp
cppInterfaceServerHub.cpp
/boost/python//boost_python
/user-config//python
: <link>shared
;
# copy and rename
install ../../lib : pythonManager ;
If I run 'bjam pythonManager' the pythonManager shared library is built and copied to my project lib folder (by the final install command). However if I run 'bjam test', both testServerHub and pythonManager are built, but the libpythonManager.so is not copied to the project lib folder - the install command doens't run!
2) Okay, so as a temporary workaround, I build libpythonManager.so first and then build testServerHub executable. Both compile and link. At runtime, the executable complains about not being able to find libpythonManager.so. Not a great surprise since the runtime linker doesn't know about my project lib folder. How do I tell it to look in a certain directory for shared libraries? or how do I install libpythonManager.so into /usr/local/lib if the install command has no effect on dependent library builds?
Thank you very much
Si

I think that you could use <install-dependencies>on in the exe Jamfile, like in
install ..\\bin : testServerHub : <install-dependencies>on <install-type>LIB ;
This will install all the libraries (LIB) on which the exe depends.
See eg http://www.boost.org/doc/tools/build/doc/html/bbv2/tasks/installing.html as a reference.

Related

linking shared libraries with cmake on ubuntu

I am trying to link to shared object library (json-c) using cmake. I am a super beginner with cmake, and have tried the solutions in various SO posts to no avail.
I am following the directions laid out here
first step is to build the json-c library, which I did sucesfully.
Second step is linking to it. That's what I am having issues with. After following these instructions, my project directory structure looks like this:
Project Structure (before running make, build directory was empty)
base
|-json-c-master
|-libjson-c.so
|-libjson-c.a
|-json.h
:
:
:
|-CMakeLists.txt
|-Makefile
|-libjson-c.so
|-build
|-CMakeFiles
|-
:
:
|-Makefile.
|-src
|-main.c
|-project1.c
|-project1.h
The contents of my CMakeLists.txt in ~/base/project1/CMakeLists.txt
cmake_minimum_required(VERSION 3.10)
project(project1) set(CMAKE_CXX_STANDARD 14)
set(SOURCES src/main.c src/project1.h src/project1.c )
add_executable(project1 ${SOURCES})
find_package(json-c CONFIG)
target_link_libraries(${PROJECT_NAME} PRIVATE json-c::json-c)
then, I ran
cd build
cmake -DCMAKE_PREFIX_PATH=/home/<username>/base/project1/json-c-master ..
this populated the previously empty build directory.
then, I ran make
and I got the error that in project1.h, which #includes "json.h", the file "json.h" was not found.
How can I link json.h to project1.h?
I again appreciate your patience, I am a super noob with cmake and I've been pouring over their documentation and other SO posts to no avail.

Shared Libraries not linking together after installation with CMake

I have run into a rather strange problem with Google Tests.
In my project, I am using externalProject_add in order to download google tests and add them into my project. In my function, I believe I am asking for the project to be built, and then installed into a specific directory:
ExternalProject_Add(gTest_download
URL ${GTEST_url}
URL_HASH ${GTEST_hash}
UPDATE_COMMAND ""
BUILD_COMMAND cmake --build . --target install
CMAKE_CACHE_ARGS
-DCMAKE_C_COMPILER:PATH=${Compiler_C}
-DCMAKE_CXX_COMPILER:PATH=${Compiler_CXX}
-DBUILD_SHARED_LIBS:BOOL=ON
-DCMAKE_INSTALL_PREFIX:PATH=<BINARY_DIR>/installation
)
I can then tell the program where all the source files are living with this:
ExternalProject_Get_Property(gTest_download BINARY_DIR)
set(gTest_LIBRARY_DIR ${BINARY_DIR}/installation/lib CACHE INTERNAL "Google Test Binary Dir")
set(gTest_INCLUDE_DIR ${BINARY_DIR}/installation/include CACHE INTERNAL "Google Test Include Dir")
However, when I try to run a cmake test with protobufs I get the run time error:
./Protobuf_test: error while loading shared libraries: libgmock.so.1.11.0: cannot open shared object file: No such file or directory
Which is super odd, because I know I specifically told the program where to find the libraries in the same externalProject_add file:
set(gTest_LIBRARIES
${gTest_LIBRARY_DIR}/${prefix}gmock${suffix}
${gTest_LIBRARY_DIR}/${prefix}gmock_main${suffix}
${gTest_LIBRARY_DIR}/${prefix}gtest${suffix}
${gTest_LIBRARY_DIR}/${prefix}gtest_main${suffix}
CACHE INTERNAL "Google Test Libraries"
)
Where ${prefix} is "lib" and ${suffix} is ".lib". And I make sure to link them in my CMakeLists.txt file properly by doing target_link_libraries(Protobuf_test ${gTest_LIBRARIES} ${protobuf_LIBRARIES}) ex:
CUSTOM_PROTOBUF_GENERATE_CPP(PROTO_SRCS PROTO_HDS ${CMAKE_CURRENT_LIST_DIR} hello.proto)
include_directories(
${protoBuf_INCLUDE_DIR}
${gTest_INCLUDE_DIR}
${CMAKE_CURRENT_LIST_DIR}
)
add_executable(Protobuf_test protobuf_test.cc ${PROTO_SRCS} ${PROTO_HDS})
add_dependencies(Protobuf_test
gTest_download
protoBuf_download
)
target_link_libraries(Protobuf_test
${gTest_LIBRARIES}
${protoBuf_LIBRARIES}
)
add_test(NAME testing_protobuf COMMAND Protobuf_test)
So I went into the installation folder which is located in d/linuxBuild/lib/src/gTest_download-build/installation/lib and confirmed it exists there. I then ran ldd libgmock.so and got the following output:
libgtest.so.1.11.0 => not found
Which I thought was odd as well. gtest is in the same directory! How is that possible? So I ran ldd on gmock_main:
libgmock.so.1.11.0 => not found
libgtest.so.1.11.0 => not found
So now I have two libraries that are in the same directory however they cannot be found. Confused, I decide to go to where the libraries should have originally installed to and copied over from. So two folders up: d/linuxBuild/lib/src/gTest_download-build. I then go into that folders lib folder and verify the libraries are there. I then run the same ldd command on gmock:
libgtest.so.1.11.0 => /mnt/d/linuxBuild/lib/src/gTest_download-build/lib/libgtest.so.1.11.0 (0x00007fb651729000)
I'm confused by this and again, run it on gmock_main:
libgmock.so.1.11.0 => /mnt/d/linuxBuild/lib/src/gTest_download-build/lib/libgmock.so.1.11.0 (0x00007f58b0db3000)
libgtest.so.1.11.0 => /mnt/d/linuxBuild/lib/src/gTest_download-build/lib/libgtest.so.1.11.0 (0x00007f58b0c9c000)
I am sorry for the lengthy question, but I need to know what happened here? Why is it when I install the libraries the links break from each other and they don't know their locations compared to the ones in the original installation path? Did their symbolic links break? Did I do something incorrectly in the CMake build? I'm scratching my head on this problem since I have never encountered this before. Any advice would be greatly appreciated.
Forget about externalProject_add and use FetchContent / FetchContent_MakeAvailable, especially when dealing with CMake-ready projects:
See https://cmake.org/cmake/help/latest/module/FetchContent.html for details
include(FetchContent)
FetchContent_Declare(
googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG release-1.11.0
)
FetchContent_MakeAvailable(googletest)
# ...
target_link_libraries(Protobuf_test
PRIVATE
gmock_main
${protoBuf_LIBRARIES}
)
This way, you are linking against the gmock_main target, which will set up your libraries, includes, and any indirect dependencies correctly like any cmake project added via add_subdirectory() or find_package().

"undefined reference" error from static lib of external parquet project

I want to link expernal 'parquet' project ( https://github.com/apache/arrow/tree/master/cpp ) as part of my current project on Linux.
For this purposes I ran cmake of parquet with such parameters
cd build_parquet
cmake -DCMAKE_BUILD_TYPE=Release -DARROW_PARQUET=ON \
-DBoost_NO_BOOST_CMAKE=TRUE -DBoost_NO_SYSTEM_PATHS=TRUE -DBOOST_ROOT=${BOOST_BUILD_DIR}/include -DBOOST_LIBRARYDIR=${BOOST_BUILD_DIR}/lib/boost -DARROW_BOOST_USE_SHARED=OFF -DBOOST_INCLUDEDIR=${BOOST_BUILD_DIR}/include/boost ..
cmake --build . --config Release
// There are a lot of dependencies except boost, but only boost required to be installed on system, since other could be downloaded and installed by cmake script
Project successfully compiled. I got executable which could launch, generated static libs libarrow.a, libparquet.a, shared libraries libarrow.so, libparquet.so
In my main project I want to use such libraries and I use such commands in cmake to find them
find_path(PARQUET_INCLUDE_DIR NAMES arrow/api.h PATHS ${PARQUET_DIR}/src)
find_library(PARQUET_LIBRARY_RELEASE NAMES parquet.a
PATHS build_parquet/release/Release/ )
find_library(ARROW_LIBRARY_RELEASE NAMES arrow.a
PATHS build_parquet/release/Release/ )
set(PARQUET_LIBRARIES_RELEASE ${PARQUET_LIBRARY_RELEASE} ${ARROW_LIBRARY_RELEASE} )
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(Parquet DEFAULT_MSG PARQUET_INCLUDE_DIR
${PARQUET_LIBRARIES_RELEASE } )
That's work okay, libraries and includes are found.
Then I link this libraries to my project
target_link_libraries(${myExe} ${PARQUET_LIBRARIES_RELEASE} ${mySomeOtherLibraries} )
after this I got enormous amount of linker errors such that
libparquet.a(column_writer.cc.o): In function `apache::thrift::transport::TMemoryBuffer::~TMemoryBuffer()':
column_writer.cc:(.text._ZN6apache6thrift9transport13TMemoryBufferD0Ev[_ZN6apache6thrift9transport13TMemoryBufferD5Ev]+0x3): undefined reference to `vtable for apache::thrift::transport::TMemoryBuffer'
.....
so that's what I don't understand much, why lib compiled well in parquet project itself but has a lot of unresolved now, when I use it to link in my own project? Moreover I compiled project for windows and when I did the same things, but with arrow.lib and parquet.lib (instead of libparquet.a and libarrow.a ) things worked fine! I was needed only to put arrow.dll, parquet.dll to executables to run project. But in Linux I've already crashed my head
So, why it doesn't work, what I should do to finally link the project with library ?
Update
I found the problem, I had to link libraries with adding .so files (not only .a files) like this
find_library(PARQUET_LIBRARY_RELEASE NAMES parquet.so parquet.a
PATHS build_parquet/release/Release/ )
find_library(ARROW_LIBRARY_RELEASE NAMES arrow.so arrow.a
PATHS build_parquet/release/Release/ )
set(PARQUET_LIBRARIES_RELEASE ${PARQUET_LIBRARY_RELEASE} ${ARROW_LIBRARY_RELEASE} )
Project is builded. So now the question is, why I need to add .so files to linker (in Windows only static .lib are enough), is it always a case when I build project in Linux ? Is order of linkage important ( .so files first and .a files next ? )
As Uwe wrote in a comment, the https://github.com/apache/parquet-cpp repository is deprecated, and the Parquet C++ library is being developed as part of the Apache Arrow C++ codebase https://github.com/apache/arrow/tree/master/cpp. Can you try building based on that, and if you have trouble can you post on the dev#arrow.apache.org mailing list?
You have succeed to build the project when link with the shared (.so) libraries instead of the static (.a) ones.
(The command find_library actually looks for one library, which name is listed in NAMES option. In your case it found .so library because its name comes before the .a one).
Actually, both shared and static parquet libraries contain the same set of symbols, and both sets are insufficient for link. The difference is that the shared library contains information, where to find remaining symbols (in the thrift library in your case), but the static library doesn't.
For correctly link with the static libraries, you need to list dependent libraries manually.
On Windows .lib file may mean either a static library, or a import file for the shared (.dll) one. It seems that you link with the dynamic one (it has no lib prefix), which succeed like on Linux.

cmake error while building OpenCV in native on Android Studio

I am trying to compile OpenCV in native on Android Studio. I am trying to follow : https://sriraghu.com/2017/03/11/opencv-in-android-an-introduction-part-1/
I was succesfully able to do it by following these steps on Windows. However on Ubuntu, its not working. The error is :
Execution failed for task ‘:app:externalNativeBuildDebug’.
Build command failed. Error while executing process /home/user/Android/Sdk/cmake/3.6.4111459/bin/cmake with arguments {–build
/home/user/AndroidStudioProjects/project_folders/app/.externalNativeBuild/cmake/debug/arm64-v8a
–target native-lib}
ninja: error: ‘../../../../jniLibs/src/main/jniLibs/arm64-v8a/libopencv_java3.so’,
needed by
‘../../../../build/intermediates/cmake/debug/obj/arm64-v8a/libnative-lib.so’,
missing and no known rule to make it
The CMakeLists.txt file is:
cmake_minimum_required(VERSION 3.4.1)
set(pathToOpenCV /media/user/Stuff/Softwares/OpenCV-android-sdk/sdk/native)
include_directories(${pathToOpenCV}/jni/include)
add_library( # Sets the name of the library.
native-lib
# Sets the library as a shared library.
SHARED
# Provides a relative path to your source file(s).
src/main/cpp/native-lib.cpp )
# Searches for a specified prebuilt library and stores the path as a
# variable. Because CMake includes system libraries in the search path by
# default, you only need to specify the name of the public NDK library
# you want to add. CMake verifies that the library exists before
# completing its build.
add_library(lib_opencv SHARED IMPORTED)
set_target_properties(lib_opencv PROPERTIES IMPORTED_LOCATIONS /home/user/cmu/android-apps/OpenCV_app_June23/OpenCV_Android/app/src/main/jniLibs/arm64-v8a/libopencv_java3.so)
find_library( # Sets the name of the path variable.
log-lib
# Specifies the name of the NDK library that
# you want CMake to locate.
log )
# Specifies libraries CMake should link to your target library. You
# can link multiple libraries, such as libraries you define in this
# build script, prebuilt third-party libraries, or system libraries.
target_link_libraries( # Specifies the target library.
native-lib
# Links the target library to the log library
# included in the NDK.
${log-lib} lib_opencv)
In native-lib.cpp, I am able to add #include without any error prompts in Android Studio.
My previous searches suggest that there is something wrong with the paths. I have checked that the paths are correct, or is there a different way to put the path in Ubuntu?

Creating binary with CMake removes runtime path

I am using CMake to build a program on linux. The program compiles successfully and runs from the project build directory. The program is linked with a custom library in the directory ${HOME}/build/lib
I have an install stage with:
install(TARGETS ProgName RUNTIME DESTINATION bin)
When I run make install the program gets put in the correct place, but the cmake installer removes the runtime path from the binary.
-- Install configuration: "Debug"
-- Installing: *binary name*
-- Removed runtime path from "*binary name*"
I have read articles on the internet discussing the misuse of the LD_LIBRARY_PATH variable so I like to keep mine limited to system library locations if possible. I am not sysadmin so I cannot add the location to the default linker search path either.
Does anyone know how I can keep the development-time linking paths when installing or at least customising which paths are added to the runtime?
Cheers
Note: if you don't want to modify the cmake scripts themselves, setting property around, you can launch you cmake with a directive asking to not remove the runtime path:
See "Variables that Control the Build", with variable: "CMAKE_SKIP_RPATH"
If true, do not add run time path information.
If this is set to TRUE, then the rpath information is not added to compiled executables.
The default is to add rpath information if the platform supports it. This allows for easy running from the build tree.
To omit RPATH in the install step, but not the build step, use CMAKE_SKIP_INSTALL_RPATH instead.
If the deliveries already contained the right runtime path, that directive will avoid cmake to do any modification to the current runtime path included in said deliveries.
cmake -DCMAKE_SKIP_RPATH=ON xxx.cmake
You should look at set_target_properties command and the property BUILD_WITH_INSTALL_RPATH
http://www.cmake.org/cmake/help/cmake-2-8-docs.html#command:set_target_properties
This works for CMake 2.8
set_target_properties(foo PROPERTIES INSTALL_RPATH_USE_LINK_PATH TRUE)
where foo is the target you defined earlier:
project(foo)
add_executable(foo ...)
...
install(TARGETS foo DESTINATION bin)
...
Before
% sudo make install
Install the project...
-- Install configuration: ""
-- Installing: /opt/mystuff/bin/foo
-- Removed runtime path from "/opt/mystuff/bin/foo"
After
% sudo make install
Install the project...
-- Install configuration: ""
-- Installing: /opt/mystuff/bin/foo
-- Set runtime path of "/opt/mystuff/bin/foo" to "/opt/zzyzx/lib:/opt/bar/lib/x86_64"

Resources