I have a complex build I'm trying to sort out on a new build server. Using CMake for creating the makefiles. Old version of CMake on a different server was 2.8.5. New version is 2.8.12.2. For some reason, the autogenerated CMakeDirectoryInformation.cmake files are missing the CMAKE_C_INCLUDE_PATH and CMAKE_CXX_INCLUDE_PATH, which specify the absolute location of the include headers available to that particular directory. Without that information, the source code in that directory that references headers via a local path (such as foo.h versus ../../../asdf/foo.h) won't compile. Any ideas why these are missing? As far as I can tell this is supposed to be autogenerated by CMake.
I found a useful command that got me further into my build:
SET(CMAKE_INCLUDE_DIRECTORIES_BEFORE ON)
But that didn't actually solve my problem. The real issue was that in one of my CMakeLists.txt files I had:
include_directories(
$[XXXX_INCLUDE_DIR}
)
The '[' bracket was causing the build to fail because that include directory was not being used. However, on the old build server, it had no issue with this square bracket.
Related
I'm porting a large project to linux. I wrote all the CMakeLists.txt files, and everything compiles in my machine.
For whatever reason we still use TFS. The old version, not git with TFS.
I'm working in my own branch, but that branch has no build definition for linux. Before I check in, I want to be sure that everything compiles on the server too. So I need to merge my branch to another one, and submit that shelve set to the build job.
In my machine everything compiles fine. But when I run the build in the server, applying a shelveset to the branch that has a linux build definition, I get an error from the build, saying
CMake Error at
/myproject/subproject/CMakeLists.txt:165 (add_library):
Cannot find source file:
/myproject/subproject/IInternalTransactionManager.h
Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
.hpp .hxx .in .txx
Indeed, that file is not there. Cmake complains about the file not being in the sources directory, which is true, because it is in another directory. But the fact is that I'm not asking for it either! My CMakeFiles.txt file does not include that file. That file is a header which is used in a few files, contains only classes definitions (no implementations), and the directory in which myHeader.h resides has been defined in include_directories. My CMakeLists.txt looks something like this:
set(PROJECT_NAME project)
project(${PROJECT_NAME})
include_directories(
../_include
)
set(source_files
main.cpp
file_that_includes_myHeader.cpp
)
add_library( ${PROJECT_NAME} STATIC ${source_files} )
and my file structure is something like:
/myproject/subproject/main.cpp
/myproject/subproject/file_that_includes_myHeader.cpp
/myproject/subproject/CMakeLists.txt
/myproject/_include/myHeader.h
So, why should cmake complaining about a missing file, if such file is not included in the CMakeLists.txt file? And why would this happen only the build in TFS? My guess is that there is something wrong when applying the shelvetset and is not related to my code, but I cannot prove it.
I compared the code after the shelveset is applyied, and still in that version the CMakeLists.txt does not mention myHeader.h
Or, there is some rule about including headers in CMakeLists.txt files which I'm not aware of.
So, after expending too much debuging I contacted the team in charge of the build process. And as it turns out, the building process in the TFS building definition was definetly NOT what I expected. And of course this was not documented.
Our development is mostly in windows (by far). The linux build has a step before building: a script is launched which parses each Visual Studio project file, gets the included files, and substitutes the source files in the CMakeLists.txt files with the one parsed from VS. Right or wrong, is just the way it is.
I could build the linux build in my local machine because everything was done correctly. The windows build worked too, even though the VS project files sometimes included some files which were not in the source directory but in some header only directory, and somehow that compiled. I guess because the directory was defined in the include directory. But When the CMakeLists.txt files were updated, cmake complained (rightly so) about not finding the files.
So, if anybody experiences similar issues, contact your devops team or whoever is in charge of such things.
I'm porting an SDK written in C++ from Windows to Linux. There are other binaries, but at its simplest, our SDK is this:
core.dll - implicitly loaded DLL ("libcore.so" shared library on Linux)
tests.exe - an app use to test the DLL (uses google test)
All of my binaries must live in one folder somewhere that apps can find. I've achieved that on Windows. I wanted to achieve the same thing in Linux. I'm failing miserably
To illustrate, Here's the basic project tree. We use CMake. After I build I've got
mysdk
|---CMakeLists.txt (has add_subdirectory() statements for "tests" and "core")
|---/tests (source code + CMakeLists.txt)
|---/core (source code + CMakeLists.txt)
|---/build (all build ouput, CMake output, etc)
|---tests (build output)
|---core (build output)
The goal is to "flatten" the "build" tree and put all the binary outputs of tests, core, etc into one folder.
I tried adding CMake's "install" command, to each of my CMakeLists.txt files (e.g. install(TARGETS core DESTINATION bin). I then then executed sudo make install after my normal build. This put all my binaries in /usr/local/bin with no errors. But when I ran tests from there, it failed to find libcore.so, even though it was sitting right there in the same folder
tests: error while loading shared libraries: libcore.so: Cannot open shared object file: No such file or directory
I read up on the LD_LIBRARY_PATH environment variable and so tried adding that folder (/usr/local/bin) into it and running. I can see I've properly altered LD_LIBRARY_PATH but it still doesn't work. "tests" still can't find libcore.so. I even tried changing the PATH environment variable as well. Same result.
In frustration, I tried brute-force copying the output binaries to a temporary subfolder (of /mysdk/build) and running tests from there. To my surprise it ran.
Then I realized why: Instead of loading the local copy of libcore.so it had loaded the one from the build output folder (as if the full path were "baked in" to the app at build time). Subsequently deleting that build-output copy of libcore.so made "tests" fail altogether as before, instead of loading the local copy. So maybe the path really was baked in.
I'm at a loss. I've read the CMake tutorial and reference. It makes this sound so easy. Aside from the obvious (What am I doing wrong?) I would appreciate if anyone could answer any of the following questions:
What is the correct way to control where my app looks for my shared libraries?
Is there a relationship between my project build structure and how my binaries must then appear when installed?
Am I even close to the right way of doing this?
Is it possible I've somehow inadvertently "baked" (into my app) full paths to my shared libraries? Is that a thing? I use all CMAKE variables in my CMakeLists files.
You can run ldd file to print the shared object dependencies for file. It will tell you where are its dependencies being read from.
You can export the environment variable LD_LIBRARY_PATH with the paths you want the linker to look for. If a dependency is not found, try adding the path where that dependency is located at to LD_LIBRARY_PATH and then run ldd again (make sure you export the variable).
Also, make sure the dependencies have the right permissions.
Updating LD_LIBRARY_PATH is an option. Another option is using RPATH. Please check the example.
https://github.com/mustafagonul/cmake-examples/blob/master/005-executable-with-shared-library/CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
# Project
project(005-executable-with-shared-library)
# Directories
set(example_BIN_DIR bin)
set(example_INC_DIR include)
set(example_LIB_DIR lib)
set(example_SRC_DIR src)
# Library files
set(library_SOURCES ${example_SRC_DIR}/library.cpp)
set(library_HEADERS ${example_INC_DIR}/library.h)
set(executable_SOURCES ${example_SRC_DIR}/main.cpp)
# Setting RPATH
# See https://cmake.org/Wiki/CMake_RPATH_handling
set(CMAKE_INSTALL_RPATH ${CMAKE_INSTALL_PREFIX}/${example_LIB_DIR})
# Add library to project
add_library(library SHARED ${library_SOURCES})
# Include directories
target_include_directories(library PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/${example_INC_DIR})
# Add executable to project
add_executable(executable ${executable_SOURCES})
# Linking
target_link_libraries(executable PRIVATE library)
# Install
install(TARGETS executable DESTINATION ${example_BIN_DIR})
install(TARGETS library DESTINATION ${example_LIB_DIR})
install(FILES ${library_HEADERS} DESTINATION ${example_INC_DIR})
I have SCons code in which I am using SConscripts to build different directories separately. In my Src directory, my SConscript builds a shared library, and then returns the resulting Node as the Python variable libMyLibrary. I typically use the install option to copy this library to a directory that is on my system's LD_LIBRARY_PATH (I'm using OpenSUSE).
So far, so good. Now, in another directory, Src/Test, another SConscript imports libMyLibrary and builds some Programs using code like this:
env.Program('myProgram', 'myProgram.cpp', LIBS=[env['LIBS'], libMyLibrary])
The program then gets installed to my local bin folder. This code does track the library dependency and build the program, but the problem is that since the library is in a sub-directory (Src), that sub-directory gets included in the linker command. Here is an abbreviated example of the linker command that SCons generates:
g++ -o Src/Test/myProgram Src/Test/myProgram.o Src/libMyLibrary.so
I believe this happens because the Node,libMyLibrary, is essentially a path. The problem is that when I try to run the program, it is not looking for libMyLibrary.so in my library folder, but rather Src/libMyLibrary.so, and of course it doesn't find it.
I do NOT want the libraries I build to be installed in sub-directories of my install folder.
I already add the Src folder to LIBPATH, so SCons adds the -LSrc option to the linker command, but that doesn't solve the problem. My preference would be that when I add a Node, the path should automatically get parsed out to add the appropriate -L and -l options.
I know that I can get around this problem by adding the string 'MyLibrary' to the LIBS variable instead of the libMyLibrary Node, but then I have to explicitly tell SCons that each Program Depends() on libMyLibrary. It seems very inefficient to short-circuit SCons's built-in dependency tracking this way. Does anyone know the correct, SCons-y way to do this?
I'm referring to your latest comment: It looks to me as if this is not really a SCons problem, but more a general linker question (XY problem). Are you perhaps simply searching for RPATH? Please also check this old SO question: scons executable + shared library in project directory
When I try to build an specific application using cmake 2.8.9, everything seems fine at first. cmake tells me that all the required header files and libraries are found, no errors. However, when I run make the build fails because of missing header files. These header files where found by cmake just seconds before running make.
I want to find out why cmake can find the file, but make cannot. How can I debug this?
Sounds like CMake "found" the headers by running a Find module, which finds files on disk and stores these in CMake variables. These variables must then be used to configure the targets defined in the CMakeList, and perhaps this step is missing in your CMakeList. Check all find_package() calls in the CMakeList and make sure their results are used where necessary.
Here's an example how that could look for a simple library (GLUT):
find_package(GLUT) # find GLUT
include_directories(${GLUT_INCLUDE_DIR}) # use variable set up by find_package() call
add_target(MyGlutUsingProgram main.cpp)
target_link_libraries(${GLUT_LIBRARIES}) # use variable set up by find_package() call
In short: This question is basically about telling Linux to load the development version of the .so file for executables in the dev directory and the installed .so file for others.
In long: Imagine a shared library, let's call it libasdf.so. And imagine the following directories:
/home/user/asdf/lib: libasdf.so
/home/user/asdf/test: ... perform_test
/opt/asdf/lib: libasdf.so
/home/user/jkl: ... use_asdf
In other words, you have a development directory for your library (/home/user/asdf) and you have an installed copy of its previous stable version (/opt/asdf) and some other programs using it (/home/user/jkl).
My question is, how can I tell Linux, to load /home/user/asdf/lib/libasdf.so when executing /home/user/asdf/test/perform_test and to load /opt/asdf/lib/libasdf.so when executing /home/user/jkl/use_asdf? Note that, even though I specify the directory by -L during link, Linux uses other methods (for example /ect/ld.so.conf and $LD_LIBRARY_PATH) to find the .so file.
The reason I need such a thing is that, of course the executables in the development directory need to link with the latest version of the library, while the other programs, would want to use the stable version.
Putting ../lib in the library path doesn't seem like a secure idea, not to mention not completely correct since you can't run the test from a different directory.
One solution I thought about is to have perform_test link with libasdf-dev.so and upon install, copy libasdf-dev.so as libasdf.so and have others link with that. This solution has one problem though. Imagine the following additional directory:
/home/user/asdf/tool: ... use_asdf_too
Which gets installed to:
/opt/asdf/bin: use_asdf_too
In my solution, it is unknown what use_asdf_too should be linked against. If linked against libasdf.so, it wouldn't work properly if invoked from the dev directory and if linked against libasdf-dev.so, it wouldn't work properly if invoked from the installed location.
What can I do? How is this managed by other people?
Installed shared objects usually don't just end with ".so". Usually they also include their soname, such as libadsf.so.42.1. The .so file for development is typically a symlink to a fully-versioned filename. The linker will look for the .so file and resolve it to the full filename, and the loader will then load the fully-versioned library instead.