CMake cannot find source file, but file was not specified in CMakeLists.txt, in TFS build definition - linux

I'm porting a large project to linux. I wrote all the CMakeLists.txt files, and everything compiles in my machine.
For whatever reason we still use TFS. The old version, not git with TFS.
I'm working in my own branch, but that branch has no build definition for linux. Before I check in, I want to be sure that everything compiles on the server too. So I need to merge my branch to another one, and submit that shelve set to the build job.
In my machine everything compiles fine. But when I run the build in the server, applying a shelveset to the branch that has a linux build definition, I get an error from the build, saying
CMake Error at
/myproject/subproject/CMakeLists.txt:165 (add_library):
Cannot find source file:
/myproject/subproject/IInternalTransactionManager.h
Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm
.hpp .hxx .in .txx
Indeed, that file is not there. Cmake complains about the file not being in the sources directory, which is true, because it is in another directory. But the fact is that I'm not asking for it either! My CMakeFiles.txt file does not include that file. That file is a header which is used in a few files, contains only classes definitions (no implementations), and the directory in which myHeader.h resides has been defined in include_directories. My CMakeLists.txt looks something like this:
set(PROJECT_NAME project)
project(${PROJECT_NAME})
include_directories(
../_include
)
set(source_files
main.cpp
file_that_includes_myHeader.cpp
)
add_library( ${PROJECT_NAME} STATIC ${source_files} )
and my file structure is something like:
/myproject/subproject/main.cpp
/myproject/subproject/file_that_includes_myHeader.cpp
/myproject/subproject/CMakeLists.txt
/myproject/_include/myHeader.h
So, why should cmake complaining about a missing file, if such file is not included in the CMakeLists.txt file? And why would this happen only the build in TFS? My guess is that there is something wrong when applying the shelvetset and is not related to my code, but I cannot prove it.
I compared the code after the shelveset is applyied, and still in that version the CMakeLists.txt does not mention myHeader.h
Or, there is some rule about including headers in CMakeLists.txt files which I'm not aware of.

So, after expending too much debuging I contacted the team in charge of the build process. And as it turns out, the building process in the TFS building definition was definetly NOT what I expected. And of course this was not documented.
Our development is mostly in windows (by far). The linux build has a step before building: a script is launched which parses each Visual Studio project file, gets the included files, and substitutes the source files in the CMakeLists.txt files with the one parsed from VS. Right or wrong, is just the way it is.
I could build the linux build in my local machine because everything was done correctly. The windows build worked too, even though the VS project files sometimes included some files which were not in the source directory but in some header only directory, and somehow that compiled. I guess because the directory was defined in the include directory. But When the CMakeLists.txt files were updated, cmake complained (rightly so) about not finding the files.
So, if anybody experiences similar issues, contact your devops team or whoever is in charge of such things.

Related

Finding my Linux shared libraries at runtime

I'm porting an SDK written in C++ from Windows to Linux. There are other binaries, but at its simplest, our SDK is this:
core.dll - implicitly loaded DLL ("libcore.so" shared library on Linux)
tests.exe - an app use to test the DLL (uses google test)
All of my binaries must live in one folder somewhere that apps can find. I've achieved that on Windows. I wanted to achieve the same thing in Linux. I'm failing miserably
To illustrate, Here's the basic project tree. We use CMake. After I build I've got
mysdk
|---CMakeLists.txt (has add_subdirectory() statements for "tests" and "core")
|---/tests (source code + CMakeLists.txt)
|---/core (source code + CMakeLists.txt)
|---/build (all build ouput, CMake output, etc)
|---tests (build output)
|---core (build output)
The goal is to "flatten" the "build" tree and put all the binary outputs of tests, core, etc into one folder.
I tried adding CMake's "install" command, to each of my CMakeLists.txt files (e.g. install(TARGETS core DESTINATION bin). I then then executed sudo make install after my normal build. This put all my binaries in /usr/local/bin with no errors. But when I ran tests from there, it failed to find libcore.so, even though it was sitting right there in the same folder
tests: error while loading shared libraries: libcore.so: Cannot open shared object file: No such file or directory
I read up on the LD_LIBRARY_PATH environment variable and so tried adding that folder (/usr/local/bin) into it and running. I can see I've properly altered LD_LIBRARY_PATH but it still doesn't work. "tests" still can't find libcore.so. I even tried changing the PATH environment variable as well. Same result.
In frustration, I tried brute-force copying the output binaries to a temporary subfolder (of /mysdk/build) and running tests from there. To my surprise it ran.
Then I realized why: Instead of loading the local copy of libcore.so it had loaded the one from the build output folder (as if the full path were "baked in" to the app at build time). Subsequently deleting that build-output copy of libcore.so made "tests" fail altogether as before, instead of loading the local copy. So maybe the path really was baked in.
I'm at a loss. I've read the CMake tutorial and reference. It makes this sound so easy. Aside from the obvious (What am I doing wrong?) I would appreciate if anyone could answer any of the following questions:
What is the correct way to control where my app looks for my shared libraries?
Is there a relationship between my project build structure and how my binaries must then appear when installed?
Am I even close to the right way of doing this?
Is it possible I've somehow inadvertently "baked" (into my app) full paths to my shared libraries? Is that a thing? I use all CMAKE variables in my CMakeLists files.
You can run ldd file to print the shared object dependencies for file. It will tell you where are its dependencies being read from.
You can export the environment variable LD_LIBRARY_PATH with the paths you want the linker to look for. If a dependency is not found, try adding the path where that dependency is located at to LD_LIBRARY_PATH and then run ldd again (make sure you export the variable).
Also, make sure the dependencies have the right permissions.
Updating LD_LIBRARY_PATH is an option. Another option is using RPATH. Please check the example.
https://github.com/mustafagonul/cmake-examples/blob/master/005-executable-with-shared-library/CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
# Project
project(005-executable-with-shared-library)
# Directories
set(example_BIN_DIR bin)
set(example_INC_DIR include)
set(example_LIB_DIR lib)
set(example_SRC_DIR src)
# Library files
set(library_SOURCES ${example_SRC_DIR}/library.cpp)
set(library_HEADERS ${example_INC_DIR}/library.h)
set(executable_SOURCES ${example_SRC_DIR}/main.cpp)
# Setting RPATH
# See https://cmake.org/Wiki/CMake_RPATH_handling
set(CMAKE_INSTALL_RPATH ${CMAKE_INSTALL_PREFIX}/${example_LIB_DIR})
# Add library to project
add_library(library SHARED ${library_SOURCES})
# Include directories
target_include_directories(library PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/${example_INC_DIR})
# Add executable to project
add_executable(executable ${executable_SOURCES})
# Linking
target_link_libraries(executable PRIVATE library)
# Install
install(TARGETS executable DESTINATION ${example_BIN_DIR})
install(TARGETS library DESTINATION ${example_LIB_DIR})
install(FILES ${library_HEADERS} DESTINATION ${example_INC_DIR})

Why does cmake create a 'source' directory in my build directory?

I am getting kind of frustrated with cmake, as I am trying to learn it and use it properly.
Here is my setup:
I have a directory called ~/project. In this directory, I have:
build directory
source directory
includes directory.
CMakeLists.txt file.
The contents of this CMakeLists.txt file is:
cmake_minimum_required(VERSION 2.8)
project(myProject)
subdirs(source)
I also have another CMakeLists.txt in ~/project/source, and its content is:
cmake_minimum_required(VERSION 2.8)
include_directories("~/project/includes")
add_executable(exec entry.cpp)
Now, I go into the build directory which is empty, and do cmake ... This works fine. However I then see a 'source' directory get created as shown here.
Why is this being created? I do not know what is going on. As I understand it, it should not be doing this, it should give me everything I see here, except for the 'source' directory.
Thanks.
Inside your build directory, CMake re-creates the whole directory structure of your project. The rational is, to keep the project structure. Image a bigger project with several levels of subfolders and sources, libraries and tests scattered in a meaningful way. To run a test, you follow the structure where the test's source is located, just in the build directory instead of the source directory.
As your project, at least as far as CMake knows it, is only the source subdirectory, only this folder is created.
If you really have just the source project, I am not sure whether would be better to place the CMake project just inside source.

sharing source between Linux eclipse and MS Visual Studio

I have a linux eclipse project checked into our company svn. Works great.
The project is intended to be cross compiled on Windows.
Untill now, I have simply moved the source files between OSes. However, I thought I'd like to let svn do this for me. Should be simple enough, just checkout the eclipse linux src into the VS project dir, right? Wrong!
The correct source was checked out of svn and it worked fine. But when I tried to check it back in i kept getting "Commit not completed filename remains in conflict" errors. I hadn't even changed anything!
Did a little checking. Turns out the linux src directory is pretty much just the source and headers. On the MSVS side the project directory contains the source and headers but also contains a bunch of files that are used by VS with names like projname.vcproj etc. etc.
So, I did a checkout into a scratch dirextory, .\fred. Checked .\fred back in. No problems. Added a new file to .fred, xxx.xxx. Check in reported:
svn: E200009: Commit failed (details follow):
svn: E200009: 'C:\Projects\fred\xxx.xxx' is not under version control
Makes me wonder about those uncommitted Visual Studio files.
So, are those files my problem? Are they breaking the commit operation?
As an alternate solution I am thinking of adding the VS files to the src dir in svn. If linux/eclipse checks them out I can tell eclipse to ignore them (I think it'll just ignore them for me). Any thoughts or recommendations for this approach?
(BTW, i still had fresh source on the linux side so any thing that got clobbered could be safely restored.)
So here is one solution I have working for the moment. I am not sure how totally stable it is.
Caveat: The project i am using already existed as a MSVS project.
In the MSVS solution dir, rename the source dir (MSVS likes the source dir name to match the solution dir name, so this means the source dir may not be named src) to something uninvolved in the solution, like temp.
SVN checkout the src (eclipse like to call source dirs src).
cd into the source dir. Issue the command:
svn changelist msvs *.cpp *.h
Add *.c if needed. "msvs" is the changelist name. It can be whatever you want ti to be.
This will created a changelist for the checked out directory.
Now, copy the remaining files from the temp directory into your source dir.
When you need to do a checkin, cd into the source dir and issue this command:
svn ci --changelist msvs
Note. You have to be in the src dir for this to work.

Including DLL's In VC++(VS2010) Project Output

Disclaimer: Am a C# developer trying to understand a few C++ fundamentals
Have created a command line project which has the .lib of a DLL file linked under Linker > Input > Additional dependencies, and also has the location of the header files specified under C/C++ > General > Additional Include Directories. Ran a build and all compiled okay, with the .exe being built in the Debug/Release dir.
Problem is I also expected the .dll file the project depends on to be there (just like when you add a reference in a .Net project) - but it isn't. When launching the .exe it complains that it can't find the DLL. Fair enough, but why didn't the VC++ put the DLL there if it knows it's required?
And is there a "best practice" to ensuring the dependent DLL files are in the output dir, other than manually copying them there? I have a project that will require use of some third-party libraries, and I was hoping the IDE would help me manage them...
Big thanks for any guidance given!
If you create solution which contains exe and dll, all output files are created in $(SolutionDir) Debug or Release subdirectories, and exe runs successfully. Otherwise, you have one of the following options:
Copy Dll in Post-build step to directory where it is available for loading (usually .exe directory)
Add Dll directory to PATH.

scons: foiling an IDE when using alternate build directories

So I have scons working with an alternate build location (build/ for my output files, src/ for my input files) and it works great. Except for one thing.
I'm using an IDE (TI Code Composer 4) to debug my program. I point the IDE at the output executable to run it, and what it uses for the source files for debugging is the build/ directory. I don't really care, except when I go to edit the file in the IDE, for example main.cpp, the file is really build/main.cpp which gets clobbered as soon as I run scons again. I have to remember to edit src/main.cpp instead. I am aware of this issue and yet I make the same mistake often.
Is there a way to have scons make the source files it copies into the build path read-only? (I'd ask how to get TI CCS4 to use the right source files when it is debugging an executable, but I doubt I'd get any answers.)
This page has information about wrapping InstallTargets with a chmod call.
FYI, the scons user list is quite active with many knowledgeable people and you can get answers pretty quickly.
You could always tell scons not to duplicate source files in the build directory:
SConscript('src/SConscript', variant_dir='build', duplicate=0)

Resources