Cmake vcpkg and Bullet - linux

Question

I am very much lost here and could really use some help.
I'm working on an Honours project for next year that involves a physics simulation using Bullet and Vulkan for rendering. After a few months of work I have most of the project functioning. It needs a lot of refactoring and cleaning which will be the next stage.
I have been using a makefile but wish to migrate to CMake for a few reasons. Mainly because it seems to be the standard and because I want to compile for different OS's in the future (I'm running Linux but may need to deploy on Windows or Mac). Finally, I was recompiling the whole project for even a small change, which was beginning to become a problem as I started Unit Testing more.
The old makefile is as follows :
ROOT_DIR:=$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
OWN_INCLUDES = \
-I$(ROOT_DIR)/src/Domain \
-I$(ROOT_DIR)/src/Vk \
-I$(ROOT_DIR)/src/Ui \
-I$(ROOT_DIR)/src/Service
ADD_INCLUDES = \
-I/opt/bullet3-master/src \
-I/opt/vk_mem_alloc \
-I/opt/stb_image \
-I/opt/tiny_obj_loader/ \
-I/opt/imgui-vulkan/
BULLET_INCLUDE_PATHS_LIBS = -L/opt/bullet3-master/src/BulletCollision/ \
-L/opt/bullet3-master/src/BulletDynamics/ \
-L/opt/bullet3-master/src/LinearMath/ \
-lBulletDynamics -lBulletCollision -lLinearMath
VULKAN_SDK_PATH = /opt/Vulkan_SDK/1.2.162.1/x86_64
CFLAGS = -std=c++17 -I$(VULKAN_SDK_PATH)/include $(OWN_INCLUDES) $(ADD_INCLUDES)
LDFLAGS = -L$(VULKAN_SDK_PATH)/lib `pkg-config --static --libs glfw3` -lvulkan $(BULLET_INCLUDE_PATHS_LIBS)
IMGUI_CPP_PATHS = /opt/imgui-vulkan/*.cpp
OWN_CPP_PATHS = src/*.cpp src/Domain/*.cpp src/Vk/*.cpp src/Ui/*.cpp src/Service/*.cpp
###### Unit Testing Paths
UNIT_TEST_INCLUDE = -I/opt/catch-header/
UNIT_TESTS_PATH = $(ROOT_DIR)/unit_tests/*.cpp
VulkanRun: $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS)
g++ $(CFLAGS) -o VulkanRun $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS) $(LDFLAGS)
Unit_Test: $(UNIT_TESTS_PATH) src/Domain/*.cpp src/Vk/*.cpp src/Ui/*.cpp src/Service/*.cpp $(IMGUI_CPP_PATHS)
g++ $(UNIT_TEST_INCLUDE) $(CFLAGS) -o Unit_Test $(UNIT_TESTS_PATH) src/Domain/*.cpp src/Vk/*.cpp src/Ui/*.cpp src/Service/*.cpp $(IMGUI_CPP_PATHS) $(LDFLAGS)
VulkanDebug: $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS)
g++ $(CFLAGS) -g -o VulkanDebug $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS) $(LDFLAGS)
VulkanOpt: $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS)
g++ $(CFLAGS) -O3 -o VulkanOpt $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS) $(LDFLAGS)
.PHONY: test clean
run: VulkanRun
LD_LIBRARY_PATH=$(VULKAN_SDK_PATH)/lib
VK_LAYER_PATH=$(VULKAN_SDK_PATH)/etc/vulkan/explicit_layer.d
./VulkanRun
test: Unit_Test
LD_LIBRARY_PATH=$(VULKAN_SDK_PATH)/lib
VK_LAYER_PATH=$(VULKAN_SDK_PATH)/etc/vulkan/explicit_layer.d
./Unit_Test
debug: VulkanDebug
LD_LIBRARY_PATH=$(VULKAN_SDK_PATH)/lib
VK_LAYER_PATH=$(VULKAN_SDK_PATH)/etc/vulkan/explicit_layer.d
optimise: VulkanOpt
LD_LIBRARY_PATH=$(VULKAN_SDK_PATH)/lib
VK_LAYER_PATH=$(VULKAN_SDK_PATH)/etc/vulkan/explicit_layer.d
./VulkanOpt
7 clean:
rm -f VulkanRun
rm -f Unit_Test
rm -f VulkanDebug
rm -f VulkanOpt
I installed cmake using the latest install script for 3.21.0.
I created a CMakeLists.txt in the root of the project as follows :
cmake_minimum_required(VERSION 3.21.0)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17")
project(LanderSim)
file(GLOB_RECURSE SOURCES "src/**.cpp")
add_executable(main ${SOURCES})
find_package(Bullet CONFIG REQUIRED)
if (BULLET_FOUND)
include_directories(${BULLET_INCLUDE_DIRS})
target_link_libraries(main PRIVATE LinearMath Bullet3Common BulletDynamics BulletSoftBody)
endif (BULLET_FOUND)
After many hours of trying I decided to try vcpkg. Following the install instructions from bullet :
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install bullet3
This resulted in errors of
CMake Error at CMakeLists.txt:11 (find_package):
Could not find a package configuration file provided by "Bullet" with any
of the following names: BulletConfig.cmake bullet-config.cmake
Looking in CMakeCache.txt i see "Bullet_DIR:PATH=Bullet_DIR-NOTFOUND"
I found the BulletConfig.make file in "/home/ash/vcpkg/installed/x64-linux/share/bullet3" and in "/home/ash/vcpkg/packages/bullet3_x64-linux/share/bullet3" and set the MakeCache.txt var Bullet_DIR:PATH to these variables (tested one at a time).
Running again I get CMake set_and_check() function not recognised. Or something to that effect. Looking in the BulletConfig.make file I see these set_and_check() functions aren't recognised by the linter. I cant find any information about them being deprecated online but I assume this is the case. So I change to set() and CMake then succeeds and builds its files.
Running make I then get an error.
fatal error: btBulletDynamicsCommon.h: No such file or directory,
#include <btBulletDynamicsCommon.h>
I tried prepending bullet/ to the include path as others had this issue but it causes the same error.
So I must be doing something wrong and I'm obviously not understanding the process that CMake uses to add includes and link libraries. I'm sure, given the popularity of CMake, that there must be something obvious. But I've spent about 10 hours over a few days searching and trying different variations and I'm starting to get very frustrated.
I've bounced off CMake before (hence why I was working with a makefile for months), but I'm determined to do this properly. I just could really use some help if anyone knows how to get CMake to generate a makefile that can see a package installed with vcpkg.
Or indeed if the vcpkg of Bullet is out of date, then a way to link and include it with CMake alone would be great. I just thought vcpkg would be easier as it provides a cleaner file structure by default as well as a CMake config file.
Thanks.
EDIT1
I've used 'cmake .' and 'cmake . -DCMAKE_TOOLCHAIN_FILE=/home/ash/vcpkg/scripts/buildsystems/vcpkg.cmake' to build the makefile. Both result in the same missing headers errors when calling make.
EDIT2
All CMake files were removed from the project (except CMakeLists.txt) before each call to cmake to ensure no values were stored there.
EDIT3
Poked around a bit more. Here is the BulletConfig.cmake file :
#
# BulletConfig.cmake(.in)
#
# Use the following variables to compile and link against Bullet:
# BULLET_FOUND - True if Bullet was found on your system
# BULLET_USE_FILE - The file making Bullet usable
# BULLET_DEFINITIONS - Definitions needed to build with Bullet
# BULLET_INCLUDE_DIR - Directory where Bullet-C-Api.h can be found
# BULLET_INCLUDE_DIRS - List of directories of Bullet and it's dependencies
# BULLET_LIBRARIES - List of libraries to link against Bullet library
# BULLET_LIBRARY_DIRS - List of directories containing Bullet' libraries
# BULLET_ROOT_DIR - The base directory of Bullet
# BULLET_VERSION_STRING - A human-readable string containing the version
set(PACKAGE_PREFIX_DIR /home/ash/installed/x64-linux)
set ( BULLET_FOUND 1 )
set ( BULLET_USE_FILE "${PACKAGE_PREFIX_DIR}/share/bullet3/UseBullet.cmake" )
set ( BULLET_DEFINITIONS "" )
set ( BULLET_INCLUDE_DIR "${PACKAGE_PREFIX_DIR}/include/bullet" )
set ( BULLET_INCLUDE_DIRS "${PACKAGE_PREFIX_DIR}/include/bullet" )
set ( BULLET_LIBRARIES "LinearMath;Bullet3Common;BulletInverseDynamics;BulletCollision;BulletDynamics;BulletSoftBody" )
set ( BULLET_LIBRARY_DIRS "${PACKAGE_PREFIX_DIR}/lib" )
set ( BULLET_ROOT_DIR "${PACKAGE_PREFIX_DIR}" )
set ( BULLET_VERSION_STRING "3.17" )
# Load targets
if(NOT TARGET Bullet3Common)
file(GLOB CONFIG_FILES "${PACKAGE_PREFIX_DIR}/share/bullet3/*Targets.cmake")
foreach(f ${CONFIG_FILES})
include(${f})
endforeach()
set(_DIR)
endif()
As stated before a few of the set functions were set_and_check(). So I changed to set() as apparently cmake 3.21 has no set_and_check() function. After a little testing by printing message(), i found that PACKAGE_PREFIX_DIR was not being set anywhere. So that is why I've set it explicitly in this file. The variables are now set correctly as reported by message() in the CMakeLists.txt file. But still it make cannot find the header files.
EDIT4
I created an empty project and ran through each library I wanted to include. Everything works except for Bullet3. However it does now see the header files. What changed between the two CMakeFiles? Nothing as far as I can tell. I'll need to find out because I have to port this project over but in the meantime this is another issue with the package.
from /home/ash/projects/C++/CMakeImportTests/src/main.cpp:22:
/home/ash/vcpkg/installed/x64-linux/include/bullet/BulletCollision/CollisionDispatch/btCollisionWorld.h:77:10:
fatal error: LinearMath/btVector3.h: No such file or directory
77 | #include "LinearMath/btVector3.h"
I think this is the same issue as described #7877
If i remove all includes of Bullet but leave the CMakeList.txt untouched, we get this error:
[ 50%] Building CXX object CMakeFiles/main.dir/src/main.cpp.o
[100%] Linking CXX executable main
/usr/bin/ld: cannot find -lLinearMath
/usr/bin/ld: cannot find -lBullet3Common
/usr/bin/ld: cannot find -lBulletDynamics
/usr/bin/ld: cannot find -lBulletSoftBody
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/main.dir/build.make:104: main] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/main.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
Is this a clue that some environment variable is not set?
EDIT5
There seems to be an ordering dependency for the target_link_library call. The suggested usage is:
target_link_libraries(main PRIVATE LinearMath Bullet3Common BulletDynamics BulletSoftBody)
Checking bullet.pc in the libs/ directory i found
Libs: -L${libdir} -lBulletSoftBody -lBulletDynamics -lBulletCollision -lLinearMath
So I tried rearranging and following the pattern:
target_link_libraries(main PRIVATE BulletSoftBody BulletDynamics BulletCollision Bullet3Common LinearMath)
Additionally there was also a need to manually link directories.
target_link_directories(main PRIVATE ${BULLET_LIBRARY_DIRS})
This now compiles without error in my test project. It seems LinearMath must be after most of the other libraries (although it can be before Bullet3Common it seems).
For some reason it's still not finding the header files when I copy the exact same CMake commands over to my main project. So I'm not free of this yet.
I should say that I was able to remove the change I made to BulletConfig.cmake of setting PACKAGE_PREFIX_DIR statically.
So just to recap my issue. A small test project works and I can use bullet and number of other libraries that I use in my main project. But if i copy this working CMakeLists.txt to my main project it can no longer find the headers and throws this error :
btBulletDynamicsCommon.h: No such file or directory
8 | #include <btBulletDynamicsCommon.h>
Bullet_DIR:PATH=/home/ash/vcpkg/installed/x64-linux/share/bullet3 is the same in both cases.

Answer 1

After all that.
The set_and_include() error is a known issue and mathisloge over at vcpkg git said the Bullet package needs to be updated. The workaround is to change the calls to set().
The ordering of the target libraries is important. The suggested way in the Bullet vcpkg package is :
target_link_libraries(main PRIVATE LinearMath Bullet3Common BulletDynamics BulletSoftBody)
But this fails to compile. It should be:
target_link_libraries(main PRIVATE BulletSoftBody BulletDynamics BulletCollision Bullet3Common LinearMath)
Also had to tell cmake the link directories using :
target_link_directories(main PRIVATE ${BULLET_LIBRARY_DIRS})
Then I still had header missing errors. But after a restart things just started working again. Hopefully there is enough here to help someone if they hit similar problems.