How avoid recompile different targets in different paths using autotools? - release

I already could compile different targets or flavours(debug release), but the problem is when I make: make debug or make release this generate objects and the library in the respective folder folder.
This is the Makefile.am:
AM_CXXFLAGS = #AM_CXXFLAGS#
ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS}
lib_LIBRARIES = libInitDB.a
libInitDB_a_SOURCES = \
InitDB.cpp
.PHONY: debug release
debug:
make CXXFLAGS='$(CXX_DEBUG_FLAGS) $(CXXFLAGS)'
mkdir -p $(DEBUG_DIR)
mv $(lib_LIBRARIES) $(DEBUG_DIR)/$(lib_LIBRARIES)
mv *.o $(DEBUG_DIR)
release:
make CXXFLAGS='$(CXX_RELEASE_FLAGS) $(CXXFLAGS)'
mkdir -p $(RELEASE_DIR)
mv $(lib_LIBRARIES) $(RELEASE_DIR)/$(lib_LIBRARIES)
mv *.o $(RELEASE_DIR)
but the problem is when I make: make debug or make release again, as I move the objects and the library, that generates again the objects and the library that already stored in the debug or release folder.
Could someone help me to find how to avoid this and when I compile that search in the correct folder?

Instead of having one Makefile generating both flavours of your targets, you could have two separate build directories configured with different options.
For example:
mkdir debug
(cd debug && ../configure --enable-debug)
mkdir release
(cd release && ../configure --enable-release)
This way, you can go to either directory and recompile only what is needed simply typing make.

Related

CMake (CLion / Android Studio) Sub-builds fail with ninja

I have multiple dependencies for a piece of software. To make everything easier, we made CMake build files for these dependencies. For example, we have lo libfoo, so we make a folder called "make", and in there put the file "buildLibFoo.cmake", which looks like this:
include(ExternalProject)
externalproject_add(LIBFOO
URL https://github.com/lib/foo/archive/refs/tags/v1.6.0.tar.gz
URL_MD5 58e4e09322f2d1e417469eb0987f0531b
BUILD_IN_SOURCE 0
PREFIX ${CMAKE_BINARY_DIR}/deps/libfoo
SOURCE_DIR ${CMAKE_BINARY_DIR}/deps/libfoo/src/libfoo
CONFIGURE_COMMAND
cd <SOURCE_DIR> &&
autoreconf -i &&
./configure
BUILD_COMMAND
cd <SOURCE_DIR> &&
make -j$(nproc)
INSTALL_COMMAND ""
)
set(LIBFOO_INCLUDE_PATH ${CMAKE_BINARY_DIR}/deps/libfoo/src/libfoo/utils)
set(LIBFOO_LIB_PATH ${CMAKE_BINARY_DIR}/deps/libfoo/src/libfoo/utils/.libs)
In my primary CMakeFiles.cmake I add the following:
cmake_minimum_required(VERSION 3.18.1)
project("testproject")
set (COMPILE_FLAGS "-O2")
## Need LIBFOO
if(NOT (${LIBFOO_INCLUDE_PATH} AND ${LIBFOO_LIB_PATH}))
include(buildTSS.cmake)
include_directories(${LIBFOO_INCLUDE_PATH})
link_directories(${LIBFOO_LIB_PATH})
list(APPEND DEPENDENCY_LIST "LIBFOO")
endif()
add_library(
testproject
SHARED
testproject.cpp)
target_link_libraries(testproject libfoo)
add_dependencies(testproject ${DEPENDENCY_LIST})
While the compiles fine if I run cmake CMakeLists.txt and make, it fails to do so if I use Android Studio (with native code) or CLion. Here is an example in Android Studio.
Build command failed.
Error while executing process /home/derp/Android/Sdk/cmake/3.18.1/bin/ninja with arguments {-C /home/derp/AndroidStudioProjects/testproject/app/.cxx/Debug/565m494g/arm64-v8a testproject}
ninja: Entering directory `/home/derp/AndroidStudioProjects/testproject/app/.cxx/Debug/565m494g/arm64-v8a'
ninja: error: build.ninja:178: bad $-escape (literal $ must be written as $$)
Any ideas how to make it work through the IDE?
I ran into this same error for an existing project I was trying to load with CLion that also built fine using cmake from the command line.
In my case the solution was to override the default setting CLion uses for the generator tool (ninja) and set it to the "Let CMake decide"

Cmake vcpkg and Bullet

I am very much lost here and could really use some help.
I'm working on an Honours project for next year that involves a physics simulation using Bullet and Vulkan for rendering. After a few months of work I have most of the project functioning. It needs a lot of refactoring and cleaning which will be the next stage.
I have been using a makefile but wish to migrate to CMake for a few reasons. Mainly because it seems to be the standard and because I want to compile for different OS's in the future (I'm running Linux but may need to deploy on Windows or Mac). Finally, I was recompiling the whole project for even a small change, which was beginning to become a problem as I started Unit Testing more.
The old makefile is as follows :
ROOT_DIR:=$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))
OWN_INCLUDES = \
-I$(ROOT_DIR)/src/Domain \
-I$(ROOT_DIR)/src/Vk \
-I$(ROOT_DIR)/src/Ui \
-I$(ROOT_DIR)/src/Service
ADD_INCLUDES = \
-I/opt/bullet3-master/src \
-I/opt/vk_mem_alloc \
-I/opt/stb_image \
-I/opt/tiny_obj_loader/ \
-I/opt/imgui-vulkan/
BULLET_INCLUDE_PATHS_LIBS = -L/opt/bullet3-master/src/BulletCollision/ \
-L/opt/bullet3-master/src/BulletDynamics/ \
-L/opt/bullet3-master/src/LinearMath/ \
-lBulletDynamics -lBulletCollision -lLinearMath
VULKAN_SDK_PATH = /opt/Vulkan_SDK/1.2.162.1/x86_64
CFLAGS = -std=c++17 -I$(VULKAN_SDK_PATH)/include $(OWN_INCLUDES) $(ADD_INCLUDES)
LDFLAGS = -L$(VULKAN_SDK_PATH)/lib `pkg-config --static --libs glfw3` -lvulkan $(BULLET_INCLUDE_PATHS_LIBS)
IMGUI_CPP_PATHS = /opt/imgui-vulkan/*.cpp
OWN_CPP_PATHS = src/*.cpp src/Domain/*.cpp src/Vk/*.cpp src/Ui/*.cpp src/Service/*.cpp
###### Unit Testing Paths
UNIT_TEST_INCLUDE = -I/opt/catch-header/
UNIT_TESTS_PATH = $(ROOT_DIR)/unit_tests/*.cpp
VulkanRun: $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS)
g++ $(CFLAGS) -o VulkanRun $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS) $(LDFLAGS)
Unit_Test: $(UNIT_TESTS_PATH) src/Domain/*.cpp src/Vk/*.cpp src/Ui/*.cpp src/Service/*.cpp $(IMGUI_CPP_PATHS)
g++ $(UNIT_TEST_INCLUDE) $(CFLAGS) -o Unit_Test $(UNIT_TESTS_PATH) src/Domain/*.cpp src/Vk/*.cpp src/Ui/*.cpp src/Service/*.cpp $(IMGUI_CPP_PATHS) $(LDFLAGS)
VulkanDebug: $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS)
g++ $(CFLAGS) -g -o VulkanDebug $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS) $(LDFLAGS)
VulkanOpt: $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS)
g++ $(CFLAGS) -O3 -o VulkanOpt $(OWN_CPP_PATHS) $(IMGUI_CPP_PATHS) $(LDFLAGS)
.PHONY: test clean
run: VulkanRun
LD_LIBRARY_PATH=$(VULKAN_SDK_PATH)/lib
VK_LAYER_PATH=$(VULKAN_SDK_PATH)/etc/vulkan/explicit_layer.d
./VulkanRun
test: Unit_Test
LD_LIBRARY_PATH=$(VULKAN_SDK_PATH)/lib
VK_LAYER_PATH=$(VULKAN_SDK_PATH)/etc/vulkan/explicit_layer.d
./Unit_Test
debug: VulkanDebug
LD_LIBRARY_PATH=$(VULKAN_SDK_PATH)/lib
VK_LAYER_PATH=$(VULKAN_SDK_PATH)/etc/vulkan/explicit_layer.d
optimise: VulkanOpt
LD_LIBRARY_PATH=$(VULKAN_SDK_PATH)/lib
VK_LAYER_PATH=$(VULKAN_SDK_PATH)/etc/vulkan/explicit_layer.d
./VulkanOpt
7 clean:
rm -f VulkanRun
rm -f Unit_Test
rm -f VulkanDebug
rm -f VulkanOpt
I installed cmake using the latest install script for 3.21.0.
I created a CMakeLists.txt in the root of the project as follows :
cmake_minimum_required(VERSION 3.21.0)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17")
project(LanderSim)
file(GLOB_RECURSE SOURCES "src/**.cpp")
add_executable(main ${SOURCES})
find_package(Bullet CONFIG REQUIRED)
if (BULLET_FOUND)
include_directories(${BULLET_INCLUDE_DIRS})
target_link_libraries(main PRIVATE LinearMath Bullet3Common BulletDynamics BulletSoftBody)
endif (BULLET_FOUND)
After many hours of trying I decided to try vcpkg. Following the install instructions from bullet :
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install bullet3
This resulted in errors of
CMake Error at CMakeLists.txt:11 (find_package):
Could not find a package configuration file provided by "Bullet" with any
of the following names: BulletConfig.cmake bullet-config.cmake
Looking in CMakeCache.txt i see "Bullet_DIR:PATH=Bullet_DIR-NOTFOUND"
I found the BulletConfig.make file in "/home/ash/vcpkg/installed/x64-linux/share/bullet3" and in "/home/ash/vcpkg/packages/bullet3_x64-linux/share/bullet3" and set the MakeCache.txt var Bullet_DIR:PATH to these variables (tested one at a time).
Running again I get CMake set_and_check() function not recognised. Or something to that effect. Looking in the BulletConfig.make file I see these set_and_check() functions aren't recognised by the linter. I cant find any information about them being deprecated online but I assume this is the case. So I change to set() and CMake then succeeds and builds its files.
Running make I then get an error.
fatal error: btBulletDynamicsCommon.h: No such file or directory,
#include <btBulletDynamicsCommon.h>
I tried prepending bullet/ to the include path as others had this issue but it causes the same error.
So I must be doing something wrong and I'm obviously not understanding the process that CMake uses to add includes and link libraries. I'm sure, given the popularity of CMake, that there must be something obvious. But I've spent about 10 hours over a few days searching and trying different variations and I'm starting to get very frustrated.
I've bounced off CMake before (hence why I was working with a makefile for months), but I'm determined to do this properly. I just could really use some help if anyone knows how to get CMake to generate a makefile that can see a package installed with vcpkg.
Or indeed if the vcpkg of Bullet is out of date, then a way to link and include it with CMake alone would be great. I just thought vcpkg would be easier as it provides a cleaner file structure by default as well as a CMake config file.
Thanks.
EDIT1
I've used 'cmake .' and 'cmake . -DCMAKE_TOOLCHAIN_FILE=/home/ash/vcpkg/scripts/buildsystems/vcpkg.cmake' to build the makefile. Both result in the same missing headers errors when calling make.
EDIT2
All CMake files were removed from the project (except CMakeLists.txt) before each call to cmake to ensure no values were stored there.
EDIT3
Poked around a bit more. Here is the BulletConfig.cmake file :
#
# BulletConfig.cmake(.in)
#
# Use the following variables to compile and link against Bullet:
# BULLET_FOUND - True if Bullet was found on your system
# BULLET_USE_FILE - The file making Bullet usable
# BULLET_DEFINITIONS - Definitions needed to build with Bullet
# BULLET_INCLUDE_DIR - Directory where Bullet-C-Api.h can be found
# BULLET_INCLUDE_DIRS - List of directories of Bullet and it's dependencies
# BULLET_LIBRARIES - List of libraries to link against Bullet library
# BULLET_LIBRARY_DIRS - List of directories containing Bullet' libraries
# BULLET_ROOT_DIR - The base directory of Bullet
# BULLET_VERSION_STRING - A human-readable string containing the version
set(PACKAGE_PREFIX_DIR /home/ash/installed/x64-linux)
set ( BULLET_FOUND 1 )
set ( BULLET_USE_FILE "${PACKAGE_PREFIX_DIR}/share/bullet3/UseBullet.cmake" )
set ( BULLET_DEFINITIONS "" )
set ( BULLET_INCLUDE_DIR "${PACKAGE_PREFIX_DIR}/include/bullet" )
set ( BULLET_INCLUDE_DIRS "${PACKAGE_PREFIX_DIR}/include/bullet" )
set ( BULLET_LIBRARIES "LinearMath;Bullet3Common;BulletInverseDynamics;BulletCollision;BulletDynamics;BulletSoftBody" )
set ( BULLET_LIBRARY_DIRS "${PACKAGE_PREFIX_DIR}/lib" )
set ( BULLET_ROOT_DIR "${PACKAGE_PREFIX_DIR}" )
set ( BULLET_VERSION_STRING "3.17" )
# Load targets
if(NOT TARGET Bullet3Common)
file(GLOB CONFIG_FILES "${PACKAGE_PREFIX_DIR}/share/bullet3/*Targets.cmake")
foreach(f ${CONFIG_FILES})
include(${f})
endforeach()
set(_DIR)
endif()
As stated before a few of the set functions were set_and_check(). So I changed to set() as apparently cmake 3.21 has no set_and_check() function. After a little testing by printing message(), i found that PACKAGE_PREFIX_DIR was not being set anywhere. So that is why I've set it explicitly in this file. The variables are now set correctly as reported by message() in the CMakeLists.txt file. But still it make cannot find the header files.
EDIT4
I created an empty project and ran through each library I wanted to include. Everything works except for Bullet3. However it does now see the header files. What changed between the two CMakeFiles? Nothing as far as I can tell. I'll need to find out because I have to port this project over but in the meantime this is another issue with the package.
from /home/ash/projects/C++/CMakeImportTests/src/main.cpp:22:
/home/ash/vcpkg/installed/x64-linux/include/bullet/BulletCollision/CollisionDispatch/btCollisionWorld.h:77:10:
fatal error: LinearMath/btVector3.h: No such file or directory
77 | #include "LinearMath/btVector3.h"
I think this is the same issue as described #7877
If i remove all includes of Bullet but leave the CMakeList.txt untouched, we get this error:
[ 50%] Building CXX object CMakeFiles/main.dir/src/main.cpp.o
[100%] Linking CXX executable main
/usr/bin/ld: cannot find -lLinearMath
/usr/bin/ld: cannot find -lBullet3Common
/usr/bin/ld: cannot find -lBulletDynamics
/usr/bin/ld: cannot find -lBulletSoftBody
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/main.dir/build.make:104: main] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/main.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
Is this a clue that some environment variable is not set?
EDIT5
There seems to be an ordering dependency for the target_link_library call. The suggested usage is:
target_link_libraries(main PRIVATE LinearMath Bullet3Common BulletDynamics BulletSoftBody)
Checking bullet.pc in the libs/ directory i found
Libs: -L${libdir} -lBulletSoftBody -lBulletDynamics -lBulletCollision -lLinearMath
So I tried rearranging and following the pattern:
target_link_libraries(main PRIVATE BulletSoftBody BulletDynamics BulletCollision Bullet3Common LinearMath)
Additionally there was also a need to manually link directories.
target_link_directories(main PRIVATE ${BULLET_LIBRARY_DIRS})
This now compiles without error in my test project. It seems LinearMath must be after most of the other libraries (although it can be before Bullet3Common it seems).
For some reason it's still not finding the header files when I copy the exact same CMake commands over to my main project. So I'm not free of this yet.
I should say that I was able to remove the change I made to BulletConfig.cmake of setting PACKAGE_PREFIX_DIR statically.
So just to recap my issue. A small test project works and I can use bullet and number of other libraries that I use in my main project. But if i copy this working CMakeLists.txt to my main project it can no longer find the headers and throws this error :
btBulletDynamicsCommon.h: No such file or directory
8 | #include <btBulletDynamicsCommon.h>
Bullet_DIR:PATH=/home/ash/vcpkg/installed/x64-linux/share/bullet3 is the same in both cases.
After all that.
The set_and_include() error is a known issue and mathisloge over at vcpkg git said the Bullet package needs to be updated. The workaround is to change the calls to set().
The ordering of the target libraries is important. The suggested way in the Bullet vcpkg package is :
target_link_libraries(main PRIVATE LinearMath Bullet3Common BulletDynamics BulletSoftBody)
But this fails to compile. It should be:
target_link_libraries(main PRIVATE BulletSoftBody BulletDynamics BulletCollision Bullet3Common LinearMath)
Also had to tell cmake the link directories using :
target_link_directories(main PRIVATE ${BULLET_LIBRARY_DIRS})
Then I still had header missing errors. But after a restart things just started working again. Hopefully there is enough here to help someone if they hit similar problems.

Unable to cross compile PostgreSQL with OpenSSL, fails on <openssl/opensslconf.h> not found - despite specifying include search path

I am trying to cross compile PostgreSQL on an x86 host for an AArch64 target, and want to compile with OpenSSL support.
I already went ahead and successfully cross compiled OpenSSL for AArch64 using the following arguments:
../Configure linux-aarch64 --prefix=$(pwd)/packaged no-dso --cross-compile-prefix="/usr/bin/aarch64-linux-gnu-"
make -j$(nproc)
make -j$(nproc) install
Now on to cross compiling PostgreSQL, I am using the following build script:
test -e postgresql-12.2.tar.gz || wget https://ftp.postgresql.org/pub/source/v12.2/postgresql-12.2.tar.gz
test -e postgresql-12.2 || tar -xzvf postgresql-12.2.tar.gz
cd postgresql-12.2
test -e build_aarch64 && rm -rf build_aarch64
mkdir build_aarch64
cd build_aarch64
../configure --host=aarch64-linux-gnu --without-readline --without-zlib CFLAGS="-O3 -fPIC" CXXFLAGS="-fPIC" CPPFLAGS="-fPIC" --prefix=$PWD/packaged USE_DEV_URANDOM=1 --with-openssl --with-libraries=../../openssl-OpenSSL_1_1_1k/build_aarch64/packaged/lib/ --with-includes=../../openssl-OpenSSL_1_1_1k/build_aarch64/packaged/include/
make -j$(nproc)
The output of the configure commands shows that the include directory has been properly set:
configure: using CPPFLAGS=-fPIC -D_GNU_SOURCE -I../../openssl-OpenSSL_1_1_1k/build_aarch64/packaged/include/
configure: using LDFLAGS= -L../../openssl-OpenSSL_1_1_1k/build_aarch64/packaged/lib/
Running the make command fails on:
/usr/include/openssl/e_os2.h:13:11: fatal error: openssl/opensslconf.h: No such file or directory
13 | # include <openssl/opensslconf.h>
However if I run find ../../openssl-OpenSSL_1_1_1k/build_aarch64/packaged/include/ | grep opensslconf.h it outputs:
../../openssl-OpenSSL_1_1_1k/build_aarch64/packaged/include/openssl/opensslconf.h
so the file is definitely there in the include paths. Is this a bug? Am I doing something incorrectly?
Figured it out, looks like I had to use absolute paths instead of relative paths for the search directories:
../configure --host=aarch64-linux-gnu --without-readline --without-zlib CFLAGS="-O3 -fPIC" CXXFLAGS="-fPIC" CPPFLAGS="-fPIC" --prefix=$PWD/packaged USE_DEV_URANDOM=1 --with-openssl --with-libraries=$(pwd)/../../openssl-OpenSSL_1_1_1k/build_aarch64/packaged/lib/ --with-includes=$(pwd)/../../openssl-OpenSSL_1_1_1k/build_aarch64/packaged/include/
So basically just added $(pwd)/ to the start of the search directory paths.

scons: how to extract archives depending on timestamps

My project uses external packages. These packages are distributed in tarball and their contents is extracted when the tarballs are updated.
For example, I use boost. The corresponding external package is boost.tar.bz2. This package contains the header files and libraries. What I would like to do is to automatically extract the contents of this archive with scons when the tarball is updated.
I can achieve this with a Makefile using a "timestamp file". When the tarball is newer than the timestamp file, the archive is automatically extracted:
all: external-packages
external-packages: boost xml2
boost: .boost-timestamp
xml2: .xml2-timestamp
.boost-timestamp: boost.tar.bz2
#echo updating boost externals
#tar xjf boost.tar.bz2
#touch .boost-timestamp
.xml2-timestamp: xml2.tar.bz2
#echo updating xml2 externals
#tar xjf xml2.tar.bz2
#touch .xml2-timestamp
clean:
rm -rf .*-timestamp boost xml2
How can I achieve the same with scons?
I think your best bet is to have a look at the untar builder.
After you add the addition of the builder function/emitter in your site_scons folder your scons file might look like this:
env = Environment()
env..Append(BUILDERS = {'UnTar' : unTarBuilder})
external_package = "packages/boost.tar.bz2"
archive = env.UnTar(source=external_package)
You should however note that the untar builder doesn't take directories into the emitter, and thus does not delete these on a clean
The point with this builder is the emitter it will make sure that SCons knows that it can create the headers, so whenever something depends on them (someone include them) it will launch the untar builder whenever the tar file has changed.

How to do an out of source build with scons?

I have been using cmake to build my projects out of source, which is really convenient as you avoid polluting your source directory with unnecessary files.
Assuming the CMakeLists.txt is in the current directory, this could be done as follows:
mkdir build
cd build
cmake ..
make
How can I do the same in scons?
In your SConstruct file, you use a variant dir:
SConscript("main.scons", variant_dir="build", duplicate=0)
Then in main.scons you set up everything as usual:
env = Environment()
env.Program(target='foo', source=Split('foo.c bar.c'))
It's possible to do this without hardcoding the variant dir into the SConstruct by (ab)using repositories, but that approach has its bugs. For the record, you would run the above as follows to build in another directory:
mkdir mybuild
cd mybuild
scons -Y .. -f ../main.scons
The easiest and most workable is to just use variant_dir. You then run this as usual from the top level source directory. All the build artefacts get produced in the build sub directory.
In response to JesperE's comment, here is how you could write the top level SConstruct to add an optionally named build directory:
AddOption('--build', default='build')
SConscript("main.scons", variant_dir=GetOption('build'), duplicate=0)
Then you would call this from the command line as follows, to create a build directory called "baz":
$ scons --build=baz

Resources