My project uses external packages. These packages are distributed in tarball and their contents is extracted when the tarballs are updated.
For example, I use boost. The corresponding external package is boost.tar.bz2. This package contains the header files and libraries. What I would like to do is to automatically extract the contents of this archive with scons when the tarball is updated.
I can achieve this with a Makefile using a "timestamp file". When the tarball is newer than the timestamp file, the archive is automatically extracted:
all: external-packages
external-packages: boost xml2
boost: .boost-timestamp
xml2: .xml2-timestamp
.boost-timestamp: boost.tar.bz2
#echo updating boost externals
#tar xjf boost.tar.bz2
#touch .boost-timestamp
.xml2-timestamp: xml2.tar.bz2
#echo updating xml2 externals
#tar xjf xml2.tar.bz2
#touch .xml2-timestamp
clean:
rm -rf .*-timestamp boost xml2
How can I achieve the same with scons?
I think your best bet is to have a look at the untar builder.
After you add the addition of the builder function/emitter in your site_scons folder your scons file might look like this:
env = Environment()
env..Append(BUILDERS = {'UnTar' : unTarBuilder})
external_package = "packages/boost.tar.bz2"
archive = env.UnTar(source=external_package)
You should however note that the untar builder doesn't take directories into the emitter, and thus does not delete these on a clean
The point with this builder is the emitter it will make sure that SCons knows that it can create the headers, so whenever something depends on them (someone include them) it will launch the untar builder whenever the tar file has changed.
Related
From the description of https://stackoverflow.com/a/2531841/5983841 and https://stackoverflow.com/a/26832773/5983841 , my understanding is that Makefile.am is not needed in the dist tarball.
I tried
wget https://dist.libuv.org/dist/v1.44.2/libuv-v1.44.2-dist.tar.gz
tar xf libuv-v1.44.2-dist.tar.gz
cd libuv-1.44.2/
mv Makefile.am Makefile.am.bak
./configure
make
it gives error when running make:
tian#tian-B250M-Wind:~/Desktop/playground/garage/libuv-1.44.2$ make
make: *** No rule to make target 'Makefile.am', needed by 'Makefile.in'. Stop.
I answered there: autotools are intended to be used with free software. A dist tarball for a free software project should include all the files needed for someone to be able to make changes to the project and rebuild it, as they want to: that's the foundational goal of Free Software.
They can't do that if you omit critical build files, like Makefile.am. If they wanted to add a new file or something to the project, they need the Makefile.am to modify it. So it should be included in the dist tarball.
Saying that the file is not required in order to build the software as-is without modification, is not the same thing as saying that it can be omitted.
In this specific case, automake-generated makefiles contain rules to check whether someone modified the Makefile.am file and if so, the rules will re-run automake to ensure everything is up to date and correct so you don't have to remember to do it by hand. However this of course requires that the Makefile.am file be present so make can determine that it's up to date.
I am working with Linux built with Yocto. I would like to add to the image my app to /bin and some header file to /usr/include. I have no problem with adding the app to /bin, but I am not able to add the header file to my rootfs. The .h file is added to a proper package, but it is not copied to rootfs.
Here is my recipe:
bindir = "${localdir}/bin"
incldir = "${localdir}/usr/include"
FILESEXTRAPATHS_prepend := "${THISDIR}/files/:"
SRC_URI = "file://My_app_dir/* \
\
"
S = "${WORKDIR}"
FILES_${PN} += "${incldir}/*"
do_compile() {
cd My_app_dir/src
make
}
do_install() {
install -d ${D}${bindir}
cp "${S}/My_app_dir/src/my_app" "${D}${bindir}/my_app"
install -d ${D}${incldir}
cp "${S}/My_app_dir/some_lib.h" "${D}${incldir}/some_lib.h"
}
After building the image, the include file exists in /build/tmp/work/<machine>/<my_app>/image/usr/include.
Do you have any idea why I cannot add .h file to /usr/include in rootfs? Thank you in advance for any help.
The header files (among other files like pkgconfig and shared library symlinks) are not added to the main package (say foo), but to the development package (e.g. foo-dev). This is called package split and you can learn more in the Package Splitting of the official documentation. The development packages (and BTW also the debug foo-dbg) are not installed by default.
But please be aware that adding the development package may pull other dependencies (because of various runtime dependencies) and files (there are other files in the development package).
Please note that your line FILES_${PN} += "${incldir}/*" has no effect, as the files in $includedir (i.e. FILES_${PN}-dev) are split before the FILES_${PN} are processed. The order is defined in the variable PACKAGES (check the official documentation).
BTW, there are minor things in the recipe which you can update (unrelated to your question though):
You can use location of standard system paths in the respective variables bindir, includedir etc.
install is preferred over the cp in do_install.
The line FILESEXTRAPATHS_prepend := "${THISDIR}/files/:" is needed only in bbappends. The files directory inside the recipe's directory is in the standard search path of files (among other paths like ${PN} etc.).
I already could compile different targets or flavours(debug release), but the problem is when I make: make debug or make release this generate objects and the library in the respective folder folder.
This is the Makefile.am:
AM_CXXFLAGS = #AM_CXXFLAGS#
ACLOCAL_AMFLAGS = ${ACLOCAL_FLAGS}
lib_LIBRARIES = libInitDB.a
libInitDB_a_SOURCES = \
InitDB.cpp
.PHONY: debug release
debug:
make CXXFLAGS='$(CXX_DEBUG_FLAGS) $(CXXFLAGS)'
mkdir -p $(DEBUG_DIR)
mv $(lib_LIBRARIES) $(DEBUG_DIR)/$(lib_LIBRARIES)
mv *.o $(DEBUG_DIR)
release:
make CXXFLAGS='$(CXX_RELEASE_FLAGS) $(CXXFLAGS)'
mkdir -p $(RELEASE_DIR)
mv $(lib_LIBRARIES) $(RELEASE_DIR)/$(lib_LIBRARIES)
mv *.o $(RELEASE_DIR)
but the problem is when I make: make debug or make release again, as I move the objects and the library, that generates again the objects and the library that already stored in the debug or release folder.
Could someone help me to find how to avoid this and when I compile that search in the correct folder?
Instead of having one Makefile generating both flavours of your targets, you could have two separate build directories configured with different options.
For example:
mkdir debug
(cd debug && ../configure --enable-debug)
mkdir release
(cd release && ../configure --enable-release)
This way, you can go to either directory and recompile only what is needed simply typing make.
I am considering switching a cross platform project from separate build management systems in Visual C++, XCode and makefiles to CMake.
One essential feature I need is to add automatically all files in a directory to a target. While this is easy to do with make, it is not easily doable with Visual C++ and XCode (correct me if I am wrong). Is it possible to do it in directly in CMake? How?
As of CMake 3.1+ the developers strongly discourage users from using file(GLOB or file(GLOB_RECURSE to collect lists of source files.
Note: We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate. The CONFIGURE_DEPENDS flag may not work reliably on all generators, or if a new generator is added in the future that cannot support it, projects using it will be stuck. Even if CONFIGURE_DEPENDS works reliably, there is still a cost to perform the check on every rebuild.
See the documentation here.
There are two goods answers ([1], [2]) here on SO detailing the reasons to manually list source files.
It is possible. E.g. with file(GLOB:
cmake_minimum_required(VERSION 2.8)
file(GLOB helloworld_SRC
"*.h"
"*.cpp"
)
add_executable(helloworld ${helloworld_SRC})
Note that this requires manual re-running of cmake if a source file is added or removed, since the generated build system does not know when to ask CMake to regenerate, and doing it at every build would increase the build time.
As of CMake 3.12, you can pass the CONFIGURE_DEPENDS flag to file(GLOB to automatically check and reset the file lists any time the build is invoked. You would write:
cmake_minimum_required(VERSION 3.12)
file(GLOB helloworld_SRC CONFIGURE_DEPENDS "*.h" "*.cpp")
This at least lets you avoid manually re-running CMake every time a file is added.
The answer by Kleist certainly works, but there is an important caveat:
When you write a Makefile manually, you might generate a SRCS variable using a function to select all .cpp and .h files. If a source file is later added, re-running make will include it.
However, CMake (with a command like file(GLOB ...)) will explicitly generate a file list and place it in the auto-generated Makefile. If you have a new source file, you will need to re-generate the Makefile by re-running cmake.
edit: No need to remove the Makefile.
Extension for #Kleist answer:
Since CMake 3.12 additional option CONFIGURE_DEPENDS is supported by commands file(GLOB) and file(GLOB_RECURSE). With this option there is no needs to manually re-run CMake after addition/deletion of a source file in the directory - CMake will be re-run automatically on next building the project.
However, the option CONFIGURE_DEPENDS implies that corresponding directory will be re-checked every time building is requested, so build process would consume more time than without CONFIGURE_DEPENDS.
Even with CONFIGURE_DEPENDS option available CMake documentation still does not recommend using file(GLOB) or file(GLOB_RECURSE) for collect the sources.
To use Visual Studio project hierarchy inside Clion with cmake:
cmake_minimum_required(VERSION 3.17)
project(MyProject)
set(CMAKE_CXX_STANDARD 17)
file(GLOB APP_SOURCES */*.cpp)
foreach (testsourcefile ${APP_SOURCES})
get_filename_component(testname ${testsourcefile} NAME_WLE)
get_filename_component(dirname ${testsourcefile} DIRECTORY)
file(GLOB dir_src CONFIGURE_DEPENDS
"${dirname}/*.h"
"${dirname}/*.cpp"
)
message("${testname}.cpp | ${dir_src}")
add_executable("${testname}.cpp" ${dir_src})
endforeach (testsourcefile ${APP_SOURCES})
So Why not use powershell to create the list of source files for you. Take a look at this script
param (
[Parameter(Mandatory=$True)]
[string]$root
)
if (-not (Test-Path -Path $root)) {
throw "Error directory does not exist"
}
#get the full path of the root
$rootDir = get-item -Path $root
$fp=$rootDir.FullName;
$files = Get-ChildItem -Path $root -Recurse -File |
Where-Object { ".cpp",".cxx",".cc",".h" -contains $_.Extension} |
Foreach {$_.FullName.replace("${fp}\","").replace("\","/")}
$CMakeExpr = "set(SOURCES "
foreach($file in $files){
$CMakeExpr+= """$file"" " ;
}
$CMakeExpr+=")"
return $CMakeExpr;
Suppose you have a folder with this structure
C:\Workspace\A
--a.cpp
C:\Workspace\B
--b.cpp
Now save this file as "generateSourceList.ps1" for example, and run the script as
~>./generateSourceList.ps1 -root "C:\Workspace" > out.txt
out.txt file will contain
set(SOURCE "A/a.cpp" "B/b.cpp")
I have been using cmake to build my projects out of source, which is really convenient as you avoid polluting your source directory with unnecessary files.
Assuming the CMakeLists.txt is in the current directory, this could be done as follows:
mkdir build
cd build
cmake ..
make
How can I do the same in scons?
In your SConstruct file, you use a variant dir:
SConscript("main.scons", variant_dir="build", duplicate=0)
Then in main.scons you set up everything as usual:
env = Environment()
env.Program(target='foo', source=Split('foo.c bar.c'))
It's possible to do this without hardcoding the variant dir into the SConstruct by (ab)using repositories, but that approach has its bugs. For the record, you would run the above as follows to build in another directory:
mkdir mybuild
cd mybuild
scons -Y .. -f ../main.scons
The easiest and most workable is to just use variant_dir. You then run this as usual from the top level source directory. All the build artefacts get produced in the build sub directory.
In response to JesperE's comment, here is how you could write the top level SConstruct to add an optionally named build directory:
AddOption('--build', default='build')
SConscript("main.scons", variant_dir=GetOption('build'), duplicate=0)
Then you would call this from the command line as follows, to create a build directory called "baz":
$ scons --build=baz