Shared library versioning with cmake on github - linux

I have a fairly new project on github that produces a shared library. Going forward, I would like to use semantic versioning (as described at semver.org) for the shared library major/minor/patch numbers in the file name. The project uses CMake. The CMakeLists.txt file refers to CPACK_PACKAGE_VERSION_MAJOR, CPACK_PACKAGE_VERSION_MINOR and CPACK_PACKAGE_VERSION_PATCH, and sets these to default values if they are not passed in on the command line.
My plan is to branch on ABI changes and API additions, according to semantic versioning principles.
I know github has support for creating and naming release packages containing the project source based on git tags. But I do not see a way to propagate the major, minor and patch numbers to the shared library name when the github user builds a release on their machine.
For example, if I have a branch called, myproj_1_2, and a release tag called myproj_rel_1_2_9, is there a way to have the shared library built by a user be name libmyproj.so.1.2.9?
Is this just a matter of documenting that a user should pass the build name information on the cmake command line, and the have the CMakeLists.txt file parse this and set CPACK_PACKAGE_VERSION_MAJOR, CPACK_PACKAGE_VERSION_MINOR and CPACK_PACKAGE_VERSION_PATCH accordingly, or is there a more elegant way to do this?

Your statement about how CPACK_PACKAGE_VERSION_XXX is set is incorrect. The CPack variables in question are set by the project command if the project command specifies versioning. So when you create the 1.2.9 branch you would set 1.2.9 as the version number in the project command.
From CPack Help
CPACK_PACKAGE_VERSION_MAJOR
Package major version. This variable will always be set, but its default value depends on whether or not version details were given to
the project() command in the top level CMakeLists.txt file. If version
details were given, the default value will be
CMAKE_PROJECT_VERSION_MAJOR. If no version details were given, a
default version of 0.1.1 will be assumed, leading to
CPACK_PACKAGE_VERSION_MAJOR having a default value of 0.
Project command
> project(<PROJECT-NAME>
> [VERSION <major>[.<minor>[.<patch>[.<tweak>]]]]
> [DESCRIPTION <project-description-string>]
> [HOMEPAGE_URL <url-string>]
> [LANGUAGES <language-name>...])
If you don't want to set the VERSION via the project command then there are multiple other ways of setting the relevant variables.
Examples are located:
https://cmake.org/cmake-tutorial/
Also look at how CMake handles versions:
https://gitlab.kitware.com/cmake/cmake/blob/master/Source/CMakeVersionSource.cmake
https://gitlab.kitware.com/cmake/cmake/blob/master/Source/cmVersionConfig.h.in
Another example of how to get git meta data for setting version related information:
https://github.com/pmirshad/cmake-with-git-metadata/blob/master/CMakeLists.txt

Related

Project-specific override for Cargo

I primarily want to use Debian's Rust packages, rather than fetching some random code from the wider Internet (I'm old-fashioned, I know, let's not get into that part). To this end, my ~/.cargo/config.toml looks like
[net]
offline = true
[source]
[source.crates-io]
replace-with = "debian"
[source.debian]
directory = "/usr/share/cargo/registry"
This works great after I install the librust-*-dev packages that I desire. However, in some specific projects, I'd like to lift this rule and tell Cargo "hey, you can in fact go wild and get whatever you want from crates.io". According to the Cargo book, a project-specific /project/.cargo/config.toml should take precedence over my user one. Assume this project-specific .cargo/config.toml:
[net]
offline = false
[source]
[source.crates-io]
I'm still not able to cargo build a project with dependencies from outside of my replacement source. If for example, I make a Cargo.toml that depends on yew (a randomly chosen crate that I know isn't available in my replacement source) I get
$ cargo build
error: no matching package found
searched package name: `yew`
What am I misunderstanding about Cargo's sources, replacement and per-project overrides?
The answer suggested by #blackgreen is one possible workaround for the underlying problem until issues 10045 and 10057 (or a combination thereof) are solved. Another, perhaps slightly less ugly, workaround follows below for those who need it.
I ended up working around the problem using UnionFS (I guess the more modern OverlayFS should work well too).
I simply add
[source.crates-io]
replace-with = "union"
[source.union]
directory = "/home/gspr/.cargo-overlay/union-registry"
to my ~/.cargo/config.toml and then do
unionfs -o ro /usr/share/cargo/registry:/home/gspr/.cargo-overlay/local-registry /home/gspr/.cargo-overlay/union-registry
Now /home/gspr/.cargo-overlay/union-registry reflects the union of /usr/share/cargo/registry and /home/gspr/.cargo-overlay/local-registry, with priority to the former in case of conflicts.
So what goes in ~/.cargo-overlay/local-registry? Individual extra crates, in the same way as in Debian's /usr/share/cargo/registry. That is to say, directories named cratename-version as they are distributed by upstream – but with a single extra file, namely .cargo-checksum.json added to them. The content of that extra file can be extracted from the crates.io index as follows.
Suppose we have cloned the crates.io index into ~/.cargo-overlay/crates.io-index, i.e.
git clone https://github.com/rust-lang/crates.io-index.git ~/.cargo-overlay/crates.io-index
Then suppose we've extracted a crate foo at version 0.1.2 into ~/.cargo-overlay/local-registry/foo-0.1.2. We can generate the missing .cargo-checksum.json like so:
cd ~/.cargo-overlay
index_file=$(find crates.io-index -type f -name foo)
cksum=$(jq -r "select(.name == \"foo\" and .vers == \"0.1.2\" ) | .cksum" ${index_file})
jo package="${cksum}" files="{}" > local-registry/foo-0.1.2/.cargo-checksum.json
It looks as if you are suffering from this issue: https://github.com/rust-lang/cargo/issues/8687
You would like to unset a config key on a upper-level config.toml but this is not supported.
I've played a bit with the config, and the only way I got it to work was to overwrite in the project-local config.toml the properties that were set in the upper-level config.toml.
In your case your upper-level config.toml specifies replace-with, so you have to overwrite that. But you can't overwrite it with crates-io, which is the registry you want to use, because that is exactly the registry with the replace-with key.
So until the above issue gets acted upon, we have to, essentially, use a mirror, both in the config and as an actual registry to download from:
[net]
offline = false
[source]
[source.crates-io]
replace-with = "crates-io-mirror"
[source.crates-io-mirror]
registry = "https://gitlab.com/integer32llc/crates.io-index"
As we both tested, it seems it's not possible to reuse the normal crates.io registry url because that is already defined and will fail with:
error: source crates-io-mirror defines source registry https://github.com/rust-lang/crates.io-index, but that source is already defined by crates-io note: Sources are not allowed to be defined multiple times.
So instead the URL above is an actual mirror server of crates.io. Then you can run cargo build successfully in the local project.
The recently released Cargo 1.56 adds a feature that should let one do what my question asks for: patch tables can now be specified in a project-specific .cargo/config.toml, which means that [patch] stanzas can now be introduced outside of Cargo.toml. That should do the trick! I haven't yet verified this, as I am stuck with an older Cargo for a little while still.

wxWidgets fails to build due to missing wxxml.lib

Apparently anything GUI-related in terms of components involves XML. I cannot go around actually configuring and building wxWidgets from source because of that. I'm new to wxWidgets.
My current setup is on Win10 with MSVC v141 (Visual Studio 2017) with the latest CMake version (currently 3.21).
Inside the config.cmake of the wxWidgets projects (using latest master branch) I see
wx_get_dependencies(EXTRALIBS_XML xml)
I am also calling CMake with -DwxUSE_XML=ON (among other parameters) but this still leads to:
the XML dependency is nowhere to be found
respectively it's not built
Linking then fails with the following error:
LINK : fatal error LNK1104: cannot open file 'wxxml.lib' [C:\Users\...\CMakeBuilds\ef5b5ada-ee42-7735-988a-ae37c735ccff\build\deps\build\wxwidgets\libs\qa\wxqa.vcxproj]
What library is actually wxWidgets using and how do I trigger it's retrieval and accordingly configuration and building? Since I am adding wxWidgets to my CMake project as an ExternalProject component, I would appreciated something in that line of thought. However any kind of information regarding this issue is more than welcome especially since it will shine light on how to configure other features (if I want them in the future) such as WebView.
The wxxml.lib issue is fixed now. While fixing it I also discovered a bug (of sort) in the build system of wxWidgets.
The reason why it failed to build this library in particular was actually quite simple but due to the lack of knowledge in the dependencies of wxWidgets. I thought that wxWidgets, given it depends on XML so much, has its own XML parser. Well, not really. The wxXML component actually uses and underlying 3rd party dependency called EXPAT, which - as you can see in my question - I have deactivated since it was giving me issues during the build (due to the still present problem of not being able to automatically retrieve dependencies).
What I did was to clone the libexpat repository, add it as an ExternalProject, set the variables for the libraries and include directory and pass them onto my wxWidgets project. But there is a catch...
The expat.cmake file looks as follows:
#############################################################################
# Name: build/cmake/lib/expat.cmake
# Purpose: Use external or internal expat lib
# Author: Tobias Taschner
# Created: 2016-09-21
# Copyright: (c) 2016 wxWidgets development team
# Licence: wxWindows licence
#############################################################################
if(wxUSE_EXPAT STREQUAL "builtin")
# TODO: implement building expat via its CMake file, using
# add_subdirectory or ExternalProject_Add
wx_add_builtin_library(wxexpat
src/expat/expat/lib/xmlparse.c
src/expat/expat/lib/xmlrole.c
src/expat/expat/lib/xmltok.c
)
set(EXPAT_LIBRARIES wxexpat)
set(EXPAT_INCLUDE_DIRS ${wxSOURCE_DIR}/src/expat/expat/lib)
elseif(wxUSE_EXPAT)
find_package(EXPAT REQUIRED)
endif()
I would use the *.cmake files of the 3rd party dependencies stored inside <ROOT_OF_WXWIDGETS_PROJECT>/build/cmake/lib to determine which variables I need to set if builtin is selected as the value for the respective library. Since I want to use my own I need sys (e.g. -DwxUSE_EXPAT=sys as a CMAKE_ARGS inside my wxWidgets ExternalProject) and also to pass the headers and libraries accordingly.
Given the file above one would assume that EXPAT_LIBRARIES is required. However after failing to build (yet again) and seeing that the reason was the activated expat build and that it was set as builtin I checked the log in detail and found the following error:
Could NOT find EXPAT (missing: EXPAT_LIBRARY) (found version "2.2.6")
Notice the EXPAT_LIBRARY. After passing it (-DEXPAT_LIBRARY=...) my build was complete. For me this is a bug or simply inconsistency between the dependency cmake file and the rest of the wxWidgets project.
It is important to note that I do not retrieve the external dependency through wxWidgets itself (see config.cmake and more precisely the macro wx_get_dependencies(...)). This solves the problem with a basic configuration and build of wxWidgets but if you don't want to tackle every dependency of wxWidgets on your own (why should you?), I recommend looking for a solution where the dependencies (at least the ones you don't want to deal with) are automatically retrieved, configured and build as builtin.

Delphi Linux64: how to retrieve the version information set by Project Options Version?

I gather there is no universal standard for putting version numbers into executables on Linux, in contrast to Windows which requires a certain structure for those details.
FreePascal has made its own standard and Delphi lets you define Version information for a Linux executable.
If we can put Version information in, we must be able to get it back out? How? Specifically on Linux64?
I have searched *.pas and *.inc in Studio\19.0\source\rtl\posix and I have not found anything on 'version' nor 'fileinfo' that could help.
Back with Kylix, I used to use argp_program_version from libc.
On a clean new DUnitX project, after adding {$ *.res} to the DPR so that the version information will stick, I can look at Project Options for Linux64 and I can see that they have a CFBundleVersion number defaulting to 1.0.0 for the project. How can I get the CFBundleVersion at runtime in my Linux64 executable?
Or, if that is not possible, could/should Delphi match the FreePascal standard and put the fileinfo into something which both Lazarus and Delphi could view at runtime??
This excerpt (metioned at both Windows and MacOS/iOS sections) from https://docwiki.embarcadero.com/RADStudio/Alexandria/en/Version_Info may help:
Go to Tools > Options > IDE > Environment Variables and add a new User variable called SAVEVRC with Value=TRUE. This variable enables the generation of a .vrc file with an auto generated build number and other information.
another excerpt from there (not sure if related to the aforementioned SAVEVRC or not) is:
Note: If you define the VersionInfo resource and add it to the project via a $R directive in the project file, the compiler will ignore settings of Version Info at Project Options dialog box and use the custom resource data instead, given an application can have only one VersionInfo resource on Windows.

How to link ".a" fat-static-libs with Apportable?

How do I link ".a" fat-static-libs with Apportable?
In the beginning, I did get a warning,
Warning: Library not found for lib-name. Try adding this to the
'deps' array in the 'add_params' section of your configuration.json
file. Check ~/.apportable/SDK/System for the directory names that
correspond to 'deps' entries.
I did add it to "deps" in the JSON right where it says this:
// A list of dependencies. Typically these correspond to frameworks in the xcode project.
My library isn't a framework, though. It's just a .a static library that has armv7, armv7s, and i386 parts which I assembled using lipo from two libraries (an armv7 and armv7s .a and a i386 .a) built with Xcode. They both use a single framework which is this Boost framework.
Adding it under deps squelched that warning message, but the apportable linker is still giving undefined references, so it is still not properly linking this file.
Now I know that Apportable has to re-jitter all this stuff, since Android won't know what to do with a Mac-executable format, so it's probably got to go pick apart my library and possibly turn it into an ELF-library before final linking. I'm not sure how to go about debugging this at this point, but is this supported at all?
Since you get "library not found" that means Apportable simply can't find the file. Hence the problem is merely with the file's location (or existence) and not what's in the library or how it is built.
The most common issue with dependent targets, especially Xcode projects dropped into other Xcode projects respectively workspaces with multiple projects, is that the resulting output of each target/project goes to different folders. Then Apportable (as well as xcodebuild under some circumstances) can't find the resulting libraries.
First step you should try is to make sure that the target dependencies are set. Select the app target, go to the Build Phases pane and under Target Dependencies add all frameworks and libraries that appear in the list and that your project depends on. This should ensure that the dependent frameworks/libraries do get built - because Xcode's built-in automatic dependency resolution isn't available to command line tools from what I understand. So you need to explicitly specify the dependent projects respectively their output.
If that doesn't help, you can force all targets to write their output to the same folder. Under Build Settings for every target change the Build Products Path (symbolic name: SYMROOT) to the same folder, for example ~/myprojectsbuildoutput
That way even dependent other projects will place their output in the same folder and xcodebuild as well as Apportable (it depends on xcodebuild) will be able to find the library files.
Tip: make sure your project builds successfully on the command line. Open Terminal, cd into the folder where the .xcodeproj bundle is and enter xcodebuild. If xcodebuild fails merely due to validation, disable Validate Built Product under Build Settings of the app target. If xcodebuild fails, Apportable likely isn't going to work either because it depends on xcodebuild. So as a prerequisite make sure that xcodebuild works on your project.
If xcodebuild also gives you "library not found" try calling it with a specific SYMROOT:
xcodebuild SYMROOT=~/myprojectsbuildoutput
If that then works you know you have to update each target's Build Projects Path. From what I know it's not currently possible (or not documented) to pass custom xcodebuild parameters via Apportable, so it needs to be set up in the .xcodeproj itself.

Building Boost on Linux - library names

I am trying to build an application which depends on Boost. So I downloaded Boost 1_41_0 to my Linux box and followed the instructions found on the Boost site for Unix variants,
http://www.boost.org/doc/libs/1_41_0/more/getting_started/unix-variants.html.
They basically suggest that I run ./bjam install, which I did. The build completed successfully. However, the library names don't seem to match the Boost naming convention described both in the documentation above, and what is specified in the makefile of the application I am trying to build.
I noticed that there are a bunch of options that I can specify to bjam and I tried to play with those, but no matter what happens I can't seem to get it quite right. My understanding is that the libraries should go into the $BOOST_ROOT/lib directory. This is where the libraries show up, but named:
libboost_thread.a
libboost_thread.so
libboost_thread.so.1.41.0
I'd expect them to be named libboost_thread-gcc41-mt-d-1_41_0 or something similar.
I did try ./bjam --build-type=complete --layout=tagged and I see:
libboost_thread.a
libboost_thread-mt.a
libboost_thread-mt-d.a
libboost_thread-mt-d.so
libboost_thread-mt-d.so.1.41.0
libboost_thread-mt-s.a
libboost_thread-mt-sd.a
libboost_thread-mt.so
libboost_thread-mt.so.1.41.0
libboost_thread.so
libboost_thread.so.1.41.0
So, I am not sure if I should just make stage my -L directory? Is there any documentation which describe this in more detail?
The names was changed in 1.40.0 - see in release notes:
Build System
The default naming of libraries in
Unix-like environment now matches
system conventions, and does not
include various decorations.
They probably forgot to update this part in the build documentation.
There are two variables here. First is "install" vs. "stage" (default). "install" copies both libraries and headers into a directory -- /usr/local by default, and you can then remove source tree. "stage" puts libraries to "stage/lib", and you should add "-L /stage/lib -I " flags.
Second is --layout=versioned and --layout=system. It seems like you have discovered what they do already, and indeed, system is default since 1.40. The getting started guide fails to mention this, and I've added an action item to update it. Ideally, you should talk to the authors of the application to use the system naming of boost libraries. If that's not possible, then building with --layout=versioned is the only option.
From the Boost documentation at http://www.boost.org/doc/libs/1_35_0/more/getting_started/windows.html#library-naming, the convention is:
-mt Threading tag: indicates that the library was built with multithreading support enabled. Libraries built without multithreading support can be identified by the absence of -mt.
-d ABI tag: encodes details that affect the library's interoperability with other compiled code. For each such feature, a single letter is added to the tag:
Key Use this library when:
s linking statically to the C++ standard library and compiler runtime support libraries.
g using debug versions of the standard and runtime support libraries.
y using a special debug build of Python.
d building a debug version of your code.
p using the STLPort standard library rather than the default one supplied with your compiler.
n using STLPort's deprecated “native iostreams” feature.
For example, if you build a debug version of your code for use with debug versions of the static runtime library and the STLPort standard library in “native iostreams” mode, the tag would be: -sgdpn. If none of the above apply, the ABI tag is ommitted.

Resources