Set linker search path for build in CMake - linux

It seems this question has been asked very often before but none of the solutions seem to apply in my case.
I'm in a CMake/Linux environment and have to run an executable binary during the build step (protoc in particular).
This binary needs a library but it's not installed (and cannot be) in the in the standard directories like /usr, so the library cannot be found.
Unfortunately I cannot manipulate the protoc call because it's embedded in a 3rd party script.
I can now set LD_LIBRARY_PATH before every make or set it system wide but this is very inconvenient especially when it comes to IDEs in which the build takes place or distributed build scenarios with continuous build environments.
I tried to set LD_LIBRARY_PATH via
set(ENV{LD_LIBRARY_PATH} "/path/to/library/dir")
but this seems to have no effect during the build step.
So my question is: can I set a library search path in CMake which is used during the build?

Try this
SET(ENV{LD_LIBRARY_PATH} "/path/to/library/dir:$ENV{LD_LIBRARY_PATH}")
I also used this dirty trick to temporary change some environment variables:
LD_LIBRARY_PATH="/path/to/library/dir:$LD_LIBRARY_PATH" cmake ...
After execution of this line LD_LIBRARY_PATH is not changed in the current shell.
Also, I do not find it bad to change LD_LIBRARY_PATH before invoking cmake:
export LD_LIBRARY_PATH=...
It won't change anything system-wide, but it would be used for your current shell, current build process. The same holds for CI builds. You can save the variable and restore it after cmake invocation:
MY_LD=$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=...
cmake...
export LD_LIBRARY_PATH=$MY_LD

I have recently run into a somewhat similar problem.
My solution was to incorporate sourcing a file that set the appropriate environment into every command.
For example, this custom command:
add_custom_command(
OUTPUT some_output
COMMAND some_command
ARGS some_args
DEPENDS some_dependencies
COMMENT "Running some_command some_args to produce some_output"
)
Would become:
set(my_some_command_with_environment "source my_environment_script.sh && some_command")
add_custom_command(
OUTPUT some_output
COMMAND bash
ARGS -c "${my_some_command_with_environment} some_args"
DEPENDS some_dependencies
COMMENT "Running some_command some_args to produce some_output"
VERBATIM
)
Obviously, this has some disadvantages:
It relies on a bash shell being available.
It sources the environment script for every command invocation (performance issue) and you will have to change all invocations of commands that rely on that environment variables.
It changes the normal syntax of having the command follow COMMAND and the arguments follow ARGS, as now the 'real' command is part of the ARGS.
My CMake-Fu has proven insufficient to find a syntactically nicer way of doing this, but maybe somebody can comment a nicer way.

I had a similar issue for an executable provided by a third party library. The binary was linked against a library not provided by the distribution but the required library was included in the libs directory of the third party library.
So running LD_LIBRARY_PATH=/path/to/thirdparty/lib /path/to/thirdparty/bin/executable worked. But the package config script didn't set up the executable to search /path/to/thirdparty/lib for the runtime dependent so CMake would complain when CMake tried to run the executable.
I got around this by configuring a bootstrap script and replacing the IMPORTED_LOCATION property with the configured bootstrapping script.
_thirdpartyExe.in
#!/bin/bash
LD_LIBRARY_PATH=#_thirdpartyLibs# #_thirdpartyExe_LOCATION# "$#"
CMakeLists.txt
find_package(ThirdPartyLib)
get_target_property(_component ThirdPartyLib::component LOCATION)
get_filename_component(_thirdpartyLibs ${_component} DIRECTORY)
get_target_property(_thirdpartyExe_LOCATION ThirdPartyLib::exe IMPORTED_LOCATION)
configure_file(
${CMAKE_CURRENT_LIST_DIR} _thirdpartyExe.in
${CMAKE_BINARY_DIR}/thirdpartyExeWrapper #ONLY
)
set_target_properties(ThirdPartyLib::exe PROPERTIES IMPORTED_LOCATION ${CMAKE_BINARY_DIR}/thirdpartyExeWrapper)
Honestly I view this as a hack and temporary stop gap until I fix the third party library itself. But as far as I've tried this seems to work on all the IDE's I've thrown at it, Eclipse, VSCode, Ninja, QtCreator, ... etc

Related

How to manage the environment PATH used by the Android NDK build?

I use swig from my within Android.mk. I reference it directly relying on the $PATH variable to contain the path to it's executable.
The issue is that the $PATH variable is different depending on where the android-ndk tool is run from.
Background
In my Android.mk file:
# some stuff ...
#echo "$(PATH)"
swig # swig parameters here...
# more stuff ...
From the terminal, we see the system path includes the path to swig:
which swig
/usr/local/bin/swig
echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:
Issue
When the android-ndk is run from within Android Studio I get the following for PATH (note: it's missing the swig path):
/usr/bin:/bin:/usr/sbin:/sbin
This causes my build to fail because "swig" isn't recognised:
make: swig: Command not found
However, if I run the android-ndk directly from the terminal, then the PATH used is the same as my system path and the build works fine:
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:
Bad solution
Yes, I could add the path manually to my Android.mk file:
export PATH:=/usr/local/bin/:$(PATH)
However, I'd prefer not to define specific paths for my machine knowing that the whole team uses this file.
Question
I'd prefer the Android.mk file to use my systems PATH instead.
Any ideas how to do this?
Thanks!
The thing in my opinion is that environment variables may be different whether the process is launched from terminal or from the GUI (launchpad, spotlight, ...).
The former would use PATH as set in bashrc (or other shell related startup files) while the other will not.
I think you might find interesting information on how to change the environment variables for the launch deamon in the following question:
Setting environment variables in OS X?

Run from clean login shell without credentials

So I've got an executable with conflicting dependencies with the build system it is running in. (AKA Xilinx doesn't play well with others). I'd love to run a cmake script as part of the build process, but it is depended on different dlls. I can try to figure out a version of cmake that matches the xylinx dlls. Xylinx redirects the dependencies to the different C++ runtimes by setting a bunch of environment variables.
Sanitize environment with command or bash script?
Will completely clean out all environments, but what i really want to do is run the script after the .bash_rc or .bash_profile is called. (cmake seems to require some of that to be set, as env -i cmake returns an error) I'd like to do it without having to require the login credentials. Is that possible?
You can pass environment variables directly in a command line, right before the executable name:
ENV_VAR=foo command -options ...
If you want to pass multiple variables just add them one after one:
ENV_VAR_FOO=foo ENV_VAR_BAR=bar command -options ...
In your case, you'll need to find out which environment variable cmake needs to access the libraries. Then pass this to the call, like:
LIB_PATH=/path/to/libs cmake -options ...

Compiler options basic ./configure explanation

recently, i try to compile a gnu wget from source code in cygwin environtment that pop-up error if perl is not found. otherwise, perl is installed both perl and perl5 on /bin/ but the wget is try search perl on /usr/bin. i think i have missed basic ./configure to setup path executable. so my question is basic.
what is it all about options on below:
*
--bindir=DIR
--sbindir=DIR
--libexecdir=DIR
*
Thank you
These options specify directories where a software package being compiled is going to be installed. As far as I remember it doesn't deal with checks performed by configure. Make sure that perl is in $PATH. If nothing helps, try to locate the exact place in the configure script (usually it's robot-generated and not intended for human eyes, but afterall it's a shell script, and anybody can read it) and see what checks exactly are performed to locate perl.
Update: I have checked, the tests corresponding to perl look like this in configure.ac (which essentially is a "source code" for configure):
AC_PATH_PROGS(PERL, [perl5 perl], no)
AC_PATH_PROG(POD2MAN, pod2man, no)
This means that PERL with executable named perl5 or perl (somewhere in $PATH) is checked, and then POD2MAN with executable pod2man. Carefully check the configure output and config.log file and see what tests have failed.
Update2: The third argument of the AC_PATH_PROG and AC_PATH_PROGS is value-if-no-found. Also you may specify the fourth argument, $PATH for this particular check. Make sure that configure gets rebuilt after you changed configure.ac (usually it happens automatically, but may be performed by autoconf explicitly)
what is it all about options on below:
*
--bindir=DIR
--sbindir=DIR
--libexecdir=DIR
The first (bindir), is where binaries will be installed. For wget, that's (on my system) /usr/bin.
The second (sbindir), is where static binaries will be installed. You might set that to /usr/sbin.
The third (libexecdir), is where your runtime libraries are to be installed. That's usually /usr/lib.
In fact, usually you let configure set all three by using --prefix.
./configure --prefix=/usr/local

cmake : Set environment variables from a script

I have a script that sets all variables needed for the cross-compilation. Here is just part of it :
export CONFIG_SITE=~/workspace/eldk-5.4/powerpc/site-config-powerpc-linux
export CC="powerpc-linux-gcc -m32 -mhard-float --sysroot=~/workspace/eldk-5.4/powerpc/sysroots/powerpc-linux"
export CXX="powerpc-linux-g++ -m32 -mhard-float --sysroot=~/workspace/eldk-5.4/powerpc/sysroots/powerpc-linux"
export CPP="powerpc-linux-gcc -E -m32 -mhard-float --sysroot=~/workspace/eldk-5.4/powerpc/sysroots/powerpc-linux"
export AS="powerpc-linux-as "
export LD="powerpc-linux-ld --sysroot=~/workspace/eldk-5.4/powerpc/sysroots/powerpc-linux"
export GDB=powerpc-linux-gdb
If I do source environment-setup-powerpc-linux, all environment variables are imported into the current shell session, and I can compile my example.
Is it possible to import these variables in cmake? If yes, how?
A bit more details :
I am using ELDK v 5.4, and it's install script generates a script which sets all environment variables
I found this tutorial, which explains how to manually set for cross-compilation, but not how to use the script, which sets everything
if I call the script before setting cmake, all works fine, and I can cross-compile, but I'd like that cmake calls the script
Reading through the cmake quick start, you can specify variable on a command line:
cmake -DVARIABLE1=value1 -DVARIABLE2=value2 ...
Otherwise, set command in the cmake script is probably what you want, see the reference manual. To set the environment variable PATH, do:
set(ENV{PATH} "/home/martink")
To set normal variable, do:
set(variable "value")
Not sure which ones you have to set, probably the environment ones.
That said, setting environment variable prior to calling cmake is often the easiest solution to solve the problem. If you want a cross-platform way to do this that doesn't depend on the syntax of a specific shell to set environment variables, there is the cmake -E env command.
The only way to set a compiler and flags to do cross-compilation reliably with CMake is with a toolchain-file as done in the tutorial you have found.
When we faced the same issue you have (a toolkit which produces a script so set the compile-environment) we changed the toolkit in a way that it produces a toolchain-file along with the script.
In reality a cmake-toolchain-file does not change that often. The basic flags used for the target are fixed quite early in a project - normally. And with CMake's CMAKE_BUILD_TYPE you can switch between Debug and Release compilations without changing the toolchain-file.
If you have different targets to support, create different toolchain and use the out-of-source-build with CMake.
EDIT: One thing you could do is to invoke cmake with the -D-argument setting the variables you want to and having sourced your script before:
source environment-setup-powerpc-linux
cmake -DCMAKE_C_COMPILER=$CC -DCMAKE_CXX_COMPILER=$CXX etc
The result will be identical as to having used a toolchain-file.

qmake -query internal settings in Linux - where are they?

I am building a Linux system with cross-compiler using ptxdist. It allows me to configure Qt4 for installation and it builds and installs qt-everywhere-opensource-src-4.6.3 Ok. However, the qmake internal settings are screwed up and I don't know how to fix them.
When I run qmake -query I get:
me#ubuntu:~$ qmake -query
QT_INSTALL_PREFIX:/
QT_INSTALL_DATA:/
QT_INSTALL_DOCS://doc
QT_INSTALL_HEADERS://include
QT_INSTALL_LIBS://lib
QT_INSTALL_BINS://bin
QT_INSTALL_PLUGINS://plugins
QT_INSTALL_TRANSLATIONS://translations
QT_INSTALL_CONFIGURATION:/etc/xdg
QT_INSTALL_EXAMPLES://examples
QT_INSTALL_DEMOS://demos
QMAKE_MKSPECS://mkspecs
QMAKE_VERSION:2.01a
QT_VERSION:4.6.3
Through some research, it looks like this can be fixed by simply rebuilding Qt, but it's not fixing this problem. I dug into the build output a bit and it looks like the ./configure command for the Qt build has "-prefix /usr" so I don't know why this isn't being fixed.
I would like to fix these internal values manually if possible because the Qt build takes hours. Does anyone know how to do this?
At configure time these paths are hardcoded in 'src/corelib/global/qconfig.cpp', and end up hardcoded into qmake when it is built. They are also hardcoded into many other files, like all the .la and .pc files, not to mention the Makefile install rules.
The only way to fix this is to figure out why configure keeps screwing up the prefix. configure is a big shell script, so it's easy to see where $QT_INSTALL_PREFIX is assigned from the '-prefix' argument, and then the different checks that are done on it (like running it through 'config.tests/unix/makeabs'). Try putting print statements before/after $QT_INSTALL_PREFIX is changed, and you should be able to find out where the path gets screwed up.
Also, you don't have to wait for the full build to complete to tell if the prefix was set
correctly. After configure runs, take a look in 'src/corelib/global/qconfig.cpp' and see what 'qt_configure_prefix_path_str' is set to.
You can manually set these properties using
qmake -set VARIABLE VALUE
They are stored using QSettings, the Qt built-in persistent applications settings.
see Configuring qmake's Environment
Configure scripts can be fuzzy about slashes. Are you sure that the build prefix is /usr and not /usr/ .

Resources