How to set LD_LIBRARY_PATH from a CMake toolchain file? - linux

On Linux, I want to create a CMake toolchain file for cross-compilation.
The compiler needs some shared libraries that are located in non-standard directories, so I have to set LD_LIBRARY_PATH before invoking it. That worked when calling the compiler from the command line, but not when calling it from CMake.
I tried to set LD_LIBRARY_PATH via set(ENV{LD_LIBRARY_PATH} "${CMAKE_CURRENT_LIST_DIR}/<shared library directory>") from the toolchain file. However the compiler complained that it couldn't find the shared libraries.

Table of contents:
Setting it in a toolchain file isn't going to do what you want.
I'm not sure why setting LD_LIBRARY_PATH before invoking the buildsystem isn't working for you.
There's a more idiomatic way in CMake to do what you want.
Setting it in a toolchain file isn't going to do what you want
I'm pretty sure the approach you are asking how to take won't work because set(ENV) just sets an environment variable that will only be known to CMake at the configure stage (not the build stage). Here's a minimal reproducible example of that:
CMakeLists.txt:
cmake_minimum_required(VERSION 3.23)
project(Foo)
set(ENV{FOO} "hello world!")
message("\$ENV{FOO} at configure time: $ENV{FOO}")
add_custom_target(echo
"${CMAKE_COMMAND}" -P "${CMAKE_CURRENT_SOURCE_DIR}/echo.cmake"
VERBATIM
)
echo.cmake:
cmake_minimum_required(VERSION 3.23)
message("\$ENV{FOO} at build time: $ENV{FOO}")
run the following:
$ cmake -S . -B build --fresh
<...>
$ENV{FOO} at configure time: hello world!
<...>
$ cmake --build build/ --target echo
$ENV{FOO} at build time:
Since environment variables are "passed downward" to child processes and not upward, when you do set(ENV), that's just setting it in the CMake process that performs the configure step (the one invoked by the cmake -S ... command in the example above). As the example shows, CMake doesn't do anything fancy to make those environment variables known at its configure time to the generated buildsystem at build time.
I'm not sure why setting LD_LIBRARY_PATH before invoking the buildsystem isn't working for you
So I have to set LD_LIBRARY_PATH before invoking it. It works when calling the compiler from the command line, but not from CMake.
As long as you're exporting the variable for it to be made available to the shell's child processes (using the correct mechanism for whichever shell you're using), that should work. A minimal reproducible example would help here.
There's a more idiomatic way in CMake to do what you want
Try using the find_library() command. It does what its name says it does at configuration time (instead of build time). If you use it, you'll also get the benefit of your buildsystems being more cross-platform.
It has several ways of tweaking how it searches for libraries. You can read about exactly how it works in the official docs.
For your case here, one of the suitable configuration points to guide the library search would be the CMAKE_LIBRARY_PATH variable, although as you'll read about, there are other configuration points you could use as well.

Related

ctest doesn't seem to use the settings in CMAKE_TOOLCHAIN_FILE

After making my application work on Linux, I'm trying cross-compile it for Windows and MacOSX. I already use CMake.
I began by creating a toolchain file. This works. My linux program is compiled with mingw and I receive a .exe at the end.
# build linux
$ cmake -Bbuild-linux
$ cmake --build build-linux
# build windows
$ cmake -Bbuild-windows -DCMAKE_TOOLCHAIN_FILE=`pwd`/cmake/x86_64-w64-mingw32.cmake
$ cmake --build build-windows
What happens next, is I run ctest to execute the unit tests. In Linux, this works fine. But when I do this using the cross-compiled stuff, it can't find the .exe file.
# run tests linux
(cd build-linux; ctest)
# run tests windows
(cd build-windows; ctest)
No problem, I thought -- I would append the .exe suffix depending on the environment -- using if(WIN32)
# CMakeLists.txt
if(WIN32)
SET(EXE_SUFFIX, ".exe")
endif()
ADD_TEST(NAME test_myapp
COMMAND ${CMAKE_CURRENT_BINARY_DIR}/bin/myapp${EXE_SUFFIX} ${CMAKE_CURRENT_SOURCE_DIR}/test_myapp.txt
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
But WIN32 isn't available for some reason and it never appends the suffix. Why isn't WIN32 available? Is the problem that ctest doesn't know about the settings from the toolchain file? Does it not realize that WIN32 was declared?
I am able to use if(WIN32) elsewhere in my CMakeLists.txt file so long as cmake.exe is doing something with those lines, not ctest.exe.
In summary, WIN32 is not set when ctest runs.
Is there a way to execute unit tests without involving ctest? If it can't be trusted to do this right, maybe I don't use it anymore.
This line is totally incorrect:
SET(EXE_SUFFIX, ".exe")
You have set the variable EXE_SUFFIX,, not EXE_SUFFIX. When you later expand ${EXE_SUFFIX}, it comes back empty, so that entire if (WIN32) block is a no-op from the perspective of the rest of the program.

Add linker flag during conan install

I'm working in a project that uses a number of external libraries. These libraries are included using Conan. The project is primarily written for Unix, but it also need to compile and run on Windows.
My current problem is that Windows defaults fopen() to be O_TEXT, while Unix expects this to be O_BINARY. I have a fix that works for my own code, simply include binmode.obj when linking to change this default to O_BINARY.
The problem is that this does not affect my third party libraries. Googling for this didn't turn up much, most suggestions seems to be based on where you are creating your own package and want flags added, rather than how to add flags when using other's packages.
What I have tried so far:
Make binmode.obj come before libraries, in case the linking order matters. Made no difference.
Added --env 'CL=link binmode.obj' to conan install, but this flag did not end up as part of the compile flags nor link flags.
Any suggestions for what I could try?
EDIT: I was wrong about "CL" taking no effect. This was caused by confusing output. But I did observe that CL seems to be applied for both compiler and linker, which makes it somewhat challenging what flags to give. Using "/link" prefix makes it work with compiler, but does not work with linker.
EDIT 2: More confusions... I didn't realize that the syntax of the CL value was: "<compile flags> /link <link flags>". It affected compile, but not link, however. So this environment variable apparently can't be used to make Conan add a linker flag for autotools based packages.
Hi Mats L welcome to our community!
I once had a similar problem and what I end up doing was quite hacky but quite simple also:
On my conan profile located at ~/.conan/profiles/default or any other profile actually I added an enviromental variable as such:
CXX=/usr/bin/clang++ -fms-compatibility. This one made me compile all the c++ sources with this flag (that can understand windows specific code).
So in your case you can run which c++ to find the location of your compiler
and edit the CXX environmental variable in the conan profile you use your final file will probably look like :
[settings]
os=Macos
os_build=Macos
arch=x86_64
arch_build=x86_64
compiler=clang
compiler.version=11
compiler.libcxx=libc++
build_type=Release
[options]
[build_requires]
[env]
CXX=c++ --required_flag
Some additional notes: You might also want this flag set on your CC enviromental variable .
It's preferable to not change the default profile but copy it (lets say on a file named default_edited_copy) and then invoke the other profile with conan ... --profile default_edited_copy

rust, WebAssembly, and passing arguments for increased total memory

I have a rust project I am compiling to webasm per http://asquera.de/blog/2017-04-10/the-path-to-rust-on-the-web/
Project compiles. When I run it in Chrome Canary, it runs out of memory and gives me a very helpful error message:
abort("Cannot enlarge memory arrays. Either (1) compile with -s
TOTAL_MEMORY=X with X higher than the current value 16777216, (2) compile
with -s ALLOW_MEMORY_GROWTH=1 which allows increasing the size at runtime,
...
Problem is, its not clear how to pass those flags to rustc / the build tool chain.
Neither setting EMMAKEN_CFLAGS or the following work:
cargo rustc -v --target=wasm32-unknown-emscripten --release -- -Clink-args="-s TOTAL_MEMORY=33554432"
This blog post offers a solution that I think can be applied in your case too:
As best as I can tell there is no way to pass most linker arguments through cargo. Instead, hack around the limitation by specifying a custom linker that is actually a shell script wrapping the real linker.
Create a shell script like emcc_link that calls emscripten with the appropriate options:
emcc "-s" "TOTAL_MEMORY=33554432" $#
(You may need other options to make it work. Check the blog post for details.)
And specify to use it for your project by editing/creating .cargo/config:
[target.wasm32-unknown-emscripten]
linker = "/your/project/dir/emcc_sdl"
[target.asmjs-unknown-emscripten]
linker = "/your/project/dir/emcc_sdl"
I ruthlessly assumed the build environment is Linux or the like. On Windows the shell script should probably be a batch script and I'm not sure if there are any differences in .cargo/config.
Disclaimer: I have not tried any of this.

How to compile vim 64-bit on windows using MinGW-64?

I tried to compile vim 64-bit on windows. But I don't know how to use MinGW-64. There's a mingw-32-make in the 32-bit version, which I could use to build. But I didn't find any 'make' program in the 64-bit MinGW. Could you please tell me how to use mingw-64, or any tutorial I could follow?
Thank you.
It does not matter from which source make program comes, it only just must be able to execute the Makefile. To compile vim with MinGW with specific compiler and Make_ming.mak makefile I used to use the following:
Export environment variable CC set to the appropriate compiler (in my case it was 32-bit named i686-pc-mingw32-gcc).
Export environment variable LD set to the appropriate linker (in my case it was similar, but with -ld suffix in place of -gcc). Be sure they are found on $PATH: I am not sure what kind of escaping you should do to make makefile work so just avoid the necessity for escaping.
Export environment variable prefix pointing to the directory where mingw resides (in my case it was /usr/i686-mingw32: I am cross-compiling).
Export environment variable vim_cv_toupper_broken set to yes. I am not sure why I did this.
Finally run make:
cd {path/to/vim/repository}/src
make -f Make_ming.mak FEATURES=HUGE CROSS_COMPILE=i686-pc-mingw32- OPTIMIZE=SPEED VIMRUNTIMEDIR="C:\\vim73\\runtime" CROSS=yes ARCH=i686
. You definitely do not need CROSS_COMPILE and CROSS options and ARCH should be probably omitted (or equal to x86_64). VIMRUNTIMEDIR should point to the place where you plan to install vim. Not sure about escaping though.
Exporting environment variables should be probably done with
set var=value
, e.g.
set CC=x86_64-w64-mingw32-gcc
(use actual name of the executable). If this does not work try moving them to the make command line:
make -f Make_ming.mak CC=x86_64-w64-mingw32-gcc LD=… …
.
And variables for python (should also be present on the command-line):
PYTHON="P:\\ath\\to\\directory\\with\\python" PYTHONINC="P:\\ath\\to\\directory\\with\\python\\header\\files" PYTHON_VER=27 PYTHON_VER_LONG=2.7.5
. (If using python msi installer PYTHONINC is %PYTHON%\\include. It is 90% some directory whose trailing path component is include. Should contain at least Python.h file.)
I just compiled VIM on MinGW and made a gist about it. I tried x86-64 (search for it), too, and with /etc/fstab changed to 64 it basically worked, just that my interpreters all were 32 and so it stopped there.
Try to set
ARCH=x86-64 in vim74/src/Make_ming.mak
and add option CC=x86_64-w64-mingw32-gcc, maybe it will be useful.

F#, Linux and makefiles

I intend to distribute an F# program as both binary and source so the user has the option of recompiling it if desired. On Windows, I understand how to do this: provide .fsproj and .sln files, which both Visual Studio and MSBuild can understand.
On Linux, the traditional solution for C programs is a makefile. This depends on gcc being directly available, which it always is.
The F# compiler can be installed on Linux and works under Mono, so that's fine so far. However, as far as I can tell, it doesn't create a scenario where fsc runs the compiler, instead the command is mono ...path.../fsc.exe. This is also fine, except I don't know what the path is going to be. So the full command to run the compiler in my case could be mono ~/FSharp-2.0.0.0/bin/fsc.exe types.fs tptp.fs main.fs -r FSharp.PowerPack.dll except that I'm not sure where fsc.exe will actually be located on the user's machine.
Is there a way to find that out within a makefile, or would it be better to fall back on just explaining the above in the documentation and relying on the user to modify the command according to his setup?
If you don't want to use autoconf just write up README and say us how to setup tools to compile your program.
For example, you can require as to use binfmt_misc kernel module to allow system to automatically use right starter program for files with known format as to $PATH must contain path to fsc.exe, so your Makefile simply will be like following code:
FILES=types.fs tptp.fs main.fs
target.exe: ${FILES}
fsc.exe -o $# ${FILES} -r FSharp.PowerPack.dll
Or you can allow user to point to compiler by using makefile variables:
MONO=/usr/bin/mono
FSC=/usr/local/fsharp/bin/fsc.exe
COMPILER=${MONO} ${FSC}
FILES=types.fs tptp.fs main.fs
target.exe: ${FILES}
${COMPILER} -o $# ${FILES} -r FSharp.PowerPack.dll

Resources