Cargo fails to build when invoked from inside a python script - rust

I have a python script which will symlink my binary into /usr/bin, and I wanted to add the functionality of building the binary if it is not already built.
# Check if binary exists
if path.exists(src_bin):
# If not, build it
run('cargo build --release', shell=True, check=True)
# Check if symlink already exists
if path.exists(dest_bin):
# Remove it, in case it is outdated
path.remove(dest_bin)
# Symlink binary to /usr/bin
os.symlink(src_bin, dest_bin)
But the cargo command (which works perfectly fine if I run it in the terminal) errors out with this:
no override and no default toolchain set; run 'rustup default stable' to set the stable toolchain as default
My best guess is that cargo relies on some environment variables to tell it which toolchain to use which are not present in the python environment, but I don't know what variables those might be. I also tried
run(['cargo', 'build', '--release'], check=True)
but the same thing happened.
What is causing this, and is there a way around it?
Edit more info:
$ ls ~/.cargo/bin
# no output
$ ls ~/.rustup/toolchains
stable-x86_64-unknown-linux-gnu
$ which cargo
/usr/bin/cargo
I tried
run('rustup run stable cargo build --release', shell=True, check=True)
and got
error: toolchain 'stable-x86_64-unknown-linux-gnu' is not installed
when I run cargo build --release or rustup run stable cargo build --release in the shell, both build sucessfully.

Related

Why is "anchor build" and "Cargo build-bpf" showing wrong rustc version?

I'm trying to build the (https://github.com/betterclever/solend-anchor) package using anchor build, however I get the following error
error: package `uint v0.9.3` cannot be built because it requires rustc 1.56.1 or newer, while the currently active rustc version is 1.56.0-dev
I updated rustc and running the command rustup default nightlyshows:
info: using existing install for 'nightly-x86_64-apple-darwin'
info: default toolchain set to 'nightly-x86_64-apple-darwin'
nightly-x86_64-apple-darwin unchanged - rustc 1.61.0-nightly
So it shows me that the installed and active rustc version is 1.61, however anchor build is not finding that for some reason. I also tried running cargo build-bpf but the same thing kept happening. cargo buildseemed to work fine.
I'm wondering what is causing the problem when running anchor build and cargo build-bpf?
anchor build and cargo build-bpf use a different compiler than the normal rustc compiler included in the system, so it's normal that they report a different version. The BPF compiler comes with the Solana tool suite.
If you already have the Solana tools installed on your computer, you can simply run:
solana-install init 1.9.13
And if you don't, you can run:
sh -c "$(curl -sSfL https://release.solana.com/v1.9.13/install)"
That will give you all of the newest tools, including the BPF compiler.
I have Solana latest version, I faced the same error while compiling one of the downloaded program.
Command
solana-install update
Worked for me.

ctest doesn't seem to use the settings in CMAKE_TOOLCHAIN_FILE

After making my application work on Linux, I'm trying cross-compile it for Windows and MacOSX. I already use CMake.
I began by creating a toolchain file. This works. My linux program is compiled with mingw and I receive a .exe at the end.
# build linux
$ cmake -Bbuild-linux
$ cmake --build build-linux
# build windows
$ cmake -Bbuild-windows -DCMAKE_TOOLCHAIN_FILE=`pwd`/cmake/x86_64-w64-mingw32.cmake
$ cmake --build build-windows
What happens next, is I run ctest to execute the unit tests. In Linux, this works fine. But when I do this using the cross-compiled stuff, it can't find the .exe file.
# run tests linux
(cd build-linux; ctest)
# run tests windows
(cd build-windows; ctest)
No problem, I thought -- I would append the .exe suffix depending on the environment -- using if(WIN32)
# CMakeLists.txt
if(WIN32)
SET(EXE_SUFFIX, ".exe")
endif()
ADD_TEST(NAME test_myapp
COMMAND ${CMAKE_CURRENT_BINARY_DIR}/bin/myapp${EXE_SUFFIX} ${CMAKE_CURRENT_SOURCE_DIR}/test_myapp.txt
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
But WIN32 isn't available for some reason and it never appends the suffix. Why isn't WIN32 available? Is the problem that ctest doesn't know about the settings from the toolchain file? Does it not realize that WIN32 was declared?
I am able to use if(WIN32) elsewhere in my CMakeLists.txt file so long as cmake.exe is doing something with those lines, not ctest.exe.
In summary, WIN32 is not set when ctest runs.
Is there a way to execute unit tests without involving ctest? If it can't be trusted to do this right, maybe I don't use it anymore.
This line is totally incorrect:
SET(EXE_SUFFIX, ".exe")
You have set the variable EXE_SUFFIX,, not EXE_SUFFIX. When you later expand ${EXE_SUFFIX}, it comes back empty, so that entire if (WIN32) block is a no-op from the perspective of the rest of the program.

Cargo recompiles package in workspace with "-p" after compiling with "--workspace"

In general, I compile everything in my workspace with: cargo build from the workspace directory, or with cargo build --workspace
However, if I then compile a specific package with cargo build -p package or by running cargo build from that package's directory, it will recompile it, when actually it should be completely cached, right?
It seems that the things being rerun are mostly related to macros... recompiling syn, serde_derive, etc...
Is it somehow expected that these macro crates need to be recompiled because they might give different results? I feel like that really shouldn't be the case if we want reproducible builds!
What can I do to prevent cargo from recompiling like this?
(this is with cargo 1.51.0 on a raspberry pi 4 running Raspbian 10)

How can I know in advance where cabal will place a generated executable?

When I run cabal build for my project, it places my compiled executable in
dist-newstyle/build/$PLATFORM/$GHC/$PACKAGE-$VERSION/x/$PACKAGE/build/$PACKAGE/$PACKAGE
I'm using a Makefile to run cabal build as needed when I need to run the executable.
Since I'm working on different platforms at various times, I need a way to tell my Makefile what to expect the $PLATFORM value to be.
There's a couple ways I could go about this:
run cabal build once unconditionally, and then use wildcards:
BIN := $(wildcard dist-newstyle/build/*/ghc-*/foo-*/x/foo/build/foo/foo)
use uname to predict what the platform will be
BIN := dist-newstyle/build/$(shell uname -m)-$(shell uname -s | some magic)/ghc-$(shell ghc --version | some magic)/foo-$(VERSION)/x/foo/build/foo/foo
give up on making sure the binary is up to date within the Makefile and just run cabal run unconditionally
Any of those would work, but I would like to know if I could just ask cabal to tell me where it expects to place an executable.
Is there a cabal command I can run to tell me what $PLATFORM and/or $GHC values it detects?
Say you just ran cabal build and produced a foo binary somewhere in the dist-newstyle/ directory. You can now run:
cabal exec which foo
This will print out the absolute path to the foo executable.
Note: It may be necessary to run cabal exec --verbose=0 which foo so that cabal does not print out extra information about packages or configuration.
Why does this work?
cabal exec lets you run commands with the same environment used when running other cabal commands, like cabal build. So you can run printenv and cabal exec printenv to see how it adds and changes some environment variables. Notice that it adds the whole dist-newstyle/ path to your PATH variable!
So saying cabal exec which foo works because it is just like running which foo, but with the extended PATH variable.
This means you can extract the dist-newstyle/ path itself using a fancier command:
cabal exec --verbose=0 printenv PATH | tr ':' '\n' | grep dist-newstyle | head -1
This may be useful if you need to know the path before you run cabal build, in which case the binary will not exist yet.
cabal exec which $EXENAME$ will give you the full path of the compiled executable.
Since cabal 3.8, you can use the list-bin command:
$ cabal list-bin my-executable-name
That will print the path at which cabal will place the executable if built. Note that it doesn't actually guarantee that the executable is built, so depending on your use case you might want to check that too.

Cross-compile a Rust application from Linux to Windows

Basically I'm trying to compile the simplest code to Windows while I am developing on Linux.
fn main() {
println!("Hello, and bye.")
}
I found these commands by searching the internet:
rustc --target=i686-w64-mingw32-gcc main.rs
rustc --target=i686_pc_windows_gnu -C linker=i686-w64-mingw32-gcc main.rs
Sadly, none of them work. It gives me an error about the std crate missing
$ rustc --target=i686_pc_windows_gnu -C linker=i686-w64-mingw32-gcc main.rs
main.rs:1:1: 1:1 error: can't find crate for `std`
main.rs:1 fn main() {
^
error: aborting due to previous error
Is there a way to compile code on Linux that will run on Windows?
Other answers, while technically correct, are more difficult than they need to be. There's no need to use rustc (in fact it's discouraged, just use cargo), you only need rustup, cargo and your distribution's mingw-w64.
Add the target (you can also change this for whatever target you're cross compiling for):
rustup target add x86_64-pc-windows-gnu
You can build your crate easily with:
cargo build --target x86_64-pc-windows-gnu
No need for messing around with ~/.cargo/config or anything else.
EDIT: Just wanted to add that while you can use the above it can also sometimes be a headache. I wanted to add that the rust tools team also maintains a project called cross: https://github.com/rust-embedded/cross
This might be another solution that you want to look into
The Rust distribution only provides compiled libraries for the host system. However, according to Arch Linux's wiki page on Rust, you could copy the compiled libraries from the Windows packages in the download directory (note that there are i686 and x86-64 packages) in the appropriate place on your system (in /usr/lib/rustlib or /usr/local/lib/rustlib, depending on where Rust is installed), install mingw-w64-gcc and Wine and you should be able to cross-compile.
If you're using Cargo, you can tell Cargo where to look for ar and the linker by adding this to ~/.cargo/config (where $ARCH is the architecture you use):
[target.$ARCH-pc-windows-gnu]
linker = "/usr/bin/$ARCH-w64-mingw32-gcc"
ar = "/usr/$ARCH-w64-mingw32/bin/ar"
Note: the exact paths can vary based on your distribution. Check the list of files for the mingw-w64 package(s) (GCC and binutils) in your distribution.
Then you can use Cargo like this:
$ # Build
$ cargo build --release --target "$ARCH-pc-windows-gnu"
$ # Run unit tests under wine
$ cargo test --target "$ARCH-pc-windows-gnu"
UPDATE 2019-06-11
This fails for me with:
Running `rustc --crate-name animation examples/animation.rs --color always --crate-type bin --emit=dep-info,link -C debuginfo=2 --cfg 'feature="default"' -C metadata=006e668c6384c29b -C extra-filename=-006e668c6384c29b --out-dir /home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/examples --target x86_64-pc-windows-gnu -C ar=x86_64-w64-mingw32-gcc-ar -C linker=x86_64-w64-mingw32-gcc -C incremental=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/incremental -L dependency=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps -L dependency=/home/roman/projects/rust-sdl2/target/debug/deps --extern bitflags=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/libbitflags-2c7b3e3d10e1e0dd.rlib --extern lazy_static=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/liblazy_static-a80335916d5ac241.rlib --extern libc=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/liblibc-387157ce7a56c1ec.rlib --extern num=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/libnum-18ac2d75a7462b42.rlib --extern rand=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/librand-7cf254de4aeeab70.rlib --extern sdl2=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/libsdl2-3f37ebe30a087396.rlib --extern sdl2_sys=/home/roman/projects/rust-sdl2/target/x86_64-pc-windows-gnu/debug/deps/libsdl2_sys-3edefe52781ad7ef.rlib -L native=/home/roman/.cargo/registry/src/github.com-1ecc6299db9ec823/winapi-x86_64-pc-windows-gnu-0.4.0/lib`
error: linking with `x86_64-w64-mingw32-gcc` failed: exit code: 1
Maybe this will help https://github.com/rust-lang/rust/issues/44787
Static compile sdl2
There is option to static-compile sdl but it didn't work for me.
Also mixer is not included when used with bundled.
Let's cross-compile examples from rust-sdl2 project from Ubuntu to Windows x86_64
In ~/.cargo/config
[target.x86_64-pc-windows-gnu]
linker = "x86_64-w64-mingw32-gcc"
ar = "x86_64-w64-mingw32-gcc-ar"
Then run this:
sudo apt-get install gcc-mingw-w64-x86-64 -y
# use rustup to add target https://github.com/rust-lang/rustup.rs#cross-compilation
rustup target add x86_64-pc-windows-gnu
# Based on instructions from https://github.com/AngryLawyer/rust-sdl2/
# First we need sdl2 libs
# links to packages https://www.libsdl.org/download-2.0.php
sudo apt-get install libsdl2-dev -y
curl -s https://www.libsdl.org/release/SDL2-devel-2.0.9-mingw.tar.gz | tar xvz -C /tmp
# Prepare files for building
mkdir -p ~/projects
cd ~/projects
git clone https://github.com/Rust-SDL2/rust-sdl2
cd rust-sdl2
cp -r /tmp/SDL2-2.0.9/x86_64-w64-mingw32/lib/* ~/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-pc-windows-gnu/lib/
cp /tmp/SDL2-2.0.9/x86_64-w64-mingw32/bin/SDL2.dll .
Build examples at once
cargo build --target=x86_64-pc-windows-gnu --verbose --examples
Or stop after first fail:
echo; for i in examples/*; do [ $? -eq 0 ] && cargo build --target=x86_64-pc-windows-gnu --verbose --example $(basename $i .rs); done
Run
cargo build will put binaries in target/x86_64-pc-windows-gnu/debug/examples/
Copy needed files:
cp /tmp/SDL2-2.0.4/x86_64-w64-mingw32/bin/SDL2.dll target/x86_64-pc-windows-gnu/debug/examples/
cp assets/sine.wav target/x86_64-pc-windows-gnu/debug/examples/
Then copy directory target/x86_64-pc-windows-gnu/debug/examples/ to your Windows machine and run exe files.
Run in cmd.exe
If you want to see the console output when running exe files, you may run them from cmd.exe.
To open cmd.exe in current directory in file explorer, right click with shift on empty place in window and choose Open command window here.
Backtraces with mingw should work now - if not use msvc https://github.com/rust-lang/rust/pull/39234
There is Docker based solution called cross. All the required tools are in virtualized environment so you don't need to install additional packages for your machine. See Supported targets list.
From project's README:
Features
cross will provide all the ingredients needed for cross compilation without touching your system installation.
cross provides an environment, cross toolchain and cross compiled libraries, that produces the most portable binaries.
“cross testing”, cross can test crates for architectures other than i686 and x86_64.
The stable, beta and nightly channels are supported.
Dependencies
rustup
A Linux kernel with binfmt_misc support is required for cross testing.
One of these container engines is required. If both are installed, cross will default to docker.
Docker. Note that on Linux non-sudo users need to be in the docker group. Read the official post-installation steps. Requires version 1.24 or later.
Podman. Requires version 1.6.3 or later.
Installation
$ cargo install cross
Usage
cross has the exact same CLI as Cargo but as it relies on Docker you'll have to start the daemon before you can use it.
# (ONCE PER BOOT)
# Start the Docker daemon, if it's not already running
$ sudo systemctl start docker
# MAGIC! This Just Works
$ cross build --target aarch64-unknown-linux-gnu
# EVEN MORE MAGICAL! This also Just Works
$ cross test --target mips64-unknown-linux-gnuabi64
# Obviously, this also Just Works
$ cross rustc --target powerpc-unknown-linux-gnu --release -- -C lto
The solution that worked for me was. It is similar to one of the accepted answers but I did not require to add the toolchain.
rustup target add x86_64-pc-windows-gnu
cargo build --target x86_64-pc-windows-gnu
Refer to the documentation for more details.
I've had success on Debian (testing) without using Mingw and Wine just following the official instructions. They look scary, but in the end it didn't hurt that much.
The official instructions also contain info on how to cross-compile C/C++ code. I haven't needed that, so it's something I haven't actually tested.
A couple of remarks for individual points in the official instructions. The numbers match the numbers in the official instructions.
Debian: sudo apt-get install lld
Make a symlink named lld-link to lld somewhere in your $PATH. Example: ln -s /usr/bin/lld local_bin/lld-link
I don't cross-compile C/C++, haven't used this point personally.
This is probably the most annoying part. I installed Rust on a Windows box via rustup, and copied the libraries from the directories named in the official docs to the Linux box. Beware, there were sometimes uppercase library filenames, but lld wants them all lowercase (Windows isn't case-sensitive, Linux is). I've used the following to rename all files in current directory to lowercase:
for f in `find`; do mv -v "$f" "`echo $f | tr '[A-Z]' '[a-z]'`"; done
Personally, I've needed both Kit directories and just one of the VC dirs.
I don't cross-compile C/C++, haven't used this point personally.
Just make $LIB_ROOT in the script at the end of this post point to the lib directory from point 3.
Mandatory
I don't cross-compile C/C++, haven't used this point personally.
Depending the target architecture, either of the following:
rustup target add i686-pc-windows-msvc
rustup target add x86_64-pc-windows-msvc
For cross-building itself, I'm using the following simple script (32-bit version):
#!/bin/sh
# "cargo build" for the 32-bit Windows MSVC architecture.
# Set this to proper directory
LIB_ROOT=~/opt/rust-msvc
# The rest shouldn't need modifications
VS_LIBS="$LIB_ROOT/Microsoft Visual Studio 14.0/VC/lib/"
KIT_8_1_LIBS="$LIB_ROOT/Windows Kits/8.1/Lib/winv6.3/um/x86/"
KIT_10_LIBS="$LIB_ROOT/Windows Kits/10/Lib/10.0.10240.0/ucrt/x86/"
export LIB="$VS_LIBS;$KIT_8_1_LIBS;$KIT_10_LIBS"
cargo build --target=i686-pc-windows-msvc "$#"
I'm using the script the same way I would use cargo build
Hope that helps somebody!

Resources