Can I use Cargo to build Rust code without using its standard project layout?
My source files are not in a directory called src and this will not change. My binaries must end up in the current directory (or, in some other projects, in a different directory that is not called target/SOMETHING). Can I tell Cargo that executable foo must be built from foo.rs and bar.rs in the same directory as Cargo.toml, and qux from foo.rs?
I don't care about Cargo as a build system or as a deployment system. I'm only interested in it as a library management system. Apparently Cargo is the only game in the Rust town for this.
Not really. You can control where the source files are by explicitly specifying them in the manifest:
[[bin]]
name = "foo"
src = "foo.rs"
[[bin]]
name = "qux"
src = "splong.rs"
I don't know what you mean by foo being built from foo.rs and bar.rs, whilst qux is built only from foo.rs. You can't just arbitrarily glob source files together: either foo.rs uses bar.rs, or it doesn't.
But you can't control the target directory from within the manifest. There's the build.target-dir setting (in Cargo's configuration, not the manifest), but that only lets you change the target directory, not the second level inside of it. You can change it using the CARGO_TARGET_DIR environment variable, though you can't set environment variables from within the manifest, either.
That said, setting CARGO_TARGET_DIR to the root of the project will also change where all the intermediate files go, and it'll mean every time you switch between debug and release builds, you'll have to do a full recompile.
You may want to consider opening an issue on the Cargo issue tracker about making this an option.
Related
While I am learning rust, I have a crude means of building a non-rust c/c++ library that lives in a submodule. My build script (build.rs) now looks like:
use std::process::Command;
fn main() {
// EXTERN_C
// Build the c-library
Command::new("make").args(&["-C", "cadd"]).status().unwrap();
// The name of the library to link to, i.e. like: -l<lib>
println!("cargo:rustc-link-lib=dylib=add_x64Linuxd");
// The library search path for linking, i.e. like -L<path>
println!("cargo:rustc-link-search=native=cadd/lib");
// The run-time library search path (LD_LIBRARY_PATH)
println!("cargo:rustc-env=LD_LIBRARY_PATH=cadd/lib");
}
This is working nicely, the makefile within cadd/ sorts out any build/re-build deps etc. The only thing I can't do at the moment is hook in make -C cadd clean when I run cargo clean. Ideally I would like it to run the clean make target at the same time. The command would look like:
Command::new("make").args(&["-C", "cadd", "clean"]).status().unwrap();
But I don't know how to get such a command to run during cargo clean. Is there a "clean script" like there is a "build script" - or another method?
Eventually I will get around to learning how to wrap up my makefile project into a cargo crate (I think that's the right terminology) - so I know this is not the optimal way to do this, but I want to get this working in a basic way first (so my head does not explode!).
The cargo clean command simply deletes the cargo target directory.
One solution is for your Makefile to output its compilation artifacts(all files it generates) into the target directory.
You could also change the directory cargo outputs its artifacts to, either via the --target-dir CLI option or by adding the following to the .cargo/config:
[build]
target-dir = "some/path"
I'm porting an SDK written in C++ from Windows to Linux. There are other binaries, but at its simplest, our SDK is this:
core.dll - implicitly loaded DLL ("libcore.so" shared library on Linux)
tests.exe - an app use to test the DLL (uses google test)
All of my binaries must live in one folder somewhere that apps can find. I've achieved that on Windows. I wanted to achieve the same thing in Linux. I'm failing miserably
To illustrate, Here's the basic project tree. We use CMake. After I build I've got
mysdk
|---CMakeLists.txt (has add_subdirectory() statements for "tests" and "core")
|---/tests (source code + CMakeLists.txt)
|---/core (source code + CMakeLists.txt)
|---/build (all build ouput, CMake output, etc)
|---tests (build output)
|---core (build output)
The goal is to "flatten" the "build" tree and put all the binary outputs of tests, core, etc into one folder.
I tried adding CMake's "install" command, to each of my CMakeLists.txt files (e.g. install(TARGETS core DESTINATION bin). I then then executed sudo make install after my normal build. This put all my binaries in /usr/local/bin with no errors. But when I ran tests from there, it failed to find libcore.so, even though it was sitting right there in the same folder
tests: error while loading shared libraries: libcore.so: Cannot open shared object file: No such file or directory
I read up on the LD_LIBRARY_PATH environment variable and so tried adding that folder (/usr/local/bin) into it and running. I can see I've properly altered LD_LIBRARY_PATH but it still doesn't work. "tests" still can't find libcore.so. I even tried changing the PATH environment variable as well. Same result.
In frustration, I tried brute-force copying the output binaries to a temporary subfolder (of /mysdk/build) and running tests from there. To my surprise it ran.
Then I realized why: Instead of loading the local copy of libcore.so it had loaded the one from the build output folder (as if the full path were "baked in" to the app at build time). Subsequently deleting that build-output copy of libcore.so made "tests" fail altogether as before, instead of loading the local copy. So maybe the path really was baked in.
I'm at a loss. I've read the CMake tutorial and reference. It makes this sound so easy. Aside from the obvious (What am I doing wrong?) I would appreciate if anyone could answer any of the following questions:
What is the correct way to control where my app looks for my shared libraries?
Is there a relationship between my project build structure and how my binaries must then appear when installed?
Am I even close to the right way of doing this?
Is it possible I've somehow inadvertently "baked" (into my app) full paths to my shared libraries? Is that a thing? I use all CMAKE variables in my CMakeLists files.
You can run ldd file to print the shared object dependencies for file. It will tell you where are its dependencies being read from.
You can export the environment variable LD_LIBRARY_PATH with the paths you want the linker to look for. If a dependency is not found, try adding the path where that dependency is located at to LD_LIBRARY_PATH and then run ldd again (make sure you export the variable).
Also, make sure the dependencies have the right permissions.
Updating LD_LIBRARY_PATH is an option. Another option is using RPATH. Please check the example.
https://github.com/mustafagonul/cmake-examples/blob/master/005-executable-with-shared-library/CMakeLists.txt
cmake_minimum_required(VERSION 2.8)
# Project
project(005-executable-with-shared-library)
# Directories
set(example_BIN_DIR bin)
set(example_INC_DIR include)
set(example_LIB_DIR lib)
set(example_SRC_DIR src)
# Library files
set(library_SOURCES ${example_SRC_DIR}/library.cpp)
set(library_HEADERS ${example_INC_DIR}/library.h)
set(executable_SOURCES ${example_SRC_DIR}/main.cpp)
# Setting RPATH
# See https://cmake.org/Wiki/CMake_RPATH_handling
set(CMAKE_INSTALL_RPATH ${CMAKE_INSTALL_PREFIX}/${example_LIB_DIR})
# Add library to project
add_library(library SHARED ${library_SOURCES})
# Include directories
target_include_directories(library PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/${example_INC_DIR})
# Add executable to project
add_executable(executable ${executable_SOURCES})
# Linking
target_link_libraries(executable PRIVATE library)
# Install
install(TARGETS executable DESTINATION ${example_BIN_DIR})
install(TARGETS library DESTINATION ${example_LIB_DIR})
install(FILES ${library_HEADERS} DESTINATION ${example_INC_DIR})
I have SCons code in which I am using SConscripts to build different directories separately. In my Src directory, my SConscript builds a shared library, and then returns the resulting Node as the Python variable libMyLibrary. I typically use the install option to copy this library to a directory that is on my system's LD_LIBRARY_PATH (I'm using OpenSUSE).
So far, so good. Now, in another directory, Src/Test, another SConscript imports libMyLibrary and builds some Programs using code like this:
env.Program('myProgram', 'myProgram.cpp', LIBS=[env['LIBS'], libMyLibrary])
The program then gets installed to my local bin folder. This code does track the library dependency and build the program, but the problem is that since the library is in a sub-directory (Src), that sub-directory gets included in the linker command. Here is an abbreviated example of the linker command that SCons generates:
g++ -o Src/Test/myProgram Src/Test/myProgram.o Src/libMyLibrary.so
I believe this happens because the Node,libMyLibrary, is essentially a path. The problem is that when I try to run the program, it is not looking for libMyLibrary.so in my library folder, but rather Src/libMyLibrary.so, and of course it doesn't find it.
I do NOT want the libraries I build to be installed in sub-directories of my install folder.
I already add the Src folder to LIBPATH, so SCons adds the -LSrc option to the linker command, but that doesn't solve the problem. My preference would be that when I add a Node, the path should automatically get parsed out to add the appropriate -L and -l options.
I know that I can get around this problem by adding the string 'MyLibrary' to the LIBS variable instead of the libMyLibrary Node, but then I have to explicitly tell SCons that each Program Depends() on libMyLibrary. It seems very inefficient to short-circuit SCons's built-in dependency tracking this way. Does anyone know the correct, SCons-y way to do this?
I'm referring to your latest comment: It looks to me as if this is not really a SCons problem, but more a general linker question (XY problem). Are you perhaps simply searching for RPATH? Please also check this old SO question: scons executable + shared library in project directory
I have an app with the following (I would have thought quite common) directory hierarchy:
/src
subdir1/ # Subdirs with more source files.
more.c
SConscript
foo.c # Source files.
foo.h
SConscript
/other # Other top-level directories with no source code.
/stuff # However, there are other assets I may want to build.
README # Other top-level files.
SConstruct
The problem is that when I run scons from the top-level directory, it calls gcc from that directory without cding into src, like this:
gcc -o src/foo.o src/foo.c
This is problematic for several reasons:
Within my program, I #include files giving the path relative to the src directory. For example, more.c could include foo.h with #include "foo.h". This fails because GCC is run from the parent directory. I don't want to change my includes to #include "src/foo.h".
I use the __FILE__ special macro for things like logging. When built from the top-level directory, GCC puts "src/" at the front of all filenames, since that was the path it was given to compile. This may seem picky, but I don't want that, because I think of my source tree as being relative to the src directory.
(Edit: I should add that obviously #1 can be fixed by adding -Isrc as a flag to GCC, but this seems like more hacks around the main issue.)
How can I make SCons cd into the src directory before calling gcc?
I don't want to get rid of the src directory and move everything up, because there are lots of other (non-code) files at the top level.
I don't want SCons to cd into every subdirectory. It should just cd into src and then build all files in the hierarchy from there.
I could solve this by moving SConscript inside the src directory and running it from there, perhaps using a Makefile at the top level. But this seems quite hacky, and I also do want to use SCons to build (non-code) assets in other directories than src.
I've read that you can make a custom Builder and make it change directories. However, I do not want to write a whole new Builder for C/C++. Is there a way to modify the behaviour of an existing builder without writing one from scratch? Also, on one forum, someone said that changing directories from within a Builder would break parallel builds since it would change the directory that other tasks are building from. Is this true?
This behavior is just how SCons works, and its not possible to avoid/change. I've been looking for some supporting documentation in its favor, and havent found any. Its just something Ive become used to.
As you mention, the include paths are simple to fix. The harder part is the __FILE__ macro. I hadnt ever noticed this until you mentioned it. Unfortunately, I think the only way around this is to strip the path in the logger, which is a rather ugly fix.
You can add src to your env['CPPPATH'] which will fix the include paths.
env.Append(CPPPATH=[Dir('src')])
However, this doesn't solve the problem with __FILE__.
I'm trying to modify my SCons files so that they put the generated files into a build directory. Initially I though VariantDir could be an option but judging from all I read and the examples it does not do what I want.
Is there any easy way to force SCons to put the output in a certain directory without having to rewrite all the sources and scripts?
After struggling with VariantDir for a while (it wasn't doing anything at all), I ended up using variant_dir parameter in the top level SConscript call, which causes all downstream build outputs end up in a parallel 'build' tree:
SConscript(['subdirs/SConscript'], variant_dir='build', duplicate=0)
My build structure is a hierarchy of SConscripts in subdirs/sub-subdirs, etc. With this call the outputs end up in build/sub-subdirs at the same level as they would in the source.
This eats up one level, though (subdirs), and using "../build" does not help. The solution is to have a SConscript file at the same level as SConstruct and call SConscript(['SConscript'], variant_dir='build', duplicate=0)
See also Force Scons output (exe, obj, lib & dll) to specific build directory - it has a similar answer
Using VariantDir with duplicate=0 should work.
Facing similar frustration, I added a site_scons that added replacement builders (e.g. "Exe" instead of "Program") and specified an emitter for that builder that replaced the path portion with the build directory. This requires the use of the alternate builder throughout your SConscripts though.
Alternatively you could try to subclass Environment and rewrite the main targets to use target rewrites. Then you specify your Environment as the default (modifying Scons.Script.DefaultEnvironment or something like that). This approach kept the SConscripts static but got very messy and requires more maintenance over time as scons internals change.
You might use Install or InstallAs on target output. It works for me.
lib = env.SharedLibrary(target = "some_target", source = sources);
env.InstallAs( target = "folder/output_name.ext", source = lib );