I want to create a program and compile it to WASM. The program runs on Windows and Unix.
The program generates a relative path for a filename from the given base_path and converts it to posix:
use pathdiff::diff_paths;
let relative_filename = diff_paths(filename, base_path).unwrap();
let path_parts = relative_filename.components().map(|component| component.as_os_str().to_str().unwrap()).collect::<Vec<&str>>();
let relative_posix_path = path_parts.join("/");
I thought this would allow me to generate a consistent relative path for both systems for example:
/// e.g. "/foo/", "/bar/baz.txt" -> "../bar/baz.txt"
/// e.g. "C:\foo\", "C:\foo\baz.txt" -> "../bar/baz.txt"
Unfortunately when compiled to WASM it seems not to be able to parse Windows file paths anymore.
Is there a cross platform way to handle paths in rust?
Related
I created a library that in order to work needs to parse a txt file every time you call the main method. The problem is that when importing into another project the txt file can not be found because I'm using env::current_dir(), when I call the method from the library folder current folder is the crate's root, and I can access root/src/my_file.txt. When importing and using the library the root is different and there isn't any my_file.txt.
How can resolve this? Here is the Crate
Here is how I access the file.
fn parse(&mut self, name_to_find: &str) -> () {
let p = env::current_dir().unwrap();
println!("{}", p.display());
let file = File::open(format!("{}/src/nam_dict.txt", p.display())).unwrap();
let lines = BufReader::new(file).lines();
...
Rust is a compiled language, so one can't assume access to the sourcecode and adjacent files at runtime.
So your options are
Include it at compile time with the include_str! macro. This means changes to the file won't be picked up until the library and its dependents are rebuilt
Locate the file at runtime, e.g. through a specified location (such as somewhere in the config hierarchy in the user's home directory), an environment variable, a commandline option or from the current directory. This way the file can be changed without recompiling the program but the user has to know that he must provide it.
More complicated approaches such as including a default configuration and letting the user override it are also possible.
I am trying to make a python module in rust. I am continuing to fail to get the files that I need to generate. I followed this tutorial almost exactly.
https://mycognosist.github.io/tutorial-rust-python-lib.html
Here is my toml file.
name = "pylib"
version = "0.1.0"
authors = ["Atops"]
edition = "2018"
[lib]
name = "status"
crate-type = ["cdylib"]
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies.cpython]
version = "0.5"
features = ["extension-module"]
Here is the code for lib.rs
extern crate cpython;
use cpython::{PyResult, Python, py_fn, py_module_initializer};
pub extern fn hello(_py: Python, val: String) -> PyResult<String> {
match &*val {
"hello" => Ok("world".to_string()),
_ => Ok("afdfs".to_string()),
}
}
py_module_initializer!(status, initstatus, Pyinit_status, |py, m|{
m.add(py, "__doc__", "asdhgdfs")?;
m.add(py, "hello", py_fn!(py, hello(val: String)))?;
Ok(())
});
I navigate to the appropriate folder with my cargo.toml file and use cargo build --release, as specified in the tutorial. It finishes with a few warnings about certain things being "not FFI-safe". When it finishes a new folder is created called "target". I go through there to where the files for the library should be, but the appropriate files don't seem to have been generated. I have these.
target\release
These don't seem to be usable for me. I am not sure what I have done wrong.
I copied every single one of these files to another folder and tried importing them in python. This did not work. I tried changing them to .so files, this did not work. It seems that windows is supposed to output a dll here, but the dll file did not work when attempting to import. It did not work as a dll file or a so file. I am not sure what to do here. Also, it seems that when these files are generated in every other tutorial or guide I see, there is a "lib" prefix on the name of each. I get no such prefix.
A dll file contains shared object code for COFF systems (like Windows) while a so file generally contains shared object code for ELF (and other) systems like Linux.
Renaming them will not work under any circumstance. If you have one and need the other, you need to rebuild them on the correct, matching, operating system.
I think you might be following instructions for Linux (due to the lib prefixes) on a Windows machine and then expecting some of the instructions to work. They might work, if you can identify all the "changes" between the platform and modify the Linux instructions to match the Windows platform; but until you know how to translate one environment's instructions to the other, it might be easier if you just find a set of Windows instructions that only refer to dll files and don't mention so files.
My package has a binary target that uses include_bytes!(…) to bundle a copy of some precomputed values into the compiled binary. This is an optimization, but isn't strictly necessary: the program is capable of calculating these values at run time if the bundled data slice .is_empty().
The program needs to be able to build without this data. However, include_bytes!("data/computed.bin") causes a build error if the target file does not exist.
error: couldn't read src/data/computed.bin: No such file or directory (os error 2)
Currently, I have a Bash build script that uses touch data/computed.bin to ensure the file exists before building. However, I don't want to depend on platform-specific solutions like Bash; I want to be able to build this project on any supported platform using cargo build.
How can my Rust program include_bytes!(…) or include_str!(…) from a file if it exits, but gracefully fall back to an alternative value or behaviour if the file doesn't exist, while only using the standard Cargo build tools?
We can use a build script to ensure that the included file exists before out package tries to include it. However, build scripts can only write to the current build's unique output directory, so we can't just create the missing input files in the source directory directly.
error: failed to verify package tarball
Caused by:
Source directory was modified by build.rs during cargo publish. Build scripts should not modify anything outside of OUT_DIR.
Instead, our build script can create the file-to-include in the build directory, copying the source data if it exists, and we can update our package code to include this data from the build directory instead of from the source directory. The build path will be available in the OUT_DIR environment variable during the build, so we can access it from std::env::var("OUT_DIR") in our build script and from env!("OUT_DIR") in the rest of our package.
//! build.rs
use std::{fs, io};
fn main() {
let out_dir = std::env::var("OUT_DIR").unwrap();
fs::create_dir_all(&format!("{}/src/data", out_dir))
.expect("unable to create data directory");
let path = format!("src/data/computed.bin", name);
let out_path = format!("{}/{}", out_dir, path);
let mut out_file = fs::OpenOptions::new()
.append(true)
.create(true)
.open(&out_path)
.expect("unable to open/create data file");
if let Ok(mut source_file) = fs::File::open(&path) {
io::copy(&mut source_file, &mut out_file).expect("failed to copy data after opening");
}
}
//! src/foo.rs
fn precomputed_data() -> Option<&'static [u8]> {
let data = include_bytes!(concat!(env!("OUT_DIR"), "/src/data/computed.bin")).as_ref();
if !data.is_empty() {
Some(data)
} else {
None
}
}
While using a build script (like in this answer) would work, I am not a fan of that solution:
The build script copies the file – depending on the file size, that might be prohibitively expensive. Though one could probably solve this problem using hardlinks instead.
An empty file might be perfectly fine data – the solution would misdetect an empty file as missing. However, depending on the use case, an empty file might actually be perfectly valid.
It is very verbose – this turns a simple include_bytes! into a build script of approximately 20 lines and additionally a few more lines when including to handle the data.is_empty() case.
It is hard to grasp what is happening here for a casual reader – why is this script including something from $OUT_DIR? It would probably take a moment for the reader to get the idea a build script might be involved here.
It does not scale well – most of those problems would get even worse if there were multiple files that needed to be included optionally.
I therefore decided to write the procedural macro crate include_optional to solve this problem (currently only works on nightly Rust because it depends on some unstable features).
With this, the solution to this problem is a one liner:
use include_optional::include_bytes_optional;
fn precomputed_data() -> Option<&'static [u8]> {
include_bytes_optional!("./computed.bin")
}
There are also macros wrapping include_str! and include!.
I'm writing a Cargo helper command that needs to know the default target triple used by Rust/Cargo (which I presume is the same as host's target triple). Ideally it should be a compile-time constant.
There's ARCH constant, but it's not a full triple. For example, it doesn't distinguish between soft float and hard float ARM ABIs.
env!("TARGET") would be ideal, but it's set only for build scripts, and not the lib/bin targets. I could pass it on to the lib with build.rs and dynamic source code generation (writing the value to an .rs file in OUT_DIR), but it seems like a heavy hack just to get one string that the compiler has to know anyway.
Is there a more straightforward way to get the current target triple in lib/bin target built with Cargo?
Build scripts print some additional output to a file so you can not be sure that build script only printed output of $TARGET.
Instead, try something like this in build.rs:
fn main() {
println!(
"cargo:rustc-env=TARGET={}",
std::env::var("TARGET").unwrap()
);
}
This will fetch the value of the $TARGET environment variable in the build script and set it so it will be accessible when the program is started.
In my main.rs:
const TARGET: &str = env!("TARGET");
Now I can access the target triplet in my program. If you are using this technique, you'll only read the value of theTARGET environment variable and nothing else.
I don't think this is exposed other than through a build script. A concise way to get the target triple without "dynamic source code generation" would be, in build.rs:
fn main() {
print!("{}", std::env::var("TARGET").unwrap());
}
and in src/main.rs:
const TARGET: &str = include_str!(concat!(env!("OUT_DIR"), "/../output"));
fn main() {
println!("target = {:?}", TARGET);
}
If you just want to know which target is installed, run: rustup target list --installed
This question already has answers here:
Is there a Linux equivalent of Windows' "resource files"?
(2 answers)
Closed 4 years ago.
I'm looking for a way to embed text files in my binaries (like windows resource system). I need something thats also platform independent (works in windows and linux). I found Qt resource management to be what I need but I'm not keen on my app depending on Qt for this alone. I also found this tool at http://www.taniwha.com/~paul/res/ .. but it is too platform specific.
The xxd utility can be used to create a C source file, containing your binary blobs as an array (with the -i command line option). You can compile that to an object which is linked into your executable.
xxd should be portable to most platforms.
If you're using QT 4.5, you can make sure that program is only dependent on one small piece of QT, such as libqtcore. QResource is a part of libqtcore.
You can simlpy append all kinds of data to your normal binary. Works in both Windows and Linux. You'll have to open your own binary at runtime and read the data from there.
However, I have to agree that embedding data in binaries is a strange idea. It's common practice to include such data as separate files packaged with the application.
That is not such a great idea. On Linux, for example, data is expected to be installed in a subdirectory of "$datadir" which is, by default, defined to be "$prefix/share", where "$prefix" is the installation prefix. On Mac OS X, resources are expected to be installed in $appbundle/Contents/Resources, where $appbundle is the name of the folder ending in ".app". On Windows, installing data in a folder that is a sibling of the executable is not an uncommon practice. You may be better off using the CMake build system, and using its CPack packaging features for installing/bundling in the default, preferred platform-specific manner.
Although bundling your resources into the executable, itself, may seem cool, it is actually a dangerous idea... for example, will the embedded data be allocated in an executable page? What will happen if you attempt to overwrite or modify the data? What if you want to tweak or modify the data at runtime? Things to think about.
This looks very promising: https://github.com/cyrilcode/embed-resource
CMake based and platform-independent.
As I also do not like the idea of converting files into C arrays only to have them converted back to binaries, I created my own resource compiler using LLVM and Clang:
https://github.com/nohajc/resman
I tested it on Windows, Linux and macOS but it can potentially be run on any platform supported by LLVM.
It is used like this:
Create header file, e.g. res_list.h
#pragma once
#include "resman.h"
// Define a global variable for each file
// It will be used to refer to the resource
constexpr resman::Resource<1> gRes1("resource_file1.jpg"); // resource with ID 1
constexpr resman::Resource<2> gRes2("resource_file2.txt"); // resource with ID 2
constexpr resman::Resource<3> gRes3("resource_file3.mp3"); // resource with ID 3
...
Run resource compiler
$ rescomp res_list.h -o res_bundle.o
Link res_bundle.o to your project
Use the resource files
#include "res_list.h"
...
resman::ResourceHandle handle{gRes1};
// ResourceHandle provides convenient interface to do things like:
// iterate over bytes
for (char c : handle) { ... }
// convert bytes to string
std::string str{handle.begin(), handle.end()};
// query size and id
unsigned size = handle.size();
unsigned id = handle.id();
The resource compiler parses res_list.h (using Clang) but instead of generating cpp files, it goes straight to the native object file (or static library) format (using LLVM).