include_str for null terminated string - rust

I need to read a file into a null terminated string at compile time.
Working in Rust OpenGL. I have a shader source code stored in a separate file. The function that will eventually read the source is gl::ShaderSource from the gl crate. All it needs is a pointer to a null terminated string (the std::ffi::CStr type).
Typically the guides I have seen read the shader source file using include_str!, then at run time allocate a whole new buffer of length +1, then copy the original source into the new buffer and put the terminating 0 at the end. I'd like to avoid all that redundant allocating and copying and just have the correctly null terminated string at compile time.
I realize it is somewhat petty to want to avoid an extra allocation for a short shader file, but the principle could apply to many other types of larger constants.
While scrolling through suggested questions during the preview I saw this: How do I expose a compile time generated static C string through FFI?
which led me to this solution:
let bytes1 = concat!(include_str!("triangle.vertex_shader"), "\0");
let bytes2 = bytes1.as_bytes();
let bytes3 = unsafe {
CStr::from_bytes_with_nul_unchecked(bytes2)
};
println!("{:?}", bytes3);
Does this accomplish avoiding the runtime allocation and copying?

Your code is unsound. It fails to verify there are no interior NUL bytes.
You can use the following function to validate the string (at compile time, with no runtime cost):
pub const fn to_cstr(s: &str) -> &CStr {
let bytes = s.as_bytes();
let mut i = 0;
while i < (bytes.len() - 1) {
assert!(bytes[i] != b'\0', "interior byte cannot be NUL");
i += 1;
}
assert!(bytes[bytes.len() - 1] == b'\0', "last byte must be NUL");
// SAFETY: We verified there are no interior NULs and that the string ends with NUL.
unsafe { CStr::from_bytes_with_nul_unchecked(bytes) }
}
Wrap it in a little nice macro:
macro_rules! include_cstr {
( $path:literal $(,)? ) => {{
// Use a constant to force the verification to run at compile time.
const VALUE: &'static ::core::ffi::CStr = $crate::to_cstr(concat!(include_str!($path), "\0"));
VALUE
}};
}
Then use it:
let bytes = include_cstr!("triangle.vertex_shader");
If there are interior NUL bytes the code will fail to compile.
When CStr::from_bytes_with_nul() becomes const-stable, you will be able to replace to_cstr() with it.

Yes that should work as intended. If you want you can even bundle it into a simple macro.
macro_rules! include_cstr {
($file:expr) => {{
// Create as explicit constant to force from_bytes_with_nul_unchecked to
// perform compile time saftey checks.
const CSTR: &'static ::std::ffi::CStr = unsafe {
let input = concat!($file, "\0");
::std::ffi::CStr::from_bytes_with_nul_unchecked(input.as_bytes())
};
CSTR
}};
}
const VERTEX_SHADER: &'static CStr = include_cstr!("shaders/vert.glsl");
const FRAGMENT_SHADER: &'static CStr = include_cstr!("shaders/frag.glsl");

Related

Is there a way to make a macro replace things in strings?

This macro should be able to replace entries in a string via an argument. For example, this would work:
let string = "Hello, world!";
replace_macro!(string, "world", "Rust"); // Hello, Rust!
I'm not sure how to do this, as all my previous attempts of just writing a regular function and calling that don't work inside macros. If possible, I'd like to be using macro_rules as opposed to a proc macro.
It is not possible. Macros cannot inspect and/or change the value of variables.
It is possible if the literal is embedded in the call (replace_macro!("Hello, world!", "world", "Rust");) but requires a proc-macro: macro_rules! macros cannot inspect and/or change literals.
It's a rather simple with a proc macro:
use quote::ToTokens;
use syn::parse::Parser;
use syn::spanned::Spanned;
type Args = syn::punctuated::Punctuated<syn::LitStr, syn::Token![,]>;
#[proc_macro]
pub fn replace_macro(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
let input_span = input.span();
let args = match Args::parse_terminated.parse(input) {
Ok(args) => Vec::from_iter(args),
Err(err) => return err.into_compile_error().into(),
};
let (original, text, replacement) = match args.as_slice() {
[original, text, replacement] => (original.value(), text.value(), replacement.value()),
_ => {
return syn::Error::new(
input_span,
r#"expected `"<original>", "<text>", "<replacement>"`"#,
)
.into_compile_error()
.into()
}
};
original
.replace(&text, &replacement)
.into_token_stream()
.into()
}
It parses a list of three string literals, punctated by commas, then calls str::replace() to do the real work.

Initialize Gstreamer's PadProbeId to a default value in Rust

I am familiar with Gstreamer but new to Rust,
TLDR; I want to be able to initialize PadProbeId to a default value before using it.
The details:
I have a Bin (containing audio + video encoders and hlssink).
I have been able to add this bin to the pipeline and it works fine.
The issue I have is the audio for the stream is optional and I want to do add_probe() only when audio is available. Below is a simplified version fo what I tried to implement
let mut audio_probe_id: PadProbeId;
let mut tee_audio_pad: Pad;
if media_info.audio_available {
// get encoded audio from the tee
tee_audio_pad = audio_tee.request_pad_simple("src_%u").unwrap();
audio_probe_id = tee_audio_pad.add_probe(gst::PadProbeType::BLOCK_DOWNSTREAM, |_pad, _info| {
gst::PadProbeReturn::Ok
}).unwrap();
// link the audio_tee.src to enc_bin ghost pad
let audio_sink_pad = enc_bin.static_pad("audio").unwrap();
tee_audio_pad.link(&audio_sink_pad).unwrap();
}
enc_bin.call_async(move |bin| {
bin.sync_state_with_parent().unwrap();
if media_info.audio_available {
tee_audio_pad.remove_probe(audio_probe_id);
}
}
However because of Rust compilers restriction to using uninitialized variables, it does not let me use audio_probe_id without initializing.
I tried to initialize it like this; let mut audio_probe_id: PadProbeId = PadProbeId(NonZeroU64(u64::MAX));. However compiler complains that it is a private field.
error[E0423]: cannot initialize a tuple struct which contains private fields
Thanks a lot for your help!
The rust way to have empty variables like this is to use Option, but in your case it would simpler to have a single conditional:
if media_info.audio_available {
// get encoded audio from the tee
let tee_audio_pad = audio_tee.request_pad_simple("src_%u").unwrap();
let audio_probe_id = tee_audio_pad.add_probe(gst::PadProbeType::BLOCK_DOWNSTREAM, |_pad, _info| {
gst::PadProbeReturn::Ok
}).unwrap();
// link the audio_tee.src to enc_bin ghost pad
let audio_sink_pad = enc_bin.static_pad("audio").unwrap();
tee_audio_pad.link(&audio_sink_pad).unwrap();
enc_bin.call_async(move |bin| {
bin.sync_state_with_parent().unwrap();
tee_audio_pad.remove_probe(audio_probe_id);
}
} else {
enc_bin.call_async(move |bin| {
bin.sync_state_with_parent().unwrap();
});
}

How to use mail filter context data?

I am trying to write a mail filter in Rust using the milter crate. I built the example on a Linux VM and it all works fine. However, the example is using u32 as the type of context injected into their handlers, a quite simple example. I instead need to store a string from the handle_header callback through to the handle_eom handler so I can use an incoming header to set the envelope from.
If I log the value of the header in handle_header to console, it writes correctly but by the time it arrives in handle_eom, it has been corrupted/overwritten whatever. I thought that context was supposed to be specifically for this scenario but it seems weird that it uses type inference rather than e.g. a pointer to an object that you can just assign whatever you want to it.
Is my understanding of context wrong or is the code incorrect?
I tried using value and &value in handle_header and it behaves the same way.
use milter::*;
fn main() {
Milter::new("inet:3000#localhost")
.name("BounceRewriteFilter")
.on_header(header_callback)
.on_eom(eom_callback)
.on_abort(abort_callback)
.actions(Actions::ADD_HEADER | Actions::REPLACE_SENDER)
.run()
.expect("milter execution failed");
}
#[on_header(header_callback)]
fn handle_header<'a>(mut context: Context<&'a str>, header: &str, value: &'a str) -> milter::Result<Status> {
if header == "Set-Return-Path" {
match context.data.borrow_mut() {
Some(retpath) => *retpath = &value,
None => {
context.data.replace(value)?;
}
}
}
Ok(Status::Continue)
}
#[on_eom(eom_callback)]
fn handle_eom(mut context: Context<&str>) -> milter::Result<Status> {
match context.data.take() {
Ok(result) => {
println!("Set-return-path header is {}", result.unwrap());
context.api.replace_sender(result.unwrap(), None::<&str>)?;
}
Err(_error) => {}
}
Ok(Status::Continue)
}
Thanks to glts on Github, the author of the crate, the problem was that the string slices passed into the handle_header method were not borrowed by the external code that stores the data pointer so by the time that handle_eom is called, the memory has been reused for something else.
All I had to do was change Context<&str> to Context<String> and convert the strings using mystr.to_owned() and in the reverse direction val = &*mystring

How can i convert this match statement into a HashMap?

How could I convert this match block into a HashMap? Ideally I would like to create a HashMap from a manifest file:
match layers[1][p] {
',' => tile = grass,
'*' => tile = sand,
// And so on....
}
I tried doing this earlier, but Macroquad's Texture2D is not compatible with a HashMap. (I think the compiler said 'Cannot use Option as Texture2D'.)
I would like to do something like this:
let tile = hashmap.get(layers[p]);
blit(tile);
Let's keep this simple. I don't think hashmap will simplify anything but why not demonstrate it. The true advantage of hash is that you can load tiles dynamically and make them loaded into the map. For now, though, we will hardcode some.
fn main() {
/* snip */
let mut textures = HashMap::new();
textures.insert(',', grass);
textures.insert('*', sand);
/* snip */
// mind that your program will crash if map does
// not contain the texture
let texture = textures.get(layers[p]).unwrap();
blit(texture)
}

Rust Ownership and Lifetimes using Combine

I've read the docs on ownership and lifetimes and I think I understand them but I'm having trouble with a specific bit of code.
I have a struct called Branch like this:
struct Branch {
slot: u8,
yaw: u8,
pitch: u8,
length: u8
}
I'm using the combine library (it's a parser combinator) to parse a string into a Branch. The parsers look like this:
let hex_re = Regex:new(r"[0-9a-fA-F]").unwrap();
let hex = || find(&hex_re).map(|s| u8::from_str_radix(s, 16));
let branch = |length: u8| {
(hex(), hex(), hex())
.map( |t| (t.0.unwrap(), t.1.unwrap(), t.2.unwrap()) )
.map( |(slot,yaw,pitch)| Branch { slot, yaw, pitch, length } )
}
The parsers are fairly simple, the first hex takes a regular-expression which matches a single hexidecimal character and maps it into a u8. The second branch maps 3 hex characters into a Branch e.g. 3d2.
The problem arises when I call the parser branch(1).parse("3d2"), the compiler reports an error 'length' does not live long enough. I think I understand this error, if I'm not mistaken it's because length goes out of scope when the closure completes and so the length variable is deallocated even though it is still being used by the newly created Branch.
So, I tried to get around this by converting length: u8 to length: &u8 like so:
let branch = |len: &u8| {
(hex(), hex(), hex())
.map( |t| (t.0.unwrap(), t.1.unwrap(), t.2.unwrap()) )
.map( |(slot,yaw,pitch)| Branch { slot, yaw, pitch, length: *len } )
};
// calling the parser
branch(&(1 as u8)).parse("3d2");
But that results in this error:
type of expression contains references that are not valid during the expression: `combine::combinator::Map<combine::combinator::Map<(combine::combinator::Map<combine::regex::Find<&regex::Regex, &str>, [closure#src\lsystem.rs:26:37: 26:70]>, combine::combinator::Map<combine::regex::Find<&regex::Regex, &str>, [closure#src\lsystem.rs:26:37: 26:70]>, combine::combinator::Map<combine::regex::Find<&regex::Regex, &str>, [closure#src\lsystem.rs:26:37: 26:70]>, combine::combinator::Map<combine::regex::Find<&regex::Regex, &str>, [closure#src\lsystem.rs:26:37: 26:70]>), [closure#src\lsystem.rs:30:19: 30:65]>, [closure#src\lsystem.rs:31:19: 31:80 length:&&u8]>`
I have no idea what this error is about. Any help would be much appreciated.
This solved it:
let hex_re = Regex:new(r"[0-9a-fA-F]").unwrap();
let hex = || find(&hex_re).map(|s| u8::from_str_radix(s, 16));
let branch = |length: u8| {
(hex(), hex(), hex())
.map( |t| (t.0.unwrap(), t.1.unwrap(), t.2.unwrap()) )
.map( move |(slot,yaw,pitch)| Branch { slot, yaw, pitch, length } )
}
placing move on the second .map works.

Resources