I'm trying to figure out how to access the EnumValueOption len associated with each member of Foo:
// proto/demo.proto:
syntax = "proto3";
import "google/protobuf/descriptor.proto";
package demo;
extend google.protobuf.EnumValueOptions {
optional uint32 len = 50000;
}
enum Foo {
None = 0 [(len) = 10];
One = 1 [(len) = 20];
Two = 2 [(len) = 30];
}
I think I should be able to do this through prost_types::FileDescriptorSet by collecting the reflection information at build time:
// build.rs:
use std::path::PathBuf;
fn main() {
let out_dir =
PathBuf::from(std::env::var("OUT_DIR").expect("OUT_DIR environment variable not set."));
prost_build::Config::new()
.file_descriptor_set_path(out_dir.join("file_descriptor_set.bin"))
.compile_protos(&["proto/demo.proto"], &["proto"])
.unwrap_or_else(|e| panic!("Failed to compile protos {:?}", e));
}
But I can't figure out how to actually retrieve the len field:
// src/lib.rs
use prost::Message;
use prost_types::FileDescriptorSet;
use prost_types::EnumValueOptions;
include!(concat!(env!("OUT_DIR"), concat!("/demo.rs")));
pub fn parse_file_descriptor_set() -> FileDescriptorSet {
let file_descriptor_set_bytes =
include_bytes!(concat!(env!("OUT_DIR"), "/file_descriptor_set.bin"));
prost_types::FileDescriptorSet::decode(&file_descriptor_set_bytes[..]).unwrap()
}
#[cfg(test)]
mod tests {
use super::*;
fn get_len(foo: Foo) -> u32 {
let set = parse_file_descriptor_set();
for f in &set.file {
for ext in &f.extension {
dbg!(ext);
}
for e in &f.enum_type {
dbg!(e);
for v in &e.value {
dbg!(v);
}
}
}
todo!()
}
#[test]
fn test_get_len() {
assert_eq!(get_len(Foo::One), 20);
}
}
Am I on the right track? Is this something that's even supported? I'm using prost, prost-types, and prost-build version 0.9.
Related
The below WAT, adapted from a couple of Wasmtime examples, runs absolutely fine embedded in my application but what I thought was the equivalent Rust code fails with:
Running `target\debug\hello_world.exe`
Error: expected 5 imports, found 1
error: process didn't exit successfully: `target\debug\hello_world.exe` (exit code: 1)
Here's the working WAT:
(module
(import "host" "greet" (func $greet (param i32 i32)))
(func (export "run")
i32.const 4 ;; ptr
i32.const 22 ;; len
call $greet)
(memory (export "memory") 1)
(data (i32.const 4) "Calling back from WAT!")
)
And the failing Rust:
use std::ffi::CString;
#[link(wasm_import_module = "host")]
extern "C" {
fn greet(ptr: i32, len: i32);
}
static GREETING: &str = "Calling back from Rust!";
#[no_mangle]
pub extern "C" fn run() {
let greeting = CString::new(GREETING).expect("contains null byte");
let ptr = greeting.as_ptr();
std::mem::forget(ptr);
unsafe {
greet(ptr as i32, GREETING.len() as i32);
}
}
My minimal example app that embeds the WASM modules:
use std::str;
use anyhow::Result;
use wasmtime::{Caller, Engine, Extern, Func, Instance, Module, Store};
struct MyState {
name: String,
count: usize,
}
fn main() -> Result<()> {
let engine = Engine::default();
let code = include_bytes!("../hello.wat");
let module = Module::new(&engine, code)?;
let mut store = Store::new(
&engine,
MyState {
name: "Hello, junglie85!".to_string(),
count: 0,
},
);
let greet_func = Func::wrap(
&mut store,
|mut caller: Caller<'_, MyState>, ptr: i32, len: i32| {
let mem = match caller.get_export("memory") {
Some(Extern::Memory(mem)) => mem,
_ => anyhow::bail!("failed to find host memory"),
};
let data = mem
.data(&caller)
.get(ptr as u32 as usize..)
.and_then(|arr| arr.get(..len as u32 as usize));
let string = match data {
Some(data) => match str::from_utf8(data) {
Ok(s) => s,
Err(_) => anyhow::bail!("invalid utf-8"),
},
None => anyhow::bail!("pointer/length out of bounds"),
};
println!("> {} {}", caller.data().name, string);
caller.data_mut().count += 1;
Ok(())
},
);
let imports = [greet_func.into()];
let instance = Instance::new(&mut store, &module, &imports)?;
let run = instance.get_typed_func::<(), ()>(&mut store, "run")?;
println!("# Global count = {}", store.data().count);
run.call(&mut store, ())?;
println!("# Global count = {}", store.data().count);
Ok(())
}
What is the Rust equivalent of the WAT example?
We all know that using global variables can lead to subtle bugs. I need to migrate Python programs to Rust, keeping the algorithm intact as far as possible. Once I have demonstrated Python-Rust equivalence there will be opportunities to debug and change the logic to fit Rust better. Here is a simple Python program using global variables, followed by my unsuccessful Rust version.
# global variable
a = 15
# function to perform addition
def add():
global a
a += 100
# function to perform subtraction
def subtract():
global a
a -= 100
# Using a global through functions
print("Initial value of a = ", a)
add()
print("a after addition = ", a)
subtract()
print("a after subtraction = ", a)
Here is a Rust program that runs, but I cannot get the closures to update the so-called global variable.
fn fmain() {
// global variable
let mut a = 15;
// perform addition
let add = || {
let mut _name = a;
// name += 100; // the program won't compile if this is uncommented
};
call_once(add);
// perform subtraction
let subtract = || {
let mut _name = a;
// name -= 100; // the program won't compile if this is uncommented
};
call_once(subtract);
// Using a global through functions
println!("Initial value of a = {}", a);
add();
println!("a after addition = {}", a);
subtract();
println!("a after subtraction = {}", a);
}
fn main() {
fmain();
}
fn call_once<F>(f: F)
where
F: FnOnce(),
{
f();
}
My request: Re-create the Python logic in Rust.
Your Rust code is not using global variables, the a variable is stack-allocated. While Rust doesn't particularly endorse global variables, you can certainly use them. Translated to Rust that uses actual globals, your program would look like this:
use lazy_static::lazy_static;
use parking_lot::Mutex; // or std::sync::Mutex
// global variable
lazy_static! {
static ref A: Mutex<u32> = Mutex::new(15);
}
// function to perform addition
fn add() {
*A.lock() += 100;
}
// function to perform subtraction
fn subtract() {
*A.lock() -= 100;
}
fn main() {
// Using a global through functions
println!("Initial value of a = {}", A.lock());
add();
println!("a after addition = {}", A.lock());
subtract();
println!("a after subtraction = {}", A.lock());
}
Playground
If you prefer to use closures, you can do that too, but you'll need to use interior mutability to allow multiple closures to capture the same environment. For example, you could use a Cell:
use std::cell::Cell;
fn main() {
let a = Cell::new(15);
let add = || {
a.set(a.get() + 100);
};
let subtract = || {
a.set(a.get() - 100);
};
// Using a global through functions
println!("Initial value of a = {}", a.get());
add();
println!("a after addition = {}", a.get());
subtract();
println!("a after subtraction = {}", a.get());
}
Playground
Dependency-less examples as enum and function. EDIT : Code improved, as suggested in comment and corrected match arm.
use std::sync::{Arc, Mutex, Once};
static START: Once = Once::new();
static mut ARCMUT: Vec<Arc<Mutex<i32>>> = Vec::new();
// as enum
enum Operation {
Add,
Subtract,
}
impl Operation {
// static change
fn result(self) -> i32 {
let mut arc_clone = unsafe { ARCMUT[0].clone() };
let mut unlock = arc_clone.lock().unwrap();
match self {
Operation::Add => *unlock += 100,
Operation::Subtract => *unlock -= 100,
}
*unlock
}
// dynamic change
fn amount(self, amount: i32) -> i32 {
let mut arc_clone = unsafe { ARCMUT[0].clone() };
let mut unlock = arc_clone.lock().unwrap();
match self {
Operation::Add => *unlock += amount,
Operation::Subtract => *unlock -= amount,
}
*unlock
}
}
// as a function
fn add() -> i32 {
let mut arc_clone = unsafe { ARCMUT[0].clone() };
let mut unlcok = arc_clone.lock().unwrap();
*unlcok += 100;
*unlcok
}
// as trait
trait OperationTrait {
fn add(self) -> Self;
fn subtract(self) -> Self;
fn return_value(self) ->i32;
}
impl OperationTrait for i32 {
fn add(mut self) -> Self {
let arc_clone = unsafe{ARCMUT[0].clone()};
let mut unlock = arc_clone.lock().unwrap();
*unlock += self;
self
}
fn subtract(mut self) -> Self {
let arc_clone = unsafe{ARCMUT[0].clone()};
let mut unlock = arc_clone.lock().unwrap();
*unlock -= self;
self
}
fn return_value(self)->Self{
let arc_clone = unsafe{ARCMUT[0].clone()};
let mut unlock = arc_clone.lock().unwrap();
*unlock
}
}
// fn main
fn main() {
START.call_once(|| unsafe {
ARCMUT = vec![Arc::new(Mutex::new(15))];
});
let test = Operation::Add.result();
println!("{:?}", test);
let test = Operation::Subtract.amount(100);
println!("{:?}", test);
let test = add();
println!("{:?}", test);
let test = 4000.add();
println!("{:?}", test);
}
I am trying to write some code that could be able to write a method that returns me a Vec with the names of the fields of a struct.
Code snippet below:
# Don't forget about dependencies if you try to reproduce this on local
use proc_macro2::{Span, Ident};
use quote::quote;
use syn::{
punctuated::Punctuated, token::Comma, Attribute, DeriveInput, Fields, Meta, NestedMeta,
Variant, Visibility,
};
#[proc_macro_derive(StructFieldNames, attributes(struct_field_names))]
pub fn derive_field_names(input: proc_macro::TokenStream) -> proc_macro::TokenStream {
let ast: DeriveInput = syn::parse(input).unwrap();
let (vis, ty, generics) = (&ast.vis, &ast.ident, &ast.generics);
let names_struct_ident = Ident::new(&(ty.to_string() + "FieldStaticStr"), Span::call_site());
let fields = filter_fields(match ast.data {
syn::Data::Struct(ref s) => &s.fields,
_ => panic!("FieldNames can only be derived for structs"),
});
let names_struct_fields = fields.iter().map(|(vis, ident)| {
quote! {
#vis #ident: &'static str
}
});
let mut vec_fields: Vec<String> = Vec::new();
let names_const_fields = fields.iter().map(|(_vis, ident)| {
let ident_name = ident.to_string();
vec_fields.push(ident_name);
quote! {
#vis #ident: -
}
});
let names_const_fields_as_vec = fields.iter().map(|(_vis, ident)| {
let ident_name = ident.to_string();
// vec_fields.push(ident_name)
});
let (impl_generics, ty_generics, where_clause) = generics.split_for_impl();
let tokens = quote! {
#[derive(Debug)]
#vis struct #names_struct_ident {
#(#names_struct_fields),*
}
impl #impl_generics #ty #ty_generics
#where_clause
{
#vis fn get_field_names() -> &'static str {
// stringify!(
[ #(#vec_fields),* ]
.map( |s| s.to_string())
.collect()
// )
}
}
};
tokens.into()
}
fn filter_fields(fields: &Fields) -> Vec<(Visibility, Ident)> {
fields
.iter()
.filter_map(|field| {
if field
.attrs
.iter()
.find(|attr| has_skip_attr(attr, "struct_field_names"))
.is_none()
&& field.ident.is_some()
{
let field_vis = field.vis.clone();
let field_ident = field.ident.as_ref().unwrap().clone();
Some((field_vis, field_ident))
} else {
None
}
})
.collect::<Vec<_>>()
}
const ATTR_META_SKIP: &'static str = "skip";
fn has_skip_attr(attr: &Attribute, path: &'static str) -> bool {
if let Ok(Meta::List(meta_list)) = attr.parse_meta() {
if meta_list.path.is_ident(path) {
for nested_item in meta_list.nested.iter() {
if let NestedMeta::Meta(Meta::Path(path)) = nested_item {
if path.is_ident(ATTR_META_SKIP) {
return true;
}
}
}
}
}
false
}
The code it's taken from here. Basically I just want to get those values as a String, and not to access them via Foo::FIELD_NAMES.some_random_field, because I need them for another process.
How can I achieve that?
Thanks
I have the following code, which runs fine on Mac and Windows, but panics on Ubuntu Linux. The panic is at the borrow_mut and the indication is that server is already immutably borrowed. That would seem to indicate that the if leaked the server borrow, which I wasn't expecting it to do. I then wrapped the 3 ifs within a { } and Ubuntu Linux no longer panics. Which I think proves that the if leaked the borrow. Am I wrong? is this expected? Or, is there something else going on with Rust and AtomicRefCell that I'm unaware of?
AtomicRefCell comes from atomic_refcell.
static server: AtomicRefCell<Server> = AtomicRefCell::new(Server {..});
...
if let ServerField::Executor(executor) = &server.borrow().executor {
executor.stop();
}
if let ServerField::Scheduler(scheduler) = &server.borrow().scheduler {
scheduler.stop()
}
if let ServerField::Monitor(monitor) = &server.borrow().monitor {
monitor.stop()
}
let mut s = server.borrow_mut();
...
Here's the code that works with all 3 platforms:
// ensure that server.borrows don't leak out.
{
if let ServerField::Executor(executor) = &server.borrow().executor {
executor.stop();
}
if let ServerField::Scheduler(scheduler) = &server.borrow().scheduler {
scheduler.stop()
}
if let ServerField::Monitor(monitor) = &server.borrow().monitor {
monitor.stop()
}
}
let mut s = server.borrow_mut();
...
Here is a complete impl to illustrate the problem. main.rs:
use atomic_refcell::AtomicRefCell;
#[allow(non_upper_case_globals)]
static server: AtomicRefCell<Server> = AtomicRefCell::new(Server {
value: ServerField::Uninitialized,
});
#[derive(Debug)]
pub struct Server {
value: ServerField,
}
#[derive(Debug)]
enum ServerField {
Uninitialized,
Value(usize),
}
fn start() {
server.borrow_mut().value = ServerField::Value(41);
}
fn stop() {
// enclose this in a scope to not panic
let borrow = &server.borrow();
println!("value is {:#?}", borrow.value);
drop(borrow);
// end scope
let mut s = server.borrow_mut();
s.value = ServerField::Uninitialized;
}
fn main() {
println!("Hello, world!");
start();
stop();
}
#[cfg(test)]
pub mod tests {
use super::*;
#[test]
fn test() {
start();
stop();
}
}
Add this to your Cargo.toml:
[dependencies]
atomic_refcell = "0.1"
In this example an if let is employed with a RHS of &server.borrow().value, which doesn't panic on Mac but may panic on Ubuntu Linux (I don't have a Linux box, and didn't upload to GitHub and create a workflow). However, this mirrors my code that I do have on GitHub with a workflow, where it showed that behavior.
use atomic_refcell::AtomicRefCell;
#[allow(non_upper_case_globals)]
static server: AtomicRefCell<Server> = AtomicRefCell::new(Server {
value: ServerField::Uninitialized,
});
#[derive(Debug)]
pub struct Server {
value: ServerField,
}
#[derive(Debug)]
enum ServerField {
Uninitialized,
Value(usize),
}
fn start() {
server.borrow_mut().value = ServerField::Value(41);
}
fn stop() {
if let ServerField::Value(val) = &server.borrow().value {
println!("value is {:#?}", val);
}
let mut s = server.borrow_mut();
s.value = ServerField::Uninitialized;
}
fn main() {
println!("Hello, world!");
start();
stop();
}
#[cfg(test)]
pub mod tests {
use super::*;
#[test]
fn test_start_stop() {
start();
stop();
}
}
I'm a Java developer and in the process of learning Rust. Here is a tiny example that reads the file names of the current directory into a string list/vector and then outputs it.
Java with nio:
import java.io.*;
import java.nio.file.*;
import java.util.*;
import java.util.stream.*;
public class ListFileNames {
public static void main(String[] args) {
final Path dir = Paths.get(".");
final List<String> fileNames = new ArrayList<>();
try {
final Stream<Path> dirEntries = Files.list(dir);
dirEntries.forEach(path -> {
if (Files.isRegularFile(path)) {
fileNames.add(path.getFileName().toString());
}
});
}
catch (IOException ex) {
System.err.println("Failed to read " + dir + ": " + ex.getMessage());
}
print(fileNames);
}
private static void print(List<String> names) {
for (String name : names) {
System.out.println(name);
}
}
}
Here is what I came up with in Rust:
use std::fs;
use std::path::Path;
fn main() {
let dir = Path::new(".");
let mut entries: Vec<String> = Vec::new();
let dir_name = dir.to_str().unwrap();
let dir_content = fs::read_dir(dir);
match dir_content {
Ok(dir_content) => {
for entry in dir_content {
if let Ok(dir_entry) = entry {
if let Ok(file_type) = dir_entry.file_type() {
if file_type.is_file() {
if let Some(string) = dir_entry.file_name().to_str() {
entries.push(String::from(string));
}
}
}
}
}
}
Err(error) => println!("failed to read {}: {}", dir_name, error)
}
print(entries);
}
fn print(names: Vec<String>) {
for name in &names {
println!("{}", name);
}
}
Are there means to reduce the excessive indentations in the Rust code making it more easier to read similar to the Java code?
It is possible to use ? to short-circuit execution, as demonstrated by #Bazaim, although this has slightly different semantics as it stops on the first error, rather than ignoring it.
To keep true to your semantics, you would move to a Stream-based Iterator-based approach as can be seen here:
use std::error::Error;
use std::fs;
use std::path::Path;
fn main() -> Result<(), Box<dyn Error>> {
let dir = Path::new(".");
let dir_content = fs::read_dir(dir)?;
dir_content.into_iter()
.filter_map(|entry| entry.ok())
.filter(|entry| if let Ok(t) = entry.file_type() { t.is_file() } else { false })
.filter_map(|entry| entry.file_name().into_string().ok())
.for_each(|file_name| println!("{}", file_name));
Ok(())
}
I am not quite happy with that filter step, but I do like the fact that the functional-style API cleanly separates each step.
I'm also a beginner at Rust.
The ? operator is very usefull.
Here is the smallest version I get :
use std::error::Error;
use std::fs;
use std::path::Path;
fn main() -> Result<(), Box<dyn Error>> {
let dir = Path::new(".");
let mut entries: Vec<String> = Vec::new();
let dir_content = fs::read_dir(dir)?;
for entry in dir_content {
let entry = entry?;
if entry.file_type()?.is_file() {
if let Some(string) = entry.file_name().to_str() {
entries.push(String::from(string));
}
}
}
print(entries);
Ok(())
}
fn print(names: Vec<String>) {
for name in &names {
println!("{}", name);
}
}
Slightly modified version of #Bazaim's answer.
So, you can write the fallible code in a separate function and pass it reference to the vector.
If it fails, you can print the error and still print the names from the vector you passed in.
use std::error::Error;
use std::fs;
use std::path::Path;
fn main() {
let mut names = vec![];
if let Err(e) = collect(&mut names) {
println!("Error: {}", e);
}
for name in names {
println!("{}", name);
}
}
fn collect(names: &mut Vec<String>) -> Result<(), Box<dyn Error>> {
let dir = Path::new(".");
let dir_content = fs::read_dir(dir)?;
for entry in dir_content {
let entry = entry?;
if entry.file_type()?.is_file() {
if let Some(string) = entry.file_name().to_str() {
names.push(String::from(string));
}
}
}
Ok(())
}