I'm implementing a bytecode VM and am struggling referencing data stored in a parsed representation of the bytecode. As is the nature of (most) bytecode, it and thus its parsed representation remain unmodified once it's initialized. A separate Vm contains the mutable parts (stack etc.) along with that module. I made an MCVE with additional explanatory comments to illustrate the problem; it's at the bottom and on the playground. The parsed bytecode may look like this:
Module { struct_types: {"Bar": StructType::Named(["a", "b"])} }
The strings "Bar", "a", "b" are references into the bytecode and have lifetime 'b, so we also have lifetimes in the types Module<'b> and StructType<'b>.
After creating this, I will want to create struct instances, think let bar = Bar { a: (), b: () };. At least currently, each struct instance needs to hold a reference to its type, so that type might look like this:
pub struct Struct<'b> {
struct_type: &'b bytecode::StructType<'b>,
fields: Vec<Value<'b>>,
}
The values of a struct's fields may be constants whose value is stored in the bytecode, so the Value enum has a lifetime 'b as well, and that works. The problem is that I have a &'b bytecode::StructType<'b> in the first field: how do I get a reference that lives long enough? I think the reference would actually be valid long enough.
The part of the code that I suspect to be the critical one is here:
pub fn struct_type(&self, _name: &str) -> Option<&'b StructType<'b>> {
// self.struct_types.get(name)
todo!("fix lifetime problems")
}
With the commented out code, I can't get a 'b reference because the reference self.struct_types lives too short; to fix that I'd need to do &'b self which would spread virally through the code; also, most of the times I need to borrow the Vm mutably, which doesn't work if all those exclusive self references have to live long.
Introducing a separate lifetime 'm so that I could return a &'m StructType<'b> sounds like something that I could try as well, but that sounds just as viral and in addition introduces a separate lifetime I need to keep track of; being able to replace 'b with 'm (or at least only having on in each place) would be a bit nicer.
Finally this feels like something that pinning could be helpful with, but I don't understand that topic enough to make an educated guess on how that could be approached.
MCVE
#![allow(dead_code)]
mod bytecode {
use std::collections::BTreeMap;
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum StructType<'b> {
/// unit struct type; doesn't have fields
Empty,
/// tuple struct type; fields are positional
Positional(usize),
/// "normal" struct type; fields are named
Named(Vec<&'b str>),
}
impl<'b> StructType<'b> {
pub fn field_count(&self) -> usize {
match self {
Self::Empty => 0,
Self::Positional(field_count) => *field_count,
Self::Named(fields) => fields.len(),
}
}
}
#[derive(Debug, Clone)]
pub struct Module<'b> {
struct_types: BTreeMap<&'b str, StructType<'b>>,
}
impl<'b> Module<'b> {
// here is the problem: I would like to return a reference with lifetime 'b.
// from the point I start executing instructions, I know that I won't modify
// the module (particularly, I won't add entries to the map), so I think that
// lifetime should be possible - pinning? `&'b self` everywhere? idk
pub fn struct_type(&self, _name: &str) -> Option<&'b StructType<'b>> {
// self.struct_types.get(name)
todo!("fix lifetime problems")
}
}
pub fn parse<'b>(bytecode: &'b str) -> Module<'b> {
// this would use nom to parse actual bytecode
assert_eq!(bytecode, "struct Bar { a, b }");
let bar = &bytecode[7..10];
let a = &bytecode[13..14];
let b = &bytecode[16..17];
let fields = vec![a, b];
let bar_struct = StructType::Named(fields);
let struct_types = BTreeMap::from_iter([
(bar, bar_struct),
]);
Module { struct_types }
}
}
mod vm {
use crate::bytecode::{self, StructType};
#[derive(Debug, Clone)]
pub enum Value<'b> {
Unit,
Struct(Struct<'b>),
}
#[derive(Debug, Clone)]
pub struct Struct<'b> {
struct_type: &'b bytecode::StructType<'b>,
fields: Vec<Value<'b>>,
}
impl<'b> Struct<'b> {
pub fn new(struct_type: &'b bytecode::StructType<'b>, fields: Vec<Value<'b>>) -> Self {
Struct { struct_type, fields }
}
}
#[derive(Debug, Clone)]
pub struct Vm<'b> {
module: bytecode::Module<'b>,
}
impl<'b> Vm<'b> {
pub fn new(module: bytecode::Module<'b>) -> Self {
Self { module }
}
pub fn create_struct(&mut self, type_name: &'b str) -> Value<'b> {
let struct_type: &'b StructType<'b> = self.module.struct_type(type_name).unwrap();
// just initialize the fields to something, we don't care
let fields = vec![Value::Unit; struct_type.field_count()];
let value = Value::Struct(Struct::new(struct_type, fields));
value
}
}
}
pub fn main() {
// the bytecode contains all constants needed at runtime;
// we're just interested in how struct types are handled
// obviously the real bytecode is not as human-readable
let bytecode = "struct Bar { a, b }";
// we parse that into a module that, among other things,
// has a map of all struct types
let module = bytecode::parse(bytecode);
println!("{:?}", module);
// we create a Vm that is capable of running commands
// that are stored in the module
let mut vm = vm::Vm::new(module);
// now we try to execute an instruction to create a struct value
// the instruction for this contains a reference to the type name
// stored in the bytecode.
// the struct value contains a reference to its type and holds its field values.
let value = {
let bar = &bytecode[7..10];
vm.create_struct(bar)
};
println!("{:?}", value);
}
&'b bytecode::StructType<'b> is a classic anti-pattern in Rust, it strongly indicates incorrectly annotated lifetimes. It doesn't make sense that an object would depend on some lifetime and borrowing it creates the same lifetime. That is very rare to happen on purpose.
So I suspect you need two lifetimes, which I will call 'm and 'b:
'b: the lifetime of the bytecode string, everything that references it will use &'b str.
'm: the lifetime of the Module object. Everything that references it or its contained StructType will use this lifetime.
If split into two lifetimes and adjusted correctly, it simply works:
#![allow(dead_code)]
mod bytecode {
use std::{collections::BTreeMap, iter::FromIterator};
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum StructType<'b> {
/// unit struct type; doesn't have fields
Empty,
/// tuple struct type; fields are positional
Positional(usize),
/// "normal" struct type; fields are named
Named(Vec<&'b str>),
}
impl<'b> StructType<'b> {
pub fn field_count(&self) -> usize {
match self {
Self::Empty => 0,
Self::Positional(field_count) => *field_count,
Self::Named(fields) => fields.len(),
}
}
}
#[derive(Debug, Clone)]
pub struct Module<'b> {
struct_types: BTreeMap<&'b str, StructType<'b>>,
}
impl<'b> Module<'b> {
// here is the problem: I would like to return a reference with lifetime 'b.
// from the point I start executing instructions, I know that I won't modify
// the module (particularly, I won't add entries to the map), so I think that
// lifetime should be possible - pinning? `&'b self` everywhere? idk
pub fn struct_type(&self, name: &str) -> Option<&StructType<'b>> {
self.struct_types.get(name)
}
}
pub fn parse<'b>(bytecode: &'b str) -> Module<'b> {
// this would use nom to parse actual bytecode
assert_eq!(bytecode, "struct Bar { a, b }");
let bar = &bytecode[7..10];
let a = &bytecode[13..14];
let b = &bytecode[16..17];
let fields = vec![a, b];
let bar_struct = StructType::Named(fields);
let struct_types = BTreeMap::from_iter([(bar, bar_struct)]);
Module { struct_types }
}
}
mod vm {
use crate::bytecode::{self, StructType};
#[derive(Debug, Clone)]
pub enum Value<'b, 'm> {
Unit,
Struct(Struct<'b, 'm>),
}
#[derive(Debug, Clone)]
pub struct Struct<'b, 'm> {
struct_type: &'m bytecode::StructType<'b>,
fields: Vec<Value<'b, 'm>>,
}
impl<'b, 'm> Struct<'b, 'm> {
pub fn new(struct_type: &'m bytecode::StructType<'b>, fields: Vec<Value<'b, 'm>>) -> Self {
Struct {
struct_type,
fields,
}
}
}
#[derive(Debug, Clone)]
pub struct Vm<'b> {
module: bytecode::Module<'b>,
}
impl<'b> Vm<'b> {
pub fn new(module: bytecode::Module<'b>) -> Self {
Self { module }
}
pub fn create_struct(&mut self, type_name: &str) -> Value<'b, '_> {
let struct_type: &StructType<'b> = self.module.struct_type(type_name).unwrap();
// just initialize the fields to something, we don't care
let fields = vec![Value::Unit; struct_type.field_count()];
let value = Value::Struct(Struct::new(struct_type, fields));
value
}
}
}
pub fn main() {
// the bytecode contains all constants needed at runtime;
// we're just interested in how struct types are handled
// obviously the real bytecode is not as human-readable
let bytecode = "struct Bar { a, b }";
// we parse that into a module that, among other things,
// has a map of all struct types
let module = bytecode::parse(bytecode);
println!("{:?}", module);
// we create a Vm that is capable of running commands
// that are stored in the module
let mut vm = vm::Vm::new(module);
// now we try to execute an instruction to create a struct value
// the instruction for this contains a reference to the type name
// stored in the bytecode.
// the struct value contains a reference to its type and holds its field values.
let value = {
let bar = &bytecode[7..10];
vm.create_struct(bar)
};
println!("{:?}", value);
}
Module { struct_types: {"Bar": Named(["a", "b"])} }
Struct(Struct { struct_type: Named(["a", "b"]), fields: [Unit, Unit] })
It can further be simplified, however, due to the fact that 'm is connected to 'b, and therefore everything that depends on 'm automatically also has access to 'b objects, because 'b is guaranteed to outlive 'm.
Therefore, let's introduce 'a, which we will now use inside of the vm mod to reference anything from the bytecode mod. This will further allow lifetime elysion to happen at a couple of points, simplifying the code even further:
#![allow(dead_code)]
mod bytecode {
use std::{collections::BTreeMap, iter::FromIterator};
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum StructType<'b> {
/// unit struct type; doesn't have fields
Empty,
/// tuple struct type; fields are positional
Positional(usize),
/// "normal" struct type; fields are named
Named(Vec<&'b str>),
}
impl<'b> StructType<'b> {
pub fn field_count(&self) -> usize {
match self {
Self::Empty => 0,
Self::Positional(field_count) => *field_count,
Self::Named(fields) => fields.len(),
}
}
}
#[derive(Debug, Clone)]
pub struct Module<'b> {
struct_types: BTreeMap<&'b str, StructType<'b>>,
}
impl<'b> Module<'b> {
// here is the problem: I would like to return a reference with lifetime 'b.
// from the point I start executing instructions, I know that I won't modify
// the module (particularly, I won't add entries to the map), so I think that
// lifetime should be possible - pinning? `&'b self` everywhere? idk
pub fn struct_type(&self, name: &str) -> Option<&StructType<'b>> {
self.struct_types.get(name)
}
}
pub fn parse<'b>(bytecode: &'b str) -> Module<'b> {
// this would use nom to parse actual bytecode
assert_eq!(bytecode, "struct Bar { a, b }");
let bar = &bytecode[7..10];
let a = &bytecode[13..14];
let b = &bytecode[16..17];
let fields = vec![a, b];
let bar_struct = StructType::Named(fields);
let struct_types = BTreeMap::from_iter([(bar, bar_struct)]);
Module { struct_types }
}
}
mod vm {
use crate::bytecode::{self, StructType};
#[derive(Debug, Clone)]
pub enum Value<'a> {
Unit,
Struct(Struct<'a>),
}
#[derive(Debug, Clone)]
pub struct Struct<'a> {
struct_type: &'a bytecode::StructType<'a>,
fields: Vec<Value<'a>>,
}
impl<'a> Struct<'a> {
pub fn new(struct_type: &'a bytecode::StructType, fields: Vec<Value<'a>>) -> Self {
Struct {
struct_type,
fields,
}
}
}
#[derive(Debug, Clone)]
pub struct Vm<'a> {
module: bytecode::Module<'a>,
}
impl<'a> Vm<'a> {
pub fn new(module: bytecode::Module<'a>) -> Self {
Self { module }
}
pub fn create_struct(&mut self, type_name: &str) -> Value {
let struct_type: &StructType = self.module.struct_type(type_name).unwrap();
// just initialize the fields to something, we don't care
let fields = vec![Value::Unit; struct_type.field_count()];
let value = Value::Struct(Struct::new(struct_type, fields));
value
}
}
}
pub fn main() {
// the bytecode contains all constants needed at runtime;
// we're just interested in how struct types are handled
// obviously the real bytecode is not as human-readable
let bytecode = "struct Bar { a, b }";
// we parse that into a module that, among other things,
// has a map of all struct types
let module = bytecode::parse(bytecode);
println!("{:?}", module);
// we create a Vm that is capable of running commands
// that are stored in the module
let mut vm = vm::Vm::new(module);
// now we try to execute an instruction to create a struct value
// the instruction for this contains a reference to the type name
// stored in the bytecode.
// the struct value contains a reference to its type and holds its field values.
let value = {
let bar = &bytecode[7..10];
vm.create_struct(bar)
};
println!("{:?}", value);
}
Module { struct_types: {"Bar": Named(["a", "b"])} }
Struct(Struct { struct_type: Named(["a", "b"]), fields: [Unit, Unit] })
Fun fact: This is now one of the rare cases where we legitimately have to use &'a bytecode::StructType<'a>, so take my opening statement with a grain of salt, and you were kind of right all along :)
The crazy thing is if we then rename 'a to 'b to be consistent with your original code, we get almost your code with only some minor differences:
#![allow(dead_code)]
mod bytecode {
use std::{collections::BTreeMap, iter::FromIterator};
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum StructType<'b> {
/// unit struct type; doesn't have fields
Empty,
/// tuple struct type; fields are positional
Positional(usize),
/// "normal" struct type; fields are named
Named(Vec<&'b str>),
}
impl<'b> StructType<'b> {
pub fn field_count(&self) -> usize {
match self {
Self::Empty => 0,
Self::Positional(field_count) => *field_count,
Self::Named(fields) => fields.len(),
}
}
}
#[derive(Debug, Clone)]
pub struct Module<'b> {
struct_types: BTreeMap<&'b str, StructType<'b>>,
}
impl<'b> Module<'b> {
// here is the problem: I would like to return a reference with lifetime 'b.
// from the point I start executing instructions, I know that I won't modify
// the module (particularly, I won't add entries to the map), so I think that
// lifetime should be possible - pinning? `&'b self` everywhere? idk
pub fn struct_type(&self, name: &str) -> Option<&StructType<'b>> {
self.struct_types.get(name)
}
}
pub fn parse<'b>(bytecode: &'b str) -> Module<'b> {
// this would use nom to parse actual bytecode
assert_eq!(bytecode, "struct Bar { a, b }");
let bar = &bytecode[7..10];
let a = &bytecode[13..14];
let b = &bytecode[16..17];
let fields = vec![a, b];
let bar_struct = StructType::Named(fields);
let struct_types = BTreeMap::from_iter([(bar, bar_struct)]);
Module { struct_types }
}
}
mod vm {
use crate::bytecode::{self, StructType};
#[derive(Debug, Clone)]
pub enum Value<'b> {
Unit,
Struct(Struct<'b>),
}
#[derive(Debug, Clone)]
pub struct Struct<'b> {
struct_type: &'b bytecode::StructType<'b>,
fields: Vec<Value<'b>>,
}
impl<'b> Struct<'b> {
pub fn new(struct_type: &'b bytecode::StructType, fields: Vec<Value<'b>>) -> Self {
Struct {
struct_type,
fields,
}
}
}
#[derive(Debug, Clone)]
pub struct Vm<'b> {
module: bytecode::Module<'b>,
}
impl<'b> Vm<'b> {
pub fn new(module: bytecode::Module<'b>) -> Self {
Self { module }
}
pub fn create_struct(&mut self, type_name: &str) -> Value {
let struct_type: &StructType = self.module.struct_type(type_name).unwrap();
// just initialize the fields to something, we don't care
let fields = vec![Value::Unit; struct_type.field_count()];
let value = Value::Struct(Struct::new(struct_type, fields));
value
}
}
}
pub fn main() {
// the bytecode contains all constants needed at runtime;
// we're just interested in how struct types are handled
// obviously the real bytecode is not as human-readable
let bytecode = "struct Bar { a, b }";
// we parse that into a module that, among other things,
// has a map of all struct types
let module = bytecode::parse(bytecode);
println!("{:?}", module);
// we create a Vm that is capable of running commands
// that are stored in the module
let mut vm = vm::Vm::new(module);
// now we try to execute an instruction to create a struct value
// the instruction for this contains a reference to the type name
// stored in the bytecode.
// the struct value contains a reference to its type and holds its field values.
let value = {
let bar = &bytecode[7..10];
vm.create_struct(bar)
};
println!("{:?}", value);
}
Module { struct_types: {"Bar": Named(["a", "b"])} }
Struct(Struct { struct_type: Named(["a", "b"]), fields: [Unit, Unit] })
So the actual fix for your original code is as follows:
4c4
< use std::collections::BTreeMap;
---
> use std::{collections::BTreeMap, iter::FromIterator};
36,38c36,37
< pub fn struct_type(&self, _name: &str) -> Option<&'b StructType<'b>> {
< // self.struct_types.get(name)
< todo!("fix lifetime problems")
---
> pub fn struct_type(&self, name: &str) -> Option<&StructType<'b>> {
> self.struct_types.get(name)
73c72
< pub fn new(struct_type: &'b bytecode::StructType<'b>, fields: Vec<Value<'b>>) -> Self {
---
> pub fn new(struct_type: &'b bytecode::StructType, fields: Vec<Value<'b>>) -> Self {
91,92c90,91
< pub fn create_struct(&mut self, type_name: &'b str) -> Value<'b> {
< let struct_type: &'b StructType<'b> = self.module.struct_type(type_name).unwrap();
---
> pub fn create_struct(&mut self, type_name: &str) -> Value {
> let struct_type: &StructType = self.module.struct_type(type_name).unwrap();
I hope deriving them step by step made it somewhat clear why those changes are necessary.
I'm currently building an image carousel. I want to count the contents of a directory that is only accessible at the OS level and pass the result to a static hashmap inside a web assembly module in Yew.
Here's a use_reducer paired with a Context in which I'd like the default state to hold the count of files.
use std::{cmp::max, collections::HashMap, fs, rc::Rc};
use yew::prelude::*;
extern crate web_sys;
// A macro to provide `println!(..)`-style syntax for `console.log` logging.
macro_rules! log {
( $( $t:tt )* ) => {
web_sys::console::log_1(&format!( $( $t )* ).into());
}
}
pub enum CarouselAction {
Prev,
Next,
}
#[derive(Debug, PartialEq, Eq, Clone)]
pub struct CarouselState {
chapter_state: HashMap<i32, i32>,
pub page: i32,
pub chapter: i32,
}
pub type CarouselContext = UseReducerHandle<CarouselState>;
impl Default for CarouselState {
fn default() -> Self {
let dir = "./assets/carousel/op";
let total_chapters = fs::read_dir(dir)
.unwrap()
.filter(|entry| entry.as_ref().unwrap().metadata().unwrap().is_file())
.count();
log!("TOTAL CHAPTERS {}", total_chapters);
Self {
chapter_state: HashMap::new(),
page: 1,
chapter: 1043,
}
}
}
impl Reducible for CarouselState {
// Reducer Action Type
type Action = CarouselAction;
fn reduce(self: Rc<Self>, action: Self::Action) -> Rc<Self> {
match action {
CarouselAction::Prev => Self {
page: max(self.page - 1, 1),
chapter: self.chapter,
chapter_state: self.chapter_state.to_owned(),
}
.into(),
CarouselAction::Next => {
log!("self.page {}", self.page);
Self {
page: self.page + 1,
chapter: self.chapter,
chapter_state: self.chapter_state.to_owned(),
}
.into()
}
}
}
}
#[derive(PartialEq, Debug, Properties)]
pub struct CarouselContextProps {
#[prop_or_default]
pub children: Children,
}
#[function_component(CarouselContextProvider)]
pub fn carousel_context_provider(props: &CarouselContextProps) -> Html {
let state = use_reducer(CarouselState::default);
html! {
<ContextProvider<CarouselContext> context={state}>
{props.children.clone()}
</ContextProvider<CarouselContext>>
}
}
pub fn use_carousel_context() -> impl Hook<Output = Option<UseReducerHandle<CarouselState>>> {
use_context::<CarouselContext>()
}
The code in question is:
impl Default for CarouselState {
fn default() -> Self {
let dir = "./assets/carousel/op";
let total_chapters = fs::read_dir(dir)
.unwrap()
.filter(|entry| entry.as_ref().unwrap().metadata().unwrap().is_file())
.count();
log!("TOTAL CHAPTERS {}", total_chapters);
Self {
chapter_state: HashMap::new(),
page: 1,
chapter: 1043,
}
}
}
I'm running the filesystem in the browser, which is not accessible, and it's returning an error (obviously). I want to run this block of code before it gets compiled to wasm and holds its result in the chapter_state hash map.
Is this possible?
I'm currently thinking of using a separate script that generates a file with the hashmap result and then including that file in yew.
#[derive(Debug, Deserialize)]
struct S3StorageConfig {
url: String,
}
#[derive(Debug, Deserialize)]
struct LocalStorageConfig {
root: std::path::PathBuf,
}
#[derive(Debug, Deserialize)]
struct StorageConfig {
storage_type: String
}
#[derive(Debug, Deserialize)]
pub struct Config {
storages: Vec<StorageConfig>
}
impl Config {
pub fn new(path:Option<std::path::PathBuf>) -> Result<Self, config::ConfigError> {
let mut cfg = config::Config::default();
if let Some(file_path) = path {
cfg.merge(config::File::from(file_path)).unwrap();
}
cfg.merge(config::Environment::with_prefix("datastore"))?;
cfg.try_into()
}
}
Suppose I want to have a config that has
[[storages]]
type: s3
url: ...
and
[[storages]]
type: local
root: ...
And when config does try_into, it is able to find these structs and assign them to the correct structs, by grace of the type field.
What magic would I need to do to make this happen?
Thanks,
So, I'm not 100% sure what you're trying to achieve here but you can serialize/deserialize into the types that you want with serde and using an enum instead.
Ex:
// This enum takes the place of your 'S3StorageConfig' and 'LocalStorageConfig'.
#[derive( Serialize, Deserialize, Debug )]
#[serde( tag = "type" )]
enum Storage {
Cloud{ url: String },
Local{ root: PathBuf },
}
fn main( ) {
let vec = vec![
Storage::Cloud{ url: "www.youtube.com".to_string( ) },
Storage::Local{ root: PathBuf::from( "C:\\Windows\\Fonts" ) },
];
let storage = serde_json::to_string( &vec ).unwrap( );
let vec: Vec<Storage> = serde_json::from_str( &storage ).unwrap( );
println!( "{:#?}", vec );
}
Now you will return a Storage enum variant from your Config class.
You wont need to impl TryInto if this is the direction you decide to take.
impl Config {
pub fn new( ) -> Result<Storage, config::ConfigError> {
// Read in the file here and use 'serde' to deserialize the
// content of the file into the correct enum variant that you
// can now return.
}
}
I want to write a vscode extension that displays the content of a large binary file, written with bincode:
#[macro_use]
extern crate serde_derive;
use std::collections::HashMap;
use std::fs::File;
use std::io::{BufReader, BufWriter};
#[derive(Serialize, Deserialize)]
pub struct MyValue {
pub name: String,
}
#[derive(Serialize, Deserialize)]
pub struct MyStruct {
pub data: HashMap<String, MyValue>,
}
impl MyStruct {
pub fn dump(&self, filename: &str) -> Result<(), String> {
let file = File::create(filename).map_err(|msg| msg.to_string())?;
let writer = BufWriter::new(file);
bincode::serialize_into(writer, self).map_err(|msg| msg.to_string())
}
pub fn load(filename: &str) -> Result<Self, String> {
let file = File::open(filename).map_err(|msg| msg.to_string())?;
let reader = BufReader::new(file);
bincode::deserialize_from::<BufReader<_>, Self>(reader).map_err(|msg| msg.to_string())
}
}
Therefore there is a wasm binding:
#[wasm_bindgen]
#[derive(Clone)]
pub struct PyMyStruct {
inner: Arc<MyStruct>,
}
#[wasm_bindgen]
impl PyMyStruct {
pub fn new(filename: &str) -> Self {
Self {
inner: Arc::new(MyStruct::load(filename).unwrap()),
}
}
pub fn header(self) -> Array {
let keys = Array::new();
for key in self.inner.data.keys() {
keys.push(&JsValue::from_str(key));
}
keys
}
pub fn value(&self, name: &str) -> JsValue {
if let Some(value) = self.inner.data.get(name) {
JsValue::from_serde(value).unwrap_or(JsValue::NULL)
} else {
JsValue::NULL
}
}
}
which provides a simple interface to the JavaScript world in order to access the content of that file.
Using Arc in order to prevent expensive unintended memory copy when handling on the JavaScript side.
(It might look strange that keys is not marked as mutable but the rust compiler recomended that way)
When running the test code:
const {PyMyStruct} = require("./corejs.js");
let obj = new PyMyStruct("../../dump.spb")
console.log(obj.header())
you get the error message:
Error: null pointer passed to rust
Does someone know how to handle this use case?
Thank you!
The issue here is that you are using new PyMyStruct() instead of PyMyStruct.new(). In wasm-bindgen's debug mode you will get an error about this at runtime. Using .new() will fix your issue:
let obj = PyMyStruct.new("../../dump.spb")
If you add the #[wasm_bindgen(constructor)] annotation to the new method, then new PyMyStruct() will work as well:
#[wasm_bindgen]
impl PyMyStruct {
#[wasm_bindgen(constructor)]
pub fn new(filename: &str) -> Self {
Self {
inner: 1,
}
}
}
Now this is fine:
let obj = new PyMyStruct("../../dump.spb")
I solved that problem by using https://neon-bindings.com instead of compiling the API to web-assembly.
The binding here looks as follow:
use core;
use std::rc::Rc;
use neon::prelude::*;
#[derive(Clone)]
pub struct MyStruct {
inner: Rc<core::MyStruct>,
}
declare_types! {
pub class JsMyStruct for MyStruct {
init(mut cx) {
let filename = cx.argument::<JsString>(0)?.value();
match core::MyStruct::load(&filename) {
Ok(inner) => return Ok(MyStruct{
inner: Rc::new(inner)
}),
Err(msg) => {
panic!("{}", msg)
}
}
}
method header(mut cx) {
let this = cx.this();
let container = {
let guard = cx.lock();
let this = this.borrow(&guard);
(*this).clone()
};
let keys = container.inner.data.keys().collect::<Vec<_>>();
let js_array = JsArray::new(&mut cx, keys.len() as u32);
for (i, obj) in keys.into_iter().enumerate() {
let js_string = cx.string(obj);
js_array.set(&mut cx, i as u32, js_string).unwrap();
}
Ok(js_array.upcast())
}
method value(mut cx) {
let key = cx.argument::<JsString>(0)?.value();
let this = cx.this();
let container = {
let guard = cx.lock();
let this = this.borrow(&guard);
(*this).clone()
};
if let Some(data) = container.inner.data.get(&key) {
return Ok(neon_serde::to_value(&mut cx, data)?);
} else {
panic!("No value for key \"{}\" available", key);
}
}
}
}
register_module!(mut m, {
m.export_class::<JsMyStruct>("MyStruct")?;
Ok(())
});
I want to implement a graph structure in Rust. For this goal, I wrote simple abstractions:
pub struct Graph<'a> {
pub nodes: Vec<Node>,
pub edges: Vec<Edge<'a>>,
}
#[derive(Debug)]
pub struct Node {
pub id: String,
pub label: String,
}
pub struct Edge<'a> {
pub source: &'a Node,
pub target: &'a Node,
}
Graph contains vectors of Nodes and Edges. Every Edge has a ref to a Node in the same Graph.
I don't know it's a possible write something like this.
I tried to write a static method that builds a new Graph instance from a JSON representation:
impl<'a> Graph<'a> {
pub fn from_json(json: &String) -> Graph {
if let json::JsonValue::Object(deserialized) = json::parse(json.as_ref()).unwrap() {
let nodes: Vec<Node> = deserialized
.get("nodes")
.unwrap()
.members()
.map(|v| {
if let json::JsonValue::Object(ref val) = *v {
return Node {
id: val.get("id").unwrap().to_string(),
label: val.get("label").unwrap().to_string(),
};
}
panic!("Invalid structure of json graph body.")
})
.collect::<Vec<Node>>();
let edges: Vec<Edge> = deserialized
.get("edges")
.unwrap()
.members()
.map(|v| {
if let json::JsonValue::Object(ref val) = *v {
let source = (*nodes)
.iter()
.find(|&v| v.id == val.get("source").unwrap().to_string())
.unwrap();
let target = (*nodes)
.iter()
.find(|&v| v.id == val.get("target").unwrap().to_string())
.unwrap();
return Edge { source, target };
}
panic!("Invalid structure of json graph body.")
})
.collect::<Vec<Edge>>();
return Graph { nodes, edges };
}
panic!("Incorrect struct of json contains!");
}
}
When I compile, I get this error:
error[E0373]: closure may outlive the current function, but it borrows `nodes`, which is owned by the current function
--> src/graph.rs:30:22
|
30 | .map(|v| {
| ^^^ may outlive borrowed value `nodes`
31 | if let json::JsonValue::Object(ref val) = *v {
32 | let source = (*nodes).iter().find(|&v| v.id == val.get("source").unwrap().to_string()).unwrap();
| ----- `nodes` is borrowed here
|
help: to force the closure to take ownership of `nodes` (and any other referenced variables), use the `move` keyword
|
30 | .map(move |v| {
| ^^^^^^^^
error: aborting due to previous error
A possible solution to this problem is to add move before the closure parameters, but I need the nodes vector to build the Graph instance.
What am I doing wrong?
After some research, I found this article's: Rust doc. Smart pointers, Users Rust Lang, and I understood my mistakes.
The first one: I remove lifetime parameters from structs definitions.
use std::rc::Rc;
#[derive(Debug)]
pub struct Graph {
pub nodes: Vec<Rc<Node>>,
pub edges: Vec<Edge>
}
#[derive(Debug)]
pub struct Node {
pub id: String,
pub label: String
}
#[derive(Debug)]
pub struct Edge {
pub source: Rc<Node>,
pub target: Rc<Node>
}
Second thing: I rewrote the code of from_json function for using Rc<T> instead of raw references.
impl Graph {
pub fn from_json(json: & String) -> Graph {
if let json::JsonValue::Object(deserialized) = json::parse(json.as_ref()).unwrap() {
let nodes : Vec<Rc<Node>> = deserialized.get("nodes").unwrap().members()
.map(|v| {
if let json::JsonValue::Object(ref val) = *v {
return Rc::new(Node {
id: val.get("id").unwrap().to_string(),
label: val.get("label").unwrap().to_string()
});
}
panic!("Invalid structure of json graph body.")
}).collect::<Vec<Rc<Node>>>();
let edges : Vec<Edge> = deserialized.get("edges").unwrap().members()
.map(|v| {
if let json::JsonValue::Object(ref val) = *v {
let source = nodes.iter().find(|&v| v.id == val.get("source").unwrap().to_string()).unwrap();
let target = nodes.iter().find(|&v| v.id == val.get("target").unwrap().to_string()).unwrap();
return Edge {
source: Rc::clone(&source),
target: Rc::clone(&target)
};
}
panic!("Invalid structure of json graph body.")
}).collect::<Vec<Edge>>();
return Graph {
nodes,
edges
}
}
panic!("Incorrect struct of json contains!");
}
}
Now it works. Thanks for sharing useful links. I found a lot of helpful information about building graph structs in Rust such as: Graph structure in Rust