I've got an exported-to-NodeJS Rust function that does something. Now, I want to extract part of that function into its own function (creating a JS object), so that I can reuse that logic in another, new exporter-to-NodeJS Rust function. However, I'm having great troubles figuring out how to pass the Context into that sub-function in a way that Rust is happy with. So far, the best (as in, "causes the compiler to complain the least") solution I have is this:
fn encode_num(mut cx: FunctionContext) -> JsResult<JsObject> {
let input = cx.argument::<JsNumber>(0)?.value(&mut cx);
let output = OrePlaintext::<u64>::from(input).0;
plaintext_from_u64(&mut cx, output)
}
fn plaintext_from_u64<'a>(cx: &'a mut FunctionContext, n: u64) -> JsResult<'a, JsObject> {
let bytes = n.to_ne_bytes();
let mut buf = match cx.buffer(8) {
Ok(b) => b,
Err(e) => return cx.throw_error(format!("Failed to allocate buffer: {:?}", e))
};
cx.borrow_mut(&mut buf, |data| {
let slice = data.as_mut_slice::<u8>();
for i in 0..8 {
slice[i] = bytes[i];
};
});
let obj = cx.empty_object();
obj.set(&mut cx, "kind", cx.string("OrePlainText"))?;
obj.set(&mut cx, "buf", buf)?;
Ok(obj)
}
#[neon::main]
fn main(mut cx: ModuleContext) -> NeonResult<()> {
cx.export_function("encodeNumber", encode_num)?;
Ok(())
}
This fails to compile with the following errors:
error[E0277]: the trait bound `&'a mut CallContext<'_, neon::prelude::JsObject>: neon::context::Context<'_>` is not satisfied
--> native/lib.rs:120:9
|
120 | obj.set(&mut cx, "kind", cx.string("OrePlainText"))?;
| ^^^ the trait `neon::context::Context<'_>` is not implemented for `&'a mut CallContext<'_, neon::prelude::JsObject>`
|
= help: the following implementations were found:
<CallContext<'a, T> as neon::context::Context<'a>>
error[E0277]: the trait bound `&'a mut CallContext<'_, neon::prelude::JsObject>: neon::context::Context<'_>` is not satisfied
--> native/lib.rs:121:9
|
121 | obj.set(&mut cx, "buf", buf)?;
| ^^^ the trait `neon::context::Context<'_>` is not implemented for `&'a mut CallContext<'_, neon::prelude::JsObject>`
|
= help: the following implementations were found:
<CallContext<'a, T> as neon::context::Context<'a>>
I started with plaintext_from_u64 just taking the context as mut cx: FunctionContext, which is how encode_num accepts it, but that then causes problems as soon as I want to call plaintext_from_u64 more than once, because "value borrowed after move", and if I try to borrow in the calls, I get "expected struct CallContext, found mutable reference. I've dug around in the neon examples repo, looking for exported functions that pass a context into an internal function, but I haven't been able to find anything so far, which is... frustrating.
What is the compiler-approved approach to this problem?
Related
I'm working on a compiler for a toy language and I want to be able to check for errors on each file. I have a MooFile struct that has a Vec<anyhow::Error> where I put errors when they are encountered. Now I want to go through the files, get the errors, and flatten that into a single vec to return from the compiler but I'm running up against the borrow checker. I'm going to post my various attempt but they keep running into similar issues.
fn compile(&mut self) -> Result<(), Vec<Error>> {
...
// Check for errors
let has_errors = self.files
.iter()
.map(|file| file.has_errors())
.reduce(|acc, value| acc | value)
.unwrap_or(false);
if has_errors {
let mut errors = vec![];
self.files
.iter()
.for_each(|&mut file| errors.append(&mut file.errors));
Err(errors)
} else {
Ok(())
}
}
error[E0596]: cannot borrow `file.errors` as mutable, as it is behind a `&` reference
--> src\lib.rs:62:48
|
62 | .for_each(|file| errors.append(&mut file.errors));
| ---- ^^^^^^^^^^^^^^^^ `file` is a `&` reference, so the data it refers to cannot be borrowed as mutable
| |
| help: consider changing this to be a mutable reference: `&mut MooFile`
fn compile(&mut self) -> Result<(), Vec<Error>> {
...
// Check for errors
let has_errors = self.files
.iter()
.map(|file| file.has_errors())
.reduce(|acc, value| acc | value)
.unwrap_or(false);
if has_errors {
let mut errors = vec![];
self.files
.into_iter()
.for_each(|mut file| errors.append(&mut file.errors));
Err(errors)
} else {
Ok(())
}
}
error[E0507]: cannot move out of `self.files` which is behind a mutable reference
--> src\lib.rs:63:13
|
63 | self.files
| ^^^^^^^^^^ move occurs because `self.files` has type `Vec<MooFile>`, which does not implement the `Copy` trait
64 | .into_iter()
| ----------- `self.files` moved due to this method call
fn compile(&mut self) -> Result<(), Vec<Error>> {
...
// Check for errors
let errors: Vec<Error> = self.files
.iter()
.map(|file| file.errors)
.flatten()
.collect();
if errors.len() > 0 {
Err(errors)
} else {
Ok(())
}
}
error[E0507]: cannot move out of `file.errors` which is behind a shared reference
--> src\lib.rs:54:25
|
54 | .map(|file| file.errors)
| ^^^^^^^^^^^ move occurs because `file.errors` has type `Vec<anyhow::Error>`, which does not implement the `Copy` trait
For more information about this error, try `rustc --explain E0507`.
error: could not compile `moo2` due to previous error
I feel like there has to be some kind of idiomatic way of doing this kind of thing that I'm missing.
Most of these errors come from how you create your iterators. If you assume x is a Vec<Y>:
x.iter(): Consumes &x and creates an iterator of &Y.
x.iter_mut(): Consumes &mut x and creates an iterator of &mut Y.
x.into_iter(): Consumes ownership of the entire Vec<Y> and creates an iterator of owned Y.
The first version uses .iter() when you want a mutable reference (.iter_mut()). The second uses .into_iter() when you want a mutable reference (.iter_mut()). And the third uses .iter() when you attempt to claim ownership of the errors by disposing of the files (.into_iter(), but you can't drop self.files without issues).
Here is one way you could Clone the errors without moving them out of each file.
let mut errors = Vec::new();
self.files.iter()
.filter(|file| file.has_errors())
.foreach(|file| errors.extend_from_slice(&file.errors));
if has_errors {
return Err(errors)
}
Ok(())
Or you could drain them into a new Vec using extend. This version also uses any to look a bit cleaner, but is not as efficient.
if self.files.iter().any(|f| f.has_errors()) {
let mut errors = Vec::new();
for file in &mut self.files {
errors.extend(file.errors.drain(..));
}
return Err(errors)
}
Ok(())
I am trying to pass around a HashMap which stores values through a set of nested enums/structs. The problem of multiple mutability happens during iteration, even all references should be dropped.
The general idea is to have a vector of values, iterate through them and simplify them, keeping track of them within the HashMap. There are two stages of simplification.
The general flow looks something like
run(Vec<ComplexVal>)
-for each val->
val.fix_complex(holder)
-for each `smp` SimpleVal in val->
basicval = Simplifier::step(smp, holder)
holder.insert("name", basicval)
But the problem is that the holder is borrowed mutably in each stage, and there isn't supposed to be any reference from the ComplexVal to the holder and since the borrowchecker doesn't like multiple borrows, it fails.
Full playground snippet: here
It happens in this snippet:
pub fn run(&mut self, mut vals: Vec<ComplexVal>) {
let mut holder = Holder{hold:HashMap::new()};
// .. setup holder code omitted
let len = vals.len();
for _ in 0..len {
let mut val = vals.remove(0); // remove from vec, should drop after running
println!("Running {:?}", val);
match val {
ComplexVal::Cmplx1(mut c) => {
c.fix_complex(&mut holder)
},
//... more cases of different types of values omitted for simplicity
}
// val *should* be dropped here, and therefore the mutable borrow of holder?
}
println!("Holder: {:?}", holder);
}
}
The only thing I can think of is that it somehow is related to the BasicVal::Ref(&BasicVal) value when created.
I need to return a reference of type &BasicVal so I can't use a regular fn() -> &BasicVal as the reference would be dangling, so I pass a ret value which is to be modified and used as the storage for the return value.
I have also tried just returning the enum BasicVal::Ref(&BasicVal), but run into the same mutability issues.
The example below is a much more simple version which (sort of) demonstrates the same error, just thought I'd include this context in case someone has another idea on how to implement this which wouldn't have these issues
Code (edited)
Updated playground link
Edit: I made a mistake in not needing the lifetimes of both holder and ret to explicitly be the same, so I have made an updated example for it
use std::borrow::BorrowMut;
///////////////////////////////
use std::cell::{RefCell, RefMut};
use std::collections::HashMap;
#[derive(Debug)]
enum BasicVal<'a> {
Ref(&'a BasicVal<'a>),
Val1(BasicStruct),
}
#[derive(Debug)]
struct Holder<'b> {
hold: HashMap<String, RefCell<BasicVal<'b>>>,
}
#[derive(Debug)]
struct BasicStruct {
val: i32,
}
impl<'a> BasicVal<'a> {
pub fn empty() -> Self { BasicVal::Val1(BasicStruct { val: 0 }) }
}
// must match sig of modify_val_ref
fn modify_val<'f>(holder: &'f mut Holder<'f>, mut ret: RefMut<BasicVal<'f>>) {
*ret = BasicVal::Val1(BasicStruct { val: 5 });
}
// must match sig of modify_val
fn modify_val_ref<'f>(holder: &'f mut Holder<'f>, mut ret: RefMut<BasicVal<'f>>) {
ret = holder.hold.get("reference_val").unwrap().borrow_mut();
}
fn do_modify<'f>(holder: &'f mut Holder<'f>) {
let mut v = RefCell::new(BasicVal::empty());
println!("Original {:?}", v);
modify_val(holder, v.borrow_mut());
holder.hold.insert("Data".to_string(), v);
println!("Modified {:?}", holder.hold.get("Data"));
}
pub fn test_dropborrow() {
let mut holder = Holder { hold: HashMap::new() };
holder.hold.insert(
"reference_val".to_string(),
RefCell::new(BasicVal::Val1(BasicStruct { val: 8 })),
);
do_modify(&mut holder);
}
pub fn main() {
test_dropborrow();
}
Edit: Using just the holder for a temp return value gives me a multiple mutable borrow issue, so that workaround doesn't work. I have also tried it with a RefCell with the same issue.
fn modify_val<'f>(holder: &'f mut Holder<'f>) {
holder.hold.insert("$return".to_string(), BasicVal::Val1(BasicStruct{val: 5}));
}
fn do_modify<'f>(holder: &'f mut Holder<'f>) {
modify_val(holder);
let mut v = holder.hold.remove("$return").unwrap();
holder.hold.insert("Data".to_string(), v);
println!("Modified {:?}", v);
}
Error:
935 | fn do_modify<'f>(holder: &'f mut Holder<'f>) {
| -- lifetime `'f` defined here
936 |
937 | modify_val(holder);
| ------------------
| | |
| | first mutable borrow occurs here
| argument requires that `*holder` is borrowed for `'f`
938 | let mut v = holder.hold.remove("$return").unwrap();
| ^^^^^^^^^^^ second mutable borrow occurs here
Any help is greatly appreciated!!!
Figured it out, essentially the BasicVal<'a> was causing Holder to mutably borrow itself in successive iterations of the loop, so removing the lifetime was pretty much the only solution
I am trying to rewrite an algorithm from javascript to rust. In the following code, I get borrowed value does not live long enough error at line number 17.
[dependencies]
scraper = "0.11.0"
use std::fs;
fn get_html(fname: &str) -> String {
fs::read_to_string(fname).expect("Something went wrong reading the file")
}
pub mod diff_html {
use scraper::{element_ref::ElementRef, Html};
pub struct DiffNode<'a> {
node_ref: ElementRef<'a>,
}
impl<'a> DiffNode<'a> {
fn from_html(html: &str) -> Self {
let doc = Self::get_doc(&html);
let root_element = doc.root_element().to_owned();
let diffn = Self {
node_ref: root_element,
};
diffn
}
fn get_doc(html: &str) -> Html {
Html::parse_document(html).to_owned()
}
}
pub fn diff<'a>(html1: &str, _html2: &str) -> DiffNode<'a> {
let diff1 = DiffNode::from_html(&html1);
diff1
}
}
fn main() {
//read strins
let filename1: &str = "test/test1.html";
let filename2: &str = "test/test2.html";
let html1: &str = &get_html(filename1);
let html2: &str = &get_html(filename2);
let diff1 = diff_html::diff(html1, html2);
//write html
//fs::write("test_outs/testx.html", html1).expect("unable to write file");
//written output file.
}
warning: unused variable: `diff1`
--> src\main.rs:43:9
|
43 | let diff1 = diff_html::diff(html1, html2);
| ^^^^^ help: if this is intentional, prefix it with an underscore: `_diff1`
|
= note: `#[warn(unused_variables)]` on by default
error[E0597]: `doc` does not live long enough
--> src\main.rs:17:32
|
14 | impl<'a> DiffNode<'a> {
| -- lifetime `'a` defined here
...
17 | let root_element = doc.root_element().to_owned();
| ^^^--------------------------
| |
| borrowed value does not live long enough
| assignment requires that `doc` is borrowed for `'a`
...
22 | }
| - `doc` dropped here while still borrowed
I want a detailed explanation/solution if possible.
root_element which is actually an ElementRef has reference to objects inside doc, not the actual owned object. The object doc here is created in from_html function and therefore owned by the function. Because doc is not returned, it is dropped / deleted from memory at the end of from_html function block.
ElementRef needs doc, the thing it is referencing to, to be alive when it is returned from the memory.
pub mod diff_html {
use scraper::{element_ref::ElementRef, Html};
pub struct DiffNode<'a> {
node_ref: ElementRef<'a>,
}
impl<'a> DiffNode<'a> {
fn from_html(html: &'a scraper::html::Html) -> Self {
Self {
node_ref: html.root_element(),
}
}
}
pub fn diff<'a>(html1_string: &str, _html2_string: &str) {
let html1 = Html::parse_document(&html1_string);
let diff1 = DiffNode::from_html(&html1);
// do things here
// at the end of the function, diff1 and html1 is dropped together
// this way the compiler doesn't yell at you
}
}
More or less you need to do something like this with diff function to let the HTML and ElementRef's lifetime to be the same.
This behavior is actually Rust's feature to guard values in memory so that it doesn't leak or reference not referencing the wrong memory address.
Also if you want to feel like operating detachable objects and play with reference (like java, javascript, golang) I suggest reading this https://doc.rust-lang.org/book/ch15-05-interior-mutability.html
This question already has answers here:
How to use take_while with futures::Stream?
(2 answers)
Closed 3 years ago.
I want to run an event loop in one thread and handle data from a UDP socket until another thread signals to stop work.
This is a difficult task for me, so I want to start from a simpler task:
one thread starting the event loop and waiting for another thread to signal the end:
use futures::{future, future::Future, stream::Stream, sync::mpsc};
use std::{io, io::BufRead, thread};
fn main() {
let (mut tx, rx) = mpsc::channel::<bool>(1);
let thr = thread::spawn(|| {
let mut runtime = tokio::runtime::current_thread::Runtime::new().unwrap();
runtime.spawn(
future::lazy(|| {
println!("event loop started");
Ok(())
})
.and_then(rx.take_while(|x| *x == true).into_future()),
);
runtime.run()
});
let stdin = io::stdin();
for line in stdin.lock().lines() {
let line = line.unwrap();
println!("{}", line);
if line == "exit" {
tx.try_send(false).unwrap();
break;
}
}
thr.join().unwrap().unwrap();
}
This code doesn't compile:
error[E0277]: the trait bound `bool: futures::future::Future` is not satisfied
--> src/main.rs:14:26
|
14 | .and_then(rx.take_while(|x| *x == true).into_future()),
| ^^^^^^^^^^ the trait `futures::future::Future` is not implemented for `bool`
|
= note: required because of the requirements on the impl of `futures::future::IntoFuture` for `bool`
error[E0599]: no method named `into_future` found for type `futures::stream::take_while::TakeWhile<futures::sync::mpsc::Receiver<bool>, [closure#src/main.rs:14:37: 14:51], bool>` in the current scope
--> src/main.rs:14:53
|
14 | .and_then(rx.take_while(|x| *x == true).into_future()),
| ^^^^^^^^^^^
|
= note: the method `into_future` exists but the following trait bounds were not satisfied:
`futures::stream::take_while::TakeWhile<futures::sync::mpsc::Receiver<bool>, [closure#src/main.rs:14:37: 14:51], bool> : futures::stream::Stream`
`&mut futures::stream::take_while::TakeWhile<futures::sync::mpsc::Receiver<bool>, [closure#src/main.rs:14:37: 14:51], bool> : futures::stream::Stream`
How do I fix the compilation error?
Read and understand the documentation and function signature of methods you attempt to use:
fn take_while<P, R>(self, pred: P) -> TakeWhile<Self, P, R>
where
P: FnMut(&Self::Item) -> R,
R: IntoFuture<Item = bool, Error = Self::Error>,
Self: Sized,
take_while takes a closure that returns some type that must be convertible into a future; a bool is not convertible into a future. The simplest way to do this is via future::ok:
let thr = thread::spawn(|| {
let mut runtime = tokio::runtime::current_thread::Runtime::new().unwrap();
runtime.spawn({
rx.take_while(|&x| future::ok(x))
.for_each(|x| {
println!("{}", x);
future::ok(())
})
});
runtime.run()
});
See also:
The trait bound `(): futures::Future` is not satisfied when using TcpConnectionNew
But my problem also in joining future::lazy and rx.take_while
That's an unrelated problem to what you asked about. Again, we look at the docs, this time for Future::and_then:
fn and_then<F, B>(self, f: F) -> AndThen<Self, B, F>
where
F: FnOnce(Self::Item) -> B,
B: IntoFuture<Error = Self::Error>,
Self: Sized,
Similarly to take_while, it takes a closure and the closure must return something that can be convertible into a future. Your code doesn't provide a closure.
Then look at Stream::into_future. This returns a type that implements Future and returns a tuple. The first item in the tuple is a single value from the stream, the second is the stream itself, to allow getting more values.
To get all the item and error types correct, I've make liberal use of map(drop) and map_err(drop) — you will want to do something better for your data and error handling.
runtime.spawn({
future::lazy(|| {
println!("event loop started");
Ok(())
})
.and_then(|_| {
rx.take_while(|&x| future::ok(x))
.into_future()
.map(drop)
.map_err(drop)
})
.map(drop)
});
Really, you should just use a oneshot channel; it's much simpler:
use futures::{
future::{self, Future},
sync::oneshot,
};
use std::thread;
fn main() {
let (tx, rx) = oneshot::channel();
let thr = thread::spawn(|| {
let mut runtime = tokio::runtime::current_thread::Runtime::new().unwrap();
runtime.spawn({
future::lazy(|| {
println!("event loop started");
Ok(())
})
.and_then(|_| rx.map_err(drop))
});
runtime.run()
});
let lines = ["hello", "goodbye", "exit"];
for &line in &lines {
if line == "exit" {
tx.send(()).unwrap();
break;
}
}
thr.join().unwrap().unwrap();
}
I'm interested in peeking ahead in a character stream. To my understanding, Peekable would be the way to go. I can't quite figure out how to use it.
First attempt:
fn trawl<I, E>(pk: &mut I) where I: std::iter::Peekable<Result<char, E>> {
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
_ => (),
}
}
}
fn main() {
trawl(&mut std::io::stdio::stdin().chars());
}
This fails to compile with
> rustc /tmp/main.rs
/tmp/main.rs:1:37: 1:73 error: `std::iter::Peekable` is not a trait
/tmp/main.rs:1 fn trawl<I, E>(pk: &mut I) where I: std::iter::Peekable<Result<char, E>> {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to previous error
Okay, fair enough. I don't fully understand traits yet so I try to pass an iterator in and then create a peekable version:
fn trawl<I, E>(it: &mut I) where I: Iterator<Result<char, E>> {
let mut pk = it.peekable();
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
_ => (),
}
}
}
fn main() {
trawl(&mut std::io::stdio::stdin().chars().peekable());
}
This fails with
> rustc /tmp/main.rs
/tmp/main.rs:2:18: 2:20 error: cannot move out of dereference of `&mut`-pointer
/tmp/main.rs:2 let mut pk = it.peekable();
^~
/tmp/main.rs:7:65: 7:70 error: cannot move out of dereference of `&`-pointer
/tmp/main.rs:7 (Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
^~~~~
note: in expansion of format_args!
<std macros>:2:23: 2:77 note: expansion site
<std macros>:1:1: 3:2 note: in expansion of println!
/tmp/main.rs:7:39: 7:77 note: expansion site
error: aborting due to 2 previous errors
Could someone explain:
why Peekable couldn't appear in the function type for lack of being a trait,
what the compiler means when it says 'move out of dereference of' and
how I might resolve either or both?
A third version
fn trawl<I, E>(mut it: I) where I: Iterator<Result<char, E>> {
let mut pk = it.peekable();
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
// (Some(i), ) => println!("{}", i.ok()),
_ => (),
}
}
}
fn main() {
trawl(std::io::stdio::stdin().chars().peekable());
}
This fails with:
> rustc /tmp/main.rs
/tmp/main.rs:7:65: 7:70 error: cannot move out of dereference of `&`-pointer
/tmp/main.rs:7 (Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
^~~~~
note: in expansion of format_args!
<std macros>:2:23: 2:77 note: expansion site
<std macros>:1:1: 3:2 note: in expansion of println!
/tmp/main.rs:7:39: 7:77 note: expansion site
error: aborting due to previous error
I fail to understand what rust is saying to me here, how Iterator.next would have a different return type from Peekable.peek.
Peekable is not a trait and thus cannot be used as a bound, which would suggest that it could mean one of many types. It is a single, specific, concrete type, struct Peekable<A, T>. As you have observed, it’s constructed by calling the peekable() method on an iterator, which changes it to something that is peekable.
Here’s how you’d use it if you just wanted to take an iterator:
fn trawl<I, E>(iter: I) where I: Iterator<Result<char, E>> {
let pk = pk.peekable();
…
}
Note also that the peekable() method takes self by value; you can’t take a mutable reference to an iterator there.
The alternative which is what you were aiming for but which I would be generally less inclined towards, would be to require the argument to be peekable, putting the burden onto the caller, as you had:
fn trawl<I, E>(pk: Peekable<E, I>) where I: Iterator<Result<char, E>> {
…
}
Peekable is actually a struct, not a trait. If you wanted to take a Peekable, you could define your function like this:
fn trawl<E, I>(it: Peekable<I>) where I: Iterator<Result<char, E>> {
...
}
Your second implementation is failing to compile because peek takes self by value (i.e. it consumes the iterator, returning a new one), so you can't call it through a &mut reference. Most code simply takes the iterator by value instead of by reference:
fn trawl<E, I>(it: I) where I: Iterator<Result<char, E>> {
let it = it.peekable();
...
}
If you don't want to move the iterator into a function like trawl, you can use the by_ref() method to create a new iterator that holds onto an &mut reference:
let mut my_iterator = /* whatever */;
trawl(my_iterator.by_ref());
// my_iterator is still usable here
As far as style goes, I would say that the second form is the better way to go, as the first leaks what's basically an implementation detail.
Rust has changed a bit since the previous answers. The way to do it now is:
fn trawl<I, E>(pk: Peekable<I>)
where I: Iterator<Item = Result<char, E>> {
…
}