Say I have the following,
type EpollEventCallback<'a> = FnMut(c_int) + Send + Sync + 'a;
struct EpollFdEventHandler<'a> {
on_readable: Option<Box<EpollEventCallback<'a>>>,
on_writable: Option<Box<EpollEventCallback<'a>>>,
}
// Map from c_int -> EpollFdEventHandler.
type EpollEventHandlerMap<'a> = collections::HashMap<c_int, EpollFdEventHandler<'a>>;
fn add_fd_handler
<'a, T: Fn(bool, &'a mut EpollFdEventHandler<'a>)>(
map: &'a mut EpollEventHandlerMap<'a>,
fd: c_int,
adder: T)
{
let mut hash_entry: hash_map::Entry<'a, _, _> = map.entry(fd);
match hash_entry {
hash_map::Entry::Occupied(ref mut occ_e) => {
let entry: &mut EpollFdEventHandler<'a> = occ_e.get_mut();
adder(false, entry);
},
hash_map::Entry::Vacant(vac_e) => {
/*
adder(
true,
vac_e.insert(EpollFdEventHandler {
on_readable: None,
on_writable: None,
}),
);
*/
}
};
}
add_fd_handler is supposed to be a helper function for adding an "FD handler"; here, it's going to get passed a closure (adder) that will set either on_readable or on_writable, depending on which handler is being added. add_fd_handler's job is simply doing the hash table lookup, and inserting an empty entry if required. However:
src/event_loop.rs:85:35: 85:48 error: `(hash_entry:std::collections::hash::map::Occupied).0` does not live long enough
src/event_loop.rs:85 hash_map::Entry::Occupied(ref mut occ_e) => {
^~~~~~~~~~~~~
src/event_loop.rs:82:1: 101:2 note: reference must be valid for the lifetime 'a as defined on the block at 82:0...
src/event_loop.rs:82 {
src/event_loop.rs:83 let mut hash_entry: hash_map::Entry<'a, _, _> = map.entry(fd);
src/event_loop.rs:84 match hash_entry {
src/event_loop.rs:85 hash_map::Entry::Occupied(ref mut occ_e) => {
src/event_loop.rs:86 let entry: &mut EpollFdEventHandler<'a> = occ_e.get_mut();
src/event_loop.rs:87 adder(false, entry);
...
src/event_loop.rs:83:67: 101:2 note: ...but borrowed value is only valid for the block suffix following statement 0 at 83:66
src/event_loop.rs:83 let mut hash_entry: hash_map::Entry<'a, _, _> = map.entry(fd);
src/event_loop.rs:84 match hash_entry {
src/event_loop.rs:85 hash_map::Entry::Occupied(ref mut occ_e) => {
src/event_loop.rs:86 let entry: &mut EpollFdEventHandler<'a> = occ_e.get_mut();
src/event_loop.rs:87 adder(false, entry);
src/event_loop.rs:88 },
The error about occ_e only shows up if I try to use it with adder(false, entry)! Rust claims occ_e "does not live long enough", but it's only being used right there in that branch of the match, so how can that be?
My best guess presently is that the closure's second arg, as &'a mut is what's the issue here; my reference in occ_e isn't 'a, it's something shorter (the unspecified lifetime on hash_entry, I think, but I don't know how to notate that).
Let the compiler infer the proper lifetime instead:
fn add_fd_handler
<T: Fn(bool, &mut EpollFdEventHandler)>(
map: &mut EpollEventHandlerMap,
fd: c_int,
adder: T)
{
let mut hash_entry = map.entry(fd);
match hash_entry {
hash_map::Entry::Occupied(ref mut occ_e) => {
let entry = occ_e.get_mut();
adder(false, entry);
},
hash_map::Entry::Vacant(vac_e) => {
/*
adder(
true,
vac_e.insert(EpollFdEventHandler {
on_readable: None,
on_writable: None,
}),
);
*/
}
};
}
The problem is that you're letting the caller determine a lifetime for the callback, but you then invoke the callback with a mutable reference to a local variable. The caller couldn't possibly know about the lifetime of that local variable, so the compiler assumes that 'a must outlive the current function. Yet, entry does not outlive the function, which is why you get an error.
The declaration T: Fn(bool, &mut EpollFdEventHandler) is equivalent to T: for<'a, 'b> Fn(bool, &'a mut EpollFdEventHandler<'b>). The for keyword in this context allows you to declare that T must implement Fn for any value of the specified lifetime parameters. This is only valid for lifetime parameters, because different lifetime parameters do not cause multiple versions of a function to be defined, unlike for type parameters.
Related
I am trying to pass around a HashMap which stores values through a set of nested enums/structs. The problem of multiple mutability happens during iteration, even all references should be dropped.
The general idea is to have a vector of values, iterate through them and simplify them, keeping track of them within the HashMap. There are two stages of simplification.
The general flow looks something like
run(Vec<ComplexVal>)
-for each val->
val.fix_complex(holder)
-for each `smp` SimpleVal in val->
basicval = Simplifier::step(smp, holder)
holder.insert("name", basicval)
But the problem is that the holder is borrowed mutably in each stage, and there isn't supposed to be any reference from the ComplexVal to the holder and since the borrowchecker doesn't like multiple borrows, it fails.
Full playground snippet: here
It happens in this snippet:
pub fn run(&mut self, mut vals: Vec<ComplexVal>) {
let mut holder = Holder{hold:HashMap::new()};
// .. setup holder code omitted
let len = vals.len();
for _ in 0..len {
let mut val = vals.remove(0); // remove from vec, should drop after running
println!("Running {:?}", val);
match val {
ComplexVal::Cmplx1(mut c) => {
c.fix_complex(&mut holder)
},
//... more cases of different types of values omitted for simplicity
}
// val *should* be dropped here, and therefore the mutable borrow of holder?
}
println!("Holder: {:?}", holder);
}
}
The only thing I can think of is that it somehow is related to the BasicVal::Ref(&BasicVal) value when created.
I need to return a reference of type &BasicVal so I can't use a regular fn() -> &BasicVal as the reference would be dangling, so I pass a ret value which is to be modified and used as the storage for the return value.
I have also tried just returning the enum BasicVal::Ref(&BasicVal), but run into the same mutability issues.
The example below is a much more simple version which (sort of) demonstrates the same error, just thought I'd include this context in case someone has another idea on how to implement this which wouldn't have these issues
Code (edited)
Updated playground link
Edit: I made a mistake in not needing the lifetimes of both holder and ret to explicitly be the same, so I have made an updated example for it
use std::borrow::BorrowMut;
///////////////////////////////
use std::cell::{RefCell, RefMut};
use std::collections::HashMap;
#[derive(Debug)]
enum BasicVal<'a> {
Ref(&'a BasicVal<'a>),
Val1(BasicStruct),
}
#[derive(Debug)]
struct Holder<'b> {
hold: HashMap<String, RefCell<BasicVal<'b>>>,
}
#[derive(Debug)]
struct BasicStruct {
val: i32,
}
impl<'a> BasicVal<'a> {
pub fn empty() -> Self { BasicVal::Val1(BasicStruct { val: 0 }) }
}
// must match sig of modify_val_ref
fn modify_val<'f>(holder: &'f mut Holder<'f>, mut ret: RefMut<BasicVal<'f>>) {
*ret = BasicVal::Val1(BasicStruct { val: 5 });
}
// must match sig of modify_val
fn modify_val_ref<'f>(holder: &'f mut Holder<'f>, mut ret: RefMut<BasicVal<'f>>) {
ret = holder.hold.get("reference_val").unwrap().borrow_mut();
}
fn do_modify<'f>(holder: &'f mut Holder<'f>) {
let mut v = RefCell::new(BasicVal::empty());
println!("Original {:?}", v);
modify_val(holder, v.borrow_mut());
holder.hold.insert("Data".to_string(), v);
println!("Modified {:?}", holder.hold.get("Data"));
}
pub fn test_dropborrow() {
let mut holder = Holder { hold: HashMap::new() };
holder.hold.insert(
"reference_val".to_string(),
RefCell::new(BasicVal::Val1(BasicStruct { val: 8 })),
);
do_modify(&mut holder);
}
pub fn main() {
test_dropborrow();
}
Edit: Using just the holder for a temp return value gives me a multiple mutable borrow issue, so that workaround doesn't work. I have also tried it with a RefCell with the same issue.
fn modify_val<'f>(holder: &'f mut Holder<'f>) {
holder.hold.insert("$return".to_string(), BasicVal::Val1(BasicStruct{val: 5}));
}
fn do_modify<'f>(holder: &'f mut Holder<'f>) {
modify_val(holder);
let mut v = holder.hold.remove("$return").unwrap();
holder.hold.insert("Data".to_string(), v);
println!("Modified {:?}", v);
}
Error:
935 | fn do_modify<'f>(holder: &'f mut Holder<'f>) {
| -- lifetime `'f` defined here
936 |
937 | modify_val(holder);
| ------------------
| | |
| | first mutable borrow occurs here
| argument requires that `*holder` is borrowed for `'f`
938 | let mut v = holder.hold.remove("$return").unwrap();
| ^^^^^^^^^^^ second mutable borrow occurs here
Any help is greatly appreciated!!!
Figured it out, essentially the BasicVal<'a> was causing Holder to mutably borrow itself in successive iterations of the loop, so removing the lifetime was pretty much the only solution
Consider this pseudo-pseudocode. I have quite some difficulties with the keyarg argument below,
type MapSet<K,V> = HashMap<K,HashSet<V>>;
fn contract<K,V,V1>(left: &MapSet<K,V>, right: &MapSet<V,V1>,
out: &mut MapSet<K,V1>,
keyarg: Option(__something_iterable__) {
let keys = match keyarg {
Some(keys) => {
keys
},
None => {
left.keys()
},
}
for &k in keys {
// ... do stuff ...
}
}
The function is used sometimes like this,
let x: MapSet<u32,String>;
let y: MapSet<String,u32>;
let out: MapSet<u32,u32>;
let kk: HashSet<u32>;
// ... snip ...
contract(x,y,out,kk);
And, at other times like that,
let x: MapSet<u32,String>;
let y: MapSet<String,u32>;
let out: MapSet<u32,u32>;
contract(x,y,out,None);
In other words, I am sometimes using keyarg argument to pass an Option with a reference to a HashSet containing the keys I want to iterate over and at other times I want to iterate over all the keys contained in left argument, so the keyarg becomes simply None.
But, so far I have always ended up in a problem with the match which complains that None leads to Keys object and Some branch to HashSet (type mismatch error).
My question is how to define the keyarg argument so that the branches of match are compatible with each other. That is, I want to express the fact that the variable keys is just something one can iterate over to the compiler.
Unless keyarg will always match your MapSet iterator, and in your pseudo code it doesn't, you'll need keys to be a trait object. The iterator for MapSet<K,V> has the trait Iterator<Item = &K>, so you can match it into a Box like so:
let keys: Box<dyn Iterator<Item = &K>> = match keyarg {
Some(keys) => Box::new(...),
None => Box::new(left.keys()),
}
As far as determining what keyarg should be, the generic way would be to accept any matching iterable like so:
fn contract<'a, K, V, V1, I: IntoIterator<Item = &'a K>>(
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
left: &'a MapSet<K, V>,
right: &MapSet<V, V1>,
out: &mut MapSet<K, V1>,
keyarg: Option<I>,
) {
let keys: Box<dyn Iterator<Item = &K>> = match keyarg {
Some(keys) => Box::new(keys.into_iter()),
None => Box::new(left.keys()),
};
// ...
}
And this is very ergonomic for passing a HashSet but causes a problem for the None case:
// This works
contract(&x, &y, &mut out, Some(&kk));
// This fails with error: "type annotations needed, cannot infer type for type parameter `I`"
contract(&x, &y, &mut out, None);
So you'd have to annotate None with some dummy iterator type (like Box<...> or std::iter::Empty or something) which isn't ideal.
Instead you can make keyarg the boxed iterator directly:
fn contract_1<'a, K, V, V1>(
left: &'a MapSet<K, V>,
right: &MapSet<V, V1>,
out: &mut MapSet<K, V1>,
keyarg: Option<Box<dyn Iterator<Item = &'a K> + 'a>>,
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
) {
let keys: Box<dyn Iterator<Item = &K>> = match keyarg {
Some(keys) => keys,
None => Box::new(left.keys()),
};
// ...
}
Which can be used like so:
contract(&x, &y, &mut out, Some(Box::new(kk.iter())));
contract(&x, &y, &mut out, None);
I would like to create a callback function that has access to the argument instanceContext, but I am not sure how to fix the code.
The failing code is instanceContext.instances.len().
I have tried using lifetimes, move, Rc, Arc, Box, RefCell, but nothing seems to work.
The goal is to have multiple Instances which will have callbacks that will make it possible for one Instance to call/modify another Instance, through the InstanceContext.
During my experimentation I've been getting the following errors:
may outlive borrowed value
borrowed value does not live long enough
closure is FnOnce because it moves the variable
this closure implements FnOnce, not Fn
use wasmtime::*;
struct InstanceContext {
id: i32,
instances: Vec<Instance>,
}
fn main() {
let store = Store::new(&Engine::default());
let mut instanceContext = InstanceContext { id: 5, instances: Vec::new() };
let _callback = createCallback(&store, &mut instanceContext);
}
fn createCallback(store: &Store, instanceContext: &mut InstanceContext) -> wasmtime::Func {
let f = Func::wrap(&store, || {
println!("Number of instances: {}", instanceContext.instances.len());
});
f
}
You don't provide the compiler error message or an actual reproduction case so it's hard to know whether the diagnostic achieved by just thinking about things is the correct one, but to me it looks like this would be the solution:
fn main() {
let store = Store::new(&Engine::default());
let mut instanceContext = Rc::new(RefCell::new(InstanceContext { id: 5, instances: Vec::new() }));
let _callback = createCallback(&store, instanceContext.clone());
}
fn createCallback(store: &Store, instanceContext: Rc<RefCell<InstanceContext>>) -> wasmtime::Func {
let f = Func::wrap(&store, move || {
println!("Number of instances: {}", instanceContext.borrow().instances.len());
});
f
}
The problem I want to solve is:
Given the recursively nested data structure, eg. a JSON tree, and a path pointing to (possibly non-existent) element inside it, return the mutable reference of the element, that's the closest to given path.
Example: if we have JSON document in form { a: { b: { c: "foo" } } } and a path a.b.d, we want to have a mutable pointer to value stored under key "b".
This is a code snippet, what I've got so far:
use std::collections::HashMap;
enum Json {
Number(i64),
Bool(bool),
String(String),
Array(Vec<Json>),
Object(HashMap<String, Json>)
}
struct Pointer<'a, 'b> {
value: &'a mut Json,
path: Vec<&'b str>,
position: usize
}
/// Return a mutable pointer to JSON element having shared
/// the nearest common path with provided JSON.
fn nearest_mut<'a,'b>(obj: &'a mut Json, path: Vec<&'b str>) -> Pointer<'a,'b> {
let mut i = 0;
let mut current = obj;
for &key in path.iter() {
match current {
Json::Array(array) => {
match key.parse::<usize>() {
Ok(index) => {
match array.get_mut(index) {
Some(inner) => current = inner,
None => break,
}
},
_ => break,
}
} ,
Json::Object(map) => {
match map.get_mut(key) {
Some(inner) => current = inner,
None => break
}
},
_ => break,
};
i += 1;
}
Pointer { path, position: i, value: current }
}
The problem is that this doesn't pass through Rust's borrow checker, as current is borrowed as mutable reference twice, once inside match statement and once at the end of the function, when constructing the pointer method.
I've tried a different approaches, but not figured out how to achieve the goal (maybe going the unsafe path).
I completely misread your question and I owe you an apology.
You cannot do it in one pass - you're going to need to do a read-only pass to find the nearest path (or exact path), and then a read-write pass to actually extract the reference, or pass a mutator function in the form of a closure.
I've implemented the two-pass method for you. Do note that it is still pretty performant:
fn nearest_mut<'a, 'b>(obj: &'a mut Json, path: Vec<&'b str>) -> Pointer<'a, 'b> {
let valid_path = nearest_path(obj, path);
exact_mut(obj, valid_path).unwrap()
}
fn exact_mut<'a, 'b>(obj: &'a mut Json, path: Vec<&'b str>) -> Option<Pointer<'a, 'b>> {
let mut i = 0;
let mut target = obj;
for token in path.iter() {
i += 1;
// borrow checker gets confused about `target` being mutably borrowed too many times because of the loop
// this once-per-loop binding makes the scope clearer and circumvents the error
let target_once = target;
let target_opt = match *target_once {
Json::Object(ref mut map) => map.get_mut(*token),
Json::Array(ref mut list) => match token.parse::<usize>() {
Ok(t) => list.get_mut(t),
Err(_) => None,
},
_ => None,
};
if let Some(t) = target_opt {
target = t;
} else {
return None;
}
}
Some(Pointer {
path,
position: i,
value: target,
})
}
/// Return a mutable pointer to JSON element having shared
/// the nearest common path with provided JSON.
fn nearest_path<'a, 'b>(obj: &'a Json, path: Vec<&'b str>) -> Vec<&'b str> {
let mut i = 0;
let mut target = obj;
let mut valid_paths = vec![];
for token in path.iter() {
// borrow checker gets confused about `target` being mutably borrowed too many times because of the loop
// this once-per-loop binding makes the scope clearer and circumvents the error
let target_opt = match *target {
Json::Object(ref map) => map.get(*token),
Json::Array(ref list) => match token.parse::<usize>() {
Ok(t) => list.get(t),
Err(_) => None,
},
_ => None,
};
if let Some(t) = target_opt {
target = t;
valid_paths.push(*token)
} else {
return valid_paths;
}
}
return valid_paths
}
The principle is simple - I reused the method I wrote in my initial question in order to get the nearest valid path (or exact path).
From there, I feed that straight into the function that I had in my original answer, and since I am certain the path is valid (from the prior function call) I can safely unwrap() :-)
I'm interested in peeking ahead in a character stream. To my understanding, Peekable would be the way to go. I can't quite figure out how to use it.
First attempt:
fn trawl<I, E>(pk: &mut I) where I: std::iter::Peekable<Result<char, E>> {
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
_ => (),
}
}
}
fn main() {
trawl(&mut std::io::stdio::stdin().chars());
}
This fails to compile with
> rustc /tmp/main.rs
/tmp/main.rs:1:37: 1:73 error: `std::iter::Peekable` is not a trait
/tmp/main.rs:1 fn trawl<I, E>(pk: &mut I) where I: std::iter::Peekable<Result<char, E>> {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to previous error
Okay, fair enough. I don't fully understand traits yet so I try to pass an iterator in and then create a peekable version:
fn trawl<I, E>(it: &mut I) where I: Iterator<Result<char, E>> {
let mut pk = it.peekable();
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
_ => (),
}
}
}
fn main() {
trawl(&mut std::io::stdio::stdin().chars().peekable());
}
This fails with
> rustc /tmp/main.rs
/tmp/main.rs:2:18: 2:20 error: cannot move out of dereference of `&mut`-pointer
/tmp/main.rs:2 let mut pk = it.peekable();
^~
/tmp/main.rs:7:65: 7:70 error: cannot move out of dereference of `&`-pointer
/tmp/main.rs:7 (Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
^~~~~
note: in expansion of format_args!
<std macros>:2:23: 2:77 note: expansion site
<std macros>:1:1: 3:2 note: in expansion of println!
/tmp/main.rs:7:39: 7:77 note: expansion site
error: aborting due to 2 previous errors
Could someone explain:
why Peekable couldn't appear in the function type for lack of being a trait,
what the compiler means when it says 'move out of dereference of' and
how I might resolve either or both?
A third version
fn trawl<I, E>(mut it: I) where I: Iterator<Result<char, E>> {
let mut pk = it.peekable();
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
// (Some(i), ) => println!("{}", i.ok()),
_ => (),
}
}
}
fn main() {
trawl(std::io::stdio::stdin().chars().peekable());
}
This fails with:
> rustc /tmp/main.rs
/tmp/main.rs:7:65: 7:70 error: cannot move out of dereference of `&`-pointer
/tmp/main.rs:7 (Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
^~~~~
note: in expansion of format_args!
<std macros>:2:23: 2:77 note: expansion site
<std macros>:1:1: 3:2 note: in expansion of println!
/tmp/main.rs:7:39: 7:77 note: expansion site
error: aborting due to previous error
I fail to understand what rust is saying to me here, how Iterator.next would have a different return type from Peekable.peek.
Peekable is not a trait and thus cannot be used as a bound, which would suggest that it could mean one of many types. It is a single, specific, concrete type, struct Peekable<A, T>. As you have observed, it’s constructed by calling the peekable() method on an iterator, which changes it to something that is peekable.
Here’s how you’d use it if you just wanted to take an iterator:
fn trawl<I, E>(iter: I) where I: Iterator<Result<char, E>> {
let pk = pk.peekable();
…
}
Note also that the peekable() method takes self by value; you can’t take a mutable reference to an iterator there.
The alternative which is what you were aiming for but which I would be generally less inclined towards, would be to require the argument to be peekable, putting the burden onto the caller, as you had:
fn trawl<I, E>(pk: Peekable<E, I>) where I: Iterator<Result<char, E>> {
…
}
Peekable is actually a struct, not a trait. If you wanted to take a Peekable, you could define your function like this:
fn trawl<E, I>(it: Peekable<I>) where I: Iterator<Result<char, E>> {
...
}
Your second implementation is failing to compile because peek takes self by value (i.e. it consumes the iterator, returning a new one), so you can't call it through a &mut reference. Most code simply takes the iterator by value instead of by reference:
fn trawl<E, I>(it: I) where I: Iterator<Result<char, E>> {
let it = it.peekable();
...
}
If you don't want to move the iterator into a function like trawl, you can use the by_ref() method to create a new iterator that holds onto an &mut reference:
let mut my_iterator = /* whatever */;
trawl(my_iterator.by_ref());
// my_iterator is still usable here
As far as style goes, I would say that the second form is the better way to go, as the first leaks what's basically an implementation detail.
Rust has changed a bit since the previous answers. The way to do it now is:
fn trawl<I, E>(pk: Peekable<I>)
where I: Iterator<Item = Result<char, E>> {
…
}