Chain iterators to references of different lifetimies - rust

I want to build a recursive function for traversing a tree in Rust. The function should always get the next element and an iterator over references to the ancestor elements.
For the iterator over ancestor elements, one could in principle use the chain and once methods. Consider the following simple example, where the tree is jsut a Vec (for the purpose of this demonstration):
fn proceed<'a, I>(mut remaining: Vec<String>, ancestors: I)
where
I: Iterator<Item = &'a String> + Clone,
{
if let Some(next) = remaining.pop() {
let next_ancestors = ancestors.chain(std::iter::once(&next));
proceed(remaining, next_ancestors);
}
}
Playground
This fails to compile because &next has a shorter lifetime than 'a:
error[E0597]: `next` does not live long enough
--> src/lib.rs:6:62
|
1 | fn proceed<'a, I>(mut remaining: Vec<String>, ancestors: I)
| -- lifetime `'a` defined here
...
6 | let next_ancestors = ancestors.chain(std::iter::once(&next));
| --------------------------------^^^^^--
| | |
| | borrowed value does not live long enough
| argument requires that `next` is borrowed for `'a`
7 | proceed(remaining, next_ancestors);
8 | }
| - `next` dropped here while still borrowed
I tried to overcome this by adding an explicit second lifetime 'b: 'a and forcing an explicit reference by something like let next_ref: &'b String = &next, but that yields a (different) error message as well.
One solution I came up with was to call map as follows:
let next_ancestors = ancestors.map(|r| r).chain(std::iter::once(&next));
As pointed out by #trentcl, this doesn't actually solve the problem, as the compiler then gets stuck in an infinite loop when compiling proceed for all the nested Chains when one actually tries to call the function.

The pieces of solution are already around, just to summarize:
As you already know, using map(|r| r) "decouples" the lifetime requirement of ancestors
from the lifetime of &next.
As already stated in the comments, fixing
the infinite recursion is a matter to change ancestors into a trait object.
fn proceed<'a>(mut remaining: Vec<String>, ancestors: &mut dyn Iterator<Item = &'a String>) {
if let Some(next) = remaining.pop() {
let mut next_ancestors = ancestors.map(|r| r).chain(std::iter::once(&next));
proceed(remaining, &mut next_ancestors);
}
}
fn main() {
let v = vec!["a".to_string(), "b".to_string()];
proceed(v, &mut std::iter::empty());
}

Related

How to shorten the lifetime of each item in IntoIterator?

The following is the code:
struct A;
fn f<'a, I>(default_args: I)
where
I: IntoIterator<Item = &'a A>,
{
{
let a1 = A;
let more_args = [&a1];
{
let i = default_args.into_iter();
let i = i.chain(more_args);
// I will use `i` here.
}
}
// I will NOT use `*_args` here.
}
The following is error:
error[E0597]: `a1` does not live long enough
--> src\main.rs:27:26
|
21 | fn f<'a, I>(default_args: I)
| -- lifetime `'a` defined here
...
27 | let more_args = [&a1];
| ^^^ borrowed value does not live long enough
...
30 | let i = i.chain(more_args);
| ------------------ argument requires that `a1` is borrowed for `'a`
...
33 | }
| - `a1` dropped here while still borrowed
For more information about this error, try `rustc --explain E0597`.
I want to get a new default_args_2 from default_args, where the Item inside default_args_2 has a shorter lifetime, as long as it valid inside the function.
You can add a little (almost) noop .map(|v| v):
let i = default_args.into_iter().map(|v| v);
The problem was that the iterator item types have to match, and therefore the chained iterator has to also yield &'a T, which it does not (it yields a shorter lifetime).
Since you don't really need to use the items for 'a, the trick of the noop map() is to insert a subtyping point: Now, the closure inside map() (conceptually) converts the &'a T, to a &'shorter_lifetime T, which is fine since 'a: 'shorter_lifetime, but now the chained iterator can produce the desired item type.
Putting it differently, you cannot convert impl Iterator<Item = &'a T> into impl Iterator<Item = &'shorter_lifetime T> (as it may not be covariant over the lifetime), but you can convert &'a T to &'shorter_lifetime T, and we use that fact because from the outside we have impl Iterator<Item = &'a T>, but inside map()'s closure we got &'a T.

Closure arguments: passing a function that mutates the inner variables

The idea is to to have one closure (change_x in this case) that captures state (x in this case) that takes a function as its parameter(alterer) that would dictate how the inner state changes.
pub fn plus17(h: & u64) -> u64 {
*h + 17
}
pub fn main() {
let x = 0; //take x by reference
let mut change_x = move |alterer: &dyn FnOnce(&u64) ->u64 | alterer(&x) ;
change_x(&mut plus17);
println!("{}", x);
}
I can't seem to get the types right however:
error[E0161]: cannot move a value of type dyn for<'r> FnOnce(&'r u64) -> u64: the size of dyn for<'r> FnOnce(&'r u64) -> u64 cannot be statically determined
--> playground/src/main.rs:19:69
|
19 | let mut increment_x = move |alterer: &dyn FnOnce(&u64) ->u64 | alterer(&x) ;
| ^^^^^^^
error[E0507]: cannot move out of `*alterer` which is behind a shared reference
--> playground/src/main.rs:19:69
|
19 | let mut increment_x = move |alterer: &dyn FnOnce(&u64) ->u64 | alterer(&x) ;
| ^^^^^^^ move occurs because `*alterer` has type `dyn for<'r> FnOnce(&'r u64) -> u64`, which does not implement the `Copy` trait
I'm not sure if i'm justified in putting the dyn where i put it, it was a compiler's suggestion and im not really sure why i have to put it there. Is it because the alterer can be of arbitrary size despite the input/return type of &u64->u64?
I have also tried to make alterer a FnMut as opposed to FnOnce, but im also pretty shaky as to their distinction and the fact that a given alterer would run only once (at the moment of invocation by outer closure change_x) seemed reasonable.
FnOnce needs an owned self. Thus alterer cannot be FnOnce, because it is not owned but a reference.
You can either make it &dyn Fn or &mut dyn FnMut (I'd recommend going with FnMut), or take Box<dyn FnOnce>.

Cannot borrow `*self` as mutable more than once at a time; in combination with HashMap

This is my code:
use std::collections::HashMap;
struct Foo {
pub map : HashMap<i32, String>
}
impl Foo {
fn foo(&mut self, x: &String) -> i32 {
// I'm planning to use/modify "x" here and also modify "self"
42
}
fn bar(&mut self) -> i32 {
let x = self.map.get_mut(&1).unwrap();
self.foo(x)
}
}
I'm getting:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:13:9
|
12 | let x = self.map.get_mut(&1).unwrap();
| -------------------- first mutable borrow occurs here
13 | self.foo(x)
| ^^^^^^^^^-^
| | |
| | first borrow later used here
| second mutable borrow occurs here
What's going on?
Modifying self and x here breaks memory safety (at least in the general situation, which is what Rust must deal with). Consider the following implementation of foo which is allowed by your signature (fixing &String to &str):
fn foo(&mut self, x: &str) -> i32 {
self.map.clear();
println!("{}", x);
42
}
But you're calling this with x being a reference to something inside of self.map. So x could be destroyed by the time it's used. That's invalid, and Rust can't prove you won't do that, because you said you might. (Kevin Anderson provides a helpful comment below if you're coming from a GC language like C# where "reference" has a different meaning.)
How to fix this depends on what you're really trying to do, though one approach would be to clone the string so it cannot be destroyed:
fn bar(&mut self) -> i32 {
let x = self.map.get(&1).unwrap().clone(); // <== now you have a copy
self.foo(&x)
}
Note this got rid of the get_mut(). It's unclear what that was for. If you need an exclusive (mut) reference into the map, then you'll need to do that separately, and you can't do that directly while also holding an exclusive reference to self for the reasons above. Remember that mut means "exclusive access," not "mutable." A side effect of having exclusive access is that mutation is allowed.
If you really need something along these lines, you need to wrap your values (String) in Arc so that you can maintain reference counts and have shared ownership. But I would first try to redesign your algorithm to avoid this.

How to create an iterator that allows mapping indices to mutable items in a slice [duplicate]

This question already has answers here:
How do I write an iterator that returns references to itself?
(4 answers)
Simultaneous mutable access to arbitrary indices of a large vector that are guaranteed to be disjoint
(5 answers)
How can I create my own data structure with an iterator that returns mutable references?
(1 answer)
Closed 3 years ago.
I have a slice of items and a slice of indices into the first slice, essentially giving me a sub-group of items that I want to modify. To iterate over the items and manipulate them I can create the following helper function.
fn process_things_by_index_loop<T>(things: &mut [T], indices: &[usize], f: &dyn Fn(&mut T)) {
for &ix in indices {
let t = &mut things[ix];
f(t);
}
}
This works fine, but I would actually just like to get an iterator over the items so I can apply, e.g., a filter to them before further processing.
It looks like this could just be a mapping of indices to items like so:
fn iterate_thigns<'a, T>(
things: &'a mut [T],
indices: &'a [usize],
) -> impl Iterator<Item = &'a mut T> {
indices.iter().map(|&ix| -> &mut T { &mut things[ix] })
}
This can't work of course, since I could have the same index twice in the indices slice and when collecting the iterator I would create two mutable references to the same item. So this gives appropriately a life-time error:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
--> src\main.rs:4:47
|
4 | indices.iter().map(|&ix|-> &mut T { &mut things[ix]})
| ^^^^^^^^^^
|
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
--> src\main.rs:4:47
|
4 | indices.iter().map(|&ix|-> &mut T { &mut things[ix]})
| ^^^^^^^^^^
|
note: first, the lifetime cannot outlive the lifetime '_ as defined on the body at 4:24...
--> src\main.rs:4:24
|
4 | indices.iter().map(|&ix|-> &mut T { &mut things[ix]})
| ^^^^^^^^^^^^^^^
note: ...so that reference does not outlive borrowed content
--> src\main.rs:4:47
|
4 | indices.iter().map(|&ix|-> &mut T { &mut things[ix]})
| ^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 3:19...
--> src\main.rs:3:19
|
3 | fn iterate_thigns<'a, T>(things: &'a mut [T], indices: &'a [usize]) -> impl Iterator<Item=&'a mut T>{
| ^^
= note: ...so that the types are compatible:
expected &mut T
found &'a mut T
This also means the return type probably can't be a simple Iterator at all.
This issues seems also related to this blog post from 2013 on "Iterators yielding mutable references", which states that no standard solutions for this pattern exist yet.
There is also a related question asking about mutable multi threaded access to the content of a vector, but I am really interested in a single threaded, iterator-like solution, which might even allow indices to repeat.
So, is there something that enables an iterator-like interface for sequential iteration of the items indicated by the indices?

When can the Rust compiler not prove that borrows are disjoint?

In section 3.2 of the Nomicon, under the heading "liveness", it says
However it's often the case that Rust isn't sufficiently smart to
prove that multiple borrows are disjoint.
What is an example where the Rust compiler cannot prove that they are disjoint? Will this ever occur in a tuple struct?
The key is in the previous sentence:
Rust explicitly enables [reborrowing into multiple mutable references] to be done with disjoint struct fields, because disjointness can be statically proven
Outside of this case, the compiler cannot tell that two borrows are disjoint. In practice, this means that the compiler cannot tell that borrows resulting from a function call will be disjoint.
struct Thing {
a: i32,
b: i32,
}
fn example_works(thing: &mut Thing) {
let a = &mut thing.a;
let b = &mut thing.b;
}
fn get_a(thing: &mut Thing) -> &mut i32 {
&mut thing.a
}
fn get_b(thing: &mut Thing) -> &mut i32 {
&mut thing.b
}
fn example_doesnt_work(thing: &mut Thing) {
let a = get_a(thing);
let b = get_b(thing);
println!("{}, {}", a, b);
}
error[E0499]: cannot borrow `*thing` as mutable more than once at a time
--> src/lib.rs:26:19
|
25 | let a = get_a(thing);
| ----- first mutable borrow occurs here
26 | let b = get_b(thing); // cannot borrow `*thing` as mutable more than once at a time
| ^^^^^ second mutable borrow occurs here
27 | println!("{}, {}", a, b);
| - first borrow later used here
Will this ever occur in a tuple struct?
Not specifically because it's a tuple struct, but yes, it can happen for the same reasons. If you obtain a borrow from a function call, you will get the same problem as a "traditional" struct.

Resources