Lifetime Confusion for iterator - rust

I am new to Rust and currently have been following Learning Rust With Entirely Too Many Linked Lists examples. Before section IterMut everything makes sense to me. Yet when implementing IterMut(in a differrnt way from the tutorial), I got totally confused by lifetime mechanism in Rust. Here is the case, first I'd just define the stack to be implemented:
pub struct List<T> {
head: Link<T>,
}
type Link<T> = Option<Box<Node<T>>>;
struct Node<T> {
elem: T,
next: Link<T>,
}
Ok then when I try to implement iterator in the following way:
pub struct IterMut<'a, T>{
this: &'a mut Link<T>
}
impl<T> List<T> {
pub fn iter_mut(&mut self) -> IterMut<T> {
IterMut {
this: &mut self.head
}
}
}
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
if let Some(node) = self.this {
Some(&mut node.elem)
} else {
None
}
}
}
It won't compile, the result says:
error: lifetime may not live long enough
impl<'a, T> Iterator for IterMut<'a, T>{
-- lifetime `'a` defined here
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
- let's call the lifetime of this reference `'1`
if let Some(node) = self.this {
Some(&mut node.elem)
^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'a`
Originally, I implement function next in this way, using as_mut and map:
// function body in the fn next
self.this.as_mut().map(|node|{
self.this = &mut node.next;
&mut node.elem
})
This also triggers compilation error (incompatible lifetime).
Therefore I wonder what is going on here, shouldn't self in fn next have the same lifetime as IterMut ('a)? And is there any workaround for this situation?

The principal problem lies in trying to take a reference to the whole head. While that reference lives, you can't hand out mutable references to anything inside it. You only need access to the Node that's inside head, though. So first, we refactor IterMut to only keep a reference into any given Node:
pub struct IterMut<'a, T>{
this: Option<&'a mut Node<T>>
}
Now, to get that out of the head we use the convenience method as_deref_mut() that Option provides. It just gives us a mutable reference to whatever was inside it (if anything):
impl<T> List<T> {
pub fn iter_mut(&mut self) -> IterMut<'_, T> {
IterMut {
this: self.head.as_deref_mut(),
}
}
}
Now, 'a is only tied to that Node and we can do what we want with it, like takeing it:
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
if let Some(node) = self.this.take() {
self.this = node.next.as_deref_mut();
Some(&mut node.elem)
} else {
None
}
}
}
And we can simplify that with a simple map call:
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
self.this.take().map(|node| {
self.this = node.next.as_deref_mut();
&mut node.elem
})
}
}

Related

Return a reference to a member iterator

I have a struct that has an iterator over a borrowed slice, and I want to create a method that returns an iterator that does some computation on the contained iterator
Basically this:
#[derive(Clone)]
struct Foo<'a> {
inner: ::std::iter::Cycle<::std::slice::Iter<'a, u8>>,
}
impl<'a> Foo<'a> {
pub fn new<T: AsRef<[u8]> + ?Sized>(vals: &'a T) -> Foo<'a> {
Self {
inner: vals.as_ref().iter().cycle(),
}
}
pub fn iter(&mut self) -> impl Iterator<Item = u8> + 'a {
self.inner.by_ref().map(Clone::clone) // simple case but this could be any computation
}
}
Which yields a cryptic error.
Here's my understanding: I need to bind the returned iterator's lifetime to the &mut self of iter() so that &mut self is only dropped when .collect() is called on the returned iterator.
BUT, changing to fn iter(&'a mut self) isn't good enough because &'a mut self now lives for the entire duration of the struct instance (meaning that calling collect() doesn't drop that reference). Which prevents something like this
#[test]
fn test(){
let mut foo = Foo::new("hello, there");
foo.iter().zip(0..4).collect::<Vec<_>>(); // call once, ok
foo.iter().zip(0..4).collect::<Vec<_>>(); // error because 2 &mut references exist at the same
}
rust playground link
The way I would do this is to make a struct just for iterating over those items. It makes things much easier.
...
pub fn iter(&mut self) -> ClonedFooIter<'a, '_> {
ClonedFooIter {
inner: &mut self.inner,
}
}
}
pub struct ClonedFooIter<'a, 'b> {
inner: &'b mut std::iter::Cycle<::std::slice::Iter<'a, u8>>,
}
impl<'a, 'b> Iterator for ClonedFooIter<'a, 'b> {
type Item = u8;
fn next(&mut self) -> Option<Self::Item> {
self.inner.next().cloned()
}
}
The idea is that the returned struct only lives till the mutable borrow of self lives.
Using a struct makes naming the iterator type easier too.

How to iterate over an Rc<RefCell<T>> that returns raw mutable references

I have a Vec<Rc<RefCell<MyStruct>>> member on a struct and I have an external library function that expects to be handed an Iterator with an Item that is a &'a mut dyn LibTrait which I have as a member inside MyStruct. I can't figure out how to get a raw &mut from out of the Rc<RefCell<MyStruct>> in my iterator even though I know the Vec member that holds it will stick around for longer than either the iterator or the function that gets passed the iterator.
The library looks like this:
struct Lib {}
trait LibTrait {
fn run(&mut self);
}
impl Lib {
pub fn lib_func<'a, Iter>(&mut self, trait_iter: Iter)
where
Iter: Iterator<Item = &'a mut dyn LibTrait>,
{
...
}
}
Here's my latest attempt where I tried to create a temp Vec to hold RefMut's to all the MyStructs so that those refs are owned for the whole time that the &mut would be inside the iterator and lib function. (This code complains about "cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements".)
struct TraitImpl {
dummy: f32,
}
impl LibTrait for TraitImpl {
fn run(&mut self) {
self.dummy += 1.0;
}
}
struct MyStruct {
my_impl: TraitImpl,
num: f32,
}
type MutStructSlice<'a> = &'a mut [&'a mut MyStruct];
struct TraitIterator<'a> {
my_structs: MutStructSlice<'a>,
index: usize,
}
impl<'a> TraitIterator<'a> {
fn new(my_structs: MutStructSlice<'a>) -> Self {
Self {
my_structs,
index: 0,
}
}
}
impl<'a> Iterator for TraitIterator<'a> {
type Item = &'a mut dyn LibTrait;
fn next(&mut self) -> Option<&'a mut dyn LibTrait> {
if self.index >= self.my_structs.len() {
return None;
}
Some(&mut self.my_structs[self.index].my_impl)
}
}
struct Data {
data: Vec<Rc<RefCell<MyStruct>>>,
lib: Lib,
}
impl Data {
fn do_stuff(&mut self) {
let mut struct_refs: Vec<RefMut<MyStruct>> = self
.data
.iter_mut()
.map(|s_rc| s_rc.borrow_mut())
.collect();
let mut structs_raw_refs: Vec<&mut MyStruct> =
struct_refs.iter_mut().map(|s_ref| &mut **s_ref).collect();
self.lib.lib_func(TraitIterator::new(&mut structs_raw_refs));
}
}
Here's the playground
Is there some way around this given that I can't change the library and I need to have Rc<RefCell<>>'s to the data?
I think you're overcomplicating this. You don't need a custom iterator, you can do this with .map():
fn do_stuff(&mut self) {
let mut struct_refs: Vec<RefMut<MyStruct>> = self.data
.iter_mut()
.map(|s_rc| s_rc.borrow_mut())
.collect();
self.lib.lib_func(struct_refs.iter_mut().map(|s_ref| &mut s_ref.my_impl as &mut dyn LibTrait));
}
You get the error you did because implementing Iterator for &mut T is inherently unsafe. See How to implement Iterator yielding mutable references.

How to implement IntoIterator for Tree<T>?

I try to implement IntoIterator for Tree, then I can use " for in Tree ", otherwise I have to write for in TreeIter{...}, but the lifetime error:
use std::iter::IntoIterator;
#[derive(Debug)]
struct Tree<T> {
data: T,
}
struct TreeIter<'a, T> {
tree: &'a Tree<T>,
count: i32,
}
impl<'a, T> IntoIterator for Tree<T> {
type Item = &'a Tree<T>;
type IntoIter = TreeIter<'a, T>;
fn into_iter(&'a self) -> Self::IntoIter {
TreeIter { tree: &self, count: 0 }
}
}
impl<'a, T> Iterator for TreeIter<'a, T>
{
type Item = &'a T;
fn next(&mut self) -> Option<Self::Item> {
self.count += 1;
if self.count > 5 {
return None;
} else {
return Some(&self.tree.data);
}
}
}
fn main() {
let tree = Tree { data: "abc" };
for v in tree {
println!("{:?}", v);
}
/*
let treeiter = TreeIter{tree: &tree, count: 0};
for (i, &v) in treeiter.enumerate() {
println!("{}: {}", i, v);
}
*/
}
got error: error[E0207]: the lifetime parameter 'a is not constrained by the impl trait
error[E0207]: the lifetime parameter `'a` is not constrained by the impl trait, self type, or predicates
--> test-iter/src/main.rs:13:6
|
13 | impl<'a, T> IntoIterator for Tree<T> {
| ^^ unconstrained lifetime parameter
Given your TreeIter structure and everything else, you don't want to consume the Tree on iteration, you just want it to reference the elements. So you want
impl<'a, T> IntoIterator for &'a Tree<T> {
// ^^^^^^^^^^^ implement for references to Trees
type Item = &'a T;
// ^^^^^ this needs to match Iterator::Item for TreeIter
type IntoIter = TreeIter<'a, T>;
fn into_iter(self) -> Self::IntoIter {
// ^^^^ self is a &'a Tree<T>
TreeIter { tree: self, count: 0 }
}
}
Then you can use it in a for loop like so:
let tree = Tree { data: "abc" };
for v in &tree {
// ^ only iterate via reference
println!("{:?}", v);
}
See it on the playground. See this Q&A for the difference between for _ in x vs for _ in &x.
into_iter() is meant to wholesale take ownership of the collection. The collection is moved into the iterator and is consumed by iteration, not borrowed by reference and simply looked at. That behavior is provided by iter() and iter_mut(). Your code is thus conceptually flawed, and the compiler error reflects that: into_iter doesn't take a borrow a collection with whatever lifetime it already has; it takes a collection and ends its lifetime right then and there. There's no 'a for you to impl<'a> over. Implement it with the right idea in mind and it works
struct IntoIter<T> { // e.g. same convention as std::vec::IntoIter
tree: Tree<T>,
pos: i32,
}
// due to your dummy implementation, we need T: Copy, but a real implementation shouldn't need it
impl<T: Copy> IntoIterator for Tree<T> {
type Item = T; // why would iterating over a tree give you trees?
type IntoIter = IntoIter<T>;
fn into_iter(self) -> Self::IntoIter {
IntoIter { tree: self, pos: 0 }
}
}
impl<T: Copy> Iterator for IntoIter<T> {
type Item = T; // iterating over an IntoIter should give values moved out of the container (in this case we're copying the same value a few times and pretending they were moved)
fn next(&mut self) -> Option<Self::Item> {
if self.pos < 5 {
self.pos += 1;
Some(self.tree.data)
} else {
None
}
}
}
fn main() {
for i in (Tree { data: 1 }) { println!("{}", i) }
}
Note that it is conventional to provide IntoIterator for borrows of the collection as well. Again, into_iter() should be seen as consuming it's argument... but "consuming" a borrow doesn't actually consume what it's referring to. This would use your iterator type, but note that this trait implementation is not what we're using in the above main.
struct Iter<'a, T> {
tree: &'a Tree<T>,
pos: i32
}
impl<'a, T> Iterator for Iter<'a, T> {
type Item = &'a T;
fn next(&mut self) -> Option<Self::Item> { todo!() }
}
impl<'a, T> IntoIterator for &'a Tree<T> {
type Item = &'a T;
type IntoIter = Iter<'a, T>;
fn into_iter(self) -> Iter<'a, T> { todo!() }
}
struct IterMut<'a, T> {
tree: &'a mut Tree<T>,
pos: i32
}
impl<'a, T> Iterator for IterMut<'a, T> {
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> { todo!() }
}
impl<'a, T> IntoIterator for &'a mut Tree<T> {
type Item = &'a mut T;
type IntoIter = IterMut<'a, T>;
fn into_iter(self) -> IterMut<'a, T> { todo!() }
}
These would be called if you did
fn main() {
let mut tree = Tree { data: 1 };
for i in &tree { println!("{}", i) } // IntoIter for borrow
for i in &mut tree { println!("{}", i) } // IntoIter for mutable borrow
}

How do I return the result of get_mut from a HashMap or a HashSet?

I'm trying to wrap a HashMap, as defined below, to return a mutable reference from a HashMap:
use std::{collections::HashMap, marker::PhantomData};
struct Id<T>(usize, PhantomData<T>);
pub struct IdCollection<T>(HashMap<Id<T>, T>);
impl<'a, T> std::ops::Index<Id<T>> for &'a mut IdCollection<T> {
type Output = &'a mut T;
fn index(&mut self, id: &'a Id<T>) -> Self::Output {
self.0.get_mut(id).unwrap()
}
}
And the resulting error:
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 54:5...
--> src/id_container.rs:54:5
|
54 | / fn index(&mut self, id: &'a Id<T>) -> Self::Output {
55 | | self.0.get_mut(id).unwrap()
56 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/id_container.rs:55:9
|
55 | self.0.get_mut(id).unwrap()
| ^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 52:6...
--> src/id_container.rs:52:6
|
52 | impl<'a, T> std::ops::Index<Id<T>> for &'a mut IdCollection<T> {
| ^^
= note: ...so that the types are compatible:
expected std::ops::Index<id_container::Id<T>>
found std::ops::Index<id_container::Id<T>>
Why can't the compiler extend the lifetime of the get_mut? The IdCollection would then be borrowed mutably.
Note that I tried using a std::collections::HashSet<IdWrapper<T>> instead of a HashMap:
struct IdWrapper<T> {
id: Id<T>,
t: T,
}
Implementing the proper borrow etc. so I can use the Id<T> as a key.
However, HashSet doesn't offer a mutable getter (which makes sense since you don't want to mutate what's used for your hash). However in my case only part of the object should be immutable. Casting a const type to a non-const is UB so this is out of the question.
Can I achieve what I want? Do I have to use some wrapper such as a Box? Although I'd rather avoid any indirection...
EDIT
Ok I'm an idiot. First I missed the IndexMut instead of the Index, and I forgot the & when specifying the Self::Output in the signature.
Here's my full code below:
pub struct Id<T>(usize, PhantomData<T>);
impl<T> std::fmt::Display for Id<T> {
fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
write!(f, "{}", self.0)
}
}
impl<T> Hash for Id<T> {
fn hash<H: Hasher>(&self, state: &mut H) {
self.0.hash(state);
}
}
impl<T> PartialEq for Id<T> {
fn eq(&self, o: &Self) -> bool {
self.0 == o.0
}
}
impl<T> Eq for Id<T> {}
pub struct IdCollection<T>(HashMap<Id<T>, T>);
impl<'a, T> IntoIterator for &'a IdCollection<T> {
type Item = (&'a Id<T>, &'a T);
type IntoIter = std::collections::hash_map::Iter<'a, Id<T>, T>;
fn into_iter(self) -> Self::IntoIter {
self.0.iter()
}
}
impl<'a, T> IntoIterator for &'a mut IdCollection<T> {
type Item = (&'a Id<T>, &'a mut T);
type IntoIter = std::collections::hash_map::IterMut<'a, Id<T>, T>;
fn into_iter(self) -> Self::IntoIter {
self.0.iter_mut()
}
}
impl<T> std::ops::Index<Id<T>> for IdCollection<T> {
type Output = T;
fn index(&self, id: Id<T>) -> &Self::Output {
self.0.get(&id).unwrap()
}
}
impl<T> std::ops::IndexMut<Id<T>> for IdCollection<T> {
fn index_mut(&mut self, id: Id<T>) -> &mut Self::Output {
self.0.get_mut(&id).unwrap()
}
}
impl<T> std::ops::Index<&Id<T>> for IdCollection<T> {
type Output = T;
fn index(&self, id: &Id<T>) -> &Self::Output {
self.0.get(id).unwrap()
}
}
impl<T> std::ops::IndexMut<&Id<T>> for IdCollection<T> {
fn index_mut(&mut self, id: &Id<T>) -> &mut Self::Output {
self.0.get_mut(id).unwrap()
}
}
If I understand correctly what you try to achieve, then I have to tell you, that it is a bit more complex than you originally thought it would be.
First of all, you have to realise, that if you like to use a HashMap then the type of the key required to be hashable and comparable. Therefore the generic type parameter T in Id<T> has to be bound to those traits in order to make Id hashable and comparable.
The second thing you need to understand is that there are two different traits to deal with the indexing operator: Index for immutable data access, and IndexMut for mutable one.
use std::{
marker::PhantomData,
collections::HashMap,
cmp::{
Eq,
PartialEq,
},
ops::{
Index,
IndexMut,
},
hash::Hash,
};
#[derive(PartialEq, Hash)]
struct Id<T>(usize, PhantomData<T>)
where T: PartialEq + Hash;
impl<T> Eq for Id<T>
where T: PartialEq + Hash
{}
struct IdCollection<T>(HashMap<Id<T>, T>)
where T: PartialEq + Hash;
impl<T> Index<Id<T>> for IdCollection<T>
where T: PartialEq + Hash
{
type Output = T;
fn index(&self, id: Id<T>) -> &Self::Output
{
self.0.get(&id).unwrap()
}
}
impl<T> IndexMut<Id<T>> for IdCollection<T>
where T: PartialEq + Hash
{
fn index_mut(&mut self, id: Id<T>) -> &mut Self::Output
{
self.0.get_mut(&id).unwrap()
}
}
fn main()
{
let mut i = IdCollection(HashMap::new());
i.0.insert(Id(12, PhantomData), 99i32);
println!("{:?}", i[Id(12, PhantomData)]);
i[Id(12, PhantomData)] = 54i32;
println!("{:?}", i[Id(12, PhantomData)]);
}
It may seem a bit surprising, but IndexMut is not designed to insert an element into the collection but to actually modify an existing one. That's the main reason why HashMap does not implement IndexMut -- and that's also the reason why the above example uses the HashMap::insert method to initially place the data. As you can see, later on, when the value is already available we can modify it via the IdCollection::index_mut.

How to have a struct field with the same mutability as the parent struct?

I'm trying to wrap a slice in a struct so that I will be able to instantiate the struct mutably or immutably. Here's a minimal example:
use std::ops::{ Index, IndexMut };
struct Test<'a, T: 'a> {
inner: &'a[T]
}
impl<'a, T: 'a> Test<'a, T> {
fn new (inner: &'a[T]) -> Self { Test { inner: inner } }
}
impl<'a, T> Index<usize> for Test<'a, T> {
type Output = T;
fn index (&self, i: usize) -> &T { &self.inner[i] }
}
impl<'a, T> IndexMut<usize> for Test<'a, T> {
fn index_mut (&mut self, i: usize) -> &mut T { &mut self.inner[i] }
}
fn main() {
let store = [0; 3];
let test = Test::new (&store);
println!("{}", test[1]);
let mut mut_store = [0; 3];
let mut mut_test = Test::new (&mut mut_store);
mut_test[1] = 42;
println!("{}", mut_test[1]);
}
This doesn't compile: "cannot borrow immutable indexed content self.inner[..] as mutable".
I could get it to compile by changing the definition of inner to be of type &'a mut[T], but then inner is mutable even when I don't need it to be (in the above example, I must then declare store as mutable too even though test is immutable).
Is there a way to make it so that the mutability of inner follows the mutability of the Test instance?
As well said in the question, this code compiles:
struct Test<'a, A: 'a> {
inner: &'a mut A,
}
fn main() {
let t = Test { inner: &mut 5i32 };
*t.inner = 9;
}
It is indeed possible to mutate a borrowed element, even when the borrowing content is immutable. This is a case where you must choose your guarantees, while keeping in mind that the mutability of a binding is always independent of the borrowed content's mutability.
Right now, I can think of two possible solutions: you can encapsulate the borrowed content over methods that depend on self's mutability (Playground, will no longer compile):
impl<'a, A: 'a> Test<'a, A> {
fn inner(&self) -> &A {
self.inner
}
fn inner_mut(&mut self) -> &mut A {
self.inner
}
}
Although you still need to keep a borrow to mutable content, it can no longer be mutated from an immutable binding of Test. If you also need it to point to immutable content, you should consider having two different structs (Playground):
struct Test<'a, A: 'a> {
inner: &'a A,
}
impl<'a, A: 'a> Test<'a, A> {
fn inner(&self) -> &A {
self.inner
}
}
struct TestMut<'a, A: 'a> {
inner: &'a mut A,
}
impl<'a, A: 'a> TestMut<'a, A> {
fn inner(&self) -> &A {
self.inner
}
fn inner_mut(&mut self) -> &mut A {
self.inner
}
}
There is a third option: to keep both kinds of borrows exclusively with an enum. At this point however, using the borrowed content as mutable requires run-time checks.

Resources