Mutable iterator [duplicate] - rust

This question already has answers here:
How can I create my own data structure with an iterator that returns mutable references?
(1 answer)
How to implement Iterator yielding mutable references [duplicate]
(2 answers)
Closed 2 months ago.
I am trying to write a mutable iterator over a vector, but I am unable to figure out what the compiler is trying to tell me. Here is my implementation
struct IterMut<'a> {
vec: &'a mut Vec<u32>,
index: usize,
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
let item = if self.index < self.vec.len() {
Some(&mut self.vec[self.index])
} else {
None
};
self.index += 1;
item
}
}
And the compiler error is:
error: lifetime may not live long enough
--> src/main.rs:60:9
|
48 | impl<'a> Iterator for IterMut<'a> {
| -- lifetime `'a` defined here
...
51 | fn next(&mut self) -> Option<Self::Item> {
| - let's call the lifetime of this reference `'1`
...
60 | item
| ^^^^ associated function was supposed to return data with lifetime `'a` but it is returning data with lifetime `'1`
The non mutable version compiles and works.
struct Iter<'a> {
vec: &'a Vec<u32>,
index: usize,
}
impl<'a> Iterator for Iter<'a> {
type Item = &'a u32;
fn next(&mut self) -> Option<Self::Item> {
let item = if self.index < self.vec.len() {
Some(&self.vec[self.index])
} else {
None
};
self.index += 1;
item
}
}
Here is a playground link with the code to try out.

The problem is that your IterMut holds on to the elements already passed.
It can do so only by reborrowing the item but this now means that the item isn't borrowed from the original Vec any more but from the IterMut instead so it does not live long enough. (The reborrow only lives as long as the IterMut, not as long as the original Vec.
In nightly or once take_first_mut stabilizes you can do this:
#![feature(slice_take)]
struct IterMut<'a> {
vec: &'a mut [u32],
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
self.vec.take_first_mut()
}
}
Note I switched from &mut Vec<u32> to a slice because you have to be able to 'release' the items which is not easy with a Vec
As #ChayimFriedman pointed out on stable you can do it with mem::take in the meantime:
struct IterMut<'a> {
vec: &'a mut [u32],
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
let (first, rem) = std::mem::take(&mut self.vec).split_first_mut()?;
self.vec = rem;
Some(first)
}
}

Related

How to specify Rust lifetime when returning item that lives longer than iterator

I'm trying to write an iterator that returns string slices from byte array. I'm having trouble specifying the lifetimes to indicate that the slice can live longer than the iterator.
A simplified test case would be:
struct StrStream<'a> {
data: &'a str,
}
impl<'a> StrStream<'a> {
fn new(data: &'a str) -> Self {
Self { data }
}
fn cstr(&'a mut self, len: usize) -> Option<&'a str> {
Some(self.data.get(0..len).unwrap())
}
}
struct MyIterator<'a> {
i: u32,
stream: &'a mut StrStream<'a>,
}
impl<'a> MyIterator<'a> {
fn new(stream: &'a mut StrStream<'a>) -> Self {
Self { i: 1, stream }
}
}
impl<'a> Iterator for MyIterator<'a> {
type Item = &'a str;
fn next(&mut self) -> Option<Self::Item> {
self.i += 1;
if self.i < 10 {
return Some(self.stream.cstr(self.i as usize).unwrap());
}
return None;
}
}
#[test]
fn iterates_tokens() {
let mut stream = StrStream::new("abcdefghijklmnopqrstuvwxyz");
let myiter = MyIterator::new(&mut stream);
for item in myiter {
println!(">>>{:?}", item);
}
}
This fails to compile with the error below. Any suggestions how to solve this problem?
error: lifetime may not live long enough
--> xxx/src/tests.rs:32:25
|
26 | impl<'a> Iterator for MyIterator<'a> {
| -- lifetime `'a` defined here
...
29 | fn next(&mut self) -> Option<Self::Item> {
| - let's call the lifetime of this reference `'1`
...
32 | return Some(self.stream.cstr(self.i as usize).unwrap());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'a`

How do you return a reference with a lifetime from an Iterator implementation in Rust? [duplicate]

I have a struct that lazily load data into an inner vector (but for the example this is ommited). Then I am implementing IntoIterator and Iterator for the IntoIterator type:
struct EntryOwned(u32);
struct Entry<'a>(&'a u32);
impl<'a> EntryOwned {
fn to_entry(&'a self) -> Entry<'a> {
Entry(&self.0)
}
}
struct LazyStruct {
cache: Vec<EntryOwned>,
}
impl<'a> LazyStruct {
fn new(data: Vec<EntryOwned>) -> LazyStruct {
Self {
cache: data,
}
}
fn get_entry(&'a self, index: usize) -> Option<Entry<'a>> {
match self.cache.get(index) {
Some(entry_owned) => Some(entry_owned.to_entry()),
None => None,
}
}
}
impl<'a> IntoIterator for &'a mut LazyStruct {
type Item = Entry<'a>;
type IntoIter = LazyStructIter<'a>;
fn into_iter(self) -> Self::IntoIter {
LazyStructIter {
inner: self,
current_index: 0,
}
}
}
struct LazyStructIter<'a> {
inner: &'a mut LazyStruct,
current_index: usize,
}
impl<'a> LazyStructIter<'a> {
fn next_item(&'a mut self) -> Option<Entry<'a>> {
if self.current_index > self.inner.cache.len() {
return None;
}
let ret = self.inner.get_entry(self.current_index);
self.current_index += 1;
ret
}
}
impl<'a> Iterator for LazyStructIter<'a> {
type Item = Entry<'a>;
fn next(&mut self) -> Option<Self::Item> {
self.next_item()
}
}
Which yields me to a lifetime issue:
Compiling playground v0.0.1 (/playground)
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/main.rs:64:14
|
64 | self.next_item()
| ^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 63:5...
--> src/main.rs:63:5
|
63 | / fn next(&mut self) -> Option<Self::Item> {
64 | | self.next_item()
65 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:64:9
|
64 | self.next_item()
| ^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 60:6...
--> src/main.rs:60:6
|
60 | impl<'a> Iterator for LazyStructIter<'a> {
| ^^
note: ...so that the types are compatible
--> src/main.rs:64:14
|
64 | self.next_item()
| ^^^^^^^^^
= note: expected `&mut LazyStructIter<'_>`
found `&mut LazyStructIter<'a>`
The lifetime of the references should be bind to the LazyStruct, but I cannot change the Iter trait do not accept any lifetime specifier.
I have already check some answers to similar issues:
Iterator returning items by reference, lifetime issue
How do I write an iterator that returns references to itself?
EDIT: The Entry is a more complex data structure and it is not possible to copy it. I have to bind it to a specific lifetime since it contains references to things.
One of them points that the iterator should hold a reference to the original collection instead of owning it since it is not allowed to return a reference with the self lifetime. But I could't make it work.
So, how should I play the references here?
Here is the playground example
You can't.
You require that each item returned by next() has a reference with a lifetime tied to the
existence of the LazyStructIter iterator.
This it is not possible with the Iterator interface:
pub trait Iterator {
type Item;
fn next(&mut self) -> Option<Self::Item>;
But it would be possible with a trait defined as:
trait FakeIterator<'a> {
type Item;
fn next(&'a mut self) -> std::option::Option<Self::Item>;
}
impl<'a> FakeIterator<'a> for LazyStructIter<'a> {
type Item = Entry<'a>;
fn next(&'a mut self) -> Option<Self::Item> {
self.next_item()
}
}
Tha capability to have a kind of iterator which returns items borrowing from self is the goal of
RFC 1598 - GAT.
See also this article for a collection of workarounds about this topic.
Generally you don't need to specify lifetimes in rust unless 1 of two things is true, the reference is bound to an owner that isn't known, or a reference that is static. Rust says it expected '_ lifetime, which means it understands the ownership and borrows on its own.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c54f5f2f24e635dcfaa0fcbf69fa4ce0
struct EntryOwned(u32);
struct Entry(u32);
impl EntryOwned {
fn to_entry(&self) -> Entry {
Entry(self.0)
}
}
struct LazyStruct {
cache: Vec<EntryOwned>,
}
impl LazyStruct {
fn new(data: Vec<EntryOwned>) -> LazyStruct {
Self {
cache: data,
}
}
fn get_entry(&self, index: usize) -> Option<Entry> {
match self.cache.get(index) {
Some(entry_owned) => Some(entry_owned.to_entry()),
None => None,
}
}
}
impl IntoIterator for LazyStruct {
type Item = Entry;
type IntoIter = LazyStructIter;
fn into_iter(self) -> Self::IntoIter {
LazyStructIter {
inner: self,
current_index: 0,
}
}
}
struct LazyStructIter {
inner: LazyStruct,
current_index: usize,
}
impl LazyStructIter {
fn next_item(&mut self) -> Option<Entry> {
if self.current_index > self.inner.cache.len() {
return None;
}
let ret = self.inner.get_entry(self.current_index);
self.current_index += 1;
ret
}
}
impl Iterator for LazyStructIter {
type Item = Entry;
fn next(&mut self) -> Option<Self::Item> {
self.next_item()
}
}
fn main() {
// let json_data = r#"{"item_1" : 10, "item_2" : 100, "item_3" : 1000}"#;
// if let Ok(data) = serde_json::from_str::<A>(json_data) {
// println!("{:?}", data);
// }
// else if let Ok(data) = serde_json::from_str::<B>(json_data) {
// println!("{:?}", data);
// }
for _ in 0..0 {
println!("foo");
}
}
After removing the lifetime specifications, it compiles fine and runs.

Returning References from Struct Method with Lifetimes

I am trying this code:
struct ByteIter<'a> {
remainder: &'a mut [u8],
index: usize,
}
impl<'a> ByteIter<'a> {
fn new(remainder: &'a mut [u8]) -> ByteIter<'a> {
ByteIter{remainder, index: 0}
}
fn next(&'a mut self) -> Option<&'a mut u8> {
if self.index >= self.remainder.len() {
None
} else {
let mut byte = &mut self.remainder[self.index];
self.index += 1;
Some(byte)
}
}
}
fn main() {
let mut a = [ 0x51, 0x52, 0x53, 0x54];
let mut bytes = ByteIter::new(&mut a);
{
let byte_1 = bytes.next();
}
let byte_2 = bytes.next();
}
Now, as I understand, the byte_1 was borrowed from bytes. But it's lifetime has already expired. Still when I compile this, I see the following error:
29 | let byte_2 = bytes.next();
| ^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
What is the first mutable borrow here? Shouldn't it be released when byte_1 goes out of scope?
Importantly, what should I do to fix this?
With impl<'a> ByteIter<'a>, you declare that 'a is the lifetime used in the structure, ie, it's the lifetime of the underlying slice.
Now, when you say fn next(&'a mut self) -> Option<&'a mut u8> {, you're reusing the same 'a, and you say it's the same than the returned mutable reference. You're saying that the returned lifetime is the same than the ByteIter struct content.
Remove the additional lifetime constraint and the compiler will be free to compute the appropriate lifetime:
fn next(& mut self) -> Option<& mut u8> {
if self.index >= self.remainder.len() {
None
} else {
let mut byte = &mut self.remainder[self.index];
self.index += 1;
Some(byte)
}
}

How can I fix "cannot infer an appropriate lifetime for autoref" when implementing an iterator that returns mutable references?

I'm trying to write a mutable iterator for a linked list called Thread where each element implements Block.
trait Block<'a> {
fn next(&mut self) -> Option<&'a mut (dyn Block<'a> + 'a)> {
None
}
}
pub struct Thread<'a> {
head: Box<dyn Block<'a> + 'a>,
}
impl<'a> Thread<'a> {
fn iter_mut(&mut self) -> ThreadIterator<'a> {
ThreadIterator {
next: Some(self.head.as_mut()),
}
}
}
pub struct ThreadIterator<'a> {
next: Option<&'a mut (dyn Block<'a> + 'a)>,
}
impl<'a> Iterator for ThreadIterator<'a> {
type Item = &'a mut (dyn Block<'a> + 'a);
fn next(&mut self) -> Option<&'a mut (dyn Block<'a> + 'a)> {
self.next.take().map(|mut block| {
self.next = block.next();
block
})
}
}
Compiling this will output the error:
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/lib.rs:14:34
|
14 | next: Some(self.head.as_mut()),
| ^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 12:5...
--> src/lib.rs:12:5
|
12 | / fn iter_mut(&mut self) -> ThreadIterator<'a> {
13 | | ThreadIterator {
14 | | next: Some(self.head.as_mut()),
15 | | }
16 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:14:24
|
14 | next: Some(self.head.as_mut()),
| ^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 11:6...
--> src/lib.rs:11:6
|
11 | impl<'a> Thread<'a> {
| ^^
note: ...so that the types are compatible
--> src/lib.rs:14:24
|
14 | next: Some(self.head.as_mut()),
| ^^^^^^^^^^^^^^^^^^
= note: expected `dyn Block<'_>`
found `dyn Block<'a>`
This is why I need the 'a requirement for all Blocks (they are borrowing a Runtime):
struct Runtime {}
struct ExampleBlock<'a> {
runtime: &'a Runtime,
next: Box<dyn Block<'a> + 'a>,
}
impl<'a> Block<'a> for ExampleBlock<'a> {
fn next(&mut self) -> Option<&'a mut (dyn Block<'a> + 'a)> {
Some(self.next.as_mut())
}
}
The first thing I tried was removing the mutable requirement from all references. Same errors.
I think the error is telling me that self.head.as_mut() is outliving self.head, so I must ensure that the lifetime of that reference is shorter than Thread<'a>. I thought I fulfilled this requirement with the 'a lifetime for ThreadIterator<'a>. In other words, you can't possibly drop Thread before ThreadIterator, right?
Edit:
I changed Block to a struct to simplify the code, though I need it to be a trait in the end.
struct Block {}
impl<'a> Block {
fn next(&mut self) -> Option<&'a mut Block> {
None
}
}
pub struct Thread {
head: Block,
}
impl<'a> Thread {
fn iter_mut(&mut self) -> ThreadIterator<'a> {
ThreadIterator {
next: Some(&mut self.head),
}
}
}
pub struct ThreadIterator<'a> {
next: Option<&'a mut Block>,
}
impl<'a> Iterator for ThreadIterator<'a> {
type Item = &'a mut Block;
fn next(&mut self) -> Option<&'a mut Block> {
self.next.take().map(|mut block| {
self.next = block.next();
block
})
}
}
It is based off of https://rust-unofficial.github.io/too-many-lists/second-iter-mut.html.
The answer to `cannot infer an appropriate lifetime for autoref due to conflicting requirements` but can't change anything due to trait definition constraints was to introduce a Option for the iterator, which I have done. Lifetime parameter problem in custom iterator over mutable references and Reimplementation of LinkedList: IterMut not compiling didn't answer my question, though I have a hard time connecting my code to theirs.
I finally found something that does work:
pub struct Block {}
impl<'a> Block {
fn next(&mut self) -> Option<&'a mut Block> {
None
}
}
pub struct Thread {
head: Block,
}
impl Thread {
fn iter_mut(&mut self) -> ThreadIterator<'_> { // The lifetime here is changed
ThreadIterator {
next: Some(&mut self.head),
}
}
}
pub struct ThreadIterator<'a> {
next: Option<&'a mut Block>,
}
impl<'a> Iterator for ThreadIterator<'a> {
type Item = &'a mut Block;
fn next(&mut self) -> Option<&'a mut Block> {
self.next.take().map(|mut block| {
self.next = block.next();
block
})
}
}
I'm having a hard time applying this to the original code, because there might be two different lifetimes, one for the iterator and one for the trait.
Answered by u/quixotrykd/:
The compiler seems to be choking here because it doesn't know how the lifetime on &mut self and ThreadIterator relate. As such, it has no way to guarantee that &mut self lives at least as long as the underlying borrow in ThreadIterator. Looking at your code here, that would be line 12 (note I've fixed your mistake in the above link).
You need to tell the compiler that the lifetime on the output ThreadIterator is the same as the input &mut self (or technically that it lasts at least as long, though you very likely want the same lifetime here), otherwise the compiler has no way to ensure that the "self" that's borrowed by &mut self stick around as long as the ThreadIterator's borrow. Looking at the rules for lifetime elision here, we can see that this doesn't apply to your code, and as such, you need to manually specify the lifetime on &mut self (otherwise it generates another, unrelated, anonymous lifetime, as indicated in the compiler's error message).
The fixed code:
pub trait Block<'a> { // Make public to resolve unrelated error
fn next(&mut self) -> Option<&'a mut (dyn Block<'a> + 'a)> {
None
}
}
pub struct Thread<'a> {
head: Box<dyn Block<'a> + 'a>,
}
impl<'a> Thread<'a> {
fn iter_mut(&'a mut self) -> ThreadIterator<'a> { // Add lifetime to &self
ThreadIterator {
next: Some(self.head.as_mut()),
}
}
}
pub struct ThreadIterator<'a> {
next: Option<&'a mut (dyn Block<'a> + 'a)>,
}
impl<'a> Iterator for ThreadIterator<'a> {
type Item = &'a mut (dyn Block<'a> + 'a);
fn next(&mut self) -> Option<&'a mut (dyn Block<'a> + 'a)> {
self.next.take().map(|mut block| {
self.next = block.next();
block
})
}
}

Iterator lifetime issue when returning references to inner collection

I have a struct that lazily load data into an inner vector (but for the example this is ommited). Then I am implementing IntoIterator and Iterator for the IntoIterator type:
struct EntryOwned(u32);
struct Entry<'a>(&'a u32);
impl<'a> EntryOwned {
fn to_entry(&'a self) -> Entry<'a> {
Entry(&self.0)
}
}
struct LazyStruct {
cache: Vec<EntryOwned>,
}
impl<'a> LazyStruct {
fn new(data: Vec<EntryOwned>) -> LazyStruct {
Self {
cache: data,
}
}
fn get_entry(&'a self, index: usize) -> Option<Entry<'a>> {
match self.cache.get(index) {
Some(entry_owned) => Some(entry_owned.to_entry()),
None => None,
}
}
}
impl<'a> IntoIterator for &'a mut LazyStruct {
type Item = Entry<'a>;
type IntoIter = LazyStructIter<'a>;
fn into_iter(self) -> Self::IntoIter {
LazyStructIter {
inner: self,
current_index: 0,
}
}
}
struct LazyStructIter<'a> {
inner: &'a mut LazyStruct,
current_index: usize,
}
impl<'a> LazyStructIter<'a> {
fn next_item(&'a mut self) -> Option<Entry<'a>> {
if self.current_index > self.inner.cache.len() {
return None;
}
let ret = self.inner.get_entry(self.current_index);
self.current_index += 1;
ret
}
}
impl<'a> Iterator for LazyStructIter<'a> {
type Item = Entry<'a>;
fn next(&mut self) -> Option<Self::Item> {
self.next_item()
}
}
Which yields me to a lifetime issue:
Compiling playground v0.0.1 (/playground)
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/main.rs:64:14
|
64 | self.next_item()
| ^^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 63:5...
--> src/main.rs:63:5
|
63 | / fn next(&mut self) -> Option<Self::Item> {
64 | | self.next_item()
65 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:64:9
|
64 | self.next_item()
| ^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 60:6...
--> src/main.rs:60:6
|
60 | impl<'a> Iterator for LazyStructIter<'a> {
| ^^
note: ...so that the types are compatible
--> src/main.rs:64:14
|
64 | self.next_item()
| ^^^^^^^^^
= note: expected `&mut LazyStructIter<'_>`
found `&mut LazyStructIter<'a>`
The lifetime of the references should be bind to the LazyStruct, but I cannot change the Iter trait do not accept any lifetime specifier.
I have already check some answers to similar issues:
Iterator returning items by reference, lifetime issue
How do I write an iterator that returns references to itself?
EDIT: The Entry is a more complex data structure and it is not possible to copy it. I have to bind it to a specific lifetime since it contains references to things.
One of them points that the iterator should hold a reference to the original collection instead of owning it since it is not allowed to return a reference with the self lifetime. But I could't make it work.
So, how should I play the references here?
Here is the playground example
You can't.
You require that each item returned by next() has a reference with a lifetime tied to the
existence of the LazyStructIter iterator.
This it is not possible with the Iterator interface:
pub trait Iterator {
type Item;
fn next(&mut self) -> Option<Self::Item>;
But it would be possible with a trait defined as:
trait FakeIterator<'a> {
type Item;
fn next(&'a mut self) -> std::option::Option<Self::Item>;
}
impl<'a> FakeIterator<'a> for LazyStructIter<'a> {
type Item = Entry<'a>;
fn next(&'a mut self) -> Option<Self::Item> {
self.next_item()
}
}
Tha capability to have a kind of iterator which returns items borrowing from self is the goal of
RFC 1598 - GAT.
See also this article for a collection of workarounds about this topic.
Generally you don't need to specify lifetimes in rust unless 1 of two things is true, the reference is bound to an owner that isn't known, or a reference that is static. Rust says it expected '_ lifetime, which means it understands the ownership and borrows on its own.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=c54f5f2f24e635dcfaa0fcbf69fa4ce0
struct EntryOwned(u32);
struct Entry(u32);
impl EntryOwned {
fn to_entry(&self) -> Entry {
Entry(self.0)
}
}
struct LazyStruct {
cache: Vec<EntryOwned>,
}
impl LazyStruct {
fn new(data: Vec<EntryOwned>) -> LazyStruct {
Self {
cache: data,
}
}
fn get_entry(&self, index: usize) -> Option<Entry> {
match self.cache.get(index) {
Some(entry_owned) => Some(entry_owned.to_entry()),
None => None,
}
}
}
impl IntoIterator for LazyStruct {
type Item = Entry;
type IntoIter = LazyStructIter;
fn into_iter(self) -> Self::IntoIter {
LazyStructIter {
inner: self,
current_index: 0,
}
}
}
struct LazyStructIter {
inner: LazyStruct,
current_index: usize,
}
impl LazyStructIter {
fn next_item(&mut self) -> Option<Entry> {
if self.current_index > self.inner.cache.len() {
return None;
}
let ret = self.inner.get_entry(self.current_index);
self.current_index += 1;
ret
}
}
impl Iterator for LazyStructIter {
type Item = Entry;
fn next(&mut self) -> Option<Self::Item> {
self.next_item()
}
}
fn main() {
// let json_data = r#"{"item_1" : 10, "item_2" : 100, "item_3" : 1000}"#;
// if let Ok(data) = serde_json::from_str::<A>(json_data) {
// println!("{:?}", data);
// }
// else if let Ok(data) = serde_json::from_str::<B>(json_data) {
// println!("{:?}", data);
// }
for _ in 0..0 {
println!("foo");
}
}
After removing the lifetime specifications, it compiles fine and runs.

Resources