I am trying this code:
struct ByteIter<'a> {
remainder: &'a mut [u8],
index: usize,
}
impl<'a> ByteIter<'a> {
fn new(remainder: &'a mut [u8]) -> ByteIter<'a> {
ByteIter{remainder, index: 0}
}
fn next(&'a mut self) -> Option<&'a mut u8> {
if self.index >= self.remainder.len() {
None
} else {
let mut byte = &mut self.remainder[self.index];
self.index += 1;
Some(byte)
}
}
}
fn main() {
let mut a = [ 0x51, 0x52, 0x53, 0x54];
let mut bytes = ByteIter::new(&mut a);
{
let byte_1 = bytes.next();
}
let byte_2 = bytes.next();
}
Now, as I understand, the byte_1 was borrowed from bytes. But it's lifetime has already expired. Still when I compile this, I see the following error:
29 | let byte_2 = bytes.next();
| ^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
What is the first mutable borrow here? Shouldn't it be released when byte_1 goes out of scope?
Importantly, what should I do to fix this?
With impl<'a> ByteIter<'a>, you declare that 'a is the lifetime used in the structure, ie, it's the lifetime of the underlying slice.
Now, when you say fn next(&'a mut self) -> Option<&'a mut u8> {, you're reusing the same 'a, and you say it's the same than the returned mutable reference. You're saying that the returned lifetime is the same than the ByteIter struct content.
Remove the additional lifetime constraint and the compiler will be free to compute the appropriate lifetime:
fn next(& mut self) -> Option<& mut u8> {
if self.index >= self.remainder.len() {
None
} else {
let mut byte = &mut self.remainder[self.index];
self.index += 1;
Some(byte)
}
}
Related
This question already has answers here:
How can I create my own data structure with an iterator that returns mutable references?
(1 answer)
How to implement Iterator yielding mutable references [duplicate]
(2 answers)
Closed 2 months ago.
I am trying to write a mutable iterator over a vector, but I am unable to figure out what the compiler is trying to tell me. Here is my implementation
struct IterMut<'a> {
vec: &'a mut Vec<u32>,
index: usize,
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
let item = if self.index < self.vec.len() {
Some(&mut self.vec[self.index])
} else {
None
};
self.index += 1;
item
}
}
And the compiler error is:
error: lifetime may not live long enough
--> src/main.rs:60:9
|
48 | impl<'a> Iterator for IterMut<'a> {
| -- lifetime `'a` defined here
...
51 | fn next(&mut self) -> Option<Self::Item> {
| - let's call the lifetime of this reference `'1`
...
60 | item
| ^^^^ associated function was supposed to return data with lifetime `'a` but it is returning data with lifetime `'1`
The non mutable version compiles and works.
struct Iter<'a> {
vec: &'a Vec<u32>,
index: usize,
}
impl<'a> Iterator for Iter<'a> {
type Item = &'a u32;
fn next(&mut self) -> Option<Self::Item> {
let item = if self.index < self.vec.len() {
Some(&self.vec[self.index])
} else {
None
};
self.index += 1;
item
}
}
Here is a playground link with the code to try out.
The problem is that your IterMut holds on to the elements already passed.
It can do so only by reborrowing the item but this now means that the item isn't borrowed from the original Vec any more but from the IterMut instead so it does not live long enough. (The reborrow only lives as long as the IterMut, not as long as the original Vec.
In nightly or once take_first_mut stabilizes you can do this:
#![feature(slice_take)]
struct IterMut<'a> {
vec: &'a mut [u32],
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
self.vec.take_first_mut()
}
}
Note I switched from &mut Vec<u32> to a slice because you have to be able to 'release' the items which is not easy with a Vec
As #ChayimFriedman pointed out on stable you can do it with mem::take in the meantime:
struct IterMut<'a> {
vec: &'a mut [u32],
}
impl<'a> Iterator for IterMut<'a> {
type Item = &'a mut u32;
fn next(&mut self) -> Option<Self::Item> {
let (first, rem) = std::mem::take(&mut self.vec).split_first_mut()?;
self.vec = rem;
Some(first)
}
}
I'm trying to write an iterator that returns string slices from byte array. I'm having trouble specifying the lifetimes to indicate that the slice can live longer than the iterator.
A simplified test case would be:
struct StrStream<'a> {
data: &'a str,
}
impl<'a> StrStream<'a> {
fn new(data: &'a str) -> Self {
Self { data }
}
fn cstr(&'a mut self, len: usize) -> Option<&'a str> {
Some(self.data.get(0..len).unwrap())
}
}
struct MyIterator<'a> {
i: u32,
stream: &'a mut StrStream<'a>,
}
impl<'a> MyIterator<'a> {
fn new(stream: &'a mut StrStream<'a>) -> Self {
Self { i: 1, stream }
}
}
impl<'a> Iterator for MyIterator<'a> {
type Item = &'a str;
fn next(&mut self) -> Option<Self::Item> {
self.i += 1;
if self.i < 10 {
return Some(self.stream.cstr(self.i as usize).unwrap());
}
return None;
}
}
#[test]
fn iterates_tokens() {
let mut stream = StrStream::new("abcdefghijklmnopqrstuvwxyz");
let myiter = MyIterator::new(&mut stream);
for item in myiter {
println!(">>>{:?}", item);
}
}
This fails to compile with the error below. Any suggestions how to solve this problem?
error: lifetime may not live long enough
--> xxx/src/tests.rs:32:25
|
26 | impl<'a> Iterator for MyIterator<'a> {
| -- lifetime `'a` defined here
...
29 | fn next(&mut self) -> Option<Self::Item> {
| - let's call the lifetime of this reference `'1`
...
32 | return Some(self.stream.cstr(self.i as usize).unwrap());
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ argument requires that `'1` must outlive `'a`
This question already has an answer here:
How can I create my own data structure with an iterator that returns mutable references?
(1 answer)
Closed 6 months ago.
I'm having a lifetime issue when implementing an iterator on custom struct containing borrows of vecs.
I've been trying different solutions but can't fix it by myself (I'm still a beginner) and I want to understand what is going on.
here is a playground example of my issue, that should be simple enough.
This is the example :
struct SomeData;
struct CustomIterator<'a> {
pub vec: &'a mut Vec<SomeData>,
pub index: usize,
}
struct MultipleIterator<'a> {
iter1: CustomIterator<'a>,
iter2: CustomIterator<'a>,
}
impl<'a> Iterator for MultipleIterator<'a> {
type Item = (&'a mut SomeData, &'a mut SomeData);
fn next(&mut self) -> Option<Self::Item> {
Some((
match self.iter1.vec.get_mut(self.iter1.index) {
Some(mut data) => &mut data,
None => return None,
},
match self.iter2.vec.get_mut(self.iter2.index) {
Some(mut data) => &mut data,
None => return None,
}
))
}
}
I don't unerstand why I can't borrow out of the next function, since I am borrowing the struct anyway when calling next()
This is actually quite tricky to implement safely and requires a lot of care to do correctly.
First things first though:
match self.iter1.vec.get_mut(self.iter1.index) {
Some(mut data) => &mut data,
None => return None,
},
This is a problem because Vec::get_mut already returns a Option<&mut T>. So in the Some(mut data) arm, data already is a mutable reference. When you try to return &mut data, you're trying to return a &mut &mut T which doesn't work. Instead, just do this:
match self.iter1.vec.get_mut(self.iter1.index) {
Some(data) => data,
None => return None,
},
We can tidy this up even more with the ? operator which does the same thing. I'm gonna substitute SomeData for i32 from now on to demonstrate something later.
impl<'a> Iterator for MultipleIterator<'a> {
type Item = (&'a mut i32, &'a mut i32);
fn next(&mut self) -> Option<Self::Item> {
Some((
self.iter1.vec.get_mut(self.iter1.index)?,
self.iter2.vec.get_mut(self.iter2.index)?,
))
}
}
This still doesn't work and now we're getting to the core of the problem. The signature of next is
fn next(&mut self) -> Option<(&'a mut i32, &'a mut i32)>
which can be desugared to
fn next<'b>(&'b mut self) -> Option<(&'a mut i32, &'a mut i32)>
This means that the lifetime of &mut self ('b) is completely decoupled from the lifetime of the references we return ('a). Which makes total sense. If that wasn't the case, we couldn't do
let mut v = vec![1,2,3];
let mut iter = v.iter_mut();
let next1: &mut i32 = iter.next().unwrap();
let next2: &mut i32 = iter.next().unwrap();
because the lifetime of next1 would have to be the same lifetime of iter, i.e. that of v. But you can't have multiple mutable references to v at the same time, so next2 would be illegal.
So you're getting an error because Rust only knows that self is borrowed for 'b but you're telling it that you're returning a reference with lifetime 'a which it can't verify to be true.
And for good reason, because as it stands right now, your implementation isn't safe! Let's just throw caution to the wind and tell Rust that this is okay with unsafe:
impl<'a> Iterator for MultipleIterator<'a> {
type Item = (&'a mut i32, &'a mut i32);
fn next(&mut self) -> Option<Self::Item> {
unsafe {
Some((
&mut *(self.iter1.vec.get_mut(self.iter1.index)? as *mut _),
&mut *(self.iter2.vec.get_mut(self.iter2.index)? as *mut _),
))
}
}
}
It's not important what exactly this does, it basically just tells the compiler to shut up and trust me.
But now, we can do this:
let mut v1 = vec![1, 2, 3];
let mut v2 = vec![4, 5, 6];
let mut mi = MultipleIterator {
iter1: CustomIterator {
vec: &mut v1,
index: 0,
},
iter2: CustomIterator {
vec: &mut v2,
index: 0,
},
};
let next1 = mi.next().unwrap();
let next2 = mi.next().unwrap();
assert_eq!(next1, (&mut 1, &mut 4));
assert_eq!(next2, (&mut 1, &mut 4));
*next1.0 += 1;
assert_eq!(next1, (&mut 2, &mut 4));
assert_eq!(next2, (&mut 2, &mut 4));
We have broken Rust's most important rule: never have two mutable references to the same thing at once.
This can only be safe if your Iterator implementation can never return a mutable reference to something more than once. You could increment index each time, for example (although this still requires unsafe):
impl<'a> Iterator for MultipleIterator<'a> {
type Item = (&'a mut i32, &'a mut i32);
fn next(&mut self) -> Option<Self::Item> {
let next1 = self.iter1.vec.get_mut(self.iter1.index)?;
let next2 = self.iter2.vec.get_mut(self.iter2.index)?;
self.iter1.index += 1;
self.iter2.index += 1;
// SAFETY: this is safe because we will never return a reference
// to the same index more than once
unsafe { Some((&mut *(next1 as *mut _), &mut *(next2 as *mut _))) }
}
}
Here is an interesting related read from the nomicon; the "mutable slice" example being particularly relevant to your problem.
I'm playing around with building a very simple stack based evaluator in Rust. I want the user to be able to define functions later, so I'm storing all operators in a HashMap with closures as values.
use std::collections::HashMap;
pub type Value = i32;
pub struct Evaluator<'a> {
stack: Vec<Value>,
ops: HashMap<String, &'a dyn FnMut(&'a mut Vec<Value>)>,
}
impl<'a> Evaluator<'a> {
pub fn new() -> Evaluator<'a> {
let stack: Vec<Value> = vec![];
let mut ops: HashMap<String, &'a dyn FnMut(&'a mut Vec<Value>)> = HashMap::new();
ops.insert("+".to_string(), &|stack: &'a mut Vec<Value>| {
if let (Some(x), Some(y)) = (stack.pop(), stack.pop()) {
stack.push(y + x);
}
});
Evaluator { stack, ops }
}
pub fn stack(&self) -> &[Value] {
&self.stack
}
pub fn eval(&'a mut self, input: &str) {
let symbols = input
.split_ascii_whitespace()
.collect::<Vec<_>>();
for sym in symbols {
if let Ok(n) = sym.parse::<i32>() {
self.stack.push(n);
} else {
let s = sym.to_ascii_lowercase();
if let Some(f) = self.ops.get(&s) {
f(&mut self.stack);
} else {
println!("error");
}
}
}
}
}
fn main() {
let mut e = Evaluator::new();
e.eval("1 2 +")
}
I'm currently getting two errors:
error[E0499]: cannot borrow `self.stack` as mutable more than once at a time
--> src/sample.rs:34:17
|
10 | impl<'a> Evaluator<'a> {
| -- lifetime `'a` defined here
...
34 | self.stack.push(n);
| ^^^^^^^^^^ second mutable borrow occurs here
...
38 | f(&mut self.stack);
| ------------------
| | |
| | first mutable borrow occurs here
| argument requires that `self.stack` is borrowed for `'a`
error[E0596]: cannot borrow `**f` as mutable, as it is behind a `&` reference
--> src/sample.rs:38:21
|
38 | f(&mut self.stack);
| ^ cannot borrow as mutable
error[E0499]: cannot borrow `self.stack` as mutable more than once at a time
--> src/sample.rs:38:23
|
10 | impl<'a> Evaluator<'a> {
| -- lifetime `'a` defined here
...
38 | f(&mut self.stack);
| --^^^^^^^^^^^^^^^-
| | |
| | `self.stack` was mutably borrowed here in the previous iteration of the loop
| argument requires that `self.stack` is borrowed for `'a`
error: aborting due to 3 previous errors
Some errors have detailed explanations: E0499, E0596.
For more information about an error, try `rustc --explain E0499`.
My concern is the first one. I'm not sure what I'm doing wrong as I'm not borrowing them at the same time. Can I tell Rust the previous borrow (self.stack.pop()) is done? Any help appreciated.
I think I solved my problem. The thing I kept coming back to is, "What owns the closures?" In this case I'm using references, but nothing is taking ownership of the data. When I refactored (below) with Box to take ownership, it worked.
I'm curious if there is a way to do this with with just references and/or if my explanation is wrong?
Working code:
use std::collections::HashMap;
pub type Value = i32;
pub struct Evaluator {
stack: Vec<Value>,
ops: HashMap<String, Box<dyn FnMut(&mut Vec<Value>)>>,
}
impl Evaluator {
pub fn new() -> Evaluator {
let stack: Vec<Value> = vec![];
let mut ops: HashMap<String, Box<dyn FnMut(&mut Vec<Value>)>> = HashMap::new();
ops.insert("+".to_string(), Box::new(|stack: &mut Vec<Value>| {
if let (Some(x), Some(y)) = (stack.pop(), stack.pop()) {
stack.push(y + x);
}
}));
Evaluator { stack, ops }
}
pub fn stack(&self) -> &[Value] {
&self.stack
}
pub fn eval(&mut self, input: &str) {
let symbols = input
.split_ascii_whitespace()
.collect::<Vec<_>>();
for sym in symbols {
if let Ok(n) = sym.parse::<i32>() {
self.stack.push(n);
} else {
let s = sym.to_ascii_lowercase();
if let Some(f) = self.ops.get_mut(&s) {
f(&mut self.stack);
} else {
println!("error");
}
}
}
}
}
fn main() {
let mut e = Evaluator::new();
e.eval("1 2 +")
}
You have borrows with conflicting lifetimes:
You are defining a lifetime 'a for the struct in line 5: pub struct Evaluator<'a> {
In line 7, you are stating that ops is a HashMap that holds functions that receive mutable borrows for the whole duration of 'a
Then, in line 28, you are defining an eval method that holds a mutable reference to self for the whole duration of the struct ('a)
The conflict can be solved if you use two different lifetimes, since the time that an operation borrows self should be inherently shorter than the lifetime for the whole evaluation, since in eval you are running a loop and multiple invocations to the operations.
This should fix the issues mentioned above:
pub struct Evaluator<'a, 'b> {
stack: Vec<Value>,
ops: HashMap<String, &'b dyn FnMut(&'b mut Vec<Value>)>,
}
impl<'a, 'b> Evaluator<'a, 'b> {
pub fn new() -> Evaluator<'a> {
let stack: Vec<Value> = vec![];
let mut ops: HashMap<String, &'b dyn FnMut(&'b mut Vec<Value>)> = HashMap::new();
I am developing some basic data structures to learn the syntax and Rust in general. Here is what I came up with for a stack:
#[allow(dead_code)]
mod stack {
pub struct Stack<T> {
data: Vec<T>,
}
impl<T> Stack<T> {
pub fn new() -> Stack<T> {
return Stack { data: Vec::new() };
}
pub fn pop(&mut self) -> Result<T, &str> {
let len: usize = self.data.len();
if len > 0 {
let idx_to_rmv: usize = len - 1;
let last: T = self.data.remove(idx_to_rmv);
return Result::Ok(last);
} else {
return Result::Err("Empty stack");
}
}
pub fn push(&mut self, elem: T) {
self.data.push(elem);
}
pub fn is_empty(&self) -> bool {
return self.data.len() == 0;
}
}
}
mod stack_tests {
use super::stack::Stack;
#[test]
fn basics() {
let mut s: Stack<i16> = Stack::new();
s.push(16);
s.push(27);
let pop_result = s.pop().expect("");
assert_eq!(s.pop().expect("Empty stack"), 27);
assert_eq!(s.pop().expect("Empty stack"), 16);
let pop_empty_result = s.pop();
match pop_empty_result {
Ok(_) => panic!("Should have had no result"),
Err(_) => {
println!("Empty stack");
}
}
if s.is_empty() {
println!("O");
}
}
}
I get this interesting error:
error[E0502]: cannot borrow `s` as immutable because it is also borrowed as mutable
--> src/main.rs:58:12
|
49 | let pop_empty_result = s.pop();
| - mutable borrow occurs here
...
58 | if s.is_empty() {
| ^ immutable borrow occurs here
...
61 | }
| - mutable borrow ends here
Why can't I just call pop on my mutable struct?
Why does pop borrow the value? If I add a .expect() after it, it is ok, it doesn't trigger that error. I know that is_empty takes an immutable reference, if I switch it to mutable I just get a second mutable borrow.
Your pop function is declared as:
pub fn pop(&mut self) -> Result<T, &str>
Due to lifetime elision, this expands to
pub fn pop<'a>(&'a mut self) -> Result<T, &'a str>
This says that the Result::Err variant is a string that lives as long as the stack you are calling it on. Since the input and output lifetimes are the same, the returned value might be pointing somewhere into the Stack data structure so the returned value must continue to hold the borrow.
If I add a .expect() after it, it is ok, it doesn't trigger that error.
That's because expect consumes the Result, discarding the Err variant without ever putting it into a variable binding. Since that's never stored, the borrow cannot be saved anywhere and it is released.
To solve the problem, you need to have distinct lifetimes between the input reference and output reference. Since you are using a string literal, the easiest solution is to denote that using the 'static lifetime:
pub fn pop(&mut self) -> Result<T, &'static str>
Extra notes:
Don't call return explicitly at the end of the block / method: return Result::Ok(last) => Result::Ok(last).
Result, Result::Ok, and Result::Err are all imported via the prelude, so you don't need to qualify them: Result::Ok(last) => Ok(last).
There's no need to specify types in many cases let len: usize = self.data.len() => let len = self.data.len().
This happens because of lifetimes. When you construct a method which takes a reference the compiler detects that and if no lifetimes are specified it "generates" them:
pub fn pop<'a>(&'a mut self) -> Result<T, &'a str> {
let len: usize = self.data.len();
if len > 0 {
let idx_to_rmv: usize = len - 1;
let last: T = self.data.remove(idx_to_rmv);
return Result::Ok(last);
} else {
return Result::Err("Empty stack");
}
}
This is what compiler sees actually. So, you want to return a static string, then you have to specify the lifetime for a &str explicitly and let the lifetime for the reference to mut self be inferred automatically:
pub fn pop(&mut self) -> Result<T, &'static str> {