While trying to implement an iterator which yields mutable refs to elements of a linked list, I stumbled upon a strange issue.
This works fine:
impl<'a, T> Iterator<&'a T> for LinkedListIterator<'a, T>{
fn next(&mut self) -> Option<&'a T> {
match self.current {
&Cell(ref x, ref xs) => {self.current = &**xs; Some(x)},
&End => None
}
}
}
But this doesn't work; the compiler says lifetime of self is too short to guarantee its contents can be safely reborrowed:
impl<'a, T> Iterator<&'a mut T> for LinkedListMutIterator<'a, T>{
fn next(&mut self) -> Option<&'a mut T> {
match self.current {
&Cell(ref mut x, ref mut xs) => {self.current = &mut **xs; Some(x)},
&End => None
}
}
}
I would expect that either both example work, or both do not, but I can't understand how borrowing something as mutable vs not-mutable would impact the way the compiler checks for lifetimes. Surely if something lives long enough to be safely borrowed, it lives long enough to be safely mutably borrowed?
EDIT: Here is the definition of both Iterators:
pub struct LinkedListIterator<'a, T>
current: &'a LinkedList<T>
}
pub struct LinkedListMutIterator<'a, T> {
current: &'a mut LinkedList<T>
}
LinkedLisk:
#[deriving(Eq, Clone)]
pub enum LinkedList<T> {
Cell(T, ~LinkedList<T>),
End
}
For a complete view of the file, please see https://github.com/TisButMe/rust-algo/blob/mut_iter/LinkedList/linked_list.rs
Note that you've left out the definition(s) of LinkedListMutIterator for the two variant bits of code, which might be relevant to any real attempt to reproduce and dissect your problem.
So, I'll try to guess at what's going on.
The compiler error message here might be misleading you; there are other factors beyond the lifetime of self that may be relevant here.
In particular I suspect the borrow-checker is complaining because it is trying to ensure that you are not creating multiple mutable-borrows that alias the same state.
It is sound to have multiple immutable-borrows to the same piece of state...
... but you cannot have multiple mutable-borrows to the same piece of state (because we want to ensure that if you have a &mut reference to some state, then that reference is the only way to mutate the state).
Related
I want to create a wrapper around (nested) slices for easy operations on multidimensional data, owned by a different struct.
The most basic version of the mutable version of my slice wrapper might look like this:
struct MySliceMut<'a> {
data: Vec<&'a mut [f32]>,
}
impl<'a, 'b> MySliceMut<'a> {
fn get(&'b mut self) -> &'a mut [&'b mut [f32]] {
self.data.as_mut_slice()
}
}
Now if I want to implement a trait, for instance AddAssign, Rust does not seem to infer the lifetime of &mut self from the implementing type. The compiler complains that &mut self might outlive 'a:
impl<'a> AddAssign<MySlice<'a>> for MySliceMut<'a> { // lifetime 'a
fn add_assign(&mut self, rhs: MySlice<'a>) { // lifetime '1
let a = self.get(); // lifetime may not live long enough, '1 must outlive 'a
let b = rhs.get();
// do inplace addition here
}
}
Full Code - Rust Playground
I tried to figure out the issue with the lifetimes, but can't find it. Would the trait impl require any additional annotations?
struct MySlice<'a> {
data: Vec<&'a [f32]>,
}
impl<'a, 'b> MySlice<'a> {
fn get(&'b self) -> &'a [&'b [f32]] {
self.data.as_slice()
}
}
Problem with your code is that fn get(&'b self) returns variable with wrong lifetime. Associated lifetime 'a of MySlice<'a> is lifetime of inner slice. Associated lifetime 'b of fn get(...) is lifetime of the self. So I guess the function probably should return &'b [&'a [f32]] instead.
-- Edited --
Make sure to change fn get(...) of MySliceMut either.
I'm trying to implement something like a zipper but taking advantage of mutable references to avoid having to deconstruct and reconstruct the data structure as I move through it. I've got example code for an attempt with a linked list, although I'd ideally like to apply it to other structures, like trees.
pub enum List<T> {
Empty,
Cons { head: T, tail: Box<List<T>> },
}
pub struct Zipper<'a, T: 'a> {
trail: Option<Box<Zipper<'a, T>>>,
focus: &'a mut List<T>,
}
impl<'a, T: 'a> Zipper<'a, T> {
pub fn down(&'a mut self) {
match self.focus {
&mut List::Empty => (),
&mut List::Cons {
tail: ref mut xs, ..
} => {
//We need a way to convince rust that we won't use oldZipper
//until xs goes out of scope
let oldZipper = std::mem::replace(
self,
Zipper {
trail: None,
focus: xs,
},
);
self.trail = Some(Box::new(oldZipper));
}
}
}
}
The borrow checker is not happy with this:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:21:21
|
16 | tail: ref mut xs, ..
| ---------- first mutable borrow occurs here
...
21 | self,
| ^^^^ second mutable borrow occurs here
...
30 | }
| - first borrow ends here
This isn't surprising: if we have a zipper focused on a list and call down on it, we get zipper with a mutable reference to the tail of that list, so we have mutable aliasing.
However, if we never use the Zipper's trail before focus goes out of scope, we'll never be able to "see" the mutable aliasing. This seems analogous to normal mutable borrowing: you can't use the variable you borrowed from until the borrow goes out of scope.
Is there some way to explain this to the borrow checker? If you want to "explain" to the borrow checker that borrowing two non-overlapping slices from an array is okay, you can use split_at: is there some corresponding function that will enforce that trail is never used before focus goes out of scope, and in doing so, satisfies the borrow checker?
In order to achieve your goal, we need to get rid of the mutable reference in the Zipper struct. We can use mutable raw pointers instead: they let us mutate their referent, and we can more than one such pointer pointing at a particular object, but dereferencing them is unsafe.
Here's the code:
use std::mem;
use std::marker::PhantomData;
pub enum List<T> {
Empty,
Cons { head: T, tail: Box<List<T>> },
}
pub struct Zipper<'a, T: 'a> {
trail: Option<Box<Zipper<'a, T>>>,
focus: *mut List<T>,
_list: PhantomData<&'a mut List<T>>,
}
impl<'a, T: 'a> Zipper<'a, T> {
pub fn new(list: &'a mut List<T>) -> Zipper<'a, T> {
Zipper {
trail: None,
focus: list as *mut List<T>,
_list: PhantomData,
}
}
pub fn down(&mut self) {
unsafe {
match *self.focus {
List::Empty => (),
List::Cons {
tail: ref mut xs, ..
} => {
let old_zipper = mem::replace(
self,
Zipper::new(xs),
);
self.trail = Some(Box::new(old_zipper));
}
}
}
}
}
fn main() {
let mut list = List::Cons { head: 1, tail: Box::new(List::Empty) };
let mut zipper = Zipper::new(&mut list);
zipper.down();
zipper.down();
}
The focus field in the Zipper struct is now a *mut List<T>. Because this is a raw pointer, we can copy it around freely. This resolves the compiler error you had in Zipper::down. There's also a new field, _list, of type PhantomData<&'a mut List<T>>. PhantomData is a special type that is meant to tell the compiler "pretend I'm storing/owning a T, even though I'm not". Without this field, the compiler would complain that the lifetime parameter 'a is unused.
Notice that Zipper::new still expects a &'a mut List<T> as a parameter: this allows Zipper to provide a safe interface by requiring the caller to have a unique mutable reference to the List<T>, a fact we can use to declare that the other unsafe operations in the struct are indeed safe since we have full knowledge of the available mutable references. As far as the compiler is concerned, a Zipper is mutably borrowing the List; if you try to mutate a List while a Zipper on the List is in scope, you'll get an error that the List is already mutably borrowed.
You haven't shown any code that would let the user get a reference to the Zipper's focus. I've been thinking of a possible implementation that would be unsafe, and it's tempting to go that route, but the compiler won't tell you it's wrong. Let me show you:
impl<'a, T: 'a> Zipper<'a, T> {
pub fn focus(&mut self) -> &'a mut List<T> {
unsafe { &mut *self.focus }
}
}
It's tempting to return a &'a mut List<T> because that's what we were given. However, it's wrong because the return value's lifetime is not bound to self in any way, which means that we could call focus twice to obtain two mutable references to the same List<T>. If we still had a &'a mut List<T> in Zipper, the compiler would tell us if we tried to return a &'a mut List<T> (unless we used unsafe code to work around it). A correct implementation would be:
impl<'a, T: 'a> Zipper<'a, T> {
pub fn focus(&mut self) -> &mut List<T> {
unsafe { &mut *self.focus }
}
}
In this implementation, the Zipper will be mutably borrowed as long as the returned &mut List<T> is around, which means we can't call focus (or down) until the &mut List<T> goes out of scope.
An easily overlooked feature of clone() is that it can shorten the lifetimes of any references hidden inside the value being cloned. This is usually useless for immutable references, which are the only kind for which Clone is implemented.
It would, however, be useful to be able to shorten the lifetimes of mutable references hidden inside a value. Is there something like a CloneMut trait?
I've managed to write one. My question is whether there is a trait in the standard library that I should use instead, i.e. am I reinventing the wheel?
The rest of this question consists of details and examples.
Playground.
Special case: the type is a mutable reference
As a warm-up, the following is good enough when the type you're cloning is a mutable reference, not wrapped in any way:
fn clone_mut<'a, 'b: 'a>(q: &'a mut &'b mut f32) -> &'a mut f32 {
*q
}
See this question (where it is called reborrow()) for an example caller.
Special case: the reference type, though user-defined, is known
A more interesting case is a user-defined mutable-reference-like type. Here's how to write a clone_mut() function specific to a particular type:
struct Foo<'a>(&'a mut f32);
impl<'b> Foo<'b> {
fn clone_mut<'a>(self: &'a mut Foo<'b>) -> Foo<'a> {
Foo(self.0)
}
}
Here's an example caller:
fn main() {
let mut x: f32 = 3.142;
let mut p = Foo(&mut x);
{
let q = p.clone_mut();
*q.0 = 2.718;
}
println!("{:?}", *p.0)
}
Note that this won't compile unless q gets a shorter lifetime than p. I'd like to view that as a unit test for clone_mut().
Higher-kinded type?
When trying to write a trait that admits both the above implementations, the problem at first feels like a higher-kinded-type problem. For example, I want to write this:
trait CloneMut {
fn clone_mut<'a, 'b>(self: &'a mut Self<'b>) -> Self<'a>;
}
impl CloneMut for Foo {
fn clone_mut<'a, 'b>(self: &'a mut Self<'b>) -> Self<'a> {
Foo(self.0)
}
}
Of course that's not allowed in Rust (the Self<'a> and Self<'b> parts in particular). However, the problem can be worked around.
General case
The following code compiles (using the preceding definition of Foo<'a>) and is compatible with the caller:
trait CloneMut<'a> {
type To: 'a;
fn clone_mut(&'a mut self) -> Self::To;
}
impl<'a, 'b> CloneMut<'a> for Foo<'b> {
type To = Foo<'a>;
fn clone_mut(&'a mut self) -> Self::To {
Foo(self.0)
}
}
It's a little ugly that there is no formal relationship between Self and Self::To. For example, you could write an implementation of clone_mut() that returns 77, completely ignoring the Self type. The following two attempts show why I think the associated type is unavoidable.
Attempt 1
This compiles:
trait CloneMut<'a> {
fn clone_mut(&'a mut self) -> Self;
}
impl<'a> CloneMut<'a> for Foo<'a> {
fn clone_mut(&'a mut self) -> Self {
Foo(self.0)
}
}
However, it's not compatible with the caller, because it does not have two distinct lifetime variables.
error[E0502]: cannot borrow `*p.0` as immutable because `p` is also borrowed as mutable
The immutable borrow mentioned in the error message is the one in the println!() statement, and the mutable borrow is the call to clone_mut(). The trait constrains the two lifetimes to be the same.
Attempt 2
This uses the same trait definition as attempt 1, but a different implementation:
trait CloneMut<'a> {
fn clone_mut(&'a mut self) -> Self;
}
impl<'a, 'b: 'a> CloneMut<'a> for Foo<'b> {
fn clone_mut(&'a mut self) -> Self {
Foo(self.0)
}
}
This doesn't even compile. The return type has the longer lifetime, and can't be made from the argument, which has the shorter lifetime.
Moving the lifetime parameter onto the method declaration gives the same error:
trait CloneMut {
fn clone_mut<'a>(&'a mut self) -> Self;
}
impl<'b> CloneMut for Foo<'b> {
fn clone_mut<'a>(&'a mut self) -> Self {
Foo(self.0)
}
}
Relationship with Clone
Incidentally, notice that CloneMut<'a, To=Self> is strictly stronger than Clone:
impl<'a, T: 'a> CloneMut<'a> for T where T: Clone {
type To = Self;
fn clone_mut(&'a mut self) -> Self {
self.clone()
}
}
That's why I think "CloneMut" is a good name.
The key property of &mut references is that they are unique exclusive references.
So it's not really a clone. You can't have two exclusive references. It's a reborrow, as the source will be completely unusable as long as the "clone" is in scope.
I'm working on a program involving a struct along these lines:
struct App {
data: Vec<u8>,
overlay: Vec<(usize, Vec<u8>)>,
sink: Sink,
}
In brief the data field holds some bytes and overlay is a series of byte sequences to be inserted at specific indices. The Sink type is unimportant except that it has a function like:
impl Sink {
fn process<'a>(&mut self, input: Vec<&'a [u8]>) {
// ...
}
}
I've implemented an iterator to merge the information from data and overlay for consumption by Sink.
struct MergeIter<'a, 'b> {
data: &'a Vec<u8>,
overlay: &'b Vec<(usize, Vec<u8>)>,
// iterator state etc.
}
impl<'a, 'b> Iterator for MergeIter<'a, 'b> {
type Item = &'a [u8];
// ...
}
This is I think a slight lie, because the lifetime of each &[u8] returned by the iterator isn't always that of the original data. The data inserted from overlay has a different lifetime, but I don't see how I can annotate this more accurately. Anyway, the borrow checker doesn't seem to mind - the following approach works:
fn merge<'a, 'b>(data: &'a Vec<u8>, overlay: &'b Vec<(usize, Vec<u8>)>, start: usize) -> Vec<&'a [u8]> {
MergeIter::new(data, overlay, start).collect()
}
impl App {
fn process(&mut self) {
let merged = merge(&self.data, &self.overlay, 0);
// inspect contents of 'merged'
self.sink.process(merged);
}
}
I end up using this merge function all over the place, but always against the same data/overlay. So I figure I'll add an App::merge function for convenience, and here's where the problem begins:
impl App {
fn merge<'a>(&'a self, start: usize) -> Vec<&'a [u8]> {
MergeIter::new(&self.data, &self.overlay, start).collect()
}
fn process(&mut self) {
let merged = self.merge(0);
// inspect contents of 'merged'
self.sink.process(merged);
}
}
App::process now fails to pass the borrow checker - it refuses to allow the mutable borrow of self.sink while self is borrowed.
I've wrestled with this for some time, and if I've understood correctly the problem isn't with process but with this signature:
fn merge<'a>(&'a self, start: usize) -> Vec<&'a [u8]> {
Here I've essentially told the borrow checker that the references returned in the vector are equivalent to the self borrow.
Even though I feel like I've now understood the problem, I still feel like my hands are tied. Leaving the lifetime annotations out doesn't help (because the compiler does the equivalent?), and with only the two references involved there's no way I can see to tell rust that the output reference has a lifetime bound to something else.
I also tried this:
fn merge<'a, 'b>(&'b self, start: usize) -> Vec<&'a [u8]> {
let data: &'a Vec<u8> = &self.data;
MergeIter::new(&self.data, &self.overlay, start).collect()
}
but the compiler complains about the let statement ("unable to infer appropriate lifetime due to conflicting requirements" -- I also find it infuriating that the compiler doesn't explain said requirements).
Is it possible to achieve this? The Rust Reference is kind of light on lifetime annotations and associated syntax.
rustc 1.0.0-nightly (706be5ba1 2015-02-05 23:14:28 +0000)
As long as the method merge takes &self, you cannot accomplish what you desire: it borrows all of each of its arguments and this cannot be altered.
The solution is to change it so that it doesn’t take self, but instead takes the individual fields you wish to be borrowed:
impl App {
...
fn merge(data: &Vec<u8>, overlay: &Vec<(usize, Vec<u8>)>, start: usize) -> Vec<&[u8]> {
MergeIter::new(data, overlay, start).collect()
}
fn process(&mut self) {
let merged = Self::merge(&self.data, &self.overlay, 0);
... // inspect contents of 'merged'
self.sink.process(merged);
}
}
Yes, you've guessed correctly - the error happens because when you have merge method accept &self, the compiler can't know at its call site that it uses only some fields - merge signature only tells it that the data it returns is somehow derived from self, but it doesn't tell how - and so the compiler assumes the "worst" case and prevents you from accessing other fields self has.
I'm afraid there is no way to fix this at the moment, and I'm not sure there ever will be any. However, you can use macros to shorten merge invocations:
macro_rules! merge {
($this:ident, $start:expr) => {
MergeIter::new(&$this.data, &$this.overlay, $start).collect()
}
}
fn process(&mut self) {
let merged = merge!(self, 0);
// inspect contents of 'merged'
self.sink.process(merged);
}
use std::io::BufReader;
struct Foo {
buf: [u8, ..10]
}
trait Bar<'a> {
fn test(&self, arg: BufReader<'a>) {}
}
impl<'a, T: Bar<'a>> Foo {
fn bar(&'a mut self, t: T) {
t.test(BufReader::new(&self.buf));
let b = &mut self.buf;
}
fn baz(&self, t: T) {
t.test(BufReader::new(&self.buf));
}
}
fn main() {}
The code above fails to compile, with the error message:
lifetimes.rs:17:31: 17:40 error: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
lifetimes.rs:17 t.test(BufReader::new(&self.buf));
^~~~~~~~~
lifetimes.rs:16:5: 18:6 help: consider using an explicit lifetime parameter as shown: fn baz(&'a self, t: T)
lifetimes.rs:16 fn baz(&self, t: T) {
lifetimes.rs:17 t.test(BufReader::new(&self.buf));
lifetimes.rs:18 }
error: aborting due to previous error
However, if I add the named lifetime parameter, I cannot mutable borrow the buf field after calling test, as seen in fn bar. Commenting out the fn baz and trying to compile results in:
lifetimes.rs:13:22: 13:30 error: cannot borrow `self.buf` as mutable because it is also borrowed as immutable
lifetimes.rs:13 let b = &mut self.buf;
^~~~~~~~
lifetimes.rs:12:32: 12:40 note: previous borrow of `self.buf` occurs here; the immutable borrow prevents subsequent moves or mutable borrows of `self.buf` until the borrow ends
lifetimes.rs:12 t.test(BufReader::new(&self.buf));
^~~~~~~~
lifetimes.rs:14:6: 14:6 note: previous borrow ends here
lifetimes.rs:11 fn bar(&'a mut self, t: T) {
lifetimes.rs:12 t.test(BufReader::new(&self.buf));
lifetimes.rs:13 let b = &mut self.buf;
lifetimes.rs:14 }
^
error: aborting due to previous error
My understanding of this is that by adding the named lifetime 'a to the &'a mut self parameter, the reference taken by BufReader has a lifetime as long as the self reference is valid, which is until the end of the function. This conflicts with the mutable borrow of self.buf on the line after.
However, I am not sure why I need the named lifetime parameter on the self. It seems to me that the BufReader reference should be able to only exist for the lifetime of the t.test method call. Is the compiler complaining because the self.buf borrow must be ensured to live only as long as the &self borrow? How would I go about doing that while still only borrowing it for the lifetime of the method call?
Any help in going about fixing this problem and understanding more about the semantics here would be much appreciated!
Update
So I am still looking into this problem, and I have found this test case and this issue that show basically what I am trying to do. I would very much like to understand why the error pointed to by the test case link is an error.
I can see in the issue rustc output that attempts to point out what the error is, but I am having trouble understanding what exactly it is trying to say.
Removing all explicit lifetimes also works. I've found that I only add lifetimes when I'm sure I need them (i.e. to specifiy that two lifetimes should intersect at a given point which can't be known to the compiler).
I'm not sure exactly what you're going for, but this compiles (on rustc 0.13.0-nightly (cc19e3380 2014-12-20 20:00:36 +0000)).
use std::io::BufReader;
struct Foo {
buf: [u8, ..10]
}
trait Bar {
fn test(&self, arg: BufReader) {}
}
impl<T: Bar> Foo {
fn bar(&mut self, t: T) {
t.test(BufReader::new(&self.buf));
let b = &mut self.buf;
}
fn baz(&self, t: T) {
t.test(BufReader::new(&self.buf));
}
}
Edit
I'm going to copy-edit my comment here:
I originally thought that adding a lifetime or generic parameter to the trait / struct / enum was a shorthand for putting it on every method in the trait, but I was wrong. My current understanding is that you add a lifetime to the trait / struct / enum when that item needs to participate in the lifetime, likely because it is storing a reference with that lifetime.
struct Keeper<'a> {
counts: Vec<&'a i32>,
}
impl<'a> Keeper<'a> {
fn add_one(&mut self, count: &'a i32) {
if *count > 5 {
self.counts.push(count);
}
}
fn add_two<'b>(&mut self, count: &'b i32) -> i32 {
*count + 1
}
}
fn main() {
let mut cnt1 = 1;
let mut cnt2 = 2;
let mut k = Keeper { counts: Vec::new() };
k.add_one(&cnt1);
k.add_two(&cnt2);
// cnt1 += 1; // Errors: cannot assign to `cnt1` because it is borrowed
cnt2 += 1; // Just fine
println!("{}, {}", cnt1, cnt2)
}
Here, we've added a lifetime to Keeper because it might store the reference it is given. The borrow checker must assume that the reference is stored for good when we call add_one, so once we call that method, we can no longer mutate the value.
add_two, on the other hand, creates a fresh lifetime that can only be applied to that function invocation, so the borrow checker knows that once the function returns, it is the One True Owner.
The upshot is, if you need to store a reference, then there's nothing you can do at this level. Rust can't make sure you are safe, and that's something it takes seriously.
However, I bet you don't need to store the reference. Move the <'a, T: Bar<'a>> from the impl to the fn and you'll be good to go.
Said another way: I bet you should never have impl<A> if your trait or struct don't require it. Put the generics on the methods instead.
Original
This compiles, but I'm not 100% sure it does what you intended:
impl Foo {
fn baz<'a, T: Bar<'a>>(&'a self, t: T) {
t.test(BufReader::new(&self.buf));
}
}
I fell into this trap myself, so I'll paste what I was told:
Everything in the impl block is parameterized. I've actually never
seen type parameters added to impl blocks themselves that aren't part
of the trait or type definition. It's far more common to parameterize
the individual methods that need it.
Perhaps other comments / answers can help explain in further detail.