Save mutable reference for later even when aliased - rust

I'm trying to implement something like a zipper but taking advantage of mutable references to avoid having to deconstruct and reconstruct the data structure as I move through it. I've got example code for an attempt with a linked list, although I'd ideally like to apply it to other structures, like trees.
pub enum List<T> {
Empty,
Cons { head: T, tail: Box<List<T>> },
}
pub struct Zipper<'a, T: 'a> {
trail: Option<Box<Zipper<'a, T>>>,
focus: &'a mut List<T>,
}
impl<'a, T: 'a> Zipper<'a, T> {
pub fn down(&'a mut self) {
match self.focus {
&mut List::Empty => (),
&mut List::Cons {
tail: ref mut xs, ..
} => {
//We need a way to convince rust that we won't use oldZipper
//until xs goes out of scope
let oldZipper = std::mem::replace(
self,
Zipper {
trail: None,
focus: xs,
},
);
self.trail = Some(Box::new(oldZipper));
}
}
}
}
The borrow checker is not happy with this:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:21:21
|
16 | tail: ref mut xs, ..
| ---------- first mutable borrow occurs here
...
21 | self,
| ^^^^ second mutable borrow occurs here
...
30 | }
| - first borrow ends here
This isn't surprising: if we have a zipper focused on a list and call down on it, we get zipper with a mutable reference to the tail of that list, so we have mutable aliasing.
However, if we never use the Zipper's trail before focus goes out of scope, we'll never be able to "see" the mutable aliasing. This seems analogous to normal mutable borrowing: you can't use the variable you borrowed from until the borrow goes out of scope.
Is there some way to explain this to the borrow checker? If you want to "explain" to the borrow checker that borrowing two non-overlapping slices from an array is okay, you can use split_at: is there some corresponding function that will enforce that trail is never used before focus goes out of scope, and in doing so, satisfies the borrow checker?

In order to achieve your goal, we need to get rid of the mutable reference in the Zipper struct. We can use mutable raw pointers instead: they let us mutate their referent, and we can more than one such pointer pointing at a particular object, but dereferencing them is unsafe.
Here's the code:
use std::mem;
use std::marker::PhantomData;
pub enum List<T> {
Empty,
Cons { head: T, tail: Box<List<T>> },
}
pub struct Zipper<'a, T: 'a> {
trail: Option<Box<Zipper<'a, T>>>,
focus: *mut List<T>,
_list: PhantomData<&'a mut List<T>>,
}
impl<'a, T: 'a> Zipper<'a, T> {
pub fn new(list: &'a mut List<T>) -> Zipper<'a, T> {
Zipper {
trail: None,
focus: list as *mut List<T>,
_list: PhantomData,
}
}
pub fn down(&mut self) {
unsafe {
match *self.focus {
List::Empty => (),
List::Cons {
tail: ref mut xs, ..
} => {
let old_zipper = mem::replace(
self,
Zipper::new(xs),
);
self.trail = Some(Box::new(old_zipper));
}
}
}
}
}
fn main() {
let mut list = List::Cons { head: 1, tail: Box::new(List::Empty) };
let mut zipper = Zipper::new(&mut list);
zipper.down();
zipper.down();
}
The focus field in the Zipper struct is now a *mut List<T>. Because this is a raw pointer, we can copy it around freely. This resolves the compiler error you had in Zipper::down. There's also a new field, _list, of type PhantomData<&'a mut List<T>>. PhantomData is a special type that is meant to tell the compiler "pretend I'm storing/owning a T, even though I'm not". Without this field, the compiler would complain that the lifetime parameter 'a is unused.
Notice that Zipper::new still expects a &'a mut List<T> as a parameter: this allows Zipper to provide a safe interface by requiring the caller to have a unique mutable reference to the List<T>, a fact we can use to declare that the other unsafe operations in the struct are indeed safe since we have full knowledge of the available mutable references. As far as the compiler is concerned, a Zipper is mutably borrowing the List; if you try to mutate a List while a Zipper on the List is in scope, you'll get an error that the List is already mutably borrowed.
You haven't shown any code that would let the user get a reference to the Zipper's focus. I've been thinking of a possible implementation that would be unsafe, and it's tempting to go that route, but the compiler won't tell you it's wrong. Let me show you:
impl<'a, T: 'a> Zipper<'a, T> {
pub fn focus(&mut self) -> &'a mut List<T> {
unsafe { &mut *self.focus }
}
}
It's tempting to return a &'a mut List<T> because that's what we were given. However, it's wrong because the return value's lifetime is not bound to self in any way, which means that we could call focus twice to obtain two mutable references to the same List<T>. If we still had a &'a mut List<T> in Zipper, the compiler would tell us if we tried to return a &'a mut List<T> (unless we used unsafe code to work around it). A correct implementation would be:
impl<'a, T: 'a> Zipper<'a, T> {
pub fn focus(&mut self) -> &mut List<T> {
unsafe { &mut *self.focus }
}
}
In this implementation, the Zipper will be mutably borrowed as long as the returned &mut List<T> is around, which means we can't call focus (or down) until the &mut List<T> goes out of scope.

Related

Rust lifetimes for implementing a trait on nested slices

I want to create a wrapper around (nested) slices for easy operations on multidimensional data, owned by a different struct.
The most basic version of the mutable version of my slice wrapper might look like this:
struct MySliceMut<'a> {
data: Vec<&'a mut [f32]>,
}
impl<'a, 'b> MySliceMut<'a> {
fn get(&'b mut self) -> &'a mut [&'b mut [f32]] {
self.data.as_mut_slice()
}
}
Now if I want to implement a trait, for instance AddAssign, Rust does not seem to infer the lifetime of &mut self from the implementing type. The compiler complains that &mut self might outlive 'a:
impl<'a> AddAssign<MySlice<'a>> for MySliceMut<'a> { // lifetime 'a
fn add_assign(&mut self, rhs: MySlice<'a>) { // lifetime '1
let a = self.get(); // lifetime may not live long enough, '1 must outlive 'a
let b = rhs.get();
// do inplace addition here
}
}
Full Code - Rust Playground
I tried to figure out the issue with the lifetimes, but can't find it. Would the trait impl require any additional annotations?
struct MySlice<'a> {
data: Vec<&'a [f32]>,
}
impl<'a, 'b> MySlice<'a> {
fn get(&'b self) -> &'a [&'b [f32]] {
self.data.as_slice()
}
}
Problem with your code is that fn get(&'b self) returns variable with wrong lifetime. Associated lifetime 'a of MySlice<'a> is lifetime of inner slice. Associated lifetime 'b of fn get(...) is lifetime of the self. So I guess the function probably should return &'b [&'a [f32]] instead.
-- Edited --
Make sure to change fn get(...) of MySliceMut either.

Is there a CloneMut trait?

An easily overlooked feature of clone() is that it can shorten the lifetimes of any references hidden inside the value being cloned. This is usually useless for immutable references, which are the only kind for which Clone is implemented.
It would, however, be useful to be able to shorten the lifetimes of mutable references hidden inside a value. Is there something like a CloneMut trait?
I've managed to write one. My question is whether there is a trait in the standard library that I should use instead, i.e. am I reinventing the wheel?
The rest of this question consists of details and examples.
Playground.
Special case: the type is a mutable reference
As a warm-up, the following is good enough when the type you're cloning is a mutable reference, not wrapped in any way:
fn clone_mut<'a, 'b: 'a>(q: &'a mut &'b mut f32) -> &'a mut f32 {
*q
}
See this question (where it is called reborrow()) for an example caller.
Special case: the reference type, though user-defined, is known
A more interesting case is a user-defined mutable-reference-like type. Here's how to write a clone_mut() function specific to a particular type:
struct Foo<'a>(&'a mut f32);
impl<'b> Foo<'b> {
fn clone_mut<'a>(self: &'a mut Foo<'b>) -> Foo<'a> {
Foo(self.0)
}
}
Here's an example caller:
fn main() {
let mut x: f32 = 3.142;
let mut p = Foo(&mut x);
{
let q = p.clone_mut();
*q.0 = 2.718;
}
println!("{:?}", *p.0)
}
Note that this won't compile unless q gets a shorter lifetime than p. I'd like to view that as a unit test for clone_mut().
Higher-kinded type?
When trying to write a trait that admits both the above implementations, the problem at first feels like a higher-kinded-type problem. For example, I want to write this:
trait CloneMut {
fn clone_mut<'a, 'b>(self: &'a mut Self<'b>) -> Self<'a>;
}
impl CloneMut for Foo {
fn clone_mut<'a, 'b>(self: &'a mut Self<'b>) -> Self<'a> {
Foo(self.0)
}
}
Of course that's not allowed in Rust (the Self<'a> and Self<'b> parts in particular). However, the problem can be worked around.
General case
The following code compiles (using the preceding definition of Foo<'a>) and is compatible with the caller:
trait CloneMut<'a> {
type To: 'a;
fn clone_mut(&'a mut self) -> Self::To;
}
impl<'a, 'b> CloneMut<'a> for Foo<'b> {
type To = Foo<'a>;
fn clone_mut(&'a mut self) -> Self::To {
Foo(self.0)
}
}
It's a little ugly that there is no formal relationship between Self and Self::To. For example, you could write an implementation of clone_mut() that returns 77, completely ignoring the Self type. The following two attempts show why I think the associated type is unavoidable.
Attempt 1
This compiles:
trait CloneMut<'a> {
fn clone_mut(&'a mut self) -> Self;
}
impl<'a> CloneMut<'a> for Foo<'a> {
fn clone_mut(&'a mut self) -> Self {
Foo(self.0)
}
}
However, it's not compatible with the caller, because it does not have two distinct lifetime variables.
error[E0502]: cannot borrow `*p.0` as immutable because `p` is also borrowed as mutable
The immutable borrow mentioned in the error message is the one in the println!() statement, and the mutable borrow is the call to clone_mut(). The trait constrains the two lifetimes to be the same.
Attempt 2
This uses the same trait definition as attempt 1, but a different implementation:
trait CloneMut<'a> {
fn clone_mut(&'a mut self) -> Self;
}
impl<'a, 'b: 'a> CloneMut<'a> for Foo<'b> {
fn clone_mut(&'a mut self) -> Self {
Foo(self.0)
}
}
This doesn't even compile. The return type has the longer lifetime, and can't be made from the argument, which has the shorter lifetime.
Moving the lifetime parameter onto the method declaration gives the same error:
trait CloneMut {
fn clone_mut<'a>(&'a mut self) -> Self;
}
impl<'b> CloneMut for Foo<'b> {
fn clone_mut<'a>(&'a mut self) -> Self {
Foo(self.0)
}
}
Relationship with Clone
Incidentally, notice that CloneMut<'a, To=Self> is strictly stronger than Clone:
impl<'a, T: 'a> CloneMut<'a> for T where T: Clone {
type To = Self;
fn clone_mut(&'a mut self) -> Self {
self.clone()
}
}
That's why I think "CloneMut" is a good name.
The key property of &mut references is that they are unique exclusive references.
So it's not really a clone. You can't have two exclusive references. It's a reborrow, as the source will be completely unusable as long as the "clone" is in scope.

Aquire &'a T from Option<T<'a>>

I'm afraid this may be very basic but I haven't been able to figure it out on my own. I have this map:
subscriptions_map: HashMap<SubscriptionKey, Subscription<'a>>
and this vector:
subscriptions: Vec<&'a Subscription<'a>>,
I want to insert a value into the HashMap and a reference to the same item into the vector. I've tried to do it like this:
let subs: &'a Subscription = &self.subscriptions_map.insert(id, item).unwrap();
self.subscriptions.push(subs);
But it gets this error:
error: borrowed value does not live long enough
let subs: &'a Subscription = &self.subscriptions_map.insert(id, item).unwrap();
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: reference must be valid for the lifetime 'a as defined on the block at 40:70...
pub fn add_subscription(&'a mut self, mut item: Subscription<'a>) {
let id = item.get_id();
let _lock = self.lock.lock().unwrap();
let subs: &'a Subscription = &self.subscriptions_map.insert(id, item).unwrap();
...
note: ...but borrowed value is only valid for the block suffix following statement 2 at 45:87
let subs: &'a Subscription = &self.subscriptions_map.insert(id, item).unwrap();
self.subscriptions.push(subs);
}
error: aborting due to previous error
I guess my question boils down to: If I have an Option<T<'a>>, how can I get a &'a T?
HashMap.insert() returns the old value for the given key, not the value you just passed. That's not what you want!
After you've inserted an item into the HashMap, you must call HashMap.get() to retrieve a pointer to the value. As HashMap.insert() takes ownership of both the key and the value, we need to pass a clone of id to insert() so we can use the original id for the get() call. (If the type of id is Copy, you may omit the call to clone() and let the compiler copy the value.)
use std::collections::HashMap;
#[derive(Eq, PartialEq, Hash, Clone)]
struct SubscriptionKey;
struct Subscription<'a>(&'a ());
struct Foo<'a> {
subscriptions_map: HashMap<SubscriptionKey, Subscription<'a>>,
subscriptions: Vec<&'a Subscription<'a>>,
}
impl<'a> Foo<'a> {
fn add(&'a mut self, id: SubscriptionKey, item: Subscription<'a>) {
self.subscriptions_map.insert(id.clone(), item);
let subs = self.subscriptions_map.get(&id).unwrap();
self.subscriptions.push(subs);
}
}
fn main() {
let subscription_data = &();
let mut f = Foo {
subscriptions_map: HashMap::new(),
subscriptions: Vec::new(),
};
f.add(SubscriptionKey, Subscription(subscription_data));
}
This works fine, but it falls apart if we try to add another subscription. If we do this:
fn main() {
let subscription_data = &();
let subscription_data2 = &();
let mut f = Foo {
subscriptions_map: HashMap::new(),
subscriptions: Vec::new(),
};
f.add(SubscriptionKey, Subscription(subscription_data));
f.add(SubscriptionKey, Subscription(subscription_data2));
}
the compiler gives the following messages:
<anon>:30:5: 30:6 error: cannot borrow `f` as mutable more than once at a time [E0499]
<anon>:30 f.add(SubscriptionKey, Subscription(subscription_data2));
^
<anon>:30:5: 30:6 help: see the detailed explanation for E0499
<anon>:29:5: 29:6 note: previous borrow of `f` occurs here; the mutable borrow prevents subsequent moves, borrows, or modification of `f` until the borrow ends
<anon>:29 f.add(SubscriptionKey, Subscription(subscription_data));
^
<anon>:31:2: 31:2 note: previous borrow ends here
<anon>:20 fn main() {
...
<anon>:31 }
^
What's going on? Why does the mutable borrow persist after the first call to Foo::add?
The problem comes from the definition of the subscriptions field. It's defined as a Vec<&'a Subscription<'a>>. Satisfying the 'a in Subscription<'a> is easy, since we receive the object with the correct lifetime in add. Satisfying the 'a in &'a ... is harder, since the Subscription<'a> value doesn't have a fixed address until we insert it into subscriptions_map (in my example, a Subscription<'a> is moved from a local variable in main() to a parameter in Foo::add() to inside self.subscriptions_map).
In order to satisfy the outer 'a, Foo::add() must define its self parameter as &'a mut self. If we defined it as &mut self, we couldn't be sure that the references we get out of subscriptions_map would live long enough (their lifetime could be shorter than 'a).
However, by inserting a &'a Subscription<'a> inside of a Foo<'a>, we are effectively locking down the Foo for further modifications, since we are now storing a borrow from self.subscriptions_map in self.subscriptions. Consider what would happen if we inserted another item in subscriptions_map: how can we be sure that the HashMap won't move its items around in memory? If the HashMap does move our item, the pointer in self.subscriptions wouldn't be updated automatically and would be dangling.
Now, suppose that we have this buggy remove() method:
impl<'a> Foo<'a> {
fn remove(&mut self, id: &SubscriptionKey) {
self.subscriptions_map.remove(id);
}
}
This method compiles fine. However, if we tried to call this on a Foo on which we called add() earlier, then self.subscriptions would contain a dangling reference to an item that used to be in self.subscriptions_map.
So the reason why the mutable borrow persists after calling add() is that, since the 'a in Foo<'a> is equal to the lifetime of the Foo<'a> itself, the compiler sees that the object borrows from itself. As you know, we can't have a mutable borrow and another borrow (mutable or not) active at the same time, so Rust prevents us from taking a mutable borrow on f while f itself retains an active borrow. In fact, since we used a method that takes self by mutable reference, Rust assumes that Foo<'a> stores a mutable reference, even though that's not the case, since Rust only looks at the signatures to determine borrows (this is to ensure that changing a private field from &'a T to &'a mut T doesn't cause borrow checking failures to you and, if you're developing a library, to your users). Since the type of an object never changes, the Foo<'a> is locked for the rest of its lifetime.
Now, what can you do? Clearly, you can't usefully have a Vec<&'a Subscription<'a>> in your struct. HashMap provides a values() iterator, but it enumerates the values in an unspecified order, so it won't help you if you want to enumerate the values in the order in which they were added. Instead of using borrowed pointers, you could use Rc:
use std::collections::HashMap;
use std::rc::Rc;
#[derive(Eq, PartialEq, Hash)]
struct SubscriptionKey;
struct Subscription<'a>(&'a ());
struct Foo<'a> {
subscriptions_map: HashMap<SubscriptionKey, Rc<Subscription<'a>>>,
subscriptions: Vec<Rc<Subscription<'a>>>,
}
impl<'a> Foo<'a> {
fn add(&mut self, id: SubscriptionKey, item: Subscription<'a>) {
let item = Rc::new(item);
self.subscriptions_map.insert(id, item.clone());
self.subscriptions.push(item);
}
}
fn main() {
let subscription_data = &();
let mut f = Foo {
subscriptions_map: HashMap::new(),
subscriptions: Vec::new(),
};
f.add(SubscriptionKey, Subscription(subscription_data));
}

Tying a trait lifetime variable to &self lifetime

I'd like to do something along the following lines:
trait GetRef<'a> {
fn get_ref(&self) -> &'a [u8];
}
struct Foo<'a> {
buf: &'a [u8]
}
impl <'a> GetRef<'a> for Foo<'a> {
fn get_ref(&self) -> &'a [u8] {
&self.buf[1..]
}
}
struct Bar {
buf: Vec<u8>
}
// this is the part I'm struggling with:
impl <'a> GetRef<'a> for Bar {
fn get_ref(&'a self) -> &'a [u8] {
&self.buf[1..]
}
The point of the explicit lifetime variable in the GetRef trait is to allow the return value of get_ref() on a Foo object to outlive the Foo itself, tying the return value's lifetime to that of the lifetime of Foo's buffer.
However, I haven't found a way to implement GetRef for Bar in a way that the compiler accepts. I've tried several variations of the above, but can't seem to find one that works. Is there any there any reason that this fundamentally cannot be done? If not, how can I do this?
Tying a trait lifetime variable to &self lifetime
Not possible.
Is there any there any reason that this fundamentally cannot be done?
Yes. An owning vector is something different than a borrowed slice. Your trait GetRef only makes sense for things that already represent a “loan” and don't own the slice. For an owning type like Bar you can't safely return a borrowed slice that outlives Self. That's what the borrow checker prevents to avoid dangling pointers.
What you tried to do is to link the lifetime parameter to the lifetime of Self. But the lifetime of Self is not a property of its type. It just depends on the scope this value was defined in. And that's why your approach cannot work.
Another way of looking at it is: In a trait you have to be explicit about whether Self is borrowed by a method and its result or not. You defined the GetRef trait to return something that is not linked to Self w.r.t. lifetimes. So, no borrowing. So, it's not implementable for types that own the data. You can't create a borrowed slice referring to a Vec's elements without borrowing the Vec.
If not, how can I do this?
Depends on what exactly you mean by “this”. If you want to write a “common denominator” trait that can be implemented for both borrowed and owning slices, you have to do it like this:
trait GetRef {
fn get_ref(&self) -> &[u8];
}
The meaning of this trait is that get_ref borrows Self and returns a kind of “loan” because of the current lifetime elision rules. It's equivalent to the more explicit form
trait GetRef {
fn get_ref<'s>(&self) -> &'s [u8];
}
It can be implemented for both types now:
impl<'a> GetRef for Foo<'a> {
fn get_ref(&self) -> &[u8] { &self.buf[1..] }
}
impl GetRef for Bar {
fn get_ref(&self) -> &[u8] { &self.buf[1..] }
}
You could make different lifetimes for &self and result in your trait like that:
trait GetRef<'a, 'b> {
fn get_ref(&'b self) -> &'a [u8];
}
struct Foo<'a> {
buf: &'a [u8]
}
impl <'a, 'b> GetRef<'a, 'b> for Foo<'a> {
fn get_ref(&'b self) -> &'a [u8] {
&self.buf[1..]
}
}
struct Bar {
buf: Vec<u8>
}
// Bar, however, cannot contain anything that outlives itself
impl<'a> GetRef<'a, 'a> for Bar {
fn get_ref(&'a self) -> &'a [u8] {
&self.buf[1..]
}
}
fn main() {
let a = vec!(1 as u8, 2, 3);
let b = a.clone();
let tmp;
{
let x = Foo{buf: &a};
tmp = x.get_ref();
}
{
let y = Bar{buf: b};
// Bar's buf cannot outlive Bar
// tmp = y.get_ref();
}
}

Borrow vs mutable borrow strange failure in lifetimes

While trying to implement an iterator which yields mutable refs to elements of a linked list, I stumbled upon a strange issue.
This works fine:
impl<'a, T> Iterator<&'a T> for LinkedListIterator<'a, T>{
fn next(&mut self) -> Option<&'a T> {
match self.current {
&Cell(ref x, ref xs) => {self.current = &**xs; Some(x)},
&End => None
}
}
}
But this doesn't work; the compiler says lifetime of self is too short to guarantee its contents can be safely reborrowed:
impl<'a, T> Iterator<&'a mut T> for LinkedListMutIterator<'a, T>{
fn next(&mut self) -> Option<&'a mut T> {
match self.current {
&Cell(ref mut x, ref mut xs) => {self.current = &mut **xs; Some(x)},
&End => None
}
}
}
I would expect that either both example work, or both do not, but I can't understand how borrowing something as mutable vs not-mutable would impact the way the compiler checks for lifetimes. Surely if something lives long enough to be safely borrowed, it lives long enough to be safely mutably borrowed?
EDIT: Here is the definition of both Iterators:
pub struct LinkedListIterator<'a, T>
current: &'a LinkedList<T>
}
pub struct LinkedListMutIterator<'a, T> {
current: &'a mut LinkedList<T>
}
LinkedLisk:
#[deriving(Eq, Clone)]
pub enum LinkedList<T> {
Cell(T, ~LinkedList<T>),
End
}
For a complete view of the file, please see https://github.com/TisButMe/rust-algo/blob/mut_iter/LinkedList/linked_list.rs
Note that you've left out the definition(s) of LinkedListMutIterator for the two variant bits of code, which might be relevant to any real attempt to reproduce and dissect your problem.
So, I'll try to guess at what's going on.
The compiler error message here might be misleading you; there are other factors beyond the lifetime of self that may be relevant here.
In particular I suspect the borrow-checker is complaining because it is trying to ensure that you are not creating multiple mutable-borrows that alias the same state.
It is sound to have multiple immutable-borrows to the same piece of state...
... but you cannot have multiple mutable-borrows to the same piece of state (because we want to ensure that if you have a &mut reference to some state, then that reference is the only way to mutate the state).

Resources