When I wondered how a mutable reference could move into a method, all the questions began.
let a = &mut x;
a.somemethod(); // value of a should have moved
a.anothermethod(); // but it works.
I've googled a lot. (really a lot)
And I've noticed that a mutable reference passed as a parameter to a function, always undergoes the following transformation. (which is called reborrowing)
fn test(&mut a) -> ();
let a = &mut x;
test(a); // what we write in code.
test(&mut *a); // the actual action in code.
So, I have googled more about "reborrowing", for its detail.
And this is what I've got.
In any of codes, x refers to an arbitrary data. I don't mention it because I don't think the type of it is important for discussion. (However, I used i32 in my own).
let a = &mut x;
let b = &mut *a; // a isn't available from now on
*a = blahblah; // error! no more access allowed for a
*b = blahblah; // I've wrote this code not to confuse you of the lifetime of b. If not mentioned, it always live till the scope ends.
let a = &mut x;
{
let b = &*a;
// *a = blahblah in this scope will throw an error, just like above case.
}
*a = blahblah; // but this works.
So alright. It's quite interesting. It seems like b borrows not only x but also a.
Perhaps, we can clarify reborrowing like this : &'a *(&'b mut x).
It has borrowed x (which has a lifetime 'a in here),
but also borrowed a (which has a lifetime 'b).
So I ran the following code to confirm my conjecture.
let x: i32 = 1; // I wanted to make it clear that x lives in this scope.
let b;
{
let a = &mut x;
b = &mut *a;
}
*b = 0;
But this works!
What??
But I just decided to get this.
&'a mut *&mutx, not &'a mut *&'b mutx.
I had no idea why mut x is unavailable during the lifetime of &mut *&mutx nor why mut x is re-available after the lifetime of &mut *&mutx, but "okay, let's just say so".
But look at this. It's totally out of my mind for a clear and general understanding.
let x: i32 = 1;
let b;
{
let a = &mut x;
let b = &**&a;
} // error!! a should live longer than b!
Wasn't lifetime simply relying on what the real data b is referring to???
&'a **& &mut x, not &'a **&'b &'c mut x.
And now what??
&'a **&'b &mut x ??? (which was my guess).
How should I accept this complicated situation?
These are some great questions! I'll do my best to answer the ones I can.
The Rust Reference is a great place for finding answers to questions like this, about the deeper semantics of Rust.
First, for your question about method resolution, the Reference
says:
When looking up a method call, the receiver may be automatically
dereferenced or borrowed in order to call a method. This requires a
more complex lookup process than for other functions, since there may
be a number of possible methods to call. The following procedure is
used:
The first step is to build a list of candidate receiver types. Obtain
these by repeatedly dereferencing the receiver
expression's type, adding each type encountered to the list, then
finally attempting an unsized coercion at the end, and adding the
result type if that is successful. Then, for each candidate T, add
&T and &mut T to the list immediately after T.
For instance, if the receiver has type Box<[i32;2]>, then the
candidate types will be Box<[i32;2]>, &Box<[i32;2]>, &mut Box<[i32;2]>, [i32; 2] (by dereferencing), &[i32; 2], &mut [i32; 2], [i32] (by unsized coercion), &[i32], and finally &mut [i32].
The link above goes into more detail.
For the rest of your questions, I think this is mostly about type coercion. Some relevant coercions are:
&mut T to &T
&T or &mut T to &U if T implements Deref<Target = U>
&mut T to &mut U if T implements DerefMut<Target = U>
Notably, &U and &mut U both implement Deref<Target = U>, and &mut U also implements DerefMut<Target = U>. Therefore, the second/third rules lead to the following coercions:
Let T be:
Coerce from
To
&U
&&U
&U
&U
&mut &U
&U
&mut U
&&mut U
&U
&mut U
&mut &mut U
&mut U
Now with the reference out of the way, let's look at your questions in more detail:
let a = &mut x;
let b = &mut *a; // a isn't available from now on
*a = blahblah; // error! no more access allowed for a
*b = blahblah; // I've wrote this code not to confuse you of the lifetime of b. If not mentioned, it always live till the scope ends.
In order to write into a reference, it clearly must be a mutable (aka unique) reference. When you write let a = &mut x, now a is the only way to access x. Now when you write let b = &mut *a, it essentially means let b = &mut *(&mut x), which simplifies to let b = &mut x.
Here, b mutably borrows a, Rust will not let you use a until b is destroyed... or so it seems for now.
let a = &mut x;
{
let b = &*a;
// *a = blahblah in this scope will throw an error, just like above case.
}
*a = blahblah; // but this works.
Here, let b = &*a turns into let b = &*(&mut x), so let b = &x. No matter what kind of reference b is, a is a mutable reference and must be unique, so you can't use it until b is gone (out of scope).
let mut x: i32 = 1; // I wanted to make it clear that x lives in this scope.
let b;
{
let a = &mut x;
b = &mut *a;
}
*b = 0;
(I assume you meant let mut x).
Now this is where it gets interesting. Here, since a and b both mutably point to the same object, Rust will not let you use a until b is destroyed. It seems like the reasoning should be, "since b mutably borrows a", but that's actually not the case. Normally, b would in fact borrow a, and this is the case for smart pointers like Box, std::cell::RefMut, etc. However, references get special treatment: Rust can analyze the memory they point to.
So when you write let b = &mut *a, b doesn't actually borrow a! It only borrows the data a points to. This distinction wasn't relevant before, but here it is what allows the code to compile.
As for your last example, I can't get it to break in the way you described.
Related
I'm having some trouble understanding the compiler error message for this code:
struct A {
b: B,
c: C,
}
struct B {
f: u32,
}
struct C {
f: u32,
}
fn foo(b: &mut B) -> &u32 { &b.f }
fn bar(c: &mut C) -> &u32 { &c.f }
fn quux(z: &mut A) -> u32 { z.b.f }
fn baz(a: &mut A) {
let x = foo(&mut a.b);
let y = bar(&mut a.c);
let z = quux(a); // equivalent to: quux(&mut *a)
println!("{} {}", x, y);
}
error[E0499]: cannot borrow `*a` as mutable more than once at a time
--> src/main.rs:22:18
|
20 | let x = foo(&mut a.b);
| -------- first mutable borrow occurs here
21 | let y = bar(&mut a.c);
22 | let z = quux(a);
| ^ second mutable borrow occurs here
23 | println!("{} {}", x, y);
| - first borrow later used here
For more information about this error, try `rustc --explain E0499`.
error: could not compile `playground` due to previous error
If line 20 is mutably borrowing *a (as the compilation error asserts), then line 21 must also mutably borrow *a (How couldn't it? It's nearly identical to line 20 -- only difference is which field is borrowed), so by my reasoning the code should still fail to compile after deleting line 22; however, the code actually compiles just fine if I delete line 22. The fact that this would then compile seems to contradict the compiler's claim that &mut a.b results in a mutable borrow of *a.
x and y are immutable borrows, so if anything I would expect a compilation error regarding the overlapping scopes of the immutable x/y and the mutable borrow of *a in line 22 (otherwise quux could conceivably mutate a while we're borrowing its fields).
What knowledge am I likely missing that makes this error message confusing to me? If you're not surprised by this message, could you break down how you reason about this?
Some more thinking before I submitted this question:
I suppose the immutable references returned from foo and bar somehow extend the original mutable borrows of a.b and a.c (why that would be necessary, I don't know). If that's the case, maybe the way to see this is that &mut a.b and &mut a.c in turn extend the mutable borrow of *a. The error message says the first mutable borrow of *a occurs on line 20, but maybe the way to see this is:
The true first borrow of *a is when when the caller gives us the &mut A.
While the compiler suggests there's a mutable borrow of *a on line 20, this isn't exactly the case: the mut &a.b expression borrows (*a).b, but merely extends the existing borrow given to us (as opposed to being a new mutable borrow of *a).
The last point, but with respect to mut &a.c.
Insert the aforementioned reasoning of "extending borrows" wrt *a.
The call to quux then mutably borrows the *a a second time.
So maybe what the compilation error calls the initial borrow of *a is actually reporting the location of the expression responsible for extending the original *a borrow?
Line 20 does not conflict with 21 since they borrow different fields within a and therefore cause no collision.
The issue is that foo and bar return references that inherit the lifetime of the mutable reference. Even though they return immutable references, their lifetimes rely on the lifetime of the mutable borrow. So by requiring they live for a period of time, by extension the mutable borrow must do so as well. The compiler always takes the function signature on face value so it can not downgrade the initial mutable reference. If it did not, it could lead to issues if the function was modified later or if it contained unsafe code that relied on consuming a mutable reference for safety.
Here is a counter example where all of the function signatures remain the same. Since the compiler can not make assumptions on how quux and foo will use their values quux may make the references returned by foo or bar invalid.
struct A {
b: B,
c: C,
}
struct B {
f: Option<u32>,
}
struct C {
f: u32,
}
fn foo(b: &mut B) -> &u32 {
b.f.as_ref().unwrap()
}
fn bar(c: &mut C) -> &u32 { &c.f }
fn quux(z: &mut A) -> u32 {
z.b.f = None;
z.c.f
}
fn baz(a: &mut A) {
let x = foo(&mut a.b);
let y = bar(&mut a.c);
let z = quux(a);
println!("{} {}", x, y);
}
There are two issues here. First: why does let x = foo(&mut a.b); prevent you from doing let z = quux(a);, and second why does let x = foo(&mut a.b); not prevent you from doing let y = bar(&mut a.c);?
Why does let x = foo(&mut a.b); prevent you from doing let z = quux(a);?
This is due to the lifetime elision rules. You declared foo as:
fn foo(b: &mut B) -> &u32
Since you use references there must be a lifetime, but since you didn't specify one yourself the compiler assumes that all references have the same lifetime. In other words, the compiler rewrites your code as:
fn foo<'a>(b: &'a mut B) -> &'a u32
Which means that the returned value has the same lifetime as the mutable borrow, so the mutable borrow lasts until the println where you use the return value x, which prevents other borrows of a.b or a as a whole.
But then why does let x = foo(&mut a.b); not prevent you from doing let y = bar(&mut a.c);?
That's because the compiler is smart enough to realize that a.b and a.c are disjoint parts of a, which can be borrowed without conflict (in the same way that you can borrow different local variables simultaneously).
I am trying to understand what exactly happens when functions reborrow mutable references.
fn main() {
let mut a = String::new();
let mut b = String::new();
let aa = &mut a;
let c = my_fun(aa, &mut b);
let d = &mut a;
println!("{}", c);
}
fn my_fun<'b>(x: &'b mut String, y: &'b mut String) -> &'b mut String { y }
From my understanding the my_fun reborrows aa as &*aa whose scope would be the my_fun. But due to the lifetime bound I created in the function signature the reborrow should live at least as long as &mut b exist. So the println force the reborrow to live until it.
Shouldn't this be throwing an error of use after free because the anonymous reborrow has only scope inside my_fun? Outside of this function this reference shouldn't be valid.
But the error I get is:
error[E0499]: cannot borrow `a` as mutable more than once at a time
--> src/main.rs:7:13
|
5 | let aa= &mut a;
| ------ first mutable borrow occurs here
6 | let c = my_fun(aa, &mut b);
7 | let d = &mut a;
| ^^^^^^ second mutable borrow occurs here
8 | println!("{}", c);
| - first borrow later used
which would have made sense if the mutable reference was merely copied instead of reborrowed inside the function.
From my understanding the my_fun reborrows aa as &*aa whose scope would be the my_fun.
It's not quite that.
Let's backtrack a bit: why reborrowing?
There is a fundamental difference between &T and &mut T: &T is Copy, whereas &mut T is not even Clone. The result is that &mut T can only be moved and therefore a call such as:
let x: &mut T = /*...*/;
func(x);
Would result in x being unusable after the call. The work-around would then be to introduce a temporary:
let x: &mut T = /*...*/;
let tmp = &mut *x;
func(tmp);
This temporary would re-borrow the pointee of x, and be consumed by func.
And... that's re-borrowing! The compiler has this "temporary" creation built-in purely for ergonomics!
With that in mind, let's go back to:
From my understanding the my_fun reborrows aa as &*aa whose scope would be the my_fun.
Lifetimes are generally more a range than a point, and this is true here.
The lifetime of tmp in my example above is constrained in 2 ways:
It cannot be greater than that of x.
It cannot be less than that of func.
Which is another way of saying that it can be anything in between those bounds.
I believe you're overthinking "reborrowing" here.
The lifetime requirements you applied say that the thing being referenced by the return value will have at least the lifetime of the things being referenced by the parameters. That's true (and if it weren't provably true, this wouldn't compile). So there is no dangling reference possible.
There isn't a separate "reborrowed" reference. Borrowing is bookkeeping inside the compiler to keep track of lifetimes. There is no let x = &*aa step that actually occurs or is even particularly implied. This isn't like reference counting where memory actually changes at runtime.
Inside of my_fun, y is a reference that's scoped to the function. But the return value is scoped to the caller. (Functions would be impossible to use if this weren't true, having nothing to do with &mut.)
I was playing around with Rust closures when I hit this interesting scenario:
fn main() {
let mut y = 10;
let f = || &mut y;
f();
}
This gives an error:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/main.rs:4:16
|
4 | let f = || &mut y;
| ^^^^^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 4:13...
--> src/main.rs:4:13
|
4 | let f = || &mut y;
| ^^^^^^^^^
note: ...so that closure can access `y`
--> src/main.rs:4:16
|
4 | let f = || &mut y;
| ^^^^^^
note: but, the lifetime must be valid for the call at 6:5...
--> src/main.rs:6:5
|
6 | f();
| ^^^
note: ...so type `&mut i32` of expression is valid during the expression
--> src/main.rs:6:5
|
6 | f();
| ^^^
Even though the compiler is trying to explain it line by line, I still haven't understood what exactly it is complaining about.
Is it trying to say that the mutable reference cannot outlive the enclosing closure?
The compiler does not complain if I remove the call f().
Short version
The closure f stores a mutable reference to y. If it were allowed to return a copy of this reference, you would end up with two simultaneous mutable references to y (one in the closure, one returned), which is forbidden by Rust's memory safety rules.
Long version
The closure can be thought of as
struct __Closure<'a> {
y: &'a mut i32,
}
Since it contains a mutable reference, the closure is called as FnMut, essentially with the definition
fn call_mut(&mut self, args: ()) -> &'a mut i32 { self.y }
Since we only have a mutable reference to the closure itself, we can't move the field y out, neither are we able to copy it, since mutable references aren't Copy.
We can trick the compiler into accepting the code by forcing the closure to be called as FnOnce instead of FnMut. This code works fine:
fn main() {
let x = String::new();
let mut y: u32 = 10;
let f = || {
drop(x);
&mut y
};
f();
}
Since we are consuming x inside the scope of the closure and x is not Copy, the compiler detects that the closure can only be FnOnce. Calling an FnOnce closure passes the closure itself by value, so we are allowed to move the mutable reference out.
Another more explicit way to force the closure to be FnOnce is to pass it to a generic function with a trait bound. This code works fine as well:
fn make_fn_once<'a, T, F: FnOnce() -> T>(f: F) -> F {
f
}
fn main() {
let mut y: u32 = 10;
let f = make_fn_once(|| {
&mut y
});
f();
}
There are two main things at play here:
Closures cannot return references to their environment
A mutable reference to a mutable reference can only use the lifetime of the outer reference (unlike with immutable references)
Closures returning references to environment
Closures cannot return any references with the lifetime of self (the closure object). Why is that? Every closure can be called as FnOnce, since that's the super-trait of FnMut which in turn is the super-trait of Fn. FnOnce has this method:
fn call_once(self, args: Args) -> Self::Output;
Note that self is passed by value. So since self is consumed (and now lives within the call_once function`) we cannot return references to it -- that would be equivalent to returning references to a local function variable.
In theory, the call_mut would allow to return references to self (since it receives &mut self). But since call_once, call_mut and call are all implemented with the same body, closures in general cannot return references to self (that is: to their captured environment).
Just to be sure: closures can capture references and return those! And they can capture by reference and return that reference. Those things are something different. It's just about what is stored in the closure type. If there is a reference stored within the type, it can be returned. But we can't return references to anything stored within the closure type.
Nested mutable references
Consider this function (note that the argument type implies 'inner: 'outer; 'outer being shorter than 'inner):
fn foo<'outer, 'inner>(x: &'outer mut &'inner mut i32) -> &'inner mut i32 {
*x
}
This won't compile. On the first glance, it seems like it should compile, since we're just peeling one layer of references. And it does work for immutable references! But mutable references are different here to preserve soundness.
It's OK to return &'outer mut i32, though. But it's impossible to get a direct reference with the longer (inner) lifetime.
Manually writing the closure
Let's try to hand code the closure you were trying to write:
let mut y = 10;
struct Foo<'a>(&'a mut i32);
impl<'a> Foo<'a> {
fn call<'s>(&'s mut self) -> &'??? mut i32 { self.0 }
}
let mut f = Foo(&mut y);
f.call();
What lifetime should the returned reference have?
It can't be 'a, because we basically have a &'s mut &'a mut i32. And as discussed above, in such a nested mutable reference situation, we can't extract the longer lifetime!
But it also can't be 's since that would mean the closure returns something with the lifetime of 'self ("borrowed from self"). And as discussed above, closures can't do that.
So the compiler can't generate the closure impls for us.
Consider this code:
fn main() {
let mut y: u32 = 10;
let ry = &mut y;
let f = || ry;
f();
}
It works because the compiler is able to infer ry's lifetime: the reference ry lives in the same scope of y.
Now, the equivalent version of your code:
fn main() {
let mut y: u32 = 10;
let f = || {
let ry = &mut y;
ry
};
f();
}
Now the compiler assigns to ry a lifetime associated to the scope of the closure body, not to the lifetime associated with the main body.
Also note that the immutable reference case works:
fn main() {
let mut y: u32 = 10;
let f = || {
let ry = &y;
ry
};
f();
}
This is because &T has copy semantics and &mut T has move semantics, see Copy/move semantics documentation of &T/&mut T types itself for more details.
The missing piece
The compiler throws an error related to a lifetime:
cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
but as pointed out by Sven Marnach there is also a problem related to the error
cannot move out of borrowed content
But why doesn't the compiler throw this error?
The short answer is that the compiler first executes type checking and then borrow checking.
the long answer
A closure is made up of two pieces:
the state of the closure: a struct containing all the variables captured by the closure
the logic of the closure: an implementation of the FnOnce, FnMut or Fn trait
In this case the state of the closure is the mutable reference y and the logic is the body of the closure { &mut y } that simply returns a mutable reference.
When a reference is encountered, Rust controls two aspects:
the state: if the reference points to a valid memory slice, (i.e. the read-only part of lifetime validity);
the logic: if the memory slice is aliased, in other words if it is pointed from more than one reference simultaneously;
Note the move out from borrowed content is forbidden for avoiding memory aliasing.
The Rust compiler executes its job through several stages, here's a simplified workflow:
.rs input -> AST -> HIR -> HIR postprocessing -> MIR -> HIR postprocessing -> LLVM IR -> binary
The compiler reports a lifetime problem because it first executes the type checking phase in HIR postprocessing (which comprises lifetime analysis) and after that, if successful, executes borrow checking in the MIR postprocessing phase.
In the chapter Lifetimes of the Rust book, there's an example:
struct Foo<'a> {
x: &'a i32,
}
fn main() {
let y = &5; // this is the same as `let _y = 5; let y = &_y;`
let f = Foo { x: y };
println!("{}", f.x);
}
Why do they use x: &'a i32?
I think if it is just x: i32 then they cannot demonstrate the lifetime usage. However, is there any other reason behind it? Is there any production code that uses immutable reference to a primitive type like i32?
In this particular case the reason is indeed to show the concept of lifetimes. As for the general case, however, I see no reason making an immutable reference to a primitive type (mutable references, of course, is another matter) except of when it is done in generic code:
struct Holder<'a, T> {
r: &'a T
}
let x: i32 = 123;
let h: Holder<i32> = Holder { r: &x };
Here if you have such structure, you have no other choice as to use a reference to an i32. Naturally, this structure can also be used with other, non-primitive and non-movable types.
As Shepmaster has mentioned in comments, there is indeed a case where you have references to primitive types - it is by-reference iterators. Remember, by a convention (which the standard library follows) iter() method on a collection should return an iterator of references into the collection:
let v: Vec<i32> = vec![1, 2, 3, 4];
let i = v.iter(); // i is Iterator<Item=&i32>
Then almost all methods on the iterator which take a closure will accept closures whose argument is a reference:
i.map(|n| *n + 1) // n is of type &i32
Note that this is in fact a consequence of the more general case with generics. Vectors and slices may contain arbitrary types, including non-moveable ones, so they just have to have methods which would allow their users to borrow their contents.
This code fails as expected at let c = a; with compile error "use of moved value: a":
fn main() {
let a: &mut i32 = &mut 0;
let b = a;
let c = a;
}
a is moved into b and is no longer available for an assignment to c. So far, so good.
However, if I just annotate b's type and leave everything else alone:
fn main() {
let a: &mut i32 = &mut 0;
let b: &mut i32 = a;
let c = a;
}
the code fails again at let c = a;
But this time with a very different error message: "cannot move out of a because it is borrowed ... borrow of *a occurs here: let b: &mut i32 = a;"
So, if I just annotate b's type: no move of a into b, but instead a "re"-borrow of *a?
What am I missing?
Cheers.
So, if I just annotate b's type: no move of a into b, but instead a "re"-borrow of *a?
What am I missing?
Absolutely nothing, as in this case these two operations are semantically very similar (and equivalent if a and b belong to the same scope).
Either you move the reference a into b, making a a moved value, and no longer available.
Either you reborrow *a in b, making a unusable as long as b is in scope.
The second case is less definitive than the first, you can show this by putting the line defining b into a sub-scope.
This example won't compile because a is moved:
fn main() {
let a: &mut i32 = &mut 0;
{ let b = a; }
let c = a;
}
But this one will, because once b goes out of scope a is unlocked:
fn main() {
let a: &mut i32 = &mut 0;
{ let b = &mut *a; }
let c = a;
}
Now, to the question "Why does annotating the type of b change the behavior ?", my guess would be:
When there is no type annotation, the operation is a simple and straightforward move. Nothing is needed to be checked.
When there is a type annotation, a conversion may be needed (casting a &mut _ into a &_, or transforming a simple reference into a reference to a trait object). So the compiler opts for a re-borrow of the value, rather than a move.
For example, this code is perflectly valid:
fn main() {
let a: &mut i32 = &mut 0;
let b: &i32 = a;
}
and here moving a into b would not make any sense, as they are of different type. Still this code compiles: b simply re-borrows *a, and the value won't be mutably available through a as long as b is in scope.
To complement #Levans's answer on the specific question "Why does annotating the type change the behaviour?":
When you don't write the type, the compiler performs a simple move. When you do put the type, the let statement becomes a coercion site as documented in "Coercion sites":
Possible coercion sites are:
let statements where an explicit type is given.
In the present case the compiler performs a reborrow coercion, which is a special case of coercion going from &mut to &mut, as explained in this issue comment on GitHub.
Note that reborrowing in general and reborrow coercion in particular are currently poorly documented. There is an open issue on the Rust Reference to improve that point.