Despite thoroughly reading the documentation, I'm rather confused about the meaning of the & and * symbol in Rust, and more generally about what is a Rust reference exactly.
In this example, it seems to be similar to a C++ reference (that is, an address that is automatically dereferenced when used):
fn main() {
let c: i32 = 5;
let rc = &c;
let next = rc + 1;
println!("{}", next); // 6
}
However, the following code works exactly the same:
fn main() {
let c: i32 = 5;
let rc = &c;
let next = *rc + 1;
println!("{}", next); // 6
}
Using * to dereference a reference wouldn't be correct in C++. So I'd like to understand why this is correct in Rust.
My understanding so far, is that, inserting * in front of a Rust reference dereferences it, but the * is implicitly inserted anyway so you don't need to add it (while in C++, it's implicitly inserted and if you insert it you get a compilation error).
However, something like this doesn't compile:
fn main() {
let mut c: i32 = 5;
let mut next: i32 = 0;
{
let rc = &mut c;
next = rc + 1;
}
println!("{}", next);
}
error[E0369]: binary operation `+` cannot be applied to type `&mut i32`
--> src/main.rs:6:16
|
6 | next = rc + 1;
| ^^^^^^
|
= note: this is a reference to a type that `+` can be applied to; you need to dereference this variable once for this operation to work
= note: an implementation of `std::ops::Add` might be missing for `&mut i32`
But this works:
fn main() {
let mut c: i32 = 5;
let mut next: i32 = 0;
{
let rc = &mut c;
next = *rc + 1;
}
println!("{}", next); // 6
}
It seems that implicit dereferencing (a la C++) is correct for immutable references, but not for mutable references. Why is this?
Using * to dereference a reference wouldn't be correct in C++. So I'd like to understand why this is correct in Rust.
A reference in C++ is not the same as a reference in Rust. Rust's references are much closer (in usage, not in semantics) to C++'s pointers. With respect to memory representation, Rust's references often are just a single pointer, while C++'s references are supposed to be alternative names of the same object (and thus have no memory representation).
The difference between C++ pointers and Rust references is that Rust's references are never NULL, never uninitialized and never dangling.
The Add trait is implemented (see the bottom of the doc page) for the following pairs and all other numeric primitives:
&i32 + i32
i32 + &i32
&i32 + &i32
This is just a convenience thing the std-lib developers implemented. The compiler can figure out that a &mut i32 can be used wherever a &i32 can be used, but that doesn't work (yet?) for generics, so the std-lib developers would need to also implement the Add traits for the following combinations (and those for all primitives):
&mut i32 + i32
i32 + &mut i32
&mut i32 + &mut i32
&mut i32 + &i32
&i32 + &mut i32
As you can see that can get quite out of hand. I'm sure that will go away in the future. Until then, note that it's rather rare to end up with a &mut i32 and trying to use it in a mathematical expression.
This answer is for those looking for the basics (e.g. coming from Google).
From the Rust book's References and Borrowing:
fn main() {
let s1 = String::from("hello");
let len = calculate_length(&s1);
println!("The length of '{}' is {}.", s1, len);
}
fn calculate_length(s: &String) -> usize {
s.len()
}
These ampersands represent references, and they allow you to refer to some value without taking ownership of it [i.e. borrowing].
The opposite of referencing by using & is dereferencing, which is accomplished with the dereference operator, *.
And a basic example:
let x = 5;
let y = &x; //set y to a reference to x
assert_eq!(5, x);
assert_eq!(5, *y); // dereference y
If we tried to write assert_eq!(5, y); instead, we would get a compilation error can't compare `{integer}` with `&{integer}`.
(You can read more in the Smart Pointers chapter.)
And from Method Syntax:
Rust has a feature called automatic referencing and dereferencing. Calling methods is one of the few places in Rust that has this behavior.
Here’s how it works: when you call a method with object.something(), Rust automatically adds in &, &mut, or * so object matches the signature of the method. In other words, the following are the same:
p1.distance(&p2);
(&p1).distance(&p2);
From the docs for std::ops::Add:
impl<'a, 'b> Add<&'a i32> for &'b i32
impl<'a> Add<&'a i32> for i32
impl<'a> Add<i32> for &'a i32
impl Add<i32> for i32
It seems the binary + operator for numbers is implemented for combinations of shared (but not mutable) references of the operands and owned versions of the operands. It has nothing to do with automatic dereferencing.
Related
I was playing around with Rust closures when I hit this interesting scenario:
fn main() {
let mut y = 10;
let f = || &mut y;
f();
}
This gives an error:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/main.rs:4:16
|
4 | let f = || &mut y;
| ^^^^^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 4:13...
--> src/main.rs:4:13
|
4 | let f = || &mut y;
| ^^^^^^^^^
note: ...so that closure can access `y`
--> src/main.rs:4:16
|
4 | let f = || &mut y;
| ^^^^^^
note: but, the lifetime must be valid for the call at 6:5...
--> src/main.rs:6:5
|
6 | f();
| ^^^
note: ...so type `&mut i32` of expression is valid during the expression
--> src/main.rs:6:5
|
6 | f();
| ^^^
Even though the compiler is trying to explain it line by line, I still haven't understood what exactly it is complaining about.
Is it trying to say that the mutable reference cannot outlive the enclosing closure?
The compiler does not complain if I remove the call f().
Short version
The closure f stores a mutable reference to y. If it were allowed to return a copy of this reference, you would end up with two simultaneous mutable references to y (one in the closure, one returned), which is forbidden by Rust's memory safety rules.
Long version
The closure can be thought of as
struct __Closure<'a> {
y: &'a mut i32,
}
Since it contains a mutable reference, the closure is called as FnMut, essentially with the definition
fn call_mut(&mut self, args: ()) -> &'a mut i32 { self.y }
Since we only have a mutable reference to the closure itself, we can't move the field y out, neither are we able to copy it, since mutable references aren't Copy.
We can trick the compiler into accepting the code by forcing the closure to be called as FnOnce instead of FnMut. This code works fine:
fn main() {
let x = String::new();
let mut y: u32 = 10;
let f = || {
drop(x);
&mut y
};
f();
}
Since we are consuming x inside the scope of the closure and x is not Copy, the compiler detects that the closure can only be FnOnce. Calling an FnOnce closure passes the closure itself by value, so we are allowed to move the mutable reference out.
Another more explicit way to force the closure to be FnOnce is to pass it to a generic function with a trait bound. This code works fine as well:
fn make_fn_once<'a, T, F: FnOnce() -> T>(f: F) -> F {
f
}
fn main() {
let mut y: u32 = 10;
let f = make_fn_once(|| {
&mut y
});
f();
}
There are two main things at play here:
Closures cannot return references to their environment
A mutable reference to a mutable reference can only use the lifetime of the outer reference (unlike with immutable references)
Closures returning references to environment
Closures cannot return any references with the lifetime of self (the closure object). Why is that? Every closure can be called as FnOnce, since that's the super-trait of FnMut which in turn is the super-trait of Fn. FnOnce has this method:
fn call_once(self, args: Args) -> Self::Output;
Note that self is passed by value. So since self is consumed (and now lives within the call_once function`) we cannot return references to it -- that would be equivalent to returning references to a local function variable.
In theory, the call_mut would allow to return references to self (since it receives &mut self). But since call_once, call_mut and call are all implemented with the same body, closures in general cannot return references to self (that is: to their captured environment).
Just to be sure: closures can capture references and return those! And they can capture by reference and return that reference. Those things are something different. It's just about what is stored in the closure type. If there is a reference stored within the type, it can be returned. But we can't return references to anything stored within the closure type.
Nested mutable references
Consider this function (note that the argument type implies 'inner: 'outer; 'outer being shorter than 'inner):
fn foo<'outer, 'inner>(x: &'outer mut &'inner mut i32) -> &'inner mut i32 {
*x
}
This won't compile. On the first glance, it seems like it should compile, since we're just peeling one layer of references. And it does work for immutable references! But mutable references are different here to preserve soundness.
It's OK to return &'outer mut i32, though. But it's impossible to get a direct reference with the longer (inner) lifetime.
Manually writing the closure
Let's try to hand code the closure you were trying to write:
let mut y = 10;
struct Foo<'a>(&'a mut i32);
impl<'a> Foo<'a> {
fn call<'s>(&'s mut self) -> &'??? mut i32 { self.0 }
}
let mut f = Foo(&mut y);
f.call();
What lifetime should the returned reference have?
It can't be 'a, because we basically have a &'s mut &'a mut i32. And as discussed above, in such a nested mutable reference situation, we can't extract the longer lifetime!
But it also can't be 's since that would mean the closure returns something with the lifetime of 'self ("borrowed from self"). And as discussed above, closures can't do that.
So the compiler can't generate the closure impls for us.
Consider this code:
fn main() {
let mut y: u32 = 10;
let ry = &mut y;
let f = || ry;
f();
}
It works because the compiler is able to infer ry's lifetime: the reference ry lives in the same scope of y.
Now, the equivalent version of your code:
fn main() {
let mut y: u32 = 10;
let f = || {
let ry = &mut y;
ry
};
f();
}
Now the compiler assigns to ry a lifetime associated to the scope of the closure body, not to the lifetime associated with the main body.
Also note that the immutable reference case works:
fn main() {
let mut y: u32 = 10;
let f = || {
let ry = &y;
ry
};
f();
}
This is because &T has copy semantics and &mut T has move semantics, see Copy/move semantics documentation of &T/&mut T types itself for more details.
The missing piece
The compiler throws an error related to a lifetime:
cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
but as pointed out by Sven Marnach there is also a problem related to the error
cannot move out of borrowed content
But why doesn't the compiler throw this error?
The short answer is that the compiler first executes type checking and then borrow checking.
the long answer
A closure is made up of two pieces:
the state of the closure: a struct containing all the variables captured by the closure
the logic of the closure: an implementation of the FnOnce, FnMut or Fn trait
In this case the state of the closure is the mutable reference y and the logic is the body of the closure { &mut y } that simply returns a mutable reference.
When a reference is encountered, Rust controls two aspects:
the state: if the reference points to a valid memory slice, (i.e. the read-only part of lifetime validity);
the logic: if the memory slice is aliased, in other words if it is pointed from more than one reference simultaneously;
Note the move out from borrowed content is forbidden for avoiding memory aliasing.
The Rust compiler executes its job through several stages, here's a simplified workflow:
.rs input -> AST -> HIR -> HIR postprocessing -> MIR -> HIR postprocessing -> LLVM IR -> binary
The compiler reports a lifetime problem because it first executes the type checking phase in HIR postprocessing (which comprises lifetime analysis) and after that, if successful, executes borrow checking in the MIR postprocessing phase.
I was playing around with Rust closures when I hit this interesting scenario:
fn main() {
let mut y = 10;
let f = || &mut y;
f();
}
This gives an error:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/main.rs:4:16
|
4 | let f = || &mut y;
| ^^^^^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 4:13...
--> src/main.rs:4:13
|
4 | let f = || &mut y;
| ^^^^^^^^^
note: ...so that closure can access `y`
--> src/main.rs:4:16
|
4 | let f = || &mut y;
| ^^^^^^
note: but, the lifetime must be valid for the call at 6:5...
--> src/main.rs:6:5
|
6 | f();
| ^^^
note: ...so type `&mut i32` of expression is valid during the expression
--> src/main.rs:6:5
|
6 | f();
| ^^^
Even though the compiler is trying to explain it line by line, I still haven't understood what exactly it is complaining about.
Is it trying to say that the mutable reference cannot outlive the enclosing closure?
The compiler does not complain if I remove the call f().
Short version
The closure f stores a mutable reference to y. If it were allowed to return a copy of this reference, you would end up with two simultaneous mutable references to y (one in the closure, one returned), which is forbidden by Rust's memory safety rules.
Long version
The closure can be thought of as
struct __Closure<'a> {
y: &'a mut i32,
}
Since it contains a mutable reference, the closure is called as FnMut, essentially with the definition
fn call_mut(&mut self, args: ()) -> &'a mut i32 { self.y }
Since we only have a mutable reference to the closure itself, we can't move the field y out, neither are we able to copy it, since mutable references aren't Copy.
We can trick the compiler into accepting the code by forcing the closure to be called as FnOnce instead of FnMut. This code works fine:
fn main() {
let x = String::new();
let mut y: u32 = 10;
let f = || {
drop(x);
&mut y
};
f();
}
Since we are consuming x inside the scope of the closure and x is not Copy, the compiler detects that the closure can only be FnOnce. Calling an FnOnce closure passes the closure itself by value, so we are allowed to move the mutable reference out.
Another more explicit way to force the closure to be FnOnce is to pass it to a generic function with a trait bound. This code works fine as well:
fn make_fn_once<'a, T, F: FnOnce() -> T>(f: F) -> F {
f
}
fn main() {
let mut y: u32 = 10;
let f = make_fn_once(|| {
&mut y
});
f();
}
There are two main things at play here:
Closures cannot return references to their environment
A mutable reference to a mutable reference can only use the lifetime of the outer reference (unlike with immutable references)
Closures returning references to environment
Closures cannot return any references with the lifetime of self (the closure object). Why is that? Every closure can be called as FnOnce, since that's the super-trait of FnMut which in turn is the super-trait of Fn. FnOnce has this method:
fn call_once(self, args: Args) -> Self::Output;
Note that self is passed by value. So since self is consumed (and now lives within the call_once function`) we cannot return references to it -- that would be equivalent to returning references to a local function variable.
In theory, the call_mut would allow to return references to self (since it receives &mut self). But since call_once, call_mut and call are all implemented with the same body, closures in general cannot return references to self (that is: to their captured environment).
Just to be sure: closures can capture references and return those! And they can capture by reference and return that reference. Those things are something different. It's just about what is stored in the closure type. If there is a reference stored within the type, it can be returned. But we can't return references to anything stored within the closure type.
Nested mutable references
Consider this function (note that the argument type implies 'inner: 'outer; 'outer being shorter than 'inner):
fn foo<'outer, 'inner>(x: &'outer mut &'inner mut i32) -> &'inner mut i32 {
*x
}
This won't compile. On the first glance, it seems like it should compile, since we're just peeling one layer of references. And it does work for immutable references! But mutable references are different here to preserve soundness.
It's OK to return &'outer mut i32, though. But it's impossible to get a direct reference with the longer (inner) lifetime.
Manually writing the closure
Let's try to hand code the closure you were trying to write:
let mut y = 10;
struct Foo<'a>(&'a mut i32);
impl<'a> Foo<'a> {
fn call<'s>(&'s mut self) -> &'??? mut i32 { self.0 }
}
let mut f = Foo(&mut y);
f.call();
What lifetime should the returned reference have?
It can't be 'a, because we basically have a &'s mut &'a mut i32. And as discussed above, in such a nested mutable reference situation, we can't extract the longer lifetime!
But it also can't be 's since that would mean the closure returns something with the lifetime of 'self ("borrowed from self"). And as discussed above, closures can't do that.
So the compiler can't generate the closure impls for us.
Consider this code:
fn main() {
let mut y: u32 = 10;
let ry = &mut y;
let f = || ry;
f();
}
It works because the compiler is able to infer ry's lifetime: the reference ry lives in the same scope of y.
Now, the equivalent version of your code:
fn main() {
let mut y: u32 = 10;
let f = || {
let ry = &mut y;
ry
};
f();
}
Now the compiler assigns to ry a lifetime associated to the scope of the closure body, not to the lifetime associated with the main body.
Also note that the immutable reference case works:
fn main() {
let mut y: u32 = 10;
let f = || {
let ry = &y;
ry
};
f();
}
This is because &T has copy semantics and &mut T has move semantics, see Copy/move semantics documentation of &T/&mut T types itself for more details.
The missing piece
The compiler throws an error related to a lifetime:
cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
but as pointed out by Sven Marnach there is also a problem related to the error
cannot move out of borrowed content
But why doesn't the compiler throw this error?
The short answer is that the compiler first executes type checking and then borrow checking.
the long answer
A closure is made up of two pieces:
the state of the closure: a struct containing all the variables captured by the closure
the logic of the closure: an implementation of the FnOnce, FnMut or Fn trait
In this case the state of the closure is the mutable reference y and the logic is the body of the closure { &mut y } that simply returns a mutable reference.
When a reference is encountered, Rust controls two aspects:
the state: if the reference points to a valid memory slice, (i.e. the read-only part of lifetime validity);
the logic: if the memory slice is aliased, in other words if it is pointed from more than one reference simultaneously;
Note the move out from borrowed content is forbidden for avoiding memory aliasing.
The Rust compiler executes its job through several stages, here's a simplified workflow:
.rs input -> AST -> HIR -> HIR postprocessing -> MIR -> HIR postprocessing -> LLVM IR -> binary
The compiler reports a lifetime problem because it first executes the type checking phase in HIR postprocessing (which comprises lifetime analysis) and after that, if successful, executes borrow checking in the MIR postprocessing phase.
I want to implement a generic fibonacci function that works with any type implementing Zero, One, and AddAssign. I first implemented a version that works fine, but is specialized for num::BigUint (see on play.rust-lang.org). I than came up with the following generic implementation (see on play.rust-lang.org):
extern crate num;
use num::{One, Zero};
use std::mem::swap;
use std::ops::AddAssign;
fn fib<T: Zero + One + AddAssign<&T>>(n: usize) -> T {
let mut f0 = Zero::zero();
let mut f1 = One::one();
for _ in 0..n {
f0 += &f1;
swap(&mut f0, &mut f1);
}
f0
}
This doesn't compile:
error[E0106]: missing lifetime specifier
--> src/main.rs:7:34
|
7 | fn fib<T: Zero + One + AddAssign<&T>>(n: usize) -> T {
| ^ expected lifetime parameter
Rust wants me to add a lifetime parameter to AddAssign<&T> but I don't know how to express the lifetime of f1.
You need to use Higher Rank Trait Bounds. This one means basically "For any lifetime 'a, T satisfies the AddAssign<&'a T> trait":
fn fib<T>(n: usize) -> T
where
for<'a> T: Zero + One + AddAssign<&'a T>,
I also had to change the way fib is called because the compiler couldn't figure out the return type, which could be literally any type that implements those traits. Declaring x's type gives sufficient context to the compiler so that it knows what you want.
fn main() {
let x: num::BigUint = fib(10);
// let x = fib::<BigUint>(10); // Also works
println!("fib(10) = {}", x);
}
playground
I'm new to Rust and I don't understand the following piece of code:
let mut x = 5;
{
let y = &mut x;
*y += 1;
}
println!("{}", x);
Explanation from the Rust site:
You'll also notice we added an asterisk (*) in front of y, making it *y, this is because y is a &mut reference. You'll need to use astrisks [sic] to access the contents of a reference as well.
If *y is a reference, why does the following code work
fn main() {
let mut x = 5;
{
let y = &mut x;
println!("{}", y);
}
}
I know I'm not modifying the value here, but what is the difference and why
would y += 1; not work?
If *y is a reference
*y is not a reference. y is a reference; *y dereferences y, allowing you access to the referred-to value.
what is the difference [between += and println!]
println! is a macro that automatically references the arguments given to it. In addition, the Display trait (used via {} in the format string) is implemented for all references to types that themselves implement Display (impl<'a, T> Display for &'a T where T: Display + ?Sized).
Thus, println!("{}", y); is actually printing out a reference to a reference to a value. Those intermediate references are automatically dereferenced due to the implementation of Display.
+=, on the other hand, is implemented via the AddAssign trait. The standard library only implements adding an integer type to itself (impl AddAssign<i32> for i32). That means that you have to add an appropriate level of dereferencing in order to get both sides to an integer.
In the chapter Lifetimes of the Rust book, there's an example:
struct Foo<'a> {
x: &'a i32,
}
fn main() {
let y = &5; // this is the same as `let _y = 5; let y = &_y;`
let f = Foo { x: y };
println!("{}", f.x);
}
Why do they use x: &'a i32?
I think if it is just x: i32 then they cannot demonstrate the lifetime usage. However, is there any other reason behind it? Is there any production code that uses immutable reference to a primitive type like i32?
In this particular case the reason is indeed to show the concept of lifetimes. As for the general case, however, I see no reason making an immutable reference to a primitive type (mutable references, of course, is another matter) except of when it is done in generic code:
struct Holder<'a, T> {
r: &'a T
}
let x: i32 = 123;
let h: Holder<i32> = Holder { r: &x };
Here if you have such structure, you have no other choice as to use a reference to an i32. Naturally, this structure can also be used with other, non-primitive and non-movable types.
As Shepmaster has mentioned in comments, there is indeed a case where you have references to primitive types - it is by-reference iterators. Remember, by a convention (which the standard library follows) iter() method on a collection should return an iterator of references into the collection:
let v: Vec<i32> = vec![1, 2, 3, 4];
let i = v.iter(); // i is Iterator<Item=&i32>
Then almost all methods on the iterator which take a closure will accept closures whose argument is a reference:
i.map(|n| *n + 1) // n is of type &i32
Note that this is in fact a consequence of the more general case with generics. Vectors and slices may contain arbitrary types, including non-moveable ones, so they just have to have methods which would allow their users to borrow their contents.