I am coding in Rust for a couple of months now, but there are still things I don't understand:
// copy values from src to dest (&mut) and return immutable ref
fn copy_into<'a, T>(src: &Vec<T>, dest: &'a mut Vec<T>) -> &'a Vec<T>
where T: Clone {
dest.clear();
dest.extend(src.into_iter().cloned());
dest
}
fn main() {
let mut v1 = vec!['a', 'b', 'c'];
let v2 = vec!['1', '2', '3'];
let v3 = copy_into(&v2, &mut v1);
// let v3 = &v1; // But this line would work
println!("{:?}", v1);
// ^^ immutable borrow occurs here
println!("{:?}", v2);
println!("{:?}", v3);
// --- mutable borrow later used here
}
Since copy_into returns an immutable reference, my understanding was that rust would know that we only have an immutable reference, but it seems that rust does remember the parameter dest and that it was a mutable reference.
In comparison, if I try to only take a reference &v1 instead of using the function, this works.
I also tried to return dest.as_ref(), but I got the following error: "cannot return reference to function parameter dest returns a reference to data owned by the current function". This also conforts me in thinking that rust refers to dest and not to v1 to know how to borrow.
This is just for learning purpose. My questions are:
Is it correct that rust remembers the dest parameter and, if yes, why?
If I did understand wrong, what is it?
In any case, I don't get why, after the println! of v1, I cannot do the same for v3, v3 is a reference to v1, not its content ?
Is it possible, and how, to pass a mutable reference to a function (because I need it) and return it as an immutable one so that it behaves as if I only did let v3 = &v1; ?
Thank you all for your help.
Since copy_into returns an immutable reference
I'm not sure where you get this from, but copy_into returns &'a mut Vec<T>, which is not an immutable reference, it's a mutable one.
Either way, it doesn't really make a difference. The important part is that the return value carries the lifetime 'a. The easiest way to think about lifetimes is to regard them as dependency indicators. Returning an 'a return value means that it depends on all input values with the 'a lifetime. That means to the compiler that as long as the return value exists, the dest input value is potentially still borrowed in it, and it can't be used until the return value is dropped.
In any case, I don't get why, after the println! of v1, I cannot do the same for v3, v3 is a reference to v1, not its content?
That's not how references work. Rust doesn't care about what exactly is happening inside of the copy_into function, you annotated with 'a that the return value depends on the dest input value, and that's all it knows. It doesn't understand what exactly v3 references, all it knows that it has to lock v1 as long as v3 exists, because that's what the lifetime annotations say.
Is it possible, and how, to pass a mutable reference to a function (because I need it) and return it as an immutable one?
Not that I know of.
Related
fn main() {
let mut name = String::from("Charlie");
let x = &mut name;
let y = x; // x has been moved
say_hello(y);
say_hello(y); // but y has not been moved, it is still usable
change_string(y);
change_string(y);
}
fn say_hello(s: &str) {
println!("Hello {}", s);
}
fn change_string(s: &mut String) {
s.push_str(" Brown");
}
When I assign x to y x has been moved. However, I would expect something with move semantics to be moved when I use it in a function. However, I can still use the reference after subsequent calls. Maybe this has to do with say_hello() taking a immutable reference but change_string() takes a mutable reference but the reference is still not moved.
You are completely right with both your reasoning and your observations. It definitely looks like things should be happening the way you describe it. However, the compiler applies some convenience magic here.
Move semantics generally apply in Rust for all types that do not implement the Copy trait. Shared references are Copy, so they are simply copied when assigned or passed to a function. Mutable references are not Copy, so they should be moved.
That's where the magic starts. Whenever a mutable reference is assigned to a name with a type already known to be a mutable reference by the compiler, the original reference is implicitly reborrowed instead of being moved. So the function called
change_string(y);
is transformed by the compiler to mean
change_string(&mut *y);
The original reference is derefenced, and a new mutable borrow is created. This new borrow is moved into the function, and the original borrow gets released once the function returns.
Note that this isn't a difference between function calls and assignments. Implicit reborrows happen whenever the target type is already known to be a mutable reference by the compiler, e.g. because the pattern has an explicit type annotation. So this line also creates an implicit reborrow, since we explicitly annotated it as a mutable reference type:
let y: &mut _ = x;
This function call on the other hand moves (and thus consumes) the mutable reference y:
fn foo<T>(_: T) {}
[...]
foo(y);
The generic type T here isn't explicitly a mutable reference type, so no implicit reborrow occurs, even though the compiler infers that the type is a mutable reference – just as in the case of your assignment let y = x;.
In some cases, the compiler can infer a generic type is a mutable reference even in the absence of an explicit type annotation:
fn bar<T>(_a: T, _b: T) {}
fn main() {
let mut i = 42;
let mut j = 43;
let x = &mut i;
let y = &mut j;
bar(x, y); // Moves x, but reborrows y.
let _z = x; // error[E0382]: use of moved value: `x`
let _t = y; // Works fine.
}
When inferring the type of the first parameter, the compiler doesn't know yet it's a mutable reference, so no implicit reborrow occurs and x is moved into the function. However, when reaching the second parameter, the compiler has already inferred that T is a mutable reference, so y is implicitly reborrowed. (This example is a good illustration why adding compiler magic to make things "just work" generally is a bad idea. Explicit is better than implicit.)
Unfortunately, this behaviour currently isn't documented in the Rust reference.
See also:
Stuff the Identity Function Does (in Rust)
Discussion of the topic on the Rust users forum
Why is the mutable reference not moved here?
The following minimal example does not compile:
fn main() {
let mut v = Vec::new();
foo(&mut v);
}
fn foo(v_ref: &mut Vec<u8>) {
for v in v_ref{
println!("{}", v);
}
for v in v_ref{
println!("{}", v);
}
}
The compiler suggests to modify the first for loop to this, which does compile.
for v in &mut *v_ref{
The reasoning given by the compiler is:
move occurs because v_ref has type &mut Vec<u8>, which does not implement the Copy trait
v_ref moved due to this implicit call to .into_iter()
Question 1: Why is this necessary? It's already a reference, shouldn't the for loop give the reference back after we're done with it?
Question 2: Why does the fix work? As I understand it, the new mutable reference should invalidate the old one.
Why is this necessary? It's already a reference, shouldn't the for loop give the reference back after we're done with it?
for loops desugar into (roughly):
{
let iterator = IntoIterator::into_iter(iter);
while let Some(v) = Iterator::next(&mut iterator) { ... }
}
Notice the IntoIterator::into_iter(). As explained in Do mutable references have move semantics?, mutable references, as opposed to shared references, are not Copy and thus once you move them you cannot use them anymore. Usually, the compiler inserts a reborrow, &mut *reference that creates a new reference from the existing one and allows the existing one to still be used (after the new is no longer used), but this only happens when it is known without inference that the type is a mutable reference, and here we need inference to determine that.
Why does the fix work? As I understand it, the new mutable reference should invalidate the old one.
You can think of references as a stack; each reference created from existing one pushes an item on the stack, and when we finished using it we pop it. &mut *reference creates a new reference, whose lifetime can be shorter than the original reference and allows the original reference to be used again after the end of the lifetime of the new reference.
fn main() {
let mut name = String::from("Charlie");
let x = &mut name;
let y = x; // x has been moved
say_hello(y);
say_hello(y); // but y has not been moved, it is still usable
change_string(y);
change_string(y);
}
fn say_hello(s: &str) {
println!("Hello {}", s);
}
fn change_string(s: &mut String) {
s.push_str(" Brown");
}
When I assign x to y x has been moved. However, I would expect something with move semantics to be moved when I use it in a function. However, I can still use the reference after subsequent calls. Maybe this has to do with say_hello() taking a immutable reference but change_string() takes a mutable reference but the reference is still not moved.
You are completely right with both your reasoning and your observations. It definitely looks like things should be happening the way you describe it. However, the compiler applies some convenience magic here.
Move semantics generally apply in Rust for all types that do not implement the Copy trait. Shared references are Copy, so they are simply copied when assigned or passed to a function. Mutable references are not Copy, so they should be moved.
That's where the magic starts. Whenever a mutable reference is assigned to a name with a type already known to be a mutable reference by the compiler, the original reference is implicitly reborrowed instead of being moved. So the function called
change_string(y);
is transformed by the compiler to mean
change_string(&mut *y);
The original reference is derefenced, and a new mutable borrow is created. This new borrow is moved into the function, and the original borrow gets released once the function returns.
Note that this isn't a difference between function calls and assignments. Implicit reborrows happen whenever the target type is already known to be a mutable reference by the compiler, e.g. because the pattern has an explicit type annotation. So this line also creates an implicit reborrow, since we explicitly annotated it as a mutable reference type:
let y: &mut _ = x;
This function call on the other hand moves (and thus consumes) the mutable reference y:
fn foo<T>(_: T) {}
[...]
foo(y);
The generic type T here isn't explicitly a mutable reference type, so no implicit reborrow occurs, even though the compiler infers that the type is a mutable reference – just as in the case of your assignment let y = x;.
In some cases, the compiler can infer a generic type is a mutable reference even in the absence of an explicit type annotation:
fn bar<T>(_a: T, _b: T) {}
fn main() {
let mut i = 42;
let mut j = 43;
let x = &mut i;
let y = &mut j;
bar(x, y); // Moves x, but reborrows y.
let _z = x; // error[E0382]: use of moved value: `x`
let _t = y; // Works fine.
}
When inferring the type of the first parameter, the compiler doesn't know yet it's a mutable reference, so no implicit reborrow occurs and x is moved into the function. However, when reaching the second parameter, the compiler has already inferred that T is a mutable reference, so y is implicitly reborrowed. (This example is a good illustration why adding compiler magic to make things "just work" generally is a bad idea. Explicit is better than implicit.)
Unfortunately, this behaviour currently isn't documented in the Rust reference.
See also:
Stuff the Identity Function Does (in Rust)
Discussion of the topic on the Rust users forum
Why is the mutable reference not moved here?
It seems that I cannot mutate anything if there is any immutable reference in my chain of dereferencing. A sample:
fn main() {
let mut x = 42;
let y: &mut i32 = &mut x; // first layer
let z: &&mut i32 = &y; // second layer
**z = 100; // Attempt to change `x`, gives compiler error.
println!("Value is: {}", z);
}
I'm getting the compiler error:
error[E0594]: cannot assign to `**z` which is behind a `&` reference
--> src/main.rs:5:5
|
4 | let z: &&mut i32 = &y; // second layer
| -- help: consider changing this to be a mutable reference: `&mut y`
5 | **z = 100; // Attempt to change `x`, gives compiler error.
| ^^^^^^^^^ `z` is a `&` reference, so the data it refers to cannot be written
In some way, this makes sense, as otherwise the compiler would not be able to prevent having multiple mutable access paths to the same variable.
However, when looking at the types, the semantics seem to be counter-intuitive:
Variable y has type &mut i32, or in plain English "A mutable reference to an integer".
Variable z has type &&mut i32, or in plain English "An immutable reference to a mutable reference to an integer".
By dereferencing z once (i.e. *z) I will get something of type &mut i32, i.e. something of the same type as y. However, dereferencing this again (i.e. **z) gets me something of type i32, but I am not allowed to mutate that integer.
In essence, the types of references in some sense lie to me, as they don't actually do what they claim they do. How should I read types of references properly in this case, or how else can I restore faith in that concept?
Testing with this sample:
fn main() {
let mut x = 42;
let y: &mut i32 = &mut x; // first layer
let m: &&mut i32 = &y; // second layer
let z: &&&mut i32 = &m; // third layer
compiler_builtin_deref_first_layer(*z);
}
fn compiler_builtin_deref_first_layer(v: &&mut i32) {
compiler_builtin_deref_second_layer(*v);
}
fn compiler_builtin_deref_second_layer(w: &mut i32) {
println!("Value is: {}", w);
}
The parameter types of those last two functions are correct. If I change any of those, the compiler will complain about mismatched types. However, if I compile the example as-is, I get this error:
error[E0596]: cannot borrow `**v` as mutable, as it is behind a `&` reference
Somehow, the call to compiler_builtin_deref_first_layer seems to be okay, but the call to compiler_builtin_deref_second_layer isn't. The compiler error talks about **v, but I only see a *v.
In essence, the types of references in some sense lie to me, as they don't actually do what they claim they do. How should I read types of references properly in this case, or how else can I restore faith in that concept?
The right way to read references in Rust is as permissions.
Ownership of an object, when it's not borrowed, gives you permission to do whatever you want to the object; create it, destroy it, move it from one place to another. You are the owner, you can do what you want, you control the life of that object.
A mutable reference borrows the object from the owner. While the mutable reference is alive, it grants exclusive access to the object. No one else can read, write, or do anything else to the object. A mutable reference could also be called an exclusive reference, or exclusive borrow. You have to return control of the object back to the original owner, but in the meantime, you get to do whatever you want with it.
An immutable reference, or shared borrow, means you get to access it at the same time as others. Because of that, you can only read it, and no one can modify it, or there would be undefined results based on the exact order that the actions happened in.
Both mutable (or exclusive) references and immutable (or shared) references can be made to owned objects, but that doesn't mean that you own the object when you're referring to it through the reference. What you can do with an object is constrained by what kind of reference you're reaching it through.
So don't think of an &&mut T reference as "an immutable reference to a mutable reference to T", and then think "well, I can't mutate the outer reference, but I should be able to mutate the inner reference."
Instead, think of it as "Someone owns a T. They've given out exclusive access, so right now there's someone who has the right to modify the T. But in the meantime, that person has given out shared access to the &mut T, which means they've promised to not mutate it for a period of time, and all of the users can use the shared reference to &mut T, including dereferencing to the underlying T but only for things which you can normally do with a shared reference, which means reading but not writing."
The final thing to keep in mind is that the mutable or immutable part aren't actually the fundamental difference between the references. It's really the exclusive vs. shared part that are. In Rust, you can modify something through a shared reference, as long as there is some kind of inner protection mechanism that ensures that only one person does so at a time. There are multiple ways of doing that, such as Cell, RefCell, or Mutex.
So what &T and &mut T provide isn't really immutable or mutable access, though they are named as such because that's the default level of access they provide at the language level in the absence of any library features. But what they really provide is shared or exclusive access, and then methods on data types can provide different functionality to callers depending on whether they take an owned value, an exclusive reference, or a shared reference.
So think of references as permissions; and it's the reference that you reach something through that determines what you are allowed to do with it. And when you have ownership or an exclusive reference, giving out an exclusive or shared reference temporarily prevents you from mutably accessing the object while those borrowed references are still alive.
I suppose this question is about lifetimes in general, but I'm having difficulty with closures specifically because you can't write out their type.
This example is a bit contrived - I'm just starting to learn Rust, and this is something I've been hung up on.
This program won't compile:
fn main () {
let mut list: Vec<&Fn() -> i32> = Vec::new();
{
list.push(&|| 1);
}
}
Because:
src/main.rs:5:25: 5:24 error: borrowed value does not live long enough
src/main.rs:5 list.push(&|| 1);
^~~~
src/main.rs:2:50: 7:2 note: reference must be valid for the block suffix following statement 0 at 2:49...
src/main.rs:2 let mut list: Vec<&Fn() -> i32> = Vec::new();
src/main.rs:3
src/main.rs:4 {
src/main.rs:5 list.push(&move || 1);
src/main.rs:6 }
src/main.rs:7 }
src/main.rs:5:9: 5:26 note: ...but borrowed value is only valid for the statement at 5:8
src/main.rs:5 list.push(&|| 1);
^~~~~~~~~~~~~~~~~
src/main.rs:5:9: 5:26 help: consider using a `let` binding to increase its lifetime
src/main.rs:5 list.push(&|| 1);
^~~~~~~~~~~~~~~~~
What I gather from this error is that the closure's lifetime is limited to the
statement inside the block, but it needs to live for the entire body of main.
I know (or, I think) that passing the closure to push as a reference means that push is only borrowing the closure, and that ownership will be returned to the block. This code would work if I could just give the closure to push (i.e. if push took ownership of the closure), but since a closure isn't sized, I must pass it as a reference.
Is that right? How can I make this code work?
There are two things you are asking about:
specifying a typename for something that has no specifyable typename
letting a closure live longer than the block where it's defined.
The first issue is fixed by NOT specifying the typename, and letting rust's type inference do the work.
let mut list: Vec<_> = Vec::new();
The second issue is fixed by not trying to make the closure live longer, but by making it "by value" so you can move it. This enforces that your closure does not reference anything, but owns all the captured values.
for i in 0..10 {
list.push(move || i);
}
Now this gives us a new problem. If we add a different closure to the Vec, the types won't match. Therefore to achieve that, we need to box the closures.
fn main () {
let mut list: Vec<Box<Fn() -> i32>> = Vec::new();
for i in 0..10 {
list.push(Box::new(move|| i));
}
{
list.push(Box::new(move|| 42));
}
}
Borrows do not own the thing they point to. Your problem is that you're borrowing a temporary which is going to cease to exist right after it's borrowed because you haven't stored it anywhere. If it helps, consider that borrows don't borrow values, they borrow storage, and a temporary has only transient storage.
If you want a borrow to something to last for any given period, you must borrow from storage that will last at least that long. In this case, because you want to store the borrow in a Vec, this means that whatever storage you borrow from must outlive the Vec as well. Thus:
fn main () {
let closure;
let mut list: Vec<&Fn() -> i32> = Vec::new();
{
closure = || 1;
list.push(&closure);
}
}
Note that closure is defined before list is. In Rust, values are dropped in reverse lexical order at the end of their scope, so any variable defined after list will necessarily be dropped before it, thus leading to list containing invalid pointers.
If you want to push multiple closures, you will need a separate variable for each one.
To forestall a possible "my actual problem isn't this simple" addendum (:P): f you need to return list or in some way persist it beyond a single function call, note that there is no way to extend a borrow. In that case, what you need to do is change list to a vector of owned, boxed closures (i.e. Vec<Box<Fn() -> i32>>).