Troubles understanding rust borrowing system - rust

I'm going thru the rustlings exercises but I don't seem to understand how borrow semantics works.
I created this simple example (check the playground):
fn main() {
let mut vec0: Vec<i64> = Vec::new();
let vec1 = &mut vec0;
println!("{} has content `{:?}`", "vec0", vec0);
println!("{} has content `{:?}`", "vec1", vec1);
}
which gives the following error:
Compiling playground v0.0.1 (/playground)
error[E0502]: cannot borrow `vec0` as immutable because it is also borrowed as mutable
--> src/main.rs:9:47
|
7 | let vec1 = &mut vec0;
| --------- mutable borrow occurs here
8 |
9 | println!("{} has content `{:?}`", "vec0", vec0);
| ^^^^ immutable borrow occurs here
10 |
11 | println!("{} has content `{:?}`", "vec1", vec1);
| ---- mutable borrow later used here
For more information about this error, try `rustc --explain E0502`.
error: could not compile `playground` due to previous error
I though that when borrowing a value the original binding kept the ownership, that is, after the let vec1 = &mut vec0; vec0 should still own the vec.
Moreover, I don't understand why on line 9 on println!("{} has content `{:?}`", "vec0", vec0) an immutable borrow occurs. Isn't the vec still owned by vec0?

You're correct that vec0 is still the owner of the the underlying memory.
The reason println! needs to borrow vec0 is that the println! macro needs to be able to read from the vector in order to print it, therefore it needs a read-only reference. A read-only reference is automatically created and goes out of scope inside this macro.
The reason that you are getting a compiler error here is that you are violating the borrow checker rules:
At any given time, you can have either one mutable reference or any number of immutable references.

The best way to understand how println references it's arguments is by using cargo-expand which will show the result of macro expansion. For your code it expands to
#![feature(prelude_import)]
#[prelude_import]
use std::prelude::rust_2018::*;
#[macro_use]
extern crate std;
fn main() {
let mut vec0: Vec<i64> = Vec::new();
let vec1 = &mut vec0;
{
::std::io::_print(::core::fmt::Arguments::new_v1(
&["", " has content `", "`\n"],
&match (&"vec0", &vec0) { // Immutable borrow
_args => [
::core::fmt::ArgumentV1::new(_args.0, ::core::fmt::Display::fmt),
::core::fmt::ArgumentV1::new(_args.1, ::core::fmt::Debug::fmt),
],
},
));
};
{
::std::io::_print(::core::fmt::Arguments::new_v1(
&["", " has content `", "`\n"],
&match (&"vec1", &vec1) { // Immutable borrow
_args => [
::core::fmt::ArgumentV1::new(_args.0, ::core::fmt::Display::fmt),
::core::fmt::ArgumentV1::new(_args.1, ::core::fmt::Debug::fmt),
],
},
));
};
}
As you can see println takes immutable reference to its arguments.(Which means vec0 stills owns the vector). But the problem here is that Rust enforces "multiple readers or single writer" rule at compile time. As long as there is mutable reference to a value you cannot use the owner until the mutable reference goes away. Similarly as long as there is multiple shared references to value not even it's owner can modify it.
For example, this will compile,
fn main() {
let mut vec0: Vec<i64> = Vec::new();
{
let vec1 = &mut vec0;
println!("{} has content `{:?}`", "vec1", vec1);
} // The mutable reference goes out of scope here.
println!("{} has content `{:?}`", "vec0", vec0);
}

You can't have two mutable references to the same value at the same time. If you reorder the lines in your function like this:
fn main() {
let mut vec0: Vec<i64> = Vec::new();
println!("{} has content `{:?}`", "vec0", vec0);
let vec1 = &mut vec0;
println!("{} has content `{:?}`", "vec1", vec1);
}
, there will be no problem, because in that variant vec0 is not accessed after the moment a mutable reference to it was created.
If vec1 was a normal (immutable) reference, there would be no problem even with your original line order, because having multiple immutable (read-only) references is not a problem:
fn main() {
let mut vec0: Vec<i64> = Vec::new();
let vec1 = &vec0;
println!("{} has content `{:?}`", "vec0", vec0);
println!("{} has content `{:?}`", "vec1", vec1);
}
References are most often used for function parameters, when you don't want to move a value, but only pass a reference to it. Only when you need to change the underlying value inside the function, you need to make the reference mutable.

Related

Cannot use iterator from one method in another because the "borrow might be used when that temporary is dropped"

I have a struct with two methods. The first method returns an iterator over some field. The second method calls the first one and subsequently tries to modify that field (Playground):
struct Container {
vec: Vec<f64>,
}
impl Container {
fn large_positions<'a>(&'a self) -> impl Iterator<Item = usize> + 'a {
self.vec
.iter()
.enumerate()
.filter_map(|(i, &f)| if f > 3.14 { Some(i) } else { None })
}
fn negate_first_large_entry(&mut self) {
if let Some(i) = self.large_positions().next() {
self.vec[i] *= -1.0;
}
}
}
The borrow checker complains:
error[E0502]: cannot borrow `self.vec` as mutable because it is also borrowed as immutable
--> src/lib.rs:15:13
|
14 | if let Some(i) = self.large_positions().next() {
| ----------------------
| |
| immutable borrow occurs here
| a temporary with access to the immutable borrow is created here ...
15 | self.vec[i] *= -1.0;
| ^^^^^^^^ mutable borrow occurs here
16 | }
17 | }
| - ... and the immutable borrow might be used here, when that temporary is dropped and runs the destructor for type `impl Iterator`
If I manually inline large_positions, the code compiles so the destructor thing is not a real issue.
Why does the borrow checker know that there is no conflict in the inlined version?
And how can I explain this to the borrow checker in the non-inlined version?
Update:
The following slightly different version of the second function exhibits the same behavior: (Playground)
fn negate_first_large_entry_below_100(&mut self) {
for i in self.large_positions() {
if self.vec[i] < 100.0 {
self.vec[i] *= -1.0;
return;
}
}
}
Here the return statement causes the borrow checker to understand that the borrow of self.vec can be released as soon as the if branch is entered. But again this only works if large_positions is inlined.
Ideally I would like to modify the signature of large_positions in such a way that the borrow checker is still able to perform this clever "early releasing".
You can extrat the iterator into a variable that you can later drop in order to get a &mut afterwards:
fn negate_first_large_entry(&mut self) {
let mut iter = self.large_positions();
if let Some(i) = iter.next() {
drop(iter);
self.vec[i] *= -1.0;
}
}
Playground
For the second method you just need to use a while loop instead of a for loop to perform the same trick:
fn negate_first_large_entry_below_100(&mut self) {
let mut iter = self.large_positions();
while let Some(i) = iter.next() {
if self.vec[i] < 100.0 {
drop(iter);
self.vec[i] *= -1.0;
return;
}
}
}
Playground
(I believe to have now gained a superficial understanding of the underlying problem, so I will formulate this as an answer. Comments are welcome.)
As the error message hints and the answer by Netwave demonstrates, the issue is related to dropping iter. Upon dropping iter, the borrow of self.vec is freed. If this happens before self.vec is modified, then the code borrow-checks successfully.
In the non-inlined version, the drop of iter happens when iter goes out of scope. The compiler is not able to move this forward, because it is not aware whether iter has any user-defined Drop implementation.
In contrast, in the manually inlined version of the code, the compile is able to perform this drop immediately after the last time iter is used. I presume this optimization is possible because the compiler knows here that no custom code is run upon dropping iter (not sure how I could verify this).
A proper solution to the problem would thus be to extend the return type of large_positions, indicating that the returned value may be dropped before it leaves scope. If I understand this proposal correctly, this is currently impossible because such a "TrivialDrop" trait does not exist in Rust.

Append value to Array inside of Rc<RefCell<Option<Vec>>>>

What's the proper way to append a value to an Option<Vec> wrapped in an Rc<RefCell<>>?
use std::{cell::RefCell, rc::Rc};
fn main(){
let my_vec = Some(vec![String::from("Foo"), String::from("Bar")]);
let mut rc_vec = Rc::from(RefCell::from(my_vec));
rc_vec.borrow_mut().as_ref().unwrap().push(String::from("Baz"));}
gives error:
error[E0596]: cannot borrow data in a `&` reference as mutable
rc_vec.borrow_mut().as_ref().unwrap().push(String::from("Baz"));
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
Just use as_mut instead of as_ref. Also you can skip the unwrap, and just map the push operation:
rc_vec
.borrow_mut()
.as_mut()
.map(|v| v.push(String::from("Baz")));
Playground

Directory traversal in vanilla Rust

I'm new to Rust and trying to understand basic directory traversal. Nearly all the examples I have found utilize the walkdir or glob library, which I've had good success with. However, I'm trying to do this now with just the std lib.
There is a primitive example in the standard lib docs listing the following function:
fn visit(path: &Path, cb: &dyn Fn(&PathBuf)) -> io::Result<()> {
for e in read_dir(path)? {
let e = e?;
let path = e.path();
if path.is_dir() {
visit(&path, cb)?;
} else if path.is_file() {
cb(&path);
}
}
Ok(())
}
The part I'm confused about is how to access the cb function in the context of a closure. I'm having a hard time finding an example.
For instance, I want to do something basic like collect the resulting paths into a Vec. Obviously, this does not work:
fn main() {
// create a new path
let path = Path::new(PATH);
let mut files = Vec::new();
visit(path, |e| {
files.push(e);
});
}
The error I'm receiving is:
expected reference `&dyn for<'r> std::ops::Fn(&'r std::path::PathBuf)`
found closure `[closure#src/main.rs:24:17: 26:6 files:_]
So my question is, how can I return a Fn and process the result in a closure context?
There are multiple issues with your code, but the first one that you are getting the error message for is because &dyn Fn(&PathBuf) expects a reference to a function. You can resolve that error by following the suggestion from the error message: help: consider borrowing here: '&|e| files.push(e)'
That turns your call into:
visit(path, &|e| files.push(e));
However, this code is still incorrect and results in yet another error:
error[E0596]: cannot borrow `files` as mutable, as it is a captured variable in a `Fn` closure
--> playground\src\main.rs:48:22
|
48 | visit(path, &|e| files.push(e));
| ^^^^^ cannot borrow as mutable
This time, it's because you're mutating files inside a Fn (immutable closure). To fix that, you need to change your function type to FnMut (see Closures As Input Parameters for more information):
fn visit(path: &Path, cb: &dyn FnMut(&PathBuf))
But you're still not done. There is now another error, but like the first, it comes with a suggestion for what needs to be changed:
error[E0596]: cannot borrow `*cb` as mutable, as it is behind a `&` reference
--> playground\src\main.rs:39:13
|
32 | fn visit(path: &Path, cb: &dyn FnMut(&PathBuf)) -> io::Result<()> {
| -------------------- help: consider changing this to be a mutable reference: `&mut dyn for<'r> std::ops::FnMut(&'r std::path::PathBuf)`
...
39 | cb(&path);
| ^^ `cb` is a `&` reference, so the data it refers to cannot be borrowed as mutable
In order for your closure to mutably borrow the data it uses, you also have to take a mutable reference to the closure itself, and you'll need to update your visit() call to match:
fn visit(path: &Path, cb: &mut dyn FnMut(&PathBuf))
...
visit(path, &mut |e| files.push(e));
Almost there, but there is one final error to resolve:
error[E0521]: borrowed data escapes outside of closure
--> playground\src\main.rs:48:26
|
47 | let mut files = Vec::new();
| --------- `files` declared here, outside of the closure body
48 | visit(path, &mut |e| files.push(e));
| - ^^^^^^^^^^^^^ `e` escapes the closure body here
| |
| `e` is a reference that is only valid in the closure body
You've defined your closure to take a reference to a PathBuf (&PathBuf), but you're trying to push those references into a Vec that is outside of the closure, which won't work because those references will be invalid once the closure goes out of scope. Instead, you should use an owned value -- simply PathBuf. You'll also need to update your usage of the closure to pass the PathBuf instead of a reference:
fn visit(path: &Path, cb: &mut dyn FnMut(PathBuf))
...
cb(path);
It finally works! Here is what the full program looks like now. Note that you should also unwrap() your call to visit() since it returns a Result. I've also added a simple for loop to print out the file names.
use std::path::{Path, PathBuf};
use std::fs::*;
use std::io;
fn visit(path: &Path, cb: &mut dyn FnMut(PathBuf)) -> io::Result<()> {
for e in read_dir(path)? {
let e = e?;
let path = e.path();
if path.is_dir() {
visit(&path, cb)?;
} else if path.is_file() {
cb(path);
}
}
Ok(())
}
fn main() {
let path = Path::new("./your/path/here");
let mut files = Vec::new();
visit(path, &mut |e| files.push(e)).unwrap();
for file in files {
println!("{:?}", file);
}
}

Getting first member of a BTreeSet

In Rust, I have a BTreeSet that I'm using to keep my values in order. I have a loop that should retrieve and remove the first (lowest) member of the set. I'm using a cloned iterator to retrieve the first member. Here's the code:
use std::collections::BTreeSet;
fn main() {
let mut start_nodes = BTreeSet::new();
// add items to the set
while !start_nodes.is_empty() {
let mut start_iter = start_nodes.iter();
let mut start_iter_cloned = start_iter.cloned();
let n = start_iter_cloned.next().unwrap();
start_nodes.remove(&n);
}
}
This, however, gives me the following compile error:
error[E0502]: cannot borrow `start_nodes` as mutable because it is also borrowed as immutable
--> prog.rs:60:6
|
56 | let mut start_iter = start_nodes.iter();
| ----------- immutable borrow occurs here
...
60 | start_nodes.remove(&n);
| ^^^^^^^^^^^ mutable borrow occurs here
...
77 | }
| - immutable borrow ends here
Why is start_nodes.iter() considered an immutable borrow? What approach should I take instead to get the first member?
I'm using version 1.14.0 (not by choice).
Why is start_nodes.iter() considered an immutable borrow?
Whenever you ask a question like this one, you need to look at the prototype of the function, in this case the prototype of BTreeSet::iter():
fn iter(&self) -> Iter<T>
If we look up the Iter type that is returned, we find that it's defined as
pub struct Iter<'a, T> where T: 'a { /* fields omitted */ }
The lifetime 'a is not explicitly mentioned in the definition of iter(); however, the lifetime elision rules make the function definition equivalent to
fn iter<'a>(&'a self) -> Iter<'a, T>
From this expanded version, you can see that the return value has a lifetime that is bound to the lifetime of the reference to self that you pass in, which is just another way of stating that the function call creates a shared borrow that lives as long as the return value. If you store the return value in a variable, the borrow lives at least as long as the variable.
What approach should I take instead to get the first member?
As noted in the comments, your code works on recent versions of Rust due to non-lexical lifetimes – the compiler figures out by itself that start_iter and start_iter_cloned don't need to live longer than the call to next(). In older versions of Rust, you can artificially limit the lifetime by introducing a new scope:
while !start_nodes.is_empty() {
let n = {
let mut start_iter = start_nodes.iter();
let mut start_iter_cloned = start_iter.cloned();
start_iter_cloned.next().unwrap()
};
start_nodes.remove(&n);
}
However, note that this code is needlessly long-winded. The new iterator you create and its cloning version only live inside the new scope, and they aren't really used for any other purpose, so you could just as well write
while !start_nodes.is_empty() {
let n = start_nodes.iter().next().unwrap().clone();
start_nodes.remove(&n);
}
which does exactly the same, and avoids the issues with long-living borrows by avoiding to store the intermediate values in variables, to ensure their lifetime ends immediately after the expression.
Finally, while you don't give full details of your use case, I strongly suspect that you would be better off with a BinaryHeap instead of a BTreeSet:
use std::collections::BinaryHeap;
fn main() {
let mut start_nodes = BinaryHeap::new();
start_nodes.push(42);
while let Some(n) = start_nodes.pop() {
// Do something with `n`
}
}
This code is shorter, simpler, completely sidesteps the issue with the borrow checker, and will also be more efficient.
Not sure this is the best approach, but I fixed it by introducing a new scope to ensure that the immutable borrow ends before the mutable borrow occurs:
use std::collections::BTreeSet;
fn main() {
let mut start_nodes = BTreeSet::new();
// add items to the set
while !start_nodes.is_empty() {
let mut n = 0;
{
let mut start_iter = start_nodes.iter();
let mut start_iter_cloned = start_iter.cloned();
let x = &mut n;
*x = start_iter_cloned.next().unwrap();
}
start_nodes.remove(&n);
}
}

Why can't I reuse a &mut reference after passing it to a function that accepts a generic type?

Why doesn't this code compile:
fn use_cursor(cursor: &mut io::Cursor<&mut Vec<u8>>) {
// do some work
}
fn take_reference(data: &mut Vec<u8>) {
{
let mut buf = io::Cursor::new(data);
use_cursor(&mut buf);
}
data.len();
}
fn produce_data() {
let mut data = Vec::new();
take_reference(&mut data);
data.len();
}
The error in this case is:
error[E0382]: use of moved value: `*data`
--> src/main.rs:14:5
|
9 | let mut buf = io::Cursor::new(data);
| ---- value moved here
...
14 | data.len();
| ^^^^ value used here after move
|
= note: move occurs because `data` has type `&mut std::vec::Vec<u8>`, which does not implement the `Copy` trait
The signature of io::Cursor::new is such that it takes ownership of its argument. In this case, the argument is a mutable reference to a Vec.
pub fn new(inner: T) -> Cursor<T>
It sort of makes sense to me; because Cursor::new takes ownership of its argument (and not a reference) we can't use that value later on. At the same time it doesn't make sense: we essentially only pass a mutable reference and the cursor goes out of scope afterwards anyway.
In the produce_data function we also pass a mutable reference to take_reference, and it doesn't produce a error when trying to use data again, unlike inside take_reference.
I found it possible to 'reclaim' the reference by using Cursor.into_inner(), but it feels a bit weird to do it manually, since in normal use-cases the borrow-checker is perfectly capable of doing it itself.
Is there a nicer solution to this problem than using .into_inner()? Maybe there's something else I don't understand about the borrow-checker?
Normally, when you pass a mutable reference to a function, the compiler implicitly performs a reborrow. This produces a new borrow with a shorter lifetime.
When the parameter is generic (and is not of the form &mut T), the compiler doesn't do this reborrowing automatically1. However, you can do it manually by dereferencing your existing mutable reference and then referencing it again:
fn take_reference(data: &mut Vec<u8>) {
{
let mut buf = io::Cursor::new(&mut *data);
use_cursor(&mut buf);
}
data.len();
}
1 — This is because the current compiler architecture only allows a chance to do a coercion if both the source and target types are known at the coercion site.

Resources