I'm writing a linked list to wrap my head around Rust lifetimes, ownership and references. I have the following code:
pub struct LinkedList {
head: Option<Box<LinkedListNode>>,
}
pub struct LinkedListNode {
next: Option<Box<LinkedListNode>>,
}
impl LinkedList {
pub fn new() -> LinkedList {
LinkedList { head: None }
}
pub fn prepend_value(&mut self) {
let mut new_node = LinkedListNode { next: None };
match self.head {
Some(ref head) => new_node.next = Some(*head),
None => new_node.next = None,
};
self.head = Some(Box::new(new_node));
}
}
fn main() {}
But I am getting the following compilation error:
error[E0507]: cannot move out of borrowed content
--> src/main.rs:18:52
|
18 | Some(ref head) => new_node.next = Some(*head),
| ^^^^^ cannot move out of borrowed content
Newer versions of Rust have a slightly different error:
error[E0507]: cannot move out of `*head` which is behind a shared reference
--> src/main.rs:18:52
|
18 | Some(ref head) => new_node.next = Some(*head),
| ^^^^^ move occurs because `*head` has type `std::boxed::Box<LinkedListNode>`, which does not implement the `Copy` trait
I'm thinking that the head node must currently be owned by self, which is the linked list. When I assign it to new_node.next, there is probably a change of ownership that will happen.
I would rather not clone the value if possible as that seems wasteful. I don't want to just "borrow" it for the duration of the function. I really want to transfer its ownership.
How do I do that?
I have already looked at cannot move out of borrowed content when unwrapping a member variable in a &mut self method and Cannot move out of borrowed content / cannot move out of behind a shared reference.
I tried removing the match arm as suggested in the accepted answer in one of those questions and defining next in the creation of the new LinkedListNode, but I get the same error message.
I have successfully added an append method which takes a LinkedListNode to add to the end of the list.
Cannot move out of borrowed content when trying to transfer ownership
At a high-level, this is against-the-grain for Rust. You cannot transfer ownership of something borrowed because you don't own it. You shouldn't borrow my car (&Car) and then give it to the first person you see on the street! This is still true even if I lend you my car and allow you to make changes to it (&mut Car).
You cannot move head out of a &self at all because you cannot mutate the value.
You cannot move head out of a &mut self because this would leave the LinkedList struct in an inconsistent state - one of the fields would have an undefined value. This is a core measure of Rust's safety guarantees.
In general, you will need to follow something from How can I swap in a new value for a field in a mutable reference to a structure? to replace the existing value.
In this case, you can use Option::take. This will leave the variable where it is, changing it in-place to a None and returning the previous value. You can then use that value to build the new head of the list:
pub fn prepend_value(&mut self) {
let head = self.head.take();
self.head = Some(Box::new(LinkedListNode { next: head }));
}
A more generic solution is to take ownership of the struct instead of borrowing it. This allows you to do whatever you want to it. Note that we take self by-value, not by-reference:
pub fn prepend_value(mut self) -> LinkedList {
self.head = Some(Box::new(LinkedListNode { next: self.head }));
self
}
Related
I'm trying to learn a bit of Rust through a toy application, which involves a tree data structure that is filled dynamically by querying an external source. In the beginning, only the root node is present. The tree structure provides a method get_children(id) that returns a [u32] of the IDs of all the node's children — either this data is already known, or the external source is queried and all the nodes are inserted into the tree.
I'm running into the following problem with the borrow checker that I can't seem to figure out:
struct Node {
id: u32,
value: u64, // in my use case, this type is much larger and should not be copied
children: Option<Vec<u32>>,
}
struct Tree {
nodes: std::collections::HashMap<u32, Node>,
}
impl Tree {
fn get_children(&mut self, id: u32) -> Option<&[u32]> {
// This will perform external queries and add new nodes to the tree
None
}
fn first_even_child(&mut self, id: u32) -> Option<u32> {
let children = self.get_children(id)?;
let result = children.iter().find(|&id| self.nodes.get(id).unwrap().value % 2 == 0)?;
Some(*result)
}
}
Which results in:
error[E0502]: cannot borrow `self.nodes` as immutable because it is also borrowed as mutable
--> src/lib.rs:19:43
|
18 | let children = self.get_children(id)?;
| ---- mutable borrow occurs here
19 | let result = children.iter().find(|&id| self.nodes.get(id).unwrap().value % 2 == 0)?;
| ---- ^^^^^ ---------- second borrow occurs due to use of `self.nodes` in closure
| | |
| | immutable borrow occurs here
| mutable borrow later used by call
Since get_children might insert nodes into the tree, we need a &mut self reference. However, the way I see it, after the value of children is known, self no longer needs to be borrowed mutably. Why does this not work, and how would I fix it?
EDIT -- my workaround
After Chayim Friedman's answer, I decided against returning Self. I mostly ran into the above problem when first calling get_children to get a list of IDs and then using nodes.get() to obtain the corresponding Node. Instead, I refactored to provide the following functions:
impl Tree {
fn load_children(&mut self, id: u32) {
// If not present yet, perform queries to add children to the tree
}
fn iter_children(&self, id: u32) -> Option<IterChildren> {
// Provides an iterator over the children of node `id`
}
}
Downgrading a mutable reference into a shared reference produces a reference that should be kept unique. This is necessary for e.g. Cell::from_mut(), which has the following signature:
pub fn from_mut(t: &mut T) -> &Cell<T>
This method relies on the uniqueness guarantee of &mut T to ensure no references to T are kept directly, only via Cell. If downgrading the reference would mean the unqiueness could have been violated, this method would be unsound, because the value inside the Cell could have been changed by another shared references (via interior mutability).
For more about this see Common Rust Lifetime Misconceptions: downgrading mut refs to shared refs is safe.
To solve this you need to get both shared references from the same shared reference that was created from the mutable reference. You can, for example, also return &Self from get_children():
fn get_children(&mut self, id: u32) -> Option<(&Self, &[u32])> {
// This will perform external queries and add new nodes to the tree
Some((self, &[]))
}
fn first_even_child(&mut self, id: u32) -> Option<u32> {
let (this, children) = self.get_children(id)?;
let result = children.iter().find(|&id| this.nodes.get(id).unwrap().value % 2 == 0)?;
Some(*result)
}
I have to iterate on keys, find the value in HashMap by key, possibly do some heavy computation in the found struct as a value (lazy => mutate the struct) and cached return it in Rust.
I'm getting the following error message:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:25:26
|
23 | fn it(&mut self) -> Option<&Box<Calculation>> {
| - let's call the lifetime of this reference `'1`
24 | for key in vec!["1","2","3"] {
25 | let result = self.find(&key.to_owned());
| ^^^^ `*self` was mutably borrowed here in the previous iteration of the loop
...
28 | return result
| ------ returning this value requires that `*self` is borrowed for `'1`
Here is the code in playground.
use std::collections::HashMap;
struct Calculation {
value: Option<i32>
}
struct Struct {
items: HashMap<String, Box<Calculation>> // cache
}
impl Struct {
fn find(&mut self, key: &String) -> Option<&Box<Calculation>> {
None // find, create, and/or calculate items
}
fn it(&mut self) -> Option<&Box<Calculation>> {
for key in vec!["1","2","3"] {
let result = self.find(&key.to_owned());
if result.is_some() {
return result
}
}
None
}
}
I can't avoid the loop as I have to check multiple keys
I have to make it mutable (self and the structure) as the possible calculation changes it
Any suggestion on how to change the design (as Rust forces to think in a bit different way that makes sense) or work around it?
PS. There are some other issues with the code, but let's split the problems and solve this one first.
You can't do caching with exclusive access. You can't treat Rust references like general-purpose pointers (BTW: &String and &Box<T> are double indirection, and very unidiomatic in Rust. Use &str or &T for temporary borrows).
&mut self means not just mutable, but exclusive and mutable, so your cache supports returning only one item, because the reference it returns has to keep self "locked" for as long as it exists.
You need to convince the borrow checker that the thing that find returns won't suddenly disappear next time you call it. Currently there's no such guarantee, because the interface doesn't stop you from calling e.g. items.clear() (borrow checker checks what function's interface allows, not what function actually does).
You can do that either by using Rc, or using a crate that implements a memory pool/arena.
struct Struct {
items: HashMap<String, Rc<Calculation>>,
}
fn find(&mut self, key: &str) -> Rc<Calculation>
This way if you clone the Rc, it will live for as long as it needs, independently of the cache.
You can also make it nicer with interior mutability.
struct Struct {
items: RefCell<HashMap<…
}
This will allow your memoizing find method to use a shared borrow instead of an exclusive one:
fn find(&self, key: &str) -> …
which is much easier to work with for the callers of the method.
Probably not the cleanest way to do that, but it compiles. The idea is not to store the value found in a temporary result, to avoid aliasing: if you store the result, self is kept borrowed.
impl Struct {
fn find(&mut self, key: &String) -> Option<&Box<Calculation>> {
None
}
fn it(&mut self) -> Option<&Box<Calculation>> {
for key in vec!["1","2","3"] {
if self.find(&key.to_owned()).is_some() {
return self.find(&key.to_owned());
}
}
None
}
}
I had similar issues, and I found a workaround by turning the for loop into a fold, which convinced the compiler that self was not mutably borrowed twice.
It works without using interior mutability or duplicated function call; the only downside is that if the result was found early, it will not short-circuit but continue iterating until the end.
Before:
for key in vec!["1","2","3"] {
let result = self.find(&key.to_owned());
if result.is_some() {
return result
}
}
After:
vec!["1", "2,", "3"]
.iter()
.fold(None, |result, key| match result {
Some(result) => Some(result),
None => self.find(&key.to_string())
})
Working playground link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=92bc73e4bac556ce163e0790c7d3f154
Working code first:
use std::cell::{Ref, RefCell};
use std::rc::Rc;
struct ValueHolder {
value: i32
}
fn give_value(wrapped: &Rc<RefCell<ValueHolder>>) -> Ref<i32> {
Ref::map(
(**wrapped).borrow(),
|borrowed| { &(*borrowed).value },
)
}
fn main() {
println!("Our value: {}", *give_value(
&Rc::new(RefCell::new(ValueHolder { value: 1337 }))
));
}
The relevant part is the give_value function.
I want to return a Ref to something inside a Rc<RefCell<>>, in this case a value inside of a struct.
That works just fine, but since I just started to learn Rust, I wonder, how I could achieve the same thing without using Ref::map.
The "naive" approach:
fn give_value(wrapped: &Rc<RefCell<ValueHolder>>) -> &i32 {
&(*(**wrapped).borrow()).value
}
fails for obvious reasons:
error[E0515]: cannot return value referencing temporary value
--> src/bin/rust_example.rs:9:5
|
9 | &(*(**wrapped).borrow()).value
| ^^^--------------------^^^^^^^
| | |
| | temporary value created here
| returns a value referencing data owned by the current function
So my question is: How can I recreate what the Ref::map function does by myself?
You can't. RefCell requires that Ref be used whenever the value is used, so it knows when it should allow a borrow. When the Ref gets dropped, it signals to the RefCell that it can decrease the borrow count, and if the borrow count is 0, then a mutable borrow can happen. Being able to obtain a reference to the internal data that doesn't borrow the Ref (note that a reference you can return from a function cannot borrow the Ref otherwise you'd be referencing a temporary) would be problematic, because then the Ref could get dropped and the RefCell thinks that it can lend out a mutable borrow, but there's still an immutable borrow to the data. The source code for Ref::map is:
pub fn map<U: ?Sized, F>(orig: Ref<'b, T>, f: F) -> Ref<'b, U>
where
F: FnOnce(&T) -> &U,
{
Ref { value: f(orig.value), borrow: orig.borrow }
}
which uses private fields that you can't access. So, no, you cannot recreate Ref::map yourself.
I am implementing a singly-linked list. I want to add a method add_all_at_index which will take a new list and an index, insert the new list at the specified index and rearrange the tail of the current list to follow after the new list.
Assume the current list is is [1,2,3,4,5]. Calling add_all_at_index at position 2 with list [8,9,10] should result in [1,2,8,9,10,3,4,5].
I'm especially having trouble in assigning the old tail of the list after the new list. I do not know how to append [3,4,5] to the node with value 10 in the list.
The algorithm I am trying to implement is
Save the current next of the node at (index - 1) in a variable called current_next
Set the next of node at (index - 1) to the head of the list
Iterate to the last of the newly added list and set its next to the current_next.
I am having trouble accomplishing the last step. Below is the code I came up with:
use std::fmt::*;
fn main() {
let list: List<i32> = List::new();
}
#[derive(PartialEq, Debug)]
pub struct Node<T: Debug> {
pub element: T,
pub next: Option<Box<Node<T>>>,
}
#[derive(PartialEq, Debug)]
pub struct List<T: Debug> {
pub head: Option<Node<T>>,
}
impl<T: Debug> List<T> {
pub fn new() -> Self {
List { head: None }
}
pub fn add_all_at_index(&mut self, list_to_add: List<T>, index: usize) {
if index > 0 {
let nth_node = self.get_nth_node_mut(index).take(); // Gets a mutable reference to node at index
nth_node.map(|node| {
let current_next = node.next.take(); // I store a reference to the next of nth node,
node.next = list_to_add.head.map(|node| Box::new(node));
// The 3rd step in the algorithm I mentioned above.
let last_node = self.get_nth_node_mut(self.length()); // This line does not compile. Getting multiple errors in this line
last_node.map(|node| node.next = current_next);
});
} else {
self.head = list_to_add.head
}
}
fn get_nth_node_mut(&mut self, n: usize) -> Option<&mut Node<T>> {
let mut nth_node = self.head.as_mut();
for _ in 0..n {
nth_node = match nth_node {
None => return None,
Some(node) => node.next.as_mut().map(|node| &mut **node),
}
}
nth_node
}
pub fn length(&self) -> usize {
let mut count = 0;
let mut current_node = self.head.as_ref();
while let Some(node) = current_node {
count = count + 1;
current_node = node.next.as_ref().map(|node| &**node)
}
count
}
}
The error I am getting is
warning: unused variable: `list`
--> src/main.rs:4:9
|
4 | let list: List<i32> = List::new();
| ^^^^
|
= note: #[warn(unused_variables)] on by default
= note: to avoid this warning, consider using `_list` instead
error[E0500]: closure requires unique access to `self` but `*self` is already borrowed
--> src/main.rs:26:26
|
25 | let nth_node = self.get_nth_node_mut(index).take(); // Gets a mutable reference to node at index
| ---- borrow occurs here
26 | nth_node.map(|node| {
| ^^^^^^ closure construction occurs here
...
31 | let last_node = self.get_nth_node_mut(self.length()); // This line does not compile. Getting multiple errors in this line
| ---- borrow occurs due to use of `self` in closure
...
34 | } else {
| - borrow ends here
error[E0502]: cannot borrow `**self` as immutable because it is also borrowed as mutable
--> src/main.rs:31:55
|
31 | let last_node = self.get_nth_node_mut(self.length()); // This line does not compile. Getting multiple errors in this line
| ---- ^^^^ immutable borrow occurs here
| |
| mutable borrow occurs here
32 | last_node.map(|node| node.next = current_next);
33 | });
| - mutable borrow ends here
Is this even the right approach to implement add_all_at_index?
I even tried implementing an iterator that returns a mutable reference to a node but I was not able to do that either. I have pasted the entire code in a gist at https://gist.github.com/hardvain/32fca033bb61a5e3bf8bbeeb32fbbd5e
First, the solution:
pub fn add_all_at_index(&mut self, list_to_add: List<T>, index: usize) {
if index > 0 {
let tail = {
let nth_node = self.get_nth_node_mut(index).take();
nth_node.map(|node| {
let current_next = node.next.take();
node.next = list_to_add.head.map(|node| Box::new(node));
current_next
})
};
if let Some(current_next) = tail {
let n = self.length();
let last_node = self.get_nth_node_mut(n);
last_node.map(|node| node.next = current_next);
}
} else {
self.head = list_to_add.head
}
}
Ugly, right? Right. Getting this working required several changes:
I moved step 3 (reattaching the tail of the list) outside of the closure passed to map, so that nth_node (which borrows self) wouldn't still be alive when you try to borrow self again to get the length.
I therefore had to save current_next, so I had the closure return it, and stored the result of map in a new variable called tail. So tail is an Option<Box<Node<T>>>.
I wrapped the tail-reattaching part in an if let to destructure tail and get current_next back out.
Then, I separated self.get_nth_node_mut(self.length()) into two statements to resolve the remaining borrowing error.
Some followup suggestions:
Using .map() for side effects and then ignoring the return value is unidiomatic. Use if let to run code on the content of an Option.
You're ignoring all the None cases. If you try to call add_all_at_index with an index that is out of range, nothing happens to self and list_to_add is just lost. The function should probably return a Result of some kind, or perhaps a bool.
.take() is for getting an Option<T> out of a &mut Option<T>. It doesn't do anything useful to an Option<&mut T>.
|node| Box::new(node) is just Box::new.
Because the first node is boxed but none of the others are, you will have to write a lot of special case code, and there's not a transparent conversion between a Node and a List.
Using get_nth_node_mut in the implementation of add_all_at_index almost forces you to traverse the list twice. Because it's implemented on List instead of on Node, you can't easily get a reference to the last element of the list, so you end up calling length() (making the total number of traversals 3) and then get_nth_node_mut again to dig up the last element.
Some of the ugliness can be mitigated by careful interface design -- for instance, this method becomes cleaner if List has a split_at_index method -- but some of it is just due to the fact that linked lists are ugly. Particularly ugly in Rust, because the language forbids shared mutable references, even temporarily. You would have to use unsafe to write many linked list operations in Rust the same way you would in C.
If you have not already, please read Learning Rust With Entirely Too Many Linked Lists. This book addresses many of the subtleties that arise when you try to implement a linked list in Rust.
Why doesn't this code compile:
fn use_cursor(cursor: &mut io::Cursor<&mut Vec<u8>>) {
// do some work
}
fn take_reference(data: &mut Vec<u8>) {
{
let mut buf = io::Cursor::new(data);
use_cursor(&mut buf);
}
data.len();
}
fn produce_data() {
let mut data = Vec::new();
take_reference(&mut data);
data.len();
}
The error in this case is:
error[E0382]: use of moved value: `*data`
--> src/main.rs:14:5
|
9 | let mut buf = io::Cursor::new(data);
| ---- value moved here
...
14 | data.len();
| ^^^^ value used here after move
|
= note: move occurs because `data` has type `&mut std::vec::Vec<u8>`, which does not implement the `Copy` trait
The signature of io::Cursor::new is such that it takes ownership of its argument. In this case, the argument is a mutable reference to a Vec.
pub fn new(inner: T) -> Cursor<T>
It sort of makes sense to me; because Cursor::new takes ownership of its argument (and not a reference) we can't use that value later on. At the same time it doesn't make sense: we essentially only pass a mutable reference and the cursor goes out of scope afterwards anyway.
In the produce_data function we also pass a mutable reference to take_reference, and it doesn't produce a error when trying to use data again, unlike inside take_reference.
I found it possible to 'reclaim' the reference by using Cursor.into_inner(), but it feels a bit weird to do it manually, since in normal use-cases the borrow-checker is perfectly capable of doing it itself.
Is there a nicer solution to this problem than using .into_inner()? Maybe there's something else I don't understand about the borrow-checker?
Normally, when you pass a mutable reference to a function, the compiler implicitly performs a reborrow. This produces a new borrow with a shorter lifetime.
When the parameter is generic (and is not of the form &mut T), the compiler doesn't do this reborrowing automatically1. However, you can do it manually by dereferencing your existing mutable reference and then referencing it again:
fn take_reference(data: &mut Vec<u8>) {
{
let mut buf = io::Cursor::new(&mut *data);
use_cursor(&mut buf);
}
data.len();
}
1 — This is because the current compiler architecture only allows a chance to do a coercion if both the source and target types are known at the coercion site.