This question already has an answer here:
Rust not allowing mutable borrow when splitting properly
(1 answer)
Closed 9 months ago.
Consider these two code examples
struct A {
v: Vec<usize>,
a: usize,
}
impl A {
fn f(&mut self) {
for _ in self.v.iter_mut() {
self.f_mut();
}
}
fn f_mut(&mut self) {
self.a += 1;
}
}
which produces the compile error
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/binaries/main.rs:9:13
|
8 | for _ in self.v.iter_mut() {
| -----------------
| |
| first mutable borrow occurs here
| first borrow later used here
9 | self.f_mut();
| ^^^^ second mutable borrow occurs here
error: aborting due to previous error
whereas this code
struct A {
v: Vec<usize>,
a: usize,
}
impl A {
fn f(&mut self) {
for _ in self.v.iter_mut() {
self.a += 1;
}
}
fn f_mut(&mut self) {
self.a += 1;
}
}
compiles just fine. Obviously these two pieces of code have identical behaviour so I am wondering if there is some way to get Rust to accept the first solution?
In my project I have some quite large for loops which I would like to refactor by splitting the body of the for-loop into separate function calls, but Rust doesn't allow me to do it.
That is, my more complicated code structured as example 2 compiles fine, but refactored into easier digestable functions as example 1 gives me the cannot borrow *self as mutable more than once at a time error.
Your second example works because it performs a splitting borrow. The compiler can see that you're only borrowing self.v mutably, so it lets you borrow other fields on self mutably as well, since they don't overlap.
However, in the first example, self.f_mut() requires borrowing all of *self mutably (because it takes &mut self), which overlaps with the borrow of self.v, and is therefore disallowed. This requires borrowing the whole value because the borrow checker only uses the signature of the called function -- it does not look into the function to see that it doesn't use self.v. (If it did look into functions, then changing the body of a function without changing its signature could be an API-breaking change!)
Simply put, a function call is a "hard" boundary for the borrow checker. By using a function call in the first example, the detail that only self.a is used is hidden from the borrow checker.
I am interested in knowing the idiomatic/canonical way of making self-referential structures in Rust. The related question Why can't I store a value and a reference to that value in the same struct explains the problem, but try as I might, I couldn't figure out the answer in the existing question (although there were some useful hints).
I have come up with a solution, but I am unsure of how safe it is, or if it is the idiomatic way to solve this problem; if it isn't, I would very much like to know what the usual solution is.
I have an existing structure in my program that holds a reference to a sequence. Sequences hold information about chromosomes so they can be rather long, and copying them isn't a viable idea.
// My real Foo is more complicated than this and is an existing
// type I'd rather not have to rewrite if I can avoid it...
struct Foo<'a> {
x: &'a [usize],
// more here...
}
impl<'a> Foo<'a> {
pub fn new(x: &'a [usize]) -> Self {
Foo {
x, /* more here... */
}
}
}
I now need a new structure that reduces the sequence to something smaller and then builds a Foo structure over the reduced string, and since someone has to own both reduced string and Foo object, I would like to put both in a structure.
// My real Bar is slightly more complicated, but it boils down to having
// a vector it owns and a Foo over that vector.
struct Bar<'a> {
x: Vec<usize>,
y: Foo<'a>, // has a reference to &x
}
// This doesn't work because x is moved after y has borrowed it
impl<'a> Bar<'a> {
pub fn new() -> Self {
let x = vec![1, 2, 3];
let y = Foo::new(&x);
Bar { x, y }
}
}
Now, this doesn't work because the Foo object in a Bar refers into the Bar object
and if the Bar object moves, the reference will point into memory that is no longer occupied by the Bar object
To avoid this problem, the x element in Bar must sit on the heap and not move around. (I think the data in a Vec already sits happily on the heap, but that doesn't seem to help me here).
A pinned box should do the trick, I belive.
struct Bar<'a> {
x: Pin<Box<Vec<usize>>>,
y: Foo<'a>,
}
Now the structure looks like this
and when I move it, the references point to the same memory.
However, moving x to the heap isn't enough for the type-checker. It still thinks that moving the pinned box will move what it points to.
If I implement Bar's constructor like this:
impl<'a> Bar<'a> {
pub fn new() -> Self {
let v: Vec<usize> = vec![1, 2, 3];
let x = Box::pin(v);
let y = Foo::new(&x);
Bar { x, y }
}
}
I get the error
error[E0515]: cannot return value referencing local variable `x`
--> src/main.rs:22:9
|
21 | let y = Foo::new(&x);
| -- `x` is borrowed here
22 | Bar { x, y }
| ^^^^^^^^^^^^ returns a value referencing data owned by the current function
error[E0505]: cannot move out of `x` because it is borrowed
--> src/main.rs:22:15
|
17 | impl<'a> Bar<'a> {
| -- lifetime `'a` defined here
...
21 | let y = Foo::new(&x);
| -- borrow of `x` occurs here
22 | Bar { x, y }
| ------^-----
| | |
| | move out of `x` occurs here
| returning this value requires that `x` is borrowed for `'a`
Some errors have detailed explanations: E0505, E0515.
For more information about an error, try `rustc --explain E0505`.
(Playground)
Even though the object I take a reference of sits on the heap, and doesn't move, the checker still sees me borrowing from an object that moves, and that, of course, is a no-no.
Here, you might stop and notice that I am trying to make two pointers to the same object, so Rc or Arc is an obvious solution. And it is, but I would have to change the implementation of Foo to have an Rc member instead of a reference. While I do have control of the source code for Foo, and I could update it and all the code that uses it, I am reluctant to make such a major change if I can avoid it. And I could have been in a situation where I am not in control of the Foo, so I couldn't change its implementation, and I would love to know how I would solve that situation then.
The only solution I could get to work was to get a raw pointer to x, so the type-checker doesn't see that I borrow it, and then connect x and y though that.
impl<'a> Bar<'a> {
pub fn new() -> Self {
let v: Vec<usize> = vec![1, 2, 3];
let x = Box::new(v);
let (x, y) = unsafe {
let ptr: *mut Vec<usize> = Box::into_raw(x);
let w: &Vec<usize> = ptr.as_ref().unwrap();
(Pin::new(Box::from_raw(ptr)), Foo::new(&w))
};
Bar { x, y }
}
}
Playground code here
What I don't know is if this is the right way to do it. It seems rather complicated, but perhaps it is the only way to make a structure like this in Rust? That some sort of unsafe is needed to trick the compiler. So that is the first of my questions.
The second is, if this is safe to do? Of course it is unsafe in the technical sense, but am I risking creating a reference to memory that might not be valid later? It is my impression that Pin should guarantee that the object remains where it is supposed to sit, and that the lifetime of the Bar<'a> and Foo<'a> objects should ensure that the reference doesn't out-live the vector, but once I have gone unsafe, could that promise be broken?
Update
The owning_ref crate has functionality that looks like what I need. You can create owned objects that present their references as well.
There is an OwningRef type that wraps an object and a reference, and it would be wonderful if you could have the slice in that and getting the reference wasn't seen as borrowing from the object, but obviously that isn't the case. Code such as this
use owning_ref::OwningRef;
struct Bar<'a> {
x: OwningRef<Vec<usize>, [usize]>,
y: Foo<'a>, // has a reference to &x
}
// This doesn't work because x is moved after y has borrowed it
impl<'a> Bar<'a> {
pub fn new() -> Self {
let v: Vec<usize> = vec![1, 2, 3];
let x = OwningRef::new(v);
let y = Foo::new(x.as_ref());
Bar { x, y }
}
}
you get the error
error[E0515]: cannot return value referencing local variable `x`
--> src/main.rs:22:9
|
21 | let y = Foo::new(x.as_ref());
| ---------- `x` is borrowed here
22 | Bar { x, y }
| ^^^^^^^^^^^^ returns a value referencing data owned by the current function
error[E0505]: cannot move out of `x` because it is borrowed
--> src/main.rs:22:15
|
17 | impl<'a> Bar<'a> {
| -- lifetime `'a` defined here
...
21 | let y = Foo::new(x.as_ref());
| ---------- borrow of `x` occurs here
22 | Bar { x, y }
| ------^-----
| | |
| | move out of `x` occurs here
| returning this value requires that `x` is borrowed for `'a`
Some errors have detailed explanations: E0505, E0515.
For more information about an error, try `rustc --explain E0505`.
error: could not compile `foo` due to 2 previous errors
The reason is the same as before: I borrow a reference to x and then I move it.
There are different wrapper objects in the crate, and in various combinations they will let me get close to a solution and then snatch it away from me, because what I borrow I still cannot move later, e.g.:
use owning_ref::{BoxRef, OwningRef};
struct Bar<'a> {
x: OwningRef<Box<Vec<usize>>, Vec<usize>>,
y: Foo<'a>, // has a reference to &x
}
// This doesn't work because x is moved after y has borrowed it
impl<'a> Bar<'a> {
pub fn new() -> Self {
let v: Vec<usize> = vec![1, 2, 3];
let v = Box::new(v); // Vector on the heap
let x = BoxRef::new(v);
let y = Foo::new(x.as_ref());
Bar { x, y }
}
}
error[E0515]: cannot return value referencing local variable `x`
--> src/main.rs:23:9
|
22 | let y = Foo::new(x.as_ref());
| ---------- `x` is borrowed here
23 | Bar { x, y }
| ^^^^^^^^^^^^ returns a value referencing data owned by the current function
error[E0505]: cannot move out of `x` because it is borrowed
--> src/main.rs:23:15
|
17 | impl<'a> Bar<'a> {
| -- lifetime `'a` defined here
...
22 | let y = Foo::new(x.as_ref());
| ---------- borrow of `x` occurs here
23 | Bar { x, y }
| ------^-----
| | |
| | move out of `x` occurs here
| returning this value requires that `x` is borrowed for `'a`
Some errors have detailed explanations: E0505, E0515.
For more information about an error, try `rustc --explain E0505`.
I can get around this by going unsafe and work with a pointer, of course, but then I am back to the solution I had with Pin and pointer hacking. I strongly feel that there is a solution here, (especially because having a Box<Vec<...>> and the corresponding Vec<...> isn't adding much to the table so there must be more to the crate), but what it is is eluding me.
(I think the data in a Vec already sits happily on the heap, but that doesn't seem to help me here).
Indeed the data in a Vec does already sit on the heap, and the x: &'a [usize] in Foo is already a reference to that heap allocation; so your problem here is not (as shown in your graphics) that moving Bar would result in (the undefined behaviour of) a dangling reference.
However, what happens if the Vec were to outgrow its current allocation? It would reallocate and be moved from its present heap allocation to another—and this would result in a dangling reference. Hence the borrow checker must enforce that, so long as anything (e.g. a Foo) that borrows from the Vec exists, the Vec cannot be mutated. Yet here we already have an expressivity problem: the Rust language has no way to annotate Bar to indicate this relationship.
Your proposed unsafe solution uses <*mut _>::as_ref, whose safety documentation includes the following requirement (emphasis added):
You must enforce Rust’s aliasing rules, since the returned lifetime 'a is arbitrarily chosen and does not necessarily reflect the actual lifetime of the data. In particular, for the duration of this lifetime, the memory the pointer points to must not get mutated (except inside UnsafeCell).
This is the key bit of the compiler's safety checks that you are trying to opt out of—but because accessing Bar now requires that one uphold this requirement, you do not have a completely safe abstraction. In my view, a raw pointer would be a tad safer here because it forces one to consider the safety of every access.
For example, one issue that immediately springs to mind is that x is declared before y in Bar and therefore, upon destruction, it will be dropped first: the Vec's heap allocation will be freed while Foo still holds references into it: undefined behaviour! Simply reordering the fields would avoid this particular problem, but there would have been no such problem with raw pointers (and any attempt to dereference them in Foo's drop handler would have forced one to consider whether they were still dereferenceable at that time).
Personally, I would try to avoid self-referencing here and probably use an arena.
I think I have finally grokked ouroboros and that is an elegant solution.
You use a macro, self_referencing when defining a structure, and inside the structure you can specify that one entry borrows others. For my application, I got it to work like this:
use ouroboros::self_referencing;
#[self_referencing]
struct _Bar {
x: Vec<usize>,
#[borrows(x)]
#[covariant]
y: Foo<'this>,
}
struct Bar(pub _Bar);
The y element references x, so I specify that. I'm sure why co-/contra-varianse is needed in this particular case where there is only one lifetime, but it specififes whether other references should live longer or can live shorter than the object. I've defined the struct as _Bar and then wrapped it in Bar. This is because macro will create a new method, and I don't want the default one. At the same time I wnat to call my constructor new to stick with tradition. So I wrap the type and write my own constructor:
impl Bar {
pub fn new() -> Self {
let x: Vec<usize> = vec![1, 2, 3];
let _bar = _BarBuilder {
x,
y_builder: |x: &Vec<usize>| Foo::new(&x),
}
.build();
Bar(_bar)
}
}
I don't use the generated _Bar::new but a generated _BarBuilder object where I can specify how to get the y value from the x reference.
I have also written accessors to get the two values. There isn't anything special here.
impl Bar {
pub fn x(&self) -> &Vec<usize> {
self.0.borrow_x()
}
pub fn y(&self) -> &Foo {
self.0.borrow_y()
}
}
and with that my trivial little test case runs...
fn main() {
let bar = Bar::new();
let vec = bar.x();
for &i in vec {
println!("i == {}", i);
}
let vec = bar.y().x;
for &i in vec {
println!("i == {}", i);
}
}
This is probably the best solution so far, assuming that there are no hidden costs that I am currently unaware of.
This is my code:
use std::collections::HashMap;
struct Foo {
pub map : HashMap<i32, String>
}
impl Foo {
fn foo(&mut self, x: &String) -> i32 {
// I'm planning to use/modify "x" here and also modify "self"
42
}
fn bar(&mut self) -> i32 {
let x = self.map.get_mut(&1).unwrap();
self.foo(x)
}
}
I'm getting:
error[E0499]: cannot borrow `*self` as mutable more than once at a time
--> src/main.rs:13:9
|
12 | let x = self.map.get_mut(&1).unwrap();
| -------------------- first mutable borrow occurs here
13 | self.foo(x)
| ^^^^^^^^^-^
| | |
| | first borrow later used here
| second mutable borrow occurs here
What's going on?
Modifying self and x here breaks memory safety (at least in the general situation, which is what Rust must deal with). Consider the following implementation of foo which is allowed by your signature (fixing &String to &str):
fn foo(&mut self, x: &str) -> i32 {
self.map.clear();
println!("{}", x);
42
}
But you're calling this with x being a reference to something inside of self.map. So x could be destroyed by the time it's used. That's invalid, and Rust can't prove you won't do that, because you said you might. (Kevin Anderson provides a helpful comment below if you're coming from a GC language like C# where "reference" has a different meaning.)
How to fix this depends on what you're really trying to do, though one approach would be to clone the string so it cannot be destroyed:
fn bar(&mut self) -> i32 {
let x = self.map.get(&1).unwrap().clone(); // <== now you have a copy
self.foo(&x)
}
Note this got rid of the get_mut(). It's unclear what that was for. If you need an exclusive (mut) reference into the map, then you'll need to do that separately, and you can't do that directly while also holding an exclusive reference to self for the reasons above. Remember that mut means "exclusive access," not "mutable." A side effect of having exclusive access is that mutation is allowed.
If you really need something along these lines, you need to wrap your values (String) in Arc so that you can maintain reference counts and have shared ownership. But I would first try to redesign your algorithm to avoid this.
I'm new to Rust and looks like I'm seriously missing some concept here.
use std::thread;
fn main() {
let mut children = vec![];
//spawn threads
for i in 0..10 {
let c = thread::spawn(|| {
println!("thread id is {}", i);
});
children.push(c);
}
for j in children {
j.join().expect("thread joining issue");
}
}
It fails with the error:
error[E0373]: closure may outlive the current function, but it borrows `i`, which is owned by the current function
Since the type of i is i32 , and there are no references involved shouldn't Rust copy the value instead of forcing to move ?
The answer to your original question is that println! borrows its arguments. However, as you pointed out in the comments, even (apparently) moving the integer into the closure still causes a compile error.
For the purposes of this answer, we'll work with this code.
fn use_closure<F: FnOnce() + 'static>(_: F) {}
fn main() {
let x: i32 = 0;
use_closure(|| {
let _y = x;
});
}
(playground)
use_closure simulates what thread::spawn does in the original code: it consumes a closure whose type has to be 'static.
Attempting to compile this gives the error
error[E0373]: closure may outlive the current function, but it borrows `x`, which is owned by the current function
--> src/main.rs:5:17
|
5 | use_closure(|| {
| ^^ may outlive borrowed value `x`
6 | let _y = x;
| - `x` is borrowed here
|
note: function requires argument type to outlive `'static`
--> src/main.rs:5:5
|
5 | / use_closure(|| {
6 | | let _y = x;
7 | | });
| |______^
help: to force the closure to take ownership of `x` (and any other referenced variables), use the `move` keyword
|
5 | use_closure(move || {
| ^^^^^^^
Wait, what?
6 | let _y = x;
| - `x` is borrowed here
Why is x borrowed there? Shouldn't it be a copy? The answer lies in "capture modes" for closures. From the documentation
The compiler prefers to capture a closed-over variable by immutable borrow, followed by unique immutable borrow (see below), by mutable borrow, and finally by move. It will pick the first choice of these that allows the closure to compile. The choice is made only with regards to the contents of the closure expression; the compiler does not take into account surrounding code, such as the lifetimes of involved variables.
Precisely because x has a Copy type, the closure itself can compile with a mere immutable borrow. Given an immutable borrow of x (call it bor), we can do our assignment to _y with _y = *bor. This isn't a "move out of data behind a reference" because this is a copy instead of a move.
However, since the closure borrows a local variable, its type won't be 'static, so it won't be usable in use_closure or thread::spawn.
Trying the same code with a type that isn't Copy, it actually works perfectly, since the closure is forced to capture x by moving it.
fn use_closure<F: FnOnce() + 'static>(_: F) {}
fn main() {
let x: Vec<i32> = vec![];
use_closure(|| {
let _y = x;
});
}
(playground)
Of course, as you already know, the solution is to use the move keyword in front of the closure. This forces all captured variables to be moved into the closure. Since the variable won't be borrowed, the closure will have a static type and will be able to be used in use_closure or thread::spawn.
fn use_closure<F: FnOnce() + 'static>(_: F) {}
fn main() {
let x: i32 = 0;
use_closure(move || {
let _y = x;
});
}
(playground)
I'm learning Rust and tried coding a doubly-linked list. However, I'm stuck already at a typical iterative traversal implementation. I'm getting the impression that the borrow checker / drop checker is too strict and cannot infer the correct lifetime for the borrow when it crosses the function boundary from RefCell. I need to repeatedly set a variable binding (curr in this case) to the borrow of its current contents:
use std::cell::RefCell;
use std::rc::Rc;
pub struct LinkedList<T> {
head: Option<Rc<RefCell<LinkedNode<T>>>>,
// ...
}
struct LinkedNode<T> {
value: T,
next: Option<Rc<RefCell<LinkedNode<T>>>>,
// ...
}
impl<T> LinkedList<T> {
pub fn insert(&mut self, value: T, idx: usize) -> &mut LinkedList<T> {
// ... some logic ...
// This is the traversal that fails to compile.
let mut curr = self.head.as_ref().unwrap();
for _ in 1..idx {
curr = curr.borrow().next.as_ref().unwrap()
}
// I want to use curr here.
// ...
unimplemented!()
}
}
The compiler complains:
Without NLL
error[E0597]: borrowed value does not live long enough
--> src/lib.rs:22:20
|
22 | curr = curr.borrow().next.as_ref().unwrap()
| ^^^^^^^^^^^^^ temporary value does not live long enough
23 | }
| - temporary value dropped here while still borrowed
...
28 | }
| - temporary value needs to live until here
|
= note: consider using a `let` binding to increase its lifetime
With NLL
error[E0716]: temporary value dropped while borrowed
--> src/lib.rs:22:20
|
22 | curr = curr.borrow().next.as_ref().unwrap()
| ^^^^^^^^^^^^^
| |
| creates a temporary which is freed while still in use
| a temporary with access to the borrow is created here ...
23 | }
| -
| |
| temporary value is freed at the end of this statement
| ... and the borrow might be used here, when that temporary is dropped and runs the destructor for type `std::cell::Ref<'_, LinkedNode<T>>`
|
= note: consider using a `let` binding to create a longer lived value
= note: The temporary is part of an expression at the end of a block. Consider adding semicolon after the expression so its temporaries are dropped sooner, before the local variables declared by the block are dropped.
I would really appreciate a iterative solution (non-recursive) to this problem.
You can clone Rc to avoid lifetime issues:
let mut curr = self.head.as_ref().unwrap().clone();
for _ in 1..idx {
let t = curr.borrow().next.as_ref().unwrap().clone();
curr = t;
}
Here's a smaller reproduction that I believe shows the same problem:
use std::cell::RefCell;
fn main() {
let foo = RefCell::new(Some(42));
let x = foo.borrow().as_ref().unwrap();
}
As I read it:
foo.borrow() returns a cell::Ref, a type of smart pointer. In this case, the smart pointer acts like an &Option<i32>.
as_ref() creates an Option<&i32> where the inner reference has the same lifetime as the smart pointer.
The Option is discarded, yielding only an &i32, still with a lifetime of the smart pointer.
Notably, the smart pointer Ref only lasts for the statement, but the code attempts to return a reference into the Ref that would outlive the statement.
Generally, the solution would be to do something like this:
let foo_borrow = foo.borrow();
let x = foo_borrow.as_ref().unwrap();
This keeps the smart pointer around longer, allowing the lifetime of the reference to be valid for as long as foo_borrow (representing the borrow itself) exists.
In the case of a loop, there's not much you can do, as you essentially want to borrow every previous node until you get to the next one.