According to this question, it is not possible to borrow parts of a struct, only the entire struct. This makes sense, however I would not expect the following code (playground) to compile, which it does:
struct Struct (u32, u32);
fn foo(a: &mut u32, b: &u32) {
}
fn main() {
let mut s = Struct ( 1, 2 );
foo(&mut s.0, &s.1);
}
Why does this work?
Additionally, is there any way to get the compiler to make the same differentiation between borrowing members when some indirection is introduced via. a method, without using a RefCell or other run-time checking. See (playground):
struct Struct (u32, u32);
impl Struct {
fn a(&mut self) -> &mut u32 {
&mut self.0
}
fn b(&self) -> &u32 {
&self.1
}
}
fn foo(a: &mut u32, b: &u32) {
}
fn main() {
let mut s = Struct ( 1, 2 );
foo(s.a(), s.b());
}
At the moment this fails with:
error[E0502]: cannot borrow `s` as immutable because it is also borrowed as mutable
--> src/main.rs:18:16
|
18 | foo(s.a(), s.b());
| --- ----- ^^^^^ immutable borrow occurs here
| | |
| | mutable borrow occurs here
| mutable borrow later used by call
I realise this is a bit of an obtuse example, my real case involves borrowing a number of members so the borrowing methods' implementations are more complex. There are still no members which are borrowed by both functions however.
Rust very deliberately does not extend its inference powers across function boundaries. This helps in making code more forwards compatible as you only need to keep the signature consistent and not its internals.
For example, consider your own code. If you decided later that a() should actually return a reference to self.1 then it would break all the code that used a() and b() together like you did. With the current limitations, you can easily change what field the reference comes from and not have to worry about breaking anyone.
Unfortunately, this makes what you want to do impossible.
I suggest giving the problem a higher level look. Do a() and b() really belong together on the same struct? Would it perhaps be better to split the two fields into their own structs?
Ideally, when you take a mutable reference to a struct, you would be using all (or most) of the struct, and there would be no need for someone else to be using that struct.
Related
Given the following struct and impl:
use std::slice::Iter;
use std::cell::RefCell;
struct Foo {
bar: RefCell<Vec<u32>>,
}
impl Foo {
pub fn iter(&self) -> Iter<u32> {
self.bar.borrow().iter()
}
}
fn main() {}
I get an error message about a lifetime issue:
error: borrowed value does not live long enough
--> src/main.rs:9:9
|
9 | self.bar.borrow().iter()
| ^^^^^^^^^^^^^^^^^ does not live long enough
10 | }
| - temporary value only lives until here
|
note: borrowed value must be valid for the anonymous lifetime #1 defined on the body at 8:36...
--> src/main.rs:8:37
|
8 | pub fn iter(&self) -> Iter<u32> {
| _____________________________________^ starting here...
9 | | self.bar.borrow().iter()
10 | | }
| |_____^ ...ending here
How am I able to return and use bars iterator?
You cannot do this because it would allow you to circumvent runtime checks for uniqueness violations.
RefCell provides you a way to "defer" mutability exclusiveness checks to runtime, in exchange allowing mutation of the data it holds inside through shared references. This is done using RAII guards: you can obtain a guard object using a shared reference to RefCell, and then access the data inside RefCell using this guard object:
&'a RefCell<T> -> Ref<'a, T> (with borrow) or RefMut<'a, T> (with borrow_mut)
&'b Ref<'a, T> -> &'b T
&'b mut RefMut<'a, T> -> &'b mut T
The key point here is that 'b is different from 'a, which allows one to obtain &mut T references without having a &mut reference to the RefCell. However, these references will be linked to the guard instead and can't live longer than the guard. This is done intentionally: Ref and RefMut destructors toggle various flags inside their RefCell to force mutability checks and to force borrow() and borrow_mut() panic if these checks fail.
The simplest thing you can do is to return a wrapper around Ref, a reference to which would implement IntoIterator:
use std::cell::Ref;
struct VecRefWrapper<'a, T: 'a> {
r: Ref<'a, Vec<T>>
}
impl<'a, 'b: 'a, T: 'a> IntoIterator for &'b VecRefWrapper<'a, T> {
type IntoIter = Iter<'a, T>;
type Item = &'a T;
fn into_iter(self) -> Iter<'a, T> {
self.r.iter()
}
}
(try it on playground)
You can't implement IntoIterator for VecRefWrapper directly because then the internal Ref will be consumed by into_iter(), giving you essentially the same situation you're in now.
Alternate Solution
Here is an alternate solution that uses interior mutability as it was intended. Instead of creating an iterator for &T values, we should create an iterator for Ref<T> values, which deference automatically.
struct Iter<'a, T> {
inner: Option<Ref<'a, [T]>>,
}
impl<'a, T> Iterator for Iter<'a, T> {
type Item = Ref<'a, T>;
fn next(&mut self) -> Option<Self::Item> {
match self.inner.take() {
Some(borrow) => match *borrow {
[] => None,
[_, ..] => {
let (head, tail) = Ref::map_split(borrow, |slice| {
(&slice[0], &slice[1..])
});
self.inner.replace(tail);
Some(head)
}
},
None => None,
}
}
}
Playground
Explanation
The accepted answer has a few significant drawbacks that may confuse those new to Rust. I will explain how, in my personal experience, the accepted answer might actually be harmful to a beginner, and why I believe this alternative uses interior mutability and iterators as they were intended.
As the previous answer importantly highlights, using RefCell creates a divergent type hierarchy that isolates mutable and immutable access to a shared value, but you do not have to worry about lifetimes to solve the iteration problem:
RefCell<T> .borrow() -> Ref<T> .deref() -> &T
RefCell<T> .borrow_mut() -> RefMut<T> .deref_mut() -> &mut T
The key to solving this without lifetimes is the Ref::map method, which is critically missed in the book. Ref::map "makes a new reference to a component of the borrowed data", or in other words converts a Ref<T> of the outer type to a Ref<U> of some inner value:
Ref::map(Ref<T>, ...) -> Ref<U>
Ref::map and its counterpart RefMut::map are the real stars of the interior mutability pattern, not borrow() and borrow_mut().
Why? Because unlike borrow() and borrow_mut(), Ref::mut and RefMut::map, allow you to create references to interior values that can be "returned".
Consider adding a first() method to the Foo struct described in the question:
fn first(&self) -> &u32 {
&self.bar.borrow()[0]
}
Nope, .borrow() makes a temporary Ref that only lives until the method returns:
error[E0515]: cannot return value referencing temporary value
--> src/main.rs:9:11
|
9 | &self.bar.borrow()[0]
| ^-----------------^^^
| ||
| |temporary value created here
| returns a value referencing data owned by the current function
error: aborting due to previous error; 1 warning emitted
We can make it more obvious what is happening if we break it up and make the implicit deference explicit:
fn first(&self) -> &u32 {
let borrow: Ref<_> = self.bar.borrow();
let bar: &Vec<u32> = borrow.deref();
&bar[0]
}
Now we can see that .borrow() creates a Ref<T> that is owned by the method's scope, and isn't returned and therefore dropped even before the reference it provided can be used. So, what we really need is to return an owned type instead of a reference. We want to return a Ref<T>, as it implements Deref for us!
Ref::map will help us do just that for component (internal) values:
fn first(&self) -> Ref<u32> {
Ref::map(self.bar.borrow(), |bar| &bar[0])
}
Of course, the .deref() will still happen automatically, and Ref<u32> will be mostly be referentially transparent as &u32.
Gotcha. One easy mistake to make when using Ref::map is to try to create an owned value in the closure, which is not possible as when we tried to use borrow(). Consider the type signature of the second parameter, the function: FnOnce(&T) -> &U,. It returns a reference, not an owned type!
This is why we use a slice in the answer &v[..] instead of trying to use the vector's .iter() method, which returns an owned std::slice::Iter<'a, T>. Slices are a reference type.
Additional Thoughts
Alright, so now I will attempt to justify why this solution is better than the accepted answer.
First, the use of IntoIterator is inconsistent with the Rust standard library, and arguably the purpose and intent of the trait. The trait method consumes self: fn into_iter(self) -> ....
let v = vec![1,2,3,4];
let i = v.into_iter();
// v is no longer valid, it was moved into the iterator
Using IntoIterator indirectly for a wrapper is inconsistent as you consume the wrapper and not the collection. In my experience, beginners will benefit from sticking with the conventions. We should use a regular Iterator.
Next, the IntoIterator trait is implemented for the reference &VecRefWrapper and not the owned type VecRefWrapper.
Suppose you are implementing a library. The consumers of your API will have to seemingly arbitrarily decorate owned values with reference operators, as is demonstrated in the example on the playground:
for &i in &foo.iter() {
println!("{}", i);
}
This is a subtle and confusing distinction if you are new to Rust. Why do we have to take a reference to the value when it is anonymously owned by - and should only exist for - the scope of the loop?
Finally, the solution above shows how it is possible to drill all they way into your data with interior mutability, and makes the path forward for implementing a mutable iterator clear as well. Use RefMut.
From my research there is currently no solution to this problem. The biggest problem here is self-referentiality and the fact that rust cannot prove your code to be safe. Or at least not in the generic fashion.
I think it's safe to assume that crates like ouroboros, self-cell and owning_ref are solution if you know that your struct (T in Ref<T>) does not contain any smart pointers nor anything which could invalidate any pointers you might obtain in your "dependent" struct.
Note that self-cell does this safely with extra heap allocation which might be ok in some cases.
There was also RFC for adding map_value to Ref<T> but as you can see, there is always some way to invalidate pointers in general (which does not mean your specific case is wrong it's just that it probably will never be added to the core library/language because it cannot be guaranteed for any T)
Yeah, so no answer, sorry. impl IntoIterator for &T works but I think it's rather hack and it forces you to write for x in &iter instead of for x in iter
So, I have the following problem. I have a structure implementation, and one of the methods of this structure consumes the instance of the structure, which is ok by me, as it is literally the last thing I want to do in my program with this instance (my_struct), but it's totally not OK with the borrow checker:
use std::thread;
struct MyStruct {
some_field: Vec<String>
}
impl MyStruct {
fn new() -> Self {
MyStruct {
some_field: vec!("str".to_string())
}
}
fn do_something_with_self_in_thread(&'static mut self) {
thread::spawn(move || self.some_field = vec!("another_str".to_string()));
}
}
fn main() {
let my_struct: &'static mut MyStruct = &mut MyStruct::new();
my_struct.do_something_with_self_in_thread()
// Some blocking call will follow
}
Error:
error[E0716]: temporary value dropped while borrowed
--> src/main.rs:20:49
|
20 | let my_struct: &'static mut MyStruct = &mut MyStruct::new();
| --------------------- ^^^^^^^^^^^^^^^ creates a temporary which is freed while still in use
| |
| type annotation requires that borrow lasts for `'static`
21 | my_struct.do_something_with_self_in_thread()
22 | }
| - temporary value is freed at the end of this statement
For more information about this error, try `rustc --explain E0716`.
I tried playing around with lifetimes, but resultless.
How do I get myself out of this situation?
Playground link
When we say a method consumes the instance, then it should take self and not a mutable borrow &mut self in the definition.
i.e.
impl MyStruct {
...
fn do_something_with_self_in_thread(self) {
...
}
}
As commenters have already said, your second problem is with lifetimes. The compiler can't reason about references not outliving owners of resources if they are in separate threads. So to this end the thread::spawn method has trait bounds requiring that the closure is 'static, which means the variables it captures can live as long as they like. Examples which are valid would be owned types T, not references &T or &mut T, unless they are &'static T types. That's not the case in your code as you've instantiated a struct inside main (albeit erroneously annotated as having a 'static lifetime) and then tried to move a mutable borrow of a field inside your closure.
I'm having trouble understanding why rust doesn't like my remove_str method in there:
use std::cell::RefCell;
use std::collections::HashSet;
#[derive(Hash, Eq, PartialEq)]
struct StringWrap<'a>{
s: &'a String,
}
struct Container<'a>{
m: HashSet<StringWrap<'a>>
}
impl<'a> Container<'a>{
fn remove_str(&mut self, s: &str){
let string = String::from(s);
let to_remove = StringWrap{s: &string};
self.m.remove(&to_remove);
}
}
It chokes with:
error[E0597]: `string` does not live long enough
--> tests/worksheet.rs:17:39
|
14 | impl<'a> Container<'a>{
| -- lifetime `'a` defined here
...
17 | let to_remove = StringWrap{s: &string};
| ^^^^^^^ borrowed value does not live long enough
18 | self.m.remove(&to_remove);
| ------------------------- argument requires that `string` is borrowed for `'a`
19 | }
| - `string` dropped here while still borrowed
As far as I can see, my string and to_remove live long enough to allow the .remove call to do its job. Is it because remove is potentially asynchronous or something like that?
Thanks for any help or insight!
As far as I can see, my string and to_remove live long enough to allow the .remove call to do its job. Is it because remove is potentially asynchronous or something like that?
No, it's because HashSet::remove must be called with something that the item becomes when borrowed:
pub fn remove<Q: ?Sized>(&mut self, value: &Q) -> bool
where
T: Borrow<Q>,
Q: Hash + Eq,
However, unless you manually implement Borrow for StringWrap, only the blanket reflexive implementation will apply—and thus remove can only be called with value of type &StringWrap<'a>. Note the lifetime requirement.
What you need to do to make this work is to implement Borrow for StringWrap. You could, for example, do the following:
impl Borrow<str> for StringWrap<'_> {
fn borrow(&self) -> &str {
self.s
}
}
and then Container::remove_str can merely forward its argument to HashMap::remove:
impl Container<'_> {
fn remove_str(&mut self, s: &str) {
self.m.remove(s);
}
}
See it on the playground.
All that said, it's rather unusual to store references in a HashSet: typically one would move ownership of the stored Strings into the set, which would render this problem moot as no lifetimes would be at play.
Here, I pass some mutable references into a function to perform some action on those. Then, I drop those mutable references by turning them into immutable references. However, the Rust borrow checker still seems to think they are mutable. Here's the code:
//! src/lib.rs
fn append_1_to_all(strings: Vec<&mut String>) -> Vec<&mut String> {
strings.into_iter().map(|s| { s.push_str("1"); s }).collect()
}
fn get_shared_references(strings: Vec<&mut String>) -> Vec<&String> {
strings.into_iter().map(|c| &(*c)).collect()
}
#[test]
fn test() {
let strings = vec!["one".to_string(), "two".to_string(), "three".to_string()];
let strings_appended = append_1_to_all(strings.iter_mut().collect());
let strings_shared = get_shared_references(strings_appended);
assert_ne!(strings[0], *strings_shared[0]);
}
Compile error:
error[E0502]: cannot borrow `strings` as immutable because it is also borrowed as mutable
--> src/lib.rs:16:16
|
12 | let strings_appended = append_1_to_all(strings.iter_mut().collect());
| ------- mutable borrow occurs here
...
16 | assert_ne!(strings[0], *strings_shared[0]);
| ^^^^^^^ -------------- mutable borrow later used here
| |
| immutable borrow occurs here
How can I fix this error? I would think strings_shared shouldn't be related to the mutable borrow anymore.
The problem is not that your reference is actually mutable, but that it's keeping the mutable reference alive.
If you have a reference, type &'a ... or &'a mut ..., and you produce another reference using it, whether that is a reborrow &*some_reference or a field access &some_struct_reference.some_field or a method or anything else, then that is always considered to be a borrow from that original reference, requiring the original reference to live as long as the derived one.
Notice that in your function declaration,
fn get_shared_references(strings: Vec<&mut String>) -> Vec<&String> {...}
if we change it to write out the elided lifetimes, we get this:
fn get_shared_references<'a>(strings: Vec<&'a mut String>) -> Vec<&'a String> {...}
In order to make this work, we'd have to be writing some other lifetime for the result, &'b String instead of `&'a String. But what would that lifetime be, and how is it constrained? It's got to last no longer than the value it refers to does, but be longer than the mutable reference. Rust's lifetime system doesn't have a way to express that. A near miss is:
fn get_shared_references<'m, 'i: 'm>(strings: Vec<&'m mut String>) -> Vec<&'i String> {...}
But even if you could implement a function with this type, it wouldn't be useful to the caller, because they don't know anything about the lifetime 'i other than "at least as long as 'm".
We could imagine a system where mutable references carry two lifetimes — let's suppose the syntax &'m mut 'i T, where 'm is the lifetime for which the reference is mutable, and 'i is (no longer than) the lifetime of the value. In that case, we could possibly have
fn consumes_mut_ref_produces_ref<'m, 'i>(x: &'m mut 'i i32) -> &'i i32 {...}
But a whole new set of lifetime rules would have to be designed to make this work, and it might not even be possible to do consistently. It's certainly not something today's Rust supports.
My recommendation, in the absence of further details on what you're trying to do, would be to make this code to work on owned values. Owned values can be passed around in the way you're trying to do — because taking them as function arguments transfers ownership, so there's no lifetimes to worry about, and it's much more possible to say: "my function consumes this and produces that," with no obligatory relationship.
fn append_1_to_all(mut strings: Vec<String>) -> Vec<String> {
for s in strings.iter_mut() {
s.push_str("1")
}
strings
}
fn get_shared_references(strings: &Vec<String>) -> Vec<&String> {
// Or even better, just use strings.iter() instead of this function
strings.iter().collect()
}
#[test]
fn test() {
let strings = vec!["one".to_string(), "two".to_string(), "three".to_string()];
// .clone() here is necessarily only because strings is used in the assert below
let strings_appended = append_1_to_all(strings.clone());
let strings_shared = get_shared_references(&strings_appended);
assert_ne!(strings[0], *strings_shared[0]);
}
I ran into an issue while trying to define and use a trait with methods that borrow self mutably.
Some context that might make it easier: I am working on a toy compiler, and the problem I was trying to solve was to define a trait for code nodes, which are either statements or expressions. The trait was meant to be used for traversing code mutably (for rewriting purposes). The abstraction I was trying to create was a "code node" that may have any number of children that are either statements or expressions. This is how it went:
// Actually these are enums with different payload types for different kinds of exprs/stmts,
// but this is not relevant.
struct Expression;
struct Statement;
trait CodeNode<'a>
where
Self::ExprIter: Iterator<Item = &'a mut Expression>,
Self::StmtIter: Iterator<Item = &'a mut Statement>,
{
type ExprIter;
type StmtIter;
fn child_exprs(&'a mut self) -> Self::ExprIter;
fn child_stmts(&'a mut self) -> Self::StmtIter;
}
This trait would be then implemented for quite a few types (I have a separate type for different kinds of statements and expressions).
The way I tried to use it was:
fn process<'a>(node: &'a mut impl CodeNode<'a>) {
for _stmt in node.child_stmts() {
// ...
}
for _expr in node.child_exprs() {
// ...
}
}
And this is where the problem lies. Rust compiler treats a call to node.child_stmts as a mutable borrow of node for the entire lifetime 'a, and so it does not allow a call to node.child_exprs later in the same function. Here is how the error looks:
error[E0499]: cannot borrow `*node` as mutable more than once at a time
--> src/main.rs:21:18
|
16 | fn process<'a>(node: &'a mut impl CodeNode<'a>) {
| -- lifetime `'a` defined here
17 | for _stmt in node.child_stmts() {
| ------------------
| |
| first mutable borrow occurs here
| argument requires that `*node` is borrowed for `'a`
...
21 | for _expr in node.child_exprs() {
| ^^^^ second mutable borrow occurs here
What I want to do is to somehow make compiler aware of the fact that node implements CodeNode<'a> for any lifetime parameter, and so it should use two separate lifetimes for two
calls, but I can't quite figure out a way to do it.
Any suggestions are welcome, I don't have a lot of experience with Rust, so maybe I am missing some more high-level solution to the original problem.
Your lifetime 'a is constrained by the CodeNode so both functions will be called with the same lifetime, but what you want are two lifetimes constrained by the two functions. So why not do something like this.
struct Expression;
struct Statement;
trait CodeNode
{
type ExprIter<'a> : Iterator<Item = &'a mut Expression>; //unstable
type StmtIter<'a> : Iterator<Item = &'a mut Statement>; //unstable
fn child_exprs<'a>(&'a mut self) -> Self::ExprIter<'a>;
fn child_stmts<'a>(&'a mut self) -> Self::StmtIter<'a>;
}
fn process(node: &mut impl CodeNode) {
for _stmt in node.child_stmts() {
// ...
}
for _expr in node.child_exprs() {
// ...
}
}
Unfortunately I had to use the unstable feature of generic associated types, but I believe this is what you want.
I also want to express that iterating over mutable references might not be a good idea and maybe you should change your program structure if that is possible.
EDIT:
#pretzelhammer proposed in the comments the following link which might be interesting: Solving the generalized streaming iterator problem without gats