A beginner of rust. After I read chapter 4.3, I have confusion about the content of chapter 4.3 which has a cross-reference to the principle
At any given time, you can have either one mutable reference or any number of immutable references.
The simplified example is
fn main() {
let mut str: String = String::from("hello");
let slice: &str = &str[0..2]; // #1
str.clear(); // #2
println!("{}", slice);
}
This example results in an error when compiling it.
error[E0502]: cannot borrow `str` as mutable because it is also borrowed as immutable
--> src/main.rs:4:5
|
3 | let slice: &str = &str[0..2]; // #1
| --- immutable borrow occurs here
4 | str.clear(); // #2
| ^^^^^^^^^^^ mutable borrow occurs here
5 | println!("{}", slice);
| ----- immutable borrow later used here
The tutorial annotation says the reason is that it violates the principle above. However, I cannot understand it. In my mind, #1 creates an immutable reference with the type &str, instead, #2 makes a mutable reference with type &String, according to the type, they seem not to refer to the same things, since they have different reference types. Why does it violate the principle above that seems to only apply to the reference with the same type? Is there any principle that can clarify this issue?
I think you misunderstand.
String is not the mutable version of str. It's its own type.
let mut x: String is the mutable version of let x: String.
String is owned and can be modified.
str is a "slice" type and refers to the content of a string, either inside of a String, or as &'static str in the global memory.
There is no mut str because str by definition is a reference to an immutable part of a string.
Let's look at your code. (renamed str to s because this got too confusing)
fn main() {
// Your variable `s` is `mut String`. It is a mutable string.
let mut s: String = String::from("hello");
// Your variable `slice` is a `&str`.
// It isn't mutable, it is a reference to a substring of `s`.
let slice: &str = &s[0..2]; // #1
// Here we already hold an immutable reference to `s` through the `slice` variable.
// This prevents us from modifying `s`, because you cannot reference an object mutably while
// it is borrowed immutably.
s.clear(); // #2
// This line is only important to force the variable `slice` to exist.
// Otherwise the compiler would be allowed to drop it before the `s.clear()` call,
// and everything would compile fine.
println!("{}", slice);
}
There is no &String in there anywhere. Taking a slice of a String via &s[0..2] automatically creates a &str instead, because that's what the specification of String says:
fn index(&self, index: Range) -> &str
Why does it violate the principle above that seems to only apply to the reference with the same type?
This is incorrect. They do not have to be the same type. If you hold a &str that references the content of a String, then the String object is also blocked from being mutated while the &str reference exists. You can even store references in other objects and then the existance of those objects still block the original String.
They are definitely different objects
This doesn't mean that they can't be connected.
To demonstrate that two objects of different types can have connected lifetimes, look at the following code:
#[derive(Debug)]
struct A {
pub value: u32,
}
#[derive(Debug)]
struct B<'a> {
pub reference: &'a u32,
}
impl A {
pub fn new(value: u32) -> Self {
Self { value }
}
pub fn set(&mut self, value: u32) {
self.value = value;
}
}
impl<'a> B<'a> {
pub fn new(a: &'a A) -> Self {
Self {
reference: &a.value,
}
}
}
fn main() {
let mut a = A::new(69);
println!("a: {:?}", a);
// Can be modified
a.set(42);
println!("a: {:?}", a);
// Create a B object that references the content of `a`
let b = B::new(&a);
println!("b: {:?}", b);
// While `b exists, it borrows a part of `a` (indicated through the fact that it has a lifetime type attached)
// That means, while `b` exists, `a` cannot be modified
a.set(420); // FAILS
// This ensures that `b` actually still exists
println!("b: {:?}", b);
}
The error message is quite clear:
error[E0502]: cannot borrow `a` as mutable because it is also borrowed as immutable
--> src/main.rs:43:5
|
38 | let b = B::new(&a);
| -- immutable borrow occurs here
...
43 | a.set(420); // FAILS
| ^^^^^^^^^^ mutable borrow occurs here
...
46 | println!("b: {:?}", b);
| - immutable borrow later used here
Note that the B type has a lifetime 'a attached. This lifetime will automatically be derived by the compiler upon instantiation and is used to prevent mutable usage of the referenced A object for as long as B exists.
&str also has a lifetime attached that is used to prevent mutable access of the referenced String object.
Related
This is a simplified example of something I've encountered:
trait Bla<'a> {
fn create(a: &'a Foo) -> Self;
fn consume(self) -> u8;
}
struct Foo;
impl Foo {
fn take(&mut self, u: u8) {}
}
struct Bar {
foo: Foo,
}
impl Bar {
fn foobar<'a, 'b, B: Bla<'b>>(&'a mut self)
where 'a: 'b {
let u = {
// immutable borrow occurs here
// type annotation requires that `self.foo` is borrowed for `'b`
let foo: &'b Foo = &self.foo;
let b = B::create(foo);
b.consume()
};
// error[E0502]: cannot borrow `self.foo` as mutable because it is also borrowed as immutable
self.foo.take(u);
}
}
Why is self.foo still considered to be borrowed even after exiting the block it was borrowed on, with everything that used the borrow also dropped?
The data might flow to another place.
To understand why the compiler is correct that the immutable borrow might still exist, consider the following implementation of Bla that is valid:
#![feature(once_cell)]
// The data stored in this global static variable exists for the entirety of runtime.
static REFS: SyncLazy<Mutex<Vec<&'static Foo>>> = SyncLazy::new(|| Mutex::new(vec![]));
struct Bad;
impl Bla<'static> for Bad {
fn create(a: &'static Foo) -> Self {
// The reference can go somewhere other than `Self`!
REFS.lock().unwrap().push(a);
Bad
}
fn consume(self) -> u8 {
// Even if `self` is consumed here,
// `self` doesn't necessarily destory the immutable borrow.
0
}
}
When your generic foobar function accepts any valid implementation of Bla, a valid implementation might take a &'static borrow that guarantees the borrow to last for the entire runtime. Therefore, it is an error because consume does not guarantee that the immutable borrow goes away.
You are probably looking for the for<> syntax as that will allow the Bad implementation to exist but will disallow anyone from calling foobar with Bad:
impl Bar {
fn foobar<B: for<'b> Bla<'b>>(&mut self) {
let u = {
let foo: &Foo = &self.foo;
let b = B::create(foo);
b.consume()
};
self.foo.take(u);
}
}
fn main() {
Bar::foobar::<Bad>(&mut Bar { foo: Foo })
// ^ error: implementation of `Bla` is not general enough
// | `Bad` must implement `Bla<'0>`, for any lifetime `'0`...
// | ...but it actually implements `Bla<'static>`
}
If this isn't the case, you might have to provide more implementation details rather than an example causing the diagnostic error.
Normally, when you don't specify your own lifetime, the lifetime of an immutable borrow to a variable ends when the variable goes out of scope. But in your code, you are specifying an explicit lifetime 'b of the immutable borrow.
To the borrow checker, the only thing known about the lifetime 'b is that it is a subset of 'a (due to where 'a: 'b). In particular, the borrow checker will not assume that the scope of the immutable borrow ends at }, because nothing of that sort is specified about 'b.
The compiler actually says this out loud:
// type annotation requires that self.foo is borrowed for 'b
Summary
Why is self.foo still considered to be borrowed even after exiting the block it was borrowed on, with everything that used the borrow also dropped?
Because the lifetime you specify does not end when the block ends. The borrow checker does not care about that some variables related to the borrow goes out of scope.
Here, I pass some mutable references into a function to perform some action on those. Then, I drop those mutable references by turning them into immutable references. However, the Rust borrow checker still seems to think they are mutable. Here's the code:
//! src/lib.rs
fn append_1_to_all(strings: Vec<&mut String>) -> Vec<&mut String> {
strings.into_iter().map(|s| { s.push_str("1"); s }).collect()
}
fn get_shared_references(strings: Vec<&mut String>) -> Vec<&String> {
strings.into_iter().map(|c| &(*c)).collect()
}
#[test]
fn test() {
let strings = vec!["one".to_string(), "two".to_string(), "three".to_string()];
let strings_appended = append_1_to_all(strings.iter_mut().collect());
let strings_shared = get_shared_references(strings_appended);
assert_ne!(strings[0], *strings_shared[0]);
}
Compile error:
error[E0502]: cannot borrow `strings` as immutable because it is also borrowed as mutable
--> src/lib.rs:16:16
|
12 | let strings_appended = append_1_to_all(strings.iter_mut().collect());
| ------- mutable borrow occurs here
...
16 | assert_ne!(strings[0], *strings_shared[0]);
| ^^^^^^^ -------------- mutable borrow later used here
| |
| immutable borrow occurs here
How can I fix this error? I would think strings_shared shouldn't be related to the mutable borrow anymore.
The problem is not that your reference is actually mutable, but that it's keeping the mutable reference alive.
If you have a reference, type &'a ... or &'a mut ..., and you produce another reference using it, whether that is a reborrow &*some_reference or a field access &some_struct_reference.some_field or a method or anything else, then that is always considered to be a borrow from that original reference, requiring the original reference to live as long as the derived one.
Notice that in your function declaration,
fn get_shared_references(strings: Vec<&mut String>) -> Vec<&String> {...}
if we change it to write out the elided lifetimes, we get this:
fn get_shared_references<'a>(strings: Vec<&'a mut String>) -> Vec<&'a String> {...}
In order to make this work, we'd have to be writing some other lifetime for the result, &'b String instead of `&'a String. But what would that lifetime be, and how is it constrained? It's got to last no longer than the value it refers to does, but be longer than the mutable reference. Rust's lifetime system doesn't have a way to express that. A near miss is:
fn get_shared_references<'m, 'i: 'm>(strings: Vec<&'m mut String>) -> Vec<&'i String> {...}
But even if you could implement a function with this type, it wouldn't be useful to the caller, because they don't know anything about the lifetime 'i other than "at least as long as 'm".
We could imagine a system where mutable references carry two lifetimes — let's suppose the syntax &'m mut 'i T, where 'm is the lifetime for which the reference is mutable, and 'i is (no longer than) the lifetime of the value. In that case, we could possibly have
fn consumes_mut_ref_produces_ref<'m, 'i>(x: &'m mut 'i i32) -> &'i i32 {...}
But a whole new set of lifetime rules would have to be designed to make this work, and it might not even be possible to do consistently. It's certainly not something today's Rust supports.
My recommendation, in the absence of further details on what you're trying to do, would be to make this code to work on owned values. Owned values can be passed around in the way you're trying to do — because taking them as function arguments transfers ownership, so there's no lifetimes to worry about, and it's much more possible to say: "my function consumes this and produces that," with no obligatory relationship.
fn append_1_to_all(mut strings: Vec<String>) -> Vec<String> {
for s in strings.iter_mut() {
s.push_str("1")
}
strings
}
fn get_shared_references(strings: &Vec<String>) -> Vec<&String> {
// Or even better, just use strings.iter() instead of this function
strings.iter().collect()
}
#[test]
fn test() {
let strings = vec!["one".to_string(), "two".to_string(), "three".to_string()];
// .clone() here is necessarily only because strings is used in the assert below
let strings_appended = append_1_to_all(strings.clone());
let strings_shared = get_shared_references(&strings_appended);
assert_ne!(strings[0], *strings_shared[0]);
}
Goal
Create function/macro which has an api like such:
fn writesperse(
buf: &mut String,
items: impl IntoIterator<Item=impl fmt::Display>,
sep: impl fmt::Display,
) -> fmt::Result {
// intersperse impl elided
}
with the main consumer of this api being a struct similar to:
use std::fmt;
// Drives building of query
struct QueryBuilder<'a> {
buf: String,
data: &'a Data,
state: State,
}
impl<'a> QueryBuilder<'a> {
// example method showing how writesperse might be used
fn f(&mut self) -> fmt::Result {
writesperse(
&mut self.buf,
self.data.names().map(|n| self.state.resolve(n)),
", ",
)
}
}
// Represents mutable container for computed values
struct State;
impl State {
fn resolve(&mut self, _name: &str) -> &StateRef {
// mutate state if name has not been seen before (elided)
&StateRef
}
}
// Represents example computed value
struct StateRef;
impl fmt::Display for StateRef {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "STATEREF")
}
}
// Immutable container with various collections of objects
struct Data;
impl Data {
// example iterator of references to some owned data
fn names(&self) -> impl Iterator<Item=&str> {
::std::iter::once("name")
}
// another iterator of a different references
fn items(&self) -> impl Iterator<Item=&DataRef> {
::std::iter::once(&DataRef)
}
}
// Represents some type Data might own
struct DataRef;
Error
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/lib.rs:13:50
|
13 | self.data.names().map(|n| self.state.resolve(n)),
| ^^^^^^^
|
note: first, the lifetime cannot outlive the lifetime '_ as defined on the body at 13:35...
--> src/lib.rs:13:35
|
13 | self.data.names().map(|n| self.state.resolve(n)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:13:39
|
13 | self.data.names().map(|n| self.state.resolve(n)),
| ^^^^^^^^^^
note: but, the lifetime must be valid for the method call at 13:13...
--> src/lib.rs:13:13
|
13 | self.data.names().map(|n| self.state.resolve(n)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that a type/lifetime parameter is in scope here
--> src/lib.rs:13:13
|
13 | self.data.names().map(|n| self.state.resolve(n)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
What I've tried
My only success thus far has been to manually do the intersperse logic at each site where it is needed.
let mut iter = some_iter.into_iter();
if let Some(i) = iter.next() {
// do any state mutation here so mutable reference is released
let n = self.state.resolve(n);
write!(&mut self.buf, "{}", n)?;
}
for i in iter {
// do same thing above
}
If I try and make State::resolve immutable, (which means I would need to pre-compute the values which is not desirable), I get a different error.
error[E0502]: cannot borrow `self` as immutable because it is also borrowed as mutable
--> src/lib.rs:13:35
|
11 | writesperse(
| ----------- mutable borrow later used by call
12 | &mut self.buf,
| ------------- mutable borrow occurs here
13 | self.data.names().map(|n| self.state.resolve(n)),
| ^^^ ---- second borrow occurs due to use of `self` in closure
| |
| immutable borrow occurs here
This error is easier to understand. However, I don't understand why what I am trying to do is disallowed. Why can I not hand out a mutable reference to QueryBuilder's buf and an iterator of references to objects within State and/or Data at the same time?
Ultimately, my number one priority is abstracting the intersperse logic into some function or macro which expects an Iterator<Item=fmt::Display>. It would be an added bonus if this Iterator could possibly mutate state and return a reference to its data. I don't think this is possible though, at least from my understanding of the streaming-iterator crate.
Thanks for your help!
writesperse is not the problem here, resolve is.
Because it takes &mut self and returns a reference with a lifetime bound to self, you can't call resolve a second time unless the reference obtained from the first call has been dropped. You can see this in this simplified f (compiler error interspersed):
// error[E0499]: cannot borrow `self.state` as mutable more than once at a time
fn f(&mut self) {
let a = self.state.resolve("alec");
// ---------- first mutable borrow occurs here
let _b = self.state.resolve("brian");
// ^^^^^^^^^^ second mutable borrow occurs here
println!("{}", a);
// - first borrow later used here
}
Part of the contract of an Iterator is that it does not yield internal references. So |n| self.state.resolve(n) is simply not a closure that can be passed to Iterator::map.
Fixing resolve
If resolve took &self instead of &mut self, this would work because the closure would not need to borrow self.state exclusively; it could return references with the lifetime of the original without worrying about overlap. So let's try that:
fn resolve(&self, _name: &str) -> &StateRef {
// some kind of interior mutability thing here, probably have to return
// `std::cell:Ref<StateRef>` or `MutexGuard<StateRef>` instead, but that
// doesn't matter for this demonstration
&StateRef
}
Oh dear.
error[E0502]: cannot borrow `self` as immutable because it is also borrowed as mutable
--> src/lib.rs:23:35
|
21 | writesperse(
| ----------- mutable borrow later used by call
22 | &mut self.buf,
| ------------- mutable borrow occurs here
23 | self.data.names().map(|n| self.state.resolve(n)),
| ^^^ ---- second borrow occurs due to use of `self` in closure
| |
| immutable borrow occurs here
What's going on? Because the closure uses self in a context where an immutable reference is required, the closure borrows self, and the &mut self.buf also borrows (part of) self mutably, and they both have to exist at the same time to pass into writesperse. So this looks like a dead end, but it's actually really close: it just needs a small change to f to compile.
fn f(&mut self) -> fmt::Result {
let state = &self.state;
writesperse(
&mut self.buf,
self.data.names().map(|n| state.resolve(n)),
", ",
)
}
The compiler can reason about mutually exclusive partial borrows as long as they happen all within the body of a single function. The compiler will let you borrow self.state and mutably borrow self.buf at the same time as long as the borrows happen in the same function. Creating a variable to borrow self.state makes it so that only state is captured by the closure, not self.
Other options
The above works, if you're able to make resolve take &self. Here are some other ideas:
You could attack the problem from the other direction: make resolve return StateRef by value so it doesn't extend the &mut borrow.
You can achieve the same thing by altering the closure so that it doesn't return a reference; |n| state.resolve(n).to_string() works with no further changes (but it does do a bunch of unnecessary allocation and copying).
You could use Arc or Rc instead of & references and defer all lifetime management to runtime.
You could write a macro that does the repetitive part for you so that writesperse is not necessary.
See also
How do I return a reference to something inside a RefCell without breaking encapsulation? if you need to use interior mutability to make resolve take &self
Mutable borrow of self doesn't change to immutable explains why state is still considered "mutably borrowed" after resolve returns
I want to write a function A which takes as parameter a function B which takes as parameter a type which is parameterized by a reference type which lifetime is at least the lifetime of the local variables in A’s body.
Consider the following example:
struct Foo {}
fn consume(mut v: Vec<&Foo>) {
while let Some(..) = v.pop() {
// Do stuff
continue;
}
}
fn setup_and<F>(f: F)
where
F: FnOnce(&mut Vec<&Foo>) + Send,
{
let mut v: Vec<&Foo> = vec![];
let other_foo = Foo {};
f(&mut v);
v.push(&other_foo);
consume(v);
}
fn main() {
let foo = Foo {};
setup_and(|v| {
v.push(&foo);
});
}
rustc cannot infer lifetimes on its own. It complains:
error[E0597]: `foo` does not live long enough
--> src/main.rs:25:17
|
24 | setup_and(|v| {
| --- value captured here
25 | v.push(&foo);
| --------^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `foo` is borrowed for `'static`
26 | });
27 | }
| - `foo` dropped here while still borrowed
I tried to specify a lifetime for the reference taken by setup_and like so:
fn setup_and<'a, F>(f: F)
where
F: FnOnce(&mut Vec<&'a Foo>) + Send,
{
let mut v: Vec<&'a Foo> = vec![];
Now rustc complains about the setup_and local reference other_foo not living long enough. I assume it is because it wants a larger lifetime than the scope of setup_and.
How would I bind lifetimes correctly in that case ? I would like to express that the references must be valid until the end of the consume call.
You have a serious, serious issue with conflicting lifetimes in your implementation, and there is no simple fix without at least a partial redesign of the outside signatures of your struct and methods. They all stem from the setup_and method, and are highlighted by the compiler when you explicitly described the lifetime bounds.
The body of your method is copied below, with annotations for you to understand the issue:
let mut v: Vec<&Foo> = vec![];
let other_foo = Foo {}; // other_foo is created here
f(&mut v);
v.push(&other_foo); // other_foo requires lifetime 'a to be added to this
consume(v); // consume does not restrict the lifetime requirement 'a
// other_foo is dropped here, at lifetime less than 'a
The easiest solution to this problem is to store an Arc<Foo>, like so (playground):
fn setup_and<F>(f: F)
where
F: FnOnce(&mut Vec<Arc<Foo>>) + Send,
{
let mut v: Vec<Arc<Foo>> = vec![];
let other_foo = Foo {};
f(&mut v);
v.push(Arc::new(other_foo));
consume(&mut v);
}
Arc is an Atomic Reference-Counting pointer. It is a clonable structure that works as a dynamic pointer to an object on the heap; for all intents and purposes, it works as a read-only reference, without the requirement for a lifetime. When all copies of an Arc are dropped, the item inside it is also dropped.
This solves two problems:
Your other_foo is now moved into the Arc, and no longer causes its lifetime issue
You can now access your objects just like you would a reference (Arc implements Deref)
The choice of Arc was made because your FnOnce requires Send, which Rc (the single-threaded variant of Arc) cannot provide.
fn func_vecmut<'a>(v: &'a mut Vec<&'a i32>, s: &'a String) {}
fn func_vecmut_fresh_lf<'a, 'b>(v: &'a mut Vec<&'a i32>, s: &'b String) {}
fn func_vec<'a>(v: &'a Vec<&'a i32>, s: &'a String) {}
fn main() {
let mut v = vec![];
{
let s: String = String::from("abc");
/* Note: Below, only one function call is active at a time, other two commented out */
func_vecmut(&mut v, &s); // Understandably, compiler fails this.
func_vecmut_fresh_lf(&mut v, &s); // Understandably, compiler passes this i.e with fresh lifetime for String.
func_vec(&v, &s); // Why does compiler pass this?
}
}
In func_vecmut (as I understand it), the compiler sees that the lifetime of String is same as Vec and the elements (i32) that it holds. And since v lives longer (defined outside the block), it extends the borrow to s beyond the scope of the block (i.e afterlife for s) and hence the following error:
error[E0597]: `s` does not live long enough
--> src/main.rs:13:30
|
13 | func_vecmut(&mut v, &s); // Understandably, compiler fails this.
| ^ borrowed value does not live long enough
...
16 | }
| - `s` dropped here while still borrowed
17 | }
| - borrowed value needs to live until here
The issue can be solved by giving a fresh lifetime to the String argument (see func_vecmut_fresh_lf).
But then, why does the compilation not fail for func_vec(&v, &s)? The only difference being &mut Vec. Is this something to do with the fact that immutable references are Copy, whereas mutable ones are not? If it is, how? The function bodies are empty, what does the compiler infer here?