I am calling closures with a fold function inside another closure. While I am intending for nothing in this some_closure function to live outside the closure environment, I am receiving an error that I am dropping a value while it is still borrowed.
I have tried removing all lifetime specifiers from some_closure, because I find the compiler is much smarter than myself at figuring out lifetimes, but I'm also finding no success in this (the compiler will always ask for lifetime specifiers leading up to the point of the shown example).
What I would desire to do here is to specify a lifetime restricted to the length of the closure inside the function, rather than the function itself. But I have a feeling that what I think is the problem may not actually be my problem, and that there is some gap in my understanding of lifetimes in closures.
I've tried to minimize the example as much as possible:
struct HoldStr<'a>(&'a str);
fn clone_slice_borrows_into_vec<'a>() -> impl Fn(&[&'a HoldStr]) -> Vec<&'a HoldStr<'a>> {
|slice| {
let mut temp = vec![];
temp.clone_from_slice(slice);
temp
}
}
fn clone_slice_borrows_into_vec_same<'a>() -> impl Fn(&[&'a HoldStr]) -> Vec<&'a HoldStr<'a>> {
// Same as last function for the sake of example, but one can assume it does something else
}
fn some_closure<'a>() -> impl Fn() {
|| {
let my_vec = vec![HoldStr("one"), HoldStr("two")];
let my_vec_holding_borrow: Vec<&'a HoldStr> = my_vec.iter().collect();
let called_closures: [Box<dyn Fn(&[&'a HoldStr]) -> Vec<&'a HoldStr<'a>>>; 2] = [
Box::new(clone_slice_borrows_into_vec()),
Box::new(clone_slice_borrows_into_vec_same())
];
let _result = called_closures
.iter()
.fold(my_vec_holding_borrow, |acc, closure| closure(&acc));
}
}
I would expect everything to be dropped by the end of the closure inside some_closure and for this to be fine, especially since I am specifying that lifetime 'a does not relate to anything the function itself returns. But it seems that the borrowed value is expected to live until the end of the function itself. I get this error:
error[E0597]: `my_vec` does not live long enough
--> src/lib.rs:61:51
|
## | fn some_closure<'a>() -> impl Fn() {
| -- lifetime `'a` defined here
...
## | let my_vec_holding_borrow: Vec<&'a HoldStr> = my_vec.iter().collect();
| ---------------- ^^^^^^ borrowed value does not live long enough
| |
| type annotation requires that `my_vec` is borrowed for `'a`
...
## | }
| - `my_vec` dropped here while still borrowed
I'd be happy to hear anything from how to resolve the error, to that I've been going about this the wrong way in the first place.
You need higher-rank trait bounds for your closure types:
fn clone_slice_borrows_into_vec() -> impl for<'a> Fn(&[&'a HoldStr]) -> Vec<&'a HoldStr<'a>> {
...
(Full code in the playground)
The lifetime 'a isn't fixed for your closure. It should return a vector of references with lifetime 'a for any input slice with references of this lifetime. Your code used an externally fixed lifetime instead, which could be chosen by the caller of clone_slice_borrows_into_vec().
If you have a funciton definition like
fn foo<'a>() -> &'a Foo
then it's basically always a mistake. This lets the caller request an arbitrary lifetime, and the function promises to create a reference of this lifetime out of thin air, which is only possible if it gets static references from some global storage, in which case it should simply return &'static Foo.
Related
I've encountered a confusing error about the use of a mutable and immutable borrow at the same time, after I expect the mutable borrow to end. I've done a lot of research on similar questions (1, 2, 3, 4, 5) which has led me to believe my problem has something to do with lexical lifetimes (though turning on the NLL feature and compiling on nightly doesn't change the result), I just have no idea what; my situation doesn't seem to fit into any of the scenarios of the other questions.
pub enum Chain<'a> {
Root {
value: String,
},
Child {
parent: &'a mut Chain<'a>,
},
}
impl Chain<'_> {
pub fn get(&self) -> &String {
match self {
Chain::Root { ref value } => value,
Chain::Child { ref parent } => parent.get(),
}
}
pub fn get_mut(&mut self) -> &mut String {
match self {
Chain::Root { ref mut value } => value,
Chain::Child { ref mut parent } => parent.get_mut(),
}
}
}
#[test]
fn test() {
let mut root = Chain::Root { value: "foo".to_string() };
{
let mut child = Chain::Child { parent: &mut root };
*child.get_mut() = "bar".to_string();
} // I expect child's borrow to go out of scope here
assert_eq!("bar".to_string(), *root.get());
}
playground
The error is:
error[E0502]: cannot borrow `root` as immutable because it is also borrowed as mutable
--> example.rs:36:36
|
31 | let mut child = Chain::Child { parent: &mut root };
| --------- mutable borrow occurs here
...
36 | assert_eq!("bar".to_string(), *root.get());
| ^^^^
| |
| immutable borrow occurs here
| mutable borrow later used here
I understand why an immutable borrow happens there, but I do not understand how a mutable borrow is used there. How can both be used at the same place? I'm hoping someone can explain what is happening and how I can avoid it.
In short, &'a mut Chain<'a> is extremely limiting and pervasive.
For an immutable reference &T<'a>, the compiler is allowed to shorten the lifetime of 'a when necessary to match other lifetimes or as part of NLL (this is not always the case, it depends on what T is). However, it cannot do so for mutable references &mut T<'a>, otherwise you could assign it a value with a shorter lifetime.
So when the compiler tries to reconcile the lifetimes when the reference and the parameter are linked &'a mut T<'a>, the lifetime of the reference is conceptually expanded to match the lifetime of the parameter. Which essentially means you've created a mutable borrow that will never be released.
Applying that knowledge to your question: creating a reference-based hierarchy is really only possible if the nested values are covariant over their lifetimes. Which excludes:
mutable references
trait objects
structs with interior mutability
Refer to these variations on the playground to see how these don't quite work as expected.
See also:
Why does linking lifetimes matter only with mutable references?
How do I implement the Chain of Responsibility pattern using a chain of trait objects?
What are the differences when getting an immutable reference from a mutable reference with self-linked lifetimes?
For fun, I'll include a case where the Rust standard library does this sort of thing on purpose. The signature of std::thread::scope looks like:
pub fn scope<'env, F, T>(f: F) -> T
where
F: for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T
The Scope that is provided to the user-defined function intentionally has its lifetimes tied in a knot to ensure it is only used in intended ways. This is not always the case since structs may be covariant or contravariant over their generic types, but Scope is defined to be invariant. Also, the only function that can be called on it is .spawn() which intentionally takes &'scope self as the self-parameter as well, ensuring that the reference does not have a shorter lifetime than what is given by scope.
Internally, the standard library contains this documentation (source):
Invariance over 'scope, to make sure 'scope cannot shrink, which is necessary for soundness.
Without invariance, this would compile fine but be unsound:
std::thread::scope(|s| {
s.spawn(|| {
let a = String::from("abcd");
s.spawn(|| println!("{a:?}")); // might run after `a` is dropped
});
});
Even if the lifetime of the reference is invariant with respect to itself, this still avoids many problems above because it uses an immutable reference and interior-mutability. If the parameter to .spawn() required &'scope mut self, then this would not work and run into the same problems above when trying to spawn more than one thread.
The issue isn't lexical lifetimes, and adding an explicit drop won't change the error. The issue is with the &'a mut Chain<'a>- that forces root to be borrowed for its entire life, rendering it useless after the borrow is dropped. As per the comment below, doing this with lifetimes is basically impossible. I would suggest using a box instead. Changing the struct to
pub enum Chain{
Root {
value: String,
},
Child {
parent: Box<Chain>,
},
}
and adjusting the other methods as necesary. Alternatively, using an Rc<RefCell<Chain>> if you want the original to remain usable without consuming self.
I've encountered a confusing error about the use of a mutable and immutable borrow at the same time, after I expect the mutable borrow to end. I've done a lot of research on similar questions (1, 2, 3, 4, 5) which has led me to believe my problem has something to do with lexical lifetimes (though turning on the NLL feature and compiling on nightly doesn't change the result), I just have no idea what; my situation doesn't seem to fit into any of the scenarios of the other questions.
pub enum Chain<'a> {
Root {
value: String,
},
Child {
parent: &'a mut Chain<'a>,
},
}
impl Chain<'_> {
pub fn get(&self) -> &String {
match self {
Chain::Root { ref value } => value,
Chain::Child { ref parent } => parent.get(),
}
}
pub fn get_mut(&mut self) -> &mut String {
match self {
Chain::Root { ref mut value } => value,
Chain::Child { ref mut parent } => parent.get_mut(),
}
}
}
#[test]
fn test() {
let mut root = Chain::Root { value: "foo".to_string() };
{
let mut child = Chain::Child { parent: &mut root };
*child.get_mut() = "bar".to_string();
} // I expect child's borrow to go out of scope here
assert_eq!("bar".to_string(), *root.get());
}
playground
The error is:
error[E0502]: cannot borrow `root` as immutable because it is also borrowed as mutable
--> example.rs:36:36
|
31 | let mut child = Chain::Child { parent: &mut root };
| --------- mutable borrow occurs here
...
36 | assert_eq!("bar".to_string(), *root.get());
| ^^^^
| |
| immutable borrow occurs here
| mutable borrow later used here
I understand why an immutable borrow happens there, but I do not understand how a mutable borrow is used there. How can both be used at the same place? I'm hoping someone can explain what is happening and how I can avoid it.
In short, &'a mut Chain<'a> is extremely limiting and pervasive.
For an immutable reference &T<'a>, the compiler is allowed to shorten the lifetime of 'a when necessary to match other lifetimes or as part of NLL (this is not always the case, it depends on what T is). However, it cannot do so for mutable references &mut T<'a>, otherwise you could assign it a value with a shorter lifetime.
So when the compiler tries to reconcile the lifetimes when the reference and the parameter are linked &'a mut T<'a>, the lifetime of the reference is conceptually expanded to match the lifetime of the parameter. Which essentially means you've created a mutable borrow that will never be released.
Applying that knowledge to your question: creating a reference-based hierarchy is really only possible if the nested values are covariant over their lifetimes. Which excludes:
mutable references
trait objects
structs with interior mutability
Refer to these variations on the playground to see how these don't quite work as expected.
See also:
Why does linking lifetimes matter only with mutable references?
How do I implement the Chain of Responsibility pattern using a chain of trait objects?
What are the differences when getting an immutable reference from a mutable reference with self-linked lifetimes?
For fun, I'll include a case where the Rust standard library does this sort of thing on purpose. The signature of std::thread::scope looks like:
pub fn scope<'env, F, T>(f: F) -> T
where
F: for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T
The Scope that is provided to the user-defined function intentionally has its lifetimes tied in a knot to ensure it is only used in intended ways. This is not always the case since structs may be covariant or contravariant over their generic types, but Scope is defined to be invariant. Also, the only function that can be called on it is .spawn() which intentionally takes &'scope self as the self-parameter as well, ensuring that the reference does not have a shorter lifetime than what is given by scope.
Internally, the standard library contains this documentation (source):
Invariance over 'scope, to make sure 'scope cannot shrink, which is necessary for soundness.
Without invariance, this would compile fine but be unsound:
std::thread::scope(|s| {
s.spawn(|| {
let a = String::from("abcd");
s.spawn(|| println!("{a:?}")); // might run after `a` is dropped
});
});
Even if the lifetime of the reference is invariant with respect to itself, this still avoids many problems above because it uses an immutable reference and interior-mutability. If the parameter to .spawn() required &'scope mut self, then this would not work and run into the same problems above when trying to spawn more than one thread.
The issue isn't lexical lifetimes, and adding an explicit drop won't change the error. The issue is with the &'a mut Chain<'a>- that forces root to be borrowed for its entire life, rendering it useless after the borrow is dropped. As per the comment below, doing this with lifetimes is basically impossible. I would suggest using a box instead. Changing the struct to
pub enum Chain{
Root {
value: String,
},
Child {
parent: Box<Chain>,
},
}
and adjusting the other methods as necesary. Alternatively, using an Rc<RefCell<Chain>> if you want the original to remain usable without consuming self.
I am writing a graphical program in Rust using the sdl2 crate from crates.io.
I have created this struct in main.rs:
struct GameObject {
name: &'static str,
pub texture: Option<Texture<'static>>
}
impl GameObject {
pub fn new(name: &'static str) -> Self {
Self { name, texture: None }
}
pub fn set_texture(&mut self, t: Option<Texture<'static>>) {
self.texture = t;
}
}
I am trying to add a texture to an instance of GameObject with the following lines of code (where canvas is the result of window.into_canvas().build().unwrap() on my main window):
let tc = canvas.texture_creator();
let mut g = GameObject::new("obj");
g.texture = Some(tc.load_texture("image.png").unwrap());
which causes this error:
error[E0597]: `tc` does not live long enough
--> src/main.rs:30:22
|
30 | g.texture = Some(tc.load_texture("image.png").unwrap());
| ^^---------------------------
| |
| borrowed value does not live long enough
| argument requires that `tc` is borrowed for `'static`
What does this mean in the context of my program?
Welcome to stack-overflow community.
Code with comments
To understand error you need to know what 'static lifetime is.
A 'static lifetime is the longest possible lifetime, and lasts for the lifetime of the running program.
A 'static lifetime may also be coerced to a shorter lifetime.
So your example with comments:
// create a `tc` vriable with local lifetime 'a
let tc = canvas.texture_creator();
// "obj" has a static lifetime , and g has a local lifetime 'b
let mut g = GameObject::new("obj");
// here you try to put a variable with lifetime 'c (returned by load_texture
// that is probably equal to 'a) into a variable `g.texture` that is declared
// to contain `'static` lifetime
// lifetime of "image.png" is `'static` on input of `load_texture()` function
// but could be shorter in implementation.
g.texture = Some(tc.load_texture("image.png").unwrap());
To make it work you could try use lazy-static to declare Texture with 'static lifetime.
Ask cargo to explain
If you see a compiler error - you can always ask cargo to explain it to you with cmd:
cargo --explain E0597
Other similar questions with answers:
borrowed value doesn't live long enough
borrowed value doesn't live long enough
I've worked down a real-life example in a web app, which I've solved using unnecessary heap allocation, to the following example:
// Try replacing with (_: &String)
fn make_debug<T>(_: T) -> impl std::fmt::Debug {
42u8
}
fn test() -> impl std::fmt::Debug {
let value = "value".to_string();
// try removing the ampersand to get this to compile
make_debug(&value)
}
pub fn main() {
println!("{:?}", test());
}
As is, compiling this code gives me:
error[E0597]: `value` does not live long enough
--> src/main.rs:9:16
|
5 | fn test() -> impl std::fmt::Debug {
| -------------------- opaque type requires that `value` is borrowed for `'static`
...
9 | make_debug(&value)
| ^^^^^^ borrowed value does not live long enough
10 | }
| - `value` dropped here while still borrowed
I can fix this error in at least two ways:
Instead of passing in a reference to value in test(), pass in value itself
Instead of the parameter T, explicitly state the type of the argument for make_debug as &String or &str
My understanding of what's happening is that, when there is a parameter, the borrow checker is assuming that any lifetime on that parameter affects the output impl Debug value.
Is there a way to keep the code parameterized, continue passing in a reference, and get the borrow checker to accept it?
I think this is due to the rules around how impl trait opaque types capture lifetimes.
If there are lifetimes inside an argument T, then an impl trait has to incorporate them. Additional lifetimes in the type signature follow the normal rules.
For more information please see:
https://github.com/rust-lang/rust/issues/43396#issuecomment-349716967
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#lifetime-parameters
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#assumption-3-there-should-be-an-explicit-marker-when-a-lifetime-could-be-embedded-in-a-return-type
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#scoping-for-type-and-lifetime-parameters
A more complete answer
Original goal: the send_form function takes an input parameter of type &T which is rendered to a binary representation. That binary representation is owned by the resulting impl Future, and no remnant of the original &T remains. Therefore, the lifetime of &T need not outlive the impl Trait. All good.
The problem arises when T itself, additionally, contains references with lifetimes. If we were not using impl Trait, our signature would look something like this:
fn send_form<T>(self, data: &T) -> SendFormFuture;
And by looking at SendFormFuture, we can readily observe that there is no remnant of T in there at all. Therefore, even if T has lifetimes of its own to deal with, we know that all references are used within the body of send_form, and never used again afterward by SendFormFuture.
However, with impl Future as the output, we get no such guarantees. There's no way to know if the concrete implementation of Future in fact holds onto the T.
In the case where T has no references, this still isn't a problem. Either the impl Future references the T, and fully takes ownership of it, or it doesn't reference it, and no lifetime issues arise.
However, if T does have references, you could end up in a situation where the concrete impl Future is holding onto a reference stored in the T. Even though the impl Future has ownership of the T itself, it doesn't have ownership of the values referenced by the T.
This is why the borrow check must be conservative, and insist that any references inside T must have a 'static lifetime.
The only workaround I can see is to bypass impl Future and be explicit in the return type. Then, you can demonstrate to the borrow checker quite easily that the output type does not reference the input T type at all, and any references in it are irrelevant.
The original code in the actix web client for send_form looks like:
https://docs.rs/awc/0.2.1/src/awc/request.rs.html#503-522
pub fn send_form<T: Serialize>(
self,
value: &T,
) -> impl Future<
Item = ClientResponse<impl Stream<Item = Bytes, Error = PayloadError>>,
Error = SendRequestError,
> {
let body = match serde_urlencoded::to_string(value) {
Ok(body) => body,
Err(e) => return Either::A(err(Error::from(e).into())),
};
// set content-type
let slf = self.set_header_if_none(
header::CONTENT_TYPE,
"application/x-www-form-urlencoded",
);
Either::B(slf.send_body(Body::Bytes(Bytes::from(body))))
}
You may need to patch the library or write your own function that does the same thing but with a concrete type. If anyone else knows how to deal with this apparent limitation of impl trait I'd love to hear it.
Here's how far I've gotten on a rewrite of send_form in awc (the actix-web client library):
pub fn send_form_alt<T: Serialize>(
self,
value: &T,
// ) -> impl Future<
// Item = ClientResponse<impl Stream<Item = Bytes, Error = PayloadError>>,
// Error = SendRequestError,
) -> Either<
FutureResult<String, actix_http::error::Error>,
impl Future<
Item = crate::response::ClientResponse<impl futures::stream::Stream>,
Error = SendRequestError,
>,
> {
Some caveats so far:
Either::B is necessarily an opaque impl trait of Future.
The first param of FutureResult might actually be Void or whatever the Void equivalent in Rust is called.
Why doesn't this code compile:
fn use_cursor(cursor: &mut io::Cursor<&mut Vec<u8>>) {
// do some work
}
fn take_reference(data: &mut Vec<u8>) {
{
let mut buf = io::Cursor::new(data);
use_cursor(&mut buf);
}
data.len();
}
fn produce_data() {
let mut data = Vec::new();
take_reference(&mut data);
data.len();
}
The error in this case is:
error[E0382]: use of moved value: `*data`
--> src/main.rs:14:5
|
9 | let mut buf = io::Cursor::new(data);
| ---- value moved here
...
14 | data.len();
| ^^^^ value used here after move
|
= note: move occurs because `data` has type `&mut std::vec::Vec<u8>`, which does not implement the `Copy` trait
The signature of io::Cursor::new is such that it takes ownership of its argument. In this case, the argument is a mutable reference to a Vec.
pub fn new(inner: T) -> Cursor<T>
It sort of makes sense to me; because Cursor::new takes ownership of its argument (and not a reference) we can't use that value later on. At the same time it doesn't make sense: we essentially only pass a mutable reference and the cursor goes out of scope afterwards anyway.
In the produce_data function we also pass a mutable reference to take_reference, and it doesn't produce a error when trying to use data again, unlike inside take_reference.
I found it possible to 'reclaim' the reference by using Cursor.into_inner(), but it feels a bit weird to do it manually, since in normal use-cases the borrow-checker is perfectly capable of doing it itself.
Is there a nicer solution to this problem than using .into_inner()? Maybe there's something else I don't understand about the borrow-checker?
Normally, when you pass a mutable reference to a function, the compiler implicitly performs a reborrow. This produces a new borrow with a shorter lifetime.
When the parameter is generic (and is not of the form &mut T), the compiler doesn't do this reborrowing automatically1. However, you can do it manually by dereferencing your existing mutable reference and then referencing it again:
fn take_reference(data: &mut Vec<u8>) {
{
let mut buf = io::Cursor::new(&mut *data);
use_cursor(&mut buf);
}
data.len();
}
1 — This is because the current compiler architecture only allows a chance to do a coercion if both the source and target types are known at the coercion site.