I recently had an error which was simply resolved by changing
impl<'a> Foo<'a> {
fn foo(&'a self, path: &str) -> Boo<'a> { /* */ }
}
to
impl<'a> Foo<'a> {
fn foo(&self, path: &str) -> Boo { /* */ }
}
which did not make sense according to my understanding, as I thought that the second version is exactly the same as the first with applied lifetime elision.
In case we introduce a new lifetime for the method this seems to be the case according this example from the nomicon.
fn get_mut(&mut self) -> &mut T; // elided
fn get_mut<'a>(&'a mut self) -> &'a mut T; // expanded
So what are the differences between this and my first code snipped.
Lifetime 'a in fn foo(&'a self, ...) ... is defined for impl<'a>, that is it is the same for all foo calls.
Lifetime 'a in fn get_mut<'a>(&'a mut self) ... is defined for the function. Different calls of get_mut can have different values for 'a.
Your code
impl<'a> Foo<'a> {
fn foo(&'a self, path: &str) -> Boo<'a> { /* */ }
}
is not the expansion of elided lifetime. This code ties lifetime of borrow &'a self to the lifetime of structure Foo<'a>. If Foo<'a> is invariant over 'a, then self should remain borrowed as long as 'a.
Correct expansion of elided lifetime is
impl<'a> Foo<'a> {
fn foo<'b>(&'b self, path: &str) -> Boo<'b> { /* */ }
}
This code doesn't depend on variance of structure Foo to be able to borrow self for shorter lifetimes.
Example of differences between variant and invariant structures.
use std::cell::Cell;
struct Variant<'a>(&'a u32);
struct Invariant<'a>(Cell<&'a u32>);
impl<'a> Variant<'a> {
fn foo(&'a self) -> &'a u32 {
self.0
}
}
impl<'a> Invariant<'a> {
fn foo(&'a self) -> &'a u32 {
self.0.get()
}
}
fn main() {
let val = 0;
let mut variant = Variant(&val);// variant: Variant<'long>
let mut invariant = Invariant(Cell::new(&val));// invariant: Invariant<'long>
{
let r = variant.foo();
// Pseudocode to explain what happens here
// let r: &'short u32 = Variant::<'short>::foo(&'short variant);
// Borrow of `variant` ends here, as it was borrowed for `'short` lifetime
// Compiler can do this conversion, because `Variant<'long>` is
// subtype of Variant<'short> and `&T` is variant over `T`
// thus `variant` of type `Variant<'long>` can be passed into the function
// Variant::<'short>::foo(&'short Variant<'short>)
}
// variant is not borrowed here
variant = Variant(&val);
{
let r = invariant.foo();
// compiler can't shorten lifetime of `Invariant`
// thus `invariant` is borrowed for `'long` lifetime
}
// Error. invariant is still borrowed here
//invariant = Invariant(Cell::new(&val));
}
Playground link
Related
I am having trouble figuring out what lifetime parameter will work for this, so my current workarounds include transmutes or raw pointers. I have a structure holding a function pointer with a generic as a parameter:
struct CB<Data> {
cb: fn(Data) -> usize
}
I would like to store an instance of that, parameterized by some type containing a reference, in some other structure that implements a trait with one method, and use that trait method to call the function pointer in CB.
struct Holder<'a> {
c: CB<Option<&'a usize>>
}
trait Exec {
fn exec(&self, v: &usize) -> usize;
}
impl<'a> Holder<'a> {
fn exec_aux(&self, v: &'a usize) -> usize {
(self.c.cb)(Some(v))
}
}
impl<'a> Exec for Holder<'a> {
fn exec(&self, v: &usize) -> usize
{
self.exec_aux(v)
}
}
This gives me a lifetime error for the 'Exec' impl of Holder:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
Simply calling exec_aux works fine as long as I don't define that Exec impl:
fn main() {
let h = Holder { c: CB{cb:cbf}};
let v = 12;
println!("{}", h.exec_aux(&v));
}
Also, making CB not generic also makes this work:
struct CB {
cb: fn(Option<&usize>) -> usize
}
The parameter in my actual code is not a usize but something big that I would rather not copy.
The lifetimes in your Exec trait are implicitly this:
trait Exec {
fn exec<'s, 'a>(&'s self, v: &'a usize) -> usize;
}
In other words, types that implement Exec need to accept any lifetimes 's and 'a. However, your Holder::exec_aux method expects a specific lifetime 'a that's tied to the lifetime parameter of the Holder type.
To make this work, you need to add 'a as a lifetime parameter to the Exec trait instead, so that you can implement the trait specifically for that lifetime:
trait Exec<'a> {
// ^^^^ vv
fn exec(&self, v: &'a usize) -> usize;
}
impl<'a> Exec<'a> for Holder<'a> {
// ^^^^ vv
fn exec(&self, v: &'a usize) -> usize
{
self.exec_aux(v)
}
}
The problem here is that the Exec trait is too generic to be used in this way by Holder. First, consider the definition:
trait Exec {
fn exec(&self, v: &usize) -> usize;
}
This definition will cause the compiler to automatically assign two anonymous lifetimes for &self and &v in exec. It's basically the same as
fn exec<'a, 'b>(&'a self, v: &'b usize) -> usize;
Note that there is no restriction on who needs to outlive whom, the references just need to be alive for the duration of the method call.
Now consider the definition
impl<'a> Holder<'a> {
fn exec_aux(&self, v: &'a usize) -> usize {
// ... doesn't matter
}
}
Since we know that &self is a &Holder<'a> (this is what the impl refers to), we need to have at least a &'a Holder<'a> here, because &'_ self can't have a lifetime shorter than 'a in Holder<'a>. So this is saying that the two parameters have the same lifetime: &'a self, &'a usize.
Where it all goes wrong is when you try to combine the two. The trait forces you into the following signature, which (again) has two distinct implicit lifetimes. But the actual Holder which you then try to call a method on forces you to have the same lifetimes for &self and &v.
fn exec(&self, v: &usize) -> usize {
// Holder<'a> needs `v` to be `'a` when calling exec_aux
// But the trait doesn't say so.
self.exec_aux(v)
}
One solution is to redefine the trait as
trait Exec<'a> {
fn exec(&'a self, v: &'a usize) -> usize;
}
and then implement it as
impl<'a> Exec<'a> for Holder<'a> {
fn exec(&'a self, v: &'a usize) -> usize {
self.exec_aux(v)
}
}
I recently had an error which was simply resolved by changing
impl<'a> Foo<'a> {
fn foo(&'a self, path: &str) -> Boo<'a> { /* */ }
}
to
impl<'a> Foo<'a> {
fn foo(&self, path: &str) -> Boo { /* */ }
}
which did not make sense according to my understanding, as I thought that the second version is exactly the same as the first with applied lifetime elision.
In case we introduce a new lifetime for the method this seems to be the case according this example from the nomicon.
fn get_mut(&mut self) -> &mut T; // elided
fn get_mut<'a>(&'a mut self) -> &'a mut T; // expanded
So what are the differences between this and my first code snipped.
Lifetime 'a in fn foo(&'a self, ...) ... is defined for impl<'a>, that is it is the same for all foo calls.
Lifetime 'a in fn get_mut<'a>(&'a mut self) ... is defined for the function. Different calls of get_mut can have different values for 'a.
Your code
impl<'a> Foo<'a> {
fn foo(&'a self, path: &str) -> Boo<'a> { /* */ }
}
is not the expansion of elided lifetime. This code ties lifetime of borrow &'a self to the lifetime of structure Foo<'a>. If Foo<'a> is invariant over 'a, then self should remain borrowed as long as 'a.
Correct expansion of elided lifetime is
impl<'a> Foo<'a> {
fn foo<'b>(&'b self, path: &str) -> Boo<'b> { /* */ }
}
This code doesn't depend on variance of structure Foo to be able to borrow self for shorter lifetimes.
Example of differences between variant and invariant structures.
use std::cell::Cell;
struct Variant<'a>(&'a u32);
struct Invariant<'a>(Cell<&'a u32>);
impl<'a> Variant<'a> {
fn foo(&'a self) -> &'a u32 {
self.0
}
}
impl<'a> Invariant<'a> {
fn foo(&'a self) -> &'a u32 {
self.0.get()
}
}
fn main() {
let val = 0;
let mut variant = Variant(&val);// variant: Variant<'long>
let mut invariant = Invariant(Cell::new(&val));// invariant: Invariant<'long>
{
let r = variant.foo();
// Pseudocode to explain what happens here
// let r: &'short u32 = Variant::<'short>::foo(&'short variant);
// Borrow of `variant` ends here, as it was borrowed for `'short` lifetime
// Compiler can do this conversion, because `Variant<'long>` is
// subtype of Variant<'short> and `&T` is variant over `T`
// thus `variant` of type `Variant<'long>` can be passed into the function
// Variant::<'short>::foo(&'short Variant<'short>)
}
// variant is not borrowed here
variant = Variant(&val);
{
let r = invariant.foo();
// compiler can't shorten lifetime of `Invariant`
// thus `invariant` is borrowed for `'long` lifetime
}
// Error. invariant is still borrowed here
//invariant = Invariant(Cell::new(&val));
}
Playground link
I have code that works, but it stops compiling with a borrow checker error after a change. I don't understand how the change could affect borrow checking.
Common part to both working and non-working code:
/// Some struct that has references inside
#[derive(Debug)]
struct MyValue<'a> {
number: &'a u32,
}
/// There are many structs similar to `MyValue` and there is a
/// trait common to them all that can create them. In this
/// example I use the `From` trait.
impl<'a> From<&'a u32> for MyValue<'a> {
fn from(value: &'a u32) -> Self {
MyValue { number: value }
}
}
/// `Producer` makes objects that hold references into it. So
/// the produced object must be first dropped before any new
/// one can be made.
trait Producer<'a, T: 'a> {
fn make(&'a mut self) -> T;
}
Here is the working code:
struct MyProducer {
number: u32,
}
impl MyProducer {
fn new() -> Self {
Self { number: 0 }
}
}
impl<'a, T: 'a + From<&'a u32>> Producer<'a, T> for MyProducer {
fn make(&'a mut self) -> T {
self.number += 1;
T::from(&self.number)
}
}
fn main() {
let mut producer = MyProducer::new();
println!(
"made this: {:?}",
<MyProducer as Producer<MyValue>>::make(&mut producer)
);
println!(
"made this: {:?}",
<MyProducer as Producer<MyValue>>::make(&mut producer)
);
}
This compiles and prints the expected output:
made this: MyValue { number: 1 }
made this: MyValue { number: 2 }
I don't like that MyProducer actually implements Producer for every T as it makes it impossible to call make directly on it. I would like to have a type that is a MyProducer for a specific T (for example for MyValue).
To achieve this, I want to add a generic parameter to MyProducer. Because the MyProducer does not really use the T, I use PhantomData to prevent the compiler from complaining.
Here is the code after changes:
use std::marker::PhantomData;
struct MyProducer<'a, T: 'a + From<&'a u32>> {
number: u32,
_phantom: PhantomData<&'a T>,
}
impl<'a, T: 'a + From<&'a u32>> MyProducer<'a, T> {
fn new() -> Self {
Self {
number: 0,
_phantom: PhantomData::default(),
}
}
}
impl<'a, T: From<&'a u32>> Producer<'a, T> for MyProducer<'a, T> {
fn make(&'a mut self) -> T {
self.number += 1;
T::from(&self.number)
}
}
fn main() {
let mut producer = MyProducer::<MyValue>::new();
println!("made this: {:?}", producer.make());
println!("made this: {:?}", producer.make());
}
The main function now looks exactly as I would like it to look like. But the code does not compile. This is the error:
error[E0499]: cannot borrow `producer` as mutable more than once at a time
--> src/main.rs:50:33
|
49 | println!("made this: {:?}", producer.make());
| -------- first mutable borrow occurs here
50 | println!("made this: {:?}", producer.make());
| ^^^^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
I don't understand why it no longer works. The produced object is still dropped before the next one is made.
If I call the make function just once, it compiles and works.
I am using edition 2018, so NLL is active.
Rust Playground: working version before change
Rust Playground: broken version after change
I reduced the noise from the code so the following is an even shorter version of the broken case which demonstrates the same problem: (test in the playground)
use std::marker::PhantomData;
#[derive(Debug)]
struct MyValue<'a>(&'a u32);
impl<'a> From<&'a u32> for MyValue<'a> {
fn from(value: &'a u32) -> Self {
MyValue(value)
}
}
struct MyProducer<'a, T>(u32, PhantomData<&'a T>);
impl<'a, T> MyProducer<'a, T>
where
T: From<&'a u32>,
{
fn new() -> Self {
Self(0, PhantomData)
}
fn make(&'a mut self) -> T {
self.0 += 1;
T::from(&self.0)
}
}
fn main() {
let mut producer = MyProducer::<MyValue>::new();
println!("made this: {:?}", producer.make());
println!("made this: {:?}", producer.make());
}
The main problem here is that the mutable borrow's lifetime is the lifetime of MyProducer, that is, the lifetime of the instance called producer is the same as the mutable borrow taken in its make method. Because the producer instance does not go out of scope (if it would then MyValue wouldn't be able to hold a reference to a value stored in it) so the mutable borrow lives until the end of main's scope. The first rule of borrowing is that there can only be a single mutable borrow of a given value in a scope at any time, hence the compiler error.
If you are looking at my solution here, which is actually working and does what I think you wanted it to: (test in the playground):
#[derive(Debug)]
struct MyValue<'a>(&'a u32);
impl<'a> From<&'a u32> for MyValue<'a> {
fn from(value: &'a u32) -> Self {
MyValue(value)
}
}
struct MyProducer(u32);
impl MyProducer {
fn new() -> Self {
Self(0)
}
fn make<'a, T>(&'a mut self) -> T
where
T: From<&'a u32>,
{
self.0 += 1;
T::from(&self.0)
}
}
fn main() {
let mut producer = MyProducer::new();
println!("made this: {:?}", producer.make::<MyValue>());
println!("made this: {:?}", producer.make::<MyValue>());
}
then you can see that the mutable borrow only lives as long as the make method, therefore after the invocation there's no more living mutable borrow to producer in main's scope, thus you can have another one.
use std::marker;
use std::ops;
pub struct Shared<'r, T: 'r> {
data: *mut T,
_pd: marker::PhantomData<&'r T>,
}
impl<'r, T> Shared<'r, T> {
pub fn new(value: T) -> Shared<'r, T> {
let boxed = Box::new(value);
Shared {
data: Box::into_raw(boxed),
_pd: marker::PhantomData,
}
}
pub fn as_ref(&self) -> SharedRef<'r, T> {
SharedRef {
data: self.data,
_pd: marker::PhantomData,
}
}
}
impl<'r, T> ops::Deref for Shared<'r, T> {
type Target = T;
fn deref(&self) -> &T {
unsafe { &*self.data }
}
}
pub struct SharedRef<'r, T: 'r> {
data: *mut T,
_pd: marker::PhantomData<&'r T>,
}
impl<'r, T> ops::Deref for SharedRef<'r, T> {
type Target = T;
fn deref(&self) -> &T {
unsafe { &*self.data }
}
}
impl<'r, T> Drop for Shared<'r, T> {
fn drop(&mut self) {
unsafe {
Box::from_raw(self.data);
}
}
}
fn main() {
let s = Shared::new(42);
let s_ref = s.as_ref();
{
let s1 = s;
}
// lifetime should end here
println!("{}", *s_ref);
}
What I wanted to express was a mix between a Box and an Arc. A uniquely owned pointer that is also capable of giving out references.
The problem is that I want to be able to move Shared around even if there are currently immutable borrows to it. It should be legal in this scenario because it is heap allocated.
The problem is that I have no idea how to express this.
fn main() {
let s = Shared::new(42);
let s_ref = s.as_ref();
{
let s1 = s;
}
// lifetime should end here
println!("{}", *s_ref);
}
Here I move s into a scope with "less" lifetime than it had before. But now after I have moved s into s1, s_ref should not be accessible anymore. So what I want to say is that it is okay to move a Shared if the lifetime does not get smaller.
Can this be expressed in Rust?
The reason Rust allows you to move out of the Shared is that you haven't tied the lifetime of the returned SharedRef to it:
pub fn as_ref(&self) -> SharedRef<'r, T> {
SharedRef {
data: self.data,
_pd: marker::PhantomData,
}
}
Annotating the &self fixes that:
pub fn as_ref(&'r self) -> SharedRef<'r, T> { .. }
My current understanding is that the key difference here is that this says that the lifetime of the SharedRef now matches the lifetime of the borrow of self, keeping the borrow alive. Indeed it doesn't have to be the same lifetime ('r) as in the Shared; it works with a new lifetime just for the borrow/return:
pub fn as_ref<'b>(&'b self) -> SharedRef<'b, T> { .. }
This also disallows the move.
As for the bonus part of the question, where you want to allow moving as long as it's to something with a long enough lifetime, I think the answer is no. The only way I know to stop something being moved at all is to borrow it, and that stops any move.
I've this snippet that doesn't pass the borrow checker:
use std::collections::HashMap;
enum Error {
FunctionNotFound,
}
#[derive(Copy, Clone)]
struct Function<'a> {
name: &'a str,
code: &'a [u32],
}
struct Context<'a> {
program: HashMap<&'a str, Function<'a>>,
call_stack: Vec<Function<'a>>,
}
impl<'a> Context<'a> {
fn get_function(&'a mut self, fun_name: &'a str) -> Result<Function<'a>, Error> {
self.program
.get(fun_name)
.map(|f| *f)
.ok_or(Error::FunctionNotFound)
}
fn call(&'a mut self, fun_name: &'a str) -> Result<(), Error> {
let fun = try!(self.get_function(fun_name));
self.call_stack.push(fun);
Ok(())
}
}
fn main() {}
error[E0499]: cannot borrow `self.call_stack` as mutable more than once at a time
--> src/main.rs:29:9
|
27 | let fun = try!(self.get_function(fun_name));
| ---- first mutable borrow occurs here
28 |
29 | self.call_stack.push(fun);
| ^^^^^^^^^^^^^^^ second mutable borrow occurs here
...
32 | }
| - first borrow ends here
My gut feeling is that the problem is tied to the fact that HashMap returns either None or a reference of the value inside the data structure. But I don't want that: my intention is that self.get_function should return either a byte copy of the stored value or an error (that's why I put .map(|f| *f), and Function is Copy).
Changing &'a mut self to something else doesn't help.
However, the following snippet, somewhat similar in spirit, is accepted:
#[derive(Debug)]
enum Error {
StackUnderflow,
}
struct Context {
stack: Vec<u32>,
}
impl Context {
fn pop(&mut self) -> Result<u32, Error> {
self.stack.pop().ok_or(Error::StackUnderflow)
}
fn add(&mut self) -> Result<(), Error> {
let a = try!(self.pop());
let b = try!(self.pop());
self.stack.push(a + b);
Ok(())
}
}
fn main() {
let mut a = Context { stack: vec![1, 2] };
a.add().unwrap();
println!("{:?}", a.stack);
}
Now I'm confused. What is the problem with the first snippet? Why doesn't it happen in the second?
The snippets are part of a larger piece of code. In order to provide more context, this on the Rust Playground shows a more complete example with the faulty code, and this shows an earlier version without HashMap, which passes the borrow checker and runs normally.
You have fallen into the lifetime-trap. Adding the same lifetime to more references will constrain your program more. Adding more lifetimes and giving each reference the minimal possible lifetime will permit more programs. As #o11c notes, removing the constraints to the 'a lifetime will solve your issue.
impl<'a> Context<'a> {
fn get_function(&mut self, fun_name: &str) -> Result<Function<'a>, Error> {
self.program
.get(fun_name)
.map(|f| *f)
.ok_or(Error::FunctionNotFound)
}
fn call(&mut self, fun_name: &str) -> Result<(), Error> {
let fun = try!(self.get_function(fun_name));
self.call_stack.push(fun);
Ok(())
}
}
The reason this works is that Rust inserts new lifetimes, so in the compiler your function's signatures will look like this:
fn get_function<'b>(&'b mut self, fun_name: &'b str) -> Result<Function<'a>, Error>
fn call<'b>(&'b mut self, fun_name: &'b str) -> Result<(), Error>
Always try to not use any lifetimes and let the compiler be smart. If that fails, don't spray lifetimes everywhere, think about where you want to pass ownership, and where you want to limit the lifetime of a reference.
You only need to remove unnecessary lifetime qualifiers in order for your code to compile:
fn get_function(&mut self, fun_name: &str) -> Result<Function<'a>, Error> { ... }
fn call(&mut self, fun_name: &str) -> Result<(), Error> { ... }
Your problem was that you tied the lifetime of &mut self and the lifetime of the value stored in it (Function<'a>), which is in most cases unnecessary. With this dependency which was present in get_function() definition, the compiler had to assume that the result of the call self.get_function(...) borrows self, and hence it prohibits you from borrowing it again.
Lifetime on &str argument is also unnecessary - it just limits the possible set of argument values for no reason. Your key can be a string with arbitrary lifetime, not just 'a.