I have the following code which you can also see in the playground:
struct Node<'a> {
parent: Option<&'a Node<'a>>,
name: &'a str,
}
fn looper(nodes: &mut Vec<Node>) {
for i in 0..nodes.len() {
let n = &nodes[i];
let next_node = Node {
parent: Some(n),
name: "please compile",
};
nodes.push(next_node);
}
}
It's supposed to take a Vec of Nodes and append Nodes to it that reference the Nodes already in the Vec. Unfortunately I get this error:
error[E0623]: lifetime mismatch
--> src/main.rs:13:20
|
6 | fn looper(nodes: &mut Vec<Node>) {
| -------------- these two types are declared with different lifetimes...
...
13 | nodes.push(next_node);
| ^^^^^^^^^ ...but data from `nodes` flows into `nodes` here
How can I make this compile? An explanation of how to think about lifetimes in this context would be useful. I understand this may be an issue due to variance but am not sure how to rephrase my problem to give me the functionality I would like.
A general rule: when some value's type, or trait it implements, has a lifetime parameter, that lifetime is always longer than the life of the value itself — the value has a guarantee that the references it contains won't go invalid before they are dropped.
In fact, in your example we can see that if this were not the case, the lifetime checking would be unsound; you're adding values to nodes, but nothing would prevent looper from instead removing values from nodes, which would then invalidate any parent references that referred to those values.
The only practical way to build a tree or linked list of references (without special-purpose unsafe code that has a particular plan to keep things sound) is to write a recursive function; each function call frame may refer to nodes constructed by its caller (and its caller's caller and so on). This is accepted by the borrow checker because each frame has its own new, shorter lifetime for the references (a reference can always be taken as having a shorter lifetime than it started with).
Related
Writing a simple interpreter has lead me to this battle with the borrow checker.
#[derive(Clone, Debug)]
struct Context<'a> {
display_name: &'a str,
parent: Option<Box<Context<'a>>>,
parent_entry_pos: Position<'a>,
}
// --snip--
#[derive(Copy, Clone, Debug)]
pub enum BASICVal<'a> {
Float(f64, Position<'a>, Position<'a>, &'a Context<'a>),
Int(i64, Position<'a>, Position<'a>, &'a Context<'a>),
Nothing(Position<'a>, Position<'a>, &'a Context<'a>),
}
// --snip--
pub fn run<'a>(text: &'a String, filename: &'a String) -> Result<(Context<'a>, BASICVal<'a>), BASICError<'a>> {
// generate tokens
let mut lexer = Lexer::new(text, filename);
let tokens = lexer.make_tokens()?;
// parse program to AST
let mut parser = Parser::new(tokens);
let ast = parser.parse();
// run the program
let context: Context<'static> = Context {
display_name: "<program>",
parent: None,
parent_entry_pos: Position::default(),
};
Ok((context, interpreter_visit(&ast?, &context)?))
}
The error is "cannot return value referencing local variable `context`" and (secondary) the "borrow of moved value: `context`":
error[E0515]: cannot return value referencing local variable `context`
--> src\basic.rs:732:2
|
732 | Ok((context, interpreter_visit(&ast?, &context)?))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^--------^^^^
| | |
| | `context` is borrowed here
| returns a value referencing data owned by the current function
error[E0382]: borrow of moved value: `context`
--> src\basic.rs:732:40
|
727 | let context: Context<'static> = Context {
| ------- move occurs because `context` has type `basic::Context<'_>`, which does not implement the `Copy` trait
...
732 | Ok((context, interpreter_visit(&ast?, &context)?))
| ------- value moved here ^^^^^^^^ value borrowed here after move
As far as I understand it: The context references several lifetime-dependent structs. The values of these structs are static in this case, as I can see by explicitly setting the lifetime parameter to 'static and the compiler not complaining. The interpreter_visit function needs to borrow the context because it gets passed to several independent functions, including itself recursively. In addition, the interpreter_visit returns BASICVals that reference the context themselves. For this reason, the context needs to outlive the run return. I try to achieve that by passing the context itself as part of the return value, thereby giving the caller control over its life. But now, I move the context to the return value before actually using it? This makes no sense. I should be able to reference one part of the return value in another part of the return value because both values make it out of the function "alive". I have tried:
boxing the context, thereby forcing it off the stack onto the heap, but that seems to only complicate things.
switching the order of the tuple, but that doesn't help.
storing interpreter_visit's result in an intermediate variable, which as expected doesn't help.
cloning the interpreter_visit result or the context itself
The issue may lie with the result and the error. The error doesn't reference a context but giving it a separate lifetime in interpreter_visit breaks the entire careful balance I have been able to achieve until now.
Answering this so that people don't have to read the comment thread.
This is a problem apparently not solvable by Rust's borrow checker. The borrow checker cannot understand that a Box of context will live on the heap and therefore last longer than the function return, therefore being "legally" referencable by the return value of interpreter_visit which itself escapes the function. The solution in this case is to circumvent borrow checking via unsafe, namely a raw pointer. Like this:
let context = Box::new(Context {
display_name: "<program>",
parent: None,
parent_entry_pos: Position::default(),
});
// Obtain a pointer to a location on the heap
let context_ptr: *const Context = &*context;
// outsmart the borrow checker
let result = interpreter_visit(&ast?, unsafe { &*context_ptr })?;
// The original box is passed back, so it is destroyed safely.
// Because the result lives as long as the context as required by the lifetime,
// we cannot get a pointer to uninitialized memory through the value and its context.
Ok((context, result))
I store a raw pointer to the context in context_ptr. The borrowed value passed to interpreter_visit is then piped through a (completely memory-safe) raw pointer dereference and borrow. This will (for some reason, only the Rust gods know) disable the borrow check, so the context data given to interpreter_visit is considered to have a legal lifetime. As I am however still passing back the very safe Box around the context data, I can avoid creating memory leaks by leaving the context with no owner. It might be possible now to pass around the interpreter_visit return value with having the context destroyed, but because both values are printed and discarded immediately, I see no issues arising from this in the future.
If you have a deeper understanding of Rust's borrow checker and would consider this a fixable edge case that doesn't have more "safe" solutions I couldn't come up with, please do comment and I will report this to the Rust team. I'm however not that certain especially because my experience with and knowledge of Rust is limited.
This is my minimal reproducible code (playground):
struct MyStruct {
my_string: String,
}
fn accepts_string(my_string: String) {
println!("my_string: {}", my_string)
}
fn accepts_struct_reference(my_struct: &MyStruct) {
accepts_string(my_struct.my_string);
}
fn main() {
let my_struct = MyStruct {
my_string: String::from("hi"),
};
accepts_struct_reference(&my_struct);
}
which produces:
error[E0507]: cannot move out of `my_struct.my_string` which is behind a shared reference
--> src/main.rs:10:20
|
10 | accepts_string(my_struct.my_string);
| ^^^^^^^^^^^^^^^^^^^ move occurs because `my_struct.my_string` has type `std::string::String`, which does not implement the `Copy` trait
I believe I understand why this error occurs: accepts_string tries to take the string away from the struct.
Why is that reference called a shared reference? Shared with whom? Does that adjective mean that there are non-shared references? If yes, what do they look like?
This is another way to distinguish mutable and immutable references.
In Rust, there is a clear distinction: the data can be either sharable (even if they're not shared currently), or directly mutable, but not both at once. This is achieved by having two types of references:
shared, or immutable ones, which can be copied but cannot be used to directly mutate data;
unique, or mutable ones, which cannot be copied (and this is UB, if you do this somehow), but can be used to mutate the data.
Note the "directly mutable" bit. Of course, it's sometimes possible to modify data through shared reference - if the data itself allows it; this is so-called internal mutability, based on multiple types like Cell or Mutex, which all internally use UnsafeCell - the only type explicitly allowed to be mutated behind shared reference.
More info might be found here: https://limpet.net/mbrubeck/2019/02/07/rust-a-unique-perspective.html
I'm working with a friend to define a safe public API for lifetimes of a "scoped" garbage collector. The lifetimes are either overly constrained and correct code does not compile or the lifetimes are too loose and they may allow invalid behavior. After trying multiple approaches, we are still stuck getting a correct API. This is especially frustrating because Rust's lifetimes can help avoid bugs in this situation but right now it just looks stubborn.
Scoped garbage collection
I am implementing an ActionScript interpreter and need a garbage collector. I studied rust-gc but it did not suit my needs. The main reason is that it requires the garbage collected values to have a static lifetime because the GC state is a thread-local static variable. I need to get garbage-collected bindings to a dynamically created host object. The other reason to avoid globals is that it is easier for me to handle multiple independent garbage-collected scopes, control their memory limits or serialize them.
A scoped garbage collector is similar to a typed-arena. You can use it to allocate values and they are all freed once the garbage collector is dropped. The difference is that you can also trigger garbage collection during its lifetime and it will clean-up the unreachable data (and is not limited to a single type).
I have a working implementation implemented (mark & sweep GC with scopes), but the interface is not yet safe to use.
Here is a usage example of what I want:
pub struct RefNamedObject<'a> {
pub name: &'a str,
pub other: Option<Gc<'a, GcRefCell<NamedObject<'a>>>>,
}
fn main() {
// Initialize host settings: in our case the host object will be replaced by a string
// In this case it lives for the duration of `main`
let host = String::from("HostConfig");
{
// Create the garbage-collected scope (similar usage to `TypedArena`)
let gc_scope = GcScope::new();
// Allocate a garbage-collected string: returns a smart pointer `Gc` for this data
let a: Gc<String> = gc_scope.alloc(String::from("a")).unwrap();
{
let b = gc_scope.alloc(String::from("b")).unwrap();
}
// Manually trigger garbage collection: will free b's memory
gc_scope.collect_garbage();
// Allocate data and get a Gc pointer, data references `host`
let host_binding: Gc<RefNamed> = gc_scope
.alloc(RefNamedObject {
name: &host,
other: None,
})
.unwrap();
// At the end of this block, gc_scope is dropped with all its
// remaining values (`a` and `host_bindings`)
}
}
Lifetime properties
The basic intuition is that Gc can only contain data that lives as long (or longer) than the corresponding GcScope. Gc is similar to Rc but supports cycles. You need to use Gc<GcRefCell<T>> to mutate values (similar to Rc<RefCell<T>>).
Here are the properties that must be satisfied by the lifetimes of my API:
Gc cannot live longer than its GcScope
The following code must fail because a outlives gc_scope:
let a: Gc<String>;
{
let gc_scope = GcScope::new();
a = gc_scope.alloc(String::from("a")).unwrap();
}
// This must fail: the gc_scope was dropped with all its values
println("{}", *a); // Invalid
Gc cannot contain data that lives shorter than its GcScope
The following code must fail because msg does not live as long (or longer) as gc_scope
let gc_scope = GcScope::new();
let a: Gc<&string>;
{
let msg = String::from("msg");
a = gc.alloc(&msg).unwrap();
}
It must be possible to allocate multiple Gc (no exclusion on gc_scope)
The following code must compile
let gc_scope = GcScope::new();
let a = gc_scope.alloc(String::from("a"));
let b = gc_scope.alloc(String::from("b"));
It must be possible to allocate values containing references with lifetimes longer than gc_scope
The following code must compile
let msg = String::from("msg");
let gc_scope = GcScope::new();
let a: Gc<&str> = gc_scope.alloc(&msg).unwrap();
It must be possible to create cycles of Gc pointers (that's the whole point)
Similarly to the Rc<Refcell<T>> pattern, you can use Gc<GcRefCell<T>> to mutate values and create cycles:
// The lifetimes correspond to my best solution so far, they can change
struct CircularObj<'a> {
pub other: Option<Gc<'a, GcRefCell<CircularObj<'a>>>>,
}
let gc_scope = GcScope::new();
let n1 = gc_scope.alloc(GcRefCell::new(CircularObj { other: None }));
let n2 = gc_scope.alloc(GcRefCell::new(CircularObj {
other: Some(Gc::clone(&n1)),
}));
n1.borrow_mut().other = Some(Gc::clone(&n2));
Solutions so far
Automatic lifetime / lifetime tag
Implemented on the auto-lifetime branch
This solution is inspired by neon's handles.
This lets any valid code compile (and allowed me to test my implementation) but is too loose and allows invalid code. It allows Gc to outlive the gc_scope that created it. (Violates the first property)
The idea here is that I add a single lifetime 'gc to all my structs. The idea is that this lifetime represents "how long gc_scope lives".
// A smart pointer for `T` valid during `'gc`
pub struct Gc<'gc, T: Trace + 'gc> {
pub ptr: NonNull<GcBox<T>>,
pub phantom: PhantomData<&'gc T>,
pub rooted: Cell<bool>,
}
I call it automatic lifetimes because the methods never mix these struct lifetimes with the lifetime of the references they receive.
Here is the impl for gc_scope.alloc:
impl<'gc> GcScope<'gc> {
// ...
pub fn alloc<T: Trace + 'gc>(&self, value: T) -> Result<Gc<'gc, T>, GcAllocErr> {
// ...
}
}
Inner/outer lifetimes
Implemented on the inner-outer branch
This implementation tries to fix the previous issue by relating Gc to the lifetime of GcScope. It is overly constrained and prevents the creation of cycles. This violates the last property.
To constrain Gc relative to its GcScope, I introduce two lifetimes: 'inner is the lifetime of GcScope and the result is Gc<'inner, T>. 'outer represents a lifetime longer than 'inner and is used for the allocated value.
Here is the alloc signature:
impl<'outer> GcScope<'outer> {
// ...
pub fn alloc<'inner, T: Trace + 'outer>(
&'inner self,
value: T,
) -> Result<Gc<'inner, T>, GcAllocErr> {
// ...
}
// ...
}
Closure (context management)
Implemented on the with branch
Another idea was to not let the user create a GcScope manually with GcScope::new but instead expose a function GcScope::with(executor) providing a reference to the gc_scope. The closure executor corresponds to the gc_scope. So far, it either prevents the use of external references or allows to leak data to external Gc variables (first and fourth properties).
Here is the alloc signature:
impl<'gc> GcScope<'gc> {
// ...
pub fn alloc<T: Trace + 'gc>(&self, value: T) -> Result<Gc<'gc, T>, GcAllocErr> {
// ...
}
}
Here is a usage example showing the violation of the first property:
let message = GcScope::with(|scope| {
scope
.alloc(NamedObject {
name: String::from("Hello, World!"),
})
.unwrap()
});
println!("{}", message.name);
What I'd like
From what I understand, the alloc signature I'd like is:
impl<'gc> GcScope<'gc> {
pub fn alloc<T: Trace + 'gc>(&'gc self, value: T) -> Result<Gc<'gc, T>, GcAllocErr> {
// ...
}
}
Where everything lives as long or longer than self (the gc_scope). But this blows up with the most simple tests:
fn test_gc() {
let scope: GcScope = GcScope::new();
scope.alloc(String::from("Hello, World!")).unwrap();
}
causes
error[E0597]: `scope` does not live long enough
--> src/test.rs:50:3
|
50 | scope.alloc(String::from("Hello, World!")).unwrap();
| ^^^^^ borrowed value does not live long enough
51 | }
| - `scope` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
I have no idea what happens here. Playground link
Edit: As explained to me on IRC, this is because I implement Drop which requires &mut self, but the scope is already borrowed in read-only mode.
Overview
Here is a quick overview of the main components of my library.
GcScope contains a RefCell to its mutable state. This was introduced to not require &mut self for alloc because it "locked" the gc_scope and violated property 3: allocate multiple values.
This mutable state is GcState. It keeps track of all the allocated values. The values are stored as a forward-only linked list of GcBox. This GcBox is heap-allocated and contains the actual value with some metadata (how many active Gc pointers have it as their root and a boolean flag used to check if the value is reachable from the root (see rust-gc). The value here must outlive its gc_scope so GcBox uses a lifetime, and in turn GcState must then use a lifetime as well as GcScope: this is always the same lifetime meaning "longer than gc_scope". The fact that GcScope has a RefCell (interior mutability) and lifetime is maybe the reason why I can't get my lifetimes to work (it causes invariance?).
Gc is a smart pointer to some gc_scope-allocated data. You can only get it through gc_scope.alloc or by cloning it.
GcRefCell is most likely fine, it's just a RefCell wrapper adding metadata and behavior to properly support borrows.
Flexibility
I'm fine with the following requirements to get a solution:
unsafe code
nightly features
API changes (see for example my with approach). What matters is that I can create a temporary zone where I can manipulate garbage-collected values and that they are all dropped after this. These garbage-collected values need to be able to access longer-lived (but not static) variables outside of the scope.
The repository has a few tests in scoped-gc/src/lib.rs (compile-fail) as scoped-gc/src/test.rs.
I found a solution, I'll post it once redacted.
This is one of the hardest problems I had with lifetimes with Rust so far, but I managed to find a solution. Thank you to panicbit and mbrubeck for having helped me on IRC.
What helped me to move forward was the explanation of the error I posted at the end of my question:
error[E0597]: `scope` does not live long enough
--> src/test.rs:50:3
|
50 | scope.alloc(String::from("Hello, World!")).unwrap();
| ^^^^^ borrowed value does not live long enough
51 | }
| - `scope` dropped here while still borrowed
|
= note: values in a scope are dropped in the opposite order they are created
I did not understand this error because it wasn't clear to me why scope was borrowed, for how long, or why it needs to no longer be borrowed at the end of the scope.
The reason is that during the allocation of the value, the scope is immutably borrowed for the duration of the allocated value. The issue now is that the scope contains a state object that implements "Drop": custom implementations of drop use &mut self -> it is not possible to get a mutable borrow for the drop while the value is already immutably borrowed.
Understanding that drop requires &mut self and that it is incompatible with immutable borrows unlocked the situation.
It turns out that the inner-outer approach described in the question above had the correct lifetimes with alloc:
impl<'outer> GcScope<'outer> {
// ...
pub fn alloc<'inner, T: Trace + 'outer>(
&'inner self,
value: T,
) -> Result<Gc<'inner, T>, GcAllocErr> {
// ...
}
// ...
}
The returned Gc lives as long as GcScope and the allocated values must live longer than the current GcScope. As mentioned in the question, the issue with this solution is that it did not support circular values.
The circular values failed to work not because of the lifetimes of alloc but due to the custom drop. Removing drop allowed all the tests to pass (but leaked memory).
The explanation is quite interesting:
The lifetime of alloc expresses the properties of the allocated values. The allocated values cannot outlive their GcScope but their content must live as long or longer than GcScope. When creating a cycle, the value is subject to both of these constraints: it is allocated so must live as long or shorter than GcScope but also referenced by another allocated value so it must live as long or longer than GcScope. Because of this there is only one solution: the allocated value must live exactly as long as its scope.
It means that the lifetime of GcScope and its allocated values is exactly the same. When two lifetimes are the same, Rust does not guarantee the order of the drops. The reason why this happens is that the drop implementations could try to access each other and since there's no ordering it would be unsafe (the value might already have been freed).
This is explained in the Drop Check chapter of the Rustonomicon.
In our case, the drop implementation of the state of the garbage collected does not dereference the allocated values (quite the opposite, it frees their memory) so the Rust compiler is overly cautious by preventing us from implementing drop.
Fortunately, the Nomicon also explains how to work around these check of values with the same lifetimes. The solution is to use the may_dangle attribute on the lifetime parameter of the Drop implementation.
This is as unstable attribute that requires to enable the generic_param_attrs and dropck_eyepatch features.
Concretely, my drop implementation became:
unsafe impl<'gc> Drop for GcState<'gc> {
fn drop(&mut self) {
// Free all the values allocated in this scope
// Might require changes to make sure there's no use after free
}
}
And I added the following lines to lib.rs:
#![feature(generic_param_attrs)]
#![feature(dropck_eyepatch)]
You can read more about these features:
generic_param_attrs
may_dangle
I updated my library scoped-gc with the fix for this issue if you want to take a closer look at it.
I'm reading through the Rust documentation on lifetimes. I tried something like:
struct S {
x: i8,
}
impl S {
fn fun(self) {}
fn print(&self) {
println!("{}", self.x);
}
}
fn main() {
let s = S { x: 1 };
s.fun();
s.print();
}
I get the following error:
error[E0382]: borrow of moved value: `s`
--> src/main.rs:16:5
|
15 | s.fun();
| - value moved here
16 | s.print();
| ^ value borrowed here after move
|
= note: move occurs because `s` has type `S`, which does not implement the `Copy` trait
This is because the fun(self) method takes ownership of the s instance. This is solved by changing to fun(&self).
I can't see why you would ever want to have a method on an object take control of itself. I can think of only one example, a destructor method, but if you wanted to do dispose of the object then it would be taken care of by the owner of the object anyway (i.e. scope of main in this example).
Why is it possible to write a method that takes ownership of the struct? Is there ever any circumstance where you would want this?
The idiomatic way to refer to a method that "takes control" of self in the Rust standard library documentation is to say that it "consumes" it. If you search for this, you should find some examples:
Option::unwrap_or_default
A lot in the Iterator trait.
As to why: you can try rewriting Iterator::map — you would end up having a lifetime parameter wandering around that would quickly become unmanageable. Why? Because the Map iterator is based upon the previous one, so the borrow checker will enforce that you can only use one of the two at the same time.
Conversion from type A to type B commonly involves functions taking self by value. See the implementors of Into and From traits for concrete examples.
I was reading the lifetimes chapter of the Rust book, and I came across this example for a named/explicit lifetime:
struct Foo<'a> {
x: &'a i32,
}
fn main() {
let x; // -+ x goes into scope
// |
{ // |
let y = &5; // ---+ y goes into scope
let f = Foo { x: y }; // ---+ f goes into scope
x = &f.x; // | | error here
} // ---+ f and y go out of scope
// |
println!("{}", x); // |
} // -+ x goes out of scope
It's quite clear to me that the error being prevented by the compiler is the use-after-free of the reference assigned to x: after the inner scope is done, f and therefore &f.x become invalid, and should not have been assigned to x.
My issue is that the problem could have easily been analyzed away without using the explicit 'a lifetime, for instance by inferring an illegal assignment of a reference to a wider scope (x = &f.x;).
In which cases are explicit lifetimes actually needed to prevent use-after-free (or some other class?) errors?
The other answers all have salient points (fjh's concrete example where an explicit lifetime is needed), but are missing one key thing: why are explicit lifetimes needed when the compiler will tell you you've got them wrong?
This is actually the same question as "why are explicit types needed when the compiler can infer them". A hypothetical example:
fn foo() -> _ {
""
}
Of course, the compiler can see that I'm returning a &'static str, so why does the programmer have to type it?
The main reason is that while the compiler can see what your code does, it doesn't know what your intent was.
Functions are a natural boundary to firewall the effects of changing code. If we were to allow lifetimes to be completely inspected from the code, then an innocent-looking change might affect the lifetimes, which could then cause errors in a function far away. This isn't a hypothetical example. As I understand it, Haskell has this problem when you rely on type inference for top-level functions. Rust nipped that particular problem in the bud.
There is also an efficiency benefit to the compiler — only function signatures need to be parsed in order to verify types and lifetimes. More importantly, it has an efficiency benefit for the programmer. If we didn't have explicit lifetimes, what does this function do:
fn foo(a: &u8, b: &u8) -> &u8
It's impossible to tell without inspecting the source, which would go against a huge number of coding best practices.
by inferring an illegal assignment of a reference to a wider scope
Scopes are lifetimes, essentially. A bit more clearly, a lifetime 'a is a generic lifetime parameter that can be specialized with a specific scope at compile time, based on the call site.
are explicit lifetimes actually needed to prevent [...] errors?
Not at all. Lifetimes are needed to prevent errors, but explicit lifetimes are needed to protect what little sanity programmers have.
Let's have a look at the following example.
fn foo<'a, 'b>(x: &'a u32, y: &'b u32) -> &'a u32 {
x
}
fn main() {
let x = 12;
let z: &u32 = {
let y = 42;
foo(&x, &y)
};
}
Here, the explicit lifetimes are important. This compiles because the result of foo has the same lifetime as its first argument ('a), so it may outlive its second argument. This is expressed by the lifetime names in the signature of foo. If you switched the arguments in the call to foo the compiler would complain that y does not live long enough:
error[E0597]: `y` does not live long enough
--> src/main.rs:10:5
|
9 | foo(&y, &x)
| - borrow occurs here
10 | };
| ^ `y` dropped here while still borrowed
11 | }
| - borrowed value needs to live until here
The lifetime annotation in the following structure:
struct Foo<'a> {
x: &'a i32,
}
specifies that a Foo instance shouldn't outlive the reference it contains (x field).
The example you came across in the Rust book doesn't illustrate this because f and y variables go out of scope at the same time.
A better example would be this:
fn main() {
let f : Foo;
{
let n = 5; // variable that is invalid outside this block
let y = &n;
f = Foo { x: y };
};
println!("{}", f.x);
}
Now, f really outlives the variable pointed to by f.x.
Note that there are no explicit lifetimes in that piece of code, except the structure definition. The compiler is perfectly able to infer lifetimes in main().
In type definitions, however, explicit lifetimes are unavoidable. For example, there is an ambiguity here:
struct RefPair(&u32, &u32);
Should these be different lifetimes or should they be the same? It does matter from the usage perspective, struct RefPair<'a, 'b>(&'a u32, &'b u32) is very different from struct RefPair<'a>(&'a u32, &'a u32).
Now, for simple cases, like the one you provided, the compiler could theoretically elide lifetimes like it does in other places, but such cases are very limited and do not worth extra complexity in the compiler, and this gain in clarity would be at the very least questionable.
If a function receives two references as arguments and returns a reference, then the implementation of the function might sometimes return the first reference and sometimes the second one. It is impossible to predict which reference will be returned for a given call. In this case, it is impossible to infer a lifetime for the returned reference, since each argument reference may refer to a different variable binding with a different lifetime. Explicit lifetimes help to avoid or clarify such a situation.
Likewise, if a structure holds two references (as two member fields) then a member function of the structure may sometimes return the first reference and sometimes the second one. Again explicit lifetimes prevent such ambiguities.
In a few simple situations, there is lifetime elision where the compiler can infer lifetimes.
I've found another great explanation here: http://doc.rust-lang.org/0.12.0/guide-lifetimes.html#returning-references.
In general, it is only possible to return references if they are
derived from a parameter to the procedure. In that case, the pointer
result will always have the same lifetime as one of the parameters;
named lifetimes indicate which parameter that is.
The case from the book is very simple by design. The topic of lifetimes is deemed complex.
The compiler cannot easily infer the lifetime in a function with multiple arguments.
Also, my own optional crate has an OptionBool type with an as_slice method whose signature actually is:
fn as_slice(&self) -> &'static [bool] { ... }
There is absolutely no way the compiler could have figured that one out.
As a newcomer to Rust, my understanding is that explicit lifetimes serve two purposes.
Putting an explicit lifetime annotation on a function restricts the type of code that may appear inside that function. Explicit lifetimes allow the compiler to ensure that your program is doing what you intended.
If you (the compiler) want(s) to check if a piece of code is valid, you (the compiler) will not have to iteratively look inside every function called. It suffices to have a look at the annotations of functions that are directly called by that piece of code. This makes your program much easier to reason about for you (the compiler), and makes compile times managable.
On point 1., Consider the following program written in Python:
import pandas as pd
import numpy as np
def second_row(ar):
return ar[0]
def work(second):
df = pd.DataFrame(data=second)
df.loc[0, 0] = 1
def main():
# .. load data ..
ar = np.array([[0, 0], [0, 0]])
# .. do some work on second row ..
second = second_row(ar)
work(second)
# .. much later ..
print(repr(ar))
if __name__=="__main__":
main()
which will print
array([[1, 0],
[0, 0]])
This type of behaviour always surprises me. What is happening is that df is sharing memory with ar, so when some of the content of df changes in work, that change infects ar as well. However, in some cases this may be exactly what you want, for memory efficiency reasons (no copy). The real problem in this code is that the function second_row is returning the first row instead of the second; good luck debugging that.
Consider instead a similar program written in Rust:
#[derive(Debug)]
struct Array<'a, 'b>(&'a mut [i32], &'b mut [i32]);
impl<'a, 'b> Array<'a, 'b> {
fn second_row(&mut self) -> &mut &'b mut [i32] {
&mut self.0
}
}
fn work(second: &mut [i32]) {
second[0] = 1;
}
fn main() {
// .. load data ..
let ar1 = &mut [0, 0][..];
let ar2 = &mut [0, 0][..];
let mut ar = Array(ar1, ar2);
// .. do some work on second row ..
{
let second = ar.second_row();
work(second);
}
// .. much later ..
println!("{:?}", ar);
}
Compiling this, you get
error[E0308]: mismatched types
--> src/main.rs:6:13
|
6 | &mut self.0
| ^^^^^^^^^^^ lifetime mismatch
|
= note: expected type `&mut &'b mut [i32]`
found type `&mut &'a mut [i32]`
note: the lifetime 'b as defined on the impl at 4:5...
--> src/main.rs:4:5
|
4 | impl<'a, 'b> Array<'a, 'b> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...does not necessarily outlive the lifetime 'a as defined on the impl at 4:5
--> src/main.rs:4:5
|
4 | impl<'a, 'b> Array<'a, 'b> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
In fact you get two errors, there is also one with the roles of 'a and 'b interchanged. Looking at the annotation of second_row, we find that the output should be &mut &'b mut [i32], i.e., the output is supposed to be a reference to a reference with lifetime 'b (the lifetime of the second row of Array). However, because we are returning the first row (which has lifetime 'a), the compiler complains about lifetime mismatch. At the right place. At the right time. Debugging is a breeze.
The reason why your example does not work is simply because Rust only has local lifetime and type inference. What you are suggesting demands global inference. Whenever you have a reference whose lifetime cannot be elided, it must be annotated.
I think of a lifetime annotation as a contract about a given ref been valid in the receiving scope only while it remains valid in the source scope. Declaring more references in the same lifetime kind of merges the scopes, meaning that all the source refs have to satisfy this contract.
Such annotation allow the compiler to check for the fulfillment of the contract.