I'm a newbie in Rust and I'm still struggling with lifetimes in Rust. Below is an example from a book I'm reading. Could anyone help explain why the author can get this information just by looking at the function signature? I already have basic understanding of borrowing, references etc. but still can't understand it.
For example, suppose we have a parsing function that takes a slice of bytes, and returns a structure holding the results of the parse:
fn parse_record<'i>(input: &'i [u8]) -> Record<'i> { ... }
Without looking into the definition of the Record type at all, we can tell that, if we
receive a Record from parse_record, whatever references it contains must point into
the input buffer we passed in, and nowhere else (except perhaps at 'static values).
The Record can get references only from the function body. In theory, these can come from
values in input, which are all references with lifetime 'i
values defined outside parse_record, which must have 'static lifetime
values defined in parse_record. These can be
dynamically created. Such values would be dropped by the end of the function scope, so any references to them would end up as dangling pointers. As such, this isn't allowed by the compiler.
literals (1, "cat"). These are actually baked into the binary, and so are effectively defined outside the function. In the function they're 'static references
The input:
input: &'i [u8]
Says the following:
I am a reference to a series of bytes ([u8]) that will live for at least as long as 'i.
So, when saying that I've a type that looks like this: Record<'i>, I can say the following about it:
I am a struct (Named Record<'i>) that contains something (Perhaps a reference, perhaps something else) that depends on something living for at least as long as 'i.
What lifetimes on references/structs/enums/whatever are telling you is that there is a dependency that an object that lives in 'i must live as long as I do.
In other words, this function signature tells you that the Record must live as long as the bytes referenced by input (The u8s under the reference, not the reference itself).
A lack of a lifetime parameter was recently deprecated and now causes a warning, so keep in mind when reading examples like the following:
fn parse_record(input: &[u8]) -> Record
that there may be a lifetime attached to Record so you must consult some kind of documentation. The compile desugars this (And the warning asks you to do this yourself) to this:
fn parse_record(input: &'_ [u8]) -> Record<'_>
This is identical to your 'i example.
Related
I want to create a struct that will hold a bunch of Path references for later lookup.
I first started with this:
struct AbsoluteSystemPaths{
home_folder:Path,
shortcuts_file:Path
}
The compiler complained about -> Path doesn't have a size known at compile-time.
It is sensible, a struct can only have as much as one non sized var member and I added 2 (home_folder and shortcuts_file). So, I decided to move them to references:
struct AbsoluteSystemPaths{
home_folder:& Path,
shortcuts_file: &Path
}
That made the sized error go away but brought another one related to lifetimes: "Missing lifetime specifier" for the two var members of type Path.
And again, it is pretty sensible, Rust needs to know how much time this 2 references will live to avoid dangling references or memory leaks.
So, I added the life time specifiers:
struct AbsoluteSystemPaths<'a> {
home_folder:&'a Path,
shortcuts_file: &'a Path
}
and then everything compiled without errors.
What I would like to know is:
Is my general reasoning correct?
The lifetime annotations I added could be read as: Both path references will last at least as 'a that is the lifetime of an instance contrusted from AbsoluteSystemPaths struct?
Yes, your understanding is correct.
One thing to note with Path, as it is an unsized type as you've mentioned, is that the standard library provides a struct called PathBuf that represents "An owned, mutable path (akin to String).". This is an alternative to using &Path if you'd like your struct to own the Path instead of hold references to them. This may be useful depending on the way you're using your struct.
I encountered a lifetime error which I am not able to explain why it is emitted by the compiler. I need this (which works fine):
fn iter<'a>() -> impl Iterator<Item = &'a f64> {
[3.14].iter()
}
However, when I try to use a float value which is transmuted from a specific byte representation using from_bits, like this:
fn iter<'a>() -> impl Iterator<Item = &'a f64> {
[f64::from_bits(0x7fffffffffffffff)].iter()
}
it gives me "creates a temporary which is freed while still in use". Playground here (stable 1.45.2).
My reasoning was that since f64 is Copy type (and it indeed works as expected if I use a constant value), this should work because no freeing should be performed on this value.
So the question is why the compiler emits the error in the second case?
Thanks for any pointers and explanations!
P.S. I need the iterator over references because it fits nicely with my other API.
Your code has two related problems:
You return a reference to a temporary object that go out of scope at the end of the function body.
Your return type contains a lifetime parameter that isn't bound to any function input.
References can only live as long as the data they point to. The result of the method call f64::from_bits(0x7fffffffffffffff) is a temporary object which goes out of scope at the end of the expression. Returning a reference to a temporary value or a local variable from a function is not possible, since the value referred to won't be alive anymore once the function returns.
The Copy trait, or whether the object is stored on the heap, is completely unrelated to the fact that values that go out of scope can no longer be referred to. Any value created inside a function will go out of scope at the end of the function body, unless you move it out of the function via the return value. However, you need to move ownership of the value for this to work – you can't simply return a reference.
Since you can't return a reference to any value created inside your function, any reference in the return value necessarily needs to refer to something that was passed in via the function parameters. This implies that the lifetime of any reference in the return value needs to match the lifetime of some reference that was passed to the function. Which gets us to the second point – a lifetime parameter that only occurs in the return type is always an error. The lifetime parameter is chosen by the calling code, so you are essentially saying that your function returns a reference that lives for an arbitrary time chosen by the caller, which is only possible if the reference refers to static data that lives as long as the program.
This also gives an explanation why your first example works. The literal [3.14] defines a constant static array. This array will live as long as the program, so you can return references with arbitrary lifetime to it. However, you'd usually express this by explicitly specifying the static lifetime to make clear what is happening:
fn iter() -> impl Iterator<Item = &'static f64> {
[3.14].iter()
}
A lifetime parameter that only occurs in the return value isn't ever useful.
So how do you fix your problem? You probably need to return an iterator over an owned type, since your iter() function doesn't accept any arguments.
It is Copy, it is semantically copied, and that is indeed the problem, as it semantically only exists on the stack until function returns, reference is now, semantically, pointing to memory that is outside the stack and is very likely to be overwritten soon, and if Rust allowed it, that would result in undefined behaviour.
On top of that, from_bits isn't const, and that means that you cannot convert static values at compile time, it is a runtime operation. Why do you want to convert each time when you already know the value?
Why is this the case anyway?
from_bits:
This is currently identical to transmute::<u64, f64>(v) on all
platforms.
If you take a look at transmute, you will find:
transmute is semantically equivalent to a bitwise move of one type
into another. It copies the bits from the source value into the
destination value, then forgets the original. It's equivalent to C's
memcpy under the hood, just like transmute_copy.
While generated code may indeed just be a simple reinterpretation of static value, Rust cannot semantically allow this value to be moved onto stack, then dropped, while a reference is still pointing to it.
Solution.
Since you want to return aNaN, you should just do the same thing you did in the first example:
fn iter<'a>() -> impl Iterator<Item = &'a f64> {
[f64::NAN].iter()
}
This will iterate over a static slice directly, and there will be no issues.
So I'm trying to squeeze in callbacks to this enum variant (Visual) - which will be stored in a vector as seen within the struct EntityComponents:
enum Component {
Position([f64; 2]),
Visual(& Fn(Entity) -> ()),
}
struct EntityComponents {
components_of_entity: HashMap<TypeId, Vec<Component>>,
}
However, Rust requires me to provide explicit lifetime parameters here.
My idea is that I want the function reference to live at least as long as its argument (the Entity), but I have no idea how the syntax for that would look like? Is it even possible?
The idea is that as long as an Entity has a Visual component, we can use this callback to render it!
A few things:
You used &Fn() instead of fn(). The former is a trait object of the Fn trait, the latter is a good ol' function pointer. While the former is more general (in particular, it supports closures), it's rather unusual to store a reference to a closure that lives somewhere else.
Therefore, if you want to store closures as well, you'd probably use a Box<Fn()> which solves your lifetime problems.
If you don't care about closures, but just want to save function pointer, you can simply write Visual(fn(Entity)),. Function pointer always have a static lifetime. So this solves your lifetime problems as well.
But what if you really want to store a reference to an Fn trait object, which lives somewhere else? The solution depends on where it lives:
In Entity: see here
somehere else: use a lifetime <'a>
You wrote Fn(Entity) -> (). The -> () is always useless, you can omit it.
I'm trying to get the hang of Rust lifetimes. While I seem to understand them, I don't seem to know the best way to fix it. Here is a function where I am using the *ring* package to generate a SHA256 HMAC. Here is a simplified version of the function that reproduced the problem:
fn sign<'b>(data: &[u8], key: &[u8]) -> &'b [u8] {
let hmac_key = hmac::SigningKey::new(&digest::SHA256, key);
let signature = hmac::sign(&hmac_key, data);
let data = signature.as_ref();
data
}
This doesn't work because signature doesn't live long enough. That makes sense; as_ref has a reference to signature, and signature doesn't live past the end of the function.
as_ref is the recommended way in *ring* to get a &[u8] from its Digest structure as seen in the documentation.
How do I correct the issue where signature does not live long enough without copying the entire contents of the byte array?
signature binds to a resource that, as it is, will only live in the scope of that function. Naturally, it would only be wrong to lend the signature's array, also living in the function, to something living outside the function. Therefore, expanding its lifetime, as you intended, is out of the question.
With that in mind, there are two ways to go around this, ordered by preference:
We can just pass the ownership of signature to the outside, by making the function return the Digest. Note that this doesn't mean that there will be deep copies of the content. Return-value optimization can take place, which will produce the returned object in-place if it's larger than a pointer. On the other hand, this seems to be more of an implementation detail rather than a guarantee of the language. I would look into the compiled assembly if this is a true concern.
Alternatively, the library could be rewritten (or modified with an expanded API) to accept a mutable reference to a buffer (which might need to support resizing, such as &mut Vec<u8>). Of course, this implies proposing and implementing acceptable changes to this library, but the former approach to your problem might just suffice.
I'm trying to wrap my head around Rust lifetimes (as the official guides don't really explain them that well).
Do rust lifetimes only refer to references, or can they refer to base/primitive values as well?
Lifetimes are the link between values and references to said values.
In order to understand this link, I will use a broken parallel: houses and addresses.
A house is a physical entity. It is built on a piece of land at some time, will live for a few dozen or hundred years, may be renovated multiple times during this time, and will most likely be destroyed at some point.
An address is a logical entity, it may point to a house, or to other physical entities (a field, a school, a train station, a company's HQ, ...).
The lifetime of a house is relatively clear: it represents the duration during which a house is usable, from the moment it is built to the moment it is destroyed. The house may undergo several renovations during this time, and what used to be a simple cabana may end up being a full-fledged manor, but that is of no concern to us; for our purpose the house is living throughout those transformations. Only its creation and ultimate destruction matter... even though it might be better if no one happen to be in the bedroom when we tear the roof down.
Now, imagine that you are a real estate agent. You do not keep the houses you sell in your office, it's impractical; you do, however, keep their addresses!
Without the notion of lifetime, from time to time your customers will complain because the address you sent them to... was the address of a garbage dump, and not at all that lovely two-story house you had the photography of. You might also get a couple of inquiries from the police station asking why people holding onto a booklet from your office were found in a just destroyed house, the ensuing lawsuit might shut down your business.
This is obviously a risk to your business, and therefore you should seek a better solution. What if each address could be tagged with the lifetime of the house it refers to, so that you know not to send people to their death (or disappointment) ?
You may have recognized the C manual memory management strategy in that garbage dump; in C it's up to you, the real estate agent developer, to make sure that your addresses (pointers/references) always refer to living houses.
In Rust, however, the references are tagged with a special marker: 'enough; it represents the a lower-bound on the lifetime of the value referred.
When the compiler checks whether your usage of the reference is safe or not, it asks the question:
Is the value still alive ?
It does not matter whether the value will be there for a 100 years afterward, as long as it lives long 'enough for the use you have of it.
No, they refer to values as well. If it is not clear from the context how long they will live, they have to be annotated as well. It is then called a lifetime bound.
In the following example it is necessary to specify that the value, the reference is referring to, lives at least as long as the reference itself:
use std::num::Primitive;
struct Foo<'a, T: Primitive + 'a> {
a: &'a T
}
Try deleting the + 'a and the compiler will complain. This is required since T could be anything implementing Primitive.
Yes, they only refer to references, however those references can refer to primitive types. Rust is not like Java (and similar languages) that make a distinction between primitive types, which are passed by value, and more complex types (Objects in Java) that are passed by reference. Complex types can be allocated on the stack and passed by value, and references can be taken to primitive types.
For example, here is a function that takes two references to i32's, and returns a reference to the larger one:
fn bigger<'a>(a: &'a i32, b: &'a i32) -> &'a i32 {
if a > b { a } else { b }
}
It uses the lifetime 'a to communicate that the lifetime of the returned reference is the same as that of the references passed in.
When you see a lifetime annotation (e.g. 'a) in the code, there's almost always a reference, or borrowed pointer, involved.
The full syntax for borrowed pointers is &'a T. 'a is the lifetime of the referent. T is the type of the referent.
Structs and enums can have lifetime parameters. This is usually a consequence of the struct or enum containing a borrowed pointer. When you store a borrowed pointer in a struct or enum, you must explicitly state the referent's lifetime. For example, the Cow enum in the standard library contains a borrowed pointer in one of its variants. Therefore, it has a lifetime parameter that is used in the borrowed pointer's type to define the referent's lifetime.
Traits can have type bounds and also a lifetime bound. The lifetime bound indicates the largest region in which all the borrowed pointers in a concrete implementation of that trait are valid (i.e. their referents are alive). If the implementation contains no borrowed
pointers, then the lifetime is inferred as 'static. Lifetime bounds can appear in type parameter definitions, in where clauses and on trait objects.
Sometimes, you might want to define a struct or enum with a lifetime parameter, but without a corresponding value to borrow from. You can use a marker type, such as ContravariantLifetime<'a>, to ensure the lifetime parameter has the proper variance (ContravariantLifetime corresponds to the variance of borrowed pointers; without a marker, the lifetime would be bivariant, which means the lifetime could be substituted with any other lifetime... not very useful!). See an example of this use case here.