I need to call a function with a lifetime specified on self from a static context like
impl <'w> World<'w> {
pub fn test_with_lifetime(&'w mut self) {
println!("test with lifetime");
}
pub fn test(&mut self) {
println!("test");
}
}
pub fn main(){
let mut world:World = World::new();
let world_rc:Rc<RefCell<World<'static>>> = Rc::new(RefCell::new(world));
let world_in_closure = Rc::clone(&world_rc);
let bx = Box::new(move ||{
if let Ok(mut borrowed_world) = world_in_closure.try_borrow_mut() {
borrowed_world.test_with_lifetime();
}
});
(bx)();
}
and if fails with 'borrowed value does not live long enough...' error.
So I have two questions:
what's difference between &self and &'w self in function definition? Don't they both effectively mean that object lives in the caller's context?
is there a way to make it compile?
playground
pub fn test(&mut self) is not a shortcut of pub fn test(&'w mut self). Rather, it is a shortcut for pub fn test<'a>(&'a mut self).
The difference is where the lifetime is defined: 'w is a part of the type. It is decided by the one who create the instance. And by taking &'w mut self, you usually mean that this function can be called zero (if the instance does not live as long as 'w) or one (if it does) times. More than that it cannot, because because 'w is contained within the type it must be shorter than or equal to the lifetime of the instance. And after you call it with 'w one time, it is mutably borrowed for the rest of its life.
pub fn test<'a>(&'a mut self), on the other hand, means that the caller of the function decides about the lifetime. It can be (and usually is) as short as the whole call, and doesn't need to be the whole instance's lifetime.
Related
I'm having difficulties understanding lifetime parameters in the following code snippet.
struct C {
data: Vec<u32>,
cols: usize
}
trait M<'s> {
fn get(&'s self, r: usize, c: usize) -> u32;
fn get_mut(&'s mut self, r: usize, c: usize) -> &'s mut u32;
}
impl<'s> M<'s> for C {
fn get(&'s self, r: usize, c: usize) -> u32 {
return self.data[self.cols*r+c];
}
fn get_mut(&'s mut self, r: usize, c: usize) -> &'s mut u32 {
return &mut self.data[self.cols*r+c];
}
}
#[cfg(test)]
mod tests {
use super::*;
fn create() -> C {
let data = vec![0u32,1u32,2u32,3u32,4u32,5u32];
return C{data, cols: 3};
}
fn select<'s, 'r: 's>(data: &'r mut dyn M<'s>) {
let mut _val: u32 = 0;
for r in 0..2 {
for c in 0..3 {
_val += *data.get_mut(r,c);
}
}
}
#[test]
fn test_select() {
let mut data = create();
select(&mut data);
}
}
The code snippet does not compile, because it complains that *data is borrowed multiple times in the function fn select<'s, 'r: 's>(data: &'r mut dyn M<'s>) {} when calling get_mut (once in every loop iteration). Even safeguarding the questionable line with curly braces (and thus creating a new context) does not help. My expectation (in both cases) would be, that the mutable borrow of &mut data should end right after the execution of that line.
On the other hand, when I remove all lifetime parameters, everything works as expected.
Can anyone explain what's the difference between the two versions (with and without explicit lifetimes)?
I've also tried to find information about additional lifetime parameters for traits, in particular specifying their meaning, but I have found none. So I assume, that they are just a declaration of the used labels inside the trait. But if that is so, then I would assume that leaving out the lifetime parameters completely and applying the eliding rules would lead to the same result.
There are two things to consider. The first is when you use a generic lifetime for a function, that lifetime must be larger than the life of the function call simply by construction. And the second is since the lifetime self is tied to the lifetime parameter of the trait, when you call .get_mut(), data is borrowed for the lifetime of 's. Combining those two principles, data is borrowed for longer than the function call so you can't call it again (its already mutably borrowed).
On the other hand, when I remove all lifetime parameters, everything works as expected. Can anyone explain what's the difference between the two versions (with and without explicit lifetimes)?
Without a generic lifetime on M, the methods will behave as if defined as so:
impl M for C {
fn get<'a>(&'a self, r: usize, c: usize) -> u32 {
return self.data[self.cols * r + c];
}
fn get_mut<'a>(&'a mut self, r: usize, c: usize) -> &'a mut u32 {
return &mut self.data[self.cols * r + c];
}
}
Thus there is no lifetime associated with the trait; the lifetimes given and returned from the function are generic only to those method calls. And since the compiler can choose a new lifetime 'a for each call and it will always pick the shorted lifetime to satisfy its usage, you can then call data.get_mut() multiple times without worry. And I'll be honest, having the lifetime on the trait didn't make much sense with the original code; as mentioned, the code works with all lifetime annotations removed: playground.
I want to write a program that will write a file in 2 steps.
It is likely that the file may not exist before the program is run. The filename is fixed.
The problem is that OpenOptions.new().write() can fail. In that case, I want to call a custom function trycreate(). The idea is to create the file instead of opening it and return a handle. Since the filename is fixed, trycreate() has no arguments and I cannot set a lifetime of the returned value.
How can I resolve this problem?
use std::io::Write;
use std::fs::OpenOptions;
use std::path::Path;
fn trycreate() -> &OpenOptions {
let f = OpenOptions::new().write(true).open("foo.txt");
let mut f = match f {
Ok(file) => file,
Err(_) => panic!("ERR"),
};
f
}
fn main() {
{
let f = OpenOptions::new().write(true).open(b"foo.txt");
let mut f = match f {
Ok(file) => file,
Err(_) => trycreate("foo.txt"),
};
let buf = b"test1\n";
let _ret = f.write(buf).unwrap();
}
println!("50%");
{
let f = OpenOptions::new().append(true).open("foo.txt");
let mut f = match f {
Ok(file) => file,
Err(_) => panic!("append"),
};
let buf = b"test2\n";
let _ret = f.write(buf).unwrap();
}
println!("Ok");
}
The question you asked
TL;DR: No, you cannot return a reference to a variable that is owned by a function. This applies if you created the variable or if you took ownership of the variable as a function argument.
Solutions
Instead of trying to return a reference, return an owned object. String instead of &str, Vec<T> instead of &[T], T instead of &T, etc.
If you took ownership of the variable via an argument, try taking a (mutable) reference instead and then returning a reference of the same lifetime.
In rare cases, you can use unsafe code to return the owned value and a reference to it. This has a number of delicate requirements you must uphold to ensure you don't cause undefined behavior or memory unsafety.
See also:
Proper way to return a new string in Rust
Return local String as a slice (&str)
Why can't I store a value and a reference to that value in the same struct?
Deeper answer
fjh is absolutely correct, but I want to comment a bit more deeply and touch on some of the other errors with your code.
Let's start with a smaller example of returning a reference and look at the errors:
fn try_create<'a>() -> &'a String {
&String::new()
}
Rust 2015
error[E0597]: borrowed value does not live long enough
--> src/lib.rs:2:6
|
2 | &String::new()
| ^^^^^^^^^^^^^ temporary value does not live long enough
3 | }
| - temporary value only lives until here
|
note: borrowed value must be valid for the lifetime 'a as defined on the function body at 1:15...
--> src/lib.rs:1:15
|
1 | fn try_create<'a>() -> &'a String {
| ^^
Rust 2018
error[E0515]: cannot return reference to temporary value
--> src/lib.rs:2:5
|
2 | &String::new()
| ^-------------
| ||
| |temporary value created here
| returns a reference to data owned by the current function
Is there any way to return a reference from a function without arguments?
Technically "yes", but for what you want, "no".
A reference points to an existing piece of memory. In a function with no arguments, the only things that could be referenced are global constants (which have the lifetime &'static) and local variables. I'll ignore globals for now.
In a language like C or C++, you could actually take a reference to a local variable and return it. However, as soon as the function returns, there's no guarantee that the memory that you are referencing continues to be what you thought it was. It might stay what you expect for a while, but eventually the memory will get reused for something else. As soon as your code looks at the memory and tries to interpret a username as the amount of money left in the user's bank account, problems will arise!
This is what Rust's lifetimes prevent - you aren't allowed to use a reference beyond how long the referred-to value is valid at its current memory location.
See also:
Is it possible to return either a borrowed or owned type in Rust?
Why can I return a reference to a local literal but not a variable?
Your actual problem
Look at the documentation for OpenOptions::open:
fn open<P: AsRef<Path>>(&self, path: P) -> Result<File>
It returns a Result<File>, so I don't know how you'd expect to return an OpenOptions or a reference to one. Your function would work if you rewrote it as:
fn trycreate() -> File {
OpenOptions::new()
.write(true)
.open("foo.txt")
.expect("Couldn't open")
}
This uses Result::expect to panic with a useful error message. Of course, panicking in the guts of your program isn't super useful, so it's recommended to propagate your errors back out:
fn trycreate() -> io::Result<File> {
OpenOptions::new().write(true).open("foo.txt")
}
Option and Result have lots of nice methods to deal with chained error logic. Here, you can use or_else:
let f = OpenOptions::new().write(true).open("foo.txt");
let mut f = f.or_else(|_| trycreate()).expect("failed at creating");
I'd also return the Result from main. All together, including fjh's suggestions:
use std::{
fs::OpenOptions,
io::{self, Write},
};
fn main() -> io::Result<()> {
let mut f = OpenOptions::new()
.create(true)
.write(true)
.append(true)
.open("foo.txt")?;
f.write_all(b"test1\n")?;
f.write_all(b"test2\n")?;
Ok(())
}
Is there any way to return a reference from a function without arguments?
No (except references to static values, but those aren't helpful here).
However, you might want to look at OpenOptions::create. If you change your first line in main to
let f = OpenOptions::new().write(true).create(true).open(b"foo.txt");
the file will be created if it does not yet exist, which should solve your original problem.
You can not return a reference pointing to a local variable. You have two alternatives, either return the value or use a static variable.
Here is why:
References are pointers to memory locations. Once functions are executed, local variables are popped off the execution stack and resources are de-allocated. After that point, any reference to a local variable will be pointing to some useless data. Since it is de-allocated, it is not in our program's possession any more and OS may have already given it to another process and our data may have been overwritten.
For the following example, x is created when the function runs and dropped off when the function completes executing. It is local to the function and lives on this particular function's stack. Function's stack holds local variables.
When run is pop off the execution stack, any reference to x, &x, will be pointing to some garbage data. That is what people call a dangling pointer. The Rust compiler does not allow to use dangling pointers since it is not safe.
fn run() -> &u32 {
let x: u32 = 42;
return &x;
} // x is dropped here
fn main() {
let x = run();
}
So, that is why we can not return a reference to a local variable. We have two options: either return the value or use a static variable.
Returning the value is the best option here. By returning the value, you will be passing the result of your calculation to the caller, in Rust's terms x will be owned by the caller. In our case it is main. So, no problem there.
Since a static variable lives as long as the process runs, its references will be pointing to the same memory location both inside and outside the function. No problem there either.
Note: #navigaid advises using a box, but it does not make sense because you are moving readily available data to heap by boxing it and then returning it. It does not solve the problem, you are still returning the local variable to the caller but using a pointer when accessing it. It adds an unnecessary indirection due to de-referencing hence incurring additional cost. Basically you will be using & for the sake of using it, nothing more.
This is an elaboration on snnsnn's answer, which briefly explained the problem without being too specific.
Rust doesn't allow return a reference to a variable created in a function. Is there a workaround? Yes, simply put that variable in a Box then return it. Example:
fn run() -> Box<u32> {
let x: u32 = 42;
return Box::new(x);
}
fn main() {
println!("{}", run());
}
code in rust playground
As a rule of thumb, to avoid similar problems in Rust, return an owned object (Box, Vec, String, ...) instead of reference to a variable:
Box<T> instead of &T
Vec<T> instead of &[T]
String instead of &str
For other types, refer to The Periodic Table of Rust Types to figure out which owned object to use.
Of course, in this example you can simply return the value (T instead of &T or Box<T>)
fn run() -> u32 {
let x: u32 = 42;
return x;
}
Yes! But you have to find a way to extend the lifetime. One way to do that is to provide mutable reference to a dummy/default value (&mut T) to the function and then fill/replace the value in the function and then return a reference to that value (&T). This way you can specify the lifetime so that the returned reference get the lifetime of a value outside the function.
Examples:
//&mut T -> &T
fn example2<'a>(life: &'a mut Vec<i32>) -> &'a Vec<i32> {
*life = vec![1, 2, 3, 4];
life
}
fn test_example2() {
//Could also use Vec::new()
let mut life = Vec::default();
let res = example2(&mut life);
println!("{:?}", res)
}
fn test2_example2() {
let life = &mut Vec::default();
let res = example2(life);
println!("{:?}", res)
}
//shows real use case
fn get_check_test_slices2<'a>(
lifetime: &'a mut Vec<usize>,
limit: usize,
) -> impl Iterator<Item = (&'a [usize], &'a [usize])> + 'a {
// create a list of primes using a simple primes sieve
*lifetime = primes1_iter_bitvec(limit).collect::<Vec<_>>();
// iterate through pairs of sub slices without copying the primes vec
// slices will be used to check that a complicated sieve is correct
all_test_check_slices(lifetime)
}
Edit:
How is that different from just giving &mut T to the function? (asked by Chayim Friedman (see old solution below)):
It's basically the same... I lost against the borrow checker previously, and that is why I didn't just use &mut T. But after renewed battles I have finally managed to just use &mut T. Thank you for the insightful question.
Old solution:
My solution works by creating a default value before calling the function, which the function later replaces/fills and returns a reference to.
/// Used to return references to values created in a function.
/// fn example<'a>(lt:& 'a mut LifeExtender<Vec<i32>>) -> &'a Vec<i32> {
/// lt.set(vec![1,2,3,4]);
/// lt.get()
/// }
pub struct LifeExtender<T> {
value: T,
}
impl<T> Default for LifeExtender<T>
where
T: Default,
{
/// using T default.
fn default() -> Self {
Self {
value: T::default(),
}
}
}
impl<T> LifeExtender<T> {
/// If T doesn't have default.
pub fn new(value: T) -> Self {
Self { value }
}
/// set value to be returned by reference
pub fn set(&mut self, new_value: T) {
self.value = new_value;
}
/// Get a reference with lifetime self.
pub fn get<'a>(&'a self) -> &'a T {
&self.value
}
/// Get a mut reference with lifetime self.
pub fn get_mut<'a>(&'a mut self) -> &'a mut T {
&mut self.value
}
}
fn example<'a>(life: &'a mut LifeExtender<Vec<i32>>) -> &'a Vec<i32> {
let local_value = vec![1, 2, 3, 4];
life.set(local_value);
life.get()
}
//prints: [1,2,3,4]
pub fn test_example() {
let mut life = LifeExtender::default();
let res = example(&mut life);
println!("{:?}", res);
}
//Real example code snippet, where I used this solution:
fn get_check_slices2<'a>(
lifetime: &'a mut LifeExtender<Vec<usize>>,
limit: usize,
) -> impl Iterator<Item = (&'a [usize], &'a [usize])> + 'a {
lifetime.set(primes1_iter_bitvec(limit).collect::<Vec<_>>());
all_test_check_slices(lifetime.get())
}
In the example below, I have a lifetime constraint for the one member. The first call to set_one works, however a second call doesn't:
struct One {
name: String,
}
struct Two<'a> {
one: Option<&'a One>,
}
fn set_one<'a>(two: &'a mut Two<'a>, one: &'a mut One) {
one.name = "something".to_owned();
two.one = Some(one);
}
fn main() {
let mut one = One { name: "".to_owned() };
let mut two = Two { one: None };
set_one(&mut two, &mut one);
set_one(&mut two, &mut one); // error: "two" already borrowed!
}
Playground here.
My understanding is that after the first call to set_one the ownership is given back to the original caller, so when the function returns the borrow for two is gone. But apparently I'm missing something.
What's wrong with the lifetimes of the example above?
For this to make sense you need to recognize that there are 3 lifetimes involved in the calling of set_one
fn set_one<'a, 'b, 'c>(two: &'a mut Two<'b>, one: &'c mut One)
The lifetime of the reference two the lifetime of the reference one and the lifetime of the reference one within Two. These are 'a, 'b and 'c respectively in the signature above.
When you write the signature as
fn set_one<'a>(two: &'a mut Two<'a>, one: &'a mut One)
You are forcing the 3 lifetimes to be the same. And to put it simply this forces these lifetimes to take the 'bigger' lifetime.
Examining the main function:
fn main() {
let mut one = One { name: "".to_owned() }; |
let mut two = Two { one: None }; | Lifetime of 'a
|
set_one(&mut two, &mut one); |
set_one(&mut two, &mut one); |
}
We can see that implicitly now the struct two is a Two<'a> with the lifetime 'a as marked in the code.
Then when we call set_one we force the lifetime of the three parameters to be 'a.
You could declare the lifetimes as:
fn set_one<'a, 'b>(two: &'a mut Two<'b>, one: &'a mut One)
However, this has a problem which is at the root of what you want to do:
fn set_one<'a>(two: &'a mut Two<'a>, one: &'a mut One) {
one.name = "something".to_owned();
two.one = Some(one); // <--- This
}
When you do this two.one = Some(one); you need the reference within Two to live as long as one. Therefore, there is no escaping the fact that you can't call set_one twice because you will need 2 mutable borrows that will live as long as the scope of one in the main function.
How do you pass a closure to an object-safe trait method or otherwise via dynamic dispatch?
I have answered this myself, but the answer leaves something wanting: FnOnce closures must be boxed since they are not sized and must be consumed on use (thus cannot be passed by reference).
You can pass it without using Box:
fn pass_fn_once(do_thing: &dyn FnOnce()) {
//do_thing();
}
fn main() {
pass_fn_once(&|| println!("Hello!"));
}
However, you won't be able to actually call do_thing in pass_fn_once, since it is only borrowed - calling an FnOnce consumes it.
If they are Boxed, you can be sure that pass_fn_once took ownership so that the closure can be thrown away afterwards.
Fn and FnMut closures can be passed by reference, as follows. FnOnce can be passed by reference but not called unless it is owned, e.g. by a Box.
fn pass_fn(get_num: &dyn Fn() -> i32) {
let _num = get_num();
}
fn pass_fn_mut(set_num: &mut dyn FnMut(i32)) {
set_num(6);
}
fn pass_fn_once(do_thing: Box<dyn FnOnce()>) {
do_thing();
}
fn main() {
pass_fn(&|| 2);
let mut x = 1;
pass_fn_mut(&mut |y| x = y);
pass_fn_once(Box::new(|| println!("Hello!")));
}
In Rust, I get the following error:
<anon>:14:9: 14:17 error: `mystruct` does not live long enough
<anon>:14 mystruct.update();
^~~~~~~~
<anon>:10:5: 17:6 note: reference must be valid for the lifetime 'a as defined on the block at 10:4...
<anon>:10 {
<anon>:11 let initial = vec![Box::new(1), Box::new(2)];
<anon>:12 let mystruct = MyStruct { v : initial, p : &arg };
<anon>:13
<anon>:14 mystruct.update();
<anon>:15
...
<anon>:12:59: 17:6 note: ...but borrowed value is only valid for the block suffix following statement 1 at 12:58
<anon>:12 let mystruct = MyStruct { v : initial, p : &arg };
<anon>:13
<anon>:14 mystruct.update();
<anon>:15
<anon>:16 mystruct
<anon>:17 }
error: aborting due to previous error
for the following code:
struct MyStruct<'a>
{
v : Vec<Box<i32>>,
p : &'a i32
}
impl<'a> MyStruct<'a>
{
fn new(arg : &'a i32) -> MyStruct<'a>
{
let initial = vec![Box::new(1), Box::new(2)];
let mystruct = MyStruct { v : initial, p : &arg };
mystruct.update();
mystruct
}
fn update(&'a mut self)
{
self.p = &self.v.last().unwrap();
}
}
fn main() {
let x = 5;
let mut obj = MyStruct::new(&x);
}
(Playground)
I don't understand why mystruct does not live enough. If I comment out the mystruct.update() line it works fine though. What's more is, if I comment out the body of update the code still fails. Why does calling an empty function which borrows a mutable self changes things?
I don't understand which reference is the one the error talks about. Can somebody explain this?
The reference this error talks about is the one which is implicitly created when you call update(). Because update() takes &'a mut self, it means that it accepts a value of type &'a mut MyStruct<'a>. It means that in theory you should call update() like this:
(&mut mystruct).update();
It would be very inconvenient to write this everywhere, and so Rust is able to automatically insert necessary &s, &muts and *s in order to call a method. This is called "autoreference", and the only place it happens is method invocations/field access.
The problem is the definition of update() method:
impl<'a> MyStruct<'a> {
...
fn update(&'a mut self) { ... }
...
}
Here you are requesting that update() receives the value it is called at via a reference with lifetime 'a, where 'a is the lifetime of the reference stored in the structure.
However, when you have a structure value you're calling this method on, there should be already a reference to i32 you stored in this structure. Hence the lifetime of the structure value is strictly smaller than the lifetime designated by the lifetime parameter, so it is just impossible to construct &'a mut MyStruct<'a> with local variables (as in your case).
The solution is to use &mut self instead of &'a mut self:
fn update(&mut self) { ... }
// essentially equivalent to
fn update<'b>(&'b mut self) where 'a: 'b { ... }
// `'b` is a fresh local lifetime parameter
This way the lifetime of the structure in this method call is not tied to the reference this structure contains and can be smaller.
More in-depth explanation follows below.
By itself your definition is not nonsense. For example:
struct IntRefWrapper<'a> {
value: &'a i32
}
static X: i32 = 12345;
static Y: IntRefWrapper<'static> = IntRefWrapper { value: &X };
impl<'a> IntRefWrapper<'a> {
fn update(&'a self) { ... }
}
Y.update();
Here update() invocation won't cause compilation errors because both lifetimes (of Y and of X, reference to which is contained in Y) are 'static.
Let's consider your example, for comparison:
impl<'a> MyStruct<'a> {
fn new(arg : &'a i32) -> MyStruct<'a> {
let initial = vec![Box::new(1), Box::new(2)];
let mystruct = MyStruct { v : initial, p : &arg };
mystruct.update();
mystruct
}
}
Here we have a lifetime parameter, 'a, which is supplied by the caller of the function. For example, the caller could call this function with a static reference:
static X: i32 = 12345;
MyStruct::new(&X); // here &X has static lifetime
However, when update() method is invoked, mystruct lifetime is bounded by the block it is called in:
{
let initial = vec![Box::new(1), Box::new(2)];
let mystruct = MyStruct { v : initial, p : &arg }; // +
// |
mystruct.update(); // |
// |
mystruct // |
}
Naturally, the borrow checker can't prove that this lifetime is the same as the lifetime provided by the caller (and for any possible "external" lifetime it is indeed impossible for them to match), so it throws an error.
When update is defined like this:
fn update(&mut self) { ... }
// or, equivalently
fn update<'b>(&'b mut self) where 'a: 'b { ... }
then when you call it, it is no longer required that the value you call this method on must live exactly as long as 'a - it is sufficient for it to live for any lifetime which is smaller than or equal to 'a - and the lifetime inside the function perfectly matches these requirements. Thus you can call such method on your value, and the compiler won't complain.
Additionally (as noticed in the comments) the following line is indeed invalid and there is no way around it:
self.p = &self.v.last().unwrap();
The borrow check fails here because you're trying to store a reference with lifetime of the structure into the structure itself. In general this can't be done because it has nasty soundness issues. For example, suppose you were indeed able to store this reference into the structure. But now you can't mutate Vec<Box<i32>> in the structure because it may destroy an element which the previously stored references points at, making the code memory unsafe.
It is impossible to check for such things statically, and so it is disallowed on the borrow checking level. In fact, it is just a nice consequence of general borrow checking rules.