(New to rust, FYI!)
So - I've been trying to grok the concept of lifetimes in Rust. I've read the documentation, and read some blogs and SO posts on the topic. But still not quite getting it (hence, possibly a bad title to the question).
I've got a particular problem I'm trying to figure out, which I've boiled down to this small sample code (tried to make as close to a working sample as I could):
use std::collections::HashMap;
// Imagine this is actually a more complex type:
type BigComplexType = i32;
fn some_expensive_computation() -> BigComplexType {
// Imagine this takes some arguments, and takes a long time to run
return 123;
}
fn construct() -> HashMap<i32, &'static HashMap<i32, BigComplexType>> {
let mut main = HashMap::new();
let mut nested = HashMap::new();
nested.insert(1111, some_expensive_computation());
nested.insert(2222, some_expensive_computation());
// ... and lots more inserts...
main.insert(10, &nested);
main.insert(20, &nested);
// ... and lots more inserts...
// Imagine a lot more other nested HashMaps to follow here
// let mut nested2 = ...
// ...
// main.insert(..., &nested2);
// ...
return main;
}
fn main() {
construct();
}
This example is a bit trivial - in the actual code, I'm creating much more complicated and deeper structures in the construct() function.
What I'm trying to do is to create some sort of cache that holds these pre-computed values, so that they can be easily and quickly accessed else in the code. Perhaps this can all just be done in some totally different way - but I figured there must be a way to do it this way.
But, rustc complains here, because nested only exists in construct(), and once we're out of the function, it ceases to exist, and thus all references are invalid (or, that's how I understand the problem).
I've tried to introduce a 'a lifetime to the construct() function, and use that lifetime on the nested HashMap below but no dice. Got some errors, and could never fully make it work. I've tried all sorts of variants of adding lifetime annotations, but no dice.
I have a feeling I'm just not grokking some aspect of the whole concept here. Can anyone help point me in the right direction?
(I did think about gathering the nested HashMap into a vector and returning along side the main hashmap - so that then the function is returning the main HashMap, plus a vector of the nested ones - thus, the lifetime would be guaranteed, I think - but I hit some other roadblocks trying that, and have a feeling I'm over-complicating things.)
For reference, this is the error I get in compiling the above:
error[E0515]: cannot return value referencing local variable `nested`
--> lifetime_return.rs:29:12
|
19 | main.insert(10, &nested);
| ------- `nested` is borrowed here
...
29 | return main;
| ^^^^ returns a value referencing data owned by the current function
error[E0515]: cannot return value referencing local variable `nested`
--> lifetime_return.rs:29:12
|
20 | main.insert(20, &nested);
| ------- `nested` is borrowed here
...
29 | return main;
| ^^^^ returns a value referencing data owned by the current function
error: aborting due to 2 previous errors
For more information about this error, try `rustc --explain E0515`.
Any/all help would be greatly appreciated! I tried to look for similar question, but couldn't spot one - perhaps I just didn't recognize some SO post as a duplicate, as I don't fully understand the lifetime model just yet.
Your intuition for why this doesn't work is correct: nested lives only inside construct, and you try to return references to it in the hashmap that live for longer than the function. Assuming that you don't want to clone the nested maps, presumably because they're very large, you can use Rc instead as a way to have trivially cloneable references to the nested maps that keep them alive for as long as necessary:
use std::collections::HashMap;
use std::rc::Rc;
type BigComplexType = i32;
fn some_expensive_computation() -> BigComplexType {
return 123;
}
fn construct() -> HashMap<i32, Rc<HashMap<i32, BigComplexType>>> {
let mut main = HashMap::new();
let mut nested = HashMap::new();
nested.insert(1111, some_expensive_computation());
nested.insert(2222, some_expensive_computation());
let nested_rc = Rc::new(nested);
main.insert(10, Rc::clone(&nested_rc));
main.insert(20, nested_rc); // can move the Rc for the last insert
let mut nested2 = HashMap::new();
nested2.insert(3333, some_expensive_computation());
nested2.insert(4444, some_expensive_computation());
let nested2_rc = Rc::new(nested2);
main.insert(30, Rc::clone(&nested2_rc));
main.insert(40, nested2_rc);
return main;
}
fn main() {
let map = construct();
println!("{}", map[&10][&1111]); // 123
}
Playground link
Related
We know that it's easy to convert a value of type RefCell<T> to a value of type &T for use as a parameter in a function:
fn main() {
let a: RefCell<i32> = RefCell::new(0);
my_func(a.borrow().deref());
}
fn my_func(i: &i32) {}
In my scenario, the RefCells are stored in a HashMap, so they get obtained wrapped in an Option. I also want the function I pass them to have the notion of option, but I only want to pass the non-mutable reference, not the whole RefCell. We can achieve this like so:
fn main() {
let a: Option<RefCell<i32>> = Some(RefCell::new(0));
match a {
Some(ref_cell) => my_func(Some(ref_cell.borrow().deref())),
None => my_func(None)
};
}
fn my_func(i: Option<&i32>) {}
This works, but in my specific scenario, my_func takes several of these Option<&T>s as a parameter, so doing it this way means the match just gets nested for each parameter and grows exponentially. It would therefore be helpful to have someway of doing this:
fn main() {
let a: Option<RefCell<i32>> = Some(RefCell::new(0));
let c = match a {
Some(ref_cell) => Some(ref_cell.borrow().deref()), // won't compile as this borrow won't live long enough
None => None
};
my_func(c);
}
fn my_func(i: Option<&i32>) {}
So essentially I want to be able to convert from Option<RefCell<T>> to Option<&T>. I feel like this should be possible somehow but I can't figure out a way to do it. I always run into some issue of performing the .borrow() on the RefCell but it not living long enough.
You can do this using the methods on Option:
a.as_ref().map(RefCell::borrow).as_deref()
as_ref() is used to convert the Option<RefCell<_>> into a Option<&RefCell<_>> to avoid consuming it. If you already have a Option<&RefCell<_>> because you got it from hash_map.get() or similar, then you can skip this.
map(RefCell::borrow) is used to call .borrow() on the value if it exists. This will create a Option<Ref<'_, _>>.
as_deref() is the equivalent of calling .deref() on the value if it exists.
It is important to do it this way instead of trying to merge the .borrow() and .deref() in a single .map() call because this keeps the intermediate Ref<'_, _> value alive.
a.as_ref().map(|a| a.borrow().deref())
error[E0515]: cannot return reference to temporary value
--> src/main.rs:8:24
|
8 | a.as_ref().map(|a| a.borrow().deref())
| ----------^^^^^^^^
| |
| returns a reference to data owned by the current function
| temporary value created here
Also, if have multiple parameters like this and you want to split them out into variables, be sure to take the Ref<'_, _> part on its own and use .as_deref() where you use it. Again this is needed to keep the intermediate Ref<'_, _> alive:
let a_ref = a.as_ref().map(RefCell::borrow);
let b_ref = b.as_ref().map(RefCell::borrow);
let c_ref = c.as_ref().map(RefCell::borrow);
f(a_ref.as_deref(), b_ref.as_deref(), c_ref.as_deref());
I'm trying to insert a value into a HashMap based on another value in the same HashMap, like so:
use std::collections::HashMap;
fn main() {
let mut some_map = HashMap::new();
some_map.insert("a", 1);
let some_val = some_map.get("a").unwrap();
if *some_val != 2 {
some_map.insert("b", *some_val);
}
}
which gives this warning:
warning: cannot borrow `some_map` as mutable because it is also borrowed as immutable
--> src/main.rs:10:9
|
7 | let some_val = some_map.get("a").unwrap();
| -------- immutable borrow occurs here
...
10 | some_map.insert("b", *some_val);
| ^^^^^^^^ --------- immutable borrow later used here
| |
| mutable borrow occurs here
|
= note: `#[warn(mutable_borrow_reservation_conflict)]` on by default
= warning: this borrowing pattern was not meant to be accepted, and may become a hard error in the future
= note: for more information, see issue #59159 <https://github.com/rust-lang/rust/issues/59159>
If I were instead trying to update an existing value, I could use interior mutation and RefCell, as described here.
If I were trying to insert or update a value based on itself, I could use the entry API, as described here.
I could work around the issue with cloning, but I would prefer to avoid that since the retrieved value in my actual code is somewhat complex. Will this require unsafe code?
EDIT
Since previous answer is simply false and doesn't answer the question at all, there's code which doesn't show any warning (playground)
Now it's a hashmap with Rc<_> values, and val_rc contains only a reference counter on actual data (number 1 in this case). Since it's just a counter, there's no cost of cloning it. Note though, that there's only one copy of a number exists, so if you modify a value of some_map["a"], then some_map["b"] is modified aswell, since they refer to a single piece of memory. Also note, that 1 lives on stack, so you better consider turn it into Rc<Box<_>> if you plan to add many heavy objects.
use std::collections::HashMap;
use std::rc::Rc;
fn main() {
let mut some_map = HashMap::new();
some_map.insert("a", Rc::new(1));
let val_rc = Rc::clone(some_map.get("a").unwrap());
if *val_rc != 2 {
some_map.insert("b", val_rc);
}
}
Previous version of answer
Hard to tell what exactly you're looking for, but in this particular case, if you only need to check the value, then destroy the borrowed value, before you update the hashmap. A dirty and ugly code would be like this:
fn main() {
let mut some_map = HashMap::new();
some_map.insert("a", 1);
let is_ok = false;
{
let some_val = some_map.get("a").unwrap();
is_ok = *some_val != 2;
}
if is_ok {
some_map.insert("b", *some_val);
}
}
I tried to get a reference-counted Rc<Foo> from a hash map and put it into a different container (Vec<Foo>).
Thought this would work (by incrementing the reference count), but instead I got an "expected struct std::rc::Rc, found reference" error.
How do I convert an &Rc<Foo> to a Rc<Foo>?
More info:
struct Foo();
let mut foo : HashMap<usize, Rc<Foo>> = HashMap::new();
let mut bar : Vec<Rc<Foo>> = Vec::new();
foo.insert(0, Rc::new(Foo()));
if let Some(x) = foo.get(&0) {
bar.push(x); // expected struct `std::rc::Rc`, found reference
// note: expected type `std::rc::Rc<Foo>`
// found type `&std::rc::Rc<Foo>` rustc(E0308)
}
I get that the hash map returns a reference to the value it owns. But dereferencing it doesn't work: both if let Some(&x) and bar.push(*x); result in a "cannot move out of borrowed content".
Curiously, adding a type annotation changes the error to "cannot move out of an Rc":
let x : &Rc<Foo> = x;
bar.push(*x); // cannot move out of an `Rc` rustc(E0507)
I need to store a reference to the same object, and not to a copy, so I avoided the .clone() "escape hatch".
To convert an &Rc<Foo> -> Rc<Foo>, use Rc::clone(), which gives you an Rc object of your own, increasing the reference count under the hood:
let ref_to_rc: &Rc<Foo> = &Rc::new(Foo());
let new_rc: Rc<Foo> = Rc::clone(ref_to_rc);
rc.clone() is equivalent to Rc::clone(&rc), but idiomatic Rust uses the latter to make it clear that the code only increases the refcount, not performing a deep copy of the data like some other implementations of .clone() do. (Though in some scenarios involving traits you might need to revert to ref_to_rc.clone().)
The errors above were about Rust refusing to do the copy implicitly. Why is std::rc::Rc<> not Copy? has an explanation of why it behaves like that.
I suppose this question is about lifetimes in general, but I'm having difficulty with closures specifically because you can't write out their type.
This example is a bit contrived - I'm just starting to learn Rust, and this is something I've been hung up on.
This program won't compile:
fn main () {
let mut list: Vec<&Fn() -> i32> = Vec::new();
{
list.push(&|| 1);
}
}
Because:
src/main.rs:5:25: 5:24 error: borrowed value does not live long enough
src/main.rs:5 list.push(&|| 1);
^~~~
src/main.rs:2:50: 7:2 note: reference must be valid for the block suffix following statement 0 at 2:49...
src/main.rs:2 let mut list: Vec<&Fn() -> i32> = Vec::new();
src/main.rs:3
src/main.rs:4 {
src/main.rs:5 list.push(&move || 1);
src/main.rs:6 }
src/main.rs:7 }
src/main.rs:5:9: 5:26 note: ...but borrowed value is only valid for the statement at 5:8
src/main.rs:5 list.push(&|| 1);
^~~~~~~~~~~~~~~~~
src/main.rs:5:9: 5:26 help: consider using a `let` binding to increase its lifetime
src/main.rs:5 list.push(&|| 1);
^~~~~~~~~~~~~~~~~
What I gather from this error is that the closure's lifetime is limited to the
statement inside the block, but it needs to live for the entire body of main.
I know (or, I think) that passing the closure to push as a reference means that push is only borrowing the closure, and that ownership will be returned to the block. This code would work if I could just give the closure to push (i.e. if push took ownership of the closure), but since a closure isn't sized, I must pass it as a reference.
Is that right? How can I make this code work?
There are two things you are asking about:
specifying a typename for something that has no specifyable typename
letting a closure live longer than the block where it's defined.
The first issue is fixed by NOT specifying the typename, and letting rust's type inference do the work.
let mut list: Vec<_> = Vec::new();
The second issue is fixed by not trying to make the closure live longer, but by making it "by value" so you can move it. This enforces that your closure does not reference anything, but owns all the captured values.
for i in 0..10 {
list.push(move || i);
}
Now this gives us a new problem. If we add a different closure to the Vec, the types won't match. Therefore to achieve that, we need to box the closures.
fn main () {
let mut list: Vec<Box<Fn() -> i32>> = Vec::new();
for i in 0..10 {
list.push(Box::new(move|| i));
}
{
list.push(Box::new(move|| 42));
}
}
Borrows do not own the thing they point to. Your problem is that you're borrowing a temporary which is going to cease to exist right after it's borrowed because you haven't stored it anywhere. If it helps, consider that borrows don't borrow values, they borrow storage, and a temporary has only transient storage.
If you want a borrow to something to last for any given period, you must borrow from storage that will last at least that long. In this case, because you want to store the borrow in a Vec, this means that whatever storage you borrow from must outlive the Vec as well. Thus:
fn main () {
let closure;
let mut list: Vec<&Fn() -> i32> = Vec::new();
{
closure = || 1;
list.push(&closure);
}
}
Note that closure is defined before list is. In Rust, values are dropped in reverse lexical order at the end of their scope, so any variable defined after list will necessarily be dropped before it, thus leading to list containing invalid pointers.
If you want to push multiple closures, you will need a separate variable for each one.
To forestall a possible "my actual problem isn't this simple" addendum (:P): f you need to return list or in some way persist it beyond a single function call, note that there is no way to extend a borrow. In that case, what you need to do is change list to a vector of owned, boxed closures (i.e. Vec<Box<Fn() -> i32>>).
I'm having an issue writing a Lexical Analyzer in Rust where certain functions are starting to complain about simple snippets that would otherwise appear harmless. This is starting to become an annoyance as the error messages are not helping me pinpoint the cause of my problems and so this is the second time I'm reaching out on the same program in the same week (previous question here).
I have read the book, I've understood everything I could from it. I've also watched/read numerous other articles and videos discussing lifetimes (both explicit and implicit) and for the most part the concept behind borrowing and moving make perfect sense, except in cases like the following:
My lexer has a next function who's purpose is to peek ahead at the next character and return it.
struct Lexer<'a> {
src: str::Chars<'a>,
buf: String,
// ... not important
}
impl<'a> Lexer<'a> {
// ... not relevant
// originally this -> Option<&char> which caused it's own slew of problems
// that I thought dereferencing the character would solve.
fn next(&self) -> Option<char> {
let res = self.src.peekable().peek();
// convert Option<&char> to Option<char>
match res {
Some(ref c) => Some(*c.clone()),
None => None
}
}
// ... not relevant
}
The error that I'm getting when doing this is:
error: borrowed value does not live long enough
let res = self.src.peekable().peek();
^~~~~~~~~~~~~~~~~~~
What I understand from this error is that the value from peekable() is not living long enough, which kind of makes sense to me. I'm only referencing the return in that line and calling another function which I imagine is returning a pointer to the character at the next location with the iterator. My naive solution to this was:
let mut peeker = self.src.peekable();
let res = peeker.peek();
If I implement this solution, I see a different error which also does not make sense to me:
error: cannot move out of borrowed content
let mut peeker = self.src.peekable();
^~~~
I'm not quite sure what's moving self out of a borrowed context here (I know it's borrowed from &self but not sure what's moving it out of the borrowed context.
EDIT
I proposed a question with details that were wildly inaccurate. The portion of the post that contained those details has been updated with actual facts - I mixed up two different situations where I had encountered a similar error (similar to me, at least).
Let's start with the second error.
As mentioned in the comments, Iterator::peekable is an iterator adapter that consumes the iterator it is going to make peekable. A small reproduction:
let values = [1,2,3];
let mut iterator = values.iter();
iterator.peekable(); // Consumes `iterator`
iterator.next(); // Not available anymore!
You can use Iterator::by_ref to get a reference that can then itself be consumed. Note that the underlying iterator will still be advanced!
let values = [1,2,3];
let mut iterator = values.iter();
iterator.by_ref().peekable();
iterator.next();
In your case, you are trying to consume the value out of a borrowed struct (via &self), which has the specific error cannot move out of borrowed content.
Let's look at your original error:
let values = [1,2,3];
let next = values.iter().peekable().peek();
The problem here is that peek returns a reference. This makes sense, as we don't know if the item we are iterating over is Copyable. However, that reference has to have somewhere to live. That place to live is the Peekable iterator itself! Peekable allocates enough space to store the "next" element. When you call peek, it advances the underlying iterator, stores the value, then returns the reference. Check out the function signature for peek to see this captured in code:
fn peek(&mut self) -> Option<&I::Item>
You can re-add the lifetimes to be explicit:
fn peek<'a>(&'a mut self) -> Option<&'a I::Item>
In the one-line version, you create and then destroy the Peekable, so there's nowhere for the value to live, so the reference dies in the same statement.