Rust async function parameters and lifetimes - rust

I'd like to understand why test1 causes an error but test2 compiles.
It seems like rust is being clever, and realising that when the .await is called directly on the async function result it knows to keep the parameter around for execution of the future but when the async is called on a separate line it can't do this.
Would love to have a link to the relevant functionality that makes this work to learn the details.
async fn do_async_thing(s: &String) {
println!("{s}");
}
fn get_string() -> String {
"sf".to_string()
}
#[tokio::test]
async fn test1() {
let a = do_async_thing(&get_string());
a.await;
}
#[tokio::test]
async fn test2() {
do_async_thing(&get_string()).await;
}
The error
error[E0716]: temporary value dropped while borrowed
--> crates/dynamo/src/error.rs:11:29
|
11 | let a = do_async_thing(&get_string());
| ^^^^^^^^^^^^ - temporary value is freed at the end of this statement
| |
| creates a temporary value which is freed while still in use
12 | a.await;
| - borrow later used here
|
= note: consider using a `let` binding to create a longer lived value

It is not directly to do with async, its because the future returned from do_async_thing holds the string reference.
You can create your own future with the same result
struct DoAsyncThingFuture<'a> {
s: &'a String
}
impl<'a> Future for DoAsyncThingFuture<'a> {
type Output = ();
fn poll(self: std::pin::Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll<Self::Output> {
println!("{}", self.s);
Poll::Ready(())
}
}
fn do_async_thing(s: &String) -> DoAsyncThingFuture {
DoAsyncThingFuture {
s
}
}
And even get the same result without a future
fn do_sync_thing(s: &String) -> &String {
s
}
Attempting to use the return value from either of these functions will give the same error. This happens the return value of get_string does not have an owner so it is dropped after the call to do_sync_thing witch means the return reference is dangling. So as why one works and the other does not:
let a = do_sync_thing(&get_string());
println!("{}", a);
//Same as
let _temp_value = get_string();
let a = do_async_thing(&_temp_value);
drop(_temp_value);
println!("{}", a);
vs
println!("{}", do_sync_thing(&get_string()));
//Same as
let _temp_value = get_string();
println!("{}", do_async_thing(&_temp_value));
drop(_temp_value);

Related

Hashmap multiple mutable borrow issue after reference drop

I am trying to pass around a HashMap which stores values through a set of nested enums/structs. The problem of multiple mutability happens during iteration, even all references should be dropped.
The general idea is to have a vector of values, iterate through them and simplify them, keeping track of them within the HashMap. There are two stages of simplification.
The general flow looks something like
run(Vec<ComplexVal>)
-for each val->
val.fix_complex(holder)
-for each `smp` SimpleVal in val->
basicval = Simplifier::step(smp, holder)
holder.insert("name", basicval)
But the problem is that the holder is borrowed mutably in each stage, and there isn't supposed to be any reference from the ComplexVal to the holder and since the borrowchecker doesn't like multiple borrows, it fails.
Full playground snippet: here
It happens in this snippet:
pub fn run(&mut self, mut vals: Vec<ComplexVal>) {
let mut holder = Holder{hold:HashMap::new()};
// .. setup holder code omitted
let len = vals.len();
for _ in 0..len {
let mut val = vals.remove(0); // remove from vec, should drop after running
println!("Running {:?}", val);
match val {
ComplexVal::Cmplx1(mut c) => {
c.fix_complex(&mut holder)
},
//... more cases of different types of values omitted for simplicity
}
// val *should* be dropped here, and therefore the mutable borrow of holder?
}
println!("Holder: {:?}", holder);
}
}
The only thing I can think of is that it somehow is related to the BasicVal::Ref(&BasicVal) value when created.
I need to return a reference of type &BasicVal so I can't use a regular fn() -> &BasicVal as the reference would be dangling, so I pass a ret value which is to be modified and used as the storage for the return value.
I have also tried just returning the enum BasicVal::Ref(&BasicVal), but run into the same mutability issues.
The example below is a much more simple version which (sort of) demonstrates the same error, just thought I'd include this context in case someone has another idea on how to implement this which wouldn't have these issues
Code (edited)
Updated playground link
Edit: I made a mistake in not needing the lifetimes of both holder and ret to explicitly be the same, so I have made an updated example for it
use std::borrow::BorrowMut;
///////////////////////////////
use std::cell::{RefCell, RefMut};
use std::collections::HashMap;
#[derive(Debug)]
enum BasicVal<'a> {
Ref(&'a BasicVal<'a>),
Val1(BasicStruct),
}
#[derive(Debug)]
struct Holder<'b> {
hold: HashMap<String, RefCell<BasicVal<'b>>>,
}
#[derive(Debug)]
struct BasicStruct {
val: i32,
}
impl<'a> BasicVal<'a> {
pub fn empty() -> Self { BasicVal::Val1(BasicStruct { val: 0 }) }
}
// must match sig of modify_val_ref
fn modify_val<'f>(holder: &'f mut Holder<'f>, mut ret: RefMut<BasicVal<'f>>) {
*ret = BasicVal::Val1(BasicStruct { val: 5 });
}
// must match sig of modify_val
fn modify_val_ref<'f>(holder: &'f mut Holder<'f>, mut ret: RefMut<BasicVal<'f>>) {
ret = holder.hold.get("reference_val").unwrap().borrow_mut();
}
fn do_modify<'f>(holder: &'f mut Holder<'f>) {
let mut v = RefCell::new(BasicVal::empty());
println!("Original {:?}", v);
modify_val(holder, v.borrow_mut());
holder.hold.insert("Data".to_string(), v);
println!("Modified {:?}", holder.hold.get("Data"));
}
pub fn test_dropborrow() {
let mut holder = Holder { hold: HashMap::new() };
holder.hold.insert(
"reference_val".to_string(),
RefCell::new(BasicVal::Val1(BasicStruct { val: 8 })),
);
do_modify(&mut holder);
}
pub fn main() {
test_dropborrow();
}
Edit: Using just the holder for a temp return value gives me a multiple mutable borrow issue, so that workaround doesn't work. I have also tried it with a RefCell with the same issue.
fn modify_val<'f>(holder: &'f mut Holder<'f>) {
holder.hold.insert("$return".to_string(), BasicVal::Val1(BasicStruct{val: 5}));
}
fn do_modify<'f>(holder: &'f mut Holder<'f>) {
modify_val(holder);
let mut v = holder.hold.remove("$return").unwrap();
holder.hold.insert("Data".to_string(), v);
println!("Modified {:?}", v);
}
Error:
935 | fn do_modify<'f>(holder: &'f mut Holder<'f>) {
| -- lifetime `'f` defined here
936 |
937 | modify_val(holder);
| ------------------
| | |
| | first mutable borrow occurs here
| argument requires that `*holder` is borrowed for `'f`
938 | let mut v = holder.hold.remove("$return").unwrap();
| ^^^^^^^^^^^ second mutable borrow occurs here
Any help is greatly appreciated!!!
Figured it out, essentially the BasicVal<'a> was causing Holder to mutably borrow itself in successive iterations of the loop, so removing the lifetime was pretty much the only solution

rust E0597: borrowed value does not live lnog enough

I am trying to rewrite an algorithm from javascript to rust. In the following code, I get borrowed value does not live long enough error at line number 17.
[dependencies]
scraper = "0.11.0"
use std::fs;
fn get_html(fname: &str) -> String {
fs::read_to_string(fname).expect("Something went wrong reading the file")
}
pub mod diff_html {
use scraper::{element_ref::ElementRef, Html};
pub struct DiffNode<'a> {
node_ref: ElementRef<'a>,
}
impl<'a> DiffNode<'a> {
fn from_html(html: &str) -> Self {
let doc = Self::get_doc(&html);
let root_element = doc.root_element().to_owned();
let diffn = Self {
node_ref: root_element,
};
diffn
}
fn get_doc(html: &str) -> Html {
Html::parse_document(html).to_owned()
}
}
pub fn diff<'a>(html1: &str, _html2: &str) -> DiffNode<'a> {
let diff1 = DiffNode::from_html(&html1);
diff1
}
}
fn main() {
//read strins
let filename1: &str = "test/test1.html";
let filename2: &str = "test/test2.html";
let html1: &str = &get_html(filename1);
let html2: &str = &get_html(filename2);
let diff1 = diff_html::diff(html1, html2);
//write html
//fs::write("test_outs/testx.html", html1).expect("unable to write file");
//written output file.
}
warning: unused variable: `diff1`
--> src\main.rs:43:9
|
43 | let diff1 = diff_html::diff(html1, html2);
| ^^^^^ help: if this is intentional, prefix it with an underscore: `_diff1`
|
= note: `#[warn(unused_variables)]` on by default
error[E0597]: `doc` does not live long enough
--> src\main.rs:17:32
|
14 | impl<'a> DiffNode<'a> {
| -- lifetime `'a` defined here
...
17 | let root_element = doc.root_element().to_owned();
| ^^^--------------------------
| |
| borrowed value does not live long enough
| assignment requires that `doc` is borrowed for `'a`
...
22 | }
| - `doc` dropped here while still borrowed
I want a detailed explanation/solution if possible.
root_element which is actually an ElementRef has reference to objects inside doc, not the actual owned object. The object doc here is created in from_html function and therefore owned by the function. Because doc is not returned, it is dropped / deleted from memory at the end of from_html function block.
ElementRef needs doc, the thing it is referencing to, to be alive when it is returned from the memory.
pub mod diff_html {
use scraper::{element_ref::ElementRef, Html};
pub struct DiffNode<'a> {
node_ref: ElementRef<'a>,
}
impl<'a> DiffNode<'a> {
fn from_html(html: &'a scraper::html::Html) -> Self {
Self {
node_ref: html.root_element(),
}
}
}
pub fn diff<'a>(html1_string: &str, _html2_string: &str) {
let html1 = Html::parse_document(&html1_string);
let diff1 = DiffNode::from_html(&html1);
// do things here
// at the end of the function, diff1 and html1 is dropped together
// this way the compiler doesn't yell at you
}
}
More or less you need to do something like this with diff function to let the HTML and ElementRef's lifetime to be the same.
This behavior is actually Rust's feature to guard values in memory so that it doesn't leak or reference not referencing the wrong memory address.
Also if you want to feel like operating detachable objects and play with reference (like java, javascript, golang) I suggest reading this https://doc.rust-lang.org/book/ch15-05-interior-mutability.html

Create an iterator and put it into a new struct without bothering the borrow-checker [duplicate]

This question already has answers here:
Is there any way to return a reference to a variable created in a function?
(5 answers)
Closed 3 years ago.
I am trying to create a lexical analyzer which uses itertools::PutBack to make an iterator over the characters in a String. I intend to store the pushback iterator in a struct and delegate methods to it so that I can categorize the characters by an enum, which will then be passed to a state machine at the core of the lexical analyzer (not yet written).
The borrow-checker is not happy with me. Method ParserEventIterator::new near the bottom of the listing causes the error. How do I define the lifetimes or borrowing so that I can get this to compile? Or what Rustic data structure design should I use in its stead?
Ultimately, I would like this to implement the appropriate traits to become a proper iterator. (Newbie to Rust. Prior to this, I have programmed in 28 languages, but this one has me stumped.)
Here is a code sample:
extern crate itertools;
use itertools::put_back;
use std::fmt::Display;
use std::fmt::Formatter;
use std::fmt::Result;
pub enum ParserEvent {
Letter(char),
Digit(char),
Other(char),
}
impl ParserEvent {
fn new(c: char) -> ParserEvent {
match c {
'a'...'z' | 'A'...'Z' => ParserEvent::Letter(c),
'0'...'9' => ParserEvent::Digit(c),
_ => ParserEvent::Other(c),
}
}
}
impl Display for ParserEvent {
fn fmt(&self, f: &mut Formatter) -> Result {
let mut _ctos = |c: char| write!(f, "{}", c.to_string());
match self {
ParserEvent::Letter(letter) => _ctos(*letter),
ParserEvent::Digit(digit) => _ctos(*digit),
ParserEvent::Other(o) => _ctos(*o),
}
}
}
// ParserEventIterator
// Elements ('e) must have lifetime longer than the iterator ('i).
pub struct ParserEventIterator<'i, 'e: 'i> {
char_iter: &'i mut itertools::PutBack<std::str::Chars<'e>>,
}
impl<'i, 'e: 'i> ParserEventIterator<'i, 'e> {
fn new(s: &'e std::string::String) -> ParserEventIterator<'i, 'e> {
// THIS NEXT LINE IS THE LINE WITH THE PROBLEM!!!
ParserEventIterator {
char_iter: &mut put_back(s.chars()),
}
}
fn put_back(&mut self, e: ParserEvent) -> () {
if let Some(c) = e.to_string().chars().next() {
self.char_iter.put_back(c);
}
}
}
impl<'i, 'e: 'i> Iterator for ParserEventIterator<'i, 'e> {
type Item = ParserEvent;
fn next(&mut self) -> Option<ParserEvent> {
match self.char_iter.next() {
Some(c) => Some(ParserEvent::new(c)),
None => None,
}
}
}
fn main() {
let mut _i = ParserEventIterator::new(&String::from("Hello World"));
}
On the Rust Playground
error[E0515]: cannot return value referencing temporary value
--> src/main.rs:43:9
|
43 | / ParserEventIterator {
44 | | char_iter: &mut put_back(s.chars()),
| | ------------------- temporary value created here
45 | | }
| |_________^ returns a value referencing data owned by the current function
Well, the compiler is almost telling you the solution by reflecting to the obvious problem: you can't have a borrow which doesn't live long enough, i.e. the borrow would point to a nonexistent location after the stack memory of the function has been destroyed.
This would happen because the borrow is referencing an object (in this case an itertools::struct::PutBack instance) that has been newly created within the function body. This instance gets destroyed at the end of the function along with all the references to it. So the compiler is preventing you to have a so called dangling pointer.
Thus, instead of borrowing you should move the PutBack instance into your struct:
// ...
pub struct ParserEventIterator<'e> {
char_iter: itertools::PutBack<std::str::Chars<'e>>
}
impl<'e> ParserEventIterator<'e> {
fn new(s: &'e std::string::String) -> ParserEventIterator<'e> {
ParserEventIterator { char_iter: put_back(s.chars()) }
}
// ...
}

Bind a reference to a struct property to a variable inside a function returning a mutable reference [duplicate]

I am learning Rust and I don't quite get why this is not working.
#[derive(Debug)]
struct Node {
value: String,
}
#[derive(Debug)]
pub struct Graph {
nodes: Vec<Box<Node>>,
}
fn mk_node(value: String) -> Node {
Node { value }
}
pub fn mk_graph() -> Graph {
Graph { nodes: vec![] }
}
impl Graph {
fn add_node(&mut self, value: String) {
if let None = self.nodes.iter().position(|node| node.value == value) {
let node = Box::new(mk_node(value));
self.nodes.push(node);
};
}
fn get_node_by_value(&self, value: &str) -> Option<&Node> {
match self.nodes.iter().position(|node| node.value == *value) {
None => None,
Some(idx) => self.nodes.get(idx).map(|n| &**n),
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn some_test() {
let mut graph = mk_graph();
graph.add_node("source".to_string());
graph.add_node("destination".to_string());
let source = graph.get_node_by_value("source").unwrap();
let dest = graph.get_node_by_value("destination").unwrap();
graph.add_node("destination".to_string());
}
}
(playground)
This has the error
error[E0502]: cannot borrow `graph` as mutable because it is also borrowed as immutable
--> src/main.rs:50:9
|
47 | let source = graph.get_node_by_value("source").unwrap();
| ----- immutable borrow occurs here
...
50 | graph.add_node("destination".to_string());
| ^^^^^ mutable borrow occurs here
51 | }
| - immutable borrow ends here
This example from Programming Rust is quite similar to what I have but it works:
pub struct Queue {
older: Vec<char>, // older elements, eldest last.
younger: Vec<char>, // younger elements, youngest last.
}
impl Queue {
/// Push a character onto the back of a queue.
pub fn push(&mut self, c: char) {
self.younger.push(c);
}
/// Pop a character off the front of a queue. Return `Some(c)` if there /// was a character to pop, or `None` if the queue was empty.
pub fn pop(&mut self) -> Option<char> {
if self.older.is_empty() {
if self.younger.is_empty() {
return None;
}
// Bring the elements in younger over to older, and put them in // the promised order.
use std::mem::swap;
swap(&mut self.older, &mut self.younger);
self.older.reverse();
}
// Now older is guaranteed to have something. Vec's pop method // already returns an Option, so we're set.
self.older.pop()
}
pub fn split(self) -> (Vec<char>, Vec<char>) {
(self.older, self.younger)
}
}
pub fn main() {
let mut q = Queue {
older: Vec::new(),
younger: Vec::new(),
};
q.push('P');
q.push('D');
assert_eq!(q.pop(), Some('P'));
q.push('X');
let (older, younger) = q.split(); // q is now uninitialized.
assert_eq!(older, vec!['D']);
assert_eq!(younger, vec!['X']);
}
A MRE of your problem can be reduced to this:
// This applies to the version of Rust this question
// was asked about; see below for updated examples.
fn main() {
let mut items = vec![1];
let item = items.last();
items.push(2);
}
error[E0502]: cannot borrow `items` as mutable because it is also borrowed as immutable
--> src/main.rs:4:5
|
3 | let item = items.last();
| ----- immutable borrow occurs here
4 | items.push(2);
| ^^^^^ mutable borrow occurs here
5 | }
| - immutable borrow ends here
You are encountering the exact problem that Rust was designed to prevent. You have a reference pointing into the vector and are attempting to insert into the vector. Doing so might require that the memory of the vector be reallocated, invalidating any existing references. If that happened and you used the value in item, you'd be accessing uninitialized memory, potentially causing a crash.
In this particular case, you aren't actually using item (or source, in the original) so you could just... not call that line. I assume you did that for some reason, so you could wrap the references in a block so that they go away before you try to mutate the value again:
fn main() {
let mut items = vec![1];
{
let item = items.last();
}
items.push(2);
}
This trick is no longer needed in modern Rust because non-lexical lifetimes have been implemented, but the underlying restriction still remains — you cannot have a mutable reference while there are other references to the same thing. This is one of the rules of references covered in The Rust Programming Language. A modified example that still does not work with NLL:
let mut items = vec![1];
let item = items.last();
items.push(2);
println!("{:?}", item);
In other cases, you can copy or clone the value in the vector. The item will no longer be a reference and you can modify the vector as you see fit:
fn main() {
let mut items = vec![1];
let item = items.last().cloned();
items.push(2);
}
If your type isn't cloneable, you can transform it into a reference-counted value (such as Rc or Arc) which can then be cloned. You may or may not also need to use interior mutability:
struct NonClone;
use std::rc::Rc;
fn main() {
let mut items = vec![Rc::new(NonClone)];
let item = items.last().cloned();
items.push(Rc::new(NonClone));
}
this example from Programming Rust is quite similar
No, it's not, seeing as how it doesn't use references at all.
See also
Cannot borrow `*x` as mutable because it is also borrowed as immutable
Pushing something into a vector depending on its last element
Why doesn't the lifetime of a mutable borrow end when the function call is complete?
How should I restructure my graph code to avoid an "Cannot borrow variable as mutable more than once at a time" error?
Why do I get the error "cannot borrow x as mutable more than once"?
Why does Rust want to borrow a variable as mutable more than once at a time?
Try to put your immutable borrow inside a block {...}.
This ends the borrow after the block.
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn some_test() {
let mut graph = mk_graph();
graph.add_node("source".to_string());
graph.add_node("destination".to_string());
{
let source = graph.get_node_by_value("source").unwrap();
let dest = graph.get_node_by_value("destination").unwrap();
}
graph.add_node("destination".to_string());
}
}
So for anyone else banging their head against this problem and wanting a quick way out - use clones instead of references. Eg I'm iterating this list of cells and want to change an attribute so I first copy the list:
let created = self.cells
.into_iter()
.map(|c| {
BoardCell {
x: c.x,
y: c.y,
owner: c.owner,
adjacency: c.adjacency.clone(),
}
})
.collect::<Vec<BoardCell>>();
And then modify the values in the original by looping the copy:
for c in created {
self.cells[(c.x + c.y * self.size) as usize].adjacency[dir] = count;
}
Using Vec<&BoardCell> would just yield this error. Not sure how Rusty this is but hey, it works.

Rust lifetime issue in loop

How to get this example to compile without array copying or multiple calls to b() per iteration — b() has to perform some expensive parsing?
This is not the full code that I wrote, but it illustrates the problem I had. Here, Test is attempting to perform some kind of streaming parsing work. c() is the parsing function, it returns Some when parsing was successful. b() is a function that attempts to read more data from the stream when c() can not parse using the available data yet. The returned value is a slice into the self.v containing the parsed range.
struct Test {
v: [u8; 10],
index: u8,
}
impl Test {
fn b(&mut self) {
self.index = 1
}
fn c(i: &[u8]) -> Option<&[u8]> {
Some(i)
}
fn a(&mut self) -> &[u8] {
loop {
self.b();
match Test::c(&self.v) {
Some(r) => return r,
_ => continue,
}
}
}
}
fn main() {
let mut q = Test {
v: [0; 10],
index: 0,
};
q.a();
}
When compiling, it produces the following borrow checker error:
error[E0502]: cannot borrow `*self` as mutable because `self.v` is also
borrowed as immutable
--> <anon>:17:13
|
17 | self.b();
| ^^^^ mutable borrow occurs here
18 |
19 | match Test::c(&self.v) {
| ------ immutable borrow occurs here
...
24 | }
| - immutable borrow ends here
If I change a() to:
fn a(&mut self) -> Option<&[u8]> {
loop {
self.b();
if let None = Test::c(&self.v) {
continue
}
if let Some(r) = Test::c(&self.v) {
return Some(r);
} else {
unreachable!();
}
}
}
Then it runs, but with the obvious drawback of calling the parsing function c() twice.
I kind of understand that changing self while the return value depends on it is unsafe, however, I do not understand why is the immutable borrow for self.v is still alive in the next iteration, when we attempted to call b() again.
Right now "Rustc can't "deal" with conditional borrowing returns". See this comment from Gankro on issue 21906.
It can't assign a correct lifetime to the borrow if only one execution path terminates the loop.
I can suggest this workaround, but I'm not sure it is optimal:
fn c(i: &[u8]) -> Option<(usize, usize)> {
Some((0, i.len()))
}
fn a(&mut self) -> &[u8] {
let parse_result;
loop {
self.b();
match Test::c(&self.v) {
Some(r) => {
parse_result = r;
break;
}
_ => {}
}
}
let (start, end) = parse_result;
&self.v[start..end]
}
You can construct result of parsing using array indexes and convert them into references outside of the loop.
Another option is to resort to unsafe to decouple lifetimes. I am not an expert in safe use of unsafe, so pay attention to comments of others.
fn a(&mut self) -> &[u8] {
loop {
self.b();
match Test::c(&self.v) {
Some(r) => return unsafe{
// should be safe. It decouples lifetime of
// &self.v and lifetime of returned value,
// while lifetime of returned value still
// cannot outlive self
::std::slice::from_raw_parts(r.as_ptr(), r.len())
},
_ => continue,
}
}
}

Resources