In a rebroadcaster (redis to zmq) implementation i am unable to figure out how to resolve the lifetime issue of temporary values (derived from consumed message from pubsub in a loop).
I pretty much understand why, but i am so far failing to produce Vec<&String> that i need to pass into zmq publisher. I tried with different types, cloning etc. but i'm just getting same or similar errors related to ownership and lifetime.
The problematic code:
async fn ps_rebroadcaster() -> Result<(), ApiError>{
let mut inproc_publisher: async_zmq::Publish<std::vec::IntoIter<&std::string::String>, &std::string::String> = async_zmq::publish("ipc://pricing")?.bind()?;
let con_str = &APP_CONFIG.REDIS_URL;
let conn = create_client(con_str.to_string())
.await
.expect("Can't connect to redis");
let mut pubsub = conn.get_async_connection().await?.into_pubsub();
match pubsub.psubscribe("T1050_*").await {
Ok(_) => {}
Err(_) => {}
};
let mut msg_stream = pubsub.into_on_message();
loop {
let result = msg_stream.next().await;
match result {
Some(message) => {
let channel_name = message.get_channel_name().to_string();
let payload_value: redis::Value = message.get_payload().expect("Can't get payload of message");
let payload_string: String = FromRedisValue::from_redis_value(&payload_value).expect("Can't convert from Redis value");
let item: Vec<&String> = vec![&channel_name , &payload_string ];
// problem here ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
inproc_publisher.send(item.into()).await?;
}
None => {
println!("None received");
continue;
}
}
}
}
And the error:
error[E0597]: `channel_name` does not live long enough
--> src/server.rs:248:47
|
248 | let item: Vec<&String> = vec![&channel_name , &payload_string ];
| ^^^^^^^^^^^^^ borrowed value does not live long enough
249 |
250 | inproc_publisher.send(item.into()).await?;
| ---------------------------------- borrow later used here
251 | }
| - `channel_name` dropped here while still borrowed
error[E0597]: `payload_string` does not live long enough
--> src/server.rs:248:63
|
248 | let item: Vec<&String> = vec![&channel_name , &payload_string ];
| ^^^^^^^^^^^^^^^ borrowed value does not live long enough
249 |
250 | inproc_publisher.send(item.into()).await?;
| ---------------------------------- borrow later used here
251 | }
| - `payload_string` dropped here while still borrowed
This fails because the sink wants an Into<Message> but doesn't apparently perform the conversion immediately, so the reference needs to continue to exist for some unspecified amount of time, and the values they refer to are destroyed too soon. Basically, you need to give the sink an owned value instead of a borrowed one.
You can do this by converting the Strings into Message values before giving them to the Publish sink, or by giving it an owned value that can be converted to Message. There isn't a conversion from String to Message, but there is one from Vec<u8>, and you can convert a String to a Vec<u8> without any copying by using the String::into_bytes() method.
Related
I want to do something like this:
fn some_fn() {
let mut my_map = HashMap::from([
(1, "1.0".to_string()),
(2, "2.0".to_string()),
]);
let key = 3;
let res = match my_map.get(&key) {
Some(child) => child,
None => {
let value = "3.0".to_string();
my_map.insert(key, value);
&value // HERE IT FAILS
}
};
println!("{}", res);
}
but it compiles with errors:
error[E0597]: `value` does not live long enough
--> src/lib.rs:16:13
|
16 | &value // HERE IT FAILS
| ^^^^^^
| |
| borrowed value does not live long enough
| borrow later used here
17 | }
| - `value` dropped here while still borrowed
error[E0382]: borrow of moved value: `value`
--> src/lib.rs:16:13
|
14 | let value = "3.0".to_string();
| ----- move occurs because `value` has type `String`, which does not implement the `Copy` trait
15 | my_map.insert(key, value);
| ----- value moved here
16 | &value // HERE IT FAILS
| ^^^^^^ value borrowed here after move
Playground
How can I fix it elegantly? Make a copy of string seems to me non-optimal.
The issue here is that you're inserting the value string then trying to return a reference to it, despite it already being moved into the HashMap. What you really want is to insert the value then get a reference to the inserted value, which could look something like this:
let res = match my_map.get(&key) {
Some(child) => child,
None => {
let value = "3.0".to_string();
my_map.insert(key, value);
my_map.get(&key).unwrap() // unwrap is guaranteed to work
}
};
BUT DON'T DO THIS. It's ugly and slow, as it has to look up the key in the map twice. Rust has a convenient Entry type that lets you get an entry of a HashMap then perform operations on that:
// or_insert returns the value if it exists, otherwise it inserts a default and returns it
let res = my_map.entry(key).or_insert("3.0".to_string());
Alternatively, if the operation to generate the default value is expensive, you can use or_insert_with to pass a closure:
// this won't call "3.0".to_string() unless it needs to
let res = my_map.entry(key).or_insert_with(|| "3.0".to_string());
I have a question about lifetimes. I have the following code:
async move {
let futures = FuturesUnordered::new();
let subscriptions = database.get_subscribers_for(&msg).await?;
for notifier in notifiers.iter() {
futures.push(notifier.notify(&msg, &subscriptions));
}
let results = futures.collect::<Vec<Result<()>>>().await;
results
.into_iter()
.filter(Result::is_err)
.for_each(|result| error!("Error sending notification: {}", result.unwrap_err()));
Ok(())
}
Signature from nofity:
async fn notify(&self, msg: &Notification, subs: &[Subscriptions]) -> Result<()>
I'm getting an error saying subscriptions might not live long enough and that subscriptions can be still be borrowed when dropped.
How can it be if the collect from the futures take the value from the futures and then I call into_iter which consumes all entries?
So basically the futures will be dropped before subscriptions. Am I understanding it wrong?
If I set the code like this, forcing the futures to be in a separate block, it compiles:
async move {
let subscriptions = database.get_subscribers_for(&msg).await?;
{
let futures = FuturesUnordered::new();
for notifier in notifiers.iter() {
futures.push(notifier.notify(&msg, &subscriptions));
}
let results = futures.collect::<Vec<Result<()>>>().await;
results
.into_iter()
.filter(Result::is_err)
.for_each(|result| error!("Error sending notification: {}", result.unwrap_err()));
}
Ok(())
}
Compiler message:
error[E0597]: `subscriptions` does not live long enough
--> src/notification/mod.rs:57:52
|
57 | futures.push(notifier.notify(&msg, &subscriptions));
| ^^^^^^^^^^^^^^ borrowed value does not live long enough
...
65 | })
| -
| |
| `subscriptions` dropped here while still borrowed
| borrow might be used here, when `futures` is dropped and runs the `Drop` code for type `FuturesUnordered`
|
= note: values in a scope are dropped in the opposite order they are defined
I have a borrow in the empty variable and I want to extend its life. In the commented code-block, I attempt to address it, but the reference is no longer available. I have to loop through the loop again to find the match in order to act on it.
How can I loop through a query looking for a best-match and then act on it once I know it's the best match, without having to loop through to find it again?
use bevy::prelude::*;
struct Person;
struct Name(String);
fn main() {
App::build()
.add_default_plugins()
.add_startup_system(startup.system())
.add_system(boot.system())
.run();
}
fn boot(mut query: Query<(&Person, &mut Name)>) {
let mut temp_str = String::new();
let mut empty: Option<&mut Name> = None;
for (_p, mut n_val) in &mut query.iter() {
if n_val.0.to_lowercase() > temp_str.to_lowercase() {
temp_str = n_val.0.clone();
empty = Some(&mut n_val);
}
}
println!("{}", temp_str);
if let Some(n) = empty {
// ...
}
// for (_p, mut n_val) in &mut query.iter() {
// if n_val.0 == temp_str {
// n_val.0 = "a".to_string();
// }
// }
}
fn startup(mut commands: Commands) {
commands
.spawn((Person, Name("Gene".to_string())))
.spawn((Person, Name("Candace".to_string())))
.spawn((Person, Name("Zany".to_string())))
.spawn((Person, Name("Sarah".to_string())))
.spawn((Person, Name("Carl".to_string())))
.spawn((Person, Name("Robert".to_string())));
}
Cargo.toml:
[package]
name = "sample"
version = "0.1.0"
authors = [""]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
bevy = "0.1.3"
Specific error:
error[E0716]: temporary value dropped while borrowed
--> src/main.rs:17:33
|
17 | for (_p, mut n_val) in &mut query.iter() {
| ^^^^^^^^^^^-
| | |
| | temporary value is freed at the end of this statement
| creates a temporary which is freed while still in use
...
24 | if let Some(n) = empty {
| ----- borrow later used here
|
= note: consider using a `let` binding to create a longer lived value
error[E0597]: `n_val` does not live long enough
--> src/main.rs:20:26
|
20 | empty = Some(&mut n_val);
| ^^^^^^^^^^ borrowed value does not live long enough
21 | }
22 | }
| - `n_val` dropped here while still borrowed
23 | println!("{}", temp_str);
24 | if let Some(n) = empty {
| ----- borrow later used here
You cannot extend the lifetime of a reference, but that is not your issue here, the error says temporary value dropped while borrowed, so you must extend the lifetime of your temporary.
If you wonder what temporary, the compiler also points to that (literally) in the error message: query.iter(). This is a function call, and the returned value is not bound to anything, so the compiler creates a temporary value for that. Then you iterate using a reference to that temporary. When the for loop ends, the temporary is dropped and any reference lifetime to it expires.
The solution is to bind the temporary to a local variable. This way you extend the lifetime of the object to the scope of the variable:
let mut iter = query.iter();
for (_p, n_val) in &mut iter {
if n_val.0.to_lowercase() > temp_str.to_lowercase() {
temp_str = n_val.0.clone();
empty = Some(n_val);
}
}
PS: I find quite bizarre the pattern of iterating over &mut iter. I would expect the return of iter() to implement Iterator or IntoIterator directly, but it looks like this is not the case.
I'm doing some exercises with using threads in Rust. I want to print chunked string, but I can't get pass lifetimes issue I've stumbled upon. Here's my code:
let input = "some sample string".to_string();
let mut threads = vec![];
for chunk in input.chars().collect::<Vec<char>>().chunks(2) {
threads.push(thread::spawn({
move || -> () {
println!("{:?}", chunk);
}
}))
}
When I'm trying to run it, I get a following error:
error[E0716]: temporary value dropped while borrowed
--> src\main.rs:7:18
|
7 | for chunk in input.chars().collect::<Vec<char>>().chunks(2) {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^----------
| | |
| | temporary value is freed at the end of this statement
| creates a temporary which is freed while still in use
| argument requires that borrow lasts for `'static`
I understand that the compiler doesn't know if the chunk within the thread will not outlive the input variable. But what I can do to fix this code?
I've tried to clone() the vector, but it didn't help.
The chunks are of the type &[char], with the same lifetime as the string.
Simply cloning here won't work, as at compile time it doesn't know the size of the char array. To get a owned copy of a unknown sized array, you must convert it to a Vec, which is done easily with the to_vec() function.
let input = "some sample string".to_string();
let mut threads = vec![];
for chunk in input.chars().collect::<Vec<char>>().chunks(2) {
let owned_chunk = chunk.to_vec();
threads.push(thread::spawn({
move || -> () {
println!("{:?}", owned_chunk);
}
}))
}
Alternatively, since we know the size of the array, we can create an array on the stack to copy into. This removes the need to allocate any heap memory.
let input = "some sample string".to_string();
let mut threads = vec![];
for chunk in input.chars().collect::<Vec<char>>().chunks(2) {
let mut owned_array = ['\0'; 2];
owned_array.copy_from_slice(chunk);
threads.push(thread::spawn({
move || -> () {
println!("{:?}", owned_array);
}
}))
}
I'm making a Discord chat bot using discord-rs, starting from this example. Everything was working and compiling fine until I tried to modify a value that is declared before a loop.
I'm trying to change prefix to the second word that is entered in a command:
extern crate discord;
use discord::Discord;
use discord::model::Event;
use std::env;
fn main() {
let discord = Discord::from_bot_token(&env::var("DISCORD_TOKEN").expect("Expected token"))
.expect("login failed");
let (mut connection, _) = discord.connect().expect("connect failed");
println!("Ready.");
let mut prefix = "!";
loop {
match connection.recv_event() {
Ok(Event::MessageCreate(message)) => {
if message.content.starts_with(prefix) {
let msg: Vec<&str> = message.content[prefix.len()..].split(" ").collect();
// message.content gets split into separate words and is checked for the command
match msg[0] {
"ag3nprefix" => {
prefix = msg[1];
// ^ here is the line that makes it not compile
let text = format!("AG3N's command prefix changed to: {}", prefix);
let _ = discord.send_message(message.channel_id, &text, "", false);
}
&_ => {}
}
}
}
Ok(_) => {}
Err(discord::Error::Closed(code, body)) => {
println!("Gateway closed on us with code {:?}: {}", code, body);
break;
}
Err(err) => println!("Receive error: {:?}", err),
}
}
}
I tried doing it various ways, but nothing would work. Here's the error from the compiler:
error[E0597]: `message.content` does not live long enough
--> src/main.rs:38:9
|
19 | let msg: Vec<&str> = message.content[prefix.len()..].split(" ").collect();
| --------------- borrow occurs here
...
38 | }
| ^ `message.content` dropped here while still borrowed
39 | }
40 | }
| - borrowed value needs to live until here
A smaller example
Here's a MCVE of the problem:
fn main() {
let mut prefix = "!";
let mut i = 0;
loop {
let event = String::from("hello");
match i {
0 => prefix = &event,
_ => println!("{}", prefix),
}
i += 1;
}
}
Rust 2015
error[E0597]: `event` does not live long enough
--> src/main.rs:9:28
|
9 | 0 => prefix = &event,
| ^^^^^ borrowed value does not live long enough
...
14 | }
| - `event` dropped here while still borrowed
15 | }
| - borrowed value needs to live until here
Rust 2018
error[E0597]: `event` does not live long enough
--> src/main.rs:9:27
|
9 | 0 => prefix = &event,
| ^^^^^^ borrowed value does not live long enough
10 | _ => println!("{}", prefix),
| ------ borrow used here, in later iteration of loop
...
14 | }
| - `event` dropped here while still borrowed
The core issue is that event is dropped at the end of each iteration. However, the code attempts to use prefix, a reference to event, in a subsequent iteration. If that were allowed to happen, you'd be accessing invalid memory, causing undefined behavior. Rust disallows this from happening.
You need to change the code so that event (or the part of it you need) lives longer than any one loop iteration.
Applied to the original code
The golden rule for borrowing problems is to identify who owns a variable. The compiler error messages help you here:
`message.content` does not live long enough
Ok, so we need to look at message.content or just message. Who owns that? We've matched a enum and transferred ownership to a local variable called message, so the variable message is the owner:
Ok(Event::MessageCreate(message)) => {
The compiler error message agrees, as this error points to the block where message is in scope (actually the match braces for technical reasons):
^ `message.content` dropped here while still borrowed
You are trying to take a reference to that string and store the reference somewhere that needs to live longer than a single loop iteration. The compiler has stopped you from introducing memory unsafety into your program. In other languages, you'd write this code and at some point in the future your program would crash (at best) or leak sensitive information or allow injecting code in (at worst).
Instead, allocate a String inside the loop. Because it has its own allocation, it can live longer than message. You also need to change the type of the original value of prefix and change the call to starts_with:
let mut prefix = "!".to_string();
loop {
match connection.recv_event() {
Ok(Event::MessageCreate(message)) => {
if message.content.starts_with(&prefix) {
let msg: Vec<_> = message.content[prefix.len()..].split(" ").collect();
// message.content gets split into separate words and is checked for the command
match msg[0] {
"ag3nprefix" => {
prefix = msg[1].to_string();
// ^ here is the line that makes it not compile
let text = format!("AG3N's command prefix changed to: {}", prefix);
let _ = discord.send_message(message.channel_id, &text, "", false);
}
&_ => {}
}
}
}
Ok(_) => {}
Err(discord::Error::Closed(code, body)) => {
println!("Gateway closed on us with code {:?}: {}", code, body);
break;
}
Err(err) => println!("Receive error: {:?}", err),
}
}
let _ = discord.send_message(message.channel_id, &text, "", false);
DO NOT IGNORE ERRORS. If you don't want to handle it, just add .expect("I didn't handle this error").