I have two async functions: get_message and get_event. I'd like to perform an action whenever a message arrives or an event comes and do that forever in an infinite loop.
The simplified setup looks like this:
use futures::{future::select, future::Either, pin_mut};
impl MsgReceiver {
async fn get_message(&mut self) -> Message { /* ... */ }
}
impl EventListener {
async fn get_event(&mut self) -> Event { /* ... */ }
}
async fn eternal_task(receiver: MsgReceiver, listener: EventListener) -> ! {
let get_msg_fut = receiver.get_message();
pin_mut!(get_msg_fut);
loop {
let get_event_fut = listener.get_event();
pin_mut!(get_event_fut);
match select(get_event_fut, get_msg_fut).await {
Either::Left((ev, r_get_msg_fut)) => {
/* react to the event */
// r_get_msg_fut is not done, how to reuse it in the next iteration?
}
Either::Right((msg, r_get_event_fut)) => {
/* react to the message */
// it's fine to drop get_event_fut here
// the following line causes a double-mut-borrow error on receiver,
// despite receiver isn't borrowed anymore (the old future is completed and dropped)
let new_future = receiver.get_message();
}
};
}
}
I have three major questions here:
When an event comes first, how to tell rust that I want to reuse the incomplete get_message future on the next loop iteration?
When a message comes first, how to construct a new future without a borrow error?
When (2) is solved, how to put the new future into the same pinned memory location and use it on the next loop iteration?
I had success using this, but could not get rid of the Box::pin
use futures::{future::select, future::Either, pin_mut};
use std::sync::Mutex;
#[derive(Debug)]
struct MsgReceiver;
#[derive(Debug)]
struct EventListener;
#[derive(Debug)]
struct Message;
#[derive(Debug)]
struct Event;
impl MsgReceiver {
async fn get_message(&mut self) -> Message {
Message
}
}
impl EventListener {
async fn get_event(&mut self) -> Event {
Event
}
}
async fn eternal_task(receiver: MsgReceiver, mut listener: EventListener) -> ! {
let receiver = Mutex::new(receiver);
let mut f = None;
loop {
let get_msg_fut = match f.take() {
None => {
let mut l = receiver.lock();
Box::pin(async move {
l.get_message().await
})
}
Some(f) => f,
};
let get_event_fut = listener.get_event();
pin_mut!(get_event_fut);
match select(get_event_fut, get_msg_fut).await {
Either::Left((ev, r_get_msg_fut)) => {
/* react to the event */
// store the future for next iteration
f = Some(r_get_msg_fut);
}
Either::Right((msg, r_get_event_fut)) => {
/* react to the message */
}
};
}
}
#[tokio::main]
async fn main() {
eternal_task(MsgReceiver, EventListener).await;
}
I think this is tricky to get right, even with unsafe which would probably be needed to accomplish this. Persisting and reusing the same variables isn't too hard, its actually #2 that's the hardest (at least with the current borrow checker).
I found a solution that totally circumvents the problem by using the async-stream crate to provide an intermediary:
async fn eternal_task(mut receiver: MsgReceiver, mut listener: EventListener) -> ! {
let combined = futures::stream::select(
stream! { loop { yield Either::Left(receiver.get_message().await); } },
stream! { loop { yield Either::Right(listener.get_event().await); } },
);
pin_mut!(combined);
while let Some(msg_or_evt) = combined.next().await {
match msg_or_evt {
Either::Left(msg) => {
// do something with msg
}
Either::Right(evt) => {
// do something with evt
}
};
}
unreachable!()
}
It uses the stream! macro to generate a type that continuously calls and yields values from .get_message() and .get_event(). It then uses futures::stream::select and Either to combine them. And then its just a matter of looping over the results. It works in #![no_std].
How do I check if a vector of enum has a certain member with values?
#[derive(PartialEq,Clone,Copy)]
enum TestResult {
Pass,
Fail {point: u8},
}
impl TestResult {
fn is_fail(&self) -> bool {
match *self {
TestResult::Fail{point:_} => true,
_ => false,
}
}
}
fn main() {
let v1 = vec![TestResult::Pass,
TestResult::Pass,
TestResult::Fail{point:50}];
if v1.contains(&TestResult::Pass) {
println!("result contains Pass");
}
if v1.iter().any(|&r| r.is_fail()) {
println!("result contains Fail");
}
}
This is working but is there a way to do this with Vec::contains()?
I want to check if TestResult::Fail is in a vector in the same way as for TestResult::Pass (no pattern matching. easy..)
There isn't really an easy way to do this, however we can make a simple macro to perform some pattern matching in the if statement.
/// Performs pattern matching contains on an IntoIter type
macro_rules! matches_any {
($iterable:expr, $($tokens:tt)+) => {{
let iter = ::std::iter::IntoIterator::into_iter($iterable);
iter.any(|x| matches!(x, $($tokens)+))
}};
}
The trick here is that we can use the matches! macro instead of a full match statement if all we want to know is if something matches a given pattern.
// Without matches!
match self {
TestResult::Fail {..} => true,
_ => false,
}
// With matches!
matches!(self, TestResult::Fail {..})
So now we can use this macro instead:
let v1 = vec![TestResult::Pass,
TestResult::Pass,
TestResult::Fail { point: 50 }];
if matches_any!(&v1, TestResult::Pass) {
println!("result contains Pass");
}
if matches_any!(&v1, TestResult::Fail {..}) {
println!("result contains Fail");
}
I'm quite new in Rust and I'm having an issue with lifetimes that I believe I can understand what is happening and why, but can't get around in my head how can I solve it.
For simplicity I created this short "clone" of what I'm actually trying to do, but the real code is using asyc-stripe. Will annotate the example code with the real one in case is relevant.
There is the following structure:
// https://github.com/arlyon/async-stripe/blob/9f1a84144a23cc7b2124a1252ee15dc646ce0215/src/resources/generated/subscription.rs#L385
struct ObjectA<'a> {
field: i32,
object_b_id: Option<&'a str>,
}
// https://github.com/arlyon/async-stripe/blob/9f1a84144a23cc7b2124a1252ee15dc646ce0215/src/resources/generated/subscription.rs#L570
impl<'a> ObjectA<'a> {
fn new(field: i32) -> Self {
return Self {
field,
object_b_id: Default::default(),
};
}
}
// https://github.com/arlyon/async-stripe/blob/9f1a84144a23cc7b2124a1252ee15dc646ce0215/src/resources/generated/subscription.rs#L210
fn persist_obj_a(obj_a: ObjectA<'_>) {}
// ---
// https://github.com/arlyon/async-stripe/blob/9f1a84144a23cc7b2124a1252ee15dc646ce0215/src/resources/generated/payment_method.rs#L18
struct ObjectB {
id: ObjectBId,
}
// https://github.com/arlyon/async-stripe/blob/9f1a84144a23cc7b2124a1252ee15dc646ce0215/src/ids.rs#L518
struct ObjectBId {
value: String,
}
impl ObjectBId {
fn as_str(&self) -> &str {
return self.value.as_str();
}
}
// This is a wrapper around https://github.com/arlyon/async-stripe/blob/9f1a84144a23cc7b2124a1252ee15dc646ce0215/src/resources/generated/payment_method.rs#L128 that just returns the first one found (id any, hence the Option)
fn load_object_b() -> Option<ObjectB> {
return Some(ObjectB {
id: ObjectBId {
value: String::from("some_id"),
},
});
}
And what I'm trying to do is: load the ObjectB with load_object_b and use its ID into a ObjectA.
Ok, so on to my attempts.
First attempt
fn first_try(condition: bool) {
let mut obj_a = ObjectA::new(1);
if condition {
match load_object_b() {
Some(obj_b) => obj_a.object_b_id = Some(obj_b.id.as_str()),
None => (),
}
}
persist_obj_a(obj_a);
}
In here I get
obj_b.id does not live long enough
Which I can understand, since from what I can understand the obj_b only exists during the match arm and is droped by the end of it.
Second attempt
fn second_try(condition: bool) {
let mut obj_a = ObjectA::new(1);
if condition {
let obj_b = load_object_b();
match obj_b {
Some(ref obj_b) => obj_a.object_b_id = Some(obj_b.id.as_str()),
None => (),
}
}
persist_obj_a(obj_a);
}
Here I get
obj_b.0 does not live long enough
Which I guess it is still the same idea, just in a different place. Since again, from my understanding, obj_b now only lives within the scope of the if condition.
Third and last attempt
I ended up "solving" it with:
fn third_try(condition: bool) {
let mut obj_a = ObjectA::new(1);
let obj_b = load_object_b();
let obj_b_id = match obj_b {
Some(ref obj_b) => Some(obj_b.id.as_str()),
None => None,
};
if condition {
obj_a.object_b_id = obj_b_id;
}
persist_obj_a(obj_a);
}
In here I moved the obj_b to have the same lifetime as obj_a. So it solves the issue that I was having.
My problem with this solution is that I feel that I'm wasting resource doing the (possible expensive) request to load_object_b even if I'm not gonna use it based on the condition.
Not sure if I'm missing something very obvious or just going on the overall wrong direction, but would appreciate some light on what I might be doing wrong.
This should work, I think:
fn third_try(condition: bool) {
let mut obj_a = ObjectA::new(1);
let obj_b = if condition { load_object_b() } else { None };
obj_a.object_b_id = obj_b.as_ref().map (|o| o.id.as_str());
persist_obj_a(obj_a);
}
Rust allows you to have conditionally initialized variables. You can declare obj_b ouside of the if, but only initialize it inside the if. The compiler will ensure you can use it only if it is initialized.
fn second_try(condition: bool) {
let mut obj_a = ObjectA::new(1);
let obj_b;
if condition {
obj_b = load_object_b();
match obj_b {
Some(ref obj_b) => obj_a.object_b_id = Some(obj_b.id.as_str()),
None => (),
}
}
persist_obj_a(obj_a);
}
This question already has answers here:
How to fix lifetime error when function returns a serde Deserialize type?
(2 answers)
Why do Rust lifetimes matter when I move values into a spawned Tokio task?
(1 answer)
Closed 1 year ago.
I have this websocket code that uses tokio and serde here:
use async_once::AsyncOnce;
use common_wasm::models::status::{CommandMessage, StatusMessage};
use futures_util::{SinkExt, StreamExt};
use lazy_static::lazy_static;
use std::{collections::VecDeque, net::SocketAddr};
use tokio::{
net::{TcpListener, TcpStream}, sync::{broadcast, mpsc}
};
use tokio_tungstenite::{
accept_async, tungstenite::{Error, Message, Result}
};
use tracing::*;
// https://stackoverflow.com/questions/67650879/rust-lazy-static-with-async-await
lazy_static! {
pub static ref STATUS_REPORTER: AsyncOnce<StatusWs> = AsyncOnce::new(async {
info!("Init lazy static WS");
let server = StatusWs::init("ws://localhost:44444").await;
server
});
}
use StatusMessage as SenderType;
use CommandMessage as ReceiveType;
pub struct StatusWs {
buf: VecDeque<ReceiveType>,
rx_client_msg: mpsc::Receiver<ReceiveType>,
tx_server_msg: broadcast::Sender<SenderType>,
}
impl StatusWs {
pub async fn init(addr: &str) -> StatusWs {
info!("Init Status WS on {}", addr);
let listener = TcpListener::bind(&addr).await.expect("Can't listen");
// Clients producting to server, they use the tx to send and server uses the rx to read
let (tx_client_msg, rx_client_msg) = mpsc::channel::<ReceiveType>(32);
// spmc for server to broadcast status to listeners. Server uses tx to send and client uses rx to read
let (tx_server_msg, _rx_server_msg) = broadcast::channel::<SenderType>(10);
let tx_server_2 = tx_server_msg.clone();
tokio::spawn(async move {
while let Ok((stream, peer)) = listener.accept().await {
info!("Peer address connected: {}", peer);
let tx_client = tx_client_msg.clone();
let rx_server = tx_server_msg.subscribe();
tokio::spawn(async move {
accept_connection(peer, stream, tx_client, rx_server).await;
});
}
});
StatusWs { buf: VecDeque::new(), rx_client_msg, tx_server_msg: tx_server_2 }
}
pub async fn reportinfo(&self, msg: &SenderType) {
let my_msg = msg.clone();
match &self.tx_server_msg.send(my_msg) {
Ok(_size) => {
//trace!("Server Sending OK {}", size)
},
Err(_err) => {
//trace!("Server Sending ERR {:?}", err)
},
}
}
pub async fn next(&mut self) -> Result<Option<ReceiveType>> {
loop {
// If buffer contains data, we can directly return it.
if let Some(data) = self.buf.pop_front() {
return Ok(Some(data));
}
// Fetch new response if buffer is empty.
let response = self.next_response().await?;
// Handle the response, possibly adding to the buffer
self.handle_response(response)?;
}
}
async fn next_response(&mut self) -> Result<ReceiveType> {
loop {
tokio::select! { // TODO don't need select if there's only one thing?
Some(msg) = self.rx_client_msg.recv() => {
return Ok(msg)
},
}
}
}
fn handle_response(&mut self, response: ReceiveType) -> Result<()> {
self.buf.push_back(response);
Ok(())
}
}
async fn accept_connection(peer: SocketAddr, stream: TcpStream, tx_client: mpsc::Sender<ReceiveType>, rx_server: broadcast::Receiver<SenderType>) {
info!("Accepting connection from {}", peer);
if let Err(e) = handle_connection(peer, stream, tx_client, rx_server).await {
match e {
Error::ConnectionClosed | Error::Protocol(_) | Error::Utf8 => error!("Connection closed"),
err => error!("Error processing connection: {}", err),
}
}
}
async fn handle_connection(
_peer: SocketAddr, stream: TcpStream, tx_client: mpsc::Sender<ReceiveType>, mut rx_server: broadcast::Receiver<SenderType>,
) -> Result<()> {
let ws_stream = accept_async(stream).await.expect("Failed to accept");
let (mut ws_sender, mut ws_receiver) = ws_stream.split();
loop {
tokio::select! {
remote_msg = ws_receiver.next() => {
match remote_msg {
Some(msg) => {
let msg = msg?;
match msg {
Message::Text(resptxt) => {
match serde_json::from_str::<ReceiveType>(&resptxt) {
Ok(cmd) => { let _ = tx_client.send(cmd).await; },
Err(err) => error!("Error deserializing: {}", err),
}
},
Message::Close(_) => break,
_ => { },
}
}
None => break,
}
}
Ok(msg) = rx_server.recv() => {
match serde_json::to_string(&msg) {
Ok(txt) => ws_sender.send(Message::Text(txt)).await?,
Err(_) => todo!(),
}
}
}
}
Ok(())
}
The sender and receiver types are simple (simple types all the way down):
use std::{collections::BTreeMap, fmt::Debug};
use serde::{Deserialize, Serialize};
#[derive(Default, Clone, Debug, Serialize, Deserialize)]
pub struct StatusMessage {
pub name: String,
pub entries: BTreeMap<i32, GuiEntry>,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct CommandMessage {
pub sender: String,
pub entryid: i32,
pub command: GuiValue,
}
Now I want to generalize the code so that I can create a struct that takes some other kind of Sender and Receiver type. Yes, I could just change the aliases, but I want to be able to use the generic type arguments rather than duplicate the whole file. The problem is as I follow the suggestions from the compiler, I end up in a place where I don't know what to do next. It's telling me resptext does not live long enough:
`resptxt` does not live long enough
borrowed value does not live long enoughrust cE0597
status_ws.rs(133, 29): `resptxt` dropped here while still borrowed
status_ws.rs(115, 28): lifetime `'a` defined here
status_ws.rs(129, 39): argument requires that `resptxt` is borrowed for `'a`
Here's what I have thus far:
use async_once::AsyncOnce;
use common_wasm::models::status::{CommandMessage, StatusMessage};
use futures_util::{SinkExt, StreamExt};
use lazy_static::lazy_static;
use serde::{Serialize, Deserialize};
use std::{collections::VecDeque, net::SocketAddr};
use tokio::{
net::{TcpListener, TcpStream}, sync::{broadcast, mpsc}
};
use tokio_tungstenite::{
accept_async, tungstenite::{Error, Message, Result}
};
use tracing::*;
// https://stackoverflow.com/questions/67650879/rust-lazy-static-with-async-await
lazy_static! {
pub static ref STATUS_REPORTER: AsyncOnce<StatusWs<CommandMessage, StatusMessage>> = AsyncOnce::new(async {
info!("Init lazy static WS");
let server = StatusWs::init("ws://localhost:44444").await;
server
});
}
// use StatusMessage as SenderType;
// use CommandMessage as ReceiveType;
pub struct StatusWs<ReceiveType, SenderType> {
buf: VecDeque<ReceiveType>,
rx_client_msg: mpsc::Receiver<ReceiveType>,
tx_server_msg: broadcast::Sender<SenderType>,
}
impl <'a, ReceiveType: Deserialize<'a> + Send, SenderType: Serialize + Clone + Send + Sync> StatusWs <ReceiveType, SenderType> {
pub async fn init(addr: &str) -> StatusWs<ReceiveType, SenderType> {
info!("Init Status WS on {}", addr);
let listener = TcpListener::bind(&addr).await.expect("Can't listen");
// Clients producting to server, they use the tx to send and server uses the rx to read
let (tx_client_msg, rx_client_msg) = mpsc::channel::<ReceiveType>(32);
// spmc for server to broadcast status to listeners. Server uses tx to send and client uses rx to read
let (tx_server_msg, _rx_server_msg) = broadcast::channel::<SenderType>(10);
let tx_server_2 = tx_server_msg.clone();
tokio::spawn(async move {
while let Ok((stream, peer)) = listener.accept().await {
info!("Peer address connected: {}", peer);
let tx_client = tx_client_msg.clone();
let rx_server = tx_server_msg.subscribe();
tokio::spawn(async move {
accept_connection(peer, stream, tx_client, rx_server).await;
});
}
});
StatusWs { buf: VecDeque::new(), rx_client_msg, tx_server_msg: tx_server_2 }
}
pub async fn reportinfo(&self, msg: &SenderType) {
let my_msg = msg.clone();
match &self.tx_server_msg.send(my_msg) {
Ok(_size) => {
//trace!("Server Sending OK {}", size)
},
Err(_err) => {
//trace!("Server Sending ERR {:?}", err)
},
}
}
pub async fn next(&mut self) -> Result<Option<ReceiveType>> {
loop {
// If buffer contains data, we can directly return it.
if let Some(data) = self.buf.pop_front() {
return Ok(Some(data));
}
// Fetch new response if buffer is empty.
let response = self.next_response().await?;
// Handle the response, possibly adding to the buffer
self.handle_response(response)?;
}
}
async fn next_response(&mut self) -> Result<ReceiveType> {
loop {
tokio::select! { // TODO don't need select if there's only one thing?
Some(msg) = self.rx_client_msg.recv() => {
return Ok(msg)
},
}
}
}
fn handle_response(&mut self, response: ReceiveType) -> Result<()> {
self.buf.push_back(response);
Ok(())
}
}
async fn accept_connection<'a, ReceiveType: Deserialize<'a>, SenderType: Clone + Serialize>(peer: SocketAddr, stream: TcpStream, tx_client: mpsc::Sender<ReceiveType>, rx_server: broadcast::Receiver<SenderType>) {
info!("Accepting connection from {}", peer);
if let Err(e) = handle_connection(peer, stream, tx_client, rx_server).await {
match e {
Error::ConnectionClosed | Error::Protocol(_) | Error::Utf8 => error!("Connection closed"),
err => error!("Error processing connection: {}", err),
}
}
}
async fn handle_connection<'a, ReceiveType: Deserialize<'a>, SenderType: Clone + Serialize>(
_peer: SocketAddr, stream: TcpStream, tx_client: mpsc::Sender<ReceiveType>, mut rx_server: broadcast::Receiver<SenderType>,
) -> Result<()> {
let ws_stream = accept_async(stream).await.expect("Failed to accept");
let (mut ws_sender, mut ws_receiver) = ws_stream.split();
loop {
tokio::select! {
remote_msg = ws_receiver.next() => {
match remote_msg {
Some(msg) => {
let msg = msg?;
match msg {
Message::Text(resptxt) => {
match serde_json::from_str::<ReceiveType>(&resptxt) {
Ok(cmd) => { let _ = tx_client.send(cmd).await; },
Err(err) => error!("Error deserializing: {}", err),
}
},
Message::Close(_) => break,
_ => { },
}
}
None => break,
}
}
Ok(msg) = rx_server.recv() => {
match serde_json::to_string(&msg) {
Ok(txt) => ws_sender.send(Message::Text(txt)).await?,
Err(_) => todo!(),
}
}
}
}
Ok(())
}
I think there's some confusion about the necessary lifetimes and bounds, in particular the lifetime on the Deserializer from Serde and the Send/Sync auto trait markers on the message types.
In any case, it seems a bit brute force to just copy the whole original file and change out the aliases, which would definitely work, when it seems there's some sort of useful lesson here.
You should use serde::de::DeserializeOwned instead of Deserialize<'a>.
The Deserialize trait takes a lifetime parameter to support zero-cost deserialization, but you can't take advantage of that since the source, resptxt, is a transient value that isn't persisted anywhere. The DeserializeOwned trait can be used to constrain that the deserialized type does not keep references to the source and can therefore be used beyond it.
After fixing that, you'll get errors that ReceiveType and SenderType must be 'static to be used in a tokio::spawn'd task. Adding that constraint finally makes your code compile.
See the full compiling code on the playground for brevity.
For convenience, you can find the below code on Rust Playground:
fn main() {
println!("Hello, world!");
let n = 5;
let mut my_vector : Vec<u16>;
loop {
let x: Result<i32, &str> = Ok(-3);
match x {
Ok(-3) => {
my_vector = (0..n).map(|x| x).collect();
// works
for e in my_vector {
println!("{}", e);
}
},
Err(e) => {
eprintln!("couldn't recieve a datagram: {}", e);
},
_ => {
}
}
// does not work
for e in my_vector {
println!("{}", e);
}
break;
}
}
I get the error "move occurs because my_vector has type Vec<u16>, which does not implement the Copy trait" and I am not able to make the program compile (and work), any help very well appreciated!
NB: This is a simplified example of the error I am facing in my program.
How could I set the value of the vector inside the match {} and then reuse it later on?
for e in my_vector {
println!("{}", e);
}
This consumes my_vector. If you want to iterate the elements and still be able to use them later, you can use .iter()
for e in my_vector.iter() {
println!("{}", e);
}
Here's a working playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=8c99592ef98735472165f76b6ceb9b19