Cannot use Stream::take_while on an mpsc::channel: bool: Future is not satisfied [duplicate] - rust

This question already has answers here:
How to use take_while with futures::Stream?
(2 answers)
Closed 3 years ago.
I want to run an event loop in one thread and handle data from a UDP socket until another thread signals to stop work.
This is a difficult task for me, so I want to start from a simpler task:
one thread starting the event loop and waiting for another thread to signal the end:
use futures::{future, future::Future, stream::Stream, sync::mpsc};
use std::{io, io::BufRead, thread};
fn main() {
let (mut tx, rx) = mpsc::channel::<bool>(1);
let thr = thread::spawn(|| {
let mut runtime = tokio::runtime::current_thread::Runtime::new().unwrap();
runtime.spawn(
future::lazy(|| {
println!("event loop started");
Ok(())
})
.and_then(rx.take_while(|x| *x == true).into_future()),
);
runtime.run()
});
let stdin = io::stdin();
for line in stdin.lock().lines() {
let line = line.unwrap();
println!("{}", line);
if line == "exit" {
tx.try_send(false).unwrap();
break;
}
}
thr.join().unwrap().unwrap();
}
This code doesn't compile:
error[E0277]: the trait bound `bool: futures::future::Future` is not satisfied
--> src/main.rs:14:26
|
14 | .and_then(rx.take_while(|x| *x == true).into_future()),
| ^^^^^^^^^^ the trait `futures::future::Future` is not implemented for `bool`
|
= note: required because of the requirements on the impl of `futures::future::IntoFuture` for `bool`
error[E0599]: no method named `into_future` found for type `futures::stream::take_while::TakeWhile<futures::sync::mpsc::Receiver<bool>, [closure#src/main.rs:14:37: 14:51], bool>` in the current scope
--> src/main.rs:14:53
|
14 | .and_then(rx.take_while(|x| *x == true).into_future()),
| ^^^^^^^^^^^
|
= note: the method `into_future` exists but the following trait bounds were not satisfied:
`futures::stream::take_while::TakeWhile<futures::sync::mpsc::Receiver<bool>, [closure#src/main.rs:14:37: 14:51], bool> : futures::stream::Stream`
`&mut futures::stream::take_while::TakeWhile<futures::sync::mpsc::Receiver<bool>, [closure#src/main.rs:14:37: 14:51], bool> : futures::stream::Stream`
How do I fix the compilation error?

Read and understand the documentation and function signature of methods you attempt to use:
fn take_while<P, R>(self, pred: P) -> TakeWhile<Self, P, R>
where
P: FnMut(&Self::Item) -> R,
R: IntoFuture<Item = bool, Error = Self::Error>,
Self: Sized,
take_while takes a closure that returns some type that must be convertible into a future; a bool is not convertible into a future. The simplest way to do this is via future::ok:
let thr = thread::spawn(|| {
let mut runtime = tokio::runtime::current_thread::Runtime::new().unwrap();
runtime.spawn({
rx.take_while(|&x| future::ok(x))
.for_each(|x| {
println!("{}", x);
future::ok(())
})
});
runtime.run()
});
See also:
The trait bound `(): futures::Future` is not satisfied when using TcpConnectionNew
But my problem also in joining future::lazy and rx.take_while
That's an unrelated problem to what you asked about. Again, we look at the docs, this time for Future::and_then:
fn and_then<F, B>(self, f: F) -> AndThen<Self, B, F>
where
F: FnOnce(Self::Item) -> B,
B: IntoFuture<Error = Self::Error>,
Self: Sized,
Similarly to take_while, it takes a closure and the closure must return something that can be convertible into a future. Your code doesn't provide a closure.
Then look at Stream::into_future. This returns a type that implements Future and returns a tuple. The first item in the tuple is a single value from the stream, the second is the stream itself, to allow getting more values.
To get all the item and error types correct, I've make liberal use of map(drop) and map_err(drop) — you will want to do something better for your data and error handling.
runtime.spawn({
future::lazy(|| {
println!("event loop started");
Ok(())
})
.and_then(|_| {
rx.take_while(|&x| future::ok(x))
.into_future()
.map(drop)
.map_err(drop)
})
.map(drop)
});
Really, you should just use a oneshot channel; it's much simpler:
use futures::{
future::{self, Future},
sync::oneshot,
};
use std::thread;
fn main() {
let (tx, rx) = oneshot::channel();
let thr = thread::spawn(|| {
let mut runtime = tokio::runtime::current_thread::Runtime::new().unwrap();
runtime.spawn({
future::lazy(|| {
println!("event loop started");
Ok(())
})
.and_then(|_| rx.map_err(drop))
});
runtime.run()
});
let lines = ["hello", "goodbye", "exit"];
for &line in &lines {
if line == "exit" {
tx.send(()).unwrap();
break;
}
}
thr.join().unwrap().unwrap();
}

Related

Rust async function parameters and lifetimes

I'd like to understand why test1 causes an error but test2 compiles.
It seems like rust is being clever, and realising that when the .await is called directly on the async function result it knows to keep the parameter around for execution of the future but when the async is called on a separate line it can't do this.
Would love to have a link to the relevant functionality that makes this work to learn the details.
async fn do_async_thing(s: &String) {
println!("{s}");
}
fn get_string() -> String {
"sf".to_string()
}
#[tokio::test]
async fn test1() {
let a = do_async_thing(&get_string());
a.await;
}
#[tokio::test]
async fn test2() {
do_async_thing(&get_string()).await;
}
The error
error[E0716]: temporary value dropped while borrowed
--> crates/dynamo/src/error.rs:11:29
|
11 | let a = do_async_thing(&get_string());
| ^^^^^^^^^^^^ - temporary value is freed at the end of this statement
| |
| creates a temporary value which is freed while still in use
12 | a.await;
| - borrow later used here
|
= note: consider using a `let` binding to create a longer lived value
It is not directly to do with async, its because the future returned from do_async_thing holds the string reference.
You can create your own future with the same result
struct DoAsyncThingFuture<'a> {
s: &'a String
}
impl<'a> Future for DoAsyncThingFuture<'a> {
type Output = ();
fn poll(self: std::pin::Pin<&mut Self>, cx: &mut std::task::Context<'_>) -> Poll<Self::Output> {
println!("{}", self.s);
Poll::Ready(())
}
}
fn do_async_thing(s: &String) -> DoAsyncThingFuture {
DoAsyncThingFuture {
s
}
}
And even get the same result without a future
fn do_sync_thing(s: &String) -> &String {
s
}
Attempting to use the return value from either of these functions will give the same error. This happens the return value of get_string does not have an owner so it is dropped after the call to do_sync_thing witch means the return reference is dangling. So as why one works and the other does not:
let a = do_sync_thing(&get_string());
println!("{}", a);
//Same as
let _temp_value = get_string();
let a = do_async_thing(&_temp_value);
drop(_temp_value);
println!("{}", a);
vs
println!("{}", do_sync_thing(&get_string()));
//Same as
let _temp_value = get_string();
println!("{}", do_async_thing(&_temp_value));
drop(_temp_value);

Borrow non-'static data in future with wasm_bindgen_futures

I am launching multiple threads from incoming tcp connections which need to read from a file or anything that implements the std::io::Read and std::io::Seek trait.
I have the following working implementation (1):
fn upload<R: Read + Seek + Send>(file: &mut R, listen_addr: SocketAddr) -> IoResult<()> {
let listener = TcpListener::bind(listen_addr)?;
let file: Arc<Mutex<&mut R>> = Arc::new(Mutex::new(file));
std::thread::scope(|scope| {
for socket in listener.incoming() {
let socket = socket.unwrap() // error handling omitted ..
let f = file.clone();
scope.spawn(move || {
start_upload_task(socket, f).unwrap();
});
}
}
Ok(())
}
fn start_upload_task<R: Read + Seek>(conn: TcpStream, file: Arc<Mutex<&mut R>>) -> IoResult<()>{
// do something wtih file ..
let mut buf = vec![0u8; 42];
{
let mut file = file.lock().unwrap();
file.seek(SeekFrom::Start(42))?;
file.read_exact(&mut buf)?;
}
Ok(())
}
However, since I want to use wasm_bindgen_futures::spawn_local, the future must be 'static and there is no equivalent to std::thread::scope in rust webassembly that I know of.
What I end up with is the following (2):
fn upload<R: Read + Seek + Send>(file: &mut R, listen_addr: SocketAddr) -> IoResult<()> {
let listener = TcpListener::bind(listen_addr)?;
let file: Arc<Mutex<&mut R>> = Arc::new(Mutex::new(file));
for socket in listener.incoming() {
let socket = socket.unwrap() // error handling omitted ..
let f = file.clone();
std::thread::spawn(move || {
start_upload_task(socket, f).unwrap();
});
}
Ok(())
}
But this gives me the following compile error:
borrowed data escapes outside of associated function
requirement occurs because of the type Mutex<&mut R>, which makes the generic argument &mut R invariant
the struct Mutex<T> is invariant over the parameter T
see https://doc.rust-lang.org/nomicon/subtyping.html for more information about variance rustcE0521
lib.rs(258, 47): file is a reference that is only valid in the associated function body
with the following suggestion:
--> src/lib.rs:273:25
|
258 | fn upload<R: Read + Seek + Send>(&self, file: &mut R, listen_addr: SocketAddr) -> IoResult<()> {
| ---- - let's call the lifetime of this reference `'1`
| |
| `file` is a reference that is only valid in the associated function body
How do I get implementation (2) to work?

Passing a neon::Context into an internal function

I've got an exported-to-NodeJS Rust function that does something. Now, I want to extract part of that function into its own function (creating a JS object), so that I can reuse that logic in another, new exporter-to-NodeJS Rust function. However, I'm having great troubles figuring out how to pass the Context into that sub-function in a way that Rust is happy with. So far, the best (as in, "causes the compiler to complain the least") solution I have is this:
fn encode_num(mut cx: FunctionContext) -> JsResult<JsObject> {
let input = cx.argument::<JsNumber>(0)?.value(&mut cx);
let output = OrePlaintext::<u64>::from(input).0;
plaintext_from_u64(&mut cx, output)
}
fn plaintext_from_u64<'a>(cx: &'a mut FunctionContext, n: u64) -> JsResult<'a, JsObject> {
let bytes = n.to_ne_bytes();
let mut buf = match cx.buffer(8) {
Ok(b) => b,
Err(e) => return cx.throw_error(format!("Failed to allocate buffer: {:?}", e))
};
cx.borrow_mut(&mut buf, |data| {
let slice = data.as_mut_slice::<u8>();
for i in 0..8 {
slice[i] = bytes[i];
};
});
let obj = cx.empty_object();
obj.set(&mut cx, "kind", cx.string("OrePlainText"))?;
obj.set(&mut cx, "buf", buf)?;
Ok(obj)
}
#[neon::main]
fn main(mut cx: ModuleContext) -> NeonResult<()> {
cx.export_function("encodeNumber", encode_num)?;
Ok(())
}
This fails to compile with the following errors:
error[E0277]: the trait bound `&'a mut CallContext<'_, neon::prelude::JsObject>: neon::context::Context<'_>` is not satisfied
--> native/lib.rs:120:9
|
120 | obj.set(&mut cx, "kind", cx.string("OrePlainText"))?;
| ^^^ the trait `neon::context::Context<'_>` is not implemented for `&'a mut CallContext<'_, neon::prelude::JsObject>`
|
= help: the following implementations were found:
<CallContext<'a, T> as neon::context::Context<'a>>
error[E0277]: the trait bound `&'a mut CallContext<'_, neon::prelude::JsObject>: neon::context::Context<'_>` is not satisfied
--> native/lib.rs:121:9
|
121 | obj.set(&mut cx, "buf", buf)?;
| ^^^ the trait `neon::context::Context<'_>` is not implemented for `&'a mut CallContext<'_, neon::prelude::JsObject>`
|
= help: the following implementations were found:
<CallContext<'a, T> as neon::context::Context<'a>>
I started with plaintext_from_u64 just taking the context as mut cx: FunctionContext, which is how encode_num accepts it, but that then causes problems as soon as I want to call plaintext_from_u64 more than once, because "value borrowed after move", and if I try to borrow in the calls, I get "expected struct CallContext, found mutable reference. I've dug around in the neon examples repo, looking for exported functions that pass a context into an internal function, but I haven't been able to find anything so far, which is... frustrating.
What is the compiler-approved approach to this problem?

Calling an FnMut callback from another thread

I am writing a Phoenix client library for Rust, taking advantage of the async websocket client from rust-websockets. Right now I am having trouble figuring out how to pass callback functions into the thread that is handling the websocket traffic. I have a simplified struct:
pub struct Socket {
endpoint: String,
connected: Arc<AtomicBool>,
state_change_close: Option<Box<FnMut(String)>>,
}
This struct has a connect function laid out as follows:
pub fn connect(&mut self) -> Result<(), String> {
if self.connected.load(Ordering::Relaxed) {
return Ok(())
}
// Copy endpoint string, otherwise we get an error on thread::spawn
let connection_string = self.endpoint.clone();
let (usr_msg, stdin_ch) = mpsc::channel(0);
let connection_thread = thread::spawn(move || {
// tokio core for running event loop
let mut core = Core::new().unwrap();
let runner = ClientBuilder::new(&connection_string)
.unwrap()
.add_protocol("rust-websocket")
.async_connect_insecure(&core.handle())
.and_then(|(duplex, _)| {
let (sink, stream) = duplex.split();
stream.filter_map(|message| {
println!("Received Message: {:?}", message);
match message {
OwnedMessage::Close(e) => {
// This is the line where I am trying to call the callback
if let Some(ref mut func) = self.state_change_close {
(func)(e.unwrap().reason);
}
Some(OwnedMessage::Close(e))
},
_ => None,
}
})
.select(stdin_ch.map_err(|_| WebSocketError::NoDataAvailable))
.forward(sink)
});
// Start the event loop
core.run(runner).unwrap();
});
self.connected.store(true, Ordering::Relaxed);
return Ok(())
}
When I try to compile this code I get the following error:
error[E0277]: the trait bound `std::ops::FnMut(std::string::String) + 'static: std::marker::Send` is not satisfied
--> src\socket.rs:99:29
|
99 | let connection_thread = thread::spawn(move || {
| ^^^^^^^^^^^^^ the trait `std::marker::Send` is not implemented for `std::ops::FnMut(std::string::String) + 'static`
|
I have tried changing the type of state_change_close to a Mutex<Option<...>> to avoid thread safety issues, but that did not help with this problem. Is what I'm trying to do possible?
After doing some more research I realized that I just had to modify Option<Box<FnMut(String)>> to be Option<Box<FnMut(String) + Send>> and copy that around my code to everywhere that the callback might be set. Learning more about trait objects!

Rust lifetime issue in loop

How to get this example to compile without array copying or multiple calls to b() per iteration — b() has to perform some expensive parsing?
This is not the full code that I wrote, but it illustrates the problem I had. Here, Test is attempting to perform some kind of streaming parsing work. c() is the parsing function, it returns Some when parsing was successful. b() is a function that attempts to read more data from the stream when c() can not parse using the available data yet. The returned value is a slice into the self.v containing the parsed range.
struct Test {
v: [u8; 10],
index: u8,
}
impl Test {
fn b(&mut self) {
self.index = 1
}
fn c(i: &[u8]) -> Option<&[u8]> {
Some(i)
}
fn a(&mut self) -> &[u8] {
loop {
self.b();
match Test::c(&self.v) {
Some(r) => return r,
_ => continue,
}
}
}
}
fn main() {
let mut q = Test {
v: [0; 10],
index: 0,
};
q.a();
}
When compiling, it produces the following borrow checker error:
error[E0502]: cannot borrow `*self` as mutable because `self.v` is also
borrowed as immutable
--> <anon>:17:13
|
17 | self.b();
| ^^^^ mutable borrow occurs here
18 |
19 | match Test::c(&self.v) {
| ------ immutable borrow occurs here
...
24 | }
| - immutable borrow ends here
If I change a() to:
fn a(&mut self) -> Option<&[u8]> {
loop {
self.b();
if let None = Test::c(&self.v) {
continue
}
if let Some(r) = Test::c(&self.v) {
return Some(r);
} else {
unreachable!();
}
}
}
Then it runs, but with the obvious drawback of calling the parsing function c() twice.
I kind of understand that changing self while the return value depends on it is unsafe, however, I do not understand why is the immutable borrow for self.v is still alive in the next iteration, when we attempted to call b() again.
Right now "Rustc can't "deal" with conditional borrowing returns". See this comment from Gankro on issue 21906.
It can't assign a correct lifetime to the borrow if only one execution path terminates the loop.
I can suggest this workaround, but I'm not sure it is optimal:
fn c(i: &[u8]) -> Option<(usize, usize)> {
Some((0, i.len()))
}
fn a(&mut self) -> &[u8] {
let parse_result;
loop {
self.b();
match Test::c(&self.v) {
Some(r) => {
parse_result = r;
break;
}
_ => {}
}
}
let (start, end) = parse_result;
&self.v[start..end]
}
You can construct result of parsing using array indexes and convert them into references outside of the loop.
Another option is to resort to unsafe to decouple lifetimes. I am not an expert in safe use of unsafe, so pay attention to comments of others.
fn a(&mut self) -> &[u8] {
loop {
self.b();
match Test::c(&self.v) {
Some(r) => return unsafe{
// should be safe. It decouples lifetime of
// &self.v and lifetime of returned value,
// while lifetime of returned value still
// cannot outlive self
::std::slice::from_raw_parts(r.as_ptr(), r.len())
},
_ => continue,
}
}
}

Resources