I've implemented callback_list_stream generically, but I can't seem to use it in a specialized way. The code works if I copy/paste the code per T.
use futures::channel::mpsc::Receiver;
use futures::{channel::mpsc::channel};
use libpulse_binding::{
callbacks::ListResult,
context::introspect::{SinkInfo, SinkPortInfo, SourceInfo, SourcePortInfo},
};
use std::future::Future;
use std::pin::Pin;
use std::sync::{Arc, Mutex};
use std::task::{Context, Poll, Waker};
use std::{borrow::Cow, boxed::Box};
pub fn callback_list_stream<T: MakeOwned >() -> (impl FnMut(ListResult<&T>), Receiver<<T as MakeOwned>::Owned>) {
let (mut sender, recv) = channel(1024); // TODO channel size?
let cb = {
move |c: ListResult<&T>| match c {
ListResult::Item(it) => match sender.try_send(it.make_owned()) {
Ok(_) => (),
Err(err) => eprintln!("Failed to send message {:?}", err),
},
ListResult::End => sender.disconnect(),
ListResult::Error => (), // TODO
}
};
(cb, recv)
}
pub fn foo() -> (
impl FnMut(ListResult<&SourceInfo<'_>>),
Receiver<SourceInfo<'static>>
) {
callback_list_stream()
}
The compiler complains:
error[E0277]: expected a `std::ops::FnMut<(pulse::callbacks::ListResult<&pulse::context::introspect::SourceInfo<'_>>,)>` closure, found `impl for<'r> std::ops::FnMut<(pulse::callbacks::ListResult<&'r _>,)>`
--> src/futuristic_pulse/callback_future.rs:230:5
|
230 | impl FnMut(ListResult<&SourceInfo<'_>>),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected an `FnMut<(pulse::callbacks::ListResult<&pulse::context::introspect::SourceInfo<'_>>,)>` closure, found `impl for<'r> std::ops::FnMut<(pulse::callbacks::ListResult<&'r _>,)>`
...
233 | callback_list_stream()
| ---------------------- this returned value is of type `(impl for<'r> std::ops::FnMut<(pulse::callbacks::ListResult<&'r _>,)>, futures_channel::mpsc::Receiver<pulse::context::introspect::SourceInfo<'static>>)`
|
= help: the trait `for<'r, 's> std::ops::FnMut<(pulse::callbacks::ListResult<&'r pulse::context::introspect::SourceInfo<'s>>,)>` is not implemented for `impl for<'r> std::ops::FnMut<(pulse::callbacks::ListResult<&'r _>,)>`
= note: the return type of a function must have a statically known size
error[E0271]: type mismatch resolving `for<'r, 's> <impl for<'t0> std::ops::FnMut<(pulse::callbacks::ListResult<&'t0 _>,)> as std::ops::FnOnce<(pulse::callbacks::ListResult<&'r pulse::context::introspect::SourceInfo<'s>>,)>>::Output == ()`
--> src/futuristic_pulse/callback_future.rs:230:5
|
230 | impl FnMut(ListResult<&SourceInfo<'_>>),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected bound lifetime parameter, found concrete lifetime
|
= note: the return type of a function must have a statically known size
I believe that need to add a constrain on the lifetime of the inner SourceInfo data to the original reference:
pub fn foo<'a, 'b>() -> (
impl FnMut(ListResult<&'a SourceInfo<'b>>),
Receiver<SourceInfo<'static>>,
) where 'b : 'a {
let (cb, recv) = callback_list_stream();
(cb, recv)
}
Related
I'm a new rustacean and I try to pass a function as argument in another function in order to create threads with the function pass as argument.
Here the code:
use std::os::unix::net::{UnixListener, UnixStream};
use std::thread;
use std::io::Read;
use anyhow::Context;
pub struct SocketServer {
path: String,
listener: UnixListener,
}
impl SocketServer {
pub fn new(path: &str) -> anyhow::Result<SocketServer>{
if std::fs::metadata(path).is_ok() {
println!("A socket is already present. Deleting...");
std::fs::remove_file(path).with_context(|| {
format!("could not delete previous socket at {:?}", path)
})?;
}
let socket_listener =
UnixListener::bind(path).context("Could not create the unix socket")?;
let path = path.to_string();
Ok(SocketServer{ path, listener: socket_listener })
}
pub fn start(&self, f: &dyn Fn(UnixStream)) -> anyhow::Result<()>{
for stream in self.listener.incoming() {
match stream {
Ok(stream) => {
thread::spawn(||f(stream));
}
Err(err) => {break;}
}
}
Ok(())
}
}
pub fn handle_stream(mut unix_stream: UnixStream) -> anyhow::Result<()> {
let mut message = String::new();
unix_stream
.read_to_string(&mut message)
.context("Failed at reading the unix stream")?;
println!("We received this message: {}", message);
Ok(())
}
Here the error I get when in try to compile:
error[E0277]: `dyn Fn(UnixStream)` cannot be shared between threads safely
--> src/socket.rs:34:35
|
34 | thread::spawn(||f(stream));
| ------------- ^^^^^^^^^^^ `dyn Fn(UnixStream)` cannot be shared between threads safely
| |
| required by a bound introduced by this call
|
= help: the trait `Sync` is not implemented for `dyn Fn(UnixStream)`
= note: required for `&dyn Fn(UnixStream)` to implement `Send`
I got some information in the Rust book but I still don't understand which function need to implement what.
Can you give me some hints? (Advice on other parts are welcome too)
I tried to remove closure but it goes to another error:
error[E0277]: expected a `FnOnce<()>` closure, found `()`
--> src/socket.rs:34:35
|
34 | thread::spawn(f(stream));
| ------------- ^^^^^^^^^ expected an `FnOnce<()>` closure, found `()`
| |
| required by a bound introduced by this call
|
= help: the trait `FnOnce<()>` is not implemented for `()`
= note: wrap the `()` in a closure with no arguments: `|| { /* code */ }`
note: required by a bound in `spawn`
--> /rust/lib/rustlib/src/rust/library/std/src/thread/mod.rs:661:8
|
661 | F: FnOnce() -> T,
| ^^^^^^^^^^^^^ required by this bound in `spawn`
Is there a reason you need a dyn Fn?
The easiest is probably to accept a impl Fn(UnixStream) + Send + Clone + 'static instead
pub fn start<F>(&self, f: F) -> anyhow::Result<()>
where
F: Fn(UnixStream) + Send + Clone + 'static,
{
for stream in self.listener.incoming() {
let f = f.clone();
match stream {
Ok(stream) => {
thread::spawn(move || f(stream));
}
Err(err) => break,
}
}
Ok(())
}
The dyn Fn() may capture things (such as Rcs) that are unsafe to share between threads. So you need to mark it Sync:
pub fn start(&self, f: &(dyn Fn(UnixStream) + Sync)) -> anyhow::Result<()> {
Now you'll get another error:
error[E0521]: borrowed data escapes outside of associated function
--> src/lib.rs:34:21
|
30 | pub fn start(&self, f: &(dyn Fn(UnixStream) + Sync)) -> anyhow::Result<()> {
| - - let's call the lifetime of this reference `'1`
| |
| `f` is a reference that is only valid in the associated function body
...
34 | thread::spawn(|| f(stream));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| `f` escapes the associated function body here
| argument requires that `'1` must outlive `'static`
Because the function's captured data may be dropped before the thread exits, causing use after free. The easiest solution is to use Arc:
pub fn start(&self, f: Arc<dyn Fn(UnixStream) + Sync + Send>) -> anyhow::Result<()> {
for stream in self.listener.incoming() {
match stream {
Ok(stream) => {
let f = Arc::clone(&f);
thread::spawn(move || f(stream));
}
Err(err) => {
break;
}
}
}
Ok(())
}
(You also need a Send bound).
On the code below, I'm trying to transform a reference to owned data. The method to_owned ensures a 'static lifetime for the new returned type. However, it's complaining that this returned type is not really 'static because of the impl <'a>. I don't see what does 'a have to do with this since all the returned objects in to_owned are owned and thus should be 'static.
pub enum RRtspEncodedPacket<'a> {
Owned(Data<Body>),
Ref(&'a Data<Body>)
}
pub struct Body(Inner);
enum Inner {
Vec(Vec<u8>),
Custom(Box<dyn Custom>),
}
trait Custom: AsRef<[u8]> + Send + Sync + 'static {}
pub struct Data<Body> {
pub(crate) channel_id: u8,
pub(crate) body: Body,
}
pub trait EncodedPacket<'a, T: Send>: Send {
fn to_owned(&self) -> Box<dyn EncodedPacket<'static, T>>;
}
impl<'a> EncodedPacket<'a, u8> for RRtspEncodedPacket<'a> {
fn to_owned(&self) -> Box<dyn EncodedPacket<'static, u8>> {
match self {
Self::Owned(o) => Box::new(Self::Owned(*o.clone())),
Self::Ref(data) => Box::new(Self::Owned(*(*data).clone()))
}
}
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=1eafce83690156b9df6432fd7092fec0
Error:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
--> src/lib.rs:27:40
|
27 | Self::Owned(o) => Box::new(Self::Owned(*o.clone())),
| ^^^^^^^^^^^
|
note: first, the lifetime cannot outlive the lifetime `'a` as defined on the impl at 24:6...
--> src/lib.rs:24:6
|
24 | impl<'a> EncodedPacket<'a, u8> for RRtspEncodedPacket<'a> {
| ^^
note: ...so that the types are compatible
--> src/lib.rs:27:40
|
27 | Self::Owned(o) => Box::new(Self::Owned(*o.clone())),
| ^^^^^^^^^^^
= note: expected `RRtspEncodedPacket<'_>`
found `RRtspEncodedPacket<'a>`
= note: but, the lifetime must be valid for the static lifetime...
note: ...so that the expression is assignable
--> src/lib.rs:27:31
|
27 | Self::Owned(o) => Box::new(Self::Owned(*o.clone())),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= note: expected `Box<(dyn EncodedPacket<'static, u8> + 'static)>`
found `Box<dyn EncodedPacket<'static, u8>>`
In to_owned() you are promising that you will return a Box<dyn EncodedPacket<'static, u8>>. But then you are using a Box<RRtspEncodedPacket<'a>> to construct the return value; the 'a may the shorter than 'static, which is what the compiler is complaining about. This is facilitated by the use of Self in Box::new(Self::Owned(*o.clone())). Here, Self refers to the type the impl block is for, so Self is RRtspEncodedPacket<'a>; this is how 'a sneaks into the return type. But what you want is RRtspEncodedPacket<'static>. Simply switching Self to RRtspEncodedPacket - which the compiler will infer to RRtspEncodedPacket<'static> - will solve your problem.
Reduced example that compiles:
pub enum RRtspEncodedPacket<'a> {
Owned(u32),
Ref(&'a u32)
}
pub trait EncodedPacket<'a, T> {
fn to_owned(&self) -> Box<dyn EncodedPacket<'static, T>>;
}
impl<'a> EncodedPacket<'a, u8> for RRtspEncodedPacket<'a> {
fn to_owned(&self) -> Box<dyn EncodedPacket<'static, u8>> {
// Notice: No `Self`, because we go from whatever `Self` is
// (some `RRtspEncodedPacket<'a>` for some `'a`) to
// `RRtspEncodedPacket<'static>`.
match self {
Self::Owned(o) => Box::new(RRtspEncodedPacket::Owned(o.clone())),
Self::Ref(data) => Box::new(RRtspEncodedPacket::Owned((*data).clone()))
}
}
}
I'm trying to understand Rust futures. I did:
#[derive(Debug)]
pub enum ClientError {
UrlParseError((String, String)),
BindError
}
pub async fn connect(addr: std::net::SocketAddr) -> impl std::future::Future<Output = std::result::Result<(), ClientError>> {
let listener = async_std::net::TcpListener::bind(&addr).await;
let listener = match listener {
Ok(listener) => listener,
Err(_) => return Err(ClientError::BindError)
};
//Ok(())
std::future::ready(Ok(()))
}
I get
error[E0277]: `std::result::Result<_, ClientError>` is not a future
--> src/lib.rs:7:53
|
7 | pub async fn connect(addr: std::net::SocketAddr) -> impl std::future::Future<Output = std::result::Result<(), ClientError>> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `std::result::Result<_, ClientError>` is not a future
|
= help: the trait `Future` is not implemented for `std::result::Result<_, ClientError>`
error[E0308]: mismatched types
--> src/lib.rs:14:5
|
14 | std::future::ready(Ok(()))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `std::result::Result`, found struct `std::future::Ready`
|
= note: expected type `std::result::Result<_, ClientError>`
found struct `std::future::Ready<std::result::Result<(), _>>`
note: return type inferred to be `std::result::Result<_, ClientError>` here
--> src/lib.rs:11:26
|
11 | Err(_) => return Err(ClientError::BindError)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^
I honestly don't know what is wrong. I cannot return a Result from a future?
How can I fix this?
I tried returning just Ok(()) and I don't get the second error
Is this works for you?
#[derive(Debug)]
pub enum ClientError {
UrlParseError((String, String)),
BindError
}
pub async fn connect(addr: std::net::SocketAddr) -> Result<(), ClientError> {
let listener = async_std::net::TcpListener::bind(&addr)
.await
.or(Err(ClientError::BindError))?
;
Ok(())
}
I'm new to Rust and I'm sorry if I'm using terms incorrect. Maybe my choice of words for question is incorrect.
I was playing with streams and I needed to have some delay between stream elements. So I wrote this:
use futures::stream;
use futures::StreamExt;
use tokio::time;
#[tokio::main]
async fn main() {
let mut stream = stream::iter(0..1000).then(|x| async move {
time::delay_for(std::time::Duration::from_millis(500)).await;
x + 1
});
while let Some(x) = stream.next().await {
println!("{:?}", x)
}
}
I get a lot of compilation errors, but the most important errors are connected with pinning. Here they are:
error[E0277]: `std::future::from_generator::GenFuture<[static generator#src/main.rs:7:64: 10:6 x:_ _]>` cannot be unpinned
--> src/main.rs:11:32
|
11 | while let Some(x) = stream.next().await {
| ^^^^ within `futures_util::stream::stream::then::_::__Then<'_, futures_util::stream::iter::Iter<std::ops::Range<{integer}>>, impl core::future::future::Future, [closure#src/main.rs:7:49: 10:6]>`, the trait `std::marker::Unpin` is not implemented for `std::future::from_generator::GenFuture<[static generator#src/main.rs:7:64: 10:6 x:_ _]>`
|
= note: required because it appears within the type `impl core::future::future::Future`
= note: required because it appears within the type `std::option::Option<impl core::future::future::Future>`
= note: required because it appears within the type `futures_util::stream::stream::then::_::__Then<'_, futures_util::stream::iter::Iter<std::ops::Range<{integer}>>, impl core::future::future::Future, [closure#src/main.rs:7:49: 10:6]>`
= note: required because of the requirements on the impl of `std::marker::Unpin` for `futures_util::stream::stream::then::Then<futures_util::stream::iter::Iter<std::ops::Range<{integer}>>, impl core::future::future::Future, [closure#src/main.rs:7:49: 10:6]>`
error[E0277]: `std::future::from_generator::GenFuture<[static generator#src/main.rs:7:64: 10:6 x:_ _]>` cannot be unpinned
--> src/main.rs:11:25
|
11 | while let Some(x) = stream.next().await {
| ^^^^^^^^^^^^^^^^^^^ within `futures_util::stream::stream::then::_::__Then<'_, futures_util::stream::iter::Iter<std::ops::Range<{integer}>>, impl core::future::future::Future, [closure#src/main.rs:7:49: 10:6]>`, the trait `std::marker::Unpin` is not implemented for `std::future::from_generator::GenFuture<[static generator#src/main.rs:7:64: 10:6 x:_ _]>`
|
= note: required because it appears within the type `impl core::future::future::Future`
= note: required because it appears within the type `std::option::Option<impl core::future::future::Future>`
= note: required because it appears within the type `futures_util::stream::stream::then::_::__Then<'_, futures_util::stream::iter::Iter<std::ops::Range<{integer}>>, impl core::future::future::Future, [closure#src/main.rs:7:49: 10:6]>`
= note: required because of the requirements on the impl of `std::marker::Unpin` for `futures_util::stream::stream::then::Then<futures_util::stream::iter::Iter<std::ops::Range<{integer}>>, impl core::future::future::Future, [closure#src/main.rs:7:49: 10:6]>`
= note: required because of the requirements on the impl of `core::future::future::Future` for `futures_util::stream::stream::next::Next<'_, futures_util::stream::stream::then::Then<futures_util::stream::iter::Iter<std::ops::Range<{integer}>>, impl core::future::future::Future, [closure#src/main.rs:7:49: 10:6]>>`
If I change my code to this:
use futures::stream;
use futures::StreamExt;
use tokio::time;
#[tokio::main]
async fn main() {
let mut stream = stream::iter(0..1000).then(|x| {
futures::future::ready(x + 1)
});
while let Some(x) = stream.next().await {
println!("{:?}", x)
}
}
Or to this:
use futures::stream;
use futures::StreamExt;
use tokio::time;
#[tokio::main]
async fn main() {
stream::iter(0..1000)
.then(|x| async move {
time::delay_for(std::time::Duration::from_millis(500)).await;
x + 1
})
.for_each(|x| async move { println!("{:?}", x) })
.await;
}
It compiles.
I assume it has something to do with pinning and simultaneous usage of then combinator and while, but I can't wrap my head around it.
I think that the issue boils down to the fact that async blocks are not Unpin. It can be proven with this code:
fn check_unpin<F: Unpin>(_: F) { }
fn main() {
check_unpin(async {});
}
that fails with the rather cryptic message:
error[E0277]: `std::future::from_generator::GenFuture<[static generator#src/main.rs:4:23: 4:25 _]>` cannot be unpinned
--> src/main.rs:4:5
|
1 | fn check_unpin<F: Unpin>(_: F) { }
| ----- required by this bound in `check_unpin`
...
4 | check_unpin(async {});
| ^^^^^^^^^^^ within `impl std::future::Future`, the trait `std::marker::Unpin` is not implemented for `std::future::from_generator::GenFuture<[static generator#src/main.rs:4:23: 4:25 _]>`
|
= note: required because it appears within the type `impl std::future::Future`
I reckon that GenFuture is the internal type that converts an async block into a impl Future.
Now back to your issue, the combinator then() returns an Unpin value if both the stream and the Future returned by the closure are Unpin (it is not totally clear in the documentation, but I've inferred that from the source code). The stream::iter is Unpin, but when you write |x| async move { x + 1} you are returning an async block that is not Unpin, thus your error.
If you use futures::future::ready(x + 1) it works simply because that Future implements Unpin.
If you use StreamExt::for_each it works because it does not require Self to be Unpin. It is not Unpin itself, but it does not matter because you are sending it up to tokio::main that pins everything internally before polling.
If you want your original code to work you just have to manually pin your stream (playground):
use futures::stream;
use futures::StreamExt;
use tokio::time;
use pin_utils::pin_mut;
#[tokio::main]
async fn main() {
let stream = stream::iter(0..1000).then(|x| async move {
time::delay_for(std::time::Duration::from_millis(500)).await;
x + 1
});
pin_mut!(stream); //<---- here, pinned!
while let Some(x) = stream.next().await {
println!("{:?}", x)
}
}
I'm unable to work out how to resolve the compilation error resulting from adapting a stream using a combinator.
The following Rust Playground demonstrates a fairly minimal example:
use futures::prelude::*;
use futures::StreamExt;
#[derive(Debug)]
pub enum Input {
A,
B(i32),
C(u16),
}
#[derive(Debug)]
enum Output {
Int(i32),
Short(u16),
}
pub struct StreamMaker;
impl StreamMaker {
/// make a stream with a series of inputs
pub fn create(self) -> impl Stream<Item = Input> {
stream::iter(vec![Input::A, Input::C(1u16), Input::B(2)])
}
}
/// consume the creator, and make output messages for a subset
pub fn adapt_stream(creator: StreamMaker) -> impl Stream<Item = Output> {
let mut upstream = creator.create();
upstream.filter_map(|message| async move {
match message {
Input::A => None,
Input::B(v) => Some(Output::Int(v)),
Input::C(v) => Some(Output::Short(v)),
}
})
}
#[tokio::main]
async fn main() -> Result<(), ()> {
let creator = StreamMaker {};
let mut stream = adapt_stream(creator);
while let Some(message) = stream.next().await {
println!("message: {:?}", message)
}
Ok(())
}
Compilation fails with:
error[E0277]: the trait bound `std::future::GenFuture<[static generator#src/main.rs:29:46: 35:6 message:Input {}]>: std::marker::Unpin` is not satisfied in `impl core::future::future::Future`
--> src/main.rs:43:38
|
43 | while let Some(message) = stream.next().await {
| ^^^^ within `impl core::future::future::Future`, the trait `std::marker::Unpin` is not implemented for `std::future::GenFuture<[static generator#src/main.rs:29:46: 35:6 message:Input {}]>`
|
= help: the following implementations were found:
<std::future::GenFuture<T> as std::marker::Unpin>
= note: required because it appears within the type `impl core::future::future::Future`
= note: required because of the requirements on the impl of `std::marker::Unpin` for `futures_util::stream::stream::filter_map::FilterMap<impl futures_core::stream::Stream, impl core::future::future::Future, [closure#src/main.rs:29:25: 35:6]>`
= note: required because it appears within the type `impl futures_core::stream::Stream`
error[E0277]: the trait bound `std::future::GenFuture<[static generator#src/main.rs:29:46: 35:6 message:Input {}]>: std::marker::Unpin` is not satisfied in `impl core::future::future::Future`
--> src/main.rs:43:31
|
43 | while let Some(message) = stream.next().await {
| ^^^^^^^^^^^^^^^^^^^ within `impl core::future::future::Future`, the trait `std::marker::Unpin` is not implemented for `std::future::GenFuture<[static generator#src/main.rs:29:46: 35:6 message:Input {}]>`
|
= help: the following implementations were found:
<std::future::GenFuture<T> as std::marker::Unpin>
= note: required because it appears within the type `impl core::future::future::Future`
= note: required because of the requirements on the impl of `std::marker::Unpin` for `futures_util::stream::stream::filter_map::FilterMap<impl futures_core::stream::Stream, impl core::future::future::Future, [closure#src/main.rs:29:25: 35:6]>`
= note: required because it appears within the type `impl futures_core::stream::Stream`
= note: required because of the requirements on the impl of `core::future::future::Future` for `futures_util::stream::stream::next::Next<'_, impl futures_core::stream::Stream>`
I can put a pin_mut!(stream); in the main, but I'd like to be able to push that upstream.
If you do not want the consumer of your stream to have to pin it themselves, you need to return a stream that implements the Unpin trait, meaning that it is safe to move around in memory even after is has been pinned.
pub fn adapt_stream(creator: StreamMaker) -> impl Stream<Item = Output> + Unpin {
// add Unpin trait --^
Adding this, your compiler should complain that the return value doesn't implement Unpin. This is because async move { ... } blocks don't implement Unpin, since they may be self-referential (e.g. contain references to variables they own). The most general way to work around this is to pin the stream to the heap with a Pin<Box<_>>, using the Box::pin constructor:
pub fn adapt_stream(creator: StreamMaker) -> impl Stream<Item = Output> + Unpin {
let mut upstream = creator.create();
Box::pin(upstream.filter_map(|message| async move {
// ^-- pin stream to heap
match message {
Input::A => None,
Input::B(v) => Some(Output::Int(v)),
Input::C(v) => Some(Output::Short(v)),
}
}))
}
Since we're now returning a Pin<Box<_>> pointer to the stream, that pointer can be safely moved around in memory while the inner stream is kept at the same location.
Full playground example