I'm playing with tokio websocket, I've started from the echo example and trying to change it.
What I'm trying to do is parsing an input message, extracting a number from it and sending back to the client the same message 'n' times.
What I did so far is:
let (mut write, read) = ws_stream.split();
// We should not forward messages other than text or binary.
read.try_filter(|msg| future::ready(msg.is_text() || msg.is_binary()))
.flat_map(|msg| {
let msg = msg.and_then(Message::into_text);
let times = msg.map(|v| extract(v.as_str()))
.unwrap_or(0);
let txt = msg.unwrap_or("Error".into());
repeat(Ok(Message::Text(txt)))
.take(times)
})
.forward(write)
.await
.expect("Failed to forward messages");
But I'm getting the error:
error[E0277]: the trait bound `tokio_tungstenite::tungstenite::Error: Clone` is not satisfied
--> src/main.rs:42:10
|
42 | .flat_map(|msg| {
| ^^^^^^^^ the trait `Clone` is not implemented for `tokio_tungstenite::tungstenite::Error
I'm pretty new to Rust, being the Error type not owned by me, I have no idea how to solve this
Is there some different idiomatic way to write what I'm trying to do?
Related
I need to interact with a another process.
I want to use thread two threads, one for reading in stdout and one for reading on stderr, these threads pass every line the read to a channel.
The main thread collects the lines and checks for a timeout, if a timeout occurs the process should be killed.
My Problem is how to share the std-out/err handles and be still able to kill the process later if a timeout occurs.
let mut stdout = process.stdout.unwrap();
let (tx_stdout, rx_stdout): (Sender<Result<Vec<u8>, ExecutionError>>, Receiver<Result<Vec<u8>, ExecutionError>>) = mpsc::channel();
std::thread::spawn(move || {
PowerShell::read_lines(&tx_stdout, stdout);
});
let mut stderr = process.stderr.unwrap();
let (tx_stderr, rx_stderr): (Sender<Result<Vec<u8>, ExecutionError>>, Receiver<Result<Vec<u8>, ExecutionError>>) = mpsc::channel();
std::thread::spawn(move || {
PowerShell::read_lines(&tx_stderr, &mut stderr);
});
process.kill(); // this is not possible
I understand why compiler has a problem with that, but I don't know how to solve the problem.
Using Arc causes the same issue properly.
Do you have some suggestions for me?
First of all, two things which are rather helpful and useful when asking for Rust help is to set up a reduced test case (if possible) and post it on https://play.rust-lang.org, and read and post compilation errors.
Here a reduced test case would be this:
use std::process::{Command, Stdio};
fn main() {
let mut process = Command::new("echo")
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn().expect("Failed to start echo process");
let stdout = process.stdout.unwrap();
std::thread::spawn(move || {
drop(stdout);
});
let stderr = process.stderr.unwrap();
std::thread::spawn(move || {
drop(stderr);
});
process.kill(); // this is not possible
}
and the compilation error is
error[E0382]: borrow of partially moved value: `process`
--> src/lib.rs:19:4
|
14 | let stderr = process.stderr.unwrap();
| -------- `process.stderr` partially moved due to this method call
...
19 | process.kill(); // this is not possible
| ^^^^^^^ value borrowed here after partial move
|
note: this function consumes the receiver `self` by taking ownership of it, which moves `process.stderr`
which is rather clear: process.stderr is an Option, Option::unwrap has the following signature:
pub fn unwrap(self) -> T
meaning it takes its subject by value. Meaning after the two unwrap calls the process object is not valid anymore: it's been stripped for parts. Some of the parts are still available, but methods can only be called when the object is known to be valid which is not the case here.
Hence the "partially moved" compilation error, you can't strip something for parts and still use it as-is. Well maybe you can depending on the parts, but Rust doesn't express partial dependencies so the compiler has to assume Child::kill needs the child alive and well, not just some of the bits.
There are various possibilities depending on the exact situation: in some cases you can Copy or Clone the bits you want out, or maybe you can use scoped threads to borrow them. None of these are possible here, but there is still a possibility: Option::take lets you take the content of an option, and replace it with a None. So here you could move the ChildStd* out of the structure *while leaving the structure intact`, that just requires the ability to modify the structure:
use std::process::{Command, Stdio};
fn main() {
let mut process = Command::new("echo")
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn().expect("Failed to start echo process");
let stdout = process.stdout.take().unwrap();
std::thread::spawn(move || {
drop(stdout);
});
let stderr = process.stderr.take().unwrap();
std::thread::spawn(move || {
drop(stderr);
});
process.kill(); // this is not possible
}
Im new to rust and im trying to grab the requests response body, my current code is:
let client = reqwest::Client::new();
let response_text = client.get("google").send();
println!("{}", response_text);
But this produces the error
std::result::Result<reqwest::Response, reqwest::Error>` cannot be formatted with the default formatter
Short version
Use "{:?}" instead of "{}" for debugging output.
Long version
I had to modify your code to get the same error message:
pub fn dummy() {
let client = reqwest::blocking::Client::new();
let response_text = client.get("google").send();
println!("{}", response_text);
}
which uses the blocking API rather than the async API. The async API gives a similar but slightly more complicated error.
The full error message is:
error[E0277]: `std::result::Result<reqwest::blocking::Response, reqwest::Error>` doesn't implement `std::fmt::Display`
--> src/lib.rs:5:20
|
5 | println!("{}", response_text);
| ^^^^^^^^^^^^^ `std::result::Result<reqwest::blocking::Response, reqwest::Error>` cannot be formatted with the default formatter
|
= help: the trait `std::fmt::Display` is not implemented for `std::result::Result<reqwest::blocking::Response, reqwest::Error>`
= note: in format strings you may be able to use `{:?}` (or {:#?} for pretty-print) instead
= note: required by `std::fmt::Display::fmt`
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
The first line gives the underlying cause std::result::Result<reqwest::blocking::Response, reqwest::Error> doesn't implement std::fmt::Display.
The problem is that formatting with "{}" requires an object to support std::fmt::Display, and a Result object doesn't support that. (You can check this by looking through the list of traits it implements at https://doc.rust-lang.org/std/result/enum.Result.html )
Result does implement std::fmt::Debug so long as both of its parameters do,
as is the case here.
This is the trait required to be formattable with "{:?}" rather than {}.
If you don't want debugging output you need to work a little harder to get at the content. You'll want something like
match response_text {
Err(e) => eprintln!("Error: {:?}", e);
Ok(v) => println!("Body: {}", v.text().unwrap();
}
But even this doesn't handle the case with non-UTF8 input correctly, and the unwrap will panic.
I'm trying to implement rabbitmq send/listen functionality in Rust and I have the following code:
struct RabbitMQ {
connection: Connection,
}
impl RabbitMQ {
fn connect() -> Self {
RabbitMQ {
connection: the created connection
}
}
}
impl MessageBroker for RabbitMQ {
async fn publish(&self, topic: &Topic) -> Result<PublisherConfirm, Error> {
let channel = self.connection.create_channel().await.unwrap();
RabbitMQ::create_exchange(&channel, &topic.exchange).await;
let payload = topic.message.as_bytes();
let res = channel.basic_publish(
topic.exchange.name.as_str(),
topic.queue.routing_key.as_str(),
topic.exchange.publish_options,
payload.to_vec(),
BasicProperties::default(),
);
res.await
}
}
So far so good!
Now I want to publish many messages in a for loop without waiting for the confirmation from the server, the problem is that when I spawn tokio async task I need to move my broker value and this makes it invalid for the next iteration of the loop:
let broker = RabbitMQ::connect(&connection_details).await;
for x in 1..10 {
tokio::spawn(async move {
let confirm = broker.publish(&my_topic).await.unwrap();
}).await.unwrap();
}
The above code won't compile with the following error:
error[E0382]: use of moved value: `broker`
--> src/main.rs:47:33
|
21 | let broker = RabbitMQ::connect(&connection_details).await;
| ------ move occurs because `broker` has type `message_broker::RabbitMQ`, which >does not implement the `Copy` trait
...
47 | tokio::spawn(async move {
| _________________________________^
48 | | let confirm = &broker.publish(&enable_cdn).await.unwrap();
| | ------ use occurs due to use in generator
49 | | }).await.unwrap();
| |_________^ value moved here, in previous iteration of loop
I can't implement the Copy trait as Connection isn't primitive and it seems that I can't use reference "&" to the broker.
My question is how can I accomplish this without writing n publish calls?
You're using an async move block, which means any name which is used in the block is moved into the future, regardless of the operations being performed. So writing
&broker.publish
inside the block makes no difference: first broker is moved, and the future (when polled with .await) takes an internal reference to it. So what you need to do is borrow outside the block then move that borrow inside:
let broker = RabbitMQ::connect(&connection_details).await;
for x in 1..10 {
let broker = &broker;
tokio::spawn(async move {
let confirm = broker.publish(&enable_cdn).await.unwrap();
}).await.unwrap();
}
but I think that's not going to work either: tokio::spawn is not scoped, so even though you're await-ing it, the compiler has no idea that it will not outlive broker. As far as it's concerned a tokio task can live as long as it wants. This means you're now probably going to get a lifetime error (the compiler will assume the borrow can outlive the enclosing function, and thus its origin).
An easy solution to that would be to put the Connection behind an Arc or something.
Alternatively, restructure your system to work better with the requirements of rabbitmq: no idea which you're using but amiquip states that connections are thread-safe, and channels while not thread-safe can be sent to other threads.
So rather than publish-ing to an implicit connection, in each iteration of the loop create a channel and move that into the task in order to actually perform the publication.
Also,
Now I want to publish many messages in a for loop without waiting for the confirmation from the server
aren't you still doing that since you're awaiting the result of tokio::spawn?
I am attempting to convert a Result to a Buffer:
let ufc_root: String =
String::from("https://www.ufc.com/athletes/all?filters%5B0%5D=status%3A23");
// let ufc_root: String = String::from("https://www.google.com");
let https = HttpsConnector::new(4).unwrap();
let client = Client::builder().build::<_, hyper::Body>(https);
client
.get(ufc_root.parse::<hyper::Uri>().unwrap())
.and_then(|res| {
println!("http status code: {}", res.status());
println!("http response headers:\n{:?}: ", res.headers());
res.into_body()
})
.from_err::<WebScrapeError>()
.and_then(|body| {
body.for_each(|chunk| {
println!("{}", chunk.into_bytes());
});
let jon_jones = Subject {
name: "Jon Jones".to_string(),
link: "http://www.jonjones.com".to_string(),
};
let subjects = vec![jon_jones];
Ok(subjects)
})
.from_err()
error[E0277]: the trait bound `hyper::Body: hyper::rt::Future` is not satisfied
--> src/scrapper.rs:24:14
|
24 | .and_then(|res| {
| ^^^^^^^^ the trait `hyper::rt::Future` is not implemented for `hyper::Body`
|
= note: required because of the requirements on the impl of `futures::future::IntoFuture` for `hyper::Body`
error[E0599]: no method named `from_err` found for type `futures::future::and_then::AndThen<hyper::client::ResponseFuture, hyper::Body, [closure#src/scrapper.rs:24:23: 28:14]>` in the current scope
--> src/scrapper.rs:29:14
|
29 | .from_err::<WebScrapeError>()
| ^^^^^^^^
|
= note: the method `from_err` exists but the following trait bounds were not satisfied:
`&mut futures::future::and_then::AndThen<hyper::client::ResponseFuture, hyper::Body, [closure#src/scrapper.rs:24:23: 28:14]> : hyper::rt::Future`
`&mut futures::future::and_then::AndThen<hyper::client::ResponseFuture, hyper::Body, [closure#src/scrapper.rs:24:23: 28:14]> : hyper::rt::Stream`
`futures::future::and_then::AndThen<hyper::client::ResponseFuture, hyper::Body, [closure#src/scrapper.rs:24:23: 28:14]> : hyper::rt::Future`
Why doesn't this compile?
and_then must return a future, or something that can be converted into a future via the IntoFuture trait. You're returning res.into_body(), which is not a future - it's actually a stream.
To get this to work, you'll need to convert that stream into a future which represents the body having been fully read. There's several options available to you here:
res.into_body().concat2(), which concatenates all of the read bytes into a single buffer
res.into_body().collect(), which collects all of the chunks of bytes into a Vec
res.into_body().into_future(), which will resolve to a tuple containing the first item from the stream, and the remainder of the stream itself (i.e. (T, Stream<Item = T>))
The fact that there's multiple different (and equally valid) ways to represent a Stream as a Future is likely why the former doesn't have a blanket implementation of IntoFuture.
An example of using concat2 is shown in the Hyper docs.
I'm trying to wrap my head around futures in Rust but I am confused by this code which is supposed to send messages arriving at rx to sink:
extern crate futures;
extern crate tokio_core;
extern crate websocket;
use websocket::message::OwnedMessage;
use websocket::server::InvalidConnection;
use websocket::async::Server;
use tokio_core::reactor::Core;
use futures::{Future, Sink, Stream};
use futures::sync::mpsc;
use std::{thread, time};
use futures::sync::mpsc::Receiver;
fn main() {
let mut core = Core::new().unwrap();
let (mut tx, rx) = mpsc::channel(5);
thread::spawn(|| worker(rx));
let mut i = 0;
loop {
let res = tx.clone().send(OwnedMessage::Text(format!("Test {}", i)));
core.run(res);
i += 1;
let period = time::Duration::from_millis(200);
thread::sleep(period);
}
}
fn worker(rx: Receiver<OwnedMessage>) {
let mut core = Core::new().unwrap();
let handle = core.handle();
// bind to the server
let server = Server::bind("127.0.0.1:9000", &handle).unwrap();
let f = server.incoming()
// we don't wanna save the stream if it drops
.map_err(|InvalidConnection { error, .. }| error)
.for_each(|(upgrade, addr)| {
// accept the request to be a ws connection if it does
let f = upgrade
.use_protocol("rust-websocket")
.accept()
.and_then(|(s, _)| {
let (sink, stream) = s.split();
rx // using stream (echoing back) works
.forward(sink)
.map_err(|error| {
error
})
.and_then(|(a, sink)| {
sink.send(OwnedMessage::Close(None))
})
});
handle.spawn(f.map_err(move |e| println!("Err"))
.map(move |_| println!("Done")));
Ok(())
});
core.run(f).expect("somerror");
}
As noted in the comment, using stream as input works fine. When using rx, the compiler complains about a type mismatch regarding the error types (I believe):
error[E0271]: type mismatch resolving `<futures::stream::SplitSink<websocket::client::async::Framed<tokio_core::net::TcpStream, websocket::async::MessageCodec<websocket::OwnedMessage>>> as futures::Sink>::SinkError == ()`
--> src/main.rs:47:26
|
47 | .forward(sink)
| ^^^^^^^ expected enum `websocket::WebSocketError`, found ()
|
= note: expected type `websocket::WebSocketError`
found type `()`
error[E0599]: no method named `map_err` found for type `futures::stream::Forward<futures::sync::mpsc::Receiver<websocket::OwnedMessage>, futures::stream::SplitSink<websocket::client::async::Framed<tokio_core::net::TcpStream, websocket::async::MessageCodec<websocket::OwnedMessage>>>>` in the current scope
--> src/main.rs:48:26
|
48 | .map_err(|error| {
| ^^^^^^^
|
= note: the method `map_err` exists but the following trait bounds were not satisfied:
`futures::stream::Forward<futures::sync::mpsc::Receiver<websocket::OwnedMessage>, futures::stream::SplitSink<websocket::client::async::Framed<tokio_core::net::TcpStream, websocket::async::MessageCodec<websocket::OwnedMessage>>>> : futures::Future`
These are my dependencies:
[dependencies]
websocket = "0.20.0"
futures = "0.1"
tokio-core = "0.1"
What am I missing here?
error[E0271]: type mismatch resolving
`<futures::stream::SplitSink<
websocket::client::async::Framed<
tokio_core::net::TcpStream,
websocket::async::MessageCodec<websocket::OwnedMessage>>>
as futures::Sink>::SinkError == ()`
We have two types here: <futures::stream::SplitSink<...> as futures::Sink>::SinkError and (). Where do these two types come from? Also, the first one is an unresolved associated type; perhaps we could resolve it to get some more insight? Let's trace it step by step.
First, we need to figure out why the compiler is trying to match these two types in the first place. If we look at the signature for forward, we'll see the constraint Self::Error: From<S::SinkError>. Self is the type of the stream we're calling forward on, while S is the type of the sink that's passed as an argument to forward.
We're calling forward on rx, whose type is futures::sync::mpsc::Receiver. On the documentation page for Receiver, we can see the following:
impl<T> Stream for Receiver<T>
type Item = T
type Error = ()
This shows us where the () came from. Let's look at the sink argument now.
The type of sink is futures::stream::SplitSink<websocket::client::async::Framed<tokio_core::net::TcpStream, websocket::async::MessageCodec<websocket::OwnedMessage>>> (we know this from the error message; the RLS also confirms this). On the documentation page for SplitSink, we have:
impl<S: Sink> Sink for SplitSink<S>
type SinkItem = S::SinkItem
type SinkError = S::SinkError
So SplitSink's SinkError is the same as its inner sink's SinkError. The inner sink's type is websocket::client::async::Framed<tokio_core::net::TcpStream, websocket::async::MessageCodec<websocket::OwnedMessage>>. What does the documentation for Framed say?
impl<T, U> Sink for Framed<T, U>
where
T: AsyncWrite,
U: Encoder,
<U as Encoder>::Error: From<Error>,
type SinkItem = <U as Encoder>::Item
type SinkError = <U as Encoder>::Error
Framed has two type parameters, but we only need to look at the second one, which is websocket::async::MessageCodec<websocket::OwnedMessage> here, to determine the SinkError type. Let's take a look at MessageCodec now. (Note: websocket::codec::ws::MessageCodec is reexported as websocket::async::MessageCodec.)
impl<M> Decoder for MessageCodec<M>
where
M: MessageTrait,
type Item = OwnedMessage
type Error = WebSocketError
Ah ha! The sink produces errors of type WebSocketError.
Now that we've figured out the types, let's go back to why we cared about the types in the first place. We were trying to understand why the constraint Self::Error: From<S::SinkError> wasn't met on the call to forward. We now know that the compiler is trying to resolve (): From<WebSocketError>. It looks like there's no impl From<WebSocketError> for (). Let's verify this:
extern crate websocket;
fn main() {
let a = websocket::result::WebSocketError::NoDataAvailable;
let () = From::from(a);
}
Indeed, this fails to compile:
error[E0277]: the trait bound `(): std::convert::From<websocket::WebSocketError>` is not satisfied
--> src/main.rs:5:14
|
5 | let () = From::from(a);
| ^^^^^^^^^^ the trait `std::convert::From<websocket::WebSocketError>` is not implemented for `()`
|
= note: required by `std::convert::From::from`
We can work around the missing implementation by using sink_map_err to change sink's error type.
let (sink, stream) = s.split();
let sink = sink.sink_map_err(|_| ()); // <<<<<
rx
.forward(sink)
.and_then(|(a, sink)| {
sink.send(OwnedMessage::Close(None))
})
This solves the call to forward, but now the result of this closure doesn't compose with upgrade.use_protocol("rust-websocket").accept(), which still has WebSocketError as its error type. It makes more sense to change rx's error type instead. But how do we construct a WebSocketError from a (), which carries no information?
You might be wondering, why does Receiver use () for its error type? If we look at the source code, we can see that in fact, poll never returns an error. I think it would be more appropriate if the error type was ! (the never type) or some other void type, to clearly indicate that errors are impossible; there's an issue open on futures requesting this change for futures 0.2.
Since errors are impossible, we don't need to construct a WebSocketError; we can just diverge instead, for example by panicking.
fn worker(rx: Receiver<OwnedMessage>) {
let rx = rx.map_err(|()| panic!("Receiver should never fail!"));
let mut core = Core::new().unwrap();
let handle = core.handle();
// bind to the server
let server = Server::bind("127.0.0.1:9000", &handle).unwrap();
let f = server.incoming()
// we don't wanna save the stream if it drops
.map_err(|InvalidConnection { error, .. }| error)
.for_each(|(upgrade, addr)| {
// accept the request to be a ws connection if it does
let f = upgrade
.use_protocol("rust-websocket")
.accept()
.and_then(|(s, _)| {
let (sink, stream) = s.split();
rx
.forward(sink)
.and_then(|(a, sink)| {
sink.send(OwnedMessage::Close(None))
})
});
handle.spawn(f.map_err(move |e| println!("Err"))
.map(move |_| println!("Done")));
Ok(())
});
core.run(f).expect("somerror");
}
Now, there's still an error:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/main.rs:43:31
|
30 | let rx = rx.map_err(|()| panic!("Receiver should never fail!"));
| -- captured outer variable
...
43 | .and_then(|(s, _)| {
| ^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure
Why is the closure trying to move rx? Because forward takes self by value. Why is the closure an FnMut? Watch out, Future::and_then requires an FnOnce (it's valid to move a value from a captured variable into an FnOnce closure), but Stream::for_each requires an FnMut. This makes sense: for_each will invoke the closure once for each incoming connection!
The channels you're using are multi-producer, single-consumer (hence the name mpsc), but you're trying to have multiple consumers here (each connection is trying to read from the receiver). I'll leave it to you to fix this design issue in your program. Remember that there can be multiple concurrent client connections!