How to write a hyper response body to a file? - rust

Question

I am trying to write a test program with tokio that grabs a file from a website and writes the streamed response to a file. The hyper website shows an example that uses a while loop and uses the .data() method the response body, but I'd like to manipulate the stream with .map() and a couple others.
I thought the next reasonable thing to try would be to convert the stream to an AsyncRead by using the .into_async_read() method from TryStreamExt, but that doesn't seem to work. I had to use a map to convert the hyper::error::Error into a std::error::Error to get a TryStream, but now the compiler is telling me that AsyncRead isn't implemented for the transformed stream. Here is my main.rs file and the error:
use std::error::Error;
use futures::stream::{StreamExt, TryStreamExt};
use http::Request;
use hyper::{Body, Client};
use hyper_tls::HttpsConnector;
use tokio::fs::File;
use tokio::io;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let https = HttpsConnector::new();
let client = Client::builder().build::<_, Body>(https);
let request = Request::get("some file from the internet").body(Body::empty())?;
let response = client.request(request).await?;
let mut stream = response
.body()
.map(|result| result.map_err(|error| std::io::Error::new(std::io::ErrorKind::Other, "Error!")))
.into_async_read();
let mut file = File::create("output file").await?;
io::copy(&mut stream, &mut file).await?;
Ok(())
}
error[E0277]: the trait bound `futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure#src/main.rs:20:14: 20:103]>>: tokio::io::async_read::AsyncRead` is not satisfied
--> src/main.rs:24:5
|
24 | io::copy(&mut stream, &mut file).await?;
| ^^^^^^^^ the trait `tokio::io::async_read::AsyncRead` is not implemented for `futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure#src/main.rs:20:14: 20:103]>>`
|
::: /Users/jackson/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.13/src/io/util/copy.rs:63:12
|
63 | R: AsyncRead + Unpin + ?Sized,
| --------- required by this bound in `tokio::io::util::copy::copy`
error[E0277]: the trait bound `futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure#src/main.rs:20:14: 20:103]>>: tokio::io::async_read::AsyncRead` is not satisfied
--> src/main.rs:24:5
|
24 | io::copy(&mut stream, &mut file).await?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `tokio::io::async_read::AsyncRead` is not implemented for `futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure#src/main.rs:20:14: 20:103]>>`
|
= note: required because of the requirements on the impl of `core::future::future::Future` for `tokio::io::util::copy::Copy<'_, futures_util::stream::try_stream::into_async_read::IntoAsyncRead<futures_util::stream::stream::map::Map<hyper::body::body::Body, [closure#src/main.rs:20:14: 20:103]>>, tokio::fs::file::File>`

Answer 1

You almost had it. You invoked into_async_read which gives you an implementation of futures::io::AsyncRead but you want a tokio::io::AsyncRead.
The tokio-util crate gives you a tool to do this conversion.
Add to your Cargo.toml:
tokio-util = { version = "0.3.1", features=["compat"] }
And say you add a conversion function like this:
fn to_tokio_async_read(r: impl futures::io::AsyncRead) -> impl tokio::io::AsyncRead {
tokio_util::compat::FuturesAsyncReadCompatExt::compat(r)
}
then your code could become:
let mut futures_io_async_read = response
.body()
.map(|result| result.map_err(|error| std::io::Error::new(std::io::ErrorKind::Other, "Error!")))
.into_async_read();
let tokio_async_read = to_tokio_async_read(futures_io_async_read)
let mut file = File::create("output file").await?;
io::copy(&mut tokio_async_read, &mut file).await?;