I'm struggling with the borrow checker when writing an interface for my services. The underlying meilisearch_sdk SDK requires that add_documents takes a reference to a slice, which cannot be transferred across await boundaries.
The line index.add_documents(&[document], Some(&id)).await produces the following error:
note: required for the cast to the object type `dyn futures::Future<Output = std::result::Result<meilisearch_sdk::tasks::Task, meilisearch_sdk::errors::Error>> + std::marker::Send`rustc
meili.rs(10, 143): future created by async block is not `Send`
meili.rs(13, 37): has type `&[D]` which is not `Send`
meili.rs(13, 60): await occurs here, with `&[document]` maybe used later
What is the correct way to approach / rectify this?
search.rs
#[async_trait]
pub trait SearchIndex {
type IndexResult;
type IndexError;
async fn index_document<D>(&self, index: String, document: D) -> Result<Self::IndexResult, Self::IndexError> where D: SearchableDocument;
async fn search_document(&self, index: String, query: String) -> Result<Self::IndexResult, Self::IndexError>;
}
index.rs
use meilisearch_sdk::{client::Client, errors::Error, tasks::Task};
use async_trait::async_trait;
use super::super::search::{SearchIndex, SearchableDocument};
#[async_trait]
impl SearchIndex for Client {
type IndexResult = Task;
type IndexError = Error;
async fn index_document<D>(&self, index: &String, document:D ) -> Result<Self::IndexResult, Self::IndexError> where D: SearchableDocument {
let index = self.index(index);
let id = document.get_id();
index.add_documents(&[document], Some(&id)).await
}
}
add_documents
pub trait SearchableDocument: DeserializeOwned + Serialize + Debug + Send{}
pub async fn add_documents<T: DeserializeOwned + Serialize + std::fmt::Debug>(
&self,
documents: &[T],
primary_key: Option<&str>,
) -> Result<Task, Error> {
self.add_or_replace(documents, primary_key).await
}
Cargo.toml
[dependencies]
serde = "1.0.136"
serde_json = "1.0.79"
rdkafka = "0.28.0"
tokio = { version = "1.17", features = ["rt-multi-thread", "macros"] }
meilisearch-sdk = "0.15.0"
futures = "0.3.21"
log = "0.4.16"
pretty_env_logger = "0.4.0"
async-trait = "0.1.53"
Related
I am attempting to use a generic method to retrieve values from database tables that are structurally identical, but have different types for one of their columns. Simplified example below
async fn query<'a, 'r, T: DatabaseType<Item=T> + Decode<'r, Sqlite> + Type<Sqlite>>(&self, name: &'a str) -> Result<Vec<NamedValue<'a, T>>> {
let mut connection = self.pool.acquire().await?;
let mut rows = sqlx::query("Select id, value from table where name = $1")
.bind(name)
.fetch(&mut connection);
let mut results = Vec::new();
while let Some(row) = rows.try_next().await? {
results.push(NamedValue {
name,
value: row.try_get("value")?
})
}
Ok(results)
}
This will not compile, with the error: borrowed value does not live long enough, argument requires that 'row' is borrowed for 'r. The lifetime sqlx::Decode wants ('r), has to be declared as part of the query function's signature, but the resource the lifetime refers to does not exist yet, and only exists when the query executes and the stream is iterated over. I can't omit this bound on the generic, because the type does need to be decodable for try_get to work, so how do I tell the compiler that it is actually completely safe, and that the decoding is happening against a row that will definitely live longe enough for the try_get? Once the value is decoded, it will always have a static lifetime.
Rust playground doesn't include SQLx, an example that can be compiled at home is below:
[package]
name = "sqlx-minimal-example"
version = "0.1.0"
edition = "2021"
[dependencies]
tokio = { version = "1", features = ["full"] }
sqlx = { version = "0.6", features = ["runtime-tokio-rustls", "sqlite"] }
anyhow = "1.0"
futures = "0.3"
And the full application would be:
use anyhow::Result;
use sqlx::{Decode, Row, Sqlite, SqlitePool, Type};
use futures::TryStreamExt;
#[tokio::main]
async fn main() -> Result<()> {
println!("Hello, world!");
Ok(())
}
struct NamedValue<'a ,T> {
name: &'a str,
value: T
}
struct SqliteBackend {
pool: SqlitePool
}
trait DatabaseType {
type Item;
}
impl DatabaseType for f32 {
type Item = f32;
}
impl DatabaseType for i32 {
type Item = i32;
}
impl SqliteBackend {
async fn query<'a, 'r, T: DatabaseType<Item=T> + Decode<'r, Sqlite> + Type<Sqlite>>(&self, name: &'a str) -> Result<Vec<NamedValue<'a, T>>> {
let mut connection = self.pool.acquire().await?;
let mut rows = sqlx::query("Select id, value from table where name = $1")
.bind(name)
.fetch(&mut connection);
let mut results = Vec::new();
while let Some(row) = rows.try_next().await? {
results.push(NamedValue {
name,
value: row.try_get("value")?
})
}
Ok(results)
}
}
Higher-ranked trait bounds were the answer. This tells the compiler the type is decodable for all possible lifetimes.
Working function below:
async fn query<'a, T: DatabaseType<Item=T> + for<'r> Decode<'r, Sqlite> + Type<Sqlite>>(&self, name: &'a str) -> Result<Vec<NamedValue<'a, T>>> {
let mut connection = self.pool.acquire().await?;
let mut rows = sqlx::query("Select id, value from table where name = $1")
.bind(name)
.fetch(&mut connection);
let mut results = Vec::new();
while let Some(row) = rows.try_next().await? {
results.push(NamedValue {
name,
value: row.try_get("value")?
})
}
Ok(results)
}
I want to add some error handler in the rust function, so I define the rust function response like this so that I could return the error message if encount the recoverable error:
pub fn add_bill_book() -> Result<BillBook,String> {
return Err("failed".parse().unwrap())
}
and box the result to return to client like this;
fn main() {
let result = add_bill_book();
box_rest_response(result);
}
but when I compile the project, shows error like this:
➜ rust-learn git:(multiple-statement) ✗ cargo build
Compiling rust-learn v0.1.0 (/Users/xiaoqiangjiang/source/reddwarf/backend/rust-learn)
error[E0277]: the trait bound `Result<BillBook, std::string::String>: std::default::Default` is not satisfied
--> src/main.rs:8:23
|
8 | box_rest_response(result);
| ----------------- ^^^^^^ the trait `std::default::Default` is not implemented for `Result<BillBook, std::string::String>`
| |
| required by a bound introduced by this call
|
note: required by a bound in `box_rest_response`
--> src/main.rs:11:87
|
11 | pub fn box_rest_response<T>(data: T) -> content::RawJson<String> where T: Serialize + Default {
| ^^^^^^^ required by this bound in `box_rest_response`
For more information about this error, try `rustc --explain E0277`.
error: could not compile `rust-learn` due to previous error
what should I do to fix it? I think it is hard to implement the default trait for all type of Result. This is the full minimal reproduce example:
use rocket::response::content;
use rust_wheel::model::response::api_response::ApiResponse;
use serde::Serialize;
use serde::Deserialize;
fn main() {
let result = add_bill_book();
box_rest_response(result);
}
pub fn box_rest_response<T>(data: T) -> content::RawJson<String> where T: Serialize + Default {
let res = ApiResponse {
result: data,
..Default::default()
};
let response_json = serde_json::to_string(&res).unwrap();
return content::RawJson(response_json);
}
#[derive(Debug,Serialize,Deserialize,Default,Clone)]
pub struct BillBook {
pub id: i64,
pub created_time: i64,
pub updated_time: i64,
pub deleted: i32,
pub creator: i64,
pub bill_book_template_id: i32,
pub remark: Option<String>,
pub contents: Option<String>,
}
pub fn add_bill_book() -> Result<BillBook,String> {
return Err("failed".parse().unwrap())
}
and this is the Cargo.toml dependencies:
[package]
name = "rust-learn"
version = "0.1.0"
edition = "2018"
[dependencies]
rocket = { version = "=0.5.0-rc.2", features = ["json"] }
serde = { version = "1.0.64", features = ["derive"] }
serde_json = "1.0.64"
serde_derive = "1.0"
# database
diesel = { version = "1.4.7", features = ["postgres","serde_json"] }
dotenv = "0.15.0"
jsonwebtoken = "7"
chrono = "0.4"
config = "0.11"
ring = "0.16.20"
md5 = "0.7.0"
data-encoding = "2.3.2"
# reddwarf public component
rust_wheel = { git = "https://github.com/jiangxiaoqiang/rust_wheel.git" }
I tried this way to solve this problem:
return match contents {
Ok(_) => {
box_rest_response(contents.unwrap())
},
Err(_) => {
box_rest_response(contents.unwrap_err())
}
}
but the new problem is that I have to match the result in many places, is it possible to optimized it? I tried to add a new function in public lib like this:
pub fn box_rest_result<T,E>(result: Result<T,E>) -> content::RawJson<String> where T: Serialize + Default, E: std::default::Default{
return match result {
Ok(_) => {
box_rest_response(result.unwrap())
},
Err(_) => {
box_rest_response(result.unwrap_err())
}
}
}
still facing the same problem.
Result does not implement Default because there is no one-fits-all default value.
You can create a newtype struct and implement Default on it, as in How do I implement a trait I don't own for a type I don't own?.
Note that match with unwrap() is a very bad idea, since match can already extract the value:
match contents {
Ok(v) => box_rest_response(v),
Err(e) => box_rest_response(e),
}
SOLUTION:
I needed to add "use std::io::prelude::*;" to my code. I do not know why.
I am trying to read from an std::net::TcpStream but I recieve this error when calling stream.read(&buf).unwrap;
the method read exists for struct std::net::TcpStream, but its
trait bounds were not satisfied method cannot be called on
std::net::TcpStream due to unsatisfied trait bounds note: the
following trait bounds were not satisfied:
std::net::TcpStream: futures::AsyncRead
which is required by std::net::TcpStream: futures::AsyncReadExt help: items from traits can only be used if the
trait is in scoperustc(E0599) main.rs(31, 16): method cannot be called
on std::net::TcpStream due to unsatisfied trait bounds tcp.rs(49,
1): doesn't satisfy std::net::TcpStream: futures::AsyncReadExt
tcp.rs(49, 1): doesn't satisfy std::net::TcpStream: futures::AsyncRead mod.rs(580, 8): the method is available for
std::boxed::Box<std::net::TcpStream> here
Code:
use irc::client::prelude::*;
use futures::prelude::*;
use std::net::{IpAddr, Ipv4Addr, SocketAddr, TcpStream};
use std::io;
use futures::{AsyncRead, AsyncReadExt};
const NAME: &str = "nickname";
#[derive(Debug)]
struct DCC {
ip: IpAddr,
port: u16,
}
impl DCC {
fn from_msg(msg: &str) -> Result<DCC, std::num::ParseIntError> {
let msg_split: Vec<&str> = msg.split_whitespace().collect();
let ip: u32 = msg_split[3].parse()?;
let ip_addr: IpAddr = IpAddr::V4(Ipv4Addr::from(ip));
let port_num: u16 = msg_split[4].parse()?;
let dcc = DCC{
ip: ip_addr,
port: port_num,
};
return Ok(dcc);
}
async fn connect(&self) -> Result<(), io::Error>{
let socket_addr = SocketAddr::new(self.ip, self.port);
let mut socket = TcpStream::connect(socket_addr)?;
let mut buf = vec![];
socket.read(&buf).unwrap();
return Err(io::Error::new(io::ErrorKind::Other, "oh no!"));
}
}
#[tokio::main]
async fn irc_get(name: &str) -> Result<String, irc::error::Error>{
let config = Config {
nickname: Some(NAME.to_owned()),
server: Some("irc.irchighway.net".to_owned()),
port: Some(6667),
use_tls: Some(false),
channels: vec!["#ebooks".to_owned()],
..Config::default()
};
let mut client = Client::from_config(config).await?;
client.identify()?;
let mut stream = client.stream()?;
//waits for server to log us in and then sends the search request
loop{
let m = match stream.next().await{
Some(v) => v,
None => panic!("at the disco")
};
let message = match &m {
Ok(message) => match &message.command {Command::NOTICE(_s1, s2)=> {print!("{:?} \n", s2); message}, _ => message},
Err(_e) => panic!("at the disco")};
match &message.command{
Command::NOTICE(_s, msg) => { if msg.contains("Welcome to #ebooks"){break}},
_=> ()
}
}
client.send_privmsg("#ebooks", format!["#Search {}", name])?;
loop{
let m = match stream.next().await.transpose()?{
Some(m) => m,
None => panic!("at the disco")
};
match &m.command{
Command::PRIVMSG(nm, msg) => if nm == NAME {println!("{:?}",m); return Ok(String::from(msg))},
_ => ()
}
}
}
fn main() {
let dcc = DCC::from_msg(&irc_get(&"romeo and juliet").unwrap()[..]);
println!("{:?}", dcc);
}
I'm fairly new at rust and based on all of the examples in the documentation I think I'm using .read correctly. My only thought is that maybe it's because I'm trying to write the code in the impl, but I don't know if rust treats that differently. It also fails with "async fn connect..." and with "fn connect...".
The compiler was telling you the solution:
help: items from traits can only be used if the trait is in scope
You need to import the traits in order to use them:
use futures::{AsyncRead, AsyncReadExt};
Also you would probably want to use tokio::TcpStream which is async and not the std one.
I'm trying to write a Rocket / Juniper / Rust based GraphQL Server using PickleDB - an in-memory key/value store.
The pickle db is created / loaded at the start and given to rocket to manage:
fn rocket() -> Rocket {
let pickle_path = var_os(String::from("PICKLE_PATH")).unwrap_or(OsString::from("pickle.db"));
let pickle_db_dump_policy = PickleDbDumpPolicy::PeriodicDump(Duration::from_secs(120));
let pickle_serialization_method = SerializationMethod::Bin;
let pickle_db: PickleDb = match Path::new(&pickle_path).exists() {
false => PickleDb::new(pickle_path, pickle_db_dump_policy, pickle_serialization_method),
true => PickleDb::load(pickle_path, pickle_db_dump_policy, pickle_serialization_method).unwrap(),
};
rocket::ignite()
.manage(Schema::new(Query, Mutation))
.manage(pickle_db)
.mount(
"/",
routes![graphiql, get_graphql_handler, post_graphql_handler],
)
}
And I want to retrieve the PickleDb instance from the Rocket State in my Guard:
pub struct Context {
pickle_db: PickleDb,
}
impl juniper::Context for Context {}
impl<'a, 'r> FromRequest<'a, 'r> for Context {
type Error = ();
fn from_request(_request: &'a Request<'r>) -> request::Outcome<Context, ()> {
let pickle_db = _request.guard::<State<PickleDb>>()?.inner();
Outcome::Success(Context { pickle_db })
}
}
This does not work because the State only gives me a reference:
26 | Outcome::Success(Context { pickle_db })
| ^^^^^^^^^ expected struct `pickledb::pickledb::PickleDb`, found `&pickledb::pickledb::PickleDb`
When I change my Context struct to contain a reference I get lifetime issues which I'm not yet familiar with:
15 | pickle_db: &PickleDb,
| ^ expected named lifetime parameter
I tried using 'static which does make rust quite unhappy and I tried to use the request lifetime (?) 'r of the FromRequest, but that does not really work either...
How do I get this to work? As I'm quite new in rust, is this the right way to do things?
I finally have a solution, although the need for unsafe indicates it is sub-optimal :)
#![allow(unsafe_code)]
use pickledb::{PickleDb, PickleDbDumpPolicy, SerializationMethod};
use serde::de::DeserializeOwned;
use serde::Serialize;
use std::env;
use std::path::Path;
use std::time::Duration;
pub static mut PICKLE_DB: Option<PickleDb> = None;
pub fn cache_init() {
let pickle_path = env::var(String::from("PICKLE_PATH")).unwrap_or(String::from("pickle.db"));
let pickle_db_dump_policy = PickleDbDumpPolicy::PeriodicDump(Duration::from_secs(120));
let pickle_serialization_method = SerializationMethod::Json;
let pickle_db = match Path::new(&pickle_path).exists() {
false => PickleDb::new(
pickle_path,
pickle_db_dump_policy,
pickle_serialization_method,
),
true => PickleDb::load(
pickle_path,
pickle_db_dump_policy,
pickle_serialization_method,
)
.unwrap(),
};
unsafe {
PICKLE_DB = Some(pickle_db);
}
}
pub fn cache_get<V>(key: &str) -> Option<V>
where
V: DeserializeOwned + std::fmt::Debug,
{
unsafe {
let pickle_db = PICKLE_DB
.as_ref()
.expect("cache uninitialized - call cache_init()");
pickle_db.get::<V>(key)
}
}
pub fn cache_set<V>(key: &str, value: &V) -> Result<(), pickledb::error::Error>
where
V: Serialize,
{
unsafe {
let pickle_db = PICKLE_DB
.as_mut()
.expect("cache uninitialized - call cache_init()");
pickle_db.set::<V>(key, value)?;
Ok(())
}
}
This can be simply imported and used as expected, but I think I'll run into issues when the load gets to high...
I want to use the async function to parse the inbound stream progressively, but actix-web requires impl Future<Item = HttpResponse, Error = Error> as the return value.
How can I convert the future returned by async function to what actix-web requires?
I'm using Rust 1.39 nightly and actix-web 1.0.7.
http_srv.rs :
use futures::compat::Stream01CompatExt;
use futures::future::{FutureExt, TryFutureExt};
use futures::stream::TryStreamExt;
use futures01::future::Future;
use futures01::stream::Stream;
use futures01::sync::mpsc; // for `try_next`
use actix_web::*;
use bytes::Bytes;
use futures_timer::Delay;
use std::time::Duration;
fn inbound(
req: HttpRequest,
stream: web::Payload,
) -> impl Future<Item = HttpResponse, Error = Error> {
let fut = async_inbound(&req, &stream);
fut.unit_error().boxed_local().compat() // <--- compliation error here.
}
async fn async_inbound(req: &HttpRequest, stream: &web::Payload) -> HttpResponse {
let mut compat_stream = stream.compat();
loop {
let result = compat_stream.try_next().await;
if let Err(e) = result {
warn!("Failed to read stream from {} : {}", req.path(), e);
break;
}
if let Ok(option) = result {
match option {
None => {
info!("Request ends");
break;
}
Some(data) => {
println!("{:?}", data);
}
}
}
}
HttpResponse::Ok().content_type("text/html").body("RESP")
}
pub fn start(port: u16) {
info!("Starting HTTP server listening at port {} ...", port);
let _ = HttpServer::new(|| {
App::new()
.wrap(middleware::DefaultHeaders::new().header(http::header::CACHE_CONTROL, "no-cache"))
.wrap(middleware::Logger::default())
.service(web::resource("/").route(web::put().to_async(inbound)))
})
.bind(format!("0.0.0.0:{}", port))
.expect(&format!("Unable to bind on port {}", port))
.run()
.expect("Failed to start HTTP server");
}
Cargo.toml:
dependencies]
log = "0.4.8"
env_logger = "0.6.2"
chrono = "0.4.8"
actix = "0.8.3"
bytes = "0.4.12"
actix-utils = "0.4.5"
futures-timer = "0.3"
futures01 = { package = "futures", version = "0.1", optional = false }
[dependencies.actix-web]
version = "1.0.7"
features = ["ssl"]
# https://rust-lang-nursery.github.io/futures-rs/blog/2019/04/18/compatibility-layer.html
# Rust’s futures ecosystem is currently split in two:
# On the one hand we have the vibrant ecosystem built around futures#0.1 with its many libraries working on stable Rust
# and on the other hand there’s std::future ecosystem with support for the ergonomic and powerful async/await language feature.
# To bridge the gap between these two worlds we have introduced a compatibility layer as part of the futures#0.3 extension to std::future.
[dependencies.futures-preview]
version = "0.3.0-alpha.18"
default-features = false
features = ["compat", "async-await", "nightly"]
Compilation Error:
error[E0271]: type mismatch resolving `<std::pin::Pin<std::boxed::Box<dyn core::future::future::Future<Output = std::result::Result<actix_http::response::Response, ()>>>> as core::future::future::Future>::Output == std::result::Result<_, actix_http::error::Error>`
--> src/http_server.rs:39:55
|
39 | fn inbound(req: HttpRequest, stream: web::Payload) -> impl Future<Item=HttpResponse, Error=Error> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected (), found struct `actix_http::error::Error`
|
= note: expected type `std::result::Result<actix_http::response::Response, ()>`
found type `std::result::Result<_, actix_http::error::Error>`
= note: required because of the requirements on the impl of `futures_core::future::TryFuture` for `std::pin::Pin<std::boxed::Box<dyn core::future::future::Future<Output = std::result::Result<actix_http::response::Response, ()>>>>`
= note: the return type of a function must have a statically known size
std::future -> future#0.1 conversion steps:
The future needs to be TryFuture (Output = Result<T, E>)
The future needs to be Unpin (you can use boxed combinator)
Finally, you can call the compat combinator
Your inbound function:
fn inbound(
req: HttpRequest,
stream: web::Payload,
) -> impl Future<Item = HttpResponse, Error = Error> {
let fut = async_inbound(&req, &stream);
fut.unit_error().boxed_local().compat()
}
The inbound function signature is fine, but the conversion isn't.
The async_inbound function isn't TryFuture (because of -> HttpResponse). You're trying to convert it with the unit_error combinator, but the result is Result<HttpResponse, ()> and you want Result<HttpResponse, Error>. Fixed inbound function:
fn inbound(
req: HttpRequest,
stream: web::Payload,
) -> impl Future<Item = HttpResponse, Error = Error> {
let fut = async_inbound(req, stream);
fut.boxed_local().compat()
}
Your async_inbound function:
async fn async_inbound(req: &HttpRequest, stream: &web::Payload) -> HttpResponse {
// ...
}
The first issue here is to replace -> HttpResponse with -> Result<HttpResponse>. Another problem is that you're passing reg and stream by reference. Move them as there's no need to take a reference and you'll need 'static. Fixed async_inbound function:
async fn async_inbound(req: HttpRequest, stream: web::Payload) -> Result<HttpResponse> {
let mut compat_stream = stream.compat();
while let Some(data) = compat_stream.try_next().await? {
println!("{:?}", data);
}
Ok(HttpResponse::Ok().content_type("text/html").body("RESP"))
}