at the moment I'm writing a pure Rust MQTT5 library (I know there are existing ones out there, but I'm more trying to learn Rust) and I stumpled upon this problem.
I'm using latest stable rust with tokio 1.0.1.
When I send out a packet over the wire, I often expect a response from the server (example below PingReq/PingAck, Ping/Pong).
Leaving out a lot if logic regarding timeouts and packet clashes I wrote a simplified version of the logic in JavaScript (since I know that fairly well).
How would this logic translate to Rust and its futures?
Or to be more clear: Can I somehow recreate the resolve() callback function behavior of awaitPackage + onIncomingPacket?
class Client {
awaitedPacketTypes = {};
/**
* a ping consist of a send ping and a receive pong
*/
async ping(){
await this.sendPacket("Ping");
return await this.awaitPackage("Pong");
}
async sendPacket(packetType) { /*...*/ }
/**
* This expects a specific packet type to be received in the future
* #param {*} packetType
*/
awaitPackage(packetType) {
return new Promise((resolve, reject) => {
this.awaitedPacketTypes[packetType] = {
resolve,
reject
};
});
}
/**
* This gets called for every packet from the network side and calls the correct resolver if something waits for this packet type
* #param {*} packet
*/
onIncomingPacket(packet) {
if(this.awaitedPacketTypes[packet.type]) {
this.awaitedPacketTypes[packet.type].resolve(packet);
this.awaitedPacketTypes[packet.type] = undefined;
} else {
/*...*/
}
}
}
Or to be more clear: Can I somehow recreate the resolve() callback function behavior of awaitPackage + onIncomingPacket?
Kinda? A rust Future is only "something which can be polled for readiness", it's a much lower-level concept than a JS promise.
There are libraries which claim to provide JS-style promises, but most every async library probably provides a similar object named differently e.g. in Tokio, you'd probably want a oneshot channel, that is a channel on which a single value can be sent, resulting in something along the lines of:
struct Packet { r#type: &'static str }
struct Client {
awaited: Mutex<HashMap<&'static str, Sender<Packet>>>
}
impl Client {
async fn ping(&self) -> Packet {
self.send_packet("Pong").await;
self.await_package("Pong").await.unwrap()
}
async fn send_packet(&self, _: &'static str) {}
fn await_package(&self, packet_type: &'static str) -> Receiver<Packet> {
let (tx, rx) = channel();
self.awaited.lock().unwrap().insert(packet_type, tx);
rx
}
fn on_incoming_packet(&self, packet: Packet) {
if let Some(tx) = self.awaited.lock().unwrap().remove(packet.r#type) {
tx.send(packet);
}
}
}
Related
I am trying to figure out a way to make my tokio_postgres client a variable that I can reuse in different parts of my app. Ideally, I'm trying to achieve something similar to the Prisma ORM in the Node world:
const prisma = new PrismaClient()
...
const user = await prisma.user.create({
data: {
name: 'Alice',
email: 'alice#prisma.io',
},
The code I have so far is:
async fn connect() -> Result<P::Client, PgError> {
// Connect to the database.
let (client, connection) =
tokio_postgres::connect("host=localhost user=postgres", NoTls).await?;
// The connection object performs the actual communication with the database,
// so spawn it off to run on its own.
tokio::spawn(async move {
if let Err(e) = connection.await {
eprintln!("connection error: {}", e);
}
});
// Now we can execute a simple statement that just returns its parameter.
let rows = client
.query("SELECT $1::TEXT", &[&"hello world"])
.await?;
// And then check that we got back the same string we sent over.
let value: &str = rows[0].get(0);
assert_eq!(value, "hello world");
return client;
}
However, I am getting the error:
expected type Result<tokio_postgres::Client, tokio_postgres::Error>
found struct tokio_postgres::Client
Any idea what I could be doing wrong here? I'm new to Rust and maybe I'm just bringing baggage from Node, but I haven't found any documentation on this and figured it would be good to have.
Let's consider an example import object that looks like this:
const importObject = {
exampleAsyncImportFunction: async () => fs.readFile("./someExampleFile.txt", "utf-8") // returns Promise<String>
};
I want to use it in my rust-written WASM-Module similar to this:
#[wasm_bindgen]
extern "C" {
pub async fn exampleAsyncImportFunction() -> JsValue; // Importing the JS-Function
}
#[wasm_bindgen]
pub async fn exampleExportFunction() -> Result<JsValue, JsValue> {
let theJSPromise = exampleAsyncImportFunction(); // Call the async import function
let promiseResult = theJSPromise.await; // Execute the async import function
// Do sth with the result
OK(JsValue::UNDEFINED)
}
Unfortunately, this leads to an unreachable error. Interestingly, it does work, when the JavaScript-Function returns e.g. a string, like this:
const importObject = {
exampleAsyncImportFunction: async () => "Some dummy content" // returns Promise<String> as well
};
But of course, an async import function should perform some actual asynchronous tasks, otherwise it would be better to use a synchronous function.
I tried to do some research and found 'js_sys::Promise' which represents a JS-Promise in Rust, and 'wasm_bindgen_futures::JsFuture' which enables to somehow convert a JS-Promise to a Rust-Future. But I don't understand, if/how these types may help with the problem and also how they should be used in general in this context.
Also it seems, that the return type of the JS-Import-Function, which is 'JsValue' by declaration, somehow implements the 'Future'-Trait. I understand that due to that, 'awaiting' the result is possible. But I'm confused, what's the actual underlying type (JsFuture, Promise, something else...).
I hope that someone can help me with this, by solving the actual problem, but also explaining a bit the relationships between all the types (especially regarding JsValue).
Thank you in advance!
I found the solution myself, and as there seems to be no good information elsewhere in the internet, I want to share my solution for anybody, who has similar issues.
For importing the JavaScript-Function, it is important to act like the function was synchronous, although it is asynchronous. The reason for that is, that async functions in Rust are only executed, when an executor or await is used. Therefore, the async-JS-Function, which under the hood is a synchronous function that returns a Promise can be used synchronously. The returned promise can then be called explicitly.
The import section will look like the following:
#[wasm_bindgen(module = "/some/file.js")]
extern "C" {
pub fn exampleAsyncImportFunction() -> JsValue; // JsValue <==> Promise
// Instead of:
// pub async fn exampleAsyncImportFunction() -> Result<JsValue, JsValue>
}
To consume the import function, the Promise as a JsValue has to be converted into a js_sys::Promise and then to a wasm_bindgen_futures::JsFuture. After that it can be awaited as expected.
The sample export function from the original questions will look like this:
#[wasm_bindgen]
pub async fn exampleExportFunction() {
// Get a reference to the import function as a 'JsValue'
let promiseAsJsValue = exampleAsyncImportFunction(); // No execution, yet
// Convert 'JsValue' to 'js_sys::Promise', which is a proxy-type for JS-Promises
let promise = js_sys::Promise::from(promiseAsJsValue);
// Convert 'js_sys::Promise' to 'wasm_bindgen_future::JsFuture'
let future = wasm_bindgen_future::JsFuture::from(promise);
// Actually execute the Promise/Future
let result: Result<JsValue, JsValue> = future.await;
// Interpret the result
if let Ok(content) = result {
// do sth with the content
}
}
I hope that this approach, specifically the transformation from JsValue to JsFuture, will help some of you, as the official docs don't cover that use case, but only the manual instantiation of a js_sys::Promise (e.g. Promise::resolve(&"foo".into()) ) or using web_sys::window::fetch().
My code looks like the following:
let fetches = futures::stream::iter(
hosts.into_iter().map(|url| {
async move {
match reqwest::get(&url).await {
// Ok and Err statements here!
}
But, the problem here is that it gives an error for URLs with invalid or self-signed SSL certificate. So, I tried to do the following:
let fetches = futures::stream::iter(
hosts.into_iter().map(|url| {
async move {
match reqwest::Client::builder().danger_accept_invalid_certs(true).build().unwrap().get(&url).await {
// Ok and Err statements here!
}
When I try to build it with Cargo, it says "error[E0277]: `RequestBuilder` is not a future".
So, how can I make my code accept invalid certificates?
Unlike the top-level get() function, which returns a Response, the Client::get() method which you call in the second snippet, returns a RequestBuilder, which you must send() to actually communicate.
Adding the missing send() allows the code to compile (playgropund):
fn main() {
let hosts: Vec<String> = vec![];
let fetches = futures::stream::iter(hosts.into_iter().map(|url| async move {
match reqwest::Client::builder()
.danger_accept_invalid_certs(true)
.build()
.unwrap()
.get(&url)
.send()
.await
{
Ok(x) => x,
Err(x) => panic!(),
}
}));
}
Most recent threads I have read are saying async is the better way to perform lots of I/O bound work such as sending HTTP requests and the like. I have tried to pick up async recently but am struggling with understanding how to send many groups of requests in parallel, for example:
let client = reqwest::Client::new();
let mut requests = 0;
let get = client.get("https://somesite.com").send().await?;
let response = get.text().await?;
if response.contains("some stuff") {
let get = client.get("https://somesite.com/something").send().await?;
let response = get.text().await?;
if response.contains("some new stuff") {
requests += 1;
println!("Got response {}", requests)
This does what I want, but how can I run it in parallel and control the amount of "worker threads" or whatever the equivalent is to a thread pool in async?
I understand it is similar to this question, but mine is strictly talking about the nightly Rust async/await syntax and a more specific use case where groups of requests/tasks need to be done. I also find using combinators for these situations a bit confusing, was hoping the newer style would help make it a bit more readable.
Not sure if this is the fastest way, as I am just experimenting myself, but here is my solution:
let client = reqwest::Client::new();
let links = vec![ // A vec of strings representing links
"example.net/a".to_owned(),
"example.net/b".to_owned(),
"example.net/c".to_owned(),
"example.net/d".to_owned(),
];
let ref_client = &client; // Need this to prevent client from being moved into the first map
futures::stream::iter(links)
.map(async move |link: String| {
let res = ref_client.get(&link).send().await;
// res.map(|res| res.text().await.unwrap().to_vec())
match res { // This is where I would usually use `map`, but not sure how to await for a future inside a result
Ok(res) => Ok(res.text().await.unwrap()),
Err(err) => Err(err),
}
})
.buffer_unordered(10) // Number of connection at the same time
.filter_map(|c| future::ready(c.ok())) // Throw errors out, do your own error handling here
.filter_map(|item| {
if item.contains("abc") {
future::ready(Some(item))
} else {
future::ready(None)
}
})
.map(async move |sec_link| {
let res = ref_client.get(&sec_link).send().await;
match res {
Ok(res) => Ok(res.text().await.unwrap()),
Err(err) => Err(err),
}
})
.buffer_unordered(10) // Number of connections for the secondary requests (so max 20 connections concurrently)
.filter_map(|c| future::ready(c.ok()))
.for_each(|item| {
println!("File received: {}", item);
future::ready(())
})
.await;
This requires the #![feature(async_closure)] feature.
This is my first attempt at writing a small webservice with rust, using actix-web.
The code below is a request handler that is intended to do three things, insert an entry in the database, send an email if that db call was successful, and then return a json payload as the response.
data.dal (database call) and data.email_service are references to Actors.
The issue: is I am unable to capture the error returned by data.dal. Any attempt to reconfigure the below code seems to give me an error stating the compiler wasn't able to find a conversion from Actix Mailbox to [Type].
Is there an alternate/better way to rewrite this? Basically when the request is issued, I'd like to be able to call Actor A. And if the result from A is Ok then call Actor B. If the results from both are okay return a JSON payload. If either A or B return an error (can have different error types), return an custom error message.
pub fn register_email(
invitation: Json<EmailInvitationInput>,
data: web::Data<AppState>,
) -> impl Future<Item=HttpResponse, Error=Error> {
let m = dal::queries::CreateEmailInvitation { email: invitation.email.clone() };
data.dal.send(m)
.from_err()
.and_then(move |res| {
let invite = res.unwrap();
let email_input = email::SendLoginLink {
from: "from_email".to_string(),
to: "to_email".to_string(),
};
data.email_service.send(email_input)
.from_err()
.and_then(move |res| match res {
Ok(_) => {
Ok(HttpResponse::Ok().json(EmailInvitationOutput { expires_at: invite.expires_at }))
}
Err(err) => {
debug!("{:#?}", err);
Ok(ServiceError::InternalServerError.error_response())
}
})
})
}
What I usually do is to have an Error type that agglomerates all different errors, the coercion to this type can be achieved implicitly by declaring the appropriate From implementations and what you are doing from_err() but here I am being explicit:
I haven't tested this code snippet but this is how I have done it in projects I'm working on that use Actix:
data.dal.send(m)
.map_err(Error::Mailbox)
.and_then(|res| res.map_err(Error::Service))
.and_then(move |invite| {
let email_input = email::SendLoginLink {
from: "from_email".to_string(),
to: "to_email".to_string(),
};
data.email_service.send(email_input)
.map_err(Error::Mailbox)
.and_then(|res| res.map_err(Error::Service))
.and_then(move |res| HttpResponse::Ok().json(EmailInvitationOutput { expires_at: invite.expires_at }))
})
.or_else(|err| {
debug!("{:#?}", err);
ServiceError::InternalServerError.error_response()
})
(I'm assuming ServiceError implements IntoFuture just like HttpResponse does)