I have a hyper server set up more or less exactly as the third example here: https://docs.rs/hyper/0.14.16/hyper/server/index.html . My version of the handle function calls some other async functions, and everything works fine, until I try to encode some URL query params into a string in one of those async functions. My project stops compiling when I include the four lines involving the Serializer in one of the functions called by handle:
async fn broken_func(&self, param: &str) -> Result<String, Infallible> {
// ...
let mut s = url::form_urlencoded::Serializer::new(String::new());
// the presence or absence of these two lines has no effect on the bug, but
// they demonstrate how I'm trying to use the Serializer
s.append_pair("key", "value");
println!("{:?}", s.finish());
drop(s); // <-- thought this might help, but it doesn't
// ...
Ok(query_string)
}
The error I get is
generator cannot be sent between threads safely
the trait `std::marker::Sync` is not implemented for `dyn for<'r> std::ops::Fn(&'r str) -> std::borrow::Cow<'_, [u8]>`
I have no idea what this has to do with form_urlencoded::Serializer. However, I am aware that Serializer is both !Send and !Sync, but in this case I'm only using it within a single function so I don't think that should make a difference? If I remove those four lines above, it goes back to compiling.
So instead, to serialize some key/value pairs into URL query parameters, I have to use the following, which kind of seems ridiculous -- not just because this is needlessly complex for something so simple, but also because url::Url::parse_with_params uses form_urlencoded::Serializer under the hood.
let query_string = url::Url::parse_with_params(
"http://example.com",
&[("key", "value")]
)
.unwrap()
.query()
.map(|s| s.to_owned())
.unwrap();
Any idea why trying to explicitly use Serializer inside an async function causes things to break?
Caesar hit the nail on the head. The trick of putting s in a scope fixed it.
Related
I have a function like this
fn get_html(address: &str) -> String {
let mut response = reqwest::blocking::get(
address,
);
response = response.unwrap_or_else(|_e| {String::from("")});
response = response.text().unwrap_or_else(|_e| {String::from("")});
return response
}
Where I'm checking for html content. I would like to return an empty String if any kind of an error occurs somewhere in this function.
I'm not sure how to deal with this because unwrap_or_else expecting Result not String.
The reqwest::blocking::get() function is returning a Result<Response>.
To obtain the html, you have to unwrap this result and call .text() method.
That will return a Result<String>, that you have to unwrap again.
In your code you assign a String::from("") when you unwrap the Result<Response>, and that is not right, because you have a Response when it is Ok and a String when it is an Err.
Instead you should match the result and return the String from the function:
fn get_html(address: &str) -> String {
let mut response = reqwest::blocking::get(
address,
);
match response {
Ok(response) => response.text().unwrap_or_else(|_e| String::from("")),
Err(_) => String::from(""),
}
}
In this code, you use unwrap_or_else() just on the .text() result.
While if you have an error on the response itself, you return a String from the function.
An idiomatic way to solve your issue would be to refactor slightly your code: because code in get_html could fail, it's natural that its signature reflects this, so instead of returning a String, it could return an Option<String>, leaving the caller decide on what to do in this case:
fn get_html(address: &str) -> Option<String> {
reqwest::blocking::get(address)
.ok()?
.text()
.ok()
}
See the playground.
This makes the code much more straightforward. However, you may really want the signature of the function to be -> String, not -> Option<String>. In this case, there are two solutions.
The first solution would be to use the experimental try block. I mention this solution, not because it's currently the most adequate, but because it will be one day (most likely).
fn get_html(address: &str) -> String {
let result: Option<_> = try {
reqwest::blocking::get(address)
.ok()?
.text()
.ok()?
};
result.unwrap_or_default()
}
See the playground.
Note that, as is, Rust is not able to figure out types on its own for the try block, so we have to help it, which makes this more verbose. This aspect will probably improve over time, when try blocks are stabilized.
Also note that, since the Default::default constructor of String produces an empty string, we can directly use .unwrap_or_default() instead of .unwrap_or(String::new()) or .unwrap_or_else(|| String::new()) (since an empty string is not allocated, the first option is also acceptable).
The second solution would simply be to add an other function with the wanted signature, and make it use the first function
fn get_html_failable(address: &str) -> Option<String> {
reqwest::blocking::get(address)
.ok()?
.text()
.ok()
}
fn get_html(address: &str) -> String {
get_html_failable(address).unwrap_or_default()
}
This may seem unconvenient, but lots of libraries in Rust have internal error propagation using types that represent failure (such as Result or Option), because it's really convenient to write such functions and to compose them (both reqwest::blocking::get and .text() return a Result, and see how easy it is to use them in a function that returns an Option), and have wrapper function that expose the wanted signature (most libraries will still expose fail types, but if you are writing an application it's not necessarily the best choice).
A variable of both preceding workarounds is to "simulate" a try block in stable Rust, without relying on an additional toplevel function.
fn get_html(address: &str) -> String {
(|| reqwest::blocking::get(address).ok()?.text().ok())()
.unwrap_or_default()
}
See the playground.
Note that, in this case, countrary to the try version, Rust is able to figure out the right types on its own.
This is probably the least readable solution, so I wouldn't recommend it (use try blocks instead). But it works.
I've created rust bindings to a C library and currently writing safe wrappers around it.
The question is about C functions which take in C function pointers which are not able to take in any custom user data.
It is easier to explain it with an example,
C Library:
// The function pointer I need to pass,
typedef void (*t_function_pointer_library_wants)(const char *argument);
// The function which I need to pass the function pointer to,
void register_hook(const t_function_pointer_library_wants hook);
Bindings:
// For the function pointer type
pub type t_function_pointer_library_wants = ::std::option::Option<unsafe extern "C" fn(argument: *const ::std::os::raw::c_char)>;
// For the function which accepts the pointer
extern "C" {
pub fn register_hook(hook: t_function_pointer_library_wants);
}
It would have been very nice if I could expose an api to the user like the following,
// Let's assume my safe wrapper is named on_something
// ..
on_something(|argument|{
// Do something with the argument..
});
// ..
although according to the sources below, the lack of ability to hand over the management of the part of memory which would store my closure's state to C, prevents me to create this sort of API. Because the function pointer in C is stateless and does not take in any user data of some sort. (Please correct me if I'm wrong.)
I've come to this conclusion by reading these sources and similar ones:
Trampoline Technique
Similar Trampoline Technique
Hacky Thread Local Technique
Sources in Shepmaster's answer
As a fallback, I maybe can imagine an API like this where I pass a function pointer instead.
fn handler(argument: &str) {
// Do something with the argument..
}
//..
on_something(handler);
//..
But I am a little confused about converting an fn(&str),
to an unsafe extern "C" fn(argument: *const std::os::raw::c_char)..
I'd be very glad if you could point me to the right direction.
* The actual library in focus is libpd and there is an issue I've created related to this.
Thanks a lot.
First off, this is a pretty hard problem to solve. Obviously, you need some way to pass data into a function outside of its arguments. However, pretty any method of doing that via a static could easily result in race conditions or worse, depending on what the c library does and how the library is used. The other option is to JIT some glue code that calls your closure. At first glance, that seems even worse, but libffi abstracts most of that away. A wrapper using the libffi crate would like this:
use std::ffi::CStr;
use libffi::high::Closure1;
fn on_something<F: Fn(&str) + Send + Sync + 'static>(f: F) {
let closure: &'static _ = Box::leak(Box::new(move |string: *const c_char| {
let string = unsafe { CStr::from_ptr(string).to_str().unwrap() };
f(string);
}));
let callback = Closure1::new(closure);
let &code = callback.code_ptr();
let ptr:unsafe extern "C" fn (*const c_char) = unsafe { std::mem::transmute(code) };
std::mem::forget(callback);
unsafe { register_handler(Some(ptr)) };
}
I don't have a playground link, but it worked fine when I tested it locally. There are two important things to note with this code:
It's maximally pessimistic about what the c code does, assuming the function is repeatedly called from multiple threads for the entire duration of the program. You may be able to get away with fewer restrictions, depending on what libpd does.
It leaks memory to ensure the callback is valid for the life of the program. This is probably fine since callbacks are typically only set once. There is no way to safely recover this memory without keeping around a pointer to the callback that was registered.
It's also worth noting that the libffi::high::ClosureMutN structs are unsound, as they permit aliasing mutable references to the passed wrapped closure. There is a PR to fix that waiting to be merged though.
I am using Reqwest to call some APIs. I want to have a general function to parse the response as below:
async fn response_to_result(response: &Response) -> anyhow::Result<()> {
let status = response.status().as_u16();
let response_message = response.text().await?; // Move here because of response.text()
return if status == 200 {
Ok(())
} else {
Err(anyhow::anyhow!(response_message))
};
}
This is the error I got:
move occurs because `*response` has type `reqwest::Response`, which does not implement the `Copy` trait
The move occurs by calling response.text() (method definition: pub async fn text(self) -> crate::Result<String>). Normally with params we can pass by reference, but with the method I don't have any idea. Does anyone have a solution for this?
Normally with params, we can pass with reference
That's true as long as the method doesn't consume the value. text() does, since it assembles all the chunks together for you. Here's its signature:
pub async fn text(self) -> Result<String>
Note the self rather than &self. After running text(), the whole stream is consumed, a String is constructed, and it's returned to you to manage. Response does not have an internal buffer that stores all this data. It's gone once the stream is consumed; keeping track of it is the job of the caller. Future calls to text() (or bytes()) couldn't work. So text() destroys the entire Response. You can't continue to use it after calling text().
So you'll need to pass the actual value, not a borrow of it. Remove the & on Response.
If you want a (mutable) borrowing version of this, you'll need to assemble the chunks yourself with chunk(). But you probably don't want to do this. You probably just want to pass Response (without the &) and let response_to_result() consume the value like text().
The comments on closure.rs are pretty great, however I can't make it work for returning a closure from a WebAssembly library.
I have a function like this:
#[wasm_bindgen]
pub fn start_game(
start_time: f64,
screen_width: f32,
screen_height: f32,
on_render: &js_sys::Function,
on_collision: &js_sys::Function,
) -> ClosureTypeHere {
// ...
}
Inside that function I make a closure, assuming Closure::wrap is one piece of the puzzle, and copying from closure.rs):
let cb = Closure::wrap(Box::new(move |time| time * 4.2) as Box<FnMut(f64) -> f64>);
How do I return this callback from start_game and what should ClosureTypeHere be?
The idea is that start_game will create local mutable objects - like a camera, and the JavaScript side should be able to call the function Rust returns in order to update that camera.
This is a good question, and one that has some nuance too! It's worth calling out the closures example in the wasm-bindgen guide (and the section about passing closures to JavaScript) as well, and it'd be good to contribute back to that as well if necessary!
To get you started, though, you can do something like this:
use wasm_bindgen::{Closure, JsValue};
#[wasm_bindgen]
pub fn start_game(
start_time: f64,
screen_width: f32,
screen_height: f32,
on_render: &js_sys::Function,
on_collision: &js_sys::Function,
) -> JsValue {
let cb = Closure::wrap(Box::new(move |time| {
time * 4.2
}) as Box<FnMut(f64) -> f64>);
// Extract the `JsValue` from this `Closure`, the handle
// on a JS function representing the closure
let ret = cb.as_ref().clone();
// Once `cb` is dropped it'll "neuter" the closure and
// cause invocations to throw a JS exception. Memory
// management here will come later, so just leak it
// for now.
cb.forget();
return ret;
}
Above the return value is just a plain-old JS object (here as a JsValue) and we create that with the Closure type you've seen already. This will allow you to quickly return a closure to JS and you'll be able to call it from JS as well.
You've also asked about storing mutable objects and such, and that can all be done through normal Rust closures, capturing, etc. For example the declaration of FnMut(f64) -> f64 above is the signature of the JS function, and that can be any set of types such as FnMut(String, MyCustomWasmBindgenType, f64) ->
Vec<u8> if you really want. For capturing local objects you can do:
let mut camera = Camera::new();
let mut state = State::new();
let cb = Closure::wrap(Box::new(move |arg1, arg2| { // note the `move`
if arg1 {
camera.update(&arg2);
} else {
state.update(&arg2);
}
}) as Box<_>);
(or something like that)
Here the camera and state variables will be owned by the closure and dropped at the same time. More info about just closures can be found in the Rust book.
It's also worth briefly covering the memory management aspect here. In the
example above we're calling forget() which leaks memory and can be a problem if the Rust function is called many times (as it would leak a lot of memory). The fundamental problem here is that there's memory allocated on the WASM heap which the created JS function object references. This allocated memory in theory needs to be deallocated whenever the JS function object is GC'd, but we have no way of knowing when that happens (until WeakRef exists!).
In the meantime we've chosen an alternate strategy. The associated memory is
deallocated whenever the Closure type itself is dropped, providing
deterministic destruction. This, however, makes it difficult to work with as we need to figure out manually when to drop the Closure. If forget doesn't work for your use case, some ideas for dropping the Closure are:
First, if it's a JS closure only invoked once, then you can use Rc/RefCell
to drop the Closure inside the the closure itself (using some interior
mutability shenanigans). We should also eventually
provide native support
for FnOnce in wasm-bindgen as well!
Next, you can return an auxiliary JS object to Rust which has a manual free
method. For example a #[wasm_bindgen]-annotated wrapper. This wrapper would
then need to be manually freed in JS when appropriate.
If you can get by, forget is by far the easiest thing to do for
now, but this is definitely a pain point! We can't wait for WeakRef to exist :)
As far as I understand from documentation, it isn't supposed to export Rust closures, they only might be passed over as parameters to imported JS functions, but all this happens in Rust code.
https://rustwasm.github.io/wasm-bindgen/reference/passing-rust-closures-to-js.html#passing-rust-closures-to-imported-javascript-functions
I made couple of experiments, and when a Rust function returns the mentioned 'Closure' type, compiler throws exception: the trait wasm_bindgen::convert::IntoWasmAbi is not implemented for wasm_bindgen::prelude::Closure<(dyn std::ops::FnMut() -> u32 + 'static)>
In all examples, closures are wrapped into an arbitrary sctuct, but after that you already can't call this on JS side.
I've got these structs:
#[derive(Debug, RustcDecodable)]
struct Config {
ssl: Option<SslConfig>,
}
#[derive(Debug, RustcDecodable)]
struct SslConfig {
key: Option<String>,
cert: Option<String>,
}
They get filled from a toml file. This works perfectly fine. Since I got an Option<T> in it I either have to call unwrap() or do a match.
But if I want to do the following:
let cfg: Config = read_config(); // Reads a File, parses it and returns the Config-Struct
let keypath = cfg.ssl.unwrap().key.unwrap();
let certpath = cfg.ssl.unwrap().cert.unwrap();
It won't work because cfg.ssl gets moved to keypath. But why does it get moved? I call unwrap() on ssl to get the key (and unwrap() it to). So the result of key.unwrap() should get moved?
Or am I missing a point? Whats the best way to make these structs accessible like this (or in a other neat way)? I tried to implement #[derive(Debug, RustcDecodable, Copy, Clone)] but this won't work because I have to implement Copy to String as well. Then I have to implement Copy to Vec<u8> and so on. There must be a more convenient solution?
What is the definition of Option::unwrap? From the documentation:
fn unwrap(self) -> T
it consumes its input (cfg.ssl here).
This is not what you want, you instead want to go from Option<T> to &T, which will start by consuming &self (by reference, not value)... or you want to clone the Option before calling unwrap.
Cloning is rarely the solution... the alternative here is as_ref:
fn as_ref(&self) -> Option<&T>
And therefore you can write:
let keypath /*: &String*/ = cfg.ssl.as_ref().unwrap().key.as_ref().unwrap();
^~~~~~~ ^~~~~~~~
So the result of key.unwrap() should get moved?
Yes, but not only that. The key insight here is, that a variable get's moved into of unwrap(). Let's look at the function signature:
fn unwrap(self) -> T { ... }
It takes self, so the object is moved into the function. But this applies to ssl.unwrap(), too!
So when writing:
cfg.ssl.unwrap().key.unwrap();
You first move cfg.ssl into unwrap(), then you access one field of the result and move that field into unwrap() again. So yes, cfg.ssl is moved. In order to solve this, you can save the temporary result of the first unwrap() call, like so:
let ssl = cfg.ssl.unwrap();
let keypath = ssl.key.unwrap();
let certpath = ssl.cert.unwrap();
Or you can look at the as_ref() method, if you don't want to move (which is probably the case).