How should I use dependencies of dependencies? - rust

I'm working on project using warp. Warp includes the bytes cargo as a dependency and I want to use the Bytes struct that gets passed into the callback. Here's an example:
use warp::Filter;
fn process(data: bytes::Bytes) -> impl warp::reply::Reply {
println!("{:?}", data);
"processed!"
}
#[tokio::main]
async fn main() {
let process = warp::path("process")
.and(warp::post())
.and(warp::body::bytes())
.map(process);
warp::serve(process)
.run(([127, 0, 0, 1], 3030))
.await;
}
Here's the dependencies in my Cargo.toml:
[dependencies]
bytes = "*"
tokio = { version = "0.2", features = ["full"] }
warp = "0.2"
This setup gives me this error:
error[E0631]: type mismatch in function arguments
--> src/main.rs:13:14
|
3 | fn process(data: bytes::Bytes) -> impl warp::reply::Reply {
| --------------------------------------------------------- found signature of `fn(bytes::bytes::Bytes) -> _`
...
13 | .map(process);
| ^^^^^^^ expected signature of `fn(bytes::bytes::Bytes) -> _`
|
= note: required because of the requirements on the impl of `warp::generic::Func<(bytes::bytes::Bytes,)>` for `fn(bytes::bytes::Bytes) -> impl warp::reply::Reply {process}`
Not a big fan of this error message because it's telling me that it's looking for the signature fn(bytes::bytes::Bytes) -> _ and it found fn(bytes::bytes::Bytes) -> _ which pretty much look exactly the same to me, but after some searching I realized it's because there's two different versions of Bytes getting pulled in, so I adjust my Cargo.toml like this:
bytes = "0.5.6"
So I get the same version of bytes that warp is pulling in.
Is this the conventional way to deal with this issue?

Related

Cannot trigger Outcome::Failure in FromRequest implementation

While trying to get started with development of an api using rocket, I was implementing a request guard which is supposed to check authorization headers. When the check fails it should result in a Failure, but that is where I cannot get it to work. Outcome::Success works perfectly fine and returns the correct object, but when triggering an Outcome::Failure I always run into the issue that I cannot get it to compile:
error[E0282]: type annotations needed
--> src/main.rs:43:21
|
43 | Outcome::Failure((Status::BadRequest, RequestError::ParseError));
| ^^^^^^^^^^^^^^^^ cannot infer type for type parameter S declared on the enum Outcome
To Reproduce
main.rs
#[macro_use] extern crate rocket;
use rocket::Request;
use rocket::request::{FromRequest, Outcome};
use rocket::http::Status;
use regex::Regex;
#[get("/")]
fn index() -> &'static str {
"Hello, world!"
}
#[get("/test")]
fn test(device: Device) -> &'static str {
"Hello test"
}
#[derive(Debug)]
enum RequestError {
InvalidCredentials,
ParseError,
}
struct Device {
id: i32
}
#[rocket::async_trait]
impl<'r> FromRequest<'r> for Device {
type Error = RequestError;
async fn from_request(request: &'r Request<'_>) -> Outcome<Self, Self::Error> {
Outcome::Failure((Status::BadRequest, RequestError::ParseError));
// TEST
let d1 = Device {
id: 123
};
Outcome::Success(d1)
}
}
#[launch]
fn rocket() -> _ {
rocket::build().mount("/", routes![index,test])
}
Cargo.toml
[package]
name = "api-sandbox"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
rocket = "0.5.0-rc.1"
regex = "1"
Expected Behavior
The parameter S should not needed to be declared, as I am not using the parameter Success(S), but use Failure(E). According to the docs I can either return an error or an tuple with Status and Error, but instead the error message pops up. I have double checked with only available resources and blogs, but was not able to correctly trigger an outcome with status failure.
Environment:
VERSION="20.04.3 LTS (Focal Fossa)"
5.10.60.1-microsoft-standard-WSL2
Rocket 0.5.0-rc.1
I am quite new to this topic, so please let me know, if I need to supply more information. Thanks for your assistance in this topic
The type of Outcome::Failure(_) cannot be deduced. Even if not used in this particular construction, the type parameter S must be known for it to be a complete type. There is no default type available or any context that can help infer the type of S.
The same is true of Outcome::Success(_). By itself, the type of the template parameter E is not known. However, this compiles because it does have context that can help the compiler infer it. It is used as a return value, so it must match the return type, and therefore it can be deduced that E is Self::Error.
This will also work for Outcome::Failure(_) if it were used as a return value:
async fn from_request(request: &'r Request<'_>) -> Outcome<Self, Self::Error> {
Outcome::Failure((Status::BadRequest, RequestError::ParseError))
}

MQTT connection with tokio

I'm trying to create an MQTT connection with tokio and the codec provided in the mqtt_v5 crate. My code does not compile, and I don't understand why. Here is what I have written so far, the send code might not be correct.
use tokio::net::TcpStream;
use tokio_util::codec::Framed;
use tokio_util::codec::Decoder;
use std::net::SocketAddrV4;
use mqtt_v5::types::Packet as MqttPacket;
use mqtt_v5::codec::MqttCodec;
use futures_sink::Sink;
use futures_core::stream::Stream;
struct MqttConn {
inner: Framed<TcpStream, MqttCodec>,
}
impl MqttConn {
async fn new(addr: SocketAddrV4) -> MqttConn {
let tcp = TcpStream::connect(addr).await.expect("cannot connect to mqtt");
MqttConn { inner: MqttCodec::new().framed(tcp) }
}
async fn handle(&self, handler: &dyn Fn(&MqttConn, MqttPacket) -> ()) {
while let Some(p) = self.inner.next().await {
handler(self, p)
}
}
async fn send(&self, p: MqttPacket) {
self.inner.start_send(p);
}
}
I get these errors from the compiler:
error[E0599]: no method named `framed` found for struct `MqttCodec` in the current scope
--> src/mqtt.rs:17:44
|
17 | MqttConn { inner: MqttCodec::new().framed(tcp) }
| ^^^^^^ method not found in `MqttCodec`
|
= help: items from traits can only be used if the trait is in scope
= note: the following trait is implemented but not in scope; perhaps add a `use` for it:
`use tokio_util::codec::decoder::Decoder;`
error[E0599]: no method named `next` found for struct `Framed<tokio::net::TcpStream, MqttCodec>` in the current scope
--> src/mqtt.rs:21:40
|
21 | while let Some(p) = self.inner.next().await {
| ^^^^ method not found in `Framed<tokio::net::TcpStream, MqttCodec>`
error[E0599]: no method named `start_send` found for struct `Framed<tokio::net::TcpStream, MqttCodec>` in the current scope
--> src/mqtt.rs:27:20
|
27 | self.inner.start_send(p);
| ^^^^^^^^^^ method not found in `Framed<tokio::net::TcpStream, MqttCodec>`
warning: unused import: `tokio_util::codec::Decoder`
--> src/mqtt.rs:3:5
|
3 | use tokio_util::codec::Decoder;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
warning: unused import: `futures_sink::Sink`
--> src/mqtt.rs:7:5
|
7 | use futures_sink::Sink;
| ^^^^^^^^^^^^^^^^^^
The compiler says that the Decoder trait is not in scope, but I use it. If I try the suggested import I find that the module tokio_util::codec::decoder is private. Based on the source tokio_util::codec::decoder::Decoder is reexported as tokio_util::codec::Decoder. Also if I read the docs correctly, Framed should implement Sink and Stream therefore it should have the next and start_send methods.
Relevant lines from Cargo.toml:
[dependencies]
tokio = { version = "1.0.1", features = ["full"] }
tokio-util = { version = "0.6", features = ["full"] }
futures-sink = "0.3.9"
futures-core = "0.3.9"
mqtt-v5 = "0.1.1"
How can I get this to compile?
You have a number of library incompatibilities that are causing some less-than-obvious error messages.
mqtt-v5 depends on tokio-util^0.3, which was written for tokio 0.2, not 1.0. You'll need to roll back to tokio 0.2 and tokio-util 0.3. This should fix the problem with Decoder and Sink.
Additionally, the Stream trait only provides poll_next(), which is the task-level stream method. next() is provided by the StreamExt trait, along with other convenience methods similar to those found in Iterator.

Handlebars helpers within Rocket

I am trying to register a helper for handlebars the ultimate goal is to be able to have a helper that calculates/prints the length of a Vec. But I already fail to get the samples from the documentation into my program. I managed to do a 'minimal' example that shows the same error as my rocket page.
#[macro_use]
extern crate handlebars;
extern crate rocket;
extern crate rocket_contrib;
use handlebars::{Context, Handlebars, Helper, HelperResult, JsonRender, Output, RenderContext};
use rocket_contrib::templates::Template;
handlebars_helper!(hex: |v: i64| format!("0x{:x}", v));
fn wow_helper(
h: &Helper,
_: &Handlebars,
_: &Context,
_: &mut RenderContext,
out: &mut Output,
) -> HelperResult {
if let Some(param) = h.param(0) {
out.write("<b><i>")?;
out.write(&param.value().render())?;
out.write("</b></i>")?;
}
Ok(())
}
fn main() {
rocket::ignite()
.attach(Template::custom(|engines| {
engines
.handlebars
.register_helper("mlenb", Box::new(wow_helper));
}))
.launch();
println!("Hello, world!");
}
with the dependencies in cargo.toml:
[dependencies]
handlebars="3.3.0"
rocket="0.4.5"
[dependencies.rocket_contrib]
version = "0.4.5"
default-features = false
features = ["diesel_sqlite_pool", "handlebars_templates", "serve"]
The error:
error[E0631]: type mismatch in function arguments
--> src/main.rs:184:43
|
162 | / fn wow_helper(
163 | | h: &handlebars::Helper,
164 | | _: &handlebars::Handlebars,
165 | | _: &handlebars::Context,
... |
175 | | Ok(())
176 | | }
| |_- found signature of `for<'r, 's, 't0, 't1, 't2, 't3, 't4, 't5, 't6, 't7> fn(&'r handlebars::Helper<'s, 't0>, &'t1 handlebars::Handlebars<'t2>, &'t3 handlebars::Context, &'t4 mut handlebars::RenderContext<'t5, 't6>, &'t7 mut (dyn handlebars::Output + 't7)) -> _`
...
184 | .register_helper("mlenb", Box::new(wow_helper));
| ^^^^^^^^^^^^^^^^^^^^ expected signature of `for<'r, 'reg, 'rc, 's, 't0> fn(&'r rocket_contrib::templates::handlebars::Helper<'reg, 'rc>, &'reg rocket_contrib::templates::handlebars::Handlebars, &'rc rocket_contrib::templates::handlebars::Context, &'s mut rocket_contrib::templates::handlebars::RenderContext<'reg>, &'t0 mut (dyn rocket_contrib::templates::handlebars::Output + 't0)) -> _`
|
= note: required because of the requirements on the impl of `rocket_contrib::templates::handlebars::HelperDef` for `for<'r, 's, 't0, 't1, 't2, 't3, 't4, 't5, 't6, 't7> fn(&'r handlebars::Helper<'s, 't0>, &'t1 handlebars::Handlebars<'t2>, &'t3 handlebars::Context, &'t4 mut handlebars::RenderContext<'t5, 't6>, &'t7 mut (dyn handlebars::Output + 't7)) -> std::result::Result<(), handlebars::RenderError> {wow_helper}`
= note: required for the cast to the object type `dyn rocket_contrib::templates::handlebars::HelperDef`
Don't need to use crate handlebars in cargo.toml with rocket crate.
its works well like that:
in cargo.toml only
[dependencies]
rocket = "0.4.5"
[dependencies.rocket_contrib]
version = "0.4.5"
default-features = false
features = ["handlebars_templates", "serves"]
and in the main.rs use:
use rocket_contrib :: templates :: {Template, handlebars};
use handlebars :: {Helper, Handlebars, Context, RenderContext, Output, HelperResult, RenderError};
So there will be no problems with versioning the handlebars!
Thanks to #jebrosen:mozilla.org in the rocket matrix channel the problem got resolved.
Problem:
I used incompatible versions of handlebars. The rocket library uses version 1.0 while I used Version 3.0 in the Cargo.toml.
Solution:
Change the Cargo.toml to:
[dependencies]
handlebars="1.0"
rocket="0.4.5"
[dependencies.rocket_contrib]
version = "0.4.5"
default-features = false
features = ["diesel_sqlite_pool", "handlebars_templates", "serve"]
Careful a rebuild requires a cargo clean otherwise the changes don't get built.
Solution for the initial Problem:
I did not manage to use the macro handlebars_helper! to create the length helper but instead created it using the function syntax.
fn mlen_helper_fun(
h: &handlebars::Helper<'_, '_>,
_: &handlebars::Handlebars,
_: &handlebars::Context,
_: &mut handlebars::RenderContext<'_>,
out: &mut dyn handlebars::Output,
) -> handlebars::HelperResult {
if let Some(param) = h.param(0) {
if let serde_json::value::Value::Array(arr) = param.value() {
out.write(&format!("{}", arr.len()))?;
Ok(())
} else {
Err(handlebars::RenderError::new("Not a array"))
}
} else {
Err(handlebars::RenderError::new(
"No parameter given at least one required",
))
}
}
This can then be used with:
.attach(Template::custom(|engines| {
engines
.handlebars
.register_helper("length", Box::new(mlen_helper_fun));
}))

Rust lifetimes, data flows into other references

I wrote the following code that filters a stream of data which worked fine until I changed from parsing simple numbers to also have types that are bound to lifetimes like &str and &[u8].
use wirefilter::{ExecutionContext, Filter, Scheme};
lazy_static::lazy_static! {
static ref SCHEME: Scheme = Scheme! {
port: Int,
name: Bytes,
};
}
#[derive(Debug)]
struct MyStruct {
port: i32,
name: String,
}
impl MyStruct {
fn scheme() -> &'static Scheme {
&SCHEME
}
fn filter_matches<'s>(&self, filter: &Filter<'s>) -> bool {
let mut ctx = ExecutionContext::new(Self::scheme());
ctx.set_field_value("port", self.port).unwrap();
ctx.set_field_value("name", self.name.as_str()).unwrap();
filter.execute(&ctx).unwrap()
}
}
fn main() -> Result<(), failure::Error> {
let data = expensive_data_iterator();
let scheme = MyStruct::scheme();
let filter = scheme
.parse("port in {2 5} && name matches \"http.*\"")?
.compile();
for my_struct in data
.filter(|my_struct| my_struct.filter_matches(&filter))
.take(2)
{
println!("{:?}", my_struct);
}
Ok(())
}
fn expensive_data_iterator() -> impl Iterator<Item = MyStruct> {
(0..).map(|port| MyStruct {
port,
name: format!("http {}", port % 2),
})
}
If I try to compile it the compiler will fail with this:
error[E0623]: lifetime mismatch
--> src/main.rs:26:16
|
21 | fn filter_matches<'s>(&self, filter: &Filter<'s>) -> bool {
| ----- ----------
| |
| these two types are declared with different lifetimes...
...
26 | filter.execute(&ctx).unwrap()
| ^^^^^^^ ...but data from `self` flows into `filter` here
error: aborting due to previous error
error: Could not compile `wirefilter_playground`.
To learn more, run the command again with --verbose.
Process finished with exit code 101
my first thought was that self and filter should have the same lifetime in fn filter_matches<'s>(&self, filter: &Filter<'s>) -> bool but if I change the signature to fn filter_matches<'s>(&'s self, filter: &Filter<'s>) -> bool I will start getting this error:
error: borrowed data cannot be stored outside of its closure
--> src/main.rs:38:29
|
33 | let filter = scheme
| ------ ...so that variable is valid at time of its declaration
...
38 | .filter(|my_struct| my_struct.filter_matches(&filter))
| ----------- ^^^^^^^^^ -------------- cannot infer an appropriate lifetime...
| | |
| | cannot be stored outside of its closure
| borrowed data cannot outlive this closure
error: aborting due to previous error
error: Could not compile `wirefilter_playground`.
To learn more, run the command again with --verbose.
Process finished with exit code 101
I am failing to understand the reason, Filter<'s> is bound to SCHEME which is lazily generated and is bound to 'static which makes sense not allowing filter.execute to take reference to &self.name.as_str() because it would be outlived but, isn't filter.execute(&ctx) which the signature is pub fn execute(&self, ctx: &ExecutionContext<'s>) -> Result<bool, SchemeMismatchError> supposed to drop the references as soon as it finishes as the result of it has not other lifetimes?
In order to try and compile the code above, you can use this Cargo.toml:
[package]
name = "wirefilter_playground"
version = "0.1.0"
edition = "2018"
[dependencies]
wirefilter-engine = "0.6.1"
failure = "0.1.5"
lazy_static = "1.3.0"
PS: That could be solved by compiling the as inside filter_matches method but that would be sort of bad because the user would only get the parse error when trying to filter and it could potentially be slower.
I see 2 ways to solve this problem:
1) extend lifetime of self.name. This can be achieved by collecting expensive_data_iterator into, say, Vec.
--- let data = expensive_data_iterator();
+++ let data: Vec<_> = expensive_data_iterator().collect();
2) reduce lifetime of filter.
--- let filter = scheme.parse("...")?.compile();
+++ let filter = scheme.parse("...")?;
--- .filter(|my_struct| my_struct.filter_matches(&filter))
+++ .filter(|my_struct| my_struct.filter_matches(&filter.clone().compile()))
I omitted some other minor changes. And yes, filter_matches<'s>(&'s self, ...) is mandatory in either case.
PS yes, 2nd option works because my_struct outlives filter. Well, if both approaches are somewhat bad, then you can combine them! Process data by chunks, collecting each one into vector.
const N: usize = 10; // or any other size
loop {
let cur_chunk: Vec<_> = data.by_ref().take(N).collect();
if cur_chunk.is_empty() {
break;
}
let cur_filter = filter.clone().compile();
// etc
}
it uses only O(N) memory and compiles filter N times less

How to check whether a path exists?

The choice seems to be between std::fs::PathExt and std::fs::metadata, but the latter is suggested for the time being as it is more stable. Below is the code I have been working with as it is based off the docs:
use std::fs;
pub fn path_exists(path: &str) -> bool {
let metadata = try!(fs::metadata(path));
assert!(metadata.is_file());
}
However, for some odd reason let metadata = try!(fs::metadata(path)) still requires the function to return a Result<T,E> even though I simply want to return a boolean as shown from assert!(metadata.is_file()).
Even though there will probably be a lot of changes to this soon enough, how would I bypass the try!() issue?
Below is the relevant compiler error:
error[E0308]: mismatched types
--> src/main.rs:4:20
|
4 | let metadata = try!(fs::metadata(path));
| ^^^^^^^^^^^^^^^^^^^^^^^^ expected bool, found enum `std::result::Result`
|
= note: expected type `bool`
found type `std::result::Result<_, _>`
= note: this error originates in a macro outside of the current crate
error[E0308]: mismatched types
--> src/main.rs:3:40
|
3 | pub fn path_exists(path: &str) -> bool {
| ________________________________________^
4 | | let metadata = try!(fs::metadata(path));
5 | | assert!(metadata.is_file());
6 | | }
| |_^ expected (), found bool
|
= note: expected type `()`
found type `bool`
Note that many times you want to do something with the file, like read it. In those cases, it makes more sense to just try to open it and deal with the Result. This eliminates a race condition between "check to see if file exists" and "open file if it exists". If all you really care about is if it exists...
Rust 1.5+
Path::exists... exists:
use std::path::Path;
fn main() {
println!("{}", Path::new("/etc/hosts").exists());
}
As mental points out, Path::exists simply calls fs::metadata for you:
pub fn exists(&self) -> bool {
fs::metadata(self).is_ok()
}
Rust 1.0+
You can check if the fs::metadata method succeeds:
use std::fs;
pub fn path_exists(path: &str) -> bool {
fs::metadata(path).is_ok()
}
fn main() {
println!("{}", path_exists("/etc/hosts"));
}
You can use std::path::Path::is_file:
use std::path::Path;
fn main() {
let b = Path::new("file.txt").is_file();
println!("{}", b);
}
https://doc.rust-lang.org/std/path/struct.Path.html#method.is_file

Resources