How to use zbus to list block devices? - rust

I want to list all the available block devices in the system using a dbus implementation called zbus.
The UDisks2 documentation mentions a method call on org.freedesktop.UDisks2.Manager interface called GetBlockDevices which accepts IN a{sv} options, OUT ao block_objects for method parameters.
Using zbus, I write:
use std::error::Error;
use std::result::Result;
use zbus::{Connection, Proxy};
fn main() -> Result<(), Box<dyn Error>> {
let connection = Connection::new_system()?;
let p = Proxy::new(
&connection,
"org.freedesktop.UDisks2",
"/org/freedesktop/UDisks2/Manager",
"org.freedesktop.UDisks2.Manager",
)?;
let resp: Vec<zvariant::ObjectPath> = p.call("GetBlockDevices", &std::collections::HashMap::<String, zvariant::Value>::new())?;
dbg!(resp);
Ok(())
}
As far as I understand, zvariant Values represent a DBus Variant. However I get the following error:
error: implementation of `serde::de::Deserialize` is not general enough
--> src/main.rs:13:45
|
13 | let resp: Vec<zvariant::ObjectPath> = p.call("GetBlockDevices", &std::collections::HashMap::<String, zvariant::Value>::new())?;
| ^^^^ implementation of `serde::de::Deserialize` is not general enough
|
::: /home/adnan338/.cargo/registry/src/github.com-1ecc6299db9ec823/serde-1.0.115/src/de/mod.rs:531:1
|
531 | / pub trait Deserialize<'de>: Sized {
532 | | /// Deserialize this value from the given Serde deserializer.
533 | | ///
534 | | /// See the [Implementing `Deserialize`][impl-deserialize] section of the
... |
569 | | }
570 | | }
| |_- trait `serde::de::Deserialize` defined here
|
= note: `std::vec::Vec<zvariant::object_path::ObjectPath<'_>>` must implement `serde::de::Deserialize<'0>`, for any lifetime `'0`...
= note: ...but `std::vec::Vec<zvariant::object_path::ObjectPath<'_>>` actually implements `serde::de::Deserialize<'1>`, for some specific lifetime `'1`
What's causing this and how can I avoid this error?

Firstly, thanks for trying out our crate. The issue is that zbus::Proxy::call expects the return value to be an owned one, while you're deserializing to an unowned type. Both of the the following work:
// OwnedObjectPath require zvariant >= 2.2.0
let resp: Vec<zvariant::OwnedObjectPath> = p.call(
"GetBlockDevices",
&std::collections::HashMap::<String, zvariant::Value>::new(),
)?;
let resp = p.call_method(
"GetBlockDevices",
&std::collections::HashMap::<String, zvariant::Value>::new(),
)?;
let resp: Vec<zvariant::ObjectPath> = resp.body()?;

Related

Using axum and reqwest to upload a file to cloud bucket, streaming it

I've created a small Github repo that mimics exactly what I'm trying to achieve: streaming file uploads to both file systems and the cloud (S3) using axum (and reqwest).
I can change all the code you see. There is no restriction. All signatures were invented by me who are in my early days with the awesome Rust.
If you want I can also eliminate reqwest, as long as I understand what to use instead! :smile:
Can you help me understand how to fix the only error left?
error[E0521]: borrowed data escapes outside of associated function
--> src\adapter_s3.rs:56:19
|
41 | async fn put_file<'a>(
| -- lifetime `'a` defined here
42 | &'a self,
| -------- `self` is a reference that is only valid in the associated function body
...
56 | .body(reqwest::Body::wrap_stream(ReaderStream::new(reader)))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| `self` escapes the associated function body here
| argument requires that `'a` must outlive `'static`
This is the relevant code I think:
async fn put_file<'a>(
&'a self,
filename: &'a str,
reader: Pin<Box<dyn AsyncRead + Send + Sync + 'a>>,
) -> Result<()> {
let presigned_url_duration = Duration::from_secs(60 * 60);
let action = self.bucket.put_object(Some(&self.credentials), filename);
let client = reqwest::Client::new();
client
.post(action.sign(presigned_url_duration))
// .body(reader)
// .body(reqwest::Body::wrap_stream(reader))
.body(reqwest::Body::wrap_stream(ReaderStream::new(reader)))
.send()
.await?
.error_for_status()?;
Ok(())
}
UPDATE:
I tried with hyper too and it's the same error obviously:
hyper::Client::new()
.request(
hyper::Request::builder()
.uri(action.sign(presigned_url_duration).to_string())
.body(hyper::Body::wrap_stream(ReaderStream::new(reader)))
.unwrap(),
)
.await
.unwrap();
error[E0521]: borrowed data escapes outside of associated function
|
66 | async fn put_file<'a>(
| -- lifetime `'a` defined here
67 | &'a self,
| -------- `self` is a reference that is only valid in the associated function body
...
124 | .body(hyper::Body::wrap_stream(ReaderStream::new(reader)))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| `self` escapes the associated function body here
| argument requires that `'a` must outlive `'static`

Cannot determine return type for grpc stream

I am trying to implement a simple stream rpc using tonic and grpc. I have been following the routeguide tutorial.
When I try to return a tokio_stream::wrappers::ReceiverStream from my stream method I get compile error indicating I should return a Result wrapped by a ReceiverStream.
warning: unused manifest key: package.author
Compiling prng_generator v0.1.0 (/home/babbleshack/projects/prng_generator)
error[E0308]: mismatched types
--> src/bin/server.rs:51:46
|
51 | Ok(Response::new(ReceiverStream::new(rx)))
| ^^ expected enum `Result`, found struct `PrngResponse`
|
= note: expected struct `tokio::sync::mpsc::Receiver<Result<PrngResponse, Status>>`
found struct `tokio::sync::mpsc::Receiver<PrngResponse>`
note: return type inferred to be `tokio::sync::mpsc::Receiver<Result<PrngResponse, Status>>` here
--> src/bin/server.rs:41:67
|
41 | ) -> Result<Response<Self::PRNGServiceRequestStream>, Status> {
| ___________________________________________________________________^
42 | | let (mut tx, rx) = mpsc::channel(4);
43 | | let response_data = self.data.clone();
44 | |
... |
51 | | Ok(Response::new(ReceiverStream::new(rx)))
52 | | }
| |_____^
For more information about this error, try `rustc --explain E0308`.
error: could not compile `prng_generator` due to previous error
When I wrap my return channel in a Result I get a contradicting error message:
error[E0308]: mismatched types
--> src/bin/server.rs:51:46
|
51 | Ok(Response::new(ReceiverStream::new(Ok(rx))))
| ^^^^^^ expected struct `tokio::sync::mpsc::Receiver`, found enum `Result`
|
= note: expected struct `tokio::sync::mpsc::Receiver<Result<PrngResponse, Status>>`
found enum `Result<tokio::sync::mpsc::Receiver<PrngResponse>, _>`
note: return type inferred to be `tokio::sync::mpsc::Receiver<Result<PrngResponse, Status>>` here
--> src/bin/server.rs:41:67
|
41 | ) -> Result<Response<Self::PRNGServiceRequestStream>, Status> {
| ___________________________________________________________________^
42 | | let (mut tx, rx) = mpsc::channel(4);
43 | | let response_data = self.data.clone();
44 | |
... |
51 | | Ok(Response::new(ReceiverStream::new(Ok(rx))))
52 | | }
| |_____^
Here is my proto:
use std::sync::Arc;
use tokio::sync::mpsc;
use tonic::{Request, Response, Status};
use tokio_stream::wrappers::ReceiverStream;
pub mod types {
tonic::include_proto!("types");
}
use types::prng_service_server::PrngService;
use types::{PrngRequest, PrngResponse};
And the implementing rust code:
use std::sync::Arc;
use tokio::sync::mpsc;
use tonic::{Request, Response, Status};
use tokio_stream::wrappers::ReceiverStream;
pub mod types {
tonic::include_proto!("types");
}
use types::prng_service_server::PrngService;
use types::{PrngRequest, PrngResponse};
#[derive(Debug, Default)]
pub struct PRNGServiceImpl{
data: Arc<Vec<PrngResponse>>,
}
#[tonic::async_trait]
impl PrngService for PRNGServiceImpl {
type PRNGServiceRequestStream = ReceiverStream<Result<PrngResponse, Status>>;
async fn prng_service_request(
&self,
request: Request<PrngRequest>,
) -> Result<Response<Self::PRNGServiceRequestStream>, Status> {
let (mut tx, rx) = mpsc::channel(256);
let response_data = self.data.clone();
tokio::spawn(async move {
for response in &response_data[..] {
Ok(tx.send(response.clone()).await.unwrap());
}
println!(" /// done sending");
});
Ok(Response::new(ReceiverStream::new(rx)))
//Ok(Response::new(ReceverStream::new(Ok(rx))))
}
}
How do I determine what my return type should be here?
The error message indicates that your return type declares a stream that produces Result<PrngResponse, Status> values, but the stream you have given it produces PrngResponse values. Your attempt to fix the solution wraps the receiver channel in a Result, which is not the same thing.
To fix this, you need to change the type that rx is inferred to be. It's inferred to be a stream of PrngResponse because of the tx.send() call, which sends a PrngResponse, so you can fix this by sending a Result instead:
tx.send(Ok(response.clone()))
The compiler points at the returned value and not the tx.send() line because the problem is the mismatch between the declared return type of the function and what it actually returns. The compiler could probably figure out that this is due to the tx.send() invocation, but in many cases type inference uses information spread out over multiple lines and there may not be one single line responsible for the inferred type of the returned value.
One way you could trace the problem to its source is to provide an explicit type somewhere matching the return type. For example:
let (mut tx, rx) = mpsc::channel::<Result<PrngResponse, Status>>(256);
This change would resolve the return type issue, and the compiler would have then pointed at the tx.send() line indicating that the sent value is not a Result.

In Rust, I have a large number of receiver objects I'd like manage, however I'm running into lifetime issues using Select

Due to the possibility of there being a large number of objects, I'd like to have a way to add them to the select list, remove them for processing and then add them back. All, without having to rebuild the select list each time an object is added back for waiting. It looks something like this:
use std::collections::HashMap;
use crossbeam::{Select, Sender, Receiver};
struct WaitList <'a> {
sel: Select<'a>,
objects: HashMap<u128, Object>,
sel_index: HashMap<usize, u128>,
}
impl<'a> WaitList<'a> {
fn new () -> Self { Self { sel: Select::new(), objects: HashMap::new(), sel_index: HashMap::new() } }
fn select(&self) -> &Object {
let oper = self.sel.select();
let index = oper.index();
let id = self.sel_index.get(&index).unwrap();
let obj = self.objects.get(&id).unwrap();
obj.cmd = oper.recv(&obj.receiver).unwrap();
self.sel.remove(index);
obj
}
fn add_object(&self, object: Object) {
let id = object.id;
self.objects.insert(id, object);
self.add_select(id);
}
fn add_select(&self, id: u128) {
let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
self.sel_index.insert(idx, id);
}
}
Over time the select list will contain more dead entries, then live, and I'll rebuild it at that time. But, I'd like to not have to rebuild it every time. Here's the detailed error message:
Checking test-select v0.1.0 (/Users/bruce/Projects/rust/examples/test-select)
error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
--> src/main.rs:28:47
|
28 | let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
| ^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 27:5...
--> src/main.rs:27:5
|
27 | / fn add_select(&self, id: u128) {
28 | | let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
29 | | self.sel_index.insert(idx, id);
30 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/main.rs:28:34
|
28 | let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
| ^^^^^^^^^^^^
note: but, the lifetime must be valid for the lifetime `'a` as defined on the impl at 9:6...
--> src/main.rs:9:6
|
9 | impl<'a> WaitList<'a> {
| ^^
note: ...so that the types are compatible
--> src/main.rs:28:28
|
28 | let idx = self.sel.recv(&self.objects.get(&id).unwrap().receiver);
| ^^^^
= note: expected `&mut crossbeam::Select<'_>`
found `&mut crossbeam::Select<'a>`
While I believe I understand the issue, that the borrow of the receiver from the hash table doesn't live long enough, I'm having a difficult time trying to come up with an alternative -- and I'm not seeing a clean way to borrow the information. I considered creating a struct to contain the borrow, and using that instead of a plain id in the wait sel_index however that runs into the same lifetime problem.
struct SingleWaiter<'a> {
id: u128,
receiver: &'a Receiver::<Command>
}
I feel like I'm missing something or not understanding something, as it seems like it shouldn't be that difficult to do what I want to do. I can imagine that the choice of HashMap for holding object might be the issue, a Vec felt wrong, as I'm adding and inserting. BTW, the HashMap isn't normally part of the waitlist. It is part of something else, but the problem remains the same irregardless of where the HashMap lives.

How to create a thread manager?

I have a data stream that I want to process in the background, but I want to create a struct or some functions to manage this stream.
In C++ land, I would create a class that abstracts all of this away. It would have a start method which would initialize the data stream and start a thread for processing. It would have a stop method that stops the processing and joins the thread.
However, this isn't really Rusty, and it doesn't even work in Rust.
Example (Playground)
use std::thread;
use std::time::Duration;
struct Handler {
worker_handle: Option<thread::JoinHandle<()>>,
stop_flag: bool, // Should be atomic, but lazy for example
}
impl Handler {
pub fn new() -> Handler {
let worker_handle = None;
let stop_flag = true;
return Handler { worker_handle, stop_flag };
}
pub fn start(&mut self) {
self.stop_flag = false;
self.worker_handle = Some(std::thread::spawn(move || {
println!("Spawned");
self.worker_fn();
}));
}
pub fn stop(&mut self) {
let handle = match self.worker_handle {
None => return,
Some(x) => x,
};
self.stop_flag = true;
handle.join();
}
fn worker_fn(&mut self) {
while !self.stop_flag {
println!("Working!");
}
}
}
fn main() {
let mut handler = Handler::new();
handler.start();
thread::sleep(Duration::from_millis(10000));
handler.stop();
}
Output:
error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
--> src/main.rs:20:54
|
20 | self.worker_handle = Some(std::thread::spawn(move || {
| ______________________________________________________^
21 | | println!("Spawned");
22 | | self.worker_fn();
23 | | }));
| |_________^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 18:5...
--> src/main.rs:18:5
|
18 | / pub fn start(&mut self) {
19 | | self.stop_flag = false;
20 | | self.worker_handle = Some(std::thread::spawn(move || {
21 | | println!("Spawned");
22 | | self.worker_fn();
23 | | }));
24 | | }
| |_____^
= note: ...so that the types are compatible:
expected &mut Handler
found &mut Handler
= note: but, the lifetime must be valid for the static lifetime...
note: ...so that the type `[closure#src/main.rs:20:54: 23:10 self:&mut Handler]` will meet its required lifetime bounds
--> src/main.rs:20:35
|
20 | self.worker_handle = Some(std::thread::spawn(move || {
| ^^^^^^^^^^^^^^^^^^
error: aborting due to previous error
Even if I cheat and remove the call the worker_fn, I still can't really work with JoinHandles like I might expect from C++ land.
error[E0507]: cannot move out of `self.worker_handle.0` which is behind a mutable reference
--> src/main.rs:27:28
|
27 | let handle = match self.worker_handle {
| ^^^^^^^^^^^^^^^^^^ help: consider borrowing here: `&self.worker_handle`
28 | None => return,
29 | Some(x) => x,
| -
| |
| data moved here
| move occurs because `x` has type `std::thread::JoinHandle<()>`, which does not implement the `Copy` trait
error: aborting due to previous error
So it's clear that I'm going outside of the usual Rust model and probably shouldn't be using this strategy.
But I still want to build some kind of interface that lets me simply spin up the data stream without worrying about managing threads, and I'm not really sure the best way to do this.
So it seems I have two core problems.
1) How can I create a function to be run in a thread which accepts data from an outside source, and can be signaled to quit safely? If I had an atomic bool for killing it, how would I share that between the threads?
2) How do I handle joining the thread when I'm done? The stop method needs to clean up the thread, but I don't know how to track a reference to it.

How do I pass a struct with type parameters as a function argument?

How do I pass an instance of EcsClient with the signature impl<P, D> EcsClient<P, D> where P: ProvideAwsCredentials, D: DispatchSignedRequest to a function as a reference in Rust? My attempt is thus:
extern crate rusoto;
use std::default::Default;
use rusoto::{ DefaultCredentialsProvider, Region };
use rusoto::ecs::{ EcsClient };
use rusoto::default_tls_client;
fn get_task_definition_revisions(client: &EcsClient) {
// Use EscClient instance here
}
fn main() {
let provider = DefaultCredentialsProvider::new().unwrap();
let client = EcsClient::new(default_tls_client().unwrap(), provider, Region::EuWest1).unwrap();
get_task_definition_revisions(&client);
}
This gives me the following error:
error[E0243]: wrong number of type arguments: expected 2, found 0
--> src/main.rs:9:43
|
9 | fn get_task_definition_revisions(client: &EcsClient) {
| ^^^^^^^^^ expected 2 type arguments
My attempted fix for this is such:
extern crate rusoto;
use std::default::Default;
use rusoto::{
DefaultCredentialsProvider,
Region,
ProvideAwsCredentials,
DispatchSignedRequest
};
use rusoto::ecs::{ EcsClient, ListTaskDefinitionsRequest };
use rusoto::default_tls_client;
fn get_task_definition_revisions(client: &EcsClient<ProvideAwsCredentials, DispatchSignedRequest>) {
// Use EcsClient instance here
}
fn main() {
let provider = DefaultCredentialsProvider::new().unwrap();
let client = EcsClient::new(default_tls_client().unwrap(), provider, Region::EuWest1);
get_task_definition_revisions(&client);
}
Which gives me:
error[E0277]: the trait bound `rusoto::ProvideAwsCredentials + 'static: std::marker::Sized` is not satisfied
--> src/main.rs:14:1
|
14 | fn get_task_definition_revisions(client: &EcsClient<P, D>) {
| _^ starting here...
15 | | let defs = client.list_task_definitions(&ListTaskDefinitionsRequest {
16 | | family_prefix: None,
17 | | max_results: None,
18 | | next_token: None,
19 | | sort: None,
20 | | status: Some("ACTIVE".to_string()),
21 | | });
22 | | }
| |_^ ...ending here: the trait `std::marker::Sized` is not implemented for `rusoto::ProvideAwsCredentials + 'static`
|
= note: `rusoto::ProvideAwsCredentials + 'static` does not have a constant size known at compile-time
= note: required by `rusoto::ecs::EcsClient`
error[E0277]: the trait bound `rusoto::DispatchSignedRequest + 'static: std::marker::Sized` is not satisfied
--> src/main.rs:14:1
|
14 | fn get_task_definition_revisions(client: &EcsClient<P, D>) {
| _^ starting here...
15 | | let defs = client.list_task_definitions(&ListTaskDefinitionsRequest {
16 | | family_prefix: None,
17 | | max_results: None,
18 | | next_token: None,
19 | | sort: None,
20 | | status: Some("ACTIVE".to_string()),
21 | | });
22 | | }
| |_^ ...ending here: the trait `std::marker::Sized` is not implemented for `rusoto::DispatchSignedRequest + 'static`
|
= note: `rusoto::DispatchSignedRequest + 'static` does not have a constant size known at compile-time
= note: required by `rusoto::ecs::EcsClient`
This feels like a rabbit hole I shouldn't be going down.
I've also tried changing the function signature to accept generics, however the EcsClient is a struct not a trait. Googling doesn't provide much help because I don't know the correct terms to search for.
This question seems to imply that I should be able to declare a function like fn my_func(client: &EcsClient) { ... } and it will work, so why doesn't the above example?
The problem is that EcsClient is not a type, it's a blueprint to build a type (also known as "type constructor").
As a result, you cannot use just EcsClient when a type is required, be it in functions or for struct members; instead, each time, you must use it to build a type by specifying its generic parameters.
Thus, the first step is to introduce the type parameters:
fn get_task_definition_revisions<P, D>(client: &EcsClient<P, D>) {}
Now, however, the compiler will complain that the P and D are insufficiently constrained: EcsClient only accept a very specific kind of P and D!
The next step, thus, is to look-up the bounds that are specified for P and D in the definition of EcsClient and apply them. It's just copy/paste at this point:
fn get_task_definition_revisions<P, D>(client: &EcsClient<P, D>)
where P: ProvideAwsCredentials,
D: DispatchSignedRequest
{
}
And then you're golden.
If you need more capabilities of P or D for this specific function, feel free to constrain them adequately by adding more bounds using +:
fn get_task_definition_revisions<P, D>(client: &EcsClient<P, D>)
where P: ProvideAwsCredentials + 'static,
D: DispatchSignedRequest
{
}
If you wonder why Rust chose to have you repeat the bounds for P and D when it could perfectly infer them, it's because it cares about you. More specifically, it cares about you 6 months from now, and the next maintainer to come. So, taking the stance that you write once and read many, it forces you to copy the bounds so that later you don't have to wonder what they are, and drill down recursively in each type/function used to painfully aggregate all the pieces. In Rust, the next time you read the function, you'll have all the information right there.
I was understanding the syntax incorrectly. It seems that P and D are placeholders, like T would be. I need to specify what those types are, so the signature now looks like this:
fn get_task_definition_revisions<P: ProvideAwsCredentials, D: DispatchSignedRequest>(client: &EcsClient<P, D>) {
...
}
I don't use P or D in the function body, but they must be declared.
The full example now looks like:
extern crate rusoto;
use std::default::Default;
use rusoto::{
DefaultCredentialsProvider,
Region,
ProvideAwsCredentials,
DispatchSignedRequest
};
use rusoto::ecs::{ StringList, EcsClient, ListTaskDefinitionsRequest };
use rusoto::default_tls_client;
fn get_task_definition_revisions<P: ProvideAwsCredentials, D: DispatchSignedRequest>(client: &EcsClient<P, D>) {
let res = client.list_task_definitions(&ListTaskDefinitionsRequest {
family_prefix: None,
max_results: None,
next_token: None,
sort: None,
status: Some("ACTIVE".to_string()),
});
// ...
}
fn main() {
let provider = DefaultCredentialsProvider::new().unwrap();
let client = EcsClient::new(default_tls_client().unwrap(), provider, Region::EuWest1);
get_task_definition_revisions(&client);
}
I'm still not entirely sure why this works or is required but I hope this answer helps someone else.

Resources