Creating a cyclic Tokio stream connected to a shared state - rust

I am running into a problem that I do not really understand and hoped
that somebody might be able to see what I have misunderstood.
The problem is quite straightforward: I have a global state (shared
between several tasks) and want to have an infinite cycle over a
vector in the global state. I will then zip that with an interval
stream and hence get a regular emission of the next value in the
stream.
If the vector in the state changes, the inifinite stream should just
reload the vector and start reading from the new one instead, and
discard the old array.
Here is the code that I've gotten this far, and the questions are at
the end of the post.
use futures::stream::Stream;
use futures::{Async, Poll};
use std::iter::Cycle;
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
use tokio::timer::Interval;
We define a global state that hold an array that can be
updated. Whenever the array is updated, we will step the version and
set the array.
struct State<T> {
version: u32,
array: Vec<T>,
}
impl<T> State<T> {
fn new(array: Vec<T>) -> Self {
Self {
version: 0,
array: Vec::new(),
}
}
fn update(&mut self, array: Vec<T>) {
self.version += 1;
self.array = array;
}
}
Now, we create an stream over the state. When initialized, it will
read the array and version from the state and store it and then keep
an instance of std::iter::Cycle internally that will cycle over the
array.
struct StateStream<I> {
state: Arc<Mutex<State<I::Item>>>,
version: u32,
iter: Cycle<I>,
}
impl<I> StateStream<I>
where
I: Iterator,
{
fn new(state: Arc<Mutex<State<I::Item>>>) -> Self {
let (version, array) = {
let locked_state = state.lock().unwrap();
(locked_state.version, locked_state.array)
};
Self {
state: state,
version: version,
iter: array.iter().cycle(),
}
}
}
We now implement the stream for the StateStream. With each poll, it
will check if the version of the state changed, and if it did, reload
the array and version.
We will then take the next item from the iterator and return that.
impl<I> Stream for StateStream<I>
where
I: Iterator + Clone,
{
type Item = I::Item;
type Error = ();
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
let locked_state = self.state.lock().unwrap();
if locked_state.version > self.version {
self.iter = locked_state.array.clone().iter().cycle();
self.version = locked_state.version;
}
Ok(Async::Ready(self.iter.next()))
}
}
The main program looks like this. I do not update the vector here, but
that is not important for the case at hand.
fn main() {
let state = Arc::new(Mutex::new(State::new(vec![2, 3, 5, 7, 11, 13])));
let primes = StateStream::new(state)
.take(20)
.zip(
Interval::new(Instant::now(), Duration::from_millis(500))
.map_err(|err| println!("Error: {}", err)),
)
.for_each(|(number, instant)| {
println!("fire; number={}, instant={:?}", number, instant);
Ok(())
});
tokio::run(primes);
}
When compiling this, I get the following errors:
cargo run --example cycle_stream_shared
Compiling tokio-testing v0.1.0 (/home/mats/crates/tokio-examples)
error[E0308]: mismatched types
--> examples/cycle_stream_shared.rs:66:19
|
66 | iter: array.iter().cycle(),
| ^^^^^^^^^^^^^^^^^^^^ expected type parameter, found struct `std::slice::Iter`
|
= note: expected type `std::iter::Cycle<I>`
found type `std::iter::Cycle<std::slice::Iter<'_, <I as std::iter::Iterator>::Item>>`
error[E0308]: mismatched types
--> examples/cycle_stream_shared.rs:81:25
|
81 | self.iter = locked_state.array.clone().iter().cycle();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected type parameter, found struct `std::slice::Iter`
|
= note: expected type `std::iter::Cycle<I>`
found type `std::iter::Cycle<std::slice::Iter<'_, <I as std::iter::Iterator>::Item>>`
error: aborting due to 2 previous errors
For more information about this error, try `rustc --explain E0308`.
error: Could not compile `tokio-testing`.
To learn more, run the command again with --verbose.
Now, the error and the explanation says that the concrete type is not
possible to derive, but in this case, I am using the generic struct
Cycle<I> and expect I to be instantiated to std::slice::Iter<'_,
I::Item>. Since std::slice::Iter has implemented Iterator and, the type have implemented all necessary traits to match.
Some answers to similar questions exist, but nothing that seems to
match this case:
“Expected type parameter” error in the constructor of a generic
struct is showing that the types do not match (same
as the explanation gives) because the generic struct definition allow any type, but the construction require a specific type.
In this case, we are using a generic type Cycle<I>, where I should implement the Iterator trait, and try to use a type std::slice::Iter<..> that does implement Iterator.
How do I return an instance of a trait from a
method? talk about how to return an arbitrary type
matching a trait, which is not the case here.
The other questions are mostly referring to these two, or variations
of these.
Update: Changed it to be a generic type to demonstrate that it still does not work.

Related

Copying a value in a pattern match without owning it

I am going through the too-many-linked-lists tutorial, looking to implement a simple linked list:
use std::mem;
struct Node{
elem: i32,
next : Link,
}
enum Link {
Empty,
More(Box<Node>),
}
pub struct List{
head: Link,
}
impl List{
pub fn pop_node(&mut self) -> Link{
let node = mem::replace(&mut self.head, Link::Empty);
match node {
Link::More(nd) => {self.head= nd.next;}
_ => ()
};
node
}
}
The pop_node function is to return the node at the head of the list. However it
does not seem to compile complaining that I moved the variable node while accessing nd. Is there a way I can pass the bound variable in a pattern match without owning it?
This is the error I see:
error[E0382]: use of partially moved value: `node`
--> src/first.rs:34:9
|
31 | Link::More(nd) => {self.head= nd.next;}
| -- value partially moved here
...
34 | node
| ^^^^ value used here after partial move
|
= note: partial move occurs because value has type `Box<Node>`, which does not implement the `Copy` trait
help: borrow this field in the pattern to avoid moving `node.0`
|
31 | Link::More(ref nd) => {self.head= nd.next;}
| +++
For more information about this error, try `rustc --explain E0382`.
Any idea what I should be doing here? (I tried things like unpacking the node struct but they didn't seem to work.)
Usually when you're performing this kind of operation, you want the actual element (the i32 here), so maybe return Option<i32> instead -- None would indicate that the list was empty. Doing this is much simpler than what you're trying to do. Within the match you can just return nd.elem.
impl List{
pub fn pop_node(&mut self) -> Option<i32> {
let node = mem::replace(&mut self.head, Link::Empty);
match node {
Link::More(nd) => {
self.head = nd.next;
Some(nd.elem)
}
Link::Empty => None
}
}
}
Note this operation should be called shift_node (pop_node would be expected to remove the last node, not the first).
I'd also consider replacing your Link type with Option<Box<Node>>. Then you can use utilities already present on Option. For example, your mem::replace() call could be replaced with self.head.take() and then you're just mapping the result. You can keep the Link name by making it an alias (type Link = Option<Box<Node>>;).
use std::mem;
struct Node{
elem: i32,
next: Link,
}
type Link = Option<Box<Node>>;
pub struct List {
head: Link,
}
impl List{
pub fn pop_node(&mut self) -> Option<i32> {
self.head.take().map(|nd| {
self.head = nd.next;
nd.elem
})
}
}

"my first rust": the method `unwrap_or_default` exists for enum `Option<&mut ...>`, but its trait bounds were not satisfied

My very first rust program/function is supposed to walk a dir tree and deliver a HashMap with K(mime_as_string) and V(count_sum, size_sum). And I want it in FP style.
So what I have is this:
fn files_info4(rootpath: &str) {
struct FTypeStats {
count: u64,
size: u64,
}
impl Default for FTypeStats {
fn default() -> Self {
FTypeStats {
count: 0,
size: 0,
}
}
}
// get file type stats directly/functionally/lazy
let fts = WalkDir::new(rootpath)
.min_depth(1)
.max_depth(99)
.into_iter()
.filter_map(|entry| entry.ok())
.map(|entry| (entry, entry.metadata().map_or(false, |m| m.is_file())))
.filter(|(e, f)| *f)
.fold(HashMap::new(), | mut acc: HashMap<String, FTypeStats>, (e, _) | {
let ftype = tree_magic::from_filepath(e.path());
acc.get_mut(&ftype).unwrap_or_default().count += 1;
acc.get_mut(&ftype).unwrap_or_default().size += e.metadata().map_or(0, |m| m.len());
acc
});
}
where I did the impl Default after meeting the error:
error[E0599]: the method `unwrap_or_default` exists for enum `Option<&mut FTypeStats>`, but its trait bounds were not satisfied
--> src/main.rs:54:29
|
54 | acc.get_mut(&ftype).unwrap_or_default().count += 1;
| ^^^^^^^^^^^^^^^^^ method cannot be called on `Option<&mut FTypeStats>` due to unsatisfied trait bounds
|
= note: the following trait bounds were not satisfied:
`&mut FTypeStats: Default`
But it still didn't help, i.e. the same error remains.
Questions:
how to make it compile/run
can the evaluation chain be optimised? (And is the code within fold actually correct regarding e.g. the assignments I need for FTypeStats and initialization for the default/initial values? (It feels a bit "implicit"))
TIA!
There are multiple problems here: First, calling .get_mut() on the HashMap gives you an Option which is None in case the key is not in the map. The Option itself is detached from the map (aside from lifetime constraints), so in the None-case, there is no link back into the map that would put a constructed-from-default value in it's proper place.
Second, as the error message says, the method unwrap_or_default() requires the inner type of Option to actually implement Default, in this case &mut FTypeStats. Not only does such an implementation not exist in your code, but it would also be very hard to do so because generating a (valid) mutable reference to some type out of thin air - which is the whole point of Default - is practically impossible.
The solution is to use the entry() API on the HashMap instead of get_mut():
*acc.entry(&ftype).or_default().count += 1;
The call to entry() gives you a proxy-value to wherever &ftype is - or would be - in the HashMap. The call to or_default() always gives you a &mut FTypeStats. If &ftype is not in the map, it uses Default on FTypeStats (not Default on &mut FTypeStats) to generate a new entry, inserts it, and gives you a mutable reference to that.

Why does calling a method on a dereferenced trait object or slice compile?

Given the following code:
trait Function {
fn filter (&self);
}
#[derive(Debug, Copy, Clone)]
struct Kidney {}
impl Function for Kidney {
fn filter (&self) {
println!("filtered");
}
}
fn main() {
let k = Kidney {};
let f: &Function = &k;
//let k1 = (*f); //--> This gives a "size not satisfied" error
(*f).filter(); //--> Works; what exactly happens here?
}
I am not sure why it compiles. I was expecting the last statement to fail. I guess I have overlooked some fundamentals while learning Rust, as I am failing to understand why dereferencing a trait (that lives behind a pointer) should compile.
Is this issue similar to the following case?
let v = vec![1, 2, 3, 4];
//let s: &[i32] = *v;
println!("{}", (*v)[0]);
*v gives a slice, but a slice is unsized, so again it is not clear to me how this compiles. If I uncomment the second statement I get
| let s:&[i32]= *v;
| ^^
| |
| expected &[i32], found slice
| help: consider borrowing here: `&*v`
|
= note: expected type `&[i32]`
found type `[{integer}]`
Does expected type &[i32] mean "expected a reference of slice"?
Dereferencing a trait object is no problem. In fact, it must be dereferenced at some point, otherwise it would be quite useless.
let k1 = (*f); fails not because of dereferencing but because you try to put the raw trait object on the stack (this is where local variables live). Values on the stack must have a size known at compile time, which is not the case for trait objects because any type could implement the trait.
Here is an example where a structs with different sizes implement the trait:
trait Function {
fn filter (&self);
}
#[derive(Debug, Copy, Clone)]
struct Kidney {}
impl Function for Kidney {
fn filter (&self) {
println!("filtered");
}
}
#[derive(Debug, Copy, Clone)]
struct Liver {
size: f32
}
impl Function for Liver {
fn filter (&self) {
println!("filtered too!");
}
}
fn main() {
let k = Kidney {};
let l = Liver {size: 1.0};
let f: &Function;
if true {
f = &k;
} else {
f = &l;
}
// Now what is the size of *f - Kidney (0 bytes) or Liver (4 bytes)?
}
(*f).filter(); works because the temporarily dereferenced object is not put on the stack. In fact, this is the same as f.filter(). Rust automatically applies as many dereferences as required to get to an actual object. This is documented in the book.
What happens in the second case is that Vec implements Deref to slices, so it gets all methods implemented for slices for free. *v gives you a dereferenced slice, which you assign to a slice. This is an obvious type error.
Judging by the MIR produced by the first piece of code, (*f).filter() is equivalent to f.filter(); it appears that the compiler is aware that since filter is a method on &self, dereferencing it doesn't serve any purpose and is omitted altogether.
The second case, however, is different, because dereferencing the slice introduces bounds-checking code. In my opinion the compiler should also be able to tell that this operation (dereferencing) doesn't introduce any meaningful changes (and/or that there won't be an out-of-bounds error) and treat it as regular slice indexing, but there might be some reason behind this.

Vector of traits (dynamic dispatch) which contains associated type (also dynamic dispatch) [duplicate]

I have a program that involves examining a complex data structure to see if it has any defects. (It's quite complicated, so I'm posting example code.) All of the checks are unrelated to each other, and will all have their own modules and tests.
More importantly, each check has its own error type that contains different information about how the check failed for each number. I'm doing it this way instead of just returning an error string so I can test the errors (it's why Error relies on PartialEq).
My Code So Far
I have traits for Check and Error:
trait Check {
type Error;
fn check_number(&self, number: i32) -> Option<Self::Error>;
}
trait Error: std::fmt::Debug + PartialEq {
fn description(&self) -> String;
}
And two example checks, with their error structs. In this example, I want to show errors if a number is negative or even:
#[derive(PartialEq, Debug)]
struct EvenError {
number: i32,
}
struct EvenCheck;
impl Check for EvenCheck {
type Error = EvenError;
fn check_number(&self, number: i32) -> Option<EvenError> {
if number < 0 {
Some(EvenError { number: number })
} else {
None
}
}
}
impl Error for EvenError {
fn description(&self) -> String {
format!("{} is even", self.number)
}
}
#[derive(PartialEq, Debug)]
struct NegativeError {
number: i32,
}
struct NegativeCheck;
impl Check for NegativeCheck {
type Error = NegativeError;
fn check_number(&self, number: i32) -> Option<NegativeError> {
if number < 0 {
Some(NegativeError { number: number })
} else {
None
}
}
}
impl Error for NegativeError {
fn description(&self) -> String {
format!("{} is negative", self.number)
}
}
I know that in this example, the two structs look identical, but in my code, there are many different structs, so I can't merge them. Lastly, an example main function, to illustrate the kind of thing I want to do:
fn main() {
let numbers = vec![1, -4, 64, -25];
let checks = vec![
Box::new(EvenCheck) as Box<Check<Error = Error>>,
Box::new(NegativeCheck) as Box<Check<Error = Error>>,
]; // What should I put for this Vec's type?
for number in numbers {
for check in checks {
if let Some(error) = check.check_number(number) {
println!("{:?} - {}", error, error.description())
}
}
}
}
You can see the code in the Rust playground.
Solutions I've Tried
The closest thing I've come to a solution is to remove the associated types and have the checks return Option<Box<Error>>. However, I get this error instead:
error[E0038]: the trait `Error` cannot be made into an object
--> src/main.rs:4:55
|
4 | fn check_number(&self, number: i32) -> Option<Box<Error>>;
| ^^^^^ the trait `Error` cannot be made into an object
|
= note: the trait cannot use `Self` as a type parameter in the supertraits or where-clauses
because of the PartialEq in the Error trait. Rust has been great to me thus far, and I really hope I'm able to bend the type system into supporting something like this!
When you write an impl Check and specialize your type Error with a concrete type, you are ending up with different types.
In other words, Check<Error = NegativeError> and Check<Error = EvenError> are statically different types. Although you might expect Check<Error> to describe both, note that in Rust NegativeError and EvenError are not sub-types of Error. They are guaranteed to implement all methods defined by the Error trait, but then calls to those methods will be statically dispatched to physically different functions that the compiler creates (each will have a version for NegativeError, one for EvenError).
Therefore, you can't put them in the same Vec, even boxed (as you discovered). It's not so much a matter of knowing how much space to allocate, it's that Vec requires its types to be homogeneous (you can't have a vec![1u8, 'a'] either, although a char is representable as a u8 in memory).
Rust's way to "erase" some of the type information and gain the dynamic dispatch part of subtyping is, as you discovered, trait objects.
If you want to give another try to the trait object approach, you might find it more appealing with a few tweaks...
You might find it much easier if you used the Error trait in std::error instead of your own version of it.
You may need to impl Display to create a description with a dynamically built String, like so:
impl fmt::Display for EvenError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{} is even", self.number)
}
}
impl Error for EvenError {
fn description(&self) -> &str { "even error" }
}
Now you can drop the associated type and have Check return a trait object:
trait Check {
fn check_number(&self, number: i32) -> Option<Box<Error>>;
}
your Vec now has an expressible type:
let mut checks: Vec<Box<Check>> = vec![
Box::new(EvenCheck) ,
Box::new(NegativeCheck) ,
];
The best part of using std::error::Error...
is that now you don't need to use PartialEq to understand what error was thrown. Error has various types of downcasts and type checks if you do need to retrieve the concrete Error type out of your trait object.
for number in numbers {
for check in &mut checks {
if let Some(error) = check.check_number(number) {
println!("{}", error);
if let Some(s_err)= error.downcast_ref::<EvenError>() {
println!("custom logic for EvenErr: {} - {}", s_err.number, s_err)
}
}
}
}
full example on the playground
I eventually found a way to do it that I'm happy with. Instead of having a vector of Box<Check<???>> objects, have a vector of closures that all have the same type, abstracting away the very functions that get called:
fn main() {
type Probe = Box<Fn(i32) -> Option<Box<Error>>>;
let numbers: Vec<i32> = vec![ 1, -4, 64, -25 ];
let checks = vec![
Box::new(|num| EvenCheck.check_number(num).map(|u| Box::new(u) as Box<Error>)) as Probe,
Box::new(|num| NegativeCheck.check_number(num).map(|u| Box::new(u) as Box<Error>)) as Probe,
];
for number in numbers {
for check in checks.iter() {
if let Some(error) = check(number) {
println!("{}", error.description());
}
}
}
}
Not only does this allow for a vector of Box<Error> objects to be returned, it allows the Check objects to provide their own Error associated type which doesn't need to implement PartialEq. The multiple ases look a little messy, but on the whole it's not that bad.
I'd suggest you some refactoring.
First, I'm pretty sure, that vectors should be homogeneous in Rust, so there is no way to supply elements of different types for them. Also you cannot downcast traits to reduce them to a common base trait (as I remember, there was a question about it on SO).
So I'd use algebraic type with explicit match for this task, like this:
enum Checker {
Even(EvenCheck),
Negative(NegativeCheck),
}
let checks = vec![
Checker::Even(EvenCheck),
Checker::Negative(NegativeCheck),
];
As for error handling, consider use FromError framework, so you will able to involve try! macro in your code and to convert error types from one to another.

Can't figure out return type of this Rust function that returns Iter?

I have this small Rust function:
pub fn factor(input_array: &mut [i32]) {
let x = input_array
.iter()
.filter(|&x| x % 2 == 0);
x
}
When I run this via cargo run I get this error:
Compiling gettingrusty v0.0.1 (file:///home/lowks/src/rust/gettingrusty)
src/functional.rs:22:9: 22:10 error: mismatched types:
expected `()`,
found `core::iter::Filter<core::slice::Iter<'_, i32>, [closure#src/functional.rs:21:21: 21:36]>`
(expected (),
found struct `core::iter::Filter`) [E0308]
src/functional.rs:22 x
^
src/functional.rs:22:9: 22:10 help: run `rustc --explain E0308` to see a detailed explanation
error: aborting due to previous error
Could not compile `gettingrusty`.
I tried a few return types such as slice::Iter<i32> and core::slice::Iter<i32> but seems like all of them are wrong. What should be the return type of my function?
Under normal circumstances, you could just copy+paste the found part of the error message. There are two problems with that in this particular case.
First, core isn't directly accessible. Various items are exposed by the standard library libstd, but are actually defined by libcore. Basically, the standard library is the public interface you are meant to use to access these items, but the compiler doesn't know that. Typically, you work around this by just replacing core::* with std::*.
The second problem is that the type includes a closure, and closures cannot be named. The simplest solution is to just not use a closure at all; you aren't capturing anything, anyway.
Doing that and just fixing the compile errors as they come up leads to:
pub fn factor(input_array: &mut [i32])
-> std::iter::Filter<std::slice::Iter<i32>, fn(&&i32) -> bool> {
fn even(x: &&i32) -> bool { **x % 2 == 0 }
let x = input_array
.iter()
.filter(even as for<'r> fn(&'r &_) -> _);
x
}
Your function returns a Filter object, so its actual return type is Filter<_, _> for some generic arguments. That’s fine, but chances are, you’ll want to hide all the implementation details from the type signature and just say that your function returns some iterator. Unfortunately, there is no (as of today) easy way to do this.
The pattern that seems to be rather common is to use a newtype wrapper. The problem with this is that writing the wrapper is a little bit more difficult than one might expect, e.g. one will have to deal with lifetimes explicitly.
Here is a complete example:
use std::iter::Filter;
use std::slice::Iter;
struct FactorResult<'a, T: 'a>(Filter<Iter<'a, T>, fn(&&T) -> bool>);
impl<'a, T> Iterator for FactorResult<'a, T> {
type Item = &'a T;
fn next(&mut self) -> Option<&'a T> { self.0.next() }
fn size_hint(&self) -> (usize, Option<usize>) { self.0.size_hint() }
}
fn factor(input_array: &[i32]) -> FactorResult<i32> {
fn even(x : &&i32) -> bool { **x % 2 == 0 }
FactorResult(input_array.iter().filter(even))
}
fn main () {
for x in factor(&[1,2,3,4]) {
println!("{}", x);
}
}
The factor function returns a FactorResult which is just a wrapper that hides the actual underlying type.
The only thing the user knows about FactorResult is that it is an Iterator. The implementation of the trait is trivial, but I had to spell it out.
I had to replace the closure with the function. This is because here Rust does not perform any allocations, so it needs to know the size of FactorResult<T>, but the type of the closure is anonymous so there is no way to refer to it. One could use a closure but the whole thing would have to be boxed in this case.

Resources