For some time, my Rust program used a DashMap with a parallel iterator bridge. This was a major performance bottleneck, and I have just found that these days, DashMap supposedly supports direct parallel iteration. So I updated my dependency crates and changed my code. Now I have my DashMap<K, V, S> like this
let families_by_location: DashMap<
petgraph::graph::NodeIndex<usize>,
Vec<&mut Family>,
std::hash::BuildHasherDefault<FxHasher>,
> = DashMap::default();
families_by_location.into_par_iter().map(...).collect();
However, this does not compile, and surprisingly, the error message is very unhelpful for me.
error[E0599]: the method `into_par_iter` exists for struct `dashmap::DashMap<NodeIndex<usize>, Vec<&mut Family>, BuildHasherDefault<rustc_hash::FxHasher>>`, but its trait bounds were not satisfied
--> src/lib.rs:442:10
|
442 | .into_par_iter()
| ^^^^^^^^^^^^^ method cannot be called on `dashmap::DashMap<NodeIndex<usize>, Vec<&mut Family>, BuildHasherDefault<rustc_hash::FxHasher>>` due to unsatisfied trait bounds
|
::: /home/gereon/.cargo/registry/src/github.com-1ecc6299db9ec823/dashmap-5.2.0/src/lib.rs:67:1
|
67 | pub struct DashMap<K, V, S = RandomState> {
| -----------------------------------------
| |
| doesn't satisfy `_: rayon::iter::IntoParallelIterator`
| doesn't satisfy `_: rayon::iter::ParallelIterator`
|
= note: the following trait bounds were not satisfied:
`dashmap::DashMap<NodeIndex<usize>, Vec<&mut Family>, BuildHasherDefault<rustc_hash::FxHasher>>: rayon::iter::ParallelIterator`
which is required by `dashmap::DashMap<NodeIndex<usize>, Vec<&mut Family>, BuildHasherDefault<rustc_hash::FxHasher>>: rayon::iter::IntoParallelIterator`
`&dashmap::DashMap<NodeIndex<usize>, Vec<&mut Family>, BuildHasherDefault<rustc_hash::FxHasher>>: rayon::iter::ParallelIterator`
which is required by `&dashmap::DashMap<NodeIndex<usize>, Vec<&mut Family>, BuildHasherDefault<rustc_hash::FxHasher>>: rayon::iter::IntoParallelIterator`
`&mut dashmap::DashMap<NodeIndex<usize>, Vec<&mut Family>, BuildHasherDefault<rustc_hash::FxHasher>>: rayon::iter::ParallelIterator`
which is required by `&mut dashmap::DashMap<NodeIndex<usize>, Vec<&mut Family>, BuildHasherDefault<rustc_hash::FxHasher>>: rayon::iter::IntoParallelIterator`
For more information about this error, try `rustc --explain E0599`.
error: could not compile `dispersal_model_rust` due to previous error
I also tried families_by_location.par_iter_mut() for a similarly unhelpful error concerning trait bounds.
I don't understand the issue. The trait bounds on DashMap<K, V, S> being an IntoParallelIterator are K: Send + Eq + Hash, V: Send, S: Send + Clone + BuildHasher.
K is NodeIndex<usize>, which implements Send + Sync + Eq + Hash because usize does.
The FxHasher Builder supposedly also supports Send + Sync + Clone + BuildHasher, and Vec is Send + Sync if its object type is. Now I would understand it if &mut Family is not Send, sending around mutable pointers between threads sounds dangerous. However, something like
let a: Vec<&mut Family> = families.par_iter_mut().collect();
let b: &dyn Sync = &a;
does no throw a compilation error, so it seems that Vec<&mut Family> is actually Send + Sync.
So it seems to me that the trait bounds for DashMap<K, V, S>: IntoParallelIterator trait bounds are fulfilled, why do I get the Rust compiler error?
An old comment on the DashMap issue tracker, which I found when searching for “parallel” on there, pointed out to me that activating the rayon feature is necessary. I did, and lo and behold my code compiles.
(I still find the error message which I get when not doing it quite confusing.)
While the list of feature flags on dashmap's docs.rs page states
rayon
This feature flag does not enable additional features.
that statement is a bit misleading: As Kevin Reid pointed out, “unlike most of rustdoc and docs.rs content, there is no automatic reporting of what a feature flag does, except for enabling other feature flags.” So all this statement only means is that activating dashmap's rayon feature does not activate other feature flags.
Related
I can create a Write trait object that points to a (heap-stored) File object as such:
let a : Box<dyn Write> = Box::new(File::open("/dev/null").unwrap()); // Compiles fine.
Using the turbofish notation, however, produces an error (Rust 1.56.1):
let a : Box<dyn Write> = Box::<dyn Write>::new(File::open("/dev/null").unwrap()); // Fails to compile.
The error being:
error[E0599]: the function or associated item `new` exists for struct `Box<dyn std::io::Write>`, but its trait bounds were not satisfied
--> src/main.rs:37:48
|
37 | let a : Box<dyn Write> = Box::<dyn Write>::new(File::open("/dev/null").unwrap()); // Fails to compile.
| ^^^ function or associated item cannot be called on `Box<dyn std::io::Write>` due to unsatisfied trait bounds
|
::: /home/[REDACTED]/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/io/mod.rs:1368:1
|
1368 | pub trait Write {
| --------------- doesn't satisfy `dyn std::io::Write: Sized`
|
= note: the following trait bounds were not satisfied:
`dyn std::io::Write: Sized`
I've been poking at the problem for the better part of a day. Either I'm missing something fundamental, or the special language treatment of Box is at play here (but I don't see how).
Your code is actually quite deceptive.
let a: Box<dyn Write> = Box::new(File::open("/dev/null").unwrap());
doesn't actually create a Box<dyn Write> like you think. It first creates a Box<File>, then the Box<File> is cast into a Box<dyn Write>. If you turbofish with File, then it works:
let a: Box<dyn Write> = Box::<File>::new(File::open("/dev/null").unwrap());
So, if you want to be explicit about the type, use an as cast instead of turbofish:
// a is Box<dyn Write>
let a = Box::new(File::open("/dev/null").unwrap()) as Box<dyn Write>;
When I try to compile this line, opt.positionals = Positionals::try_from(opt.args).unwrap(); I'm getting an error about a trait not being implement. This trait should be implemented though generically as I have TryFrom<Vec<T>>
Here is the error,
error[E0277]: the trait bound `Positionals: TryFrom<Vec<String>>` is not satisfied
--> bin/seq.rs:83:20
|
83 | opt.positionals = Positionals::try_from(opt.args).unwrap();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `TryFrom<Vec<String>>` is not implemented for `Positionals`
|
= help: the following implementations were found:
<Positionals as TryFrom<Vec<T>>>
And here is my implementation of the trait,
use std::convert::TryFrom;
impl<T: Debug + AsRef<String> + Into<String> + Clone> TryFrom<Vec<T>> for Positionals {
Why doesn't my implementation of TryFrom<Vec<T>> cover the concrete TryFrom<Vec<String>> and how can I remedy this problem?
String does not implement AsRef<String>. You could use AsRef<str> or Borrow<String> instead.
When I make a composite trait (name?) that is made of two other traits, the compiler doesn't seem to detect that the two traits are implemented by that composite trait.
Here is the code:
use std::io::{Read, Seek};
use zip::read::ZipArchive;
trait ReadSeek: Read + Seek {}
impl<T: Read + Seek> ReadSeek for T {}
pub struct ArchiveZip<'a> {
path: &'a str,
archive: ZipArchive<&'a (dyn ReadSeek)>,
}
And here is the error:
error[E0277]: the trait bound `&'a (dyn ReadSeek + 'a): std::io::Read` is not satisfied
--> src/archive/zipfile.rs:12:14
|
12 | archive: ZipArchive<&'a (dyn ReadSeek)>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::io::Read` is not implemented for `&'a (dyn ReadSeek + 'a)`
|
::: /Users/nhooey/.cargo/registry/src/github.com-1ecc6299db9ec823/zip-0.5.11/src/read.rs:50:26
|
50 | pub struct ZipArchive<R: Read + io::Seek> {
| ---- required by this bound in `ZipArchive`
|
= note: `std::io::Read` is implemented for `&'a mut (dyn archive::ReadSeek + 'a)`, but not for `&'a (dyn archive::ReadSeek + 'a)`
Why does it not detect that the ReadSeek trait doesn't implement both Read and Seek?
(I omitted the other error message that is also complaining about Seek in the same way.)
From the error message you can see: struct ZipArchive<R: Read + io::Seek>.
This means that the generic argument (R) to ZipArchive must implement Read and io::Seek.
You code has: ZipArchive<&'a (dyn ReadSeek)>.
R in your code is therefore &'a (dyn ReadSeek) which is a reference (&'_ _). The fact that the object the reference points to implements Read and Seek doesn't change which traits the reference itself implements.
So, you can either implement your trait for references (in my experience this usually doesn't go well) or you find a way for R to be a type that directly implements the required traits.
Looking at the documentation for references, you can see that Read and Seek are implemented for mutable references. Changing you references to be mutable should therefore solve the problem.
I have a piece of code that does not compile and that can be reduced to this snippet:
use std::error::Error;
use std::convert::TryFrom;
// A trait that provides methods for parsing data into a type T.
pub trait Deserializable<T> {
// some methods
}
pub struct MyBuffer<'a> {
inner: &'a [u8]
}
impl<'a, T> Deserializable<T> for MyBuffer<'a>
where
T: TryFrom<&'a [u8]>,
<T as TryFrom<&'a [u8]>>::Error: Error + Sync + Send + 'static
{
// some methods to implement
}
fn main() {}
The compiler rejects this program with a confusing error message:
error[E0310]: the associated type `<T as std::convert::TryFrom<&[u8]>>::Error` may not live long enough
--> src/main.rs:13:13
|
13 | impl<'a, T> Deserializable<T> for MyBuffer<'a>
| ^^^^^^^^^^^^^^^^^
|
= help: consider adding an explicit lifetime bound `<T as std::convert::TryFrom<&[u8]>>::Error: 'static`...
note: ...so that the type `<T as std::convert::TryFrom<&[u8]>>::Error` will meet its required lifetime bounds
--> src/main.rs:13:13
|
13 | impl<'a, T> Deserializable<T> for MyBuffer<'a>
| ^^^^^^^^^^^^^^^^^
The error suggests that I add the 'static lifetime bound, but I already added it:
consider adding an explicit lifetime bound `<T as std::convert::TryFrom<&[u8]>>::Error: 'static`
Can someone explain why this program does not compile, and/or how to fix it (if possible)? It seems to me that it should be possible to have <T as TryFrom<&'a [u8]>>::Error to be 'static even though T itself is bound to 'a.
The reason I want Error to be 'static is that I use failure and failure::Fail is implemented for Send + Sync + Error + 'static.
This appears to be a gap in the compiler's ability to reason about lifetimes and associated types.
Sometimes, in order to help the compiler, you can add a generic parameter to alias an associated type. This parameter doesn't "count" in that it doesn't make the item "more generic", but because generics are resolved at use instead of declaration, it defers the hard part of type checking until the exact types are known. In other words: the compiler might be able to prove that any particular T::Error works, but it can't quite prove that every T::Error must work, so we introduce a new parameter E which is bound to be T::Error and tell the compiler to figure out what E is only when we try to use it.
The following works (playground):
impl<'a, T, E> Deserializable<T> for MyBuffer<'a>
where
T: TryFrom<&'a [u8], Error = E>,
E: Error + Sync + Send + 'static,
{
// ...
}
Instead of bounding T::Error, we introduce a new type parameter E with the bounds we want and constrain T such that its TryFrom::Error is E. This is logically (as far as I can tell) the same thing as what you wrote, but it compiles without complaint.
I can't find official documentation that refers to this; it may be an inherent limitation of the solver, or merely a bug.
I'm still not sure why, but it seems just removing the 'static bound on the error worked?
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=d4f46a0ad9a5dd7dc538fe4e197d823d
I have a piece of code that does not compile and that can be reduced to this snippet:
use std::error::Error;
use std::convert::TryFrom;
// A trait that provides methods for parsing data into a type T.
pub trait Deserializable<T> {
// some methods
}
pub struct MyBuffer<'a> {
inner: &'a [u8]
}
impl<'a, T> Deserializable<T> for MyBuffer<'a>
where
T: TryFrom<&'a [u8]>,
<T as TryFrom<&'a [u8]>>::Error: Error + Sync + Send + 'static
{
// some methods to implement
}
fn main() {}
The compiler rejects this program with a confusing error message:
error[E0310]: the associated type `<T as std::convert::TryFrom<&[u8]>>::Error` may not live long enough
--> src/main.rs:13:13
|
13 | impl<'a, T> Deserializable<T> for MyBuffer<'a>
| ^^^^^^^^^^^^^^^^^
|
= help: consider adding an explicit lifetime bound `<T as std::convert::TryFrom<&[u8]>>::Error: 'static`...
note: ...so that the type `<T as std::convert::TryFrom<&[u8]>>::Error` will meet its required lifetime bounds
--> src/main.rs:13:13
|
13 | impl<'a, T> Deserializable<T> for MyBuffer<'a>
| ^^^^^^^^^^^^^^^^^
The error suggests that I add the 'static lifetime bound, but I already added it:
consider adding an explicit lifetime bound `<T as std::convert::TryFrom<&[u8]>>::Error: 'static`
Can someone explain why this program does not compile, and/or how to fix it (if possible)? It seems to me that it should be possible to have <T as TryFrom<&'a [u8]>>::Error to be 'static even though T itself is bound to 'a.
The reason I want Error to be 'static is that I use failure and failure::Fail is implemented for Send + Sync + Error + 'static.
This appears to be a gap in the compiler's ability to reason about lifetimes and associated types.
Sometimes, in order to help the compiler, you can add a generic parameter to alias an associated type. This parameter doesn't "count" in that it doesn't make the item "more generic", but because generics are resolved at use instead of declaration, it defers the hard part of type checking until the exact types are known. In other words: the compiler might be able to prove that any particular T::Error works, but it can't quite prove that every T::Error must work, so we introduce a new parameter E which is bound to be T::Error and tell the compiler to figure out what E is only when we try to use it.
The following works (playground):
impl<'a, T, E> Deserializable<T> for MyBuffer<'a>
where
T: TryFrom<&'a [u8], Error = E>,
E: Error + Sync + Send + 'static,
{
// ...
}
Instead of bounding T::Error, we introduce a new type parameter E with the bounds we want and constrain T such that its TryFrom::Error is E. This is logically (as far as I can tell) the same thing as what you wrote, but it compiles without complaint.
I can't find official documentation that refers to this; it may be an inherent limitation of the solver, or merely a bug.
I'm still not sure why, but it seems just removing the 'static bound on the error worked?
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=d4f46a0ad9a5dd7dc538fe4e197d823d