IntoIterator as a function argument doesn't accept adapter struct - rust

I want to have a function, that accepts &IntoIterator<Item=u32>, so I could pass to it both &Vec<u32> and iterators' adapter structs (like Map, Filter and any other, which I believe all implement IntoIterator)
So I have a function like
pub fn f<'a, T>(it_src: &'a T) -> u32
where &'a T: IntoIterator<Item = u32> {
let it = it_src.into_iter();
let result: u32;
// more more usage
result
}
And this is how I tried to use it (same signature, but different name)
pub fn f_with_feature()<'a, T>(it_src: &'a T) -> u32
where &'a T: IntoIterator<Item = u32> {
let adjusted_values = it_src.into_iter()
.map(|e| adjust(e));
f(&adjusted_values)
}
What I've got is an error
error[E0308]: mismatched types
--> src\main.rs:14:7
|
14 | f(&adjusted_values)
| ^^^^^^^^^^^^^^^^ expected type parameter, found struct `std::iter::Map`
|
= note: expected type `&T`
found type `&std::iter::Map<<&T as std::iter::IntoIterator>::IntoIter, [closure#src\main.rs:13:14: 13:27]>`
How is it that Map doesn't match as T?
Also, I've come up with an idea, that passing iterators' adaptors with static dispatch isn't a good idea since each other closure used to generate a Map will create a new function specialization. Though I've seen that static dispatch approach for most of the times is idiomatic in Rust. How to manage this situation?

I think you want to have trait bounds on T (and not on &'a T). So I guess you actually want the following:
pub fn f<'a, T>(it_src: &'a T) -> u32
where T: IntoIterator<Item = u32> {
let it = it_src.into_iter();
let result: u32 = 1;
// more more usage
result
}
pub fn f_with_feature<'a, T>(it_src: &'a T) -> u32
where T: IntoIterator<Item = u32> {
let adjusted_values = it_src.into_iter()
.map(|e| adjust(e));
f(&adjusted_values)
}
Which brings us to the next problem: IntoIterator's into_iter consumes self, which means that you cannot call it_src.into_iter if you only borrow it_src.
So if you really want to use into_iter, you can try this:
pub fn f<T>(it_src: T) -> u32
where T: IntoIterator<Item = u32> {
let it = it_src.into_iter();
let result: u32 = 1;
// more more usage
result
}
pub fn f_with_feature<T>(it_src: T) -> u32
where T: IntoIterator<Item = u32> {
let adjusted_values = it_src.into_iter()
.map(|e| adjust(e));
f(adjusted_values)
}
The above, however, requires you to move the values into f resp. f_with_feature.
In my experience, just taking an iterator (and doing the conversion at call site if necessary), leads to simple, straightforward solutions:
pub fn f<T>(it_src: T) -> u32
where T: Iterator<Item = u32> {
let it = it_src.into_iter();
let result: u32 = 1;
// more more usage
result
}
pub fn f_with_feature<T>(it_src: T) -> u32
where T: Iterator<Item = u32> {
let adjusted_values = it_src.into_iter()
.map(|e| adjust(e));
f(adjusted_values)
}

Related

type mismatch with Peekable of Iterator trait object

This code compiles fine:
use std::iter::Peekable;
struct Token {
length: u32,
}
fn get_tokens() -> Box<dyn Iterator<Item = Token>> {
todo!()
}
fn do_something_with_boxed_filter(boxed: Box<dyn Iterator<Item = Token>>) {}
fn main() {
let tokens = get_tokens();
let long_tokens = tokens.filter(|token| token.length > 32);
let boxed = Box::new(long_tokens);
do_something_with_boxed_filter(boxed);
}
But if I try to make a peekable version, as shown below, it fails to compile.
use std::iter::Peekable;
struct Token {
length: u32,
}
fn get_tokens() -> Box<dyn Iterator<Item = Token>> {
todo!()
}
fn do_something_with_peekable(peekable: Peekable<Box<dyn Iterator<Item = Token>>>) {}
fn main() {
let tokens = get_tokens();
let long_tokens = tokens.filter(|token| token.length > 32);
let boxed = Box::new(long_tokens);
let peekable = boxed.peekable();
do_something_with_peekable(peekable);
}
The error message is:
error[E0308]: mismatched types
--> src/main.rs:18:32
|
15 | let long_tokens = tokens.filter(|token| token.length > 32);
| ------------------------- the found closure
...
18 | do_something_with_peekable(peekable);
| ^^^^^^^^ expected trait object `dyn Iterator`, found struct `Filter`
|
= note: expected struct `Peekable<Box<(dyn Iterator<Item = Token> + 'static)>>`
found struct `Peekable<Box<Filter<Box<dyn Iterator<Item = Token>>, [closure#src/main.rs:15:37: 15:62]>>>`
For more information about this error, try `rustc --explain E0308`.
I don't understand this; the Filter struct implements Iterator so shouldn't I be able to use a Filter struct in place of the dyn Iterator trait object?
Yes, you are right with all the things you said.
The problem is that you can only feed things into functions that can be coerced.
Box<T> is coercible to Box<dyn Trait> if T: Trait. However, at the time of writing, this goes only one level deep. Peekable<Box<T>> can not be coerced into Peekable<Box<dyn Trait>>. Therefore, you need to force the coercion before creating the Peekable:
use std::iter::Peekable;
struct Token {
length: u32,
}
fn get_tokens() -> Box<dyn Iterator<Item = Token>> {
todo!()
}
fn do_something_with_peekable(peekable: Peekable<Box<dyn Iterator<Item = Token>>>) {}
fn main() {
let tokens = get_tokens();
let long_tokens = tokens.filter(|token| token.length > 32);
let boxed = Box::new(long_tokens) as Box<dyn Iterator<Item = Token>>;
let peekable = boxed.peekable();
do_something_with_peekable(peekable);
}
An alternative would be to use the impl keyword:
use std::iter::Peekable;
struct Token {
length: u32,
}
fn get_tokens() -> Box<dyn Iterator<Item = Token>> {
todo!()
}
fn do_something_with_peekable(peekable: Peekable<Box<impl Iterator<Item = Token>>>) {}
fn main() {
let tokens = get_tokens();
let long_tokens = tokens.filter(|token| token.length > 32);
let boxed = Box::new(long_tokens);
let peekable = boxed.peekable();
do_something_with_peekable(peekable);
}
With impl, no coercion happens. Box<dyn Trait> is a Box to an unsized trait type, where Box<impl Trait> is actually a real type that implements the Trait. The information what type it is does not get lost to the compiler, impl Trait is actually syntactic sugar for intruducing a generic Box<T> where T: Trait.
impl has advantages and disadvantages. The biggest disadvantage is that just like normal generics, it compiles a new version of the function for every type that gets used with it (I think).
The advantage is that there is no longer the need for a Box at all:
use std::iter::Peekable;
struct Token {
length: u32,
}
fn get_tokens() -> impl Iterator<Item = Token> {
vec![].into_iter()
}
fn do_something_with_peekable(peekable: Peekable<impl Iterator<Item = Token>>) {}
fn main() {
let tokens = get_tokens();
let long_tokens = tokens.filter(|token| token.length > 32);
let peekable = long_tokens.peekable();
do_something_with_peekable(peekable);
}
Just be aware that the types of two objects of Peekable<dyn Trait> are always identical, while the type of two Peekable<impl Trait> objects could be different. Peekable<dyn Trait> is a type, while Peekable<impl Trait> is not a type, it's more of a placeholder for a type. Therefore, you cannot store different objects of Peekable<impl Trait> in one vector, because the vector wouldn't know which type to choose.

implementing RngCore for my own rng, with function returning result

I have a usecase where I need to implement my own rng that can be passed to my function that require a RngCore rng.
So I need to implemennt the RngCore trait for my rng, that's running on a esp32 device.
What I have currently looks like this:
impl RngCore for HRNG {
fn next_u32(&mut self) -> u32 {
let mut n : u32;
unsafe {
let n = esp_idf_sys::esp_random();
}
n
}
fn next_u64(&mut self) -> u64 {
let mut n : u32;
unsafe {
let n = esp_idf_sys::esp_random();
}
n
}
fn fill_bytes(&mut self, dest: &mut [u8]) {
unsafe { esp_idf_sys::esp_fill_random(dest.as_ptr() as *mut core::ffi::c_void, dest.len().try_into().unwrap()); }
}
fn try_fill_bytes(&mut self, dest: &mut [u8]) -> Result<(), Error> {
//
}
}
My issue is with the try_fill_bytes method.
It returns a result, that can be an error.
And currenlty the code gives me this issue:
the size for values of type `(dyn std::error::Error + 'static)` cannot be known at compilation time
But I need to implement the trait, so I'm not sure what to do, since the function signature is required?
How do I actually get to implement this trait?
You need rand::Error, not (implicitly dyn) std::error::Error.

How to implement the `Index` trait for a simple struct?

I'm trying to implement the Index trait for a simple trait, and I want to use it with usize. I added SliceIndex<[T], Output = T> so I can use T to index the slice inside A.
use std::ops::Index;
use std::slice::SliceIndex;
struct A <'a, T>{
slice: &'a [T]
}
impl<'a, T: Index<T, Output = T> + SliceIndex<[T], Output = T>> Index<T>
for A<'a, T>
{
type Output = T;
#[inline(always)]
fn index(&self, index: T) -> &Self::Output {
self.slice.index(index)
}
}
fn main() {
let mut aa: Vec<u64> = vec![0; 10];
let coefficient_iterable = A{slice: &aa};
println!("{}", coefficient_iterable[1usize]);
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=9564b39061cae3e19db14217c10b9d8a
But I get:
Error:
error[E0608]: cannot index into a value of type `A<'_, u64>`
--> src/main.rs:22:20
|
22 | println!("{}", coefficient_iterable[1usize]);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For more information about this error, try `rustc --explain E0608`.
error: could not compile `playground` due to previous error
I have no idea why, since usize implements SliceIndex<[T]>.
Requiring T to be both Index and SliceIndex doesn't make sense because Index describes the behaviour of indexable containers, while SliceIndex is a bound you can put on a generic indexing type to make it more flexible.
You don't want to implement Index<T> because T here is the type of the items in the slice, not the type of the index. For the code in your main function to work you could just implement Index<usize>, but generically implementing Index<Idx> (where Idx: SliceIndex) gives more flexibility so you can use ranges too.
use std::ops::Index;
use std::slice::SliceIndex;
struct A<'a, T> {
slice: &'a [T],
}
impl<'a, T, Idx> Index<Idx> for A<'a, T>
where
Idx: SliceIndex<[T], Output = T>,
{
type Output = T;
#[inline(always)]
fn index(&self, index: Idx) -> &Self::Output {
self.slice.index(index)
}
}
fn main() {
let aa: Vec<u64> = vec![0; 10];
let coefficient_iterable = A { slice: &aa };
assert_eq!(coefficient_iterable[1], 0);
}

Generic type parameter with constraints from unsigned integer type

I am trying to use a generic datatype which implements 2 methods from the unsigned primitive integer types in rust (u8, u32, u64, etc.). The goal is to be able to read some predefined datatype from a stream (playground link):
use std::io::Read;
use std::io::BufReader;
pub trait ReadExt {
fn read<T>(&mut self) -> Result<T, std::io::Error>;
}
impl<R: Read> ReadExt for R {
fn read<T>(&mut self) -> Result<T, std::io::Error> {
let mut v = Vec::with_capacity((T::BITS >> 3).into());
self.read_exact(&mut v);
Ok(T::from_le_bytes(v))
}
}
fn main() {
let mut stream = BufReader::new(&0xffffffffu32.to_le_bytes()[..]);
let x = ReadExt::read::<u32>(&mut stream).unwrap();
assert_eq!(x, 0xffffffffu32);
}
I get the compile error:
error[E0599]: no associated item named `BITS` found for type parameter `T` in the current scope
--> src/main.rs:11:44
|
11 | let mut v = Vec::with_capacity((T::BITS >> 3).into());
| ^^^^ associated item not found in `T`
error[E0599]: no function or associated item named `from_le_bytes` found for type parameter `T` in the current scope
--> src/main.rs:13:15
|
13 | Ok(T::from_le_bytes(v))
| ^^^^^^^^^^^^^ function or associated item not found in `T`
How do I add constraints on T and specify that T must implement the traits from u8, u16, u32? Specifically, I want T to implement BITS and from_le_bytes.
If you want to use any methods or constants coming from T, then T must be bounded/constrained by some trait, because you can use only methods/constants/etc coming from the trait bounds.
In your case that would be:
pub trait FromLeBytes {
const BITS: u32;
fn from_le_bytes(bytes: [u8; 4]) -> Self;
}
which defines the required method from_le_bytes() and the constant BITS.
You will have to implement that trait for each type that you want to use. Currently I've implemented it only for u32:
impl FromLeBytes for u32 {
const BITS: u32 = u32::BITS;
fn from_le_bytes(bytes: [u8; 4]) -> Self {
u32::from_le_bytes(bytes)
}
}
Then you need to declare that constraint by modifying your ReadExt trait:
pub trait ReadExt {
fn read<T: FromLeBytes>(&mut self) -> Result<T, std::io::Error>;
}
After which you are ready to use the method and the constant in your code:
use std::io::BufReader;
use std::io::Read;
pub trait FromLeBytes {
const BITS: u32;
fn from_le_bytes(bytes: [u8; 4]) -> Self;
}
impl FromLeBytes for u32 {
const BITS: u32 = u32::BITS;
fn from_le_bytes(bytes: [u8; 4]) -> Self {
u32::from_le_bytes(bytes)
}
}
pub trait ReadExt {
fn read<T: FromLeBytes>(&mut self) -> Result<T, std::io::Error>;
}
impl<R: Read> ReadExt for R {
fn read<T: FromLeBytes>(&mut self) -> Result<T, std::io::Error> {
// Could NOT have been done like that because of: https://github.com/rust-lang/rust/issues/68436
// let mut v = [0; T::BITS as usize / 8];
// self.read_exact(&mut v);
// Ok(T::from_le_bytes(v))
let mut v = Vec::with_capacity(T::BITS as usize / 8);
v.resize(v.capacity(), 0);
self.read_exact(&mut v).unwrap();
let mut a = [0; 4];
for (i, b) in v.iter().copied().enumerate() {
a[i] = b;
}
Ok(T::from_le_bytes(a))
}
}
fn main() {
let x = &0xffffffffu32.to_le_bytes()[..];
let mut stream = BufReader::new(x);
let x = ReadExt::read::<u32>(&mut stream).unwrap();
assert_eq!(x, 0xffffffffu32);
}
PS: I've also fixed some compilation errors in your original code
This is not easily possible because BITS and from_le_bytes are both not from a trait, but associated functions of u8, u16 etc. There is no common trait that has those functions/consts.
You can partially work around this by relying on num_traits::int::PrimInt::from_le() to convert endianness and std::mem::size_of() to get the number of bytes in the integer. However, from_le() only allows converting an existing integer – it does not allow to convert a byte buffer, and your use case needs that.
If you are okay with using unsafe Rust, you can achieve what you want via std::mem::transmute_copy. Below you find code doing exactly that (I also fixed a few bugs in your original code). But please read on – there is still a subtle problem.
use std::io::Read;
use std::io::BufReader;
use num_traits::int::PrimInt;
pub trait ReadExt {
fn read<T: PrimInt>(&mut self) -> Result<T, std::io::Error>;
}
impl<R: Read> ReadExt for R {
fn read<T: PrimInt>(&mut self) -> Result<T, std::io::Error> {
let mut v = vec![0u8; std::mem::size_of::<T>()];
self.read_exact(&mut v).unwrap();
let result_in_le: T = unsafe {
std::mem::transmute_copy(&v[0])
};
Ok(T::from_le(result_in_le))
}
}
fn main() {
let buf = 0xffffffffu32.to_le_bytes();
let mut stream = BufReader::new(&buf[..]);
let x = ReadExt::read::<u32>(&mut stream).unwrap();
assert_eq!(x, 0xffffffffu32);
}
Playground
This implementation has a way for the user to trigger undefined behavior: As PrimInt is not sealed, one could implement the trait on some custom struct containing a f64. Using ReadExt::read(), one could then transform arbitrary byte patterns into a f64 – this triggers undefined behavior.
To solve this, we can construct our own trait that is a subtrait of PrimInt and unsafe:
use std::io::Read;
use std::io::BufReader;
use num_traits::int::PrimInt;
pub unsafe trait UnsafeInt: PrimInt {
}
unsafe impl UnsafeInt for u8 {}
unsafe impl UnsafeInt for u16 {}
unsafe impl UnsafeInt for u32 {}
unsafe impl UnsafeInt for u64 {}
unsafe impl UnsafeInt for usize {}
unsafe impl UnsafeInt for i8 {}
unsafe impl UnsafeInt for i16 {}
unsafe impl UnsafeInt for i32 {}
unsafe impl UnsafeInt for i64 {}
unsafe impl UnsafeInt for isize {}
pub trait ReadExt {
fn read<T: UnsafeInt>(&mut self) -> Result<T, std::io::Error>;
}
impl<R: Read> ReadExt for R {
fn read<T: UnsafeInt>(&mut self) -> Result<T, std::io::Error> {
let mut v = vec![0u8; std::mem::size_of::<T>()];
self.read_exact(&mut v).unwrap();
let result_in_le: T = unsafe {
std::mem::transmute_copy(&v[0])
};
Ok(T::from_le(result_in_le))
}
}
fn main() {
let buf = 0xffffffffu32.to_le_bytes();
let mut stream = BufReader::new(&buf[..]);
let x = ReadExt::read::<u32>(&mut stream).unwrap();
assert_eq!(x, 0xffffffffu32);
}
Playground
This guarantees that T is a primitive type, thereby making the transmute_copy sound.

How can I explicitly specify a lifetime when implementing a trait?

Given the implementation below, where essentially I have some collection of items that can be looked up via either a i32 id field or a string field. To be able to use either interchangeably, a trait "IntoKey" is used, and a match dispatches to the appropriate lookup map; this all works fine for my definition of get within the MapCollection impl:
use std::collections::HashMap;
use std::ops::Index;
enum Key<'a> {
I32Key(&'a i32),
StringKey(&'a String),
}
trait IntoKey<'a> {
fn into_key(&'a self) -> Key<'a>;
}
impl<'a> IntoKey<'a> for i32 {
fn into_key(&'a self) -> Key<'a> { Key::I32Key(self) }
}
impl<'a> IntoKey<'a> for String {
fn into_key(&'a self) -> Key<'a> { Key::StringKey(self) }
}
#[derive(Debug)]
struct Bar {
i: i32,
n: String,
}
struct MapCollection
{
items: Vec<Bar>,
id_map: HashMap<i32, usize>,
name_map: HashMap<String, usize>,
}
impl MapCollection {
fn new(items: Vec<Bar>) -> MapCollection {
let mut is = HashMap::new();
let mut ns = HashMap::new();
for (idx, item) in items.iter().enumerate() {
is.insert(item.i, idx);
ns.insert(item.n.clone(), idx);
}
MapCollection {
items: items,
id_map: is,
name_map: ns,
}
}
fn get<'a, K>(&self, key: &'a K) -> Option<&Bar>
where K: IntoKey<'a> //'
{
match key.into_key() {
Key::I32Key(i) => self.id_map.get(i).and_then(|idx| self.items.get(*idx)),
Key::StringKey(s) => self.name_map.get(s).and_then(|idx| self.items.get(*idx)),
}
}
}
fn main() {
let bars = vec![Bar { i:1, n:"foo".to_string() }, Bar { i:2, n:"far".to_string() }];
let map = MapCollection::new(bars);
if let Some(bar) = map.get(&1) {
println!("{:?}", bar);
}
if map.get(&3).is_none() {
println!("no item numbered 3");
}
if let Some(bar) = map.get(&"far".to_string()) {
println!("{:?}", bar);
}
if map.get(&"baz".to_string()).is_none() {
println!("no item named baz");
}
}
However, if I then want to implement std::ops::Index for this struct, if I attempt to do the below:
impl<'a, K> Index<K> for MapCollection
where K: IntoKey<'a> {
type Output = Bar;
fn index<'b>(&'b self, k: &K) -> &'b Bar {
self.get(k).expect("no element")
}
}
I hit a compiler error:
src/main.rs:70:18: 70:19 error: cannot infer an appropriate lifetime for automatic coercion due to conflicting requirements
src/main.rs:70 self.get(k).expect("no element")
^
src/main.rs:69:5: 71:6 help: consider using an explicit lifetime parameter as shown: fn index<'b>(&'b self, k: &'a K) -> &'b Bar
src/main.rs:69 fn index<'b>(&'b self, k: &K) -> &'b Bar {
src/main.rs:70 self.get(k).expect("no element")
src/main.rs:71 }
I can find no way to specify a distinct lifetime here; following the compiler's recommendation is not permitted as it changes the function signature and no longer matches the trait, and anything else I try fails to satisfy the lifetime specification.
I understand that I can implement the trait for each case (i32, String) separately instead of trying to implement it once for IntoKey, but I am more generally trying to understand lifetimes and appropriate usage. Essentially:
Is there actually an issue the compiler is preventing? Is there something unsound about this approach?
Am I specifying my lifetimes incorrectly? To me, the lifetime 'a in Key/IntoKey is dictating that the reference need only live long enough to do the lookup; the lifetime 'b associated with the index fn is stating that the reference resulting from the lookup will live as long as the containing MapCollection.
Or am I simply not utilizing the correct syntax to specify the needed information?
(using rustc 1.0.0-nightly (b63cee4a1 2015-02-14 17:01:11 +0000))
Do you intend on implementing IntoKey on struct's that are going to store references of lifetime 'a? If not, you can change your trait and its implementations to:
trait IntoKey {
fn into_key<'a>(&'a self) -> Key<'a>;
}
This is the generally recommended definition style, if you can use it. If you can't...
Let's look at this smaller reproduction:
use std::collections::HashMap;
use std::ops::Index;
struct Key<'a>(&'a u8);
trait IntoKey<'a> { //'
fn into_key(&'a self) -> Key<'a>;
}
struct MapCollection;
impl MapCollection {
fn get<'a, K>(&self, key: &'a K) -> &u8
where K: IntoKey<'a> //'
{
unimplemented!()
}
}
impl<'a, K> Index<K> for MapCollection //'
where K: IntoKey<'a> //'
{
type Output = u8;
fn index<'b>(&'b self, k: &K) -> &'b u8 { //'
self.get(k)
}
}
fn main() {
}
The problem lies in get:
fn get<'a, K>(&self, key: &'a K) -> &u8
where K: IntoKey<'a>
Here, we are taking a reference to K that must live as long as the Key we get out of it. However, the Index trait doesn't guarantee that:
fn index<'b>(&'b self, k: &K) -> &'b u8
You can fix this by simply giving a fresh lifetime to key:
fn get<'a, 'b, K>(&self, key: &'b K) -> &u8
where K: IntoKey<'a>
Or more succinctly:
fn get<'a, K>(&self, key: &K) -> &u8
where K: IntoKey<'a>

Resources