Passing an iterable, or a collection to a function - rust

Consider this pseudo-pseudocode. I have quite some difficulties with the keyarg argument below,
type MapSet<K,V> = HashMap<K,HashSet<V>>;
fn contract<K,V,V1>(left: &MapSet<K,V>, right: &MapSet<V,V1>,
out: &mut MapSet<K,V1>,
keyarg: Option(__something_iterable__) {
let keys = match keyarg {
Some(keys) => {
keys
},
None => {
left.keys()
},
}
for &k in keys {
// ... do stuff ...
}
}
The function is used sometimes like this,
let x: MapSet<u32,String>;
let y: MapSet<String,u32>;
let out: MapSet<u32,u32>;
let kk: HashSet<u32>;
// ... snip ...
contract(x,y,out,kk);
And, at other times like that,
let x: MapSet<u32,String>;
let y: MapSet<String,u32>;
let out: MapSet<u32,u32>;
contract(x,y,out,None);
In other words, I am sometimes using keyarg argument to pass an Option with a reference to a HashSet containing the keys I want to iterate over and at other times I want to iterate over all the keys contained in left argument, so the keyarg becomes simply None.
But, so far I have always ended up in a problem with the match which complains that None leads to Keys object and Some branch to HashSet (type mismatch error).
My question is how to define the keyarg argument so that the branches of match are compatible with each other. That is, I want to express the fact that the variable keys is just something one can iterate over to the compiler.

Unless keyarg will always match your MapSet iterator, and in your pseudo code it doesn't, you'll need keys to be a trait object. The iterator for MapSet<K,V> has the trait Iterator<Item = &K>, so you can match it into a Box like so:
let keys: Box<dyn Iterator<Item = &K>> = match keyarg {
Some(keys) => Box::new(...),
None => Box::new(left.keys()),
}
As far as determining what keyarg should be, the generic way would be to accept any matching iterable like so:
fn contract<'a, K, V, V1, I: IntoIterator<Item = &'a K>>(
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
left: &'a MapSet<K, V>,
right: &MapSet<V, V1>,
out: &mut MapSet<K, V1>,
keyarg: Option<I>,
) {
let keys: Box<dyn Iterator<Item = &K>> = match keyarg {
Some(keys) => Box::new(keys.into_iter()),
None => Box::new(left.keys()),
};
// ...
}
And this is very ergonomic for passing a HashSet but causes a problem for the None case:
// This works
contract(&x, &y, &mut out, Some(&kk));
// This fails with error: "type annotations needed, cannot infer type for type parameter `I`"
contract(&x, &y, &mut out, None);
So you'd have to annotate None with some dummy iterator type (like Box<...> or std::iter::Empty or something) which isn't ideal.
Instead you can make keyarg the boxed iterator directly:
fn contract_1<'a, K, V, V1>(
left: &'a MapSet<K, V>,
right: &MapSet<V, V1>,
out: &mut MapSet<K, V1>,
keyarg: Option<Box<dyn Iterator<Item = &'a K> + 'a>>,
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
) {
let keys: Box<dyn Iterator<Item = &K>> = match keyarg {
Some(keys) => keys,
None => Box::new(left.keys()),
};
// ...
}
Which can be used like so:
contract(&x, &y, &mut out, Some(Box::new(kk.iter())));
contract(&x, &y, &mut out, None);

Related

Temporarily cache owned value between iterator adapters

I'd like to know if there's a way to cache an owned value between iterator adapters, so that adapters later in the chain can reference it.
(Or if there's another way to allow later adapters to reference an owned value that lives inside the iterator chain.)
To illustrate what I mean, let's look at this (contrived) example:
I have a function that returns a String, which is called in an Iterator map() adapter, yielding an iterator over Strings. I'd like to get an iterator over the chars() in those Strings, but the chars() method requires a string slice, meaning a reference.
Is this possible to do, without first collecting the Strings?
Here's a minimal example that of course fails:
fn greet(c: &str) -> String {
"Hello, ".to_owned() + c
}
fn main() {
let names = ["Martin", "Helena", "Ingrid", "Joseph"];
let iterator = names.into_iter().map(greet);
let fails = iterator.flat_map(<str>::chars);
}
Playground
Using a closure instead of <str>::chars - |s| s.chars() - does of course not work either. It makes the types match, but breaks lifetimes.
Edit (2022-10-03): In response to the comments, here's some pseudocode of what I have in mind, but with incorrect lifetimes:
struct IteratorCache<'a, T, I>{
item : Option<T>,
inner : I,
_p : core::marker::PhantomData<&'a T>
}
impl<'a, T, I> Iterator for IteratorCache<'a, T,I>
where I: Iterator<Item=T>
{
type Item=&'a T;
fn next(&mut self) -> Option<&'a T> {
self.item = self.inner.next();
if let Some(x) = &self.item {
Some(&x)
} else {
None
}
}
}
The idea would be that the reference could stay valid until the next call to next(). However I don't know if this can be expressed with the function signature of the Iterator trait. (Or if this can be expressed at all.)
I don't think something like this exists yet, and collecting into a Vec<char> creates some overhead, but you can write such an iterator yourself with a little bit of trickery:
struct OwnedCharsIter {
s: String,
index: usize,
}
impl OwnedCharsIter {
pub fn new(s: String) -> Self {
Self { s, index: 0 }
}
}
impl Iterator for OwnedCharsIter {
type Item = char;
fn next(&mut self) -> Option<Self::Item> {
// Slice of leftover characters
let slice = &self.s[self.index..];
// Iterator over leftover characters
let mut chars = slice.chars();
// Query the next char
let next_char = chars.next()?;
// Compute the new index by looking at how many bytes are left
// after querying the next char
self.index = self.s.len() - chars.as_str().len();
// Return next char
Some(next_char)
}
}
fn greet(c: &str) -> String {
"Hello, ".to_owned() + c
}
fn main() {
let names = ["Martin", "Helena", "Ingrid", "Joseph"];
let iterator = names.into_iter().map(greet);
let chars_iter = iterator.flat_map(OwnedCharsIter::new);
println!("{:?}", chars_iter.collect::<String>())
}
"Hello, MartinHello, HelenaHello, IngridHello, Joseph"

Initialize a Vec with not-None values only

If I have variables like this:
let a: u32 = ...;
let b: Option<u32> = ...;
let c: u32 = ...;
, what is the shortest way to make a vector of those values, so that b is only included if it's Some?
In other words, is there something simpler than this:
let v = match b {
None => vec![a, c],
Some(x) => vec![a, x, c],
};
P.S. I would prefer a solution where we don't need to use the variables more than once. Consider this example:
let some_person: String = ...;
let best_man: Option<String> = ...;
let a_third_person: &str = ...;
let another_opt: Option<String> = ...;
...
As can be seen, we might have to use longer variable names, more than one Option (None), expressions (like a_third_person.to_string()), etc.
Yours is fine, but here's a sophisticated one:
[Some(a), b, Some(c)].into_iter().flatten().collect::<Vec<_>>()
This works since Option impls IntoIterator.
If it depends on just one variable:
b.map(|b| vec![a, b, c]).unwrap_or_else(|| vec![a, c]);
Playground
After some thinking and investigating, I've come with the following crazy thing.
The end goal is to have a macro, optional_vec![], that you can pass it either T or Option<T> and it should behave like described in the question. However, I decided on a strong restriction: it should have the best performance possible. So, you write:
optional_vec![a, b, c]
And get at least the performance of hand-written match, if not more. This forbids the use of the simple [Some(a), b, Some(c)].into_iter().flatten().collect::<Vec<_>>(), suggested in my other answer (though even this solution needs some way to differentiate between Option<T> and just T, which, like we'll see, is not an easy problem at all).
I will first warn that I've not found a way to make my macro work with Option. That is, if you want to build a vector of Option<T> from Option<T> and Option<Option<T>>, it will not work.
When a design a complex macro, I like to think first how the expanded code will look like. And in this macro, we have several hard problems to solve.
First, the macro take plain expressions. But somehow, it needs to switch on their type being T or Option<T>. How should such thing be done?
The feature we use to do such things is specialization.
#![feature(specialization)]
pub trait Optional {
fn some_method(self);
}
impl<T> Optional for T {
default fn some_method(self) {
// Just T
}
}
impl<T> Optional for Option<T> {
fn some_method(self) {
// Option<T>
}
}
Like you probably noticed, now we have two problems: first, specialization is unstable, and I'd like to stay with stable. Second, what should be inside the trait? The second problem is easier to solve, so let's begin with it.
Turns out that the most performant way to do the pushing to the vector is to pre-allocate capacity (Vec::with_capacity), write to the vector by using pointers (don't push(), it optimizes badly!) then set the length (Vec::set_len()).
We can get a pointer to the internal buffer of the vector using Vec::as_mut_ptr(), and advance the pointer via <*mut T>::add(1).
So, we need two methods: one to hint us about the capacity (can be zero for None or one for Some() and non-Option elements), and a write_and_advance() method:
pub trait Optional {
type Item;
fn len(&self) -> usize;
unsafe fn write_and_advance(self, place: &mut *mut Self::Item);
}
impl<T> Optional for T {
default type Item = Self;
default fn len(&self) -> usize { 1 }
default unsafe fn write_and_advance(self, place: &mut *mut Self) {
place.write(self);
*place = place.add(1);
}
}
impl<T> Optional<T> for Option<T> {
type Item = T;
fn len(&self) -> usize { self.is_some() as usize }
unsafe fn write_and_advance(self, place: &mut *mut T) {
if let Some(value) = self {
place.write(value);
*place = place.add(1);
}
}
}
It doesn't even compile! For the why, see Mismatch between associated type and type parameter only when impl is marked `default`. Luckily for us, the trick we'll use to workaround specialization not being stable does work in this situation. But for now, let's assume it works. How will the code using this trait look like?
match (a, b, c) { // The match is here because it's the best binding for liftimes: see https://stackoverflow.com/a/54855986/7884305
(a, b, c) => {
let len = Optional::len(&a) + Optional::len(&b) + Optional::len(&c);
let mut result = ::std::vec::Vec::with_capacity(len);
let mut next_element = result.as_mut_ptr();
unsafe {
Optional::write_and_advance(a, &mut next_element);
Optional::write_and_advance(b, &mut next_element);
Optional::write_and_advance(c, &mut next_element);
result.set_len(len);
}
result
}
}
And it works! Except that it does not, because the specialization does not compile as I said, and we also want to not repeat all of this boilerplate but insert it into a macro.
So, how do we solve the problems with specialization: being unstable and not working?
dtonlay has a very cool trick he calls autoref specialization (BTW, all of this repo is a very recommended reading!). This is a trick that can be used to emulate specialization. It works only in macros, but we're in a macro so this is fine.
I will not elaborate about the trick here (I recommend to read his post; he also used this trick in the excellent and very widely used anyhow crate). In short, the idea is to trick the typechecker by implementing a trait for T under certain conditions (the specialized impl) and other trait for &T for the general case (this could be inherent impl if not coherence). Since Rust performs automatic referencing during method resolution, that is take reference to the receiver as needed, this will work - the typechecker will autoref if needed, and will stop in the first applicable impl - i.e. the specialized impl if it matches, or the general impl otherwise.
Here's an example:
use std::fmt;
pub trait Display {
fn foo(&self);
}
// Level 1
impl<T: fmt::Display> Display for T {
fn foo(&self) { println!("Display({}), {}", std::any::type_name::<T>(), self); }
}
pub trait Debug {
fn foo(&self);
}
// Level 2
impl<T: fmt::Debug> Debug for &T {
fn foo(&self) { println!("Debug({}), {:?}", std::any::type_name::<T>(), self); }
}
macro_rules! foo {
($e:expr) => ((&$e).foo());
}
Playground.
We can use this trick in our case:
#[doc(hidden)]
pub mod autoref_specialization {
#[derive(Copy, Clone)]
pub struct OptionTag;
pub trait OptionKind {
fn optional_kind(&self) -> OptionTag;
}
impl<T> OptionKind for Option<T> {
#[inline(always)]
fn optional_kind(&self) -> OptionTag { OptionTag }
}
impl OptionTag {
#[inline(always)]
pub fn len<T>(self, this: &Option<T>) -> usize { this.is_some() as usize }
#[inline(always)]
pub unsafe fn write_and_advance<T>(self, this: Option<T>, place: &mut *mut T) {
if let Some(value) = this {
place.write(value);
*place = place.add(1);
}
}
}
#[derive(Copy, Clone)]
pub struct DefaultTag;
pub trait DefaultKind {
fn optional_kind(&self) -> DefaultTag;
}
impl<T> DefaultKind for &'_ T {
#[inline(always)]
fn optional_kind(&self) -> DefaultTag { DefaultTag }
}
impl DefaultTag {
#[inline(always)]
pub fn len<T>(self, _this: &T) -> usize { 1 }
#[inline(always)]
pub unsafe fn write_and_advance<T>(self, this: T, place: &mut *mut T) {
place.write(this);
*place = place.add(1);
}
}
}
And the expanded code will look like:
use autoref_specialization::{DefaultKind as _, OptionKind as _};
match (a, b, c) {
(a, b, c) => {
let (a_tag, b_tag, c_tag) = (
(&a).optional_kind(),
(&b).optional_kind(),
(&c).optional_kind(),
);
let len = a_tag.len(&a) + b_tag.len(&b) + c_tag.len(&c);
let mut result = ::std::vec::Vec::with_capacity(len);
let mut next_element = result.as_mut_ptr();
unsafe {
a_tag.write_and_advance(a, &mut next_element);
b_tag.write_and_advance(b, &mut next_element);
c_tag.write_and_advance(c, &mut next_element);
result.set_len(len);
}
result
}
}
It may be tempting to try to convert this immediately into a macro, but we still have one unsolved problem: our macro need to generate identifiers. This may not be obvious, but what if we pass optional_vec![1, Some(2), 3]? We need to generate the bindings for the match (in our case, (a, b, c) => ...) and the tag names ((a_tag, b_tag, c_tag)).
Unfortunately, generating names is not something macro_rules! can do in today's Rust. Fortunately, there is an excellent crate paste (another one from dtonlay!) that is a small proc-macro that allows you to do that. It is even available on the playground!
However, we need a series of identifiers. That can be done with tt-munching, by repeatedly adding some letter (I used a), so you get a, aa, aaa, ... you get the idea.
#[doc(hidden)]
pub mod reexports {
pub use std::vec::Vec;
pub use paste::paste;
}
#[macro_export]
macro_rules! optional_vec {
// Empty case
{ #generate_idents
exprs = []
processed_exprs = [$($e:expr,)*]
match_bindings = [$($binding:ident)*]
tags = [$($tag:ident)*]
} => {{
use $crate::autoref_specialization::{DefaultKind as _, OptionKind as _};
match ($($e,)*) {
($($binding,)*) => {
let ($($tag,)*) = (
$((&$binding).optional_kind(),)*
);
let len = 0 $(+ $tag.len(&$binding))*;
let mut result = $crate::reexports::Vec::with_capacity(len);
let mut next_element = result.as_mut_ptr();
unsafe {
$($tag.write_and_advance($binding, &mut next_element);)*
result.set_len(len);
}
result
}
}
}};
{ #generate_idents
exprs = [$e:expr, $($rest:expr,)*]
processed_exprs = [$($processed_exprs:tt)*]
match_bindings = [$first_binding:ident $($bindings:ident)*]
tags = [$($tags:ident)*]
} => {
$crate::reexports::paste! {
$crate::optional_vec! { #generate_idents
exprs = [$($rest,)*]
processed_exprs = [$($processed_exprs)* $e,]
match_bindings = [
[< $first_binding a >]
$first_binding
$($bindings)*
]
tags = [
[< $first_binding a_tag >]
$($tags)*
]
}
}
};
// Entry
[$e:expr $(, $exprs:expr)* $(,)?] => {
$crate::optional_vec! { #generate_idents
exprs = [$($exprs,)+]
processed_exprs = [$e,]
match_bindings = [__optional_vec_a]
tags = [__optional_vec_a_tag]
}
};
}
Playground.
I can also personally recommend
let mut v = vec![a, c];
v.extend(b);
Short and clear.
Sometime the straight forward solution is the best:
fn jim_power(a: u32, b: Option<u32>, c: u32) -> Vec<u32> {
let mut acc = Vec::with_capacity(3);
acc.push(a);
if let Some(b) = b {
acc.push(b);
}
acc.push(c);
acc
}
fn ys_iii(
some_person: String,
best_man: Option<String>,
a_third_person: String,
another_opt: Option<String>,
) -> Vec<String> {
let mut acc = Vec::with_capacity(4);
acc.push(some_person);
best_man.map(|x| acc.push(x));
acc.push(a_third_person);
another_opt.map(|x| acc.push(x));
acc
}
If you don't care about the order of the values, another option is
Iterator::chain(
[a, c].into_iter(),
[b].into_iter().flatten()
).collect()
Playground

What's the most idiomatic way of working with an Iterator of Results? [duplicate]

This question already has answers here:
How do I stop iteration and return an error when Iterator::map returns a Result::Err?
(4 answers)
Closed 3 years ago.
I have code like this:
let things = vec![/* ...*/]; // e.g. Vec<String>
things
.map(|thing| {
let a = try!(do_stuff(thing));
Ok(other_stuff(a))
})
.filter(|thing_result| match *thing_result {
Err(e) => true,
Ok(a) => check(a),
})
.map(|thing_result| {
let a = try!(thing_result);
// do stuff
b
})
.collect::<Result<Vec<_>, _>>()
In terms of semantics, I want to stop processing after the first error.
The above code works, but it feels quite cumbersome. Is there a better way? I've looked through the docs for something like filter_if_ok, but I haven't found anything.
I am aware of collect::<Result<Vec<_>, _>>, and it works great. I'm specifically trying to eliminate the following boilerplate:
In the filter's closure, I have to use match on thing_result. I feel like this should just be a one-liner, e.g. .filter_if_ok(|thing| check(a)).
Every time I use map, I have to include an extra statement let a = try!(thing_result); in order to deal with the possibility of an Err. Again, I feel like this could be abstracted away into .map_if_ok(|thing| ...).
Is there another approach I can use to get this level of conciseness, or do I just need to tough it out?
There are lots of ways you could mean this.
If you just want to panic, use .map(|x| x.unwrap()).
If you want all results or a single error, collect into a Result<X<T>>:
let results: Result<Vec<i32>, _> = result_i32_iter.collect();
If you want everything except the errors, use .filter_map(|x| x.ok()) or .flat_map(|x| x).
If you want everything up to the first error, use .scan((), |_, x| x.ok()).
let results: Vec<i32> = result_i32_iter.scan((), |_, x| x.ok());
Note that these operations can be combined with earlier operations in many cases.
Since Rust 1.27, Iterator::try_for_each could be of interest:
An iterator method that applies a fallible function to each item in the iterator, stopping at the first error and returning that error.
This can also be thought of as the fallible form of for_each() or as the stateless version of try_fold().
You can implement these iterators yourself. See how filter and map are implemented in the standard library.
map_ok implementation:
#[derive(Clone)]
pub struct MapOkIterator<I, F> {
iter: I,
f: F,
}
impl<A, B, E, I, F> Iterator for MapOkIterator<I, F>
where
F: FnMut(A) -> B,
I: Iterator<Item = Result<A, E>>,
{
type Item = Result<B, E>;
#[inline]
fn next(&mut self) -> Option<Self::Item> {
self.iter.next().map(|x| x.map(&mut self.f))
}
}
pub trait MapOkTrait {
fn map_ok<F, A, B, E>(self, func: F) -> MapOkIterator<Self, F>
where
Self: Sized + Iterator<Item = Result<A, E>>,
F: FnMut(A) -> B,
{
MapOkIterator {
iter: self,
f: func,
}
}
}
impl<I, T, E> MapOkTrait for I
where
I: Sized + Iterator<Item = Result<T, E>>,
{
}
filter_ok is almost the same:
#[derive(Clone)]
pub struct FilterOkIterator<I, P> {
iter: I,
predicate: P,
}
impl<I, P, A, E> Iterator for FilterOkIterator<I, P>
where
P: FnMut(&A) -> bool,
I: Iterator<Item = Result<A, E>>,
{
type Item = Result<A, E>;
#[inline]
fn next(&mut self) -> Option<Result<A, E>> {
for x in self.iter.by_ref() {
match x {
Ok(xx) => if (self.predicate)(&xx) {
return Some(Ok(xx));
},
Err(_) => return Some(x),
}
}
None
}
}
pub trait FilterOkTrait {
fn filter_ok<P, A, E>(self, predicate: P) -> FilterOkIterator<Self, P>
where
Self: Sized + Iterator<Item = Result<A, E>>,
P: FnMut(&A) -> bool,
{
FilterOkIterator {
iter: self,
predicate: predicate,
}
}
}
impl<I, T, E> FilterOkTrait for I
where
I: Sized + Iterator<Item = Result<T, E>>,
{
}
Your code may look like this:
["1", "2", "3", "4"]
.iter()
.map(|x| x.parse::<u16>().map(|a| a + 10))
.filter_ok(|x| x % 2 == 0)
.map_ok(|x| x + 100)
.collect::<Result<Vec<_>, std::num::ParseIntError>>()
playground
filter_map can be used to reduce simple cases of mapping then filtering. In your example there is some logic to the filter so I don't think it simplifies things. I don't see any useful functions in the documentation for Result either unfortunately. I think your example is as idiomatic as it could get, but here are some small improvements:
let things = vec![...]; // e.g. Vec<String>
things.iter().map(|thing| {
// The ? operator can be used in place of try! in the nightly version of Rust
let a = do_stuff(thing)?;
Ok(other_stuff(a))
// The closure braces can be removed if the code is a single expression
}).filter(|thing_result| match *thing_result {
Err(e) => true,
Ok(a) => check(a),
}
).map(|thing_result| {
let a = thing_result?;
// do stuff
b
})
The ? operator can be less readable in some cases, so you might not want to use it.
If you are able to change the check function to return Some(x) instead of true, and None instead of false, you can use filter_map:
let bar = things.iter().filter_map(|thing| {
match do_stuff(thing) {
Err(e) => Some(Err(e)),
Ok(a) => {
let x = other_stuff(a);
if check_2(x) {
Some(Ok(x))
} else {
None
}
}
}
}).map(|thing_result| {
let a = try!(thing_result);
// do stuff
b
}).collect::<Result<Vec<_>, _>>();
You can get rid of the let a = try!(thing); by using a match in some cases as well. However, using filter_map here doesn't seem to help.

How to solve this lifetime-related error?

Say I have the following,
type EpollEventCallback<'a> = FnMut(c_int) + Send + Sync + 'a;
struct EpollFdEventHandler<'a> {
on_readable: Option<Box<EpollEventCallback<'a>>>,
on_writable: Option<Box<EpollEventCallback<'a>>>,
}
// Map from c_int -> EpollFdEventHandler.
type EpollEventHandlerMap<'a> = collections::HashMap<c_int, EpollFdEventHandler<'a>>;
fn add_fd_handler
<'a, T: Fn(bool, &'a mut EpollFdEventHandler<'a>)>(
map: &'a mut EpollEventHandlerMap<'a>,
fd: c_int,
adder: T)
{
let mut hash_entry: hash_map::Entry<'a, _, _> = map.entry(fd);
match hash_entry {
hash_map::Entry::Occupied(ref mut occ_e) => {
let entry: &mut EpollFdEventHandler<'a> = occ_e.get_mut();
adder(false, entry);
},
hash_map::Entry::Vacant(vac_e) => {
/*
adder(
true,
vac_e.insert(EpollFdEventHandler {
on_readable: None,
on_writable: None,
}),
);
*/
}
};
}
add_fd_handler is supposed to be a helper function for adding an "FD handler"; here, it's going to get passed a closure (adder) that will set either on_readable or on_writable, depending on which handler is being added. add_fd_handler's job is simply doing the hash table lookup, and inserting an empty entry if required. However:
src/event_loop.rs:85:35: 85:48 error: `(hash_entry:std::collections::hash::map::Occupied).0` does not live long enough
src/event_loop.rs:85 hash_map::Entry::Occupied(ref mut occ_e) => {
^~~~~~~~~~~~~
src/event_loop.rs:82:1: 101:2 note: reference must be valid for the lifetime 'a as defined on the block at 82:0...
src/event_loop.rs:82 {
src/event_loop.rs:83 let mut hash_entry: hash_map::Entry<'a, _, _> = map.entry(fd);
src/event_loop.rs:84 match hash_entry {
src/event_loop.rs:85 hash_map::Entry::Occupied(ref mut occ_e) => {
src/event_loop.rs:86 let entry: &mut EpollFdEventHandler<'a> = occ_e.get_mut();
src/event_loop.rs:87 adder(false, entry);
...
src/event_loop.rs:83:67: 101:2 note: ...but borrowed value is only valid for the block suffix following statement 0 at 83:66
src/event_loop.rs:83 let mut hash_entry: hash_map::Entry<'a, _, _> = map.entry(fd);
src/event_loop.rs:84 match hash_entry {
src/event_loop.rs:85 hash_map::Entry::Occupied(ref mut occ_e) => {
src/event_loop.rs:86 let entry: &mut EpollFdEventHandler<'a> = occ_e.get_mut();
src/event_loop.rs:87 adder(false, entry);
src/event_loop.rs:88 },
The error about occ_e only shows up if I try to use it with adder(false, entry)! Rust claims occ_e "does not live long enough", but it's only being used right there in that branch of the match, so how can that be?
My best guess presently is that the closure's second arg, as &'a mut is what's the issue here; my reference in occ_e isn't 'a, it's something shorter (the unspecified lifetime on hash_entry, I think, but I don't know how to notate that).
Let the compiler infer the proper lifetime instead:
fn add_fd_handler
<T: Fn(bool, &mut EpollFdEventHandler)>(
map: &mut EpollEventHandlerMap,
fd: c_int,
adder: T)
{
let mut hash_entry = map.entry(fd);
match hash_entry {
hash_map::Entry::Occupied(ref mut occ_e) => {
let entry = occ_e.get_mut();
adder(false, entry);
},
hash_map::Entry::Vacant(vac_e) => {
/*
adder(
true,
vac_e.insert(EpollFdEventHandler {
on_readable: None,
on_writable: None,
}),
);
*/
}
};
}
The problem is that you're letting the caller determine a lifetime for the callback, but you then invoke the callback with a mutable reference to a local variable. The caller couldn't possibly know about the lifetime of that local variable, so the compiler assumes that 'a must outlive the current function. Yet, entry does not outlive the function, which is why you get an error.
The declaration T: Fn(bool, &mut EpollFdEventHandler) is equivalent to T: for<'a, 'b> Fn(bool, &'a mut EpollFdEventHandler<'b>). The for keyword in this context allows you to declare that T must implement Fn for any value of the specified lifetime parameters. This is only valid for lifetime parameters, because different lifetime parameters do not cause multiple versions of a function to be defined, unlike for type parameters.

How to use Rust's Peekable?

I'm interested in peeking ahead in a character stream. To my understanding, Peekable would be the way to go. I can't quite figure out how to use it.
First attempt:
fn trawl<I, E>(pk: &mut I) where I: std::iter::Peekable<Result<char, E>> {
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
_ => (),
}
}
}
fn main() {
trawl(&mut std::io::stdio::stdin().chars());
}
This fails to compile with
> rustc /tmp/main.rs
/tmp/main.rs:1:37: 1:73 error: `std::iter::Peekable` is not a trait
/tmp/main.rs:1 fn trawl<I, E>(pk: &mut I) where I: std::iter::Peekable<Result<char, E>> {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to previous error
Okay, fair enough. I don't fully understand traits yet so I try to pass an iterator in and then create a peekable version:
fn trawl<I, E>(it: &mut I) where I: Iterator<Result<char, E>> {
let mut pk = it.peekable();
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
_ => (),
}
}
}
fn main() {
trawl(&mut std::io::stdio::stdin().chars().peekable());
}
This fails with
> rustc /tmp/main.rs
/tmp/main.rs:2:18: 2:20 error: cannot move out of dereference of `&mut`-pointer
/tmp/main.rs:2 let mut pk = it.peekable();
^~
/tmp/main.rs:7:65: 7:70 error: cannot move out of dereference of `&`-pointer
/tmp/main.rs:7 (Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
^~~~~
note: in expansion of format_args!
<std macros>:2:23: 2:77 note: expansion site
<std macros>:1:1: 3:2 note: in expansion of println!
/tmp/main.rs:7:39: 7:77 note: expansion site
error: aborting due to 2 previous errors
Could someone explain:
why Peekable couldn't appear in the function type for lack of being a trait,
what the compiler means when it says 'move out of dereference of' and
how I might resolve either or both?
A third version
fn trawl<I, E>(mut it: I) where I: Iterator<Result<char, E>> {
let mut pk = it.peekable();
loop {
let cur = pk.next();
let nxt = pk.peek();
match (cur, nxt) {
(Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
// (Some(i), ) => println!("{}", i.ok()),
_ => (),
}
}
}
fn main() {
trawl(std::io::stdio::stdin().chars().peekable());
}
This fails with:
> rustc /tmp/main.rs
/tmp/main.rs:7:65: 7:70 error: cannot move out of dereference of `&`-pointer
/tmp/main.rs:7 (Some(i), Some(nxt_i)) => println!("{} {}", i.ok(), nxt_i.ok()),
^~~~~
note: in expansion of format_args!
<std macros>:2:23: 2:77 note: expansion site
<std macros>:1:1: 3:2 note: in expansion of println!
/tmp/main.rs:7:39: 7:77 note: expansion site
error: aborting due to previous error
I fail to understand what rust is saying to me here, how Iterator.next would have a different return type from Peekable.peek.
Peekable is not a trait and thus cannot be used as a bound, which would suggest that it could mean one of many types. It is a single, specific, concrete type, struct Peekable<A, T>. As you have observed, it’s constructed by calling the peekable() method on an iterator, which changes it to something that is peekable.
Here’s how you’d use it if you just wanted to take an iterator:
fn trawl<I, E>(iter: I) where I: Iterator<Result<char, E>> {
let pk = pk.peekable();
…
}
Note also that the peekable() method takes self by value; you can’t take a mutable reference to an iterator there.
The alternative which is what you were aiming for but which I would be generally less inclined towards, would be to require the argument to be peekable, putting the burden onto the caller, as you had:
fn trawl<I, E>(pk: Peekable<E, I>) where I: Iterator<Result<char, E>> {
…
}
Peekable is actually a struct, not a trait. If you wanted to take a Peekable, you could define your function like this:
fn trawl<E, I>(it: Peekable<I>) where I: Iterator<Result<char, E>> {
...
}
Your second implementation is failing to compile because peek takes self by value (i.e. it consumes the iterator, returning a new one), so you can't call it through a &mut reference. Most code simply takes the iterator by value instead of by reference:
fn trawl<E, I>(it: I) where I: Iterator<Result<char, E>> {
let it = it.peekable();
...
}
If you don't want to move the iterator into a function like trawl, you can use the by_ref() method to create a new iterator that holds onto an &mut reference:
let mut my_iterator = /* whatever */;
trawl(my_iterator.by_ref());
// my_iterator is still usable here
As far as style goes, I would say that the second form is the better way to go, as the first leaks what's basically an implementation detail.
Rust has changed a bit since the previous answers. The way to do it now is:
fn trawl<I, E>(pk: Peekable<I>)
where I: Iterator<Item = Result<char, E>> {
…
}

Resources