I've got code that looks (a little) like this:
struct OutputIterator<'r, 'i: 'r> {
input_handler: &'r mut InputHandler<'i>
}
impl<'r, 'i> Iterator for OutputIterator<'r, 'i> {
type Item = i32;
fn next(&mut self) -> Option<Self::Item> {
self.input_handler.inputs.next().map(|x| x * 2)
}
}
struct InputHandler<'a> {
inputs: Box<dyn Iterator<Item = i32> + 'a>
}
impl<'a> InputHandler<'a> {
fn outputs<'b>(&'b mut self) -> OutputIterator<'b, 'a> {
OutputIterator { input_handler: self }
}
}
fn main() {
let mut input_handler = InputHandler {
inputs: Box::new(vec![1,2,3,4,5].into_iter())
};
for output in input_handler.outputs() {
println!("{}", output);
}
}
Basically, the user can supply an iterator of inputs, and then get an iterator of outputs (in my real code the connection between inputs and outputs is much more complex involving a bunch of internal state. Multiple inputs might go towards one output or vice-versa).
This works, but I would like to change it use impl Iterator both to hide the OutputIterator type and to allow for easier swapping out of the return type in testing with a fake. My best attempt at that changes the InputHandler impl like so:
impl<'a> InputHandler<'a> {
fn outputs<'b>(&'b mut self) -> impl Iterator<Item = i32> + 'b {
OutputIterator { input_handler: self }
}
}
Unfortunately, that gets me: error[E0700]: hidden type for `impl Trait` captures lifetime that does not appear in bounds
Is there a way to make this work? It's important for the interface that InputHandler take an iterator with some lifetime, and that obviously has to be passed on to the OutputIterator somehow, but I'd really like to abstract those details away from the caller. In principle, the caller should only have to worry about making sure that the inputs Iterator and InputHandler outlive the OutputIterator so I think the logical lifetime bound on the OutputIterator here is the smaller of those two? Any clarity on why this error is happening or how to fix it would be great!
In case it helps, here's a rust playground with the code in it.
Using a workaround from https://github.com/rust-lang/rust/issues/34511#issuecomment-373423999, via https://stackoverflow.com/a/50548538/1757964:
trait Captures<'a> {}
impl<'a, T: ?Sized> Captures<'a> for T {}
impl<'a> InputHandler<'a> {
fn outputs<'b>(&'b mut self) -> impl Iterator<Item = i32> + Captures<'a> + 'b {
OutputIterator { input_handler: self }
}
}
Related
I currently have code that looks kind of like this:
struct People {
names: Vec<String>,
ages: Vec<i32>,
}
impl People {
fn iter_people<'a>(&'a self) -> PeopleIterator<'a> {
return PeopleIterator {
names_iterator: Box::new(self.names.iter()),
ages: Box::new(self.ages.iter()),
};
}
}
struct PeopleIterator<'a> {
names_iterator: Box<dyn Iterator<Item = &'a String>>,
ages: Box<dyn Iterator<Item = &'a i32>>,
}
impl<'a> Iterator for PeopleIterator<'a> {
...snip...
}
I am aware that I should model a person as a struct Person and then have a Vec<Person> to model people but this is just a simplification of my actual code.
Anyway, the Rust compiler tells me this:
lifetime may not live long enough
requirement occurs because of the type PeopleIterator<'_>, which makes the generic argument '_ invariant
I have looked at the suggested link for subtyping and variance but I need to read it a few more times to actually understand it.
What stumps me is that I would expect both my iterators self.names.iter() and self.ages.iter() to live as long as self and I have declared that self should live as long as PeopleIterator. However, when I look at the iter() function, it does not make this constraint but instead has an anonymous lifetime '_. I am guessing this is the problem but I am confused and don't know how to fix it :(
The problem is the lifetime of the iterator itself in Box<dyn Iterator<Item = &'a String>> is by default bound to be 'static, but that's not possible for an iterator containing non static references like anything from &'a self. The solution is to specify an explicit lifetime bound:
struct PeopleIterator<'a> {
names_iterator: Box<dyn Iterator<Item = &'a String> + 'a>,
ages: Box<dyn Iterator<Item = &'a i32> + 'a>,
}
Personally I'd just use generics instead of static dispatch avoiding some indirection and the whole problem from the beginning:
impl People {
fn iter_people(&self) -> PeopleIterator<impl Iterator<Item = &String>, impl Iterator<Item = &i32>> {
return PeopleIterator {
names_iterator: self.names.iter(),
ages: self.ages.iter(),
};
}
}
struct PeopleIterator<N, A> {
names_iterator: N,
ages: A,
}
I am new to Rust and currently have been following Learning Rust With Entirely Too Many Linked Lists examples. Before section IterMut everything makes sense to me. Yet when implementing IterMut(in a differrnt way from the tutorial), I got totally confused by lifetime mechanism in Rust. Here is the case, first I'd just define the stack to be implemented:
pub struct List<T> {
head: Link<T>,
}
type Link<T> = Option<Box<Node<T>>>;
struct Node<T> {
elem: T,
next: Link<T>,
}
Ok then when I try to implement iterator in the following way:
pub struct IterMut<'a, T>{
this: &'a mut Link<T>
}
impl<T> List<T> {
pub fn iter_mut(&mut self) -> IterMut<T> {
IterMut {
this: &mut self.head
}
}
}
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
if let Some(node) = self.this {
Some(&mut node.elem)
} else {
None
}
}
}
It won't compile, the result says:
error: lifetime may not live long enough
impl<'a, T> Iterator for IterMut<'a, T>{
-- lifetime `'a` defined here
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
- let's call the lifetime of this reference `'1`
if let Some(node) = self.this {
Some(&mut node.elem)
^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'a`
Originally, I implement function next in this way, using as_mut and map:
// function body in the fn next
self.this.as_mut().map(|node|{
self.this = &mut node.next;
&mut node.elem
})
This also triggers compilation error (incompatible lifetime).
Therefore I wonder what is going on here, shouldn't self in fn next have the same lifetime as IterMut ('a)? And is there any workaround for this situation?
The principal problem lies in trying to take a reference to the whole head. While that reference lives, you can't hand out mutable references to anything inside it. You only need access to the Node that's inside head, though. So first, we refactor IterMut to only keep a reference into any given Node:
pub struct IterMut<'a, T>{
this: Option<&'a mut Node<T>>
}
Now, to get that out of the head we use the convenience method as_deref_mut() that Option provides. It just gives us a mutable reference to whatever was inside it (if anything):
impl<T> List<T> {
pub fn iter_mut(&mut self) -> IterMut<'_, T> {
IterMut {
this: self.head.as_deref_mut(),
}
}
}
Now, 'a is only tied to that Node and we can do what we want with it, like takeing it:
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
if let Some(node) = self.this.take() {
self.this = node.next.as_deref_mut();
Some(&mut node.elem)
} else {
None
}
}
}
And we can simplify that with a simple map call:
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
self.this.take().map(|node| {
self.this = node.next.as_deref_mut();
&mut node.elem
})
}
}
Consider next function for example:
fn print(input: &dyn Display) {
println!("{}", input);
}
It works fine, as intended.
Now consider a little bit more complex example using trait with the same function inside:
trait Print {
type Input: ?Sized;
fn print(input: &Self::Input);
}
impl Print for f64 {
type Input = dyn Display;
// actually the same function as before
fn print(input: &dyn Display) {
println!("{}", input);
}
}
Now if I modify first function like this:
fn print(input: &dyn Display) {
f64::print(input);
}
I got compile error:
error[E0759]: `input` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
Why is this happening? I see no reason for 'static lifetime requirement in trait's associated type.
Worked example here.
Yes, when a dyn _ type is used as an associated type, it implicitly receives a lifetime bound of 'static. This is to prevent situations like this:
pub trait Stew {
type Item: ?Sized;
fn stew(&mut self, item: Box<Self::Item>);
}
use std::fmt::Display;
pub struct DisplayStew(Vec<Box<dyn Display>>);
impl Stew for DisplayStew {
type Item = dyn Display;
fn stew(&mut self, item: Box<Self::Item>) {
self.0.push(item);
}
}
impl DisplayStew {
pub fn new() -> Self {
DisplayStew(Vec::new())
}
pub fn look(&self) {
for item in &self.0 {
println!("- {}", item);
}
}
}
fn main() {
let mut bowl = DisplayStew::new();
bowl.stew(Box::new(123.456));
bowl.stew(Box::new("abcdef"));
{ // 'a begins
let s = "foo".to_owned();
bowl.stew(Box::new(&s)); // Box<&'a str>
} // 'a expires
// s was borrowed by bowl, but has ceased to exist, what now?
bowl.look();
}
Without a lifetime parameter, the compiler has no way of restricting how long a type may keep a reference to its associated type.
On stable Rust (1.58.1), there is currently no way to address this, but the following version works on unstable:
#![feature(generic_associated_types)]
use std::fmt::Display;
trait Print {
type Input<'a>: ?Sized;
fn print<'a>(input: &'a Self::Input<'a>);
}
impl Print for f64 {
type Input<'a> = dyn Display + 'a;
fn print<'a>(input: &'a (dyn Display + 'a)) {
println!("{}", input);
}
}
fn print(input: &dyn Display) {
println!("{}", input);
f64::print(input);
}
fn main() {
print(&123.);
}
I have a custom collection like this:
struct VecChoice<T> {
v1: Vec<T>,
v2: Vec<T>,
use_v1: Vec<bool>,
}
in the impl I can iterate this collection like this:
fn foo(&self, ...) {
let item_refs: Vec<_> = (0..self.v1.len()).map(|i| {
if self.use_v1[i] {
&self.v1[i]
} else {
&self.v2[i]
}
});
// ... do whatever I want with chosen references
}
However, I am failing to make it iterable:
impl<'a, T> IntoIterator for &'a VecChoice<T> {
type Item = &'a T;
// this fails because the trait `Sized` is not implemented for `(dyn FnMut(usize) -> Self::Item + 'static)`
type IntoIter = Map<usize, dyn FnMut(usize) -> Self::Item>;
fn into_iter(self) -> Self::IntoIter {
(0..self.v1.len()).map(|i| {
if self.use_v1[i] {
&self.v1[i]
} else {
&self.v2[i]
}
})
}
}
I could probably collect results into a Vec<&T> as above, then use its into_iter, but I suspect there should be a way to do it without constructing intermediate Vec.
The closure that you have passed to map actually does have a size. The problem though is that this type isn't nameable. You've tried to solve that with dyn, which isn't quite the right solution because the closure is sized but dyn makes it so that it isn't. dyn would be appropriate if there were different possible sizes, but then you'd have to put it behind a pointer of some kind so that the IntoIter type is Sized.
This is one of those cases where it is probably better to implement the Iterator manually, rather than using combinators.
struct VecChoiceIter<'a, T> {
index: usize,
vec_choice: &'a VecChoice<T>,
}
impl<'a, T> Iterator for VecChoiceIter<'a, T> {
type Item = &'a T;
fn next(&mut self) -> Option<Self::Item> {
if self.index == self.vec_choice.v1.len() {
None
} else {
let i = self.index;
self.index += 1;
let use_v1 = self.vec_choice.use_v1[i];
if use_v1 {
Some(&self.vec_choice.v1[i])
} else {
Some(&self.vec_choice.v2[i])
}
}
}
}
This gives you a Sized and nameable type that you can use for the IntoIterator implementation:
impl<'a, T> IntoIterator for &'a VecChoice<T> {
type Item = &'a T;
type IntoIter = VecChoiceIter<'a, T>;
fn into_iter(self) -> Self::IntoIter {
VecChoiceIter { index: 0, vec_choice: self }
}
}
There are some interesting RFCs in progress that could make this work more like how you originally wanted. In particular RFC-2515. This would let you write your IntoIterator implementation as you originally tried, but without having to name the type (playground - nightly):
impl<'a, T> IntoIterator for &'a VecChoice<T> {
type Item = &'a T;
// This is an "existential" type. That is, tell the compiler that there is
// exactly one possibility for what this type can be, which it can infer
// from the usage.
type IntoIter = impl Iterator<Item = Self::Item>;
fn into_iter(self) -> Self::IntoIter {
(0..self.v1.len()).map(move |i| {
if self.use_v1[i] {
&self.v1[i]
} else {
&self.v2[i]
}
})
}
}
It's often very tempting to try to make an iterator out of a pre-made collection, but unfortunately this tends to run into a practical problem a lot of the time: you need some way to store an offset into that collection, so you serve the right chunk of data out of it when next is called. Consequently, you almost always need to provide some custom iterator type.
In this case, you can do so like this:
struct VecChoice<T> {
v1: Vec<T>,
v2: Vec<T>,
use_v1: Vec<bool>,
}
struct VecChoiceIter<'a, T> {
off: usize,
collection: &'a VecChoice<T>,
}
impl<'a, T> Iterator for VecChoiceIter<'a, T> {
type Item = &'a T;
fn next(&mut self) -> Option<Self::Item> {
let off = self.off;
self.off += 1;
if *self.collection.use_v1.get(off)? {
self.collection.v1.get(off)
} else {
self.collection.v2.get(off)
}
}
}
impl<'a, T> IntoIterator for &'a VecChoice<T> {
type Item = &'a T;
type IntoIter = VecChoiceIter<'a, T>;
fn into_iter(self) -> Self::IntoIter {
VecChoiceIter {
off: 0,
collection: self,
}
}
}
Note that in this case, I've switched use_v1 to a Vec<bool>, because this is not C and only booleans can be used in conditionals.
You could also do the conversion up front and store it in its own Vec, but in my experience people don't expect creating an iterator, whether by calling iter or into_iter, to be expensive. Iterators are pretty fundamental in Rust, and as a consequence it's very common for folks to create lots of them, often implicitly, and making those functions be expensive would be undesirable in many cases.
Probably the most simple way is to use .zip() and return an opaque impl Iterator from a method on the type (so you don't have to write out the actual type):
struct VecChoice<T> {
v1: Vec<T>,
v2: Vec<T>,
use_v1: Vec<bool>,
}
impl<T> VecChoice<T> {
fn iter(&self) -> impl Iterator<Item = &T> {
self.v1
.iter()
.zip(self.v2.iter())
.zip(self.use_v1.iter())
.map(|((v1, v2), use_v1)| if use_v1 { v1 } else { v2 })
}
}
This will iterate over all three Vec (actually the shortest of them) and return either from v1 or v2.
Notice that I switched use_v1 from a Vec<T> to a Vec<bool>, which seems to be what you have, given the way you use it.
How can I reduce the lifetime of a closure?
I was trying to make a method, which returns an iterator related to self. I didn't want to make new struct or something, so I just made it return filters and maps, and confronted some borrow checker errors.
The following code was my first try.
fn f<'b>(&'b self) -> impl Iterator<Item = u8> {
(0..self.some_number())
.filter(|&i| self.some_bool_function(i))
.map(|i| i as u8)
}
The following code replicates my question.
struct A(bool);
impl A {
fn f<'a>(&'a self) -> impl Iterator<Item = u8> + 'a {
(0..1).filter(|&i| self.0)
}
}
or even shorter,
fn g<'a>(t:&'a ()) -> impl 'a + FnMut() {
|| *t
}
This would not compile, because the closure may outlive self. I don't know how to make this work, without moving self.
If you return a closure, you must ensure that the closure has everything it needs - even after returning (i.e. after the (temporary) function parameters are popped from the stack).
Thus, I think you want to move the stuff you return into the closure:
impl A {
fn f<'a>(&'a self) -> impl Iterator<Item = u8> + 'a {
(0..1).filter(move |&i| self.0)
}
}
Resp.
fn g<'a>(t:&'a ()) -> impl 'a + FnMut() {
move || *t
}
Resp (extending your first example):
struct A(bool);
impl A {
fn some_number(&self) -> usize {
6
}
fn some_bool_function(&self, i: usize) -> bool {
i%2==0
}
fn f<'b>(&'b self) -> impl Iterator<Item = u8> + 'b {
(0..self.some_number())
.filter(move |&i| self.some_bool_function(i))
.map(|i| i as u8)
}
}