How do I properly add lifetimes to an iterator containing other iterators in Rust? - rust

I currently have code that looks kind of like this:
struct People {
names: Vec<String>,
ages: Vec<i32>,
}
impl People {
fn iter_people<'a>(&'a self) -> PeopleIterator<'a> {
return PeopleIterator {
names_iterator: Box::new(self.names.iter()),
ages: Box::new(self.ages.iter()),
};
}
}
struct PeopleIterator<'a> {
names_iterator: Box<dyn Iterator<Item = &'a String>>,
ages: Box<dyn Iterator<Item = &'a i32>>,
}
impl<'a> Iterator for PeopleIterator<'a> {
...snip...
}
I am aware that I should model a person as a struct Person and then have a Vec<Person> to model people but this is just a simplification of my actual code.
Anyway, the Rust compiler tells me this:
lifetime may not live long enough
requirement occurs because of the type PeopleIterator<'_>, which makes the generic argument '_ invariant
I have looked at the suggested link for subtyping and variance but I need to read it a few more times to actually understand it.
What stumps me is that I would expect both my iterators self.names.iter() and self.ages.iter() to live as long as self and I have declared that self should live as long as PeopleIterator. However, when I look at the iter() function, it does not make this constraint but instead has an anonymous lifetime '_. I am guessing this is the problem but I am confused and don't know how to fix it :(

The problem is the lifetime of the iterator itself in Box<dyn Iterator<Item = &'a String>> is by default bound to be 'static, but that's not possible for an iterator containing non static references like anything from &'a self. The solution is to specify an explicit lifetime bound:
struct PeopleIterator<'a> {
names_iterator: Box<dyn Iterator<Item = &'a String> + 'a>,
ages: Box<dyn Iterator<Item = &'a i32> + 'a>,
}
Personally I'd just use generics instead of static dispatch avoiding some indirection and the whole problem from the beginning:
impl People {
fn iter_people(&self) -> PeopleIterator<impl Iterator<Item = &String>, impl Iterator<Item = &i32>> {
return PeopleIterator {
names_iterator: self.names.iter(),
ages: self.ages.iter(),
};
}
}
struct PeopleIterator<N, A> {
names_iterator: N,
ages: A,
}

Related

Rust lifetimes for implementing a trait on nested slices

I want to create a wrapper around (nested) slices for easy operations on multidimensional data, owned by a different struct.
The most basic version of the mutable version of my slice wrapper might look like this:
struct MySliceMut<'a> {
data: Vec<&'a mut [f32]>,
}
impl<'a, 'b> MySliceMut<'a> {
fn get(&'b mut self) -> &'a mut [&'b mut [f32]] {
self.data.as_mut_slice()
}
}
Now if I want to implement a trait, for instance AddAssign, Rust does not seem to infer the lifetime of &mut self from the implementing type. The compiler complains that &mut self might outlive 'a:
impl<'a> AddAssign<MySlice<'a>> for MySliceMut<'a> { // lifetime 'a
fn add_assign(&mut self, rhs: MySlice<'a>) { // lifetime '1
let a = self.get(); // lifetime may not live long enough, '1 must outlive 'a
let b = rhs.get();
// do inplace addition here
}
}
Full Code - Rust Playground
I tried to figure out the issue with the lifetimes, but can't find it. Would the trait impl require any additional annotations?
struct MySlice<'a> {
data: Vec<&'a [f32]>,
}
impl<'a, 'b> MySlice<'a> {
fn get(&'b self) -> &'a [&'b [f32]] {
self.data.as_slice()
}
}
Problem with your code is that fn get(&'b self) returns variable with wrong lifetime. Associated lifetime 'a of MySlice<'a> is lifetime of inner slice. Associated lifetime 'b of fn get(...) is lifetime of the self. So I guess the function probably should return &'b [&'a [f32]] instead.
-- Edited --
Make sure to change fn get(...) of MySliceMut either.

impl Iterator failing for iterator with multiple lifetime parameters

I've got code that looks (a little) like this:
struct OutputIterator<'r, 'i: 'r> {
input_handler: &'r mut InputHandler<'i>
}
impl<'r, 'i> Iterator for OutputIterator<'r, 'i> {
type Item = i32;
fn next(&mut self) -> Option<Self::Item> {
self.input_handler.inputs.next().map(|x| x * 2)
}
}
struct InputHandler<'a> {
inputs: Box<dyn Iterator<Item = i32> + 'a>
}
impl<'a> InputHandler<'a> {
fn outputs<'b>(&'b mut self) -> OutputIterator<'b, 'a> {
OutputIterator { input_handler: self }
}
}
fn main() {
let mut input_handler = InputHandler {
inputs: Box::new(vec![1,2,3,4,5].into_iter())
};
for output in input_handler.outputs() {
println!("{}", output);
}
}
Basically, the user can supply an iterator of inputs, and then get an iterator of outputs (in my real code the connection between inputs and outputs is much more complex involving a bunch of internal state. Multiple inputs might go towards one output or vice-versa).
This works, but I would like to change it use impl Iterator both to hide the OutputIterator type and to allow for easier swapping out of the return type in testing with a fake. My best attempt at that changes the InputHandler impl like so:
impl<'a> InputHandler<'a> {
fn outputs<'b>(&'b mut self) -> impl Iterator<Item = i32> + 'b {
OutputIterator { input_handler: self }
}
}
Unfortunately, that gets me: error[E0700]: hidden type for `impl Trait` captures lifetime that does not appear in bounds
Is there a way to make this work? It's important for the interface that InputHandler take an iterator with some lifetime, and that obviously has to be passed on to the OutputIterator somehow, but I'd really like to abstract those details away from the caller. In principle, the caller should only have to worry about making sure that the inputs Iterator and InputHandler outlive the OutputIterator so I think the logical lifetime bound on the OutputIterator here is the smaller of those two? Any clarity on why this error is happening or how to fix it would be great!
In case it helps, here's a rust playground with the code in it.
Using a workaround from https://github.com/rust-lang/rust/issues/34511#issuecomment-373423999, via https://stackoverflow.com/a/50548538/1757964:
trait Captures<'a> {}
impl<'a, T: ?Sized> Captures<'a> for T {}
impl<'a> InputHandler<'a> {
fn outputs<'b>(&'b mut self) -> impl Iterator<Item = i32> + Captures<'a> + 'b {
OutputIterator { input_handler: self }
}
}

borrow_mut() on my RefCell-like structure doesn't work

I try to write my own RefCell-like mutable memory location but without runtime borrow checking (no overhead). I adopted the code architecture from RefCell (and Ref, and RefMut). I can call .borrow() without problems but if I call .borrow_mut() then the rust compiler says cannot borrow as mutable. I don't see the problem, my .borrow_mut() impl looks fine?
code that fails:
let real_refcell= Rc::from(RefCell::from(MyStruct::new()));
let nooverhead_refcell = Rc::from(NORefCell::from(MyStruct::new()));
// works
let refmut_refcell = real_refcell.borrow_mut();
// cannot borrow as mutable
let refmut_norefcell = nooverhead_refcell.borrow_mut();
norc.rs (No Overhead RefCell)
use crate::norc_ref::{NORefMut, NORef};
use std::cell::UnsafeCell;
use std::borrow::Borrow;
#[derive(Debug)]
pub struct NORefCell<T: ?Sized> {
value: UnsafeCell<T>
}
impl<T> NORefCell<T> {
pub fn from(t: T) -> NORefCell<T> {
NORefCell {
value: UnsafeCell::from(t)
}
}
pub fn borrow(&self) -> NORef<'_, T> {
NORef {
value: unsafe { &*self.value.get() }
}
}
pub fn borrow_mut(&mut self) -> NORefMut<'_, T> {
NORefMut {
value: unsafe { &mut *self.value.get() }
}
}
}
norc_ref.rs (data structure returned by NORefCell.borrow[_mut]()
use std::ops::{Deref, DerefMut};
#[derive(Debug)]
pub struct NORef<'b, T: ?Sized + 'b> {
pub value: &'b T,
}
impl<T: ?Sized> Deref for NORef<'_, T> {
type Target = T;
#[inline]
fn deref(&self) -> &T {
self.value
}
}
/// No Overhead Ref Cell: Mutable Reference
#[derive(Debug)]
pub struct NORefMut<'b, T: ?Sized + 'b> {
pub value: &'b mut T,
}
impl<T: ?Sized> Deref for NORefMut<'_, T> {
type Target = T;
#[inline]
fn deref(&self) -> &T {
self.value
}
}
impl<T: ?Sized> DerefMut for NORefMut<'_, T> {
#[inline]
fn deref_mut(&mut self) -> &mut T {
self.value
}
}
NORefCell::borrow_mut() takes &mut self, which requires a DerefMut on the Rc in which it is wrapped. This won't work because Rc does not give mutable references just by asking nicely (you need it to check if the reference count is exactly one, otherwise there would be multiple mutable borrows).
borrow_mut has to take &self instead of &mut self.
As mentioned in my comment: What you are basically doing is providing a safe-looking abstraction around an UnsafeCell. This is incredibly dangerous. Notice the docs regarding UnsafeCell:
The compiler makes optimizations based on the knowledge that &T is not mutably aliased or mutated, and that &mut T is unique. UnsafeCell is the only core language feature to work around the restriction that &T may not be mutated.
You are providing a thin wrapper around this powerful object, with no unsafe on the API-boundary. The "No-overhead-RefCell" is really a "no-trigger-guard-foot-gun". It does work, yet be warned about its dangers.

What does it mean for a trait to have a lifetime parameter?

I understand how lifetime parameters apply to functions and structs, but what does it mean for a trait to have a lifetime parameter? Is it a shortcut to introduce a lifetime parameter to its methods, or is it something else?
If you have a trait with a lifetime bound, then implementors of the trait can participate in the same lifetime. Concretely, this allows you to store references with that lifetime. It is not a shortcut for specifying lifetimes on member methods, and difficulty and confusing error messages lie that way!
trait Keeper<'a> {
fn save(&mut self, v: &'a u8);
fn restore(&self) -> &'a u8;
}
struct SimpleKeeper<'a> {
val: &'a u8,
}
impl<'a> Keeper<'a> for SimpleKeeper<'a> {
fn save(&mut self, v: &'a u8) {
self.val = v
}
fn restore(&self) -> &'a u8 {
self.val
}
}
Note how both the struct and the trait are parameterized on a lifetime, and that lifetime is the same.
What would the non-trait versions of save() and restore() look like for SimpleKeeper<'a>?
Very similar, actually. The important part is that the struct stores the reference itself, so it needs to have a lifetime parameter for the values inside.
struct SimpleKeeper<'a> {
val: &'a u8,
}
impl<'a> SimpleKeeper<'a> {
fn save(&mut self, v: &'a u8) {
self.val = v
}
fn restore(&self) -> &'a u8 {
self.val
}
}
And would they mean exactly the same thing as the the trait version?
Yep!

Tying a trait lifetime variable to &self lifetime

I'd like to do something along the following lines:
trait GetRef<'a> {
fn get_ref(&self) -> &'a [u8];
}
struct Foo<'a> {
buf: &'a [u8]
}
impl <'a> GetRef<'a> for Foo<'a> {
fn get_ref(&self) -> &'a [u8] {
&self.buf[1..]
}
}
struct Bar {
buf: Vec<u8>
}
// this is the part I'm struggling with:
impl <'a> GetRef<'a> for Bar {
fn get_ref(&'a self) -> &'a [u8] {
&self.buf[1..]
}
The point of the explicit lifetime variable in the GetRef trait is to allow the return value of get_ref() on a Foo object to outlive the Foo itself, tying the return value's lifetime to that of the lifetime of Foo's buffer.
However, I haven't found a way to implement GetRef for Bar in a way that the compiler accepts. I've tried several variations of the above, but can't seem to find one that works. Is there any there any reason that this fundamentally cannot be done? If not, how can I do this?
Tying a trait lifetime variable to &self lifetime
Not possible.
Is there any there any reason that this fundamentally cannot be done?
Yes. An owning vector is something different than a borrowed slice. Your trait GetRef only makes sense for things that already represent a “loan” and don't own the slice. For an owning type like Bar you can't safely return a borrowed slice that outlives Self. That's what the borrow checker prevents to avoid dangling pointers.
What you tried to do is to link the lifetime parameter to the lifetime of Self. But the lifetime of Self is not a property of its type. It just depends on the scope this value was defined in. And that's why your approach cannot work.
Another way of looking at it is: In a trait you have to be explicit about whether Self is borrowed by a method and its result or not. You defined the GetRef trait to return something that is not linked to Self w.r.t. lifetimes. So, no borrowing. So, it's not implementable for types that own the data. You can't create a borrowed slice referring to a Vec's elements without borrowing the Vec.
If not, how can I do this?
Depends on what exactly you mean by “this”. If you want to write a “common denominator” trait that can be implemented for both borrowed and owning slices, you have to do it like this:
trait GetRef {
fn get_ref(&self) -> &[u8];
}
The meaning of this trait is that get_ref borrows Self and returns a kind of “loan” because of the current lifetime elision rules. It's equivalent to the more explicit form
trait GetRef {
fn get_ref<'s>(&self) -> &'s [u8];
}
It can be implemented for both types now:
impl<'a> GetRef for Foo<'a> {
fn get_ref(&self) -> &[u8] { &self.buf[1..] }
}
impl GetRef for Bar {
fn get_ref(&self) -> &[u8] { &self.buf[1..] }
}
You could make different lifetimes for &self and result in your trait like that:
trait GetRef<'a, 'b> {
fn get_ref(&'b self) -> &'a [u8];
}
struct Foo<'a> {
buf: &'a [u8]
}
impl <'a, 'b> GetRef<'a, 'b> for Foo<'a> {
fn get_ref(&'b self) -> &'a [u8] {
&self.buf[1..]
}
}
struct Bar {
buf: Vec<u8>
}
// Bar, however, cannot contain anything that outlives itself
impl<'a> GetRef<'a, 'a> for Bar {
fn get_ref(&'a self) -> &'a [u8] {
&self.buf[1..]
}
}
fn main() {
let a = vec!(1 as u8, 2, 3);
let b = a.clone();
let tmp;
{
let x = Foo{buf: &a};
tmp = x.get_ref();
}
{
let y = Bar{buf: b};
// Bar's buf cannot outlive Bar
// tmp = y.get_ref();
}
}

Resources