I am trying to avoid lifetimes because I still don't have a good understanding of the concept. I am reading this wonderful article and it clarifies many misunderstandings. Although I am not sure I can solve the problem.
I know how to implement iterable collection on dyn. Playground:
use std::collections::HashMap;
pub trait Enumerable {
fn elements<'a, 'b>(&'a self) -> Box<dyn Iterator<Item = (i32, &String)> + 'b>
where
'a: 'b;
}
#[derive(Debug)]
struct Container {
pub map: HashMap<i32, String>,
}
impl Enumerable for Container {
fn elements<'a, 'b>(&'a self) -> Box<dyn Iterator<Item = (i32, &String)> + 'b>
where
'a: 'b,
{
Box::new(self.map.iter().map(|el| (*el.0, el.1)))
}
}
My attempt to implement the same code without dyn. Playground:
use std::collections::HashMap;
pub trait Enumerable<'it, 'it2, It>
where
It: Iterator<Item = &'it2 (i32, &'it String)>,
'it: 'it2,
{
fn elements<'a, 'b>(&'a self) -> It
where
'a: 'b;
}
#[derive(Debug)]
struct Container {
pub map: HashMap<i32, String>,
}
impl<'it, 'it2> Enumerable<'it, 'it2, core::slice::Iter<'it, (i32, &String)>> for Container {
fn elements<'a, 'b>(&'a self) -> core::slice::Iter<'a, (i32, &'b String)>
where
'a: 'b,
{
self.map.iter().map(|el| (*el.0, el.1))
}
}
I was thinking about using impls but there is a restriction on using it in a trait. What is wrong with the code? What other useful articles can you recommend?
Apart of the fact that I don't think you need two separate lifetimes 'a and 'b, your first code example already looks quite promising.
Then, once you have only one lifetime, Rust can figure out lifetimes without any annotations:
use std::collections::HashMap;
pub trait Enumerable {
fn elements(&self) -> Box<dyn Iterator<Item = (i32, &String)> + '_>;
}
#[derive(Debug)]
struct Container {
pub map: HashMap<i32, String>,
}
impl Enumerable for Container {
fn elements(&self) -> Box<dyn Iterator<Item = (i32, &String)> + '_> {
Box::new(self.map.iter().map(|el| (*el.0, el.1)))
}
}
I know this is an XY-problem answer, but maybe it helps anyway. Your main reasoning behind not using dyn was to not deal with lifetimes, so I thought this might be relevant.
There is a solution to the original problem with GAT and TAIT which are not part of stable channgel for today.
Solution is
mod mod1 {
pub trait Enumerable {
type It<'it>: Iterator<Item = (i32, &'it String)>
where
Self: 'it;
fn elements(&self) -> Self::It<'_>;
}
}
//
impl mod1::Enumerable for Container {
type It<'it> = impl Iterator<Item = (i32, &'it String)>;
fn elements(&self) -> Self::It<'_> {
self.map.iter().map(|el| (*el.0, el.1))
}
}
Full solution
There are alternative solutions, but this one works even if the trait is not part of your crate.
Also, I should note, that if possible to avoid using lifetimes you can implement IntoIterator for your &Container:
impl< 'it > IntoIterator for &'it Container
{
type Item = ( &'it i32, &'it String );
type IntoIter = std::collections::hash_map::Iter< 'it, i32, String >;
fn into_iter( self ) -> Self::IntoIter
{
self.map.iter()
}
}
Full solution of a tweaked problem
Because the lifetime is dropped that works even on stable Rust.
Most probably you want to have your own InotIterator-like trait, especially if there is more than a single way how can you iterate your container, but if not you can simply implement standard IntoIterator for reference.
Related
I currently have code that looks kind of like this:
struct People {
names: Vec<String>,
ages: Vec<i32>,
}
impl People {
fn iter_people<'a>(&'a self) -> PeopleIterator<'a> {
return PeopleIterator {
names_iterator: Box::new(self.names.iter()),
ages: Box::new(self.ages.iter()),
};
}
}
struct PeopleIterator<'a> {
names_iterator: Box<dyn Iterator<Item = &'a String>>,
ages: Box<dyn Iterator<Item = &'a i32>>,
}
impl<'a> Iterator for PeopleIterator<'a> {
...snip...
}
I am aware that I should model a person as a struct Person and then have a Vec<Person> to model people but this is just a simplification of my actual code.
Anyway, the Rust compiler tells me this:
lifetime may not live long enough
requirement occurs because of the type PeopleIterator<'_>, which makes the generic argument '_ invariant
I have looked at the suggested link for subtyping and variance but I need to read it a few more times to actually understand it.
What stumps me is that I would expect both my iterators self.names.iter() and self.ages.iter() to live as long as self and I have declared that self should live as long as PeopleIterator. However, when I look at the iter() function, it does not make this constraint but instead has an anonymous lifetime '_. I am guessing this is the problem but I am confused and don't know how to fix it :(
The problem is the lifetime of the iterator itself in Box<dyn Iterator<Item = &'a String>> is by default bound to be 'static, but that's not possible for an iterator containing non static references like anything from &'a self. The solution is to specify an explicit lifetime bound:
struct PeopleIterator<'a> {
names_iterator: Box<dyn Iterator<Item = &'a String> + 'a>,
ages: Box<dyn Iterator<Item = &'a i32> + 'a>,
}
Personally I'd just use generics instead of static dispatch avoiding some indirection and the whole problem from the beginning:
impl People {
fn iter_people(&self) -> PeopleIterator<impl Iterator<Item = &String>, impl Iterator<Item = &i32>> {
return PeopleIterator {
names_iterator: self.names.iter(),
ages: self.ages.iter(),
};
}
}
struct PeopleIterator<N, A> {
names_iterator: N,
ages: A,
}
I'm new to rust, so forgive me if the question is naive.
I'm trying to build an OS in rust and I'm following this tutorial. The OS doesn't have memory management yet, so the goal is to build an object which is like a vector in that it can be pushed and popped etc, but it lives on the stack. We do this by initializing it with an array of fixed size. It looks like this:
#[derive(Debug)]
pub struct StackVec<'a, T: 'a> {
storage: &'a mut [T],
len: usize
}
impl<'a, T: 'a> StackVec<'a, T> {
pub fn new(storage: &'a mut [T]) -> StackVec<'a, T> {
StackVec {
storage: storage,
len: 0,
}
}
pub fn with_len(storage: &'a mut [T], len: usize) -> StackVec<'a, T> {
if len > storage.len(){
panic!();
}
StackVec{
storage: storage,
len: len
}
}
pub fn capacity(&self) -> usize {
self.storage.len()
}
pub fn into_slice(self) -> &'a mut [T] {
&mut self.storage[0..self.len]
}
// Other functions which aren't relevant for the question.
}
Popping and pushing increases and decreases the len variable and adds and removes entries from the appropriate place in the array.
Now, we also need to implement the IntoIterator trait. Given that the StackVec contains a reference to an array, I thought that I could just return an iterator from the underlying array:
impl <'a, T:'a> IntoIterator for StackVec<'a, T> {
type Item = T;
type IntoIter = core::array::IntoIter; // <- Throws "not found in `core::array"
fn into_iter(self) -> Self::IntoIter {
self.into_slice().into_iter()
}
}
But no matter how much I play around with it, it still doesn't want to compile. I can't find a way to express using types that into_itershould return the iterator for the array. What am I doing wrong?
Different problems here:
You cannot use array::IntoIterator because you do not have an array, you have a slice, which is quite different. It can be solved, for example, by using the proper core::slice::Iter as in the example.
You are trying to return T but in reality you only give access to &T, so return Item should be &T
Your into_slice method uses a &mut which is not necessary, you can reslice the storage for this implementation.
impl <'a, T:'a> IntoIterator for StackVec<'a, T> {
type Item = &'a T;
type IntoIter = core::slice::Iter<'a, T>;
fn into_iter(self) -> Self::IntoIter {
self.storage[0..self.len].into_iter()
}
}
Playground
I'm trying to create a trait that extends iterator from strings and return a flat map, here is what I got so far
trait WordsExt<'a, F>: Iterator<Item = &'a str>
where
Self: Sized,
F: FnMut(&'a str) -> Split<'a, &'a str>,
{
fn words(self) -> FlatMap<Self, Split<'a, &'a str>, F> {
self.flat_map(|x: &'a str| x.split(" "))
}
}
I'm stuck on closure type.
I tried this too
struct Words<'a, I>
where
I: Iterator<Item = &'a str>,
{
inner: I,
}
trait WordsExt<'a>: Iterator<Item = &'a str>
where
Self: Sized,
{
fn words(
self,
) -> Words<'a, FlatMap<Self, Split<'a, &'a str>, Fn(Self::Item) -> Split<'a, &'a str>>>
{
Words {
inner: self.flat_map(|x: &'a str| x.split(" ")),
}
}
}
I just came across this problem every time. I need to return a trait. When I'm in a function I can use impl Trait syntax. But when expressing a trait I can't express this. From what I could grasp, the closer I can get is to use generics, so that the method is monomorphized at call. But then I need to enable user to select the concrete type. In this case the closure is implementation detail, it should not leak to user.
I came across this answer How can I add new methods to Iterator?
In this case the OP use case has an inner state. In my case I just want to create an alias i.words() -> i.flat_map(|x| x.split(" "))
I know about coherence, so that I may need to create a wrapper over Iterator because Iterator is not under my crate. This was the second attempt.
I could implement a function by the way it was straightforward
fn words<'a>(i: impl Iterator<Item = &'a str>) -> impl Iterator<Item = &'a str> {
i.flat_map(|x| x.split(" "))
}
But I can't express the same with trait syntax because there is no impl Trait syntax in traits. So ... I have two options, dynamic dispatch and generics, none of them are ideal. Going even further. Since these struct is unsized I can't use static dispatch on it, so I'm stuck on dynamic dispatch on something that would be simply simple
Invalid struct bellow
struct Words<'a>(Iterator<Item = &'a str>);
Finally, I expected that given a function F(X) I would be able to always refactor it to write a X.F(), but this seems not to be true because while I'm able to write fn f(x) -> impl Y I'm not able to express trait X { fn (self) -> impl Y }, at last not recurring to generics or dynamic dispatch.
I was trying to write iterator combinators in a kind of fluent syntax: iterator.comb1().comb2().comb3()
I got your code to compile by changing it as follows:
trait WordsExt<'a>: Iterator<Item = &'a str>
where
Self: Sized,
{
fn words(self) -> FlatMap<Self, Split<'a, &'a str>, fn(&'a str) -> Split<'a, &'a str>> {
self.flat_map(|x: &'a str| x.split(" "))
}
}
The trick is to replace the function trait object by the actual function type.
I try to write my own RefCell-like mutable memory location but without runtime borrow checking (no overhead). I adopted the code architecture from RefCell (and Ref, and RefMut). I can call .borrow() without problems but if I call .borrow_mut() then the rust compiler says cannot borrow as mutable. I don't see the problem, my .borrow_mut() impl looks fine?
code that fails:
let real_refcell= Rc::from(RefCell::from(MyStruct::new()));
let nooverhead_refcell = Rc::from(NORefCell::from(MyStruct::new()));
// works
let refmut_refcell = real_refcell.borrow_mut();
// cannot borrow as mutable
let refmut_norefcell = nooverhead_refcell.borrow_mut();
norc.rs (No Overhead RefCell)
use crate::norc_ref::{NORefMut, NORef};
use std::cell::UnsafeCell;
use std::borrow::Borrow;
#[derive(Debug)]
pub struct NORefCell<T: ?Sized> {
value: UnsafeCell<T>
}
impl<T> NORefCell<T> {
pub fn from(t: T) -> NORefCell<T> {
NORefCell {
value: UnsafeCell::from(t)
}
}
pub fn borrow(&self) -> NORef<'_, T> {
NORef {
value: unsafe { &*self.value.get() }
}
}
pub fn borrow_mut(&mut self) -> NORefMut<'_, T> {
NORefMut {
value: unsafe { &mut *self.value.get() }
}
}
}
norc_ref.rs (data structure returned by NORefCell.borrow[_mut]()
use std::ops::{Deref, DerefMut};
#[derive(Debug)]
pub struct NORef<'b, T: ?Sized + 'b> {
pub value: &'b T,
}
impl<T: ?Sized> Deref for NORef<'_, T> {
type Target = T;
#[inline]
fn deref(&self) -> &T {
self.value
}
}
/// No Overhead Ref Cell: Mutable Reference
#[derive(Debug)]
pub struct NORefMut<'b, T: ?Sized + 'b> {
pub value: &'b mut T,
}
impl<T: ?Sized> Deref for NORefMut<'_, T> {
type Target = T;
#[inline]
fn deref(&self) -> &T {
self.value
}
}
impl<T: ?Sized> DerefMut for NORefMut<'_, T> {
#[inline]
fn deref_mut(&mut self) -> &mut T {
self.value
}
}
NORefCell::borrow_mut() takes &mut self, which requires a DerefMut on the Rc in which it is wrapped. This won't work because Rc does not give mutable references just by asking nicely (you need it to check if the reference count is exactly one, otherwise there would be multiple mutable borrows).
borrow_mut has to take &self instead of &mut self.
As mentioned in my comment: What you are basically doing is providing a safe-looking abstraction around an UnsafeCell. This is incredibly dangerous. Notice the docs regarding UnsafeCell:
The compiler makes optimizations based on the knowledge that &T is not mutably aliased or mutated, and that &mut T is unique. UnsafeCell is the only core language feature to work around the restriction that &T may not be mutated.
You are providing a thin wrapper around this powerful object, with no unsafe on the API-boundary. The "No-overhead-RefCell" is really a "no-trigger-guard-foot-gun". It does work, yet be warned about its dangers.
This is a simplified example of my code:
#[derive(Debug, Clone, Copy)]
enum Data<'a> {
I32(&'a [i32]),
F64(&'a [f64]),
}
impl<'a> From<&'a [i32]> for Data<'a> {
fn from(v: &'a [i32]) -> Data<'a> {
Data::I32(v)
}
}
impl<'a> From<&'a [f64]> for Data<'a> {
fn from(v: &'a [f64]) -> Data<'a> {
Data::F64(v)
}
}
#[derive(Debug, Clone, Copy)]
struct DataVar<'a> {
name: &'a str,
data: Data<'a>,
}
impl<'a> DataVar<'a> {
fn new<T>(name: &'a str, data: T) -> Self
where
T: Into<Data<'a>>,
{
Self {
name,
data: data.into(),
}
}
}
First of all, considering that I need to cast different DataVars to the same vector, and I would like to avoid using trait objects, do you think my implementation is correct or do you have suggestions for improvement?
Now my main question. I can define new DataVars passing a slice, for instance as follows:
let x = [1, 2, 3];
let xvar = DataVar::new("x", &x[..]);
How can I modify my constructor so that it works not only with a slice, but also with a reference to array or vector? For instance I would like the following to work as well:
let x = [1, 2, 3];
let xvar = DataVar::new("x", &x);
EDIT:
Now I tried implementing the same code using a trait object instead of an enum, but the result is even worse... isn't there really any solution to this?
trait Data: std::fmt::Debug {}
impl Data for &[i32] {}
impl Data for &[f64] {}
#[derive(Debug, Clone, Copy)]
struct DataVar<'a> {
name: &'a str,
data: &'a dyn Data,
}
impl<'a> DataVar<'a> {
fn new<T>(name: &'a str, data: &'a T) -> Self
where
T: Data,
{
Self { name, data }
}
}
let x = [1, 2, 3];
let xvar = DataVar::new("x", &&x[..]);
To me, AsRef doesn't seem to be the right abstraction for two reasons: first, because it's possible (if unlikely) for a type to implement both AsRef<[i32]> and AsRef<[f64]>, and it's not clear what should happen in that case; and second, because there's already a built-in language feature (coercion) that can turn Vec<T> or &[T; n] into &[T], and you're not taking advantage of it.
What I'd like is to write a new function that looks basically like this:
fn new<T>(name: &'a str, data: &'a [T]) -> Self
where
// what goes here?
This will automatically work with &[T; n], &Vec<T>, &Cow<T>, etc. if we can tell the compiler what to do with T. It makes sense that you could make a trait that knows how to convert &'a [Self] to Data and is implemented for i32 and f64, so let's do that:
trait Item: Sized {
fn into_data<'a>(v: &'a [Self]) -> Data<'a>;
}
impl Item for i32 {
fn into_data<'a>(v: &'a [i32]) -> Data<'a> {
Data::I32(v)
}
}
impl Item for f64 {
fn into_data<'a>(v: &'a [f64]) -> Data<'a> {
Data::F64(v)
}
}
The trait bound on new becomes trivial:
impl<'a> DataVar<'a> {
fn new<T>(name: &'a str, data: &'a [T]) -> Self
where
T: Item,
{
Self {
name,
data: T::into_data(data),
}
}
}
I find this more readable than the version with From and AsRef, but if you still want From, you can easily add it with a generic impl:
impl<'a, T> From<&'a [T]> for Data<'a>
where
T: Item,
{
fn from(v: &'a [T]) -> Self {
T::into_data(v)
}
}
We can use the AsRef trait to convert references to arrays or vectors to slices. AsRef is a generic trait, so we need to introduce a second type parameter to represent the "intermediate type" (the slice type). After calling as_ref, we've got a slice that can be converted to a Data using into.
impl<'a> DataVar<'a> {
fn new<T, U>(name: &'a str, data: &'a T) -> Self
where
T: AsRef<U> + ?Sized,
U: ?Sized + 'a,
&'a U: Into<Data<'a>>,
{
Self {
name,
data: data.as_ref().into(),
}
}
}
Note however that the data parameter is now a reference: this is necessary because the lifetime of the reference returned by as_ref is bound by the lifetime of the self parameter passed to as_ref. If we changed the parameter back to data: T, then data.as_ref() now implicitly references data in order to call as_ref, which expects a shared reference to self (&self). But data here is a local parameter, which means that the lifetime of the reference created by this implicit referencing operation is limited to the local function, and so is the reference returned by data.as_ref(). This lifetime is shorter than 'a, so we can't store it in the DataVar and return it.
If you need to handle data values that are not references in addition to values that are references, this solution cannot support that, unfortunately.
This is actually the best solution for my case:
impl<'a> DataVar<'a> {
fn new<T, U>(name: &'a str, data: &'a T) -> Self
where
T: AsRef<[U]> + ?Sized,
U: 'a,
&'a [U]: Into<Data<'a>>,
{
Self {
name,
data: data.as_ref().into(),
}
}
}
It works with slices, references to vectors, and references to arrays up to length 32 which implement AsRef<[T]> https://doc.rust-lang.org/beta/std/convert/trait.AsRef.html
Thanks #Francis for your hints!
Actually, this is IMHO the best solution... so similar to my initial code, I just needed a small fix in the new constructor:
#[derive(Debug, Clone, Copy)]
enum Data<'a> {
I32(&'a [i32]),
F64(&'a [f64]),
}
impl<'a> From<&'a [i32]> for Data<'a> {
fn from(data: &'a [i32]) -> Data<'a> {
Data::I32(data)
}
}
impl<'a> From<&'a [f64]> for Data<'a> {
fn from(data: &'a [f64]) -> Data<'a> {
Data::F64(data)
}
}
#[derive(Debug, Clone, Copy)]
struct DataVar<'a> {
name: &'a str,
data: Data<'a>,
}
impl<'a> DataVar<'a> {
fn new<T>(name: &'a str, data: &'a [T]) -> Self
where
&'a [T]: Into<Data<'a>>,
{
Self {
name,
data: data.into(),
}
}
}
#trentcl your solution is brilliant! Now I see how to leverage coercion.
However I tweaked it a little bit as follows, I will finally use this code unless you see any drawbacks in it, thanks!
#[derive(Debug, Clone, Copy)]
enum Data<'a> {
I32(&'a [i32]),
F64(&'a [f64]),
}
trait IntoData<'a>: Sized {
fn into_data(&self) -> Data<'a>;
}
impl<'a> IntoData<'a> for &'a [i32] {
fn into_data(&self) -> Data<'a> {
Data::I32(&self)
}
}
impl<'a> IntoData<'a> for &'a [f64] {
fn into_data(&self) -> Data<'a> {
Data::F64(&self)
}
}
#[derive(Debug, Clone, Copy)]
struct DataVar<'a> {
name: &'a str,
data: Data<'a>,
}
impl<'a> DataVar<'a> {
fn new<T>(name: &'a str, data: &'a [T]) -> Self
where
&'a [T]: IntoData<'a>,
{
Self {
name,
data: data.into_data(),
}
}
}