I have a lifetime issue, I'm trying to implement an iterator returning its items by reference, here is the code:
struct Foo {
d: [u8; 42],
pos: usize
}
impl<'a> Iterator<&'a u8> for Foo {
fn next<'a>(&'a mut self) -> Option<&'a u8> {
let r = self.d.get(self.pos);
if r.is_some() {
self.pos += 1;
}
r
}
}
fn main() {
let mut x = Foo {
d: [1; 42],
pos: 0
};
for i in x {
println!("{}", i);
}
}
However this code doesn't compile properly, I get an issue related to the lifetime of parameters, here is the corresponding error:
$ rustc test.rs
test.rs:8:5: 14:6 error: method `next` has an incompatible type for trait: expected concrete lifetime, but found bound lifetime parameter
test.rs:8 fn next<'a>(&'a mut self) -> Option<&'a u8> {
test.rs:9 let r = self.d.get(self.pos);
test.rs:10 if r.is_some() {
test.rs:11 self.pos += 1;
test.rs:12 }
test.rs:13 r
...
test.rs:8:49: 14:6 note: expected concrete lifetime is the lifetime 'a as defined on the block at 8:48
test.rs:8 fn next<'a>(&'a mut self) -> Option<&'a u8> {
test.rs:9 let r = self.d.get(self.pos);
test.rs:10 if r.is_some() {
test.rs:11 self.pos += 1;
test.rs:12 }
test.rs:13 r
...
error: aborting due to previous error
Does somebody has an idea how to fix this issue and still returning items by reference?
At least what does this message means: expected concrete lifetime, but found bound lifetime parameter ?
Note on the version of Rust used: at the time this question and answer were written, the Iterator trait used generics; it has changed to use associated types and is now defined thus:
pub trait Iterator {
type Item;
fn next(&mut self) -> Option<Self::Item>;
…
}
And so the incorrect implementation shown here would be like this:
impl<'a> Iterator for Foo {
type Item = &'a u8;
fn next<'a>(&'a mut self) -> Option<&'a u8>;
}
In practical terms this affects nothing; it is merely that A becomes Self::Item.
The definition of the Iterator trait is thus:
pub trait Iterator<A> {
fn next(&mut self) -> Option<A>;
…
}
Note carefully: fn next(&mut self) -> Option<A>.
Here is what you have:
impl<'a> Iterator<&'a u8> for Foo {
fn next<'a>(&'a mut self) -> Option<&'a u8>;
}
Note carefully: fn next<'a>(&'a mut self) -> Option<&'a u8>.
There are several problems here:
You have introduced a new generic parameter <'a> which should not be there. For convenience’s sake and to emphasise what has happened here, I shall dub the 'a defined on the impl block ρ₀ and the 'a defined on the method ρ₁. They are not the same.
The lifetime of &mut self is different from that of the trait.
The lifetime of the return type is different to the trait: where A is &'ρ₀ u8, the return type uses in the place of A &'ρ₁ u8. It expected the concrete lifetime ρ₀ but found instead the lifetime ρ₁. (I’m not certain precisely what the “bound” bit means, so I’ll keep quiet on it lest I be wrong.)
Here’s what this amounts to: you cannot connect the lifetime of the object you are iterating over to &mut self. Instead, it must be bound to something in the type you are implementing the trait for. To take an example, iterating over items in a slice is done by creating a new iterator object connected to the base slice, impl<'a, T> Iterator<&'a T> for Items<'a, T>. Expressed in another way, the way the iteration traits are designed is not, if you are producing references, for you to return something inside self, but rather to return something inside another object that you have a reference to.
For your specific, presumably simple example, you should either stop yielding references, or alter it so that your iterator object does not contain the data that you are iterating over—let it merely contain a reference to it, e.g. &'a [T] or even something like Items<'a, T>.
Related
I'm having difficulties understanding lifetime parameters in the following code snippet.
struct C {
data: Vec<u32>,
cols: usize
}
trait M<'s> {
fn get(&'s self, r: usize, c: usize) -> u32;
fn get_mut(&'s mut self, r: usize, c: usize) -> &'s mut u32;
}
impl<'s> M<'s> for C {
fn get(&'s self, r: usize, c: usize) -> u32 {
return self.data[self.cols*r+c];
}
fn get_mut(&'s mut self, r: usize, c: usize) -> &'s mut u32 {
return &mut self.data[self.cols*r+c];
}
}
#[cfg(test)]
mod tests {
use super::*;
fn create() -> C {
let data = vec![0u32,1u32,2u32,3u32,4u32,5u32];
return C{data, cols: 3};
}
fn select<'s, 'r: 's>(data: &'r mut dyn M<'s>) {
let mut _val: u32 = 0;
for r in 0..2 {
for c in 0..3 {
_val += *data.get_mut(r,c);
}
}
}
#[test]
fn test_select() {
let mut data = create();
select(&mut data);
}
}
The code snippet does not compile, because it complains that *data is borrowed multiple times in the function fn select<'s, 'r: 's>(data: &'r mut dyn M<'s>) {} when calling get_mut (once in every loop iteration). Even safeguarding the questionable line with curly braces (and thus creating a new context) does not help. My expectation (in both cases) would be, that the mutable borrow of &mut data should end right after the execution of that line.
On the other hand, when I remove all lifetime parameters, everything works as expected.
Can anyone explain what's the difference between the two versions (with and without explicit lifetimes)?
I've also tried to find information about additional lifetime parameters for traits, in particular specifying their meaning, but I have found none. So I assume, that they are just a declaration of the used labels inside the trait. But if that is so, then I would assume that leaving out the lifetime parameters completely and applying the eliding rules would lead to the same result.
There are two things to consider. The first is when you use a generic lifetime for a function, that lifetime must be larger than the life of the function call simply by construction. And the second is since the lifetime self is tied to the lifetime parameter of the trait, when you call .get_mut(), data is borrowed for the lifetime of 's. Combining those two principles, data is borrowed for longer than the function call so you can't call it again (its already mutably borrowed).
On the other hand, when I remove all lifetime parameters, everything works as expected. Can anyone explain what's the difference between the two versions (with and without explicit lifetimes)?
Without a generic lifetime on M, the methods will behave as if defined as so:
impl M for C {
fn get<'a>(&'a self, r: usize, c: usize) -> u32 {
return self.data[self.cols * r + c];
}
fn get_mut<'a>(&'a mut self, r: usize, c: usize) -> &'a mut u32 {
return &mut self.data[self.cols * r + c];
}
}
Thus there is no lifetime associated with the trait; the lifetimes given and returned from the function are generic only to those method calls. And since the compiler can choose a new lifetime 'a for each call and it will always pick the shorted lifetime to satisfy its usage, you can then call data.get_mut() multiple times without worry. And I'll be honest, having the lifetime on the trait didn't make much sense with the original code; as mentioned, the code works with all lifetime annotations removed: playground.
I am having trouble figuring out what lifetime parameter will work for this, so my current workarounds include transmutes or raw pointers. I have a structure holding a function pointer with a generic as a parameter:
struct CB<Data> {
cb: fn(Data) -> usize
}
I would like to store an instance of that, parameterized by some type containing a reference, in some other structure that implements a trait with one method, and use that trait method to call the function pointer in CB.
struct Holder<'a> {
c: CB<Option<&'a usize>>
}
trait Exec {
fn exec(&self, v: &usize) -> usize;
}
impl<'a> Holder<'a> {
fn exec_aux(&self, v: &'a usize) -> usize {
(self.c.cb)(Some(v))
}
}
impl<'a> Exec for Holder<'a> {
fn exec(&self, v: &usize) -> usize
{
self.exec_aux(v)
}
}
This gives me a lifetime error for the 'Exec' impl of Holder:
error[E0495]: cannot infer an appropriate lifetime for lifetime parameter `'a` due to conflicting requirements
Simply calling exec_aux works fine as long as I don't define that Exec impl:
fn main() {
let h = Holder { c: CB{cb:cbf}};
let v = 12;
println!("{}", h.exec_aux(&v));
}
Also, making CB not generic also makes this work:
struct CB {
cb: fn(Option<&usize>) -> usize
}
The parameter in my actual code is not a usize but something big that I would rather not copy.
The lifetimes in your Exec trait are implicitly this:
trait Exec {
fn exec<'s, 'a>(&'s self, v: &'a usize) -> usize;
}
In other words, types that implement Exec need to accept any lifetimes 's and 'a. However, your Holder::exec_aux method expects a specific lifetime 'a that's tied to the lifetime parameter of the Holder type.
To make this work, you need to add 'a as a lifetime parameter to the Exec trait instead, so that you can implement the trait specifically for that lifetime:
trait Exec<'a> {
// ^^^^ vv
fn exec(&self, v: &'a usize) -> usize;
}
impl<'a> Exec<'a> for Holder<'a> {
// ^^^^ vv
fn exec(&self, v: &'a usize) -> usize
{
self.exec_aux(v)
}
}
The problem here is that the Exec trait is too generic to be used in this way by Holder. First, consider the definition:
trait Exec {
fn exec(&self, v: &usize) -> usize;
}
This definition will cause the compiler to automatically assign two anonymous lifetimes for &self and &v in exec. It's basically the same as
fn exec<'a, 'b>(&'a self, v: &'b usize) -> usize;
Note that there is no restriction on who needs to outlive whom, the references just need to be alive for the duration of the method call.
Now consider the definition
impl<'a> Holder<'a> {
fn exec_aux(&self, v: &'a usize) -> usize {
// ... doesn't matter
}
}
Since we know that &self is a &Holder<'a> (this is what the impl refers to), we need to have at least a &'a Holder<'a> here, because &'_ self can't have a lifetime shorter than 'a in Holder<'a>. So this is saying that the two parameters have the same lifetime: &'a self, &'a usize.
Where it all goes wrong is when you try to combine the two. The trait forces you into the following signature, which (again) has two distinct implicit lifetimes. But the actual Holder which you then try to call a method on forces you to have the same lifetimes for &self and &v.
fn exec(&self, v: &usize) -> usize {
// Holder<'a> needs `v` to be `'a` when calling exec_aux
// But the trait doesn't say so.
self.exec_aux(v)
}
One solution is to redefine the trait as
trait Exec<'a> {
fn exec(&'a self, v: &'a usize) -> usize;
}
and then implement it as
impl<'a> Exec<'a> for Holder<'a> {
fn exec(&'a self, v: &'a usize) -> usize {
self.exec_aux(v)
}
}
An easily overlooked feature of clone() is that it can shorten the lifetimes of any references hidden inside the value being cloned. This is usually useless for immutable references, which are the only kind for which Clone is implemented.
It would, however, be useful to be able to shorten the lifetimes of mutable references hidden inside a value. Is there something like a CloneMut trait?
I've managed to write one. My question is whether there is a trait in the standard library that I should use instead, i.e. am I reinventing the wheel?
The rest of this question consists of details and examples.
Playground.
Special case: the type is a mutable reference
As a warm-up, the following is good enough when the type you're cloning is a mutable reference, not wrapped in any way:
fn clone_mut<'a, 'b: 'a>(q: &'a mut &'b mut f32) -> &'a mut f32 {
*q
}
See this question (where it is called reborrow()) for an example caller.
Special case: the reference type, though user-defined, is known
A more interesting case is a user-defined mutable-reference-like type. Here's how to write a clone_mut() function specific to a particular type:
struct Foo<'a>(&'a mut f32);
impl<'b> Foo<'b> {
fn clone_mut<'a>(self: &'a mut Foo<'b>) -> Foo<'a> {
Foo(self.0)
}
}
Here's an example caller:
fn main() {
let mut x: f32 = 3.142;
let mut p = Foo(&mut x);
{
let q = p.clone_mut();
*q.0 = 2.718;
}
println!("{:?}", *p.0)
}
Note that this won't compile unless q gets a shorter lifetime than p. I'd like to view that as a unit test for clone_mut().
Higher-kinded type?
When trying to write a trait that admits both the above implementations, the problem at first feels like a higher-kinded-type problem. For example, I want to write this:
trait CloneMut {
fn clone_mut<'a, 'b>(self: &'a mut Self<'b>) -> Self<'a>;
}
impl CloneMut for Foo {
fn clone_mut<'a, 'b>(self: &'a mut Self<'b>) -> Self<'a> {
Foo(self.0)
}
}
Of course that's not allowed in Rust (the Self<'a> and Self<'b> parts in particular). However, the problem can be worked around.
General case
The following code compiles (using the preceding definition of Foo<'a>) and is compatible with the caller:
trait CloneMut<'a> {
type To: 'a;
fn clone_mut(&'a mut self) -> Self::To;
}
impl<'a, 'b> CloneMut<'a> for Foo<'b> {
type To = Foo<'a>;
fn clone_mut(&'a mut self) -> Self::To {
Foo(self.0)
}
}
It's a little ugly that there is no formal relationship between Self and Self::To. For example, you could write an implementation of clone_mut() that returns 77, completely ignoring the Self type. The following two attempts show why I think the associated type is unavoidable.
Attempt 1
This compiles:
trait CloneMut<'a> {
fn clone_mut(&'a mut self) -> Self;
}
impl<'a> CloneMut<'a> for Foo<'a> {
fn clone_mut(&'a mut self) -> Self {
Foo(self.0)
}
}
However, it's not compatible with the caller, because it does not have two distinct lifetime variables.
error[E0502]: cannot borrow `*p.0` as immutable because `p` is also borrowed as mutable
The immutable borrow mentioned in the error message is the one in the println!() statement, and the mutable borrow is the call to clone_mut(). The trait constrains the two lifetimes to be the same.
Attempt 2
This uses the same trait definition as attempt 1, but a different implementation:
trait CloneMut<'a> {
fn clone_mut(&'a mut self) -> Self;
}
impl<'a, 'b: 'a> CloneMut<'a> for Foo<'b> {
fn clone_mut(&'a mut self) -> Self {
Foo(self.0)
}
}
This doesn't even compile. The return type has the longer lifetime, and can't be made from the argument, which has the shorter lifetime.
Moving the lifetime parameter onto the method declaration gives the same error:
trait CloneMut {
fn clone_mut<'a>(&'a mut self) -> Self;
}
impl<'b> CloneMut for Foo<'b> {
fn clone_mut<'a>(&'a mut self) -> Self {
Foo(self.0)
}
}
Relationship with Clone
Incidentally, notice that CloneMut<'a, To=Self> is strictly stronger than Clone:
impl<'a, T: 'a> CloneMut<'a> for T where T: Clone {
type To = Self;
fn clone_mut(&'a mut self) -> Self {
self.clone()
}
}
That's why I think "CloneMut" is a good name.
The key property of &mut references is that they are unique exclusive references.
So it's not really a clone. You can't have two exclusive references. It's a reborrow, as the source will be completely unusable as long as the "clone" is in scope.
Background: I'm creating an iterator that returns a reference to a slice &[T], but the data vector needs to remain immutable. The iterator cannot modify the original data, but must return the same slice pointer repeatedly after it has modified it. I've considered having my iterator own a Vec<T>, but I would like to avoid that (and it didn't seem to work). I avoid allocations since I plan to use it mostly in realtime audio and allocations could potentially block. Code:
pub struct Windower<'a, 'b, T: 'a + 'b> {
window_type: WindowType,
hop_size: usize,
bin_size: usize,
current_index: usize,
data: &'a [T],
out_data: &'b mut [T]
}
impl<'a, 'b, T: Float + FromPrimitive> Iterator for Windower<'a, 'b, T> {
type Item = &'b [T];
fn next(&mut self) -> Option<Self::Item> {
if self.current_index < (self.len() - 1) {
let start = self.current_index * self.hop_size;
let end = start + self.bin_size;
self.current_index += 1;
let window = self.window();
let data_iter = self.data[start..end].iter();
for &mut v in self.out_data {
let val: T = window.next().unwrap() *
*data_iter.next().unwrap();
v = val;
}
Some(self.out_data)
} else {
None
}
}
}
Returns the error:
src/waves.rs:160:18: 160:31 error: cannot infer an appropriate lifetime for automatic coercion due to conflicting requirements [E0495]
src/waves.rs:160 Some(self.out_data)
^~~~~~~~~~~~~
src/waves.rs:146:5: 164:6 help: consider using an explicit lifetime parameter as shown: fn next(&'b mut self) -> Option<Self::Item>
I cannot figure out how to fix this. I can't make the change suggested, because the trait implementation for Iterator does not have an explicit lifetime parameter.
Rust prevents you from having more than one alias to an object if one of them is a mutable alias.
Here, Windower::out_data is a mutable alias to some slice, and you're trying to return an immutable alias to the same data from your next method. In order for this to be safe, Rust must prevent you from being able to use Windower::out_data for as long as the slice returned by next is in scope. This means that the signature fn next(&'b mut self) -> Option<Self::Item> is indeed required, which means you simply cannot implement Iterator with your current implementation.
Broadly speaking my goal is this:
For some known type Bar...
Have a trait Foo with a function: get_iterator<T>() -> T where T: Iterator<Item = Bar>
The instance of the iterator borrows the original object Foo is implemented on.
I imagine it working like this:
let mut foo = Foo;
let bar = foo.get_iterator();
foo.mutable_call(); // <-- This fails, because foo is borrowed in bar
for x in bar {
...
}
So, that's the goal, and here's my attempt, which I can't seem to get working:
struct ValsFromT<'a, T: 'a> {
parent:&'a T,
offset: usize,
}
struct Val;
trait HasValsIterator<T> {
fn val_iterator(&self) -> T where T: Iterator<Item = Val>;
}
struct Foo;
impl<'a> Iterator for ValsFromT<'a, Foo> {
type Item = Val;
fn next(&mut self) -> Option<Val> {
return None;
}
}
impl<'a> HasValsIterator<ValsFromT<'a, Foo>> for Foo {
fn val_iterator(&'a self) -> ValsFromT<'a, Foo> {
return ValsFromT {
offset: 0usize,
parent: self
};
}
}
fn takes_vals<T>(instance:T) where T: HasValsIterator<T> {
// ...
}
#[test]
fn test_foo() {
let x = Foo;
takes_vals(x);
}
(playpen: http://is.gd/wys3fx)
We're getting the dreaded concrete/bound lifetime error here, because of trying to return an iterator instance that references self from the trait function:
<anon>:22:3: 27:4 error: method `val_iterator` has an incompatible type for trait:
expected bound lifetime parameter ,
found concrete lifetime [E0053]
<anon>:22 fn val_iterator(&'a self) -> ValsFromT<'a, Foo> {
<anon>:23 return ValsFromT {
<anon>:24 offset: 0usize,
<anon>:25 parent: self
<anon>:26 };
<anon>:27 }
<anon>:22:3: 27:4 help: see the detailed explanation for E0053
Is there some way of doing this?
Unfortunately, Veedrac's suggestion doesn't work directly. You will get the following error if you'd try to use val_iterator() method on instance inside takes_vals():
<anon>:31:25: 31:39 error: the trait `core::iter::Iterator` is not implemented for the type `U` [E0277]
<anon>:31 let iter = instance.val_iterator();
^~~~~~~~~~~~~~
<anon>:31:25: 31:39 help: see the detailed explanation for E0277
<anon>:31:25: 31:39 note: `U` is not an iterator; maybe try calling `.iter()` or a similar method
error: aborting due to previous error
playpen: application terminated with error code 101
This (and some other further errors) requires changing the signature of the function to this one:
fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U>
However, even this doesn't work yet:
<anon>:31:16: 31:24 error: `instance` does not live long enough
<anon>:31 let iter = instance.val_iterator();
^~~~~~~~
<anon>:30:97: 32:2 note: reference must be valid for the lifetime 'a as defined on the block at 30:96...
<anon>:30 fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U> {
<anon>:31 let iter = instance.val_iterator();
<anon>:32 }
<anon>:30:97: 32:2 note: ...but borrowed value is only valid for the scope of parameters for function at 30:96
<anon>:30 fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: T) where T: HasValsIterator<'a, U> {
<anon>:31 let iter = instance.val_iterator();
<anon>:32 }
Remember that the trait requires that val_iterator() accepts the target by reference with lifetime 'a. This lifetime in this function is an input parameter. However, when val_iterator() is called on instance, the only lifetime which can be specified for the reference is the one of instance which is strictly smaller than any possible 'a as a parameter, because it is a local variable. Therefore, it is not possible to pass instance by value; you can only pass it by reference for lifetimes to match:
fn takes_vals<'a, T: 'a, U: Iterator<Item=Val>+'a>(instance: &'a T) where T: HasValsIterator<'a, U> {
let iter = instance.val_iterator();
}
This works.
I'd like to add that using associated types instead of type parameters would be more correct semantically:
trait HasValsIterator<'a> {
type Iter: Iterator<Item=Val> + 'a;
fn val_iterator(&'a self) -> Self::Iter;
}
impl<'a> HasValsIterator<'a> for Foo {
type Iter = ValsFromT<'a, Foo>;
fn val_iterator(&'a self) -> ValsFromT<'a, Foo> { ... }
}
fn takes_vals<'a, T: 'a>(instance: &'a T) where T: HasValsIterator<'a> {
...
}
I say that this is more correct because the type of the iterator is determined by the implementor, that is, it is "output" type, which are modeled by associated types. As you can see, takes_vals() signature also shrank considerably.
Ideally, HasValsIterator trait should have been defined like this:
trait HasValsIterator {
type Iter<'a>: Iterator<Item=Val> + 'a
fn val_iterator<'a>(&'a self) -> Iter<'a>;
}
This way, val_iterator() would in any situation, including when HasValsIterator implementor is passed by value. However, Rust is not there yet.