I'm running into an issue with trait bounds and can't understand what I'm doing wrong. I'm working with the arduino-uno crate from avr-hal and I have a function that reads the ADC, implemented as follows:
fn read_signal<T: avr_hal_generic::hal::adc::Channel<board::adc::Adc, ID = u8>>(
adc: &mut board::adc::Adc,
pinA0: &mut T,
) {
let x = adc.read(&mut pinA0);
}
However, I receive the following error:
the trait bound `&mut T: avr_hal_generic::embedded_hal::adc::Channel<arduino_uno::adc::Adc>` is not satisfied
required because of the requirements on the impl of `arduino_uno::prelude::_embedded_hal_adc_OneShot<arduino_uno::adc::Adc, _, &mut T>` for `arduino_uno::adc::Adc` rustc(E0277)
I have tried using where instead, but that doesn't help. How can I fix this?
Also, though I show this specific example, I would really appreciate an explanation or pointers to better documentation on this subject/error as I've encountered it a few times and struggle to understand what is wrong.
The compiler error says:
the trait bound &mut T: avr_hal_generic::embedded_hal::adc::Channel<arduino_uno::adc::Adc> is not satisfied
Notice that the error is asking for &mut T to implement Channel, &mut T: Channel<...>, whereas your T has the bound T: Channel<...> — applying to T itself rather than &mut T. That's why the bound you've already written isn't helping.
Now, what's the right fix? If I look at the docs you linked, I can find the type of <Adc as OneShot>::read. (Note: I copied the text from the docs to construct this snippet; I didn't read the source code.)
impl<WORD, PIN> OneShot<Adc, WORD, PIN> for Adc
where
WORD: From<u16>,
PIN: Channel<Adc, ID = MUX_A>, // <---- look here
{
pub fn read(
&mut self,
_pin: &mut PIN // <---- look here
) -> Result<WORD, ...>
}
So, read should be given an &mut PIN and the type variable PIN should implement Channel. That all sounds reasonable, but in your code the compiler thinks we want &mut T to implement Channel. An extra &mut has appeared. Where did it come from? You wrote:
fn read_signal<T: avr_hal_generic::hal::adc::Channel<board::adc::Adc, ID = u8>>(
adc: &mut board::adc::Adc,
pinA0: &mut T,
) {
let x = adc.read(&mut pinA0);
}
pinA0 is of type &mut T, but you then wrote adc.read(&mut pinA0), which means the parameter to read() is of type &mut &mut T. Since read wants &mut of something implementing Channel, this results in the error you saw, asking for &mut T: Channel.
The fix, then, is to not take &mut of &mut:
let x = adc.read(pinA0);
Under some circumstances, Rust provides implicit coercions which will, in particular, turn a reference to a reference (or a reference to a 'smart pointer' type implementing Deref) into a reference. This is why you might have had extraneous & or &mut work in the past. However, these coercions are not applied when there isn't a single concrete type that's expected, such as in this case where the function parameter's type contains the type variable T.
Related
I'm having trouble understanding how values of boxed traits come into existence. Consider the following code:
trait Fooer {
fn foo(&self);
}
impl Fooer for i32 {
fn foo(&self) { println!("Fooer on i32!"); }
}
fn main() {
let a = Box::new(32); // works, creates a Box<i32>
let b = Box::<i32>::new(32); // works, creates a Box<i32>
let c = Box::<dyn Fooer>::new(32); // doesn't work
let d: Box<dyn Fooer> = Box::new(32); // works, creates a Box<Fooer>
let e: Box<dyn Fooer> = Box::<i32>::new(32); // works, creates a Box<Fooer>
}
Obviously, variant a and b work, trivially. However, variant c does not, probably because the new function takes only values of the same type which is not the case since Fooer != i32. Variant d and e work, which lets me suspect that some kind of automatic conversion from Box<i32> to Box<dyn Fooer> is being performed.
So my questions are:
Does some kind of conversion happen here?
If so, what the mechanism behind it and how does it work? (I'm also interested in the low level details, i.e. how stuff is represented under the hood)
Is there a way to create a Box<dyn Fooer> directly from an i32? If not: why not?
However, variant c does not, probably because the new function takes only values of the same type which is not the case since Fooer != i32.
No, it's because there is no new function for Box<dyn Fooer>. In the documentation:
impl<T> Box<T>
pub fn new(x: T) -> Box<T>
Most methods on Box<T> allow T: ?Sized, but new is defined in an impl without a T: ?Sized bound. That means you can only call Box::<T>::new when T is a type with a known size. dyn Fooer is unsized, so there simply isn't a new function to call.
In fact, that function can't exist in today's Rust. Box<T>::new needs to know the concrete type T so that it can allocate memory of the right size and alignment. Therefore, you can't erase T before you send it to Box::new. (It's conceivable that future language extensions may allow functions to accept unsized parameters; however, it's unclear whether even unsized_locals would actually enable Box<T>::new to accept unsized T.)
For the time being, unsized types like dyn Fooer can only exist behind a "fat pointer", that is, a pointer to the object and a pointer to the implementation of Fooer for that object. How do you get a fat pointer? You start with a thin pointer and coerce it. That's what's happening in these two lines:
let d: Box<Fooer> = Box::new(32); // works, creates a Box<Fooer>
let e: Box<Fooer> = Box::<i32>::new(32); // works, creates a Box<Fooer>
Box::new returns a Box<i32>, which is then coerced to Box<Fooer>. You could consider this a conversion, but the Box isn't changed; all the compiler does is stick an extra pointer on it and forget its original type. rodrigo's answer goes into more detail about the language-level mechanics of this coercion.
Hopefully all of this goes to explain why the answer to
Is there a way to create a Box<Fooer> directly from an i32?
is "no": the i32 has to be boxed before you can erase its type. It's the same reason you can't write let x: Fooer = 10i32.
Related
Why can't I write a function with the same type as Box::new?
Are polymorphic variables allowed?
How do you actually use dynamically sized types in Rust?
Why is `let ref a: Trait = Struct` forbidden?
I'll try to explain what conversions (coercions) happen in your code.
There is a marker trait named Unsize that, between others:
Unsize is implemented for:
T is Unsize<Trait> when T: Trait.
[...]
This trait, AFAIK, is not used directly for coercions. Instead, CoerceUnsized is used. This trait is implemented in a lot of cases, some of them are quite expected, such as:
impl<'a, 'b, T, U> CoerceUnsized<&'a U> for &'b T
where
'b: 'a,
T: Unsize<U> + ?Sized,
U: ?Sized
that is used to coerce &i32 into &Fooer.
The interesting, not so obvious implementation for this trait, that affects your code is:
impl<T, U> CoerceUnsized<Box<U>> for Box<T>
where
T: Unsize<U> + ?Sized,
U: ?Sized
This, together with the definition of the Unsize marker, can be somewhat read as: if U is a trait and T implements U, then Box<T> can be coerced into Box<U>.
About your last question:
Is there a way to create a Box<Fooer> directly from an i32? If not: why not?
Not that I know of. The problem is that Box::new(T) requires a sized value, since the value passed is moved into the box, and unsized values cannot be moved.
In my opinion, the easiest way to do that is to simply write:
let c = Box::new(42) as Box<Fooer>;
That is, you create a Box of the proper type and then coerce to the unsized one (note it looks quite similar to your d example).
I'm trying to have a MyType that supports a From<&[u8]> trait, but I'm running into "lifetime problems":
Here's a minimally viable example:
struct MyType {
i: i32
}
impl MyType {
fn from_bytes(_buf: &[u8]) -> MyType {
// for example...
MyType { i: 3 }
}
}
impl From<&[u8]> for MyType {
fn from(bytes: &[u8]) -> Self {
MyType::from_bytes(bytes)
}
}
fn do_smth<T>() -> T where T: From<&[u8]>
{
// for example...
let buf : Vec<u8> = vec![1u8, 2u8];
T::from(buf.as_slice())
}
(...and here's a Rust playground link)
For reasons I cannot understand, the Rust compiler is telling me:
error[E0637]: `&` without an explicit lifetime name cannot be used here
--> src/lib.rs:17:36
|
17 | fn do_smth<T>() -> T where T: From<&[u8]>
| ^ explicit lifetime name needed here
I'm not an expert on lifetimes and I don't understand why this piece of code needs one. What would be the best way to fix this?
Might Rust be thinking that the type T could be a &[u8] itself? But, in that case, the lifetime should be inferred to be the same as the input to From::<&[u8]>::from(), no?
One fix I was given was to do:
fn do_smth<T>() -> T where for<'a> T: From<&'a [u8]>
{
// for example...
let buf : Vec<u8> = vec![1u8, 2u8];
T::from(buf.as_slice())
}
...but I do not understand this fix, nor do I understand why lifetimes are needed in the first place.
Rust first wants you to write:
fn do_smth<'a, T>() -> T
where
T: From<&'a [u8]>,
{
// for example...
let buf: Vec<u8> = vec![1u8, 2u8];
T::from(&buf)
}
where you make explicit that this function can be called for any lifetime 'a and any type T such that T implements From<&'a [u8]>.
But Rust then complains:
error[E0597]: `buf` does not live long enough
--> src/lib.rs:24:13
|
18 | fn do_smth<'a, T>() -> T
| -- lifetime `'a` defined here
...
24 | T::from(&buf)
| --------^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `buf` is borrowed for `'a`
25 | }
| - `buf` dropped here while still borrowed
You promised that this function could work with any lifetime, but this turns out to not be true, because in the body of the function you create a fresh reference to the local Vec which has a different lifetime, say 'local. Your function only works when 'a equals 'local, but you promise that it also works for all other lifetimes. What you need is a way to express that these lifetimes are the same, and the only way I think that is possible is by changing the local reference to an argument:
fn do_smth<'a, T>(buf: &'a [u8]) -> T
where
T: From<&'a [u8]>,
{
T::from(buf)
}
And then it compiles.
If instead of the function promising it can work with any lifetime, you want to make the caller promise it can work with any lifetime, you can instead use HRTBs to make the caller promise it.
fn do_smth<T>() -> T
where
for<'a> T: From<&'a [u8]>,
{
// for example...
let buf: Vec<u8> = vec![1u8, 2u8];
T::from(&buf)
}
Now, since you can use any lifetime, a local one also works and the code compiles.
Lifetimes represent a "duration" (metaphorically), or, more pragmatically, a scope, in which a variable is valid. Outside of one's lifetime, the variable should be considered as having been freed from memory, even though you haven't done that explicitly, because that's how Rust manages memory.
It becomes a bit more complex when Rust tries to ensure that, once a variable is done for, no other parts of the code that could have had access to that variable still have access. These shared accesses are called borrows, and that's why borrows have lifetimes too. The main condition Rust enforces on them is that a borrow's lifetime is always shorter (or within, depending on how you see it) than its original variable, ie. you can't share something for more time than you actually own it.
Rust therefore enforces all borrows (as well as all variables, really) to have an established lifetime at compile-time. To lighten things, Rust has default rules about what a lifetime should be if it was not explicitly defined by the user, that is, when you talk about a type that involves a lifetime, Rust let's you not write that lifetime explicitly under certain conditions. However, this is not a "lifetime inference", in the sense of inferring types: Rust will not try to make sense out of explicit lifetimes, it's a lot less smart about it. In particular, this lifetime explicitation can fail, in the sense that Rust will not be able to figure out the right lifetime it has to assign even though it was possible to find out that worked.
Back to business: your first error simply stems from the fact that Rust has no rule to make a lifetime if it wasn't provided in the position pointed out by the error. As I said, Rust won't try to infer what the right lifetime would be, it simply checks whether not explicitly putting a lifetime there implicitly means something or not. So, you simply need to put one.
Your first reflex might be to make your function generic over the missing lifetime, which is often the right thing to do (and even the only possible action), that is, do something like that:
fn do_smth<'a, T>() -> T
where
T: From<&'a [u8]>
{
// for example...
let buf : Vec<u8> = vec![1, 2];
T::from(buf.as_slice())
}
What this means is that do_smth is generic over the lifetime 'a, just like it is generic over the type T. This has two consequences:
Rust will proceed to a monomorphisation of your function for each call, meaning it will actually provide a concrete implementation of your function for each type T and each lifetime 'a that is required. In particular, it will automatically find out what is the right lifetime. This might seem contradictory with what I said earlier, about Rust not inferring lifetimes. The difference is that type inference and monomorphisation, although similar, are not the same step, and so the compiler does not work lifetimes in the same way. Don't worry about this until you have understood the rest.
The second consequence, which is a bit disastrous, is that your function exposes the following contract: for any type T, and any lifetime 'a, such that T: From<&'a [u8]>, do_smth can produce a type T. If you think about it, it means that even if T only implements From<&'a [u8]> for a lifetime 'a that is already finished (or, if you see lifetimes as scopes, for a lifetime 'a that is disjoint from do_smth's scope), you can produce an element of type T. This is not what you actually meant: you don't want the caller to give you an arbitrary lifetime. Instead, you know that the lifetime of the borrow of the slice is the one you chose it to be, within your function (because you own the underlying vector), and you want that the type T to be buildable from that slice. That is, you want T: From<&'a [u8]> for a 'a that you have chosen, not one provided by the caller.
This last point should make you understand why the previous snippet of code is unsound, and won't compile. Your function should not take a lifetime as argument, just a type T with certain constraints. But then, how do you encode the said conditions? That's where for<'a> comes into play. If you have a type T such that T: for<'a> From<&'a [u8]>, it means that for all 'a, T: From<&'a [u8]>. In particular, it is true for the lifetime of your slice. This is why the following works
fn do_smth<T>() -> T
where
T: for<'a> From<&'a [u8]>
{
// for example...
let buf: Vec<u8> = vec![1, 2];
T::from(buf.as_slice())
}
Note that, as planned, this version of do_smth is not generic over a lifetime, that is, the caller does not provide a lifetime to the function.
Considering the following code:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || &t)
}
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
t moves to the closure, so the closure live as long as t
The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
There is no lifetime problem, the code compiles.
What actually happens:
The code does not compile:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 2:14...
--> src/lib.rs:2:14
|
2 | Box::new(move || &t)
| ^^^^^^^^^^
note: ...so that closure can access `t`
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...
--> src/lib.rs:1:8
|
1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
| ^^
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>
found std::boxed::Box<dyn std::ops::Fn() -> &T>
I do not understand the conflict. How can I fix it?
Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.
tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).
Implementing it manually
As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.
What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.
struct Baz<T>(T);
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
fn make_baz<T>(t: T) -> Box<Baz<T>> {
Box::new(Baz(t))
}
This is pretty equivalent to your boxed closure. Let's try to use it:
let outside = {
let s = "hi".to_string();
let baz = make_baz(s);
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // works too
This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.
It gets more interesting when we add a single character:
let outside = {
let s = "hi".to_string();
let baz = make_baz(&s); // <-- NOW BORROWED!
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // doesn't work!
Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.
The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.
Writing a trait for it
So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:
trait MyFn {
type Output;
fn call(&self) -> Self::Output;
}
In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.
Let's implement it!
impl<T> MyFn for Baz<T> {
type Output = ???;
fn call(&self) -> Self::Output {
&self.0
}
}
Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?
Let's check the function we implemented before:
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:
fn call<'s>(&'s self) -> &'s T
Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).
(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)
But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.
What about the future?
Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).
Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:
trait MyFn {
type Output<'a>; // <-- we added <'a> to make it generic
fn call(&self) -> Self::Output;
}
The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:
fn call<'s>(&'s self) -> Self::Output<'s>
So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:
impl<T> MyFn for Baz<T> {
type Output<'a> = &'a T;
fn call(&self) -> Self::Output {
&self.0
}
}
To return a boxed MyFn we would need to write this (according to this section of the RFC:
fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {
Box::new(Baz(t))
}
And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)
Summary
What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.
With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
This makes no sense. A value cannot "live as long" as a type, because a type doesn't live. "T has lifetime 'a" is a very imprecise statement, easy to misunderstand. What T: 'a really means is "instances of T must stay valid at least as long as lifetime 'a. For example, T must not be a reference with a lifetime shorter than 'a, or a struct containing such a reference. Note that this has nothing to do with forming references to T, i.e. &T.
The value t, then, lives as long as its lexical scope (it's a function parameter) says it does, which has nothing to do with 'a at all.
t moves to the closure, so the closure live as long as t
This is also incorrect. The closure lives as long as the closure does lexically. It is a temporary in the result expression, and therefore lives until the end of the result expression. t's lifetime concerns the closure not at all, since it has its own T variable inside, the capture of t. Since the capture is a copy/move of t, it is not in any way affected by t's lifetime.
The temporary closure is then moved into the box's storage, but that's a new object with its own lifetime. The lifetime of that closure is bound to the lifetime of the box, i.e. it is the return value of the function, and later (if you store the box outside the function) the lifetime of whatever variable you store the box in.
All of that means that a closure that returns a reference to its own capture state must bind the lifetime of that reference to its own reference. Unfortunately, this is not possible.
Here's why:
The Fn trait implies the FnMut trait, which in turn implies the FnOnce trait. That is, every function object in Rust can be called with a by-value self argument. This means that every function object must be still valid being called with a by-value self argument and returning the same thing as always.
In other words, trying to write a closure that returns a reference to its own captures expands to roughly this code:
struct Closure<T> {
captured: T,
}
impl<T> FnOnce<()> for Closure<T> {
type Output = &'??? T; // what do I put as lifetime here?
fn call_once(self, _: ()) -> Self::Output {
&self.captured // returning reference to local variable
// no matter what, the reference would be invalid once we return
}
}
And this is why what you're trying to do is fundamentally impossible. Take a step back, think of what you're actually trying to accomplish with this closure, and find some other way to accomplish it.
You expect the type T to have lifetime 'a, but t is not a reference to a value of type T. The function takes ownership of the variable t by argument passing:
// t is moved here, t lifetime is the scope of the function
fn foo<'a, T: 'a>(t: T)
You should do:
fn foo<'a, T: 'a>(t: &'a T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || t)
}
The other answers are top-notch, but I wanted to chime in with another reason your original code couldn't work. A big problem lies in the signature:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a>
This says that the caller may specify any lifetime when calling foo and the code will be valid and memory-safe. That cannot possibly be true for this code. It wouldn't make sense to call this with 'a set to 'static, but nothing about this signature would prevent that.
I'm having trouble understanding how values of boxed traits come into existence. Consider the following code:
trait Fooer {
fn foo(&self);
}
impl Fooer for i32 {
fn foo(&self) { println!("Fooer on i32!"); }
}
fn main() {
let a = Box::new(32); // works, creates a Box<i32>
let b = Box::<i32>::new(32); // works, creates a Box<i32>
let c = Box::<dyn Fooer>::new(32); // doesn't work
let d: Box<dyn Fooer> = Box::new(32); // works, creates a Box<Fooer>
let e: Box<dyn Fooer> = Box::<i32>::new(32); // works, creates a Box<Fooer>
}
Obviously, variant a and b work, trivially. However, variant c does not, probably because the new function takes only values of the same type which is not the case since Fooer != i32. Variant d and e work, which lets me suspect that some kind of automatic conversion from Box<i32> to Box<dyn Fooer> is being performed.
So my questions are:
Does some kind of conversion happen here?
If so, what the mechanism behind it and how does it work? (I'm also interested in the low level details, i.e. how stuff is represented under the hood)
Is there a way to create a Box<dyn Fooer> directly from an i32? If not: why not?
However, variant c does not, probably because the new function takes only values of the same type which is not the case since Fooer != i32.
No, it's because there is no new function for Box<dyn Fooer>. In the documentation:
impl<T> Box<T>
pub fn new(x: T) -> Box<T>
Most methods on Box<T> allow T: ?Sized, but new is defined in an impl without a T: ?Sized bound. That means you can only call Box::<T>::new when T is a type with a known size. dyn Fooer is unsized, so there simply isn't a new function to call.
In fact, that function can't exist in today's Rust. Box<T>::new needs to know the concrete type T so that it can allocate memory of the right size and alignment. Therefore, you can't erase T before you send it to Box::new. (It's conceivable that future language extensions may allow functions to accept unsized parameters; however, it's unclear whether even unsized_locals would actually enable Box<T>::new to accept unsized T.)
For the time being, unsized types like dyn Fooer can only exist behind a "fat pointer", that is, a pointer to the object and a pointer to the implementation of Fooer for that object. How do you get a fat pointer? You start with a thin pointer and coerce it. That's what's happening in these two lines:
let d: Box<Fooer> = Box::new(32); // works, creates a Box<Fooer>
let e: Box<Fooer> = Box::<i32>::new(32); // works, creates a Box<Fooer>
Box::new returns a Box<i32>, which is then coerced to Box<Fooer>. You could consider this a conversion, but the Box isn't changed; all the compiler does is stick an extra pointer on it and forget its original type. rodrigo's answer goes into more detail about the language-level mechanics of this coercion.
Hopefully all of this goes to explain why the answer to
Is there a way to create a Box<Fooer> directly from an i32?
is "no": the i32 has to be boxed before you can erase its type. It's the same reason you can't write let x: Fooer = 10i32.
Related
Why can't I write a function with the same type as Box::new?
Are polymorphic variables allowed?
How do you actually use dynamically sized types in Rust?
Why is `let ref a: Trait = Struct` forbidden?
I'll try to explain what conversions (coercions) happen in your code.
There is a marker trait named Unsize that, between others:
Unsize is implemented for:
T is Unsize<Trait> when T: Trait.
[...]
This trait, AFAIK, is not used directly for coercions. Instead, CoerceUnsized is used. This trait is implemented in a lot of cases, some of them are quite expected, such as:
impl<'a, 'b, T, U> CoerceUnsized<&'a U> for &'b T
where
'b: 'a,
T: Unsize<U> + ?Sized,
U: ?Sized
that is used to coerce &i32 into &Fooer.
The interesting, not so obvious implementation for this trait, that affects your code is:
impl<T, U> CoerceUnsized<Box<U>> for Box<T>
where
T: Unsize<U> + ?Sized,
U: ?Sized
This, together with the definition of the Unsize marker, can be somewhat read as: if U is a trait and T implements U, then Box<T> can be coerced into Box<U>.
About your last question:
Is there a way to create a Box<Fooer> directly from an i32? If not: why not?
Not that I know of. The problem is that Box::new(T) requires a sized value, since the value passed is moved into the box, and unsized values cannot be moved.
In my opinion, the easiest way to do that is to simply write:
let c = Box::new(42) as Box<Fooer>;
That is, you create a Box of the proper type and then coerce to the unsized one (note it looks quite similar to your d example).
Considering the following code:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || &t)
}
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
t moves to the closure, so the closure live as long as t
The closure returns a reference to t which was moved to the closure. So the reference is valid as long as the closure exists.
There is no lifetime problem, the code compiles.
What actually happens:
The code does not compile:
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
|
note: first, the lifetime cannot outlive the lifetime as defined on the body at 2:14...
--> src/lib.rs:2:14
|
2 | Box::new(move || &t)
| ^^^^^^^^^^
note: ...so that closure can access `t`
--> src/lib.rs:2:22
|
2 | Box::new(move || &t)
| ^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the function body at 1:8...
--> src/lib.rs:1:8
|
1 | fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a> {
| ^^
= note: ...so that the expression is assignable:
expected std::boxed::Box<(dyn std::ops::Fn() -> &'a T + 'a)>
found std::boxed::Box<dyn std::ops::Fn() -> &T>
I do not understand the conflict. How can I fix it?
Very interesting question! I think I understood the problem(s) at play here. Let me try to explain.
tl;dr: closures cannot return references to values captured by moving, because that would be a reference to self. Such a reference cannot be returned because the Fn* traits don't allow us to express that. This is basically the same as the streaming iterator problem and could be fixed via GATs (generic associated types).
Implementing it manually
As you probably know, when you write a closure, the compiler will generate a struct and impl blocks for the appropriate Fn traits, so closures are basically syntax sugar. Let's try to avoid all that sugar and build your type manually.
What you want is a type which owns another type and can return references to that owned type. And you want to have a function which returns a boxed instance of said type.
struct Baz<T>(T);
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
fn make_baz<T>(t: T) -> Box<Baz<T>> {
Box::new(Baz(t))
}
This is pretty equivalent to your boxed closure. Let's try to use it:
let outside = {
let s = "hi".to_string();
let baz = make_baz(s);
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // works too
This works just fine. The string s is moved into the Baz type and that Baz instance is moved into the Box. s is now owned by baz and then by outside.
It gets more interesting when we add a single character:
let outside = {
let s = "hi".to_string();
let baz = make_baz(&s); // <-- NOW BORROWED!
println!("{}", baz.call()); // works
baz
};
println!("{}", outside.call()); // doesn't work!
Now we cannot make the lifetime of baz bigger than the lifetime of s, since baz contains a reference to s which would be an dangling reference of s would go out of scope earlier than baz.
The point I wanted to make with this snippet: we didn't need to annotate any lifetimes on the type Baz to make this safe; Rust figured it out on its own and enforces that baz lives no longer than s. This will be important below.
Writing a trait for it
So far we only covered the basics. Let's try to write a trait like Fn to get closer to your original problem:
trait MyFn {
type Output;
fn call(&self) -> Self::Output;
}
In our trait, there are no function parameters, but otherwise it's fairly identical to the real Fn trait.
Let's implement it!
impl<T> MyFn for Baz<T> {
type Output = ???;
fn call(&self) -> Self::Output {
&self.0
}
}
Now we have a problem: what do we write instead of ???? Naively one would write &T... but we need a lifetime parameter for that reference. Where do we get one? What lifetime does the return value even have?
Let's check the function we implemented before:
impl<T> Baz<T> {
fn call(&self) -> &T {
&self.0
}
}
So here we use &T without lifetime parameter too. But this only works because of lifetime elision. Basically, the compiler fills in the blanks so that fn call(&self) -> &T is equivalent to:
fn call<'s>(&'s self) -> &'s T
Aha, so the lifetime of the returned reference is bound to the self lifetime! (more experienced Rust users might already have a feeling where this is going...).
(As a side note: why is the returned reference not dependent on the lifetime of T itself? If T references something non-'static then this has to be accounted for, right? Yes, but it is already accounted for! Remember that no instance of Baz<T> can ever live longer than the thing T might reference. So the self lifetime is already shorter than whatever lifetime T might have. Thus we only need to concentrate on the self lifetime)
But how do we express that in the trait impl? Turns out: we can't (yet). This problem is regularly mentioned in the context of streaming iterators -- that is, iterators that return an item with a lifetime bound to the self lifetime. In today's Rust, it is sadly impossible to implement this; the type system is not strong enough.
What about the future?
Luckily, there is an RFC "Generic Associated Types" which was merged some time ago. This RFC extends the Rust type system to allow associated types of traits to be generic (over other types and lifetimes).
Let's see how we can make your example (kinda) work with GATs (according to the RFC; this stuff doesn't work yet ☹). First we have to change the trait definition:
trait MyFn {
type Output<'a>; // <-- we added <'a> to make it generic
fn call(&self) -> Self::Output;
}
The function signature hasn't changed in the code, but notice that lifetime elision kicks in! The above fn call(&self) -> Self::Output is equivalent to:
fn call<'s>(&'s self) -> Self::Output<'s>
So the lifetime of the associated type is bound to the self lifetime. Just as we wanted! The impl looks like this:
impl<T> MyFn for Baz<T> {
type Output<'a> = &'a T;
fn call(&self) -> Self::Output {
&self.0
}
}
To return a boxed MyFn we would need to write this (according to this section of the RFC:
fn make_baz<T>(t: T) -> Box<for<'a> MyFn<Output<'a> = &'a T>> {
Box::new(Baz(t))
}
And what if we want to use the real Fn trait? As far as I understand, we can't, even with GATs. I think it's impossible to change the existing Fn trait to use GATs in a backwards compatible manner. So it's likely that the standard library will keep the less powerful trait as is. (side note: how to evolve the standard library in backwards incompatible ways to use new language features is something I wondered about a few times already; so far I haven't heard of any real plan in this regards; I hope the Rust team comes up with something...)
Summary
What you want is not technically impossible or unsafe (we implemented it as a simple struct and it works). However, unfortunately it is impossible to express what you want in the form of closures/Fn traits in Rust's type system right now. This is the same problem streaming iterators are dealing with.
With the planned GAT feature, it is possible to express all of this in the type system. However, the standard library would need to catch up somehow to make your exact code possible.
What I expect:
The type T has lifetime 'a.
The value t live as long as T.
This makes no sense. A value cannot "live as long" as a type, because a type doesn't live. "T has lifetime 'a" is a very imprecise statement, easy to misunderstand. What T: 'a really means is "instances of T must stay valid at least as long as lifetime 'a. For example, T must not be a reference with a lifetime shorter than 'a, or a struct containing such a reference. Note that this has nothing to do with forming references to T, i.e. &T.
The value t, then, lives as long as its lexical scope (it's a function parameter) says it does, which has nothing to do with 'a at all.
t moves to the closure, so the closure live as long as t
This is also incorrect. The closure lives as long as the closure does lexically. It is a temporary in the result expression, and therefore lives until the end of the result expression. t's lifetime concerns the closure not at all, since it has its own T variable inside, the capture of t. Since the capture is a copy/move of t, it is not in any way affected by t's lifetime.
The temporary closure is then moved into the box's storage, but that's a new object with its own lifetime. The lifetime of that closure is bound to the lifetime of the box, i.e. it is the return value of the function, and later (if you store the box outside the function) the lifetime of whatever variable you store the box in.
All of that means that a closure that returns a reference to its own capture state must bind the lifetime of that reference to its own reference. Unfortunately, this is not possible.
Here's why:
The Fn trait implies the FnMut trait, which in turn implies the FnOnce trait. That is, every function object in Rust can be called with a by-value self argument. This means that every function object must be still valid being called with a by-value self argument and returning the same thing as always.
In other words, trying to write a closure that returns a reference to its own captures expands to roughly this code:
struct Closure<T> {
captured: T,
}
impl<T> FnOnce<()> for Closure<T> {
type Output = &'??? T; // what do I put as lifetime here?
fn call_once(self, _: ()) -> Self::Output {
&self.captured // returning reference to local variable
// no matter what, the reference would be invalid once we return
}
}
And this is why what you're trying to do is fundamentally impossible. Take a step back, think of what you're actually trying to accomplish with this closure, and find some other way to accomplish it.
You expect the type T to have lifetime 'a, but t is not a reference to a value of type T. The function takes ownership of the variable t by argument passing:
// t is moved here, t lifetime is the scope of the function
fn foo<'a, T: 'a>(t: T)
You should do:
fn foo<'a, T: 'a>(t: &'a T) -> Box<Fn() -> &'a T + 'a> {
Box::new(move || t)
}
The other answers are top-notch, but I wanted to chime in with another reason your original code couldn't work. A big problem lies in the signature:
fn foo<'a, T: 'a>(t: T) -> Box<Fn() -> &'a T + 'a>
This says that the caller may specify any lifetime when calling foo and the code will be valid and memory-safe. That cannot possibly be true for this code. It wouldn't make sense to call this with 'a set to 'static, but nothing about this signature would prevent that.