Well the title says it all, but I'm just going to add my code as that makes it clearer what I want to do.
I have the following function:
fn calculate_rook_attacks_for(pos: &impl BoardPos, blockers: u64) -> u64 {
unimplemented!()
}
I now want a higher order function that takes such calculation functions (not just this one, all functions that have this signature), but I'm struggling on the type definition. Mostly with the trait part.
I've tried:
fn generate_all_possible_attacks_for(
calculate_moves_for: fn(impl BoardPos) -> u64
) {
// cool generations stuff
}
But apparently that's not allowed:
impl Trait only allowed in function and inherent method return types, not in fn.
I've also tried:
fn generate_all_possible_attacks_for<T>(
calculate_attacks_for: fn(&T, u64) -> u64,
)
where
T: BoardPos
{
// cool generation stuff
}
But that complains something about type inference when trying to call it:
cannot infer type for type parameter T declared on the function generate_all_possible_attacks_for
If I try to add the type: ::<&impl BoardPos>, I'm back the the previous error.
I've tried various other things (also stuff with &dyn), but I can't remember them all.
I'm sure the resources on how to do this are out there, I just lack the correct terminology to look it up.
(Btw, I don't want to use &dyn in the actual function call, as I've heard that results in a runtime performance impact)
Edit
Upon reading the answer I realized I'd have to specify the type at some point. I'm not sure what to do in the following case though:
fn generate_magic_number_for(pos: &impl BoardPos, piece: Piece) -> u64 {
let idx = pos.idx();
let (relevant_moves, number_of_relevant_moves, get_attacks_for): (
u64,
u64,
fn(&impl BoardPos, u64) -> u64,
) = match piece {
Piece::Bishop => (
RELEVANT_BISHOP_MOVES_PER_SQUARE[idx],
NUMBER_OF_RELEVANT_BISHOP_MOVES_PER_SQUARE[idx],
piece::calculate_bishop_attacks_for,
),
Piece::Rook => (
RELEVANT_ROOK_MOVES_PER_SQUARE[idx],
NUMBER_OF_RELEVANT_ROOK_MOVES_PER_SQUARE[idx],
piece::calculate_rook_attacks_for,
),
_ => panic!(
"this function is only callable for bishops and rooks, was called with '{:?}'",
piece
),
};
/// ...
}
If you look at this code:
trait BoardPos {}
fn calculate_rook_attacks_for(pos: &impl BoardPos, blockers: u64) -> u64 {
unimplemented!()
}
fn generate_all_possible_attacks_for<T>(calculate_attacks_for: fn(&T, u64) -> u64)
where
T: BoardPos,
{
// cool generation stuff
}
fn main() {
generate_all_possible_attacks_for(calculate_rook_attacks_for);
}
It makes sense that it complains about not knowing the type T, because neither generate_all_possible_attacks_for nor calculate_rook_attacks_for specify a type T. Both of them say that they need a type T with certain properties, but when you call the function, it needs to resolve to a concrete type that the compiler can actually compile to.
At some point, you actually need to specify what the exact type &impl BoardPos is supposed to be.
For example here:
trait BoardPos {}
struct MyBoardPos;
impl BoardPos for MyBoardPos {}
fn calculate_rook_attacks_for(pos: &impl BoardPos, blockers: u64) -> u64 {
unimplemented!()
}
fn generate_all_possible_attacks_for<T>(calculate_attacks_for: fn(&T, u64) -> u64)
where
T: BoardPos,
{
// cool generation stuff
}
fn main() {
generate_all_possible_attacks_for::<MyBoardPos>(calculate_rook_attacks_for);
}
or here:
trait BoardPos {}
struct MyBoardPos;
impl BoardPos for MyBoardPos {}
fn calculate_rook_attacks_for(pos: &impl BoardPos, blockers: u64) -> u64 {
unimplemented!()
}
fn generate_all_possible_attacks_for(calculate_attacks_for: fn(&MyBoardPos, u64) -> u64) {
// cool generation stuff
}
fn main() {
generate_all_possible_attacks_for(calculate_rook_attacks_for);
}
If you want to keep the &impl BoardPos in the function type, to call calculate_attacks_for inside if generate_all_possible_attacks_for several times with different types, I think you are out of luck. You cannot pass a generic function into another function. At the point where you pass it into the function, it needs to be a function pointer. Meaning, a real function that rust actually generated as bytecode. This means all generics have to be resolved, because I don't think the concept of an abstract function pointer exists. (but of course I might be wrong, please correct me)
The type of calculate_rook_attacks_for expands to
fn calculate_rook_attacks_for<T: BoardPos>(pos: &T, blockers: u64) -> u64
So you'd need T in generate_all_possible_attacks_for's type to be in a different place, something like
fn generate_all_possible_attacks_for<F: for<T: BoardPos> Fn(&T, u64) -> u64>(
calculate_attacks_for: F,
)
But this isn't legal (yet?). However, if your generate_all_possible_attacks_for only wants to call calculate_attacks_for with some specific implementation of BoardPos, you can just use it:
trait BoardPos {}
struct SomeBoardPos;
impl BoardPos for SomeBoardPos {}
fn calculate_rook_attacks_for(_pos: &impl BoardPos, _blockers: u64) -> u64 {
0
}
fn generate_all_possible_attacks_for(
calculate_attacks_for: impl Fn(&SomeBoardPos, u64) -> u64,
) -> u64 {
calculate_attacks_for(&SomeBoardPos, 0)
}
fn main() {
println!("{}", generate_all_possible_attacks_for(calculate_rook_attacks_for));
}
Playground
Unfortunately, if you want to call calculate_rook_attacks_for more than once with different implementations, you'll need to pass it the same number of times (so far as I know):
trait BoardPos {}
struct SomeBoardPos;
impl BoardPos for SomeBoardPos {}
struct SomeBoardPos2;
impl BoardPos for SomeBoardPos2 {}
fn calculate_rook_attacks_for(_pos: &impl BoardPos, _blockers: u64) -> u64 {
0
}
fn generate_all_possible_attacks_for(
calculate_attacks_for: impl Fn(&SomeBoardPos, u64) -> u64,
calculate_attacks_for2: impl Fn(&SomeBoardPos2, u64) -> u64,
) -> u64 {
calculate_attacks_for(&SomeBoardPos, 0) + calculate_attacks_for2(&SomeBoardPos2, 0)
}
fn main() {
println!("{}", generate_all_possible_attacks_for(calculate_rook_attacks_for, calculate_rook_attacks_for));
}
Playground
Upon reading the answer I realized I'd have to specify the type at some point. I'm not sure what to do in the following case though:
Name the type instead of using impl BoardPos:
fn generate_magic_number_for<T: BoardPos>(pos: &T, piece: Piece) -> u64 {
let idx = pos.idx();
let (relevant_moves, number_of_relevant_moves, get_attacks_for): (
u64,
u64,
fn(&T, u64) -> u64,
) = match piece {
Piece::Bishop => (
RELEVANT_BISHOP_MOVES_PER_SQUARE[idx],
NUMBER_OF_RELEVANT_BISHOP_MOVES_PER_SQUARE[idx],
piece::calculate_bishop_attacks_for,
),
Piece::Rook => (
RELEVANT_ROOK_MOVES_PER_SQUARE[idx],
NUMBER_OF_RELEVANT_ROOK_MOVES_PER_SQUARE[idx],
piece::calculate_rook_attacks_for,
),
_ => panic!(
"this function is only callable for bishops and rooks, was called with '{:?}'",
piece
),
};
/// ...
}
Related
given this example
struct Outer<T>(*mut T);
impl<T> Outer<T> {
pub fn new(value: &mut T) -> Outer<T> {
Outer(value as *mut T)
}
}
struct Inner(pub i32);
impl Inner {
pub fn do_thing(&self) {
println!("did the thing {}", self.0);
}
}
fn main() {
let outer = Outer::new(Inner(2));
outer.do_thing() // error: do_thing doesnt exist
}
how would i expose the methods of Inner as methods of Outer
im trying to achieve what Box is doing
You have to reborrow the pointer in order to implement Deref (playground):
impl<T: ?Sized> Deref for Outer<T> {
type Target = T;
fn deref(&self) -> &Self::Target {
unsafe { &*self.0 }
}
}
Answer from before the edit below:
The clean way is to have methods that return the inner type:
impl<T> Outer<T> {
pub fn get(&self) -> &T {
&self.0
}
pub fn get_mut(&mut self) -> &mut T {
&mut self.0
}
pub fn into_inner(self) -> T {
self.0
}
}
This is more useful when the inner type isn't pub (you can just do self.0 for the same effect as all three methods), but is a common pattern that users of rust libraries expect.
The other way is to implement Deref and DerefMut. This is messier because it's a very strong API commitment: you now have all the methods of the inner type added to the outer type unconditionally, and gain any future methods. It also means that any namespace collisions between the wrapper and the target are hard to notice, which is why things like Arc::get_mut are associated functions instead of methods. If you use this, consider changing methods on the wrapper to associated functions (don't take a self parameter).
impl<T> Deref for Outer<T> {
type Target = T;
fn deref(&self) -> &Self::Target {
&self.0
}
}
Can I propagate the Send trait of function parameters to its return type, so that the return type is impl Send if and only if the parameters are?
Details:
An async function has a nice feature. Its returned Future is automatically Send if it can be. In the following example, the async function will create a Future that is Send, if the inputs to the function are Send.
struct MyStruct;
impl MyStruct {
// This async fn returns an `impl Future<Output=T> + Send` if `T` is Send.
// Otherwise, it returns an `impl Future<Output=T>` without `Send`.
async fn func<T>(&self, t: T) -> T {
t
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This works
assert_is_send(MyStruct.func(4u64));
// And the following correctly fails
assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
playground
Now, I want to move such a function into a trait, which requires using async-trait (which is some codegen that effectively writes my async fn as a function returning Pin<Box<dyn Future>>) or doing something similar manually. Is there a way to write this in a way to retain this auto-Send behavior where the returned Future is made Send if T is Send? The following example implements it as two separate functions:
use std::pin::Pin;
use std::future::Future;
struct MyStruct;
impl MyStruct {
fn func_send<T: 'static + Send>(&self, t: T) -> Pin<Box<dyn Future<Output = T> + Send>> {
Box::pin(async{t})
}
fn func_not_send<T: 'static>(&self, t: T) -> Pin<Box<dyn Future<Output = T>>> {
Box::pin(async{t})
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This works
assert_is_send(MyStruct.func_send(4u64));
// And the following correctly fails
// assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
playground
But actually, I don't want them to be separate. I want them to be one function similar to how async fn does it automatically. Something along the lines of
use std::pin::Pin;
use std::future::Future;
struct MyStruct;
impl MyStruct {
fn func<T: 'static + ?Send>(&self, t: T) -> Pin<Box<dyn Future<Output = T> + ?Send>> {
Box::pin(async{t})
}
}
fn assert_is_send(_v: impl Send) {}
fn main() {
// This should
assert_is_send(MyStruct.func(4u64));
// And this should fail
assert_is_send(MyStruct.func(std::rc::Rc::new(4u64)));
}
Is something like this possible in Rust? I'm ok with writing the async-trait magic manually and modifying it instead of using the async-trait crate if that is a way to make it work.
Some ideas I had but they haven't really borne fruit yet:
Use min-specialization to specialize on Send? But doesn't seem like that feature is going to be stabilized anytime soon so maybe not the best option.
Return a custom MyFuture type instead of just impl Future and somehow impl Send for MyFuture where T: Send? Would probably be difficult though since I would have to be able to name that Future and async code usually produces impl Future types that cannot be named.
Writing a procedural macro that adds + Send to the return type if it recognizes that the input type is Send. Actually, can procedural macros detect if a certain type implements Send? My guess would be it's not possible since they just work on token streams.
(2) is the only way that could work.
There are two ways to make it work:
Write the future manually, without the help of async and .await. But that means writing the future manually:
enum ConditionalSendFut<T> {
Start { t: T },
Done,
}
impl<T> Unpin for ConditionalSendFut<T> {}
impl<T> Future for ConditionalSendFut<T> {
type Output = T;
fn poll(mut self: Pin<&mut Self>, _context: &mut Context<'_>) -> Poll<Self::Output> {
match &mut *self {
Self::Start { .. } => {
let t = match std::mem::replace(&mut *self, Self::Done) {
Self::Start { t } => t,
_ => unreachable!(),
};
Poll::Ready(t)
}
Self::Done => Poll::Pending,
}
}
}
struct MyStruct;
impl MyStruct {
fn func<T: 'static>(&self, t: T) -> ConditionalSendFut<T> {
ConditionalSendFut::Start { t }
}
}
Playground.
Store a Pin<Box<dyn Future<Output = T>>> and conditionally impl Send on the future. But this requires unsafe code and manually ensuring that you don't hold other non-Send types across .await points:
struct ConditionalSendFut<T>(Pin<Box<dyn Future<Output = T>>>);
// SAFETY: The only non-`Send` type we're holding across an `.await`
// point is `T`.
unsafe impl<T: Send> Send for ConditionalSendFut<T> {}
impl<T> Future for ConditionalSendFut<T> {
type Output = T;
fn poll(mut self: Pin<&mut Self>, context: &mut Context<'_>) -> Poll<Self::Output> {
self.0.as_mut().poll(context)
}
}
struct MyStruct;
impl MyStruct {
fn func<T: 'static>(&self, t: T) -> ConditionalSendFut<T> {
ConditionalSendFut(Box::pin(async { t }))
}
}
Playground.
(1) cannot work with traits, as each impl will have a different future. This leaves us with (2) only. I would not recommend it, but it is possible.
It is very likely that when async fns in traits will be stable there will be a mechanism to that (what is talked about currently is to impl them conditionally and use bounds on use sites to require them) but currently there is no such thing, even on the nightly implementation of async fns in traits.
I ran into a problem the other day when I started learning how generics worked on rust. The issue I encountered is in the following snippet:
struct Example<T> {
a: T
}
impl<T> Example<T> {
fn new() -> Self {
Self {
a: 5
}
}
}
Trying to compile this gives the following output:
error[E0308]: mismatched types
--> src/lib.rs:11:16
|
8 | impl<T> Example<T> {
| - this type parameter
...
11 | a: 5
| ^ expected type parameter `T`, found integer
|
= note: expected type parameter `T`
found type `{integer}`
For more information about this error, try `rustc --explain E0308`.
warning: `url_shortener` (lib) generated 2 warnings
error: could not compile `url_shortener` due to previous error; 2 warnings emitted
An easy fix would be specifying the type that we want, like so:
impl Example<i32> {
fn new() -> Self {
Self {
a: 5
}
}
}
But I don't really understand why keeping the type parameter general does not work. Also I have an example where changing the type parameter would not be so easy. Say we want to do something like the following:
use itertools::Itertools;
struct Example<T: Iterator<Item=String>> {
a: T
}
fn get_permutations(n: i32) -> impl Iterator<Item=String> {
(0..n)
.map(|_| (97u8..=122u8).map(|x| x as char))
.multi_cartesian_product().map(|v| v.iter().join(""))
}
impl<T: Iterator<Item=String>> Example<T> {
fn new() -> Self {
let test = get_permutations(4)
.chain(
get_permutations(3)
)
.chain(
get_permutations(2)
)
.chain(
get_permutations(1)
);
Self {
a: test
}
}
}
The type of test is Chain<Chain<Chain<impl Iterator<Item = String>, impl Iterator<Item = String>>, impl Iterator<Item = String>>, impl Iterator<Item = String>>, and we can't simply change the generic type parameter T to that, since the impl Trait syntax is not allowed for type parameters.
We could try changing the return type of the get_permutations function so that it does not return impl Trait, but the actual type returned by get_permutations includes closures and I don't know how to write it out (or if it even is possible considering each closure is its own type(?)). Besides we might want to chain together more functions in it at a later date, and it would be bad to change its type every time we wanted to do that.
So it seems it would be better to have get_permutations return type to be some generic Iterator, the only solution that I can think of is using Boxes like so:
use itertools::Itertools;
struct Example {
a: Box<dyn Iterator<Item = String>>
}
fn get_permutations(n: i32) -> Box<dyn Iterator<Item = String>> {
(0..n)
.map(|_| (97u8..=122u8).map(|x| x as char))
.multi_cartesian_product().map(|v| v.iter().join(""))
}
impl Example {
fn new() -> Self {
let test = Box::new(get_permutations(4)
.chain(
get_permutations(3)
)
.chain(
get_permutations(2)
)
.chain(
get_permutations(1)
));
Self {
a: test
}
}
}
But I find this kind of ugly. I can also create the object without using new():
use itertools::Itertools;
struct Example<T: Iterator<Item=String>> {
a: T
}
fn get_permutations(n: i32) -> impl Iterator<Item=String> {
(0..n)
.map(|_| (97u8..=122u8).map(|x| x as char))
.multi_cartesian_product().map(|v| v.iter().join(""))
}
fn main(){
let test = Example{a: get_permutations(4)
.chain(
get_permutations(3)
)
.chain(
get_permutations(2)
)
.chain(
get_permutations(1)
)};
for perm in test.a {
println!("{perm}")
}
}
But this wouldn't make much sense if it is the case that every Example object should contain this exact iterator. So, is there a better way of doing this without using Boxes?
Your simple example doesn't work since generics are specified by the user, i.e. Example::<String>::new() wouldn't make sense with your implementation, since it would try assigning 5 to a String.
For your actual use-case, you want to be able to store a type that you don't want to name, and want to avoid trait objects. As of right now, this isn't possible AFAIK. There is an unstable feature, #![feature(type_alias_impl_trait)], which would be ideal and would look like this:
#![feature(type_alias_impl_trait)] // <------------
use itertools::Itertools;
type Permutations = impl Iterator<Item = String>; // <------------
struct Example {
a: Permutations,
}
fn get_permutations(n: i32) -> impl Iterator<Item = String> {
(0..n)
.map(|_| (97u8..=122u8).map(|x| x as char))
.multi_cartesian_product()
.map(|v| v.iter().join(""))
}
impl Example {
fn new() -> Self {
let test = get_permutations(4)
.chain(get_permutations(3))
.chain(get_permutations(2))
.chain(get_permutations(1));
Self { a: test }
}
}
Your stable options are to either use a trait object, Box<dyn Iterator<Item = String>>, as you've already found or you can keep a generic argument and have a function return a concrete but opaque type via Example<impl Iterator<Item = String>>:
use itertools::Itertools;
struct Example<T: Iterator<Item = String>> {
a: T,
}
fn get_permutations(n: i32) -> impl Iterator<Item = String> {
(0..n)
.map(|_| (97u8..=122u8).map(|x| x as char))
.multi_cartesian_product()
.map(|v| v.iter().join(""))
}
impl<T: Iterator<Item = String>> Example<T> {
fn new() -> Example<impl Iterator<Item = String>> { // <------------
let test = get_permutations(4)
.chain(get_permutations(3))
.chain(get_permutations(2))
.chain(get_permutations(1));
Example { a: test }
}
}
Both have their drawbacks since the former would require an allocation and dynamic dispatch, but the latter method is still an unnameable type (you can't store Example<impl ...> in a struct) and can still be overridden by users.
#[doc(hidden)]
pub fn parse<T: ParseMacroInput>(token_stream: TokenStream) -> Result<T> {
T::parse.parse(token_stream)
} // Not public API.
#[doc(hidden)]
pub trait ParseMacroInput: Sized {
fn parse(input: ParseStream) -> Result<Self>;
}
impl<T: Parse> ParseMacroInput for T {
fn parse(input: ParseStream) -> Result<Self> {
<T as Parse>::parse(input)
}
}
I guess that the code you find weird is that in the title:
T::parse.parse(token_stream)
Here T is a generic type that implements ParseMacroInput, because of the constraint T: ParseMacroInput above. The only member of that trait is that fn parse(input: ParseStream) -> Result<Self>; function, so T::parse must be a reference to that function.
Now, what about the next .parse? Well functions are values too, and they can implement traits. And indeed in synthere is this trait:
pub trait Parser: Sized {
// ...
fn parse(self, tokens: proc_macro::TokenStream) -> Result<Self::Output> { ... }
}
and an implementation for functions that take a ParseStream and returns a Result<_>, just like function ParseMacroInput::parse.
impl<F, T> Parser for F
where
F: FnOnce(ParseStream) -> Result<T>
And that is the function your original T::parse.parse() calls: a function of a trait implemented by a function type or another generic type. The fact that both functions have the same name is an unfortunate coincidence.
I'm trying to write some code that will generate a random struct with a random value. I have the following trait and helper macros for the structs:
use rand::{thread_rng, Rng};
use std::fmt;
pub trait DataType {
/// generate a new instance of the type with a random value
fn random() -> Box<Self>;
/// generate a new instance of the same type with a random value
fn gen_another(&self) -> Box<Self>;
}
macro_rules! impl_data_type_for_num {
($x:ident) => {
impl DataType for $x {
fn random() -> Box<Self> {
Box::new(Self {
value: thread_rng().gen()
})
}
fn gen_another(&self) -> Box<Self> {
Self::random()
}
}
};
}
macro_rules! impl_formatting {
($x:ident, $s:expr) => {
impl fmt::Debug for $x {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, $s)
}
}
impl fmt::Display for $x {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "{}", self.value)
}
}
};
}
Then I use the macros to implement the needed traits on a bunch of structs (heres a few for example):
pub struct VirtBool {
value: bool
}
impl_data_type_for_num!(VirtBool);
impl_formatting!(VirtBool, "bool");
pub struct VirtU8 {
value: u8
}
impl_data_type_for_num!(VirtU8);
impl_formatting!(VirtU8, "u8");
pub struct VirtU16 {
value: u16
}
impl_data_type_for_num!(VirtU16);
impl_formatting!(VirtU16, "u16");
So far it all works fine, but then an issue arises when I try to implement the same traits on a struct with unsized fields:
pub struct VirtArray {
_type: Box<dyn DataType>,
value: Vec<Box<dyn DataType>>
}
impl DataType for VirtArray {
fn random() -> Box<Self> {
let t = random_var();
let s = thread_rng().gen_range(0, 10);
Box::new(Self {
_type: *t,
value: (0..s).map(|_| t.gen_another()).collect()
})
}
fn gen_another(&self) -> Box<Self> {
Box::new(Self {
_type: self._type,
value: self.value.iter().map(|t| t.gen_another()).collect::<Vec<Box<dyn DataType>>>()
})
}
}
impl fmt::Debug for VirtArray {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "[{:?}; {}]", self._type, self.value.len())
}
}
impl fmt::Display for VirtArray {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
let mut s = self.value.iter().map(|v| format!("{}, ",v)).collect::<String>();
s.truncate(s.len().checked_sub(2).unwrap_or(s.len()));
write!(f, "[{}]", s)
}
}
This throws the error the trait 'DataType' cannot be made into an object on several occasions. I then have a function to create a random struct but that also throws the same error:
/// generate a random type with random value
fn random_var() -> Box<dyn DataType> {
match thread_rng().gen_range(0,4) {
0 => Box::new(VirtBool::random()),
1 => Box::new(VirtU8::random()),
2 => Box::new(VirtU16::random()),
3 => Box::new(VirtArray::random()),
_ => panic!("invalid")
}
}
I was originally using enums to do all of this but I'm trying to switch it over to structs and traits to help with scaling/use-ability. Does anyone have any idea how to fix the code above? I've been at a roadblock here for quite a while now.
Also, I'm aware I could use type_name_of_val to print the types, but I'm trying to keep from using unstable/nightly features.
DataType cannot be made into a trait object because it uses Self and because it has a static method.
I realize it might seem like returning Box<Self> may be reasonable to call on a dyn DataType, since if you call it on dyn DataType you want a Box<dyn DataType>, but Rust doesn't try to modify methods for you to turn methods that return e.g. Box<VirtArray> into ones that return Box<dyn DataType> if they are called on a dyn DataType value. You can work around this by having the methods return Box<dyn DataType> instead.
Static methods are not allowed for trait objects because there is no implementation for the trait object type. Remember, dyn Foo implements Foo. What would (dyn DataType)::random() be? (You can use a where clause, as in the example above, to make sure dyn DataType isn't expected to have this method in a way that can be detected ahead of time; this means you can't use it on your dyn DataType objects, but it sounds like you might not want to.)