Consider the following:
// Just a sequence of adjacent fields of same the type
#[repr(C)]
#[derive(Debug)]
struct S<T> {
a : T,
b : T,
c : T,
d : T,
}
impl<T : Sized> S<T> {
fn new(a : T, b : T, c : T, d : T) -> Self {
Self {
a,
b,
c,
d,
}
}
// reinterpret it as an array
fn as_slice(&self) -> &[T] {
unsafe { std::slice::from_raw_parts(self as *const Self as *const T, 4) }
}
}
fn main() {
let s = S::new(1, 2, 3, 4);
let a = s.as_slice();
println!("s :: {:?}\n\
a :: {:?}", s, a);
}
Is this code portable?
Is it always safe to assume a repr(C) struct with fields of same type can be reinterpreted like an array? Why?
Yes, it is safe and portable, except for very large T (fix below). None of the points listed in the safety section of the documentation for std::slice::from_raw_parts are a concern here:
The data pointer is valid for mem::size_of::<T>() * 4, which is the size of S<T>, and is properly aligned.
All of the items are in the same allocation object, because they are in the same struct.
The pointer is not null, because it is a cast from the safe &self parameter, and it is properly aligned, because S<T> has (at least) the alignment of T.
The data parameter definitely points to 4 consecutive initialized Ts, because S is marked #[repr(C)] which is defined such that in your struct, no padding would be introduced. (repr(Rust) makes no such guarantee).
The memory referenced is not mutated during the lifetime of the reference, which is guaranteed by the borrow checker.
The total size of the slice must not be greater than isize::MAX. The code does not check this, so it is technically a safety hole. To be sure, add a check to as_slice, before the unsafe:
assert!(std::mem::size_of::<S<T>>() <= isize::MAX as _);
The check will normally be optimized out.
Related
the rust docs gives the following example for blanket implementations
impl<T: Display> ToString for T {
// --snip--
}
My somewhat more complicated trait does not allow me to write a blanket implementation.
/// a fixed size 2D array (WxH) that can be indexed by [usize;2]
pub trait Grid<C: Copy + Eq, const W: usize, const H: usize>:
IndexMut<[usize; 2], Output = Option<C>>
{
}
// allow to transform one grid implementation into another
// as long as they contain the same elements and have the same size
impl<C, O, I, const W: usize, const H: usize> From<I> for O
where
O: Default + Grid<C, W, H>,
I: Grid<C, W, H>,
C: Copy
{
fn from(input: O) -> O {
let mut output = O::default();
for i in 0..O::W {
for j in 0..O::H {
output[[i, j]] = input[[i, j]];
}
}
output
}
}
The errors say
the type parameter `C` is not constrained by the impl trait, self type, or predicates
the const parameter `W` is not constrained by the impl trait, self type, or predicates
expressions using a const parameter must map each value to a distinct output value
proving the result of expressions other than the parameter are unique is not supported
It feels like it is constrained via the O: Grid<Player, W, H> but that doesn't seem to be the right constraint.
Rhe errors around the const generic parameters are secondary (a red herring?). When I replace them with constants, the error around C (the element type) still remains.
The problem is that your variables C, W, and H could have more than one value, as far as the compiler is concerned. Consider what happens if I implements Grid<Foo, 1, 2> and also Grid<Foo, 500, 50>: it is ambiguous which Grid implementation your blanket From implementation should be using. (C can't actually have multiple values for a given O or I, but the compiler doesn't actually reason that out.)
The solution to that problem is to change your trait's generics to associated types (and associated constants), which means that the trait cannot be implemented more than once for the same self type:
use std::ops::IndexMut;
/// a fixed size 2D array (WxH) that can be indexed by [usize;2]
pub trait Grid: IndexMut<[usize; 2], Output = Option<Self::Component>>
{
type Component: Copy + Eq;
const WIDTH: usize;
const HEIGHT: usize;
}
// allow to transform one grid implementation into another
// as long as they contain the same elements and have the same size
impl<O, I> From<I> for O
where
O: Default + Grid,
O::Component: Copy,
I: Grid<Component = O::Component, WIDTH = O::WIDTH, HEIGHT = O::HEIGHT>,
{
fn from(input: O) -> O {
let mut output = O::default();
for i in 0..O::W {
for j in 0..O::H {
output[[i, j]] = input[[i, j]];
}
}
output
}
}
However, this still will not compile, for several reasons:
You can't write a From impl for any two types that meet some trait bounds, because it might overlap with some other crate's From impl for any two types that meet different bounds — and it always overlaps with the blanket From<T> for T in the standard library.
In general, you can only implement From<TypeYouDefined> for T or From<T> for TypeYouDefined — not a fully generic, blanket From impl. You can define your own conversion trait, though.
You can't actually write a bound that two traits' associated constants are equal, as my code suggests — the compiler assumes it must be a type constraint. I don't know if there's a solution to that problem.
I want to write an LRU Cache with a memory size limitation rather than the "number of objects" limitation in std. After trying to figure it out for myself, I cheated and looked at an existing implementation, and I almost understand it, but this stops me:
struct KeyRef<K> {
k: *const K,
}
impl<K: Hash> Hash for LruKeyRef<K> {
fn hash<H: Hasher>(&self, state: &mut H) {
unsafe { (*self.k).hash(state) }
}
}
impl<K: PartialEq> PartialEq for LruKeyRef<K> {
fn eq(&self, other: &LruKeyRef<K>) -> bool {
unsafe { (*self.k).eq(&*other.k) }
}
}
It's that last unsafe line that I don't understand. I'm using a HashMap as the underlying structure, the key is stored with the value, and I want the hasher to be able to find it. I make the working hash key a reference to the real key and provide Hash and PartialEq functions such that the HashMap can find and use the key for bucketing purposes. That's easy.
I understand then that I have to compare the two for PartialEq, and so it makes sense to me that I have to use *self.k to dereference the current object, so why &*other.k for the other object? That's what I don't understand. Why isn't it just *other.k? Aren't I just dereferencing both so I can compare the actual keys?
We wish to call PartialEq::eq:
trait PartialEq<Rhs = Self>
where
Rhs: ?Sized,
{
fn eq(&self, other: &Rhs) -> bool;
}
Assuming the default implementation where Rhs = Self and Self = K, we need to end up with two &K types
other.k is of type *const K
*other.k is of type K
&*other.k is of type &K
This much should hopefully make sense.
self.k is of type *const K
*self.k is of type K
The piece that's missing that that method calls are allowed to automatically reference the value they are called on. This is why there's no distinct syntax for a reference and a value, as there would be in C or C++ (foo.bar() vs foo->bar()).
Thus, the K is automatically referenced to get &K, fulfilling the signature.
impl<K: PartialEq> PartialEq for LruKeyRef<K> {
fn eq(&self, other: &LruKeyRef<K>) -> bool {
unsafe { (*self.k).eq(&*other.k) }
}
}
Under typical circumstances, we can call methods taking &self with just a reference to the object. In addition, a chain of references to the object is also implicitly coerced. That is, we can write:
let a: &str = "I'm a static string";
assert_eq!(str.len(), 19);
assert_eq!((&&&&str).len(), 19);
In your case however, we start with a pointer, which must be explicitly dereferenced inside an unsafe scope. Here are the types of all relevant expressions:
self.k : *const K
(*self.k) : K
other.k : *const K
&*other.k : &K
Since equals takes a reference on its right-hand member, we must make it a reference. Unlike in C++, you can not just pass an lvalue as a reference without making this reference-passing explicit, nor can you pass an rvalue to a const reference. You can however, prepend & to a literal in order to obtain a reference to it (foo(&5)). It only appears asymmetrical because (in a way) self.k is the caller and other.k is the callee.
The general setup is I have an array of values I'd like to map() and then chain() with 1 additional value. I've learned from this answer that the proper way to construct that final value is to use std::iter::once. This works and eliminated the below problem, but I would still like to understand it better.
In my broken, likely rust-anti-pattern-riddled example, I was using an array of a single element and then calling into_iter(). This produced a value / reference type-mismatch in the chain.
Question: What is the Rust-idiomatic mechanism for correcting this value / reference mismatch? Particularly if clone and copy are unavailable.
Background: Why is there a type mis-match to begin with?
This much I believe I understand. Based on the definition of std::iter::Map, the item type for the iterator is type Item = B where B is constrained by F: FnMut(<I as Iterator>::Item) -> B (i.e. the mapped type). However array defines the following 2 IntoIterator implementations, both of which appear to produce references.
impl<'a, const N: usize, T> IntoIterator for &'a [T; N] where
[T; N]: LengthAtMost32,
type Item = &'a T
impl<'a, const N: usize, T> IntoIterator for &'a mut [T; N] where
[T; N]: LengthAtMost32,
type Item = &'a mut T
Example demonstrating the issue:
#[derive(PartialEq, Eq, Clone, Copy)]
enum Enum1 {
A, B, C
}
#[derive(PartialEq, Eq, Clone, Copy)]
enum Enum2 {
X, Y, Z
}
struct Data {
// Other data omitted
e1: Enum1,
e2: Enum2
}
struct Consumer {
// Other data omitted
/** Predicate which evaluates if this consumer can consume given Data */
consumes: Box<dyn Fn(&Data) -> bool>
}
fn main() {
// Objective: 3 consumers which consume data with A, B, and X respectively
let v: Vec<Consumer> = [Enum1::A, Enum1::B].iter()
.map(|&e1| Consumer { consumes: Box::new(move |data| data.e1 == e1) })
// This chain results in an iterator type-mismatch:
// expected &Consumer, found Consumer
.chain([Consumer { consumes: Box::new(move |data| data.e2 == Enum2::X) }].into_iter())
.collect(); // Fails as well due to the chain failure
}
Error:
error[E0271]: type mismatch resolving `<std::slice::Iter<'_, Consumer> as std::iter::IntoIterator>::Item == Consumer`
--> src/main.rs:52:10
|
52 | .chain([Consumer { consumes: Box::new(move |data| data.e2 == Enum2::X) }].into_iter())
| ^^^^^ expected reference, found struct `Consumer`
|
= note: expected type `&Consumer`
found type `Consumer`
Rust playground example.
There is a long-standing issue regarding this. The technical details are a bit heavy, but essentially, due to underlying, technical reasons, you cannot take ownership of a fixed-size array and return owned references without a lot of hocus pocus. This becomes obvious when you think about what a fixed-size array is and how it is stored in memory, and how you can get elements out without cloning them.
As a result, due to the implementations you found already, you can only get borrowed references. You can bypass this with arrayvec (as they have a sound implementation of IntoIterator for ArrayVec with owned types), or you can require that all your T: Clone and deal with it that way, at a cost of extra items in memory (temporarily; 90% of the time the compiler optimizes this away).
Let's say I have the following struct in Rust:
struct Num {
pub num: i32;
}
impl Num {
pub fn new(x: i32) -> Num {
Num { num: x }
}
}
impl Clone for Num {
fn clone(&self) -> Num {
Num { num: self.num }
}
}
impl Copy for Num { }
impl Add<Num> for Num {
type Output = Num;
fn add(self, rhs: Num) -> Num {
Num { num: self.num + rhs.num }
}
}
And then I have the following code snippet:
let a = Num::new(0);
let b = Num::new(1);
let c = a + b;
let d = a + b;
This works because Num is marked as Copy. Otherwise, the second addition would be a compilation error, since a and b had already been moved into the add function during the first addition (I think).
The question is what the emitted assembly does. When the add function is called, are two copies of the arguments made into the new stack frame, or is the Rust compiler smart enough to know that in this case, it's not necessary to do that copying?
If the Rust compiler isn't smart enough, and actually does the copying like a function with a argument passed by value in C++, how do you avoid the performance overhead in cases where it matters?
The context is I am implementing a matrix class (just to learn), and if I have a 100x100 matrix, I really don't want to be invoking two copies every time I try to do a multiply or add.
The question is what the emitted assembly does
There's no need to guess; you can just look. Let's use this code:
use std::ops::Add;
#[derive(Copy, Clone, Debug)]
struct Num(i32);
impl Add for Num {
type Output = Num;
fn add(self, rhs: Num) -> Num {
Num(self.0 + rhs.0)
}
}
#[inline(never)]
fn example() -> Num {
let a = Num(0);
let b = Num(1);
let c = a + b;
let d = a + b;
c + d
}
fn main() {
println!("{:?}", example());
}
Paste it into the Rust Playground, then select the Release mode and view the LLVM IR:
Search through the result to see the definition of the example function:
; playground::example
; Function Attrs: noinline norecurse nounwind nonlazybind readnone uwtable
define internal fastcc i32 #_ZN10playground7example17h60e923840d8c0cd0E() unnamed_addr #2 {
start:
ret i32 2
}
That's right, this was completely and totally evaluated at compile time and simplified all the way down to a simple constant. Compilers are pretty good nowadays.
Maybe you want to try something not quite as hardcoded?
#[inline(never)]
fn example(a: Num, b: Num) -> Num {
let c = a + b;
let d = a + b;
c + d
}
fn main() {
let something = std::env::args().count();
println!("{:?}", example(Num(something as i32), Num(1)));
}
Produces
; playground::example
; Function Attrs: noinline norecurse nounwind nonlazybind readnone uwtable
define internal fastcc i32 #_ZN10playground7example17h73d4138fe5e9856fE(i32 %a) unnamed_addr #3 {
start:
%0 = shl i32 %a, 1
%1 = add i32 %0, 2
ret i32 %1
}
Oops, the compiler saw that we we basically doing (x + 1) * 2, so it did some tricky optimizations here to get to 2x + 2. Let's try something harder...
#[inline(never)]
fn example(a: Num, b: Num) -> Num {
a + b
}
fn main() {
let something = std::env::args().count() as i32;
let another = std::env::vars().count() as i32;
println!("{:?}", example(Num(something), Num(another)));
}
Produces
; playground::example
; Function Attrs: noinline norecurse nounwind nonlazybind readnone uwtable
define internal fastcc i32 #_ZN10playground7example17h73d4138fe5e9856fE(i32 %a, i32 %b) unnamed_addr #3 {
start:
%0 = add i32 %b, %a
ret i32 %0
}
A simple add instruction.
The real takeaway from this is:
Look at the generated assembly for your cases. Even similar-looking code might optimize differently.
Perform micro and macro benchmarking. You never know exactly how the code will play out in the big picture. Maybe all your cache will be blown, but your micro benchmarks will be stellar.
is the Rust compiler smart enough to know that in this case, it's not necessary to do that copying?
As you just saw, the Rust compiler plus LLVM are pretty smart. In general, it is possible to elide copies when it knows that the operand isn't needed. Whether it will work in your case or not is tough to answer.
Even if it did, you might not want to be passing large items via the stack as it's always possible that it will need to be copied.
And note that you don't have to implement copy for the value, you can choose to only allow it via references:
impl<'a, 'b> Add<&'b Num> for &'a Num {
type Output = Num;
fn add(self, rhs: &'b Num) -> Num {
Num(self.0 + rhs.0)
}
}
In fact, you may want to implement both ways of adding them, and maybe all 4 permutations of value / reference!
If the Rust compiler isn't smart enough, and actually does the copying like a function with a argument passed by value in C++, how do you avoid the performance overhead in cases where it matters?
The context is I am implementing a matrix class (just to learn), and if I have a 100x100 matrix, I really don't want to be invoking two copies every time I try to do a multiply or add.
All of Rust's implicit copies (be them from moves or actual Copy types) are a shallow memcpy. If you heap allocate, only the pointers and such are copied. Unlike C++, passing a vector by value will only copy three pointer-sized values.
To copy heap memory, an explicit copy must be made, normally by calling .clone(), implemented with #[derive(Clone)] or impl Clone.
I've talked in more detail about this elsewhere.
Shepmaster points out that shallow copies are often messed with a lot by the compiler - generally only heap memory and massive stack values cause problems.
Why does the following for loop with a char range fail to compile?
fn main() {
for c in 'a'..'z' {
println!("{}", c);
}
}
Error...
main.rs:11:5: 14:2 error: the trait `core::num::One` is not implemented for the type `char` [E0277]
main.rs:11 for c in 'a'..'z' {
main.rs:12 println!("{}", c);
main.rs:13 }
main.rs:14 }
main.rs:11:5: 14:2 error: the trait `core::iter::Step` is not implemented for the type `char` [E0277]
main.rs:11 for c in 'a'..'z' {
main.rs:12 println!("{}", c);
main.rs:13 }
main.rs:14 }
Why do you even need core::num::One for a iterating over a range?
The x..y syntax is sugar for std::ops::Range { start: x, end: y }. This type (Range<A>) is iterable due to the implementation of Iterator for it, specifically, from that page:
impl<A> Iterator for Range<A>
where A: One + Step,
&'a A: Add<&'a A>,
&'a A::Output == A {
type Item = A;
This is saying that Range<A> can behave as an iterator over As if the type A implements One and Step, and can be added in the right way.
In this case, char satisfies none of those: it is semantically nonsense for char to have One or be addable, and it doesn't implement Step either.
That said, since char doesn't implement those traits (and hence Range<char> doesn't behave like an iterator via that impl), it should be possible to have a manual impl:
impl Iterator for Range<char> {
type Item = char;
which would allow for x in 'a'..'z' to work.
However, this probably isn't semantically what we want: the .. range doesn't include the last element, which would be suprising for characters, one would have to write 'a'..'{' to get the letters A through Z. There's been proposals for inclusive-range syntax, e.g. one example is 'a'...'z' (more dots == more elements), and I would imagine that there would be an Iterator implementation for this type with chars.
As others have demonstrated, for ASCII characters one can use byte literals, and more generally, one can cast characters to u32s:
for i in ('à' as u32)..('æ' as u32) + 1 {
let c = std::char::from_u32(i).unwrap();
println!("{}", c);
}
Which gives:
à
á
â
ã
ä
å
æ
NB. this approach isn't perfect, it will crash if the range crosses the surrogate range, 0xD800-0xDFFF.
I just published a crate, char-iter, which handles the latter correctly and behaves like one would expect. Once added (via cargo), it can be used like:
extern crate char_iter;
// ...
for c in char_iter::new('a', 'z') {
// ...
}
for c in char_iter::new('à', 'æ') {
// ...
}
Well, you need Step to denote that the structure can be stepped over in both directions.
/// Objects that can be stepped over in both directions.
///
/// The `steps_between` function provides a way to efficiently compare
/// two `Step` objects.
pub trait Step: PartialOrd
One on the other hand is used to retrieve a value from mutable iterator, while simultaneously incrementing it:
#[inline]
fn next(&mut self) -> Option<A> {
if self.start < self.end {
let mut n = &self.start + &A::one();
mem::swap(&mut n, &mut self.start);
Some(n)
} else {
None
}
}
Source
What you could do is make your range a u8 and then convert it back to char, like this:
fn main() {
for c in (b'a'..b'z'+1) {
println!(" {:?}", c as char);
}
}
Note: That range are exclusive so ('a'..'z') is actually ('a', 'b', ... 'y'). Or in math notation [a,z) ;) .
That's why I add b'z'+1 instead of b'z'.
Note: u8 is valid, only because the characters are ASCII.