In rustc 1.0.0, I'd like to write a function that mutates a two dimensional array supplied by the caller. I was hoping this would work:
fn foo(x: &mut [[u8]]) {
x[0][0] = 42;
}
fn main() {
let mut x: [[u8; 3]; 3] = [[0; 3]; 3];
foo(&mut x);
}
It fails to compile:
$ rustc fail2d.rs
fail2d.rs:7:9: 7:15 error: mismatched types:
expected `&mut [[u8]]`,
found `&mut [[u8; 3]; 3]`
(expected slice,
found array of 3 elements) [E0308]
fail2d.rs:7 foo(&mut x);
^~~~~~
error: aborting due to previous error
I believe this is telling me I need to somehow feed the function a slice of slices, but I don't know how to construct this.
It "works" if I hard-code the nested array's length in the function signature. This isn't acceptable because I want the function to operate on multidimensional arrays of arbitrary dimension:
fn foo(x: &mut [[u8; 3]]) { // FIXME: don't want to hard code length of nested array
x[0][0] = 42;
}
fn main() {
let mut x: [[u8; 3]; 3] = [[0; 3]; 3];
foo(&mut x);
}
tldr; any zero-cost ways of passing a reference to a multidimensional array such that the function use statements like $x[1][2] = 3;$?
This comes down to a matter of memory layout. Assuming a type T with a size known at compile time (this constraint can be written T: Sized), the size of [T; n] is known at compile time (it takes n times as much memory as T does); but [T] is an unsized type; its length is not known at compile time. Therefore it can only be used through some form of indirection, such as a reference (&[T]) or a box (Box<[T]>, though this is of limited practical value, with Vec<T> which allows you to add and remove items without needing to reallocate every single time by using overallocation).
A slice of an unsized type doesn’t make sense; it’s permitted for reasons that are not clear to me, but you can never actually have an instance of it. (Vec<T>, by comparison, requires T: Sized.)
&[T; n] can coerce to &[T], and &mut [T; n] to &mut [T], but this only applies at the outermost level; the contents of slice is fixed (you’d need to create a new array or vector to achieve such a transformation, because the memory layout of each item is different). The effect of this is that arrays work for single‐dimensional work, but for multi‐dimensional work they fall apart. Arrays are currently very much second‐class citizens in Rust, and will be until the language supports making slices generic over length, which it is likely to eventually.
I recommend that you use either a single‐dimensional array (suitable for square matrices, indexed by x * width + y or similar), or vectors (Vec<Vec<T>>). There may also be libraries already out there abstracting over a suitable solution.
Related
I want to call .map() on an array of enums:
enum Foo {
Value(i32),
Nothing,
}
fn main() {
let bar = [1, 2, 3];
let foos = bar.iter().map(|x| Foo::Value(*x)).collect::<[Foo; 3]>();
}
but the compiler complains:
error[E0277]: the trait bound `[Foo; 3]: std::iter::FromIterator<Foo>` is not satisfied
--> src/main.rs:8:51
|
8 | let foos = bar.iter().map(|x| Foo::Value(*x)).collect::<[Foo; 3]>();
| ^^^^^^^ a collection of type `[Foo; 3]` cannot be built from an iterator over elements of type `Foo`
|
= help: the trait `std::iter::FromIterator<Foo>` is not implemented for `[Foo; 3]`
How do I do this?
The issue is actually in collect, not in map.
In order to be able to collect the results of an iteration into a container, this container should implement FromIterator.
[T; n] does not implement FromIterator because it cannot do so generally: to produce a [T; n] you need to provide n elements exactly, however when using FromIterator you make no guarantee about the number of elements that will be fed into your type.
There is also the difficulty that you would not know, without supplementary data, which index of the array you should be feeding now (and whether it's empty or full), etc... this could be addressed by using enumerate after map (essentially feeding the index), but then you would still have the issue of deciding what to do if not enough or too many elements are supplied.
Therefore, not only at the moment one cannot implement FromIterator on a fixed-size array; but even in the future it seems like a long shot.
So, now what to do? There are several possibilities:
inline the transformation at call site: [Value(1), Value(2), Value(3)], possibly with the help of a macro
collect into a different (growable) container, such as Vec<Foo>
...
Update
This can work:
let array: [T; N] = something_iterable.[into_]iter()
.collect::<Vec<T>>()
.try_into()
.unwrap()
In newer version of rust, try_into is included in prelude, so it is not necessary to use std::convert::TryInto. Further, starting from 1.48.0, array support directly convert from Vec type, signature from stdlib source:
fn try_from(mut vec: Vec<T, A>) -> Result<[T; N], Vec<T, A>> {
...
}
Original Answer
as of rustc 1.42.0, if your element impl Copy trait, for simplicity, this just works:
use std::convert::TryInto;
...
let array: [T; N] = something_iterable.[into_]iter()
.collect::<Vec<T>>()
.as_slice()
.try_into()
.unwrap()
collect as_slice try_into + unwrap()
Iterator<T> ------> Vec<T> -------> &[T] ------------------> [T]
But I would just call it a workaround.
You need to include std::convert::TryInto because the try_into method is defined in the TryInto trait.
Below is the signature checked when you call try_into as above, taken from the source. As you can see, that requires your type T implement Copy trait, so theoritically, it will copy all your elements once.
#[stable(feature = "try_from", since = "1.34.0")]
impl<T, const N: usize> TryFrom<&[T]> for [T; N]
where
T: Copy,
[T; N]: LengthAtMost32,
{
type Error = TryFromSliceError;
fn try_from(slice: &[T]) -> Result<[T; N], TryFromSliceError> {
<&Self>::try_from(slice).map(|r| *r)
}
}
While you cannot directly collect into an array for the reasons stated by the other answers, that doesn't mean that you can't collect into a data structure backed by an array, like an ArrayVec:
use arrayvec::ArrayVec; // 0.7.0
use std::array;
enum Foo {
Value(i32),
Nothing,
}
fn main() {
let bar = [1, 2, 3];
let foos: ArrayVec<_, 3> = array::IntoIter::new(bar).map(Foo::Value).collect();
let the_array = foos
.into_inner()
.unwrap_or_else(|_| panic!("Array was not completely filled"));
// use `.expect` instead if your type implements `Debug`
}
Pulling the array out of the ArrayVec returns a Result to deal with the case where there weren't enough items to fill it; the case that was discussed in the other answers.
For your specific problem, Rust 1.55.0 allows you to directly map an array:
enum Foo {
Value(i32),
Nothing,
}
fn main() {
let bar = [1, 2, 3];
let foos = bar.map(Foo::Value);
}
In this case you can use Vec<Foo>:
#[derive(Debug)]
enum Foo {
Value(i32),
Nothing,
}
fn main() {
let bar = [1, 2, 3];
let foos = bar.iter().map(|&x| Foo::Value(x)).collect::<Vec<Foo>>();
println!("{:?}", foos);
}
.collect() builds data structures that can have arbitrary length, because the iterator's item number is not limited in general. (Shepmaster's answer already provides plenty details there).
One possibility to get data into an array from a mapped chain without allocating a Vec or similar is to bring mutable references to the array into the chain. In your example, that'd look like this:
#[derive(Debug, Clone, Copy)]
enum Foo {
Value(i32),
Nothing,
}
fn main() {
let bar = [1, 2, 3];
let mut foos = [Foo::Nothing; 3];
bar.iter().map(|x| Foo::Value(*x))
.zip(foos.iter_mut()).for_each(|(b, df)| *df = b);
}
The .zip() makes the iteration run over both bar and foos in lockstep -- if foos were under-allocated, the higher bars would not be mapped at all, and if it were over-allocated, it'd keep its original initialization values. (Thus also the Clone and Copy, they are needed for the [Nothing; 3] initialization).
You can actually define a Iterator trait extension to do this!
use std::convert::AsMut;
use std::default::Default;
trait CastExt<T, U: Default + AsMut<[T]>>: Sized + Iterator<Item = T> {
fn cast(mut self) -> U {
let mut out: U = U::default();
let arr: &mut [T] = out.as_mut();
for i in 0..arr.len() {
match self.next() {
None => panic!("Array was not filled"),
Some(v) => arr[i] = v,
}
}
assert!(self.next().is_none(), "Array was overfilled");
out
}
}
impl<T, U: Iterator<Item = T>, V: Default + AsMut<[T]>> CastExt<T, V> for U { }
fn main () {
let a: [i32; 8] = (0..8).map(|i| i * 2).cast();
println!("{:?}", a); // -> [0, 2, 4, 6, 8, 10, 12, 14]
}
Here's a playground link.
This isn't possible because arrays do not implement any traits. You can only collect into types which implement the FromIterator trait (see the list at the bottom of its docs).
This is a language limitation, since it's currently impossible to be generic over the length of an array and the length is part of its type. But, even if it were possible, it's very unlikely that FromIterator would be implemented on arrays because it'd have to panic if the number of items yielded wasn't exactly the length of the array.
You may combine arrays map method with Iterator::next.
Example:
fn iter_to_array<Element, const N: usize>(mut iter: impl Iterator<Item = Element>) -> [Element; N] {
// Here I use `()` to make array zero-sized -> no real use in runtime.
// `map` creates new array, which we fill by values of iterator.
let res = [(); N].map(|_| iter.next().unwrap());
// Ensure that iterator finished
assert!(matches!(iter.next(), None));
res
}
I ran into this problem myself — here's a workaround.
You can't use FromIterator, but you can iterate over the contents of a fixed-size object, or, if things are more complicated, indices that slice anything that can be accessed. Either way, mutation is viable.
For example, the problem I had was with an array of type [[usize; 2]; 4]:
fn main() {
// Some input that could come from another function and thus not be mutable
let pairs: [[usize; 2]; 4] = [[0, 0], [0, 1], [1, 1], [1, 0]];
// Copy mutable
let mut foo_pairs = pairs.clone();
for pair in foo_pairs.iter_mut() {
// Do some operation or other on the fixed-size contents of each
pair[0] += 1;
pair[1] -= 1;
}
// Go forth and foo the foo_pairs
}
If this is happening inside a small function, it's okay in my book. Either way, you were going to end up with a transformed value of identical type as the same one, so copying the whole thing first and then mutating is about the same amount of effort as referencing a value in a closure and returning some function of it.
Note that this only works if you plan to compute something that is going to be the same type, up to and including size/length. But that's implied by your use of Rust arrays. (Specifically, you could Value() your Foos or Nothing them as you like, and still be within type parameters for your array.)
I have a program where I need to append two Vec<u8> before they are are serialized.
Just to be sure how to do it, I made this example program:
let a: Vec<u8> = vec![1, 2, 3, 4, 5, 6];
let b: Vec<u8> = vec![7, 8, 9];
let c = [a, b].concat();
println!("{:?}", c);
Which works perfectly. The issue is now when I have to implement it in my own project.
Here I need to write a function, the function takes a struct as input that looks like this:
pub struct Message2 {
pub ephemeral_key_r: Vec<u8>,
pub c_r: Vec<u8>,
pub ciphertext2: Vec<u8>,
}
and the serialalization function looks like this:
pub fn serialize_message_2(msg: &Message2) -> Result<Vec<u8>> {
let c_r_and_ciphertext = [msg.c_r, msg.ciphertext2].concat();
let encoded = (
Bytes::new(&msg.ephemeral_key_r),
Bytes::new(&c_r_and_ciphertext),
);
Ok(cbor::encode_sequence(encoded)?)
}
The first issue that arises here is that it complains that msg.ciphertext2 and msg.c_r are moved values. This makes sense, so I add an & in front of both of them.
However, when I do this, the call to concat() fails, with this type error:
util.rs(77, 59): method cannot be called on `[&std::vec::Vec<u8>; 2]` due to unsatisfied trait bounds
So, when I borrow the values, then the expression [&msg.c_r, &msg.ciphertext2] becomes an array of two vec's, which there is not a concat() defined for.
I also tried calling clone on both vectors:
let c_r_and_ciphertext = [msg.c_r.clone(), msg.ciphertext2.clone()].concat();
and this actually works out!
But now I'm just wondering, why does borrowing the values change the types?
and is there any things to think about when slapping on clone to values that are moved, and where I cannot borrow for some reason?
The reasons on why .concat() behaves as it does are a bit awkward.
To be able to call .concat(), the Concat trait must be implemented. It is implemented on slices of strings, and slices of V, where V can be Borrowed as slices of copyable T.
First, you're calling concat on an array, not a slice. However, auto-borrowing and unsize coercion are applied when calling a function with .. This turns the [V; 2] into a &[V] (where V = Vec<u8> in the working case and V = &Vec<u8> in the non-workin case). Try calling Concat::concat([a, b]) and you'll notice the difference.
So now is the question whether V can be borrowed as/into some &[T] (where T = u8 in your case). Two possibilities exist:
There is an impl<T> Borrow<[T]> for Vec<T>, so Vec<u8> can be turned into &[u8].
There is an impl<'_, T> Borrow<T> for &'_ T, so if you already have a &[u8], that can be used.
However, there is no impl<T> Borrow<[T]> for &'_ Vec<T>, so concatting [&Vec<_>] won't work.
So much for the theory, on the practical side: You can avoid the clones by using [&msg.c_r[..], &msg.ciphertext2[..]].concat(), because you'll be calling concat on &[&[u8]]. The &x[..] is a neat trick to turn the Vecs into slices (by slicing it, without slicing anything off…). You can also do that with .borrow(), but that's a bit more awkward, since you may need an extra type specification: [msg.c_r.borrow(), msg.ciphertext2.borrow()].concat::<u8>()
I tried to reproduce your error message, which this code does:
fn main() {
let a = vec![1, 2];
let b = vec![3, 4];
println!("{:?}", [&a, &b].concat())
}
gives:
error[E0599]: the method `concat` exists for array `[&Vec<{integer}>; 2]`, but its trait bounds were not satisfied
--> src/main.rs:4:31
|
4 | println!("{:?}", [&a, &b].concat())
| ^^^^^^ method cannot be called on `[&Vec<{integer}>; 2]` due to unsatisfied trait bounds
|
= note: the following trait bounds were not satisfied:
`[&Vec<{integer}>]: Concat<_>`
It is a simple matter of helping the compiler to see that &a works perfectly fine as a slice, by calling it &a[..]:
fn main() {
let a = vec![1, 2];
let b = vec![3, 4];
println!("{:?}", [&a[..], &b[..]].concat())
}
why does borrowing the values change the types?
Borrowing changes a type into a reference to that same type, so T to &T. These types are related, but are not the same.
is there any things to think about when slapping on clone to values that are moved, and where I cannot borrow for some reason?
Cloning is a good way to sacrifice performance to make the borrow checker happy. It (usually) involves copying the entire memory that is cloned, but if your code is not performance critical (which most code is not), then it may still be a good trade-off...
I want to write a function in Rust that will return the vector composed of start integer, then all intermediate integers and then end integer. The assertion it should hold is this:
assert_eq!(intervals(0, 4, 1..4), vec![0, 1, 2, 3, 4]);
The hint is to use chain method for iterators. The function declaration is predefined, I implemented it in one way, which is the following code:
pub fn intervals< I>(start: u32, end: u32, intermediate: I) -> Vec<u32>
where
I: IntoIterator<Item = u32>,
{
let mut a1 = vec![];
a1.push(start);
let inter: Vec<u32> = intermediate.into_iter().collect();
let mut iter : Vec<u32> = a1.iter().chain(inter.iter()).map(|x| *x).collect();
iter.push(end);
return iter;
}
But I am quite convinced this is not really optimal way to do this. I am sure I am doing lots of unnecessary things in the middle two lines. I tried to use intermediate directly like this:
let mut iter: Vec<u32> = a1.iter().chain(intermediate).map(|x| *x).collect();
But I am getting this error for chain method and I don't know how to solve it:
type mismatch resolving <I as std::iter::IntoIterator>::Item==&u32,
expected u32, found &u32
I am super new in Rust so any advice would be helpful to understand what's the right way to use intermediate parameter here.
Here are a few hints:
You have created three separate vectors (one explicitly, two using collect) when in fact you only need one.
You can use the std::iter::once iterator to produce iterators for the start and end integers
No need to collect the intermediate range. The intermediate argument implements IntoIterator, so you can feed it directly to chain. So, you can chain together the start, intermediate and end.
No need to use the 'return' keyword at the end of a function - the result of a function is the value of the last expression in it (as long as there is no semicolon on the end).
Applying those tips your function would look like this:
use std::iter::once;
pub fn intervals< I>(start: u32, end: u32, intermediate: I) -> Vec<u32>
where
I: IntoIterator<Item = u32>,
{
once(start).chain(intermediate).chain(once(end)).collect()
}
One additional thing to note, to answer your question from the comments:
why trying this: a1.iter().chain(intermediate) gives an error with chain method
Calling Vec::iter() returns an iterator that returns references to the values in the vector. This makes sense: calling iter() does not consume the vector, and its contents remain intact: you could iterate over it multiple times if you wanted.
On the other hand, invoking into_iter() from the IntoIterator trait returns an iterator that returns the values. This also makes sense: into_iter() does consume the object you are calling it on, so the iterator then takes ownership of the items that were previously owned by the object.
Trying to chain together two such iterators does not work because they are each iterating different types. One resolution would be to consume a1 as well, like this:
let mut iter : Vec<u32> = a1.into_iter().chain(intermediate).collect();
Why does the following code work?
use std::rc::Rc;
fn main () {
let c = vec![1, 2, 3, 4, 5];
let r = Rc::new(c);
println!("{:?}", (**r)[0]);
}
I can understand it working with single deference (println!("{:?}", (*r)[0]);). But not able to understand it working with double-dereference too.
Both, Rc and Vec implements Deref, whichs deref-method is called with the *.
let c = vec![1, 2, 3, 4, 5];
creates a Vec with the given elements with the vec!-macro.
let r = Rc::new(c);
creates a Reference counted Object from the Vector. The Vector is moved into the RC.
println!("{:?}", (**r)[0]);
This one is a bit more tricky: *r dereferences the Rc, so we get the underlying Vector. *rc dereferences the Vector as a slice. slice[0] indexes the first element of the slice, which results in the first element 1. println! finally prints the result.
It might be easier to understand what happens once we build a function prototype around the expression (**r)[0]:
fn foo<T, U>(r: T) -> i32
where
T: Deref<Target=U>,
U: Deref<Target=[i32]>,
{
(**r)[0]
}
Playground
Rc<T>, as is typical for most smart containers in Rust, implements Deref so that it can be used as an ordinary reference to the underlying value. In turn, Vec<T> implements Deref so that it can be used as a slice (Target = [T]). The explicit dereferencing *, when performed twice, applies the two conversions in sequence.
Of course, usually you wouldn't need to do this, because Vec also implements the Index operator.
In this code:
fn unpack_u32(data: &[u8]) -> u32 {
assert_eq!(data.len(), 4);
let res = data[0] as u32 |
(data[1] as u32) << 8 |
(data[2] as u32) << 16 |
(data[3] as u32) << 24;
res
}
fn main() {
let v = vec![0_u8, 1_u8, 2_u8, 3_u8, 4_u8, 5_u8, 6_u8, 7_u8, 8_u8];
println!("res: {:X}", unpack_u32(&v[1..5]));
}
the function unpack_u32 accepts only slices of length 4. Is there any way to replace the runtime check assert_eq with a compile time check?
Yes, kind of. The first step is easy: change the argument type from &[u8] to [u8; 4]:
fn unpack_u32(data: [u8; 4]) -> u32 { ... }
But transforming a slice (like &v[1..5]) into an object of type [u8; 4] is hard. You can of course create such an array simply by specifying all elements, like so:
unpack_u32([v[1], v[2], v[3], v[4]]);
But this is rather ugly to type and doesn't scale well with array size. So the question is "How to get a slice as an array in Rust?". I used a slightly modified version of Matthieu M.'s answer to said question (playground):
fn unpack_u32(data: [u8; 4]) -> u32 {
// as before without assert
}
use std::convert::AsMut;
fn clone_into_array<A, T>(slice: &[T]) -> A
where A: Default + AsMut<[T]>,
T: Clone
{
assert_eq!(slice.len(), std::mem::size_of::<A>()/std::mem::size_of::<T>());
let mut a = Default::default();
<A as AsMut<[T]>>::as_mut(&mut a).clone_from_slice(slice);
a
}
fn main() {
let v = vec![0_u8, 1, 2, 3, 4, 5, 6, 7, 8];
println!("res: {:X}", unpack_u32(clone_into_array(&v[1..5])));
}
As you can see, there is still an assert and thus the possibility of runtime failure. The Rust compiler isn't able to know that v[1..5] is 4 elements long, because 1..5 is just syntactic sugar for Range which is just a type the compiler knows nothing special about.
I think the answer is no as it is; a slice doesn't have a size (or minimum size) as part of the type, so there's nothing for the compiler to check; and similarly a vector is dynamically sized so there's no way to check at compile time that you can take a slice of the right size.
The only way I can see for the information to be even in principle available at compile time is if the function is applied to a compile-time known array. I think you'd still need to implement a procedural macro to do the check (so nightly Rust only, and it's not easy to do).
If the problem is efficiency rather than compile-time checking, you may be able to adjust your code so that, for example, you do one check for n*4 elements being available before n calls to your function; you could use the unsafe get_unchecked to avoid later redundant bounds checks. Obviously you'd need to be careful to avoid mistakes in the implementation.
I had a similar problem, creating a fixed byte-array on stack corresponding to const length of other byte-array (which may change during development time)
A combination of compiler plugin and macro was the solution:
https://github.com/frehberg/rust-sizedbytes