How to initialize an array of Vec? [duplicate] - rust

I’m trying to initialize a fixed-size array of some nullable, non-copyable type, like an Option<Box<Thing>> for some kind of Thing. I’d like to pack two of them into a struct without any extra indirection. I’d like to write something like this:
let array: [Option<Box<Thing>>; SIZE] = [None; SIZE];
But it doesn’t work because the [e; n] syntax requires that e implements Copy. Of course, I could expand it into SIZE Nones, but that can be unwieldy when SIZE is large. I don’t believe this can be done with a macro without an unnatural encoding of SIZE. Is there a good way to do it?
Yes, this is easy with unsafe; is there a way to do it without unsafe?

As of Rust 1.38 (released in September 2019), a cleaner alternative to previously posted answers is possible using an intermediate const initializer. This approach works for arrays of any size:
const SIZE: usize = 100;
const INIT: Option<Box<Thing>> = None;
let array: [Option<Box<Thing>>; SIZE] = [INIT; SIZE];
(It works with or without the Box; the example uses Box because it was used in the question.)
One limitation is that the array item must have a default representation that can be evaluated at compile time - a constant, enum variant, or a primitive container composed of those. None or a tuple of numbers will work, but a non-empty Vec or String won't.

You could use the Default trait to initialize the array with default values:
let array: [Option<Box<Thing>>; SIZE] = Default::default();
See this playground for a working example.
Note that this will only work for arrays with up to 32 elements, because Default::default is only implemented for up to [T; 32]. See https://doc.rust-lang.org/std/default/trait.Default.html#impl-Default-for-%5BT%3B%2032%5D.

As of Rust 1.55.0 (which introduced [T]::map()), the following will work:
const SIZE: usize = 100;
#[derive(Debug)]
struct THING { data: i64 }
let array = [(); SIZE].map(|_| Option::<THING>::default());
for x in array {
println!("x: {:?}", x);
}
Rust Playground

I'm copying the answer by chris-morgan and adapting it to match the question better, to follow the recommendation by dbaupp downthread, and to match recent syntax changes:
use std::mem;
use std::ptr;
#[derive(Debug)]
struct Thing {
number: usize,
}
macro_rules! make_array {
($n:expr, $constructor:expr) => {{
let mut items: [_; $n] = mem::uninitialized();
for (i, place) in items.iter_mut().enumerate() {
ptr::write(place, $constructor(i));
}
items
}}
}
const SIZE: usize = 50;
fn main() {
let items = unsafe { make_array!(SIZE, |i| Box::new(Some(Thing { number: i }))) };
println!("{:?}", &items[..]);
}
Note the need to use unsafe here: The problem is that if the constructor function panic!s, this would lead to undefined behavior.

Go through the heap
If you can create a Vec of your type, you can convert it into an array:
use std::convert::TryInto;
#[derive(Clone)]
struct Thing;
const SIZE: usize = 100;
fn main() {
let v: Vec<Option<Thing>> = vec![None; SIZE];
let v: Box<[Option<Thing>; SIZE]> = match v.into_boxed_slice().try_into() {
Ok(v) => v,
Err(_) => unreachable!(),
};
let v: [Option<Thing>; SIZE] = *v;
}
In many cases, you actually want to leave it as a Vec<T>, Box<[T]>, or Box<[T; N]> as these types all put the data in the heap. Large arrays tend to be... large... and you don't want all that data on the stack.
See also:
What is the use of into_boxed_slice() methods?
How to get a slice as an array in Rust?
How do I get an owned value out of a `Box`?
Keep it simple
Type out all the values:
struct Thing;
const SIZE: usize = 5;
fn main() {
let array: [Option<Box<Thing>>; SIZE] = [None, None, None, None, None];
}
You could use a build script to generate this code for you. For an example of this, see:
How to create a static string at compile time

An alternative approach using the arrayvec crate that generalizes easily to situations other than initializing everything with a fixed value:
use arrayvec::ArrayVec;
let array = std::iter::repeat(None)
.take(SIZE)
.collect::<ArrayVec<Option<Box<Thing>>, SIZE>>()
.into_inner()
.unwrap();
(playground)

On nightly Rust, you can use inline const. This is a variant of the answer by #user4815162342, but one that doesn't require you to declare a separate constant and repeat the type:
#![feature(inline_const)]
let array: [Option<Box<Thing>>; SIZE] = [const { None }; SIZE];
Until this is stabilized (hopefully soon), you can also use the inline-const crate, but this does require you to repeat the type.

let stackoverflow: [Option<&mut ()>;0xDEADBEEF] = std::array::from_fn(|_| None);
dbg!(stackoverflow);
playground

Related

How to create a Box<UnsafeCell<[T]>>

The recommended way to create a regular boxed slice (i.e. Box<[T]>) seems to be to first create a std::Vec<T>, and use .into_boxed_slice(). However, nothing similar to this seems to work if I want the slice to be wrapped in UnsafeCell.
A solution with unsafe code is fine, but I'd really like to avoid having to manually manage the memory.
The only (not-unsafe) way to create a Box<[T]> is via Box::from, given a &[T] as the parameter. This is because [T] is ?Sized and can't be passed a parameter. This in turn effectively requires T: Copy, because T has to be copied from behind the reference into the new Box. But UnsafeCell is not Copy, regardless if T is. Discussion about making UnsafeCell Copy has been going on for years, yielding no final conclusion, due to safety concerns.
If you really, really want a Box<UnsafeCell<[T]>>, there are only two ways:
Because Box and UnsafeCell are both CoerceUnsize, and [T; N] is Unsize, you can create a Box<UnsafeCell<[T; N]>> and coerce it to a Box<UnsafeCell<[T]>. This limits you to initializing from fixed-sized arrays.
Unsize coercion:
fn main() {
use std::cell::UnsafeCell;
let x: [u8;3] = [1,2,3];
let c: Box<UnsafeCell<[_]>> = Box::new(UnsafeCell::new(x));
}
Because UnsafeCell is #[repr(transparent)], you can create a Box<[T]> and unsafely mutate it to a Box<UnsafeCell<[T]>, as the UnsafeCell<[T]> is guaranteed to have the same memory layout as a [T], given that [T] doesn't use niche-values (even if T does).
Transmute:
// enclose the transmute in a function accepting and returning proper type-pairs
fn into_boxed_unsafecell<T>(inp: Box<[T]>) -> Box<UnsafeCell<[T]>> {
unsafe {
mem::transmute(inp)
}
}
fn main() {
let x = vec![1,2,3];
let b = x.into_boxed_slice();
let c: Box<UnsafeCell<[_]>> = into_boxed_unsafecell(b);
}
Having said all this: I strongly suggest you are suffering from the xy-problem. A Box<UnsafeCell<[T]>> is a very strange type (especially compared to UnsafeCell<Box<[T]>>). You may want to give details on what you are trying to accomplish with such a type.
Just swap the pointer types to UnsafeCell<Box<[T]>>:
use std::cell::UnsafeCell;
fn main() {
let mut res: UnsafeCell<Box<[u32]>> = UnsafeCell::new(vec![1, 2, 3, 4, 5].into_boxed_slice());
unsafe {
println!("{}", (*res.get())[1]);
res.get_mut()[1] = 10;
println!("{}", (*res.get())[1]);
}
}
Playground

Extract original slice from SliceStorage and SliceStorageMut

I am working on some software where I am managing a buffer of floats in a Vec<T> where T is either an f32 or f64. I sometimes need to interpret this buffer, or sections of it, as a mathematical vector. To this end, I am taking advantage of MatrixSlice and friends in nalgebra.
I can create a DVectorSliceMut, for example, the following way
fn as_vector<'a>(slice: &'a mut [f64]) -> DVectorSliceMut<'a, f64> {
DVectorSliceMut::from(slice)
}
However, sometimes I need to later extract the original slice from the DVectorSliceMut with the original lifetime 'a. Is there a way to do this?
The StorageMut trait has a as_mut_slice member function, but the lifetime of the returned slice is the lifetime of the reference to the Storage implementor, not the original slice. I am okay with a solution which consumes the DVectorSliceMut if necessary.
Update: Methods into_slice and into_slice_mut have been respectively added to the SliceStorage and SliceStorageMut traits as of nalgebra v0.28.0.
Given the current API of nalgebra (v0.27.1) there isn't much that you can do, except:
life with the shorter life-time of StorageMut::as_mut_slice
make a feature request for such a function at nalgebra (which seems you already did)
employ your own unsafe code to make StorageMut::ptr_mut into a &'a mut
You could go with the third option until nalgebra gets update and implement something like this in your own code:
use nalgebra::base::dimension::Dim;
use nalgebra::base::storage::Storage;
use nalgebra::base::storage::StorageMut;
fn into_slice<'a>(vec: DVectorSliceMut<'a, f64>) -> &'a mut [f64] {
let mut inner = vec.data;
// from nalgebra
// https://docs.rs/nalgebra/0.27.1/src/nalgebra/base/matrix_slice.rs.html#190
let (nrows, ncols) = inner.shape();
if nrows.value() != 0 && ncols.value() != 0 {
let sz = inner.linear_index(nrows.value() - 1, ncols.value() - 1);
unsafe { core::slice::from_raw_parts_mut(inner.ptr_mut(), sz + 1) }
} else {
unsafe { core::slice::from_raw_parts_mut(inner.ptr_mut(), 0) }
}
}
Methods into_slice and into_slice_mut which return the original slice have been respectively added to the SliceStorage and SliceStorageMut traits as of nalgebra v0.28.0.

How do you write an iterator to a File or Stdout (anything implementing Write)?

I have an iterator full of u8's that I want to write to a file or stdout. Calling io::stdout().write_all(foo) on my iterator gives me an expected type &[u8], and that I have an iterator type instead. I understand why this doesn't work.
What I don't understand is how to change it so that it does work. At first I tried adding a .collect() to the end of my iterator, but then it says the trait bound &[u8]: std::iter::FromIterator<u8> is not satisfied and a collection of type &[u8] cannot be built from an iterator over elements of type u8.
It seems odd to me that Write doesn't provide a way of writing with an iterator, when Read provides a bytes() function that returns an iterator. What is the idiomatic way to do this?
Here is my main function's contents:
io::stdout().write_all(
io::stdin().bytes().map(
|x| match x {
Ok(b) => b,
_ => panic!("There was an error reading from stdin"),
}
).repeat(3).collect()
);
It seems odd to me that Write doesn't provide a way of writing with an iterator, when Read provides a bytes() function that returns an iterator. What is the idiomatic way to do this?
It does feel inconsistent, indeed, but you can certainly write the equivalent yourself.
Something like:
fn write_all<W: Write, I: Iterator<Item=u8>>(writer: &mut W, iter: I) {
const SIZE: usize = 1024;
let mut buffer = [0u8; SIZE];
let mut index = 0;
for i in iter {
buffer[index] = i;
index += 1;
if index == SIZE {
writer.write_all(&buffer);
}
}
writer.write_all(&buffer[..index]);
}
There are probably ways to make it more idiomatic, and I haven't tested the boundary conditions, but hopefully it should get you going.
The problem is that you want to build a Vec<u8>, but are attempting to create a Vec<&u8>. You can do the following:
fn main() {
let array = [1u8, 2, 3];
let vector: Vec<u8> = array.iter().map(|i| *i).collect();
}
Note the .map(|i| *i) part that allows to go from &u8 references to u8 values.

Initialize a large, fixed-size array with non-Copy types

I’m trying to initialize a fixed-size array of some nullable, non-copyable type, like an Option<Box<Thing>> for some kind of Thing. I’d like to pack two of them into a struct without any extra indirection. I’d like to write something like this:
let array: [Option<Box<Thing>>; SIZE] = [None; SIZE];
But it doesn’t work because the [e; n] syntax requires that e implements Copy. Of course, I could expand it into SIZE Nones, but that can be unwieldy when SIZE is large. I don’t believe this can be done with a macro without an unnatural encoding of SIZE. Is there a good way to do it?
Yes, this is easy with unsafe; is there a way to do it without unsafe?
As of Rust 1.38 (released in September 2019), a cleaner alternative to previously posted answers is possible using an intermediate const initializer. This approach works for arrays of any size:
const SIZE: usize = 100;
const INIT: Option<Box<Thing>> = None;
let array: [Option<Box<Thing>>; SIZE] = [INIT; SIZE];
(It works with or without the Box; the example uses Box because it was used in the question.)
One limitation is that the array item must have a default representation that can be evaluated at compile time - a constant, enum variant, or a primitive container composed of those. None or a tuple of numbers will work, but a non-empty Vec or String won't.
You could use the Default trait to initialize the array with default values:
let array: [Option<Box<Thing>>; SIZE] = Default::default();
See this playground for a working example.
Note that this will only work for arrays with up to 32 elements, because Default::default is only implemented for up to [T; 32]. See https://doc.rust-lang.org/std/default/trait.Default.html#impl-Default-for-%5BT%3B%2032%5D.
As of Rust 1.55.0 (which introduced [T]::map()), the following will work:
const SIZE: usize = 100;
#[derive(Debug)]
struct THING { data: i64 }
let array = [(); SIZE].map(|_| Option::<THING>::default());
for x in array {
println!("x: {:?}", x);
}
Rust Playground
I'm copying the answer by chris-morgan and adapting it to match the question better, to follow the recommendation by dbaupp downthread, and to match recent syntax changes:
use std::mem;
use std::ptr;
#[derive(Debug)]
struct Thing {
number: usize,
}
macro_rules! make_array {
($n:expr, $constructor:expr) => {{
let mut items: [_; $n] = mem::uninitialized();
for (i, place) in items.iter_mut().enumerate() {
ptr::write(place, $constructor(i));
}
items
}}
}
const SIZE: usize = 50;
fn main() {
let items = unsafe { make_array!(SIZE, |i| Box::new(Some(Thing { number: i }))) };
println!("{:?}", &items[..]);
}
Note the need to use unsafe here: The problem is that if the constructor function panic!s, this would lead to undefined behavior.
Go through the heap
If you can create a Vec of your type, you can convert it into an array:
use std::convert::TryInto;
#[derive(Clone)]
struct Thing;
const SIZE: usize = 100;
fn main() {
let v: Vec<Option<Thing>> = vec![None; SIZE];
let v: Box<[Option<Thing>; SIZE]> = match v.into_boxed_slice().try_into() {
Ok(v) => v,
Err(_) => unreachable!(),
};
let v: [Option<Thing>; SIZE] = *v;
}
In many cases, you actually want to leave it as a Vec<T>, Box<[T]>, or Box<[T; N]> as these types all put the data in the heap. Large arrays tend to be... large... and you don't want all that data on the stack.
See also:
What is the use of into_boxed_slice() methods?
How to get a slice as an array in Rust?
How do I get an owned value out of a `Box`?
Keep it simple
Type out all the values:
struct Thing;
const SIZE: usize = 5;
fn main() {
let array: [Option<Box<Thing>>; SIZE] = [None, None, None, None, None];
}
You could use a build script to generate this code for you. For an example of this, see:
How to create a static string at compile time
An alternative approach using the arrayvec crate that generalizes easily to situations other than initializing everything with a fixed value:
use arrayvec::ArrayVec;
let array = std::iter::repeat(None)
.take(SIZE)
.collect::<ArrayVec<Option<Box<Thing>>, SIZE>>()
.into_inner()
.unwrap();
(playground)
On nightly Rust, you can use inline const. This is a variant of the answer by #user4815162342, but one that doesn't require you to declare a separate constant and repeat the type:
#![feature(inline_const)]
let array: [Option<Box<Thing>>; SIZE] = [const { None }; SIZE];
Until this is stabilized (hopefully soon), you can also use the inline-const crate, but this does require you to repeat the type.
let stackoverflow: [Option<&mut ()>;0xDEADBEEF] = std::array::from_fn(|_| None);
dbg!(stackoverflow);
playground

How to achieve equivalent of take_while on a slice?

Rust slices do not currently support some iterator methods, i.e. take_while. What is the best way to implement take_while for slices?
const STRHELLO:&'static[u8] = b"HHHello";
fn main() {
let subslice:&[u8] = STRHELLO.iter().take_while(|c|(**c=='H' as u8)).collect();
println!("Expecting: {}, Got {}",STRHELLO.slice_to(3),subslice);
assert!(subslice==STRHELLO.slice_to(3));
}
results in the error:
<anon>:6:74: 6:83 error: the trait `core::iter::FromIterator<&u8>` is not implemented for the type `&[u8]`
This code in the playpen:
http://is.gd/1xkcUa
First of all, the issue you have is that collect is about creating a new collection, while a slice is about referencing a contiguous range of items in an existing array (be it dynamically allocated or not).
I am afraid that due to the nature of traits, the fact that the original container (STRHELLO) was a contiguous range has been lost, and cannot be reconstructed after the fact. I am also afraid that any use of "generic" iterators simply cannot lead to the desired output; the type system would have to somehow carry the fact that:
the original container was a contiguous range
the chain of operations performed so far conserve this property
This may be doable or not, but I do not see it done now, and I am unsure in what way it could be elegantly implemented.
On the other hand, you can go about it in the do-it-yourself way:
fn take_while<'a>(initial: &'a [u8], predicate: |&u8| -> bool) -> &'a [u8] { // '
let mut i = 0u;
for c in initial.iter() {
if predicate(c) { i += 1; } else { break; }
}
initial.slice_to(i)
}
And then:
fn main() {
let subslice: &[u8] = take_while(STRHELLO, |c|(*c==b'H'));
println!("Expecting: {}, Got {}",STRHELLO.slice_to(3), subslice);
assert!(subslice == STRHELLO.slice_to(3));
}
Note: 'H' as u8 can be rewritten as b'H' as show here, which is symmetric with the strings.
It is possible via some heavy gymnastics to implement this functionality using the stock iterators:
use std::raw::Slice;
use std::mem::transmute;
/// Splice together to slices of the same type that are contiguous in memory.
/// Panics if the slices aren't contiguous with "a" coming first.
/// i.e. slice b must follow slice a immediately in memory.
fn splice<'a>(a:&'a[u8], b:&'a[u8]) -> &'a[u8] {
unsafe {
let aa:Slice<u8> = transmute(a);
let bb:Slice<u8> = transmute(b);
let pa = aa.data as *const u8;
let pb = bb.data as *const u8;
let off = aa.len as int; // Risks overflow into negative!!!
assert!(pa.offset(off) == pb, "Slices were not contiguous!");
let cc = Slice{data:aa.data,len:aa.len+bb.len};
transmute(cc)
}
}
/// Wrapper around splice that lets you use None as a base case for fold
/// Will panic if the slices cannot be spliced! See splice.
fn splice_for_fold<'a>(oa:Option<&'a[u8]>, b:&'a[u8]) -> Option<&'a[u8]> {
match oa {
Some(a) => Some(splice(a,b)),
None => Some(b),
}
}
/// Implementaton using pure iterators
fn take_while<'a>(initial: &'a [u8],
predicate: |&u8| -> bool) -> Option<&'a [u8]> {
initial
.chunks(1)
.take_while(|x|(predicate(&x[0])))
.fold(None, splice_for_fold)
}
usage:
const STRHELLO:&'static[u8] = b"HHHello";
let subslice: &[u8] = super::take_while(STRHELLO, |c|(*c==b'H')).unwrap();
println!("Expecting: {}, Got {}",STRHELLO.slice_to(3), subslice);
assert!(subslice == STRHELLO.slice_to(3));
Matthieu's implementation is way cleaner if you just need take_while. I am posting this anyway since it may be a path towards solving the more general problem of using iterator functions on slices cleanly.

Resources