How do I shuffle a VecDeque? - rust

I can shuffle a regular vector quite simply like this:
extern crate rand;
use rand::Rng;
fn shuffle(coll: &mut Vec<i32>) {
rand::thread_rng().shuffle(coll);
}
The problem is, my code now requires the use of a std::collections::VecDeque instead, which causes this code to not compile.
What's the simplest way of getting around this?

As of Rust 1.48, VecDeque supports the make_contiguous() method. That method doesn't allocate and has complexity of O(n), like shuffling itself. Therefore you can shuffle a VecDeque by calling make_contiguous() and then shuffling the returned slice:
use rand::prelude::*;
use std::collections::VecDeque;
pub fn shuffle<T>(v: &mut VecDeque<T>, rng: &mut impl Rng) {
v.make_contiguous().shuffle(rng);
}
Playground
Historical answer follows below.
Unfortunately, the rand::Rng::shuffle method is defined to shuffle slices. Due to its own complexity constraints a VecDeque cannot store its elements in a slice, so shuffle can never be directly invoked on a VecDeque.
The real requirement of the values argument to shuffle algorithm are finite sequence length, O(1) element access, and the ability to swap elements, all of which VecDeque fulfills. It would be nice if there were a trait that incorporates these, so that values could be generic on that, but there isn't one.
With the current library, you have two options:
Use Vec::from(deque) to copy the VecDeque into a temporary Vec, shuffle the vector, and return the contents back to VecDeque. The complexity of the operation will remain O(n), but it will require a potentially large and costly heap allocation of the temporary vector.
Implement the shuffle on VecDeque yourself. The Fisher-Yates shuffle used by rand::Rng is well understood and easy to implement. While in theory the standard library could switch to a different shuffle algorithm, that is not likely to happen in practice.
A generic form of the second option, using a trait to express the len-and-swap requirement, and taking the code of rand::Rng::shuffle, could look like this:
use std::collections::VecDeque;
// Real requirement for shuffle
trait LenAndSwap {
fn len(&self) -> usize;
fn swap(&mut self, i: usize, j: usize);
}
// A copy of an earlier version of rand::Rng::shuffle, with the signature
// modified to accept any type that implements LenAndSwap
fn shuffle(values: &mut impl LenAndSwap, rng: &mut impl rand::Rng) {
let mut i = values.len();
while i >= 2 {
// invariant: elements with index >= i have been locked in place.
i -= 1;
// lock element i in place.
values.swap(i, rng.gen_range(0..=i));
}
}
// VecDeque trivially fulfills the LenAndSwap requirement, but
// we have to spell it out.
impl<T> LenAndSwap for VecDeque<T> {
fn len(&self) -> usize {
self.len()
}
fn swap(&mut self, i: usize, j: usize) {
self.swap(i, j)
}
}
fn main() {
let mut v: VecDeque<u64> = [1, 2, 3, 4].into_iter().collect();
shuffle(&mut v, &mut rand::thread_rng());
println!("{:?}", v);
}

You can use make_contiguous (documentation) to create a mutable slice that you can then shuffle:
use rand::prelude::*;
use std::collections::VecDeque;
fn main() {
let mut deque = VecDeque::new();
for p in 0..10 {
deque.push_back(p);
}
deque.make_contiguous().shuffle(&mut rand::thread_rng());
println!("Random deque: {:?}", deque)
}
Playground Link if you want to try it out online.

Shuffle the components of the VecDeque separately, starting with VecDeque.html::as_mut_slices:
use rand::seq::SliceRandom; // 0.6.5;
use std::collections::VecDeque;
fn shuffle(coll: &mut VecDeque<i32>) {
let mut rng = rand::thread_rng();
let (a, b) = coll.as_mut_slices();
a.shuffle(&mut rng);
b.shuffle(&mut rng);
}
As Lukas Kalbertodt points out, this solution never swaps elements between the two slices so a certain amount of randomization will not happen. Depending on your needs of randomization, this may be unnoticeable or a deal breaker.

Related

Can I implement Index/IndexMut for a type that has locked data?

I've got a struct that contains some locked data. The real world is complex, but here's a minimal example (or as minimal as I can make it):
use std::fmt::Display;
use std::ops::{Index, IndexMut};
use std::sync::Mutex;
struct LockedVector<T> {
stuff: Mutex<Vec<T>>,
}
impl<T> LockedVector<T> {
pub fn new(v: Vec<T>) -> Self {
LockedVector {
stuff: Mutex::new(v),
}
}
}
impl<T> Index<usize> for LockedVector<T> {
type Output = T;
fn index(&self, index: usize) -> &Self::Output {
todo!()
}
}
impl<T> IndexMut<usize> for LockedVector<T> {
fn index_mut(&mut self, index: usize) -> &mut Self::Output {
let thing = self.stuff.get_mut().unwrap();
&mut thing[index]
}
}
impl<T: Display> Display for LockedVector<T> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let strings: Vec<String> = self
.stuff
.lock()
.unwrap()
.iter()
.map(|s| format!("{}", s))
.collect();
write!(f, "{}", strings.join(", "))
}
}
fn main() {
let mut my_stuff = LockedVector::new(vec![0, 1, 2, 3, 4]);
println!("initially: {}", my_stuff);
my_stuff[2] = 5;
println!("then: {}", my_stuff);
let a_mut_var: &mut usize = &mut my_stuff[3];
*a_mut_var = 54;
println!("Still working: {}", my_stuff);
}
What I'm trying to do here is implement the Index and IndexMut traits on a struct, where the data being indexed is behind a Mutex lock. My very fuzzy reasoning for why this should be possible is that the result of locking a mutex is sort-of like a reference, and it seems like you could map a reference onto another reference, or somehow make a sort of reference that wraps the entire lock but only de-references the specific index.
My much less fuzzy reasoning is that the code above compiles and runs (note the todo!) - I'm able to get back mutable references, and I assume I haven't somehow snuck past the mutex in an unthread-safe way. (I made an attempt to test the threaded behavior, but ran into other issues trying to get a mutable reference into another thread at all.)
The weird issue is, I can't do the same for Index - there is no get_immut() I can use, and I haven't found another approach. I can get a mutable reference out of Mutex, but not an immutable one (and of course, I can't get the mutable one if I only have an immutable reference to begin with)
My expectation is that indexing would acquire a lock, and the returned reference (in both mutable and immutable cases) would maintain the lock for their lifetimes. As a bonus, it would be nice if RwLock-ed things could only grab/hold the read lock for the immutable cases, and the write lock for mutable ones.
For context as to why I'd do this: I have a Grid trait that is used by a bunch of different code, but backed by different implementations, some of which are thread-safe. I was hoping to put the Index and IndexMut traits on it for the nice syntax. Threads don't generally have mutable references to the thread-safe Grids at all, so the IndexMut trait would see little use there, but I could see it being valuable during setup or for the non-thread-safe cases. The immutable Index behavior seems like it would be useful everywhere.
Bonus question: I absolutely hate that Display code, how can I make it less hideous?
If you look at the documentation of get_mut you'll see it's only possible precisely because a mutable reference ensures that there is no other reference or a lock to it, unfortunately for you that means that a get_ref for Mutex would only be possible by taking a mutable reference, that's just an artificially limited get_mut though.
Unfortunately for you since Index only gives you a shared reference you can't safely get a shared reference to it's contents, so you can't implement an Index so that it indexes into something behind a Mutex.

Panic if the capacity of a vector is increased

I am working on implementing a sieve of atkins as my first decently sized program in rust. This algorithm takes a number and returns a vector of all primes below that number. There are two different vectors I need use this function.
BitVec 1 for prime 0 for not prime (flipped back and forth as part of the algorithm).
Vector containing all known primes.
The size of the BitVec is known as soon as the function is called. While the final size of the vector containing all known primes is not known, there are relatively accurate upper limits for the number of primes in a range. Using these I can set the size of the vector to an upper bound then shrink_to_fit it before returning. The upshot of this neither array should ever need to have it's capacity increased while the algorithm is running, and if this happens something has gone horribly wrong with the algorithm.
Therefore, I would like my function to panic if the capacity of either the vector or the bitvec is changed during the running of the function. Is this possible and if so how would I be best off implementing it?
Thanks,
You can assert that the vecs capacity() and len() are different before each push:
assert_ne!(v.capacity(), v.len());
v.push(value);
If you want it done automatically you'd have to wrap your vec in a newtype:
struct FixedSizeVec<T>(Vec<T>);
impl<T> FixedSizeVec<T> {
pub fn push(&mut self, value: T) {
assert_ne!(self.0.len(), self.0.capacity())
self.0.push(value)
}
}
To save on forwarding unchanged methods you can impl Deref(Mut) for your newtype.
use std::ops::{Deref, DerefMut};
impl<T> Deref for FixedSizeVec<T> {
type Target = Vec<T>;
fn deref(&self) -> &Vec<T> {
&self.0
}
}
impl<T> DerefMut for FixedSizeVec<T> {
fn deref_mut(&mut self) -> &mut Vec<T> {
&mut self.0
}
}
An alternative to the newtype pattern is to create a new trait with a method that performs the check, and implement it for the vector like so:
trait PushCheck<T> {
fn push_with_check(&mut self, value: T);
}
impl<T> PushCheck<T> for std::vec::Vec<T> {
fn push_with_check(&mut self, value: T) {
let prev_capacity = self.capacity();
self.push(value);
assert!(prev_capacity == self.capacity());
}
}
fn main() {
let mut v = Vec::new();
v.reserve(4);
dbg!(v.capacity());
v.push_with_check(1);
v.push_with_check(1);
v.push_with_check(1);
v.push_with_check(1);
// This push will panic
v.push_with_check(1);
}
The upside is that you aren't creating a new type, but the obvious downside is you need to remember to use the newly defined method.

Extract original slice from SliceStorage and SliceStorageMut

I am working on some software where I am managing a buffer of floats in a Vec<T> where T is either an f32 or f64. I sometimes need to interpret this buffer, or sections of it, as a mathematical vector. To this end, I am taking advantage of MatrixSlice and friends in nalgebra.
I can create a DVectorSliceMut, for example, the following way
fn as_vector<'a>(slice: &'a mut [f64]) -> DVectorSliceMut<'a, f64> {
DVectorSliceMut::from(slice)
}
However, sometimes I need to later extract the original slice from the DVectorSliceMut with the original lifetime 'a. Is there a way to do this?
The StorageMut trait has a as_mut_slice member function, but the lifetime of the returned slice is the lifetime of the reference to the Storage implementor, not the original slice. I am okay with a solution which consumes the DVectorSliceMut if necessary.
Update: Methods into_slice and into_slice_mut have been respectively added to the SliceStorage and SliceStorageMut traits as of nalgebra v0.28.0.
Given the current API of nalgebra (v0.27.1) there isn't much that you can do, except:
life with the shorter life-time of StorageMut::as_mut_slice
make a feature request for such a function at nalgebra (which seems you already did)
employ your own unsafe code to make StorageMut::ptr_mut into a &'a mut
You could go with the third option until nalgebra gets update and implement something like this in your own code:
use nalgebra::base::dimension::Dim;
use nalgebra::base::storage::Storage;
use nalgebra::base::storage::StorageMut;
fn into_slice<'a>(vec: DVectorSliceMut<'a, f64>) -> &'a mut [f64] {
let mut inner = vec.data;
// from nalgebra
// https://docs.rs/nalgebra/0.27.1/src/nalgebra/base/matrix_slice.rs.html#190
let (nrows, ncols) = inner.shape();
if nrows.value() != 0 && ncols.value() != 0 {
let sz = inner.linear_index(nrows.value() - 1, ncols.value() - 1);
unsafe { core::slice::from_raw_parts_mut(inner.ptr_mut(), sz + 1) }
} else {
unsafe { core::slice::from_raw_parts_mut(inner.ptr_mut(), 0) }
}
}
Methods into_slice and into_slice_mut which return the original slice have been respectively added to the SliceStorage and SliceStorageMut traits as of nalgebra v0.28.0.

Using `flat_map` to create iterator over fields for slice of struct

Given a collection (vector/slice) of structs. How do I create a combined iterator over some fields in each struct?
Below is a concrete attempt using flat_map:
struct Game {
home_team: u8,
away_team: u8,
}
fn teams(games: &[Game]) -> impl Iterator<Item = u8> {
games
.iter()
.flat_map(|game| [game.home_team, game.away_team].iter().map(|x| x.clone()))
}
fn main() {
let data = &[
Game {
home_team: 1,
away_team: 2,
},
Game {
home_team: 1,
away_team: 3,
},
];
let non_unique_teams: Vec<u8> = teams(data).collect();
}
My actual use-case is very similar. In particular, the fields that form the basis of the iterator implements Copy, making cloning perfectly fine.
My intuition tells me that this should work since I'm cloning the only things I need to "take" from the incoming slice. Obviously, my understanding of the borrow checker is to poor for me to grasp this.
The iterator needs to own the memory containing the copies of the struct fields. In your code, you create a local array and call iter() on it, which results in an iterator over a slice reference that does not own the data.
The easiest way to make the iterator own the data is to allocate a vector for each struct:
fn teams(games: &[Game]) -> impl Iterator<Item = u8> + '_ {
games
.iter()
.flat_map(|game| vec![game.home_team, game.away_team])
}
This will result in a heap allocation in each iteration. The performance penalty is probably minimal, since the allocator is probably able to reuse the allocation in each iteration. However, if you want to avoid the allocation for some reason, you could also use a combination of Iterator::chain() and std::iter::once():
use std::iter::once;
fn teams(games: &[Game]) -> impl Iterator<Item = u8> + '_ {
games
.iter()
.flat_map(|game| once(game.home_team).chain(once(game.away_team)))
}
Further alternatives include implementing IntoIterator and Clone for Game, which would allow you to simply use games.iter().cloned().flatten(), the iter_vals crate or using generators, an unstable feature that makes implementing this kind of iterator more convenient.

Mutating and non-mutating method chains

I have two functions, which I'm hoping to use in method chains. They both do basically the same thing, except that one of them overwrites itself and another returns a clone. I'm coming from Ruby, and I'm used to just calling self.dup.mutable_method in the destructive method.
I believe I have a solution worked out in Rust, but I'm not sure whether it has an extra allocation going on in there somewhere, and I'm afraid that it'll consume itself. This is audio DSP code, so I want to make sure that there are no allocations in the mutable method. (I'm three days into Rust, so mea culpa for the non-generalized trait impls.)
impl Filter for DVec<f64> {
fn preemphasis_mut<'a>(&'a mut self, freq: f64, sample_rate: f64) -> &'a mut DVec<f64> {
let filter = (-2.0 * PI * freq / sample_rate).exp();
for i in (1..self.len()).rev() {
self[i] -= self[i-1] * filter;
};
self
}
fn preemphasis(&self, freq: f64, sample_rate: f64) -> DVec<f64> {
let mut new = self.clone();
new.preemphasis_mut(freq, sample_rate);
new
}
}
// Ideal code:
let mut sample: DVec<f64> = method_that_loads_sample();
let copy_of_sample = sample.preemphasis(75.0, 44100.0); // this mutates and copies, with one allocation
sample.preemphasis_mut(75.0, 44100.0); // this mutates in-place, with no allocations
copy_of_sample.preemphasis_mut(75.0, 44100.0)
.preemphasis_mut(150.0, 44100.0); // this mutates twice in a row, with no allocations
I have not seen any libraries follow any patterns similar to Ruby's foo and foo! method pairs when it comes to self mutation. I believe this mostly to be because Rust places mutability front-and-center, so it's much more difficult to "accidentally" mutate something. To that end, I would probably drop one of your methods and allow the user to decide when something should be mutated:
use std::f64::consts::PI;
trait Filter {
fn preemphasis<'a>(&'a mut self, freq: f64, sample_rate: f64) -> &'a mut Self;
}
impl Filter for Vec<f64> {
fn preemphasis<'a>(&'a mut self, freq: f64, sample_rate: f64) -> &'a mut Self {
let filter = (-2.0 * PI * freq / sample_rate).exp();
for i in (1..self.len()).rev() {
self[i] -= self[i-1] * filter;
};
self
}
}
fn main() {
let mut sample = vec![1.0, 2.0];
// this copies then mutates, with one allocation
let mut copy_of_sample = sample.clone();
copy_of_sample
.preemphasis(75.0, 44100.0);
// this mutates in-place, with no allocations
sample
.preemphasis(75.0, 44100.0);
// this mutates twice in a row, with no allocations
copy_of_sample
.preemphasis(75.0, 44100.0)
.preemphasis(150.0, 44100.0);
}
I think a key thing here is that the caller of the code can easily see when something will be mutated (because of the &mut reference to self). The caller also gets to determine when and where the clone happens.

Resources