Idiomatic way to memoize or lazy evaluate using `Cell` and `RefCell` - rust

The code below works. It evaluates x and ys lazily and caches into Foo::x: Cell, Foo::ys: RefCell respectively.
However, I feel there might be a better way to do it. I dislike I have to make a wrapper CacheVecGuard so that on the call site I can use self.borrow_ys() instead of the lengthy &self.ys.borrow().1.
How can I improve this piece of code?
Are there any canonical snippets to do lazy evaluation or memoization that is suitable in this case? (I am aware of lazy_static which doesn't fit)
use std::cell::{RefCell, Cell, Ref};
use std::ops::Deref;
struct CacheVecGuard<'a>(Ref<'a, (bool, Vec<f64>)>);
impl<'a> Deref for CacheVecGuard<'a> {
type Target = [f64];
fn deref(&self) -> &Self::Target {
&(self.0).1
}
}
fn pre_calculate_x(x: f64) -> f64 {
x
}
fn pre_calculate_ys(x: f64, ys: &mut [f64]) {
for i in 0..ys.len() {
ys[i] += 1.0;
}
}
struct Foo {
pub a: f64,
x: Cell<Option<f64>>,
ys: RefCell<(bool, Vec<f64>)>,
}
impl Foo {
pub fn new(a: f64) -> Self {
Self {
a,
x: Cell::new(None),
ys: RefCell::new((false, vec![0.0; 10])),
}
}
fn get_x(&self) -> f64 {
match self.x.get() {
None => {
let x = pre_calculate_x(self.a);
self.x.set(Some(x));
println!("Set x to {}", x);
x
}
Some(x) => x,
}
}
fn borrow_ys(&self) -> CacheVecGuard {
{
let (ref mut ready, ref mut ys) = *self.ys.borrow_mut();
if !*ready {
pre_calculate_ys(self.a, ys);
println!("Set ys to {:?}", ys);
*ready = true;
}
}
CacheVecGuard(self.ys.borrow())
}
fn clear_cache(&mut self) {
*(&mut self.ys.borrow_mut().0) = false;
self.x.set(None);
}
pub fn test(&self) -> f64 {
self.borrow_ys()[0] + self.get_x()
}
pub fn set_a(&mut self, a: f64) {
self.a = a;
self.clear_cache();
}
}
fn main() {
let mut foo = Foo::new(1.0);
println!("{}", foo.test());
foo.set_a(3.0);
println!("{}", foo.test());
}
It prints
Set ys to [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
Set x to 1
2
Set ys to [2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
Set x to 3
5
Playground

The fact that you need the ability to clear the cache means that you must have a guard. Otherwise, a call to set_a could invalidate a bare reference returned earlier by borrow_ys. The only way the compiler can verify that this doesn't happen is by returning a guard and borrowing from the guard instead.
If you can do away with the ability to clear the cache, you could use the LazyCell type from the lazycell crate instead.

Related

How to implement a "method override" on a struct (pass a closure to a struct?)

I'm new to Rust and still learning how to represent different designs patterns in the language, plus my recent background is all heavily OO design which is not helping think about the problem.
In the example below I want to eliminate the sort_ascending condition into something more elegant and ideally have it fully resolve at compile time.
struct MyList {
ladder: Vec<i32>,
sort_ascending: bool,
}
impl MyList {
pub fn new(data: &[i32], sort_ascending: bool) -> MyList {
return MyList {
ladder: data.to_vec(),
sort_ascending: sort_ascending,
};
}
pub fn get_best(&self) -> Option<&i32> {
if self.sort_ascending {
return self.ladder.iter().reduce(|a, b| if a >= b { a } else { b });
} else {
return self.ladder.iter().reduce(|a, b| if a <= b { a } else { b });
}
}
}
fn main() {
let x = MyList::new(&[10, 4, 30, 2, 5, 2], true);
let r = x.get_best();
println!("{:?}", r);
let x = MyList::new(&[10, 4, 30, 2, 5, 2], false);
let r = x.get_best();
println!("{:?}", r);
}
I think the solution lies in making the closure passed to reduce() configurable and have tried many variations along the following lines without success:
struct MyList<F>
where F : Fn(i32,i32) -> i32
{
ladder: Vec<i32>,
sort_closure : F
}
impl<F> MyList<F>
where F : Fn(i32,i32) -> i32
{
pub fn new(data: &[i32], sort_ascending: bool) -> MyList<F> {
if sort_ascending {
return MyList {
ladder: data.to_vec(),
sort_closure : |a, b| if a >= b { a } else { b },
};
} else {
return MyList {
ladder: data.to_vec(),
sort_closure : |a, b| if a <= b { a } else { b },
};
}
}
pub fn get_best(&self) -> Option<&i32> {
return self.ladder.iter().reduce(self.sort_closure);
}
}
fn main() {
let x = MyList::new(&[10, 4, 30, 2, 5, 2], true);
let r = x.get_best();
println!("{:?}", r);
let x = MyList::new(&[10, 4, 30, 2, 5, 2], false);
let r = x.get_best();
println!("{:?}", r);
}
I would really appreciate any pointers on how to make the above code compile, or a pointer towards what would be the correct approach to take to implement this pattern.
A straightforward fix is to use a function pointer (fn) instead of trying to use a generic parameter on MyList.
The following solution is similar to what user Jmb suggested in the comment above, but simplified further by
Avoiding lifetimes in the function pointer
Using std::cmp::{max, min} instead of writing our own closures (though the code still works if you were to define the closures yourself, as before)
use std::cmp;
struct MyList {
ladder: Vec<i32>,
sort_closure: fn(i32, i32) -> i32,
}
impl MyList {
pub fn new(data: &[i32], sort_ascending: bool) -> MyList {
MyList {
ladder: data.to_vec(),
sort_closure: match sort_ascending {
true => cmp::max,
false => cmp::min,
},
}
}
pub fn get_best(&self) -> Option<i32> {
self.ladder.iter().copied().reduce(self.sort_closure)
}
}
Why the original approach doesn't work
This is what the MyList definition looked like:
struct MyList<F>
where F : Fn(i32,i32) -> i32
{
ladder: Vec<i32>,
sort_closure : F
}
The reason this doesn't work has to do with the type parameter F on MyList. Having the type paramter there means the caller (the code in main()) is free to instantiate MyList<F> with any F of their choosing that satisfies the Fn(i32,i32) -> i32 constraint. However, inside the MyList::new() function, we are defining our own two closures, then trying to store one of them in MyList. This is the contradiction. The caller doesn't get to pick the type of the closure to be stored, except indirectly with the sort_ascending flag.

Recursion limit when instantiating (unused?) Rust type

I implemented worstsort in Rust:
fn bubblesort<T: PartialOrd>(l: &mut [T]) {
for i in 1..l.len() {
for j in 0..l.len() - i {
if l[j + 1] < l[j] {
l.swap(j + 1, j);
}
}
}
}
fn permutations<T: Clone>(l: &[T]) -> Vec<Vec<T>> {
if l.len() <= 1 {
return vec![l.to_vec()];
}
let mut res = Vec::new();
for i in 0..l.len() {
let mut lcopy = l.to_vec();
lcopy.remove(i);
let p = permutations(&lcopy);
for perm in p {
let mut fullperm = vec![l[i].clone()];
fullperm.extend(perm);
res.push(fullperm);
}
}
res
}
pub fn badsort<T: PartialOrd + Clone>(k: usize, l: &mut [T]) {
if k == 0 {
bubblesort(l);
} else {
let mut p = permutations(l);
badsort(k - 1, &mut p);
l.clone_from_slice(&p[0]);
}
}
pub fn worstsort<T, F>(l: &mut [T], f: F)
where
T: PartialOrd + Clone,
F: FnOnce(usize) -> usize,
{
badsort(f(l.len()), l);
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn badsort_zero() {
let mut unsorted = vec![8, 3, 5];
badsort(0, &mut unsorted);
assert_eq!(unsorted, vec![3, 5, 8]);
}
#[test]
fn worstsort_id() {
let mut unsorted = vec![8, 3, 5];
worstsort(&mut unsorted, |n| n);
assert_eq!(unsorted, vec![3, 5, 8]);
}
}
It compiles and checks fine, until I tried to write tests that actually calling the worstsort and badsort functions, and cargo test gives an error:
error: reached the recursion limit while instantiating `badsort::<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<std::vec::Vec<i32>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>`
I get this error even when calling badsort(0, l), which should just be bubblesort, with no recursive calls whatsoever. My assumption is that rustc is trying to generate monomorphised versions of badsort for every possible usize (I swapped the type to u8 and got the same error), but I don't understand why: it errors whatever the argument is, even when it doesn't recursively call itself at all! I tried setting the recursion limit all the way up to #![recursion_limit="1024"], but it still hits this limit and fails, even with u8.
Is my guess correct? Is there a way to get this (admittedly perverse) code to compile and run at all?

Chain two iterators while lazily constructing the second one

I'd like a method like Iterator::chain() that only computes the argument iterator when it's needed. In the following code, expensive_function should never be called:
use std::{thread, time};
fn expensive_function() -> Vec<u64> {
thread::sleep(time::Duration::from_secs(5));
vec![4, 5, 6]
}
pub fn main() {
let nums = [1, 2, 3];
for &i in nums.iter().chain(expensive_function().iter()) {
if i > 2 {
break;
} else {
println!("{}", i);
}
}
}
One possible approach: delegate the expensive computation to an iterator adaptor.
let nums = [1, 2, 3];
for i in nums.iter()
.cloned()
.chain([()].into_iter().flat_map(|_| expensive_function()))
{
if i > 2 {
break;
} else {
println!("{}", i);
}
}
Playground
The passed iterator is the result of flat-mapping a dummy unit value () to the list of values, which is lazy. Since the iterator needs to own the respective outcome of that computation, I chose to copy the number from the array.
You can create your own custom iterator adapter that only evaluates a closure when the original iterator is exhausted.
trait IteratorExt: Iterator {
fn chain_with<F, I>(self, f: F) -> ChainWith<Self, F, I::IntoIter>
where
Self: Sized,
F: FnOnce() -> I,
I: IntoIterator<Item = Self::Item>,
{
ChainWith {
base: self,
factory: Some(f),
iterator: None,
}
}
}
impl<I: Iterator> IteratorExt for I {}
struct ChainWith<B, F, I> {
base: B,
factory: Option<F>,
iterator: Option<I>,
}
impl<B, F, I> Iterator for ChainWith<B, F, I::IntoIter>
where
B: Iterator,
F: FnOnce() -> I,
I: IntoIterator<Item = B::Item>,
{
type Item = I::Item;
fn next(&mut self) -> Option<Self::Item> {
if let Some(b) = self.base.next() {
return Some(b);
}
// Exhausted the first, generate the second
if let Some(f) = self.factory.take() {
self.iterator = Some(f().into_iter());
}
self.iterator
.as_mut()
.expect("There must be an iterator")
.next()
}
}
use std::{thread, time};
fn expensive_function() -> Vec<u64> {
panic!("You lose, good day sir");
thread::sleep(time::Duration::from_secs(5));
vec![4, 5, 6]
}
pub fn main() {
let nums = [1, 2, 3];
for i in nums.iter().cloned().chain_with(|| expensive_function()) {
if i > 2 {
break;
} else {
println!("{}", i);
}
}
}

Create an array from a tuple

I can create an array from a tuple like this:
let a = (1, 2, 3);
let b = [a.0, a.1, a.2];
Is there a way to do it without naming each element of the tuple? Something like:
let b = a.to_array();
There is no such functionality at the moment, however it would be perfectly possible to extend the set of implementations of the From trait to cover this usecase (and its reverse).
This extension would have to be in the core crate because of the orphan rules, but we can readily demonstrate it with custom traits:
use std::convert::Into;
trait MyFrom<T> {
fn my_from(t: T) -> Self;
}
trait MyInto<U> {
fn my_into(self) -> U;
}
impl<T, U> MyInto<U> for T
where
U: MyFrom<T>
{
fn my_into(self) -> U { <U as MyFrom<T>>::my_from(self) }
}
impl<T> MyFrom<()> for [T; 0] {
fn my_from(_: ()) -> Self { [] }
}
impl<T, A> MyFrom<(A,)> for [T; 1]
where
A: Into<T>,
{
fn my_from(t: (A,)) -> Self { [t.0.into()] }
}
impl<T, A, B> MyFrom<(A, B)> for [T; 2]
where
A: Into<T>,
B: Into<T>,
{
fn my_from(t: (A, B)) -> Self { [t.0.into(), t.1.into()] }
}
Once define, it's easy enough to use:
fn main() {
{
let array: [i64; 0] = ().my_into();
println!("{:?}", array);
}
{
let array: [i64; 1] = (1u32,).my_into();
println!("{:?}", array);
}
{
let array: [i64; 2] = (1u32, 2i16).my_into();
println!("{:?}", array);
}
}
will print:
[]
[1]
[1, 2]
The reverse implementation would be as easy, there's nothing mysterious here it's just boilerplate (hurray for macros!).
No, there isn't. What is more, you can't even iterate over tuples. The tuple is heterogeneous, so it's unfit for a conversion to a homogeneous type like a vector or an array.
You could write a macro to allow iteration over the contents of a tuple of a generic length and collect them (as long as all its elements are of the same type), but you would still have to access/process every element individually.

How to define mutual recursion with closures?

I can do something like this:
fn func() -> (Vec<i32>, Vec<i32>) {
let mut u = vec![0;5];
let mut v = vec![0;5];
fn foo(u: &mut [i32], v: &mut [i32], i: usize, j: usize) {
for k in i+1..u.len() {
u[k] += 1;
bar(u, v, k, j);
}
}
fn bar(u: &mut [i32], v: &mut [i32], i: usize, j: usize) {
for k in j+1..v.len() {
v[k] += 1;
foo(u, v, i, k);
}
}
foo(&mut u, &mut v, 0, 0);
(u,v)
}
fn main() {
let (u,v) = func();
println!("{:?}", u);
println!("{:?}", v);
}
but I would prefer to do something like this:
fn func() -> (Vec<i32>, Vec<i32>) {
let mut u = vec![0;5];
let mut v = vec![0;5];
let foo = |i, j| {
for k in i+1..u.len() {
u[k] += 1;
bar(k, j);
}
};
let bar = |i, j| {
for k in j+1..v.len() {
v[k] += 1;
foo(i, k);
}
};
foo(0, 0);
(u,v)
}
fn main() {
let (u,v) = func();
println!("{:?}", u);
println!("{:?}", v);
}
The second example doesn't compile with the error: unresolved name bar.
In my task I can do it through one recursion, but it will not look clear.
Does anyone have any other suggestions?
I have a solution for mutually recursive closures, but it doesn't work with multiple mutable borrows, so I couldn't extend it to your example.
There is a way to use define mutually recursive closures, using an approach similar to how this answer does single recursion. You can put the closures together into a struct, where each of them takes a borrow of that struct as an extra argument.
fn func(n: u32) -> bool {
struct EvenOdd<'a> {
even: &'a Fn(u32, &EvenOdd<'a>) -> bool,
odd: &'a Fn(u32, &EvenOdd<'a>) -> bool
}
let evenodd = EvenOdd {
even: &|n, evenodd| {
if n == 0 {
true
} else {
(evenodd.odd)(n - 1, evenodd)
}
},
odd: &|n, evenodd| {
if n == 0 {
false
} else {
(evenodd.even)(n - 1, evenodd)
}
}
};
(evenodd.even)(n, &evenodd)
}
fn main() {
println!("{}", func(5));
println!("{}", func(6));
}
While defining mutually recursive closures works in some cases, as demonstrated in the answer by Alex Knauth, I don't think that's an approach you should usually take. It is kind of opaque, has some limitations pointed out in the other answer, and it also has a performance overhead since it uses trait objects and dynamic dispatch at runtime.
Closures in Rust can be thought of as functions with associated structs storing the data you closed over. So a more general solution is to define your own struct storing the data you want to close over, and define methods on that struct instead of closures. For this case, the code could look like this:
pub struct FooBar {
pub u: Vec<i32>,
pub v: Vec<i32>,
}
impl FooBar {
fn new(u: Vec<i32>, v: Vec<i32>) -> Self {
Self { u, v }
}
fn foo(&mut self, i: usize, j: usize) {
for k in i+1..self.u.len() {
self.u[k] += 1;
self.bar(k, j);
}
}
fn bar(&mut self, i: usize, j: usize) {
for k in j+1..self.v.len() {
self.v[k] += 1;
self.foo(i, k);
}
}
}
fn main() {
let mut x = FooBar::new(vec![0;5], vec![0;5]);
x.foo(0, 0);
println!("{:?}", x.u);
println!("{:?}", x.v);
}
(Playground)
While this can get slightly more verbose than closures, and requires a few more explicit type annotations, it's more flexible and easier to read, so I would generally prefer this approach.

Resources