Related
What's the most idiomatic way to get the Rust vec equivalent of this Python code?
import numpy as np
a = np.arange(5)
a_diff = np.diff(a) # this is the thing I'm trying to emulate in Rust
print(a_diff) # [1 1 1 1]
I can figure out multiple unsatisfactory ways of doing this, but I figure there's got to be a clean one-liner approach using iter(), right?
let a: Vec<f64> = (0..5).collect::<Vec<i64>>().iter().map(|x| *x as f64).collect();
let a_diff = ???
With stock Rust I'd use windows:
fn main() {
let a: Vec<f64> = (0..5).map(|x| x as f64).collect();
let a_diff: Vec<f64> = a
.windows(2)
.map(|vs| {
let [x, y] = vs else { unreachable!() };
y - x
})
.collect();
dbg!(a_diff);
}
(I also removed the needless collection into a Vec<i64>.)
When using nightly that can be shortened to this:
#![feature(array_windows)]
fn main() {
let a: Vec<f64> = (0..5).map(|x| x as f64).collect();
let a_diff: Vec<f64> = a.array_windows().map(|[x, y]| y - x).collect();
dbg!(a_diff);
}
If you're dealing with Vec, you can use windows:
let a: Vec<f64> = (0..5).map(|x| x as f64).collect();
let a_diff: Vec<f64> = a.windows(2).map(|s| s[1] - s[0]).collect();
If you want to do it using only iterators, you can use scan, but it's more involved:
let mut a = (0..5).map(|x| x as f64);
let a_diff: Vec<f64> = if let Some(first) = a.next() {
a.scan(first, |prev, x| {
let out = x - *prev;
*prev = x;
Some(out)
}).collect()
} else { vec![] };
Or you can use tuple_windows from the itertools crate:
use itertools::Itertools;
let a = (0..5).map(|x| x as f64);
let a_diff: Vec<f64> = a.tuple_windows().map(|(a, b)| b - a).collect();
CONTEXT
General overview
(Here is the github page with the minimal example of my problem, and the page of my whole project)
I'm very new to rust and I'm trying to simulate the behavior of a fluid in Rust. This is easy: computing large arrays with some functions for each timestep.
I'd like to parallelize the computations done each timestep using rayon. But the compiler doesn't want me to access a mutable struct containing an Array that I want to modify, even if I'm sure that each modification will be on different places in the array: which assure me that it's safe. (I think?).
So my question is: should I use unsafe rust in here? If so, how?
And: is it possible to make Rust understand that it's safe, or to do it properly without unsafe rust?
I saw this post which kind of resembled my problem, but couldn't find a way to use the solution for my problem.
I also tried to put unsafe {...} keywords everywhere, but the compilator still complains...
You may only need to read "Structs" subsection to understand the problem, but I will also put a "Function" subsection, in case it can be important. If you think it might not be necessary, you can skip to "Main function" subsection.
Structs
Here are my structs:
I'd like to keep them that way, as they would give me (I think) more flexibility with setters/getters, but I'm open to change the way it's implemented right now.
#[derive(Debug, PartialEq)]
struct vec2D {pub x: f64, pub y: f64}
#[derive(Debug, PartialEq)]
struct ScalarField2D {
s: Array2<f64>,
}
#[derive(Debug, PartialEq)]
struct VectorField2D {
x: ScalarField2D,
y: ScalarField2D
}
impl ScalarField2D {
// also a constructor new() but not shown for simplicity
fn get_pos(&self, x: usize, y: usize) -> f64
{return self.s[[y,x]];}
fn set_pos(&mut self, x: usize, y: usize, f: f64)
{self.s[[y,x]] = f;}
}
impl VectorField2D {
// also a constructor new() but not shown for simplicity
fn get_pos(&self, x: usize, y: usize) -> vec2D
{let vec_at_pos = vec2D {
x: self.x.get_pos(x, y),
y: self.y.get_pos(x, y)};
return vec_at_pos;}
fn set_pos(&mut self, x: usize, y: usize, vec: &vec2D)
{self.x.set_pos(x, y, vec.x);
self.y.set_pos(x, y, vec.y);}
}
Function
Here is my function: which takes a ScalarField2D struct, and computes a vector called the "gradient" at a particular position of the ScalarField2D array, and then returning this vector as a "vec2D" struct.
// computes the gradient of a scalar field at a given position
fn grad_scalar(a: &ScalarField2D,
x: i32, y: i32,
x_max: i32, y_max: i32) -> vec2D
{
let ip = ((x+1) % x_max) as usize;
// i-1 with Periodic Boundaries
let im = ((x - 1 + x_max) % x_max) as usize;
// j+1 with Periodic Boundaries
let jp = ((y+1) % y_max) as usize;
// j-1 with Periodic Boundaries
let jm = ((y - 1 + y_max) % y_max) as usize;
let (i, j) = (x as usize, y as usize);
let grad = vec2D {
x: (a.get_pos(ip, j) - a.get_pos(im, j))/(2.),
y: (a.get_pos(i, jp) - a.get_pos(i, jm))/(2.)};
return grad;
}
Main function
Here is my problem:
I try to iterate over all the possible x and y using (0..x_max).into_par_iter() (or y_max for y), compute the gradient associated with each position, and then set the value to the ScalarField2D struct using the set_pos method... Here is the main function, and the import commands, and I will show you the error message I get in the next subsection
use ndarray::prelude::*;
use rayon::prelude::*;
fn main() {
let (x_max, y_max) = (2usize, 50usize);
let (x_maxi32, y_maxi32) = (x_max as i32, y_max as i32);
let mut GD_grad_rho = VectorField2D::new(x_max, y_max);
let mut GD_rho = ScalarField2D::new(x_max, y_max);
let x_iterator = (0..x_max).into_par_iter();
x_iterator.map(|xi| {
let y_iterator = (0..y_max).into_par_iter();
y_iterator.map(|yi| {
// unsafe here?
GD_grad_rho
.set_pos(xi, yi,
&grad_scalar(&GD_rho,
xi as i32, yi as i32,
x_maxi32, y_maxi32));
});});
}
Error message
Here is the compilation error I get
error[E0596]: cannot borrow `GD_grad_rho` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:104:13
|
104 | / GD_grad_rho
105 | | .set_pos(xi, yi,
106 | | &grad_scalar(&GD_rho,
107 | | xi as i32, yi as i32,
108 | | x_maxi32, y_maxi32));
| |__________________________________________________________^ cannot borrow as mutable
error[E0596]: cannot borrow `GD_grad_rho` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:101:24
|
101 | y_iterator.map(|yi| {
| ^^^^ cannot borrow as mutable
...
104 | GD_grad_rho
| ----------- mutable borrow occurs due to use of `GD_grad_rho` in closure
For more information about this error, try `rustc --explain E0596`.
error: could not compile `minimal_example_para` due to 2 previous errors
If you want the complete thing, I created a github repo with everything in it.
Tests after susitsm answer
So I did something like this:
use ndarray::prelude::*;
use rayon::prelude::*;
fn grad_scalar(a: &Array2<f64>,
i: usize, j: usize) -> (f64, f64)
{
let array_shape = a.shape();
let imax = array_shape[0];
let jmax = array_shape[1];
// i-1 with Periodic Boundaries
let ip = ((i + 1) % imax);
// i-1 with Periodic Boundaries
let im = ((i + imax) - 1) % imax;
// j+1 with Periodic Boundaries
let jp = ((j + 1) % jmax);
// j-1 with Periodic Boundaries
let jm = ((j + jmax) - 1) % jmax;
let grad_i = (a[[ip, j]] - a[[im, j]])/2.;
let grad_j = (a[[i, jp]] - a[[i, jm]])/2.;
return (grad_i, grad_j);
}
fn main() {
let a = Array::<f64, Ix2>::ones((dim, dim));
let index_list: Vec<(_, _)> = (0..a.len())
.into_par_iter()
.map(|i| (i / a.dim().0, i % a.dim().1))
.collect();
let (r1, r2): (Vec<_>, Vec<_>) = (0..a.len())
.into_par_iter()
.map(|i| (index_list[i].0, index_list[i].1))
.map(|(i, j)| grad_scalar(&a, i, j))
.collect();
let grad_row = Array2::from_shape_vec(a.dim(), r1).unwrap();
let grad_col = Array2::from_shape_vec(a.dim(), r2).unwrap();
}
Which gives me the result I want, even though I wanted to access the values through my Structs. Which is not exactly what I want but we're getting closer
But I wonder about the efficiency for more arrays, I'll post a separate question for that
You can do something like this:
use ndarray::Array2;
use rayon::prelude::*;
fn main() {
let a: Vec<u64> = (0..20000).collect();
let a = Array2::from_shape_vec((100, 200), a).unwrap();
let stuff = |arr, i, j| (i + j, i * j);
let (x, y): (Vec<_>, Vec<_>) = (0..a.len())
.into_par_iter()
.map(|i| (i / a.dim().0, i % a.dim().1))
.map(|(i, j)| stuff(&a, i, j))
.collect();
let grad_x = Array2::from_shape_vec(a.dim(), x);
let grad_y = Array2::from_shape_vec(a.dim(), y);
let grad_vector_field = VectorField2D {
x: ScalarField2D { s: grad_x },
y: ScalarField2D { s: grad_y },
};
}
Or implement the FromParallelIterator<vec2D>
impl FromParallelIterator<vec2D> for VectorField2D {
fn from_par_iter<I>(par_iter: I) -> Self
where I: IntoParallelIterator<Item = T>
{
let (x, y): (Vec<_>, Vec<_>) = par_iter
.into_par_iter()
.map(|vec_2D| {
let vec2D { x, y } = vec_2D;
(x, y)
})
.collect();
Self {
x: ScalarField2D { s: Array2::from_shape_vec(a.dim(), x) },
y: ScalarField2D { s: Array2::from_shape_vec(a.dim(), y) },
}
}
}
This will enable using collect for your type when using parallel iterators
let vector_field: VectorField2D = (0..a.len())
.into_par_iter()
.map(|i| (index_list[i].0, index_list[i].1))
.map(|(i, j)| grad_scalar(&a, i, j))
.collect();
In this function parse can return an error so I use .filter_map(Result::ok) to filter them out.
fn part1(input: &str) {
let sum = input.lines()
.map(|l| l.parse::<u32>())
.filter_map(Result::ok)
.map(|n| n as f32 / 3.0)
.map(|f| f.round())
.map(|f| f as u32 - 2)
.sum::<u32>();
// println!("{}", sum);
println!("{:?}", sum);
}
However, I would like to return out of the part1 function when parse gives an error, kind of like using the question mark operator like this .map(|l| l.parse::<u32>()?). If this is done the compiler gives the error
error[E0277]: the `?` operator can only be used in a closure that returns `Result`
or `Option` (or another type that implements `std::ops::Try`)
--> src/main.rs:64:18
|
64 | .map(|l| l.parse::<u32>()?)
| ----^^^^^^^^^^^^^^^^^
| | |
| | cannot use the `?` operator in a closure that returns `u32`
| this function should return `Result` or `Option` to accept `?`
Is this because the question mark operator is used inside a closure so it returns out of the closure instead of the enclosing function? What are some idiomatic alternatives to using the question mark operator inside the closure so that I can return out of part1 if parse gives an error or unwrap the Ok if parse is successful? The result should be similar to .filter_map(Result::ok), except instead of filtering out the errors it will return out of the enclosing function when there is an error.
You can just keep passing the Result from parse further down the chain and allow the final sum to work - since Sum is implemented for Result. Then you can use ? on the final result of the chain.
An example would look like this:
fn part1(input: &str) -> Result<u32,std::num::ParseIntError> {
let sum = input.lines()
.map(|l| l.parse::<u32>())
.map(|n| n.map( |n| n as f32 / 3.0) )
.map(|f| f.map( |f| f.round() ) )
.map(|f| f.map( |f| f as u32 - 2) )
.sum::<Result<u32,_>>()?;
Ok(sum)
}
If you're using nightly rust you can get rid of the nested closures using a try block
#![feature(try_blocks)]
fn part1(input: &str) -> Result<u32, std::num::ParseIntError> {
let sum = input.lines()
.map( |l| try {
let n = l.parse::<u32>()?;
let f = n as f32 / 3.0;
let f = f.round();
f as u32 - 2
})
.sum::<Result<u32,_>>()?;
Ok(sum)
}
If you are not using nightly you can extract the processing into a closure that returns a Result.
fn part1(input: &str) -> Result<u32, std::num::ParseIntError> {
let process_line = |l:&str| -> Result<u32,std::num::ParseIntError> {
let n = l.parse::<u32>()?;
let f = n as f32 / 3.0;
let f = f.round();
Ok(f as u32 - 2)
};
let sum = input.lines().map(process_line).sum::<Result<u32,_>>()?;
Ok(sum)
}
I'm also assuming that your real use case is somewhat more complicated than you've presented here. For something this simple I'd just use a for loop
fn part1(input: &str) -> Result<u32,std::num::ParseIntError> {
let mut sum = 0;
for line in input.lines() {
let n = l.parse::<u32>()?;
let f = n as f32 / 3.0;
let f = f.round();
sum += f as u32 - 2;
}
Ok(sum)
}
The multiple calls to map might make some solutions feel cluttered.
Instead, all your math could be performed in a single call to map, that is then used with sum:
fn part1(input: &str) -> Result<(), std::num::ParseIntError> {
let sum = input.lines()
.map(|l| {
let n = l.parse::<u32>()?;
let mut f = n as f32 / 3.0;
f = f.round();
Ok(f as u32 - 2)
})
.sum::<Result<u32, _>>()?;
// println!("{}", sum);
println!("{:?}", sum);
Ok(())
}
But you could then go further by removing the ? and using map on the Result. If you do this along with returning a value from your function, you don't even need the explicit type parameter to sum:
fn part1(input: &str) -> Result<u32, std::num::ParseIntError> {
input.lines()
.map(|l| {
l.parse::<u32>().map(|n| {
let mut f = n as f32 / 3.0;
f = f.round();
f as u32 - 2
})
})
.sum()
}
You would then have to call println outside of the function.
If you don't like the nested closures, you can always extract the math to another function (with a better name):
fn part1(input: &str) -> Result<u32, std::num::ParseIntError> {
input.lines()
.map(|l| l.parse().map(math_part))
.sum()
}
fn math_part(n: u32) -> u32 {
let mut f = n as f32 / 3.0;
f = f.round();
f as u32 - 2
}
Is there an efficient way to perform a drop_while for mutable iterators?
Using .take_while() on its own is insufficient because TakeWhile is lazy:
let s = String::from("abcdefg");
let mut it = s.chars();
it.by_ref().take_while(|x| *x != 'c');
println!("{:?}", it.next());
This results in a Some('a') and the following warning:
unused `std::iter::TakeWhile` that must be used
This can be fixed by running a .collect() which results in the wanted Some('d'):
let _: String = it.by_ref().take_while(|x| *x < 'c').collect();
But, is this the correct approach? Would this not waste resources?
You can use skip_while:
let x = it.by_ref().skip_while(|x| *x <= 'c').next();
println!("{:?}", x); // Some('d')
which is equivalent to using find, with a negated predicate:
let x = it.by_ref().find(|x| *x > 'c');
println!("{:?}", x); // Some('d')
I believe most iterator methods are lazy except for for_each which seems to be eager. You can turn your take_while into a drop_while like this:
fn main() {
let s = String::from("abcdefg");
let mut it = s.chars();
it.by_ref().take_while(|x| *x != 'c').for_each(drop);
println!("{:?}", it.next()); // prints "Some('d')"
}
playground
In the following sample program, is there any way I could avoid having to define map2?
fn map2<T, U, V, F: Fn(T, U) -> V>(f: F, a: Option<T>, b: Option<U>) -> Option<V> {
match a {
Some(x) => match b {
Some(y) => Some(f(x, y)),
None => None,
},
None => None,
}
}
fn main() {
let a = Some(5);
let b = Some(10);
let f = |a, b| {
a + b
};
let res = map2(f, a, b);
println!("{:?}", res);
// prints Some(15)
}
For people who also speak Haskell, I guess this question could also be phrased as "Is there any tool we can use instead of liftM2 in Rust?"
I don't believe there's a direct function equivalent to liftM2, but you can combine Option::and_then and Option::map like this:
fn main() {
let a = Some(5);
let b = Some(10);
let f = |a, b| {
a + b
};
println!("{:?}", a.and_then(|a| b.map(|b| f(a, b))));
}
Output:
Some(15)
As of Rust 1.46.0, you can use Option::zip:
fn map2<T, U, V, F: Fn(T, U) -> V>(f: F, a: Option<T>, b: Option<U>) -> Option<V> {
match a.zip(b) {
Some((x, y)) => Some(f(x, y)),
None => None,
}
}
This can be combined with Option::map, as shown in other answers:
fn map2<T, U, V, F: Fn(T, U) -> V>(f: F, a: Option<T>, b: Option<U>) -> Option<V> {
a.zip(b).map(|(x, y)| f(x, y))
}
I don't know if you can get down to one line (Edit: oh the accepted answer gets it down to one line nicely), but you can avoid the nested match by matching on a tuple:
let a = Some(5);
let b = Some(10);
let f = |a, b| {
a + b
};
let res = match (a, b) {
(Some(a), Some(b)) => Some(f(a, b)),
_ => None,
};
println!("{:?}", res);
// prints Some(15)
let num_maybe = Some(5);
let num_maybe2 = Some(10);
let f = |a, b| {
a + b
};
Option 1
if let (Some(a), Some(b)) = (num_maybe, num_maybe2) {
f(a, b)
}
Option 2
num_maybe.and_then(|a| num_maybe2.map(|b| f(a, b))
Option 3
[num_maybe, num_maybe2].into_iter().flatten().fold(0, f)
You can use an immediately invoked function expression (IIFE) combined with the ? (try) operator:
fn main() {
let a = Some(5);
let b = Some(10);
let f = |a, b| a + b;
let res = (|| Some(f(a?, b?)))();
println!("{:?}", res);
}
In the future, you can use try blocks:
#![feature(try_blocks)]
fn main() {
let a = Some(5);
let b = Some(10);
let f = |a, b| a + b;
let res: Option<_> = try { f(a?, b?) };
println!("{:?}", res);
}
See also:
Is there a more ergonomic syntax for Either when using futures?
Is there a shortcut to unwrap or continue in a loop?
You can use the fact that Options can be iterated over. Iterate over both options, zip them together, and map the resulting iterator over your function.
fn main() {
let a = Some(5);
let b = Some(10);
let f = |(a, b)| {
a + b
};
let res = a.iter().zip(b.iter()).map(f).next();
println!("{:?}", res);
// prints Some(15)
}
This required a modification of f, so the arguments are merged into a single tuple-argument. It would be possible without modifying f, by directly mapping over |args| f.call(args), but then you would have to specify the closure kind of f.
I stumbled upon this thread and didn't find the most obvious and straightforward one-liner solution based on zip.
let one = Some(1);
let two = Some(2);
let sum = one.zip(two).map(|(a, b)| a + b);
assert_eq!(sum, Some(3));
let two: Option<i32> = None;
let sum = one.zip(two).map(|(a, b)| a + b);
assert_eq!(sum, None);
There's also the zip_with variant which is marked as unstable right now.
let sum = one.zip_with(two, |a, b| a + b);