CONTEXT
General overview
(Here is the github page with the minimal example of my problem, and the page of my whole project)
I'm very new to rust and I'm trying to simulate the behavior of a fluid in Rust. This is easy: computing large arrays with some functions for each timestep.
I'd like to parallelize the computations done each timestep using rayon. But the compiler doesn't want me to access a mutable struct containing an Array that I want to modify, even if I'm sure that each modification will be on different places in the array: which assure me that it's safe. (I think?).
So my question is: should I use unsafe rust in here? If so, how?
And: is it possible to make Rust understand that it's safe, or to do it properly without unsafe rust?
I saw this post which kind of resembled my problem, but couldn't find a way to use the solution for my problem.
I also tried to put unsafe {...} keywords everywhere, but the compilator still complains...
You may only need to read "Structs" subsection to understand the problem, but I will also put a "Function" subsection, in case it can be important. If you think it might not be necessary, you can skip to "Main function" subsection.
Structs
Here are my structs:
I'd like to keep them that way, as they would give me (I think) more flexibility with setters/getters, but I'm open to change the way it's implemented right now.
#[derive(Debug, PartialEq)]
struct vec2D {pub x: f64, pub y: f64}
#[derive(Debug, PartialEq)]
struct ScalarField2D {
s: Array2<f64>,
}
#[derive(Debug, PartialEq)]
struct VectorField2D {
x: ScalarField2D,
y: ScalarField2D
}
impl ScalarField2D {
// also a constructor new() but not shown for simplicity
fn get_pos(&self, x: usize, y: usize) -> f64
{return self.s[[y,x]];}
fn set_pos(&mut self, x: usize, y: usize, f: f64)
{self.s[[y,x]] = f;}
}
impl VectorField2D {
// also a constructor new() but not shown for simplicity
fn get_pos(&self, x: usize, y: usize) -> vec2D
{let vec_at_pos = vec2D {
x: self.x.get_pos(x, y),
y: self.y.get_pos(x, y)};
return vec_at_pos;}
fn set_pos(&mut self, x: usize, y: usize, vec: &vec2D)
{self.x.set_pos(x, y, vec.x);
self.y.set_pos(x, y, vec.y);}
}
Function
Here is my function: which takes a ScalarField2D struct, and computes a vector called the "gradient" at a particular position of the ScalarField2D array, and then returning this vector as a "vec2D" struct.
// computes the gradient of a scalar field at a given position
fn grad_scalar(a: &ScalarField2D,
x: i32, y: i32,
x_max: i32, y_max: i32) -> vec2D
{
let ip = ((x+1) % x_max) as usize;
// i-1 with Periodic Boundaries
let im = ((x - 1 + x_max) % x_max) as usize;
// j+1 with Periodic Boundaries
let jp = ((y+1) % y_max) as usize;
// j-1 with Periodic Boundaries
let jm = ((y - 1 + y_max) % y_max) as usize;
let (i, j) = (x as usize, y as usize);
let grad = vec2D {
x: (a.get_pos(ip, j) - a.get_pos(im, j))/(2.),
y: (a.get_pos(i, jp) - a.get_pos(i, jm))/(2.)};
return grad;
}
Main function
Here is my problem:
I try to iterate over all the possible x and y using (0..x_max).into_par_iter() (or y_max for y), compute the gradient associated with each position, and then set the value to the ScalarField2D struct using the set_pos method... Here is the main function, and the import commands, and I will show you the error message I get in the next subsection
use ndarray::prelude::*;
use rayon::prelude::*;
fn main() {
let (x_max, y_max) = (2usize, 50usize);
let (x_maxi32, y_maxi32) = (x_max as i32, y_max as i32);
let mut GD_grad_rho = VectorField2D::new(x_max, y_max);
let mut GD_rho = ScalarField2D::new(x_max, y_max);
let x_iterator = (0..x_max).into_par_iter();
x_iterator.map(|xi| {
let y_iterator = (0..y_max).into_par_iter();
y_iterator.map(|yi| {
// unsafe here?
GD_grad_rho
.set_pos(xi, yi,
&grad_scalar(&GD_rho,
xi as i32, yi as i32,
x_maxi32, y_maxi32));
});});
}
Error message
Here is the compilation error I get
error[E0596]: cannot borrow `GD_grad_rho` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:104:13
|
104 | / GD_grad_rho
105 | | .set_pos(xi, yi,
106 | | &grad_scalar(&GD_rho,
107 | | xi as i32, yi as i32,
108 | | x_maxi32, y_maxi32));
| |__________________________________________________________^ cannot borrow as mutable
error[E0596]: cannot borrow `GD_grad_rho` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:101:24
|
101 | y_iterator.map(|yi| {
| ^^^^ cannot borrow as mutable
...
104 | GD_grad_rho
| ----------- mutable borrow occurs due to use of `GD_grad_rho` in closure
For more information about this error, try `rustc --explain E0596`.
error: could not compile `minimal_example_para` due to 2 previous errors
If you want the complete thing, I created a github repo with everything in it.
Tests after susitsm answer
So I did something like this:
use ndarray::prelude::*;
use rayon::prelude::*;
fn grad_scalar(a: &Array2<f64>,
i: usize, j: usize) -> (f64, f64)
{
let array_shape = a.shape();
let imax = array_shape[0];
let jmax = array_shape[1];
// i-1 with Periodic Boundaries
let ip = ((i + 1) % imax);
// i-1 with Periodic Boundaries
let im = ((i + imax) - 1) % imax;
// j+1 with Periodic Boundaries
let jp = ((j + 1) % jmax);
// j-1 with Periodic Boundaries
let jm = ((j + jmax) - 1) % jmax;
let grad_i = (a[[ip, j]] - a[[im, j]])/2.;
let grad_j = (a[[i, jp]] - a[[i, jm]])/2.;
return (grad_i, grad_j);
}
fn main() {
let a = Array::<f64, Ix2>::ones((dim, dim));
let index_list: Vec<(_, _)> = (0..a.len())
.into_par_iter()
.map(|i| (i / a.dim().0, i % a.dim().1))
.collect();
let (r1, r2): (Vec<_>, Vec<_>) = (0..a.len())
.into_par_iter()
.map(|i| (index_list[i].0, index_list[i].1))
.map(|(i, j)| grad_scalar(&a, i, j))
.collect();
let grad_row = Array2::from_shape_vec(a.dim(), r1).unwrap();
let grad_col = Array2::from_shape_vec(a.dim(), r2).unwrap();
}
Which gives me the result I want, even though I wanted to access the values through my Structs. Which is not exactly what I want but we're getting closer
But I wonder about the efficiency for more arrays, I'll post a separate question for that
You can do something like this:
use ndarray::Array2;
use rayon::prelude::*;
fn main() {
let a: Vec<u64> = (0..20000).collect();
let a = Array2::from_shape_vec((100, 200), a).unwrap();
let stuff = |arr, i, j| (i + j, i * j);
let (x, y): (Vec<_>, Vec<_>) = (0..a.len())
.into_par_iter()
.map(|i| (i / a.dim().0, i % a.dim().1))
.map(|(i, j)| stuff(&a, i, j))
.collect();
let grad_x = Array2::from_shape_vec(a.dim(), x);
let grad_y = Array2::from_shape_vec(a.dim(), y);
let grad_vector_field = VectorField2D {
x: ScalarField2D { s: grad_x },
y: ScalarField2D { s: grad_y },
};
}
Or implement the FromParallelIterator<vec2D>
impl FromParallelIterator<vec2D> for VectorField2D {
fn from_par_iter<I>(par_iter: I) -> Self
where I: IntoParallelIterator<Item = T>
{
let (x, y): (Vec<_>, Vec<_>) = par_iter
.into_par_iter()
.map(|vec_2D| {
let vec2D { x, y } = vec_2D;
(x, y)
})
.collect();
Self {
x: ScalarField2D { s: Array2::from_shape_vec(a.dim(), x) },
y: ScalarField2D { s: Array2::from_shape_vec(a.dim(), y) },
}
}
}
This will enable using collect for your type when using parallel iterators
let vector_field: VectorField2D = (0..a.len())
.into_par_iter()
.map(|i| (index_list[i].0, index_list[i].1))
.map(|(i, j)| grad_scalar(&a, i, j))
.collect();
I'm trying to extract X and Y coordinates from a EcKeyopenssl::pkey::Private key in Rust.
I have managed to convert it to a point and bytes, but I don't know how to get the coordinates.
pub fn exportpubkey(key: &EcKey<openssl::pkey::Private>) -> (){
let group: EcGroup = EcGroup::from_curve_name(Nid::SECP256K1).unwrap();
let mut ctx: BigNumContext = openssl::bn::BigNumContext::new().unwrap();
let bytes = key.public_key().to_bytes(&group,openssl::ec::PointConversionForm::COMPRESSED, &mut ctx).unwrap();
println!("{}", display_bytes(&bytes));
}
You might want to look at the method affine_coordinates
use openssl::{bn::*, ec::*, nid::Nid};
pub fn print_pub_key(key: &EcKey<openssl::pkey::Private>) -> () {
let group: EcGroup = EcGroup::from_curve_name(Nid::SECP256K1).unwrap();
let mut ctx: BigNumContext = BigNumContext::new().unwrap();
let public_key = key.public_key();
let mut x = BigNum::new().unwrap();
let mut y = BigNum::new().unwrap();
public_key
.affine_coordinates_gfp(group.as_ref(), &mut x, &mut y, &mut ctx)
.expect("extract coords");
println!("{}, {}", x, y);
}
fn main() {
let group: EcGroup = EcGroup::from_curve_name(Nid::SECP256K1).unwrap();
let key = EcKey::generate(&group).unwrap();
print_pub_key(&key);
}
Note that affine_coordinates_gfp and affine_coordinates_gf2m seem to give back the same coordinate in your chosen group.
I am having a problem whilst trying to get the intersection of two Vectors.
impl Solution {
pub fn intersection(nums: Vec<Vec<i32>>) -> Vec<i32> {
// Intended strategy:
// Store the first element into an intersect_result
// Iterate over the remaining elements for each element:
// Determine the intersection of the current element and the intersect result
// set intersect result to this.
// Sort the intersect result
// Return the intersect result back to the caller.
let len:i32 = nums.len() as i32;
let intersect_result:Vec<i32> = nums[0].clone();
for i in 1..len{
println!("i is: {}", i);
let temp_vec:Vec<i32> = nums[i as usize].clone();
// find the intersection of the current element with the intersect result
let unique_a:HashSet<i32> = temp_vec.into_iter().collect();
let unique_b:HashSet<i32> = intersect_result.clone().into_iter().collect();
intersect_result = unique_a.intersection(&unique_b).collect::<Vec<_>>();
}
vec![]
}
}
The error message I get is:
= note: expected struct `Vec<i32>`
found struct `Vec<&i32>`
This happens in the call unique_a.intersection().
Any thoughts guys?
You can add a map(|i| *i) in the iterator chain that causes the error:
intersect_result = unique_a.intersection(&unique_b).map(|i| *i).collect::<Vec<_>>();
When fixing this, the code also seems to work as intended. I think there are a few improvements possible (probably more, but these immediately tracked my attention):
use hashbrown::HashSet;
pub fn intersection(nums: Vec<Vec<i32>>) -> Vec<i32> {
let mut intersect_result: Vec<i32> = nums[0].clone();
for temp_vec in nums {
let unique_a: HashSet<i32> = temp_vec.into_iter().collect();
intersect_result = unique_a
.intersection(&intersect_result.into_iter().collect())
.map(|i| *i)
.collect::<Vec<_>>();
}
intersect_result
}
fn main() {
let a = vec![1, 2, 3];
let b = vec![2, 3, 4];
let c = vec![3, 4, 5];
let v = vec![a, b, c];
let res = intersection(v);
println!("res: {:?}", res);
}
I have a function that calculates the variance of an iterator of floats, I would like to be able to call this function on an iterator after using the map method to transform it.
use num::Float;
fn compute_var_iter<'a, I, T>(vals: I) -> T
where
I: Iterator<Item = &'a T>,
T: 'a + Float + std::ops::AddAssign,
{
// online variance function
// Var = E[X^2] - E[X]^2
// corrects for 1/n -> 1/(n-1)
let mut x = T::zero();
let mut xsquare = T::zero();
let mut len = T::zero();
for &val in vals {
x += val;
xsquare += val * val;
len += T::one();
}
((xsquare / len) - (x / len) * (x / len)) / (len - T::one()) * len
}
fn main() {
let a: Vec<f64> = (1..100001).map(|i| i as f64).collect();
let b: Vec<f64> = (0..100000).map(|i| i as f64).collect();
dbg!(compute_var_iter(&mut a.iter())); // this works
dbg!(compute_var_iter(a.iter().zip(b).map(|(x, y)| x * y))); // this does not work
}
Is there a performant way to get the map output back to an iterator or to make the function take the map object as an input so that we can avoid having to .collect() and keep the execution lazy?
You can use the iterator objects directly without collect:
use num::Float;
fn compute_var_iter<I, T>(vals: I) -> T
where
I: Iterator<Item = T>,
T: Float + std::ops::AddAssign,
{
// online variance function
// Var = E[X^2] - E[X]^2
// corrects for 1/n -> 1/(n-1)
let mut x = T::zero();
let mut xsquare = T::zero();
let mut len = T::zero();
for val in vals {
x += val;
xsquare += val * val;
len += T::one();
}
((xsquare / len) - (x / len) * (x / len)) / (len - T::one()) * len
}
fn main() {
let a = (1..100001).map(|i| i as f64);
let b = (0..100000).map(|i| i as f64);
let c: Vec<f64> = (0..10000).map(|i| i as f64).collect();
dbg!(compute_var_iter(a.clone())); // this works
dbg!(compute_var_iter(c.iter().map(|i| *i))); // this works
dbg!(compute_var_iter(a.zip(b).map(|(x, y)| x * y)));
}
Playground
Notice that you would need to clone the iterator if you intend to use it several times. Also you do not really need to use references since numbers are usually Copy and the cost is the same as creating the references itself.
Is there an efficient way to perform a drop_while for mutable iterators?
Using .take_while() on its own is insufficient because TakeWhile is lazy:
let s = String::from("abcdefg");
let mut it = s.chars();
it.by_ref().take_while(|x| *x != 'c');
println!("{:?}", it.next());
This results in a Some('a') and the following warning:
unused `std::iter::TakeWhile` that must be used
This can be fixed by running a .collect() which results in the wanted Some('d'):
let _: String = it.by_ref().take_while(|x| *x < 'c').collect();
But, is this the correct approach? Would this not waste resources?
You can use skip_while:
let x = it.by_ref().skip_while(|x| *x <= 'c').next();
println!("{:?}", x); // Some('d')
which is equivalent to using find, with a negated predicate:
let x = it.by_ref().find(|x| *x > 'c');
println!("{:?}", x); // Some('d')
I believe most iterator methods are lazy except for for_each which seems to be eager. You can turn your take_while into a drop_while like this:
fn main() {
let s = String::from("abcdefg");
let mut it = s.chars();
it.by_ref().take_while(|x| *x != 'c').for_each(drop);
println!("{:?}", it.next()); // prints "Some('d')"
}
playground