Move non-copyable struct across iterations - rust

I am trying to access a variable inside a for loop. I can't implement Copy on the struct because it contains a String. How would I use the variable across iterations?
I get error E0382 when compiling. When I looked at the Rust documentation for the error, they mentioned using reference counting to solve the problem. Is this the only solution in my case?
#[derive(Clone)]
struct InputParser {
args: Vec<String>,
current: String,
consumed_quote: bool,
}
impl InputParser {
pub fn parse(input: String) -> Vec<String> {
let parser = InputParser {
args: Vec::new(),
current: String::new(),
consumed_quote: false,
};
for c in input.chars() {
match c {
'"' => parser.consume_quote(),
' ' => parser.consume_space(),
_ => parser.consume_char(c),
}
}
parser.end();
return parser.args;
}
pub fn consume_space(mut self) {
if !self.consumed_quote {
self.push_current();
}
}
pub fn consume_quote(mut self) {
self.consumed_quote = self.consumed_quote;
if self.consumed_quote {
self.push_current();
}
}
pub fn consume_char(mut self, c: char) {
self.current.push(c);
}
pub fn end(mut self) {
self.push_current();
}
pub fn push_current(mut self) {
if self.current.len() > 0 {
self.args.push(self.current);
self.current = String::new();
}
}
}
I want to access parser across iterations of the for loop.

[How do I] move [a] non-copyable struct across iterations
You don't, at least not trivially. Once you've moved the struct to a function, it's gone. The only way to get it back is for the function to give it back to you.
Instead, you most likely want to modify an existing struct inside the loop. You need to use a mutable reference for this:
use std::mem;
#[derive(Clone)]
struct InputParser {
args: Vec<String>,
current: String,
consumed_quote: bool,
}
impl InputParser {
fn consume_space(&mut self) {
if !self.consumed_quote {
self.push_current();
}
}
fn consume_quote(&mut self) {
self.consumed_quote = self.consumed_quote;
if self.consumed_quote {
self.push_current();
}
}
fn consume_char(&mut self, c: char) {
self.current.push(c);
}
fn end(&mut self) {
self.push_current();
}
fn push_current(&mut self) {
if self.current.len() > 0 {
let arg = mem::replace(&mut self.current, String::new());
self.args.push(arg);
}
}
}
fn parse(input: String) -> Vec<String> {
let mut parser = InputParser {
args: Vec::new(),
current: String::new(),
consumed_quote: false,
};
for c in input.chars() {
match c {
'"' => parser.consume_quote(),
' ' => parser.consume_space(),
_ => parser.consume_char(c),
}
}
parser.end();
parser.args
}
fn main() {}
Note that the previous way of taking the current argument would result in error[E0507]: cannot move out of borrowed content, so I switched to mem::replace. This prevents self.current from ever becoming an undefined value (which it was previously).
If you really want to pass everything by value, you need to return by value as well.
#[derive(Clone)]
struct InputParser {
args: Vec<String>,
current: String,
consumed_quote: bool,
}
impl InputParser {
fn consume_space(mut self) -> Self {
if !self.consumed_quote {
return self.push_current();
}
self
}
fn consume_quote(mut self) -> Self {
self.consumed_quote = self.consumed_quote;
if self.consumed_quote {
return self.push_current();
}
self
}
fn consume_char(mut self, c: char) -> Self {
self.current.push(c);
self
}
fn end(mut self) -> Self {
self.push_current()
}
fn push_current(mut self) -> Self {
if self.current.len() > 0 {
self.args.push(self.current);
self.current = String::new();
}
self
}
}
fn parse(input: String) -> Vec<String> {
let mut parser = InputParser {
args: Vec::new(),
current: String::new(),
consumed_quote: false,
};
for c in input.chars() {
parser = match c {
'"' => parser.consume_quote(),
' ' => parser.consume_space(),
_ => parser.consume_char(c),
}
}
parser = parser.end();
parser.args
}
fn main() {}
I believe this makes the API objectively worse in this case. However, you will see this style somewhat frequently with a builder. In that case, the methods tend to be chained together, so you never see a reassignment to the variable.

Related

How to pass on closures through multiple sctructs and functions

I have a hierarchy of structs where I need to call a method in the topmost struct from an Iterator::next implementation at the lowest level.
Current implementation is as follows:
Functional abstract:
pub struct TopLevel {
answer: usize,
}
pub struct MidLevelIter<'mli> {
count: usize,
top_level: &'mli TopLevel,
}
pub struct MidLevel<'ml> {
top_level: &'ml TopLevel,
}
pub struct LowestLevelIter<'lli> {
count: usize,
top_level: &'lli TopLevel,
}
impl TopLevel {
pub fn new() -> Self {
Self { answer: 42 }
}
pub fn iter(&self) -> MidLevelIter<'_> {
MidLevelIter {
count: 1,
top_level: self,
}
}
fn calculate(&self, _: usize) -> &usize {
&self.answer
}
}
impl<'mli> Iterator for MidLevelIter<'mli> {
type Item = MidLevel<'mli>;
fn next(&mut self) -> Option<Self::Item> {
if self.count < 2 {
self.count = 2;
Some(MidLevel {
top_level: self.top_level,
})
} else {
None
}
}
}
impl<'lli> Iterator for LowestLevelIter<'lli> {
type Item = &'lli usize;
fn next(&mut self) -> Option<Self::Item> {
if self.count < 2 {
self.count = 2;
Some(self.top_level.calculate(self.count))
} else {
None
}
}
}
impl<'ml> MidLevel<'ml> {
pub fn iter(&self) -> LowestLevelIter<'ml> {
LowestLevelIter {
count: 1,
top_level: self.top_level,
}
}
}
fn main() {
let collector = TopLevel::new();
for pc in collector.iter() {
for sc in pc.iter() {
println!("SC={}", sc);
}
}
}
This works fine, but it kind of bothers me that I have to pass a reference to TopLevel through all these structs.
So, my idea was to pass only the required method as a closure. That way, the lower levels need not to know anything about the TopLevel construct.
the following approach, however, fails because of "cannot move out of self.mapper which is behind a mutable reference".
pub struct TopLevel {
answer: usize,
}
pub struct MidLevelIter<'mli> {
count: usize,
mapper: Box<dyn Fn(usize) -> &'mli usize + 'mli>,
}
pub struct MidLevel<'ml> {
mapper: Box<dyn Fn(usize) -> &'ml usize + 'ml>,
}
pub struct LowestLevelIter<'lli> {
count: usize,
mapper: Box<dyn Fn(usize) -> &'lli usize + 'lli>,
}
impl TopLevel {
pub fn new() -> Self {
Self { answer: 42 }
}
pub fn iter(&self) -> MidLevelIter<'_> {
MidLevelIter {
count: 1,
mapper: Box::new(self.mapper()),
}
}
fn calculate(&self, _: usize) -> &usize {
&self.answer
}
fn mapper<'m>(&'m self) -> impl Fn(usize) -> &'m usize {
move |secondary_index| self.calculate(secondary_index)
}
}
impl<'mli> Iterator for MidLevelIter<'mli> {
type Item = MidLevel<'mli>;
fn next(&mut self) -> Option<Self::Item> {
if self.count < 2 {
self.count = 2;
Some(MidLevel {
mapper: self.mapper,
})
} else {
None
}
}
}
impl<'lli> Iterator for LowestLevelIter<'lli> {
type Item = &'lli usize;
fn next(&mut self) -> Option<Self::Item> {
if self.count < 2 {
self.count = 2;
Some((self.mapper)(self.count))
} else {
None
}
}
}
impl<'ml> MidLevel<'ml> {
pub fn iter(&self) -> LowestLevelIter<'ml> {
LowestLevelIter {
count: 1,
mapper: self.mapper,
}
}
}
fn main() {
let collector = TopLevel::new();
for pc in collector.iter() {
for sc in pc.iter() {
println!("SC={}", sc);
}
}
}
Although I can understand what the compiler tells me there, I don't see how to circumvent it.
Traits are the answer to my problem.
Basically, on the lowest level I just wanted to perform a transformation of some sort on the items of the iteration. Although closures looked suitable for that, Rust provides another feature to accomplish this: Traits.
On the lowest level, I want to convert a numerical index to a key reference. So make a trait for it:
trait IndexToKey {
fn calculate(&self, _: usize) -> &usize;
}
This trait can now be passed on, e.g.:
pub struct MidLevelIter<'mli> {
count: usize,
mapper: &'mli dyn IndexToKey,
}
Originally, my TopLevel struct provided the logic, so let's implement the trait:
impl IndexToKey for TopLevel {
fn calculate(&self, _ix: usize) -> &usize {
&self.answer
}
}
Now we can pass a reference to trait implementation down to the lowest level, which now simply performs the conversion:
impl<'lli> Iterator for LowestLevelIter<'lli> {
type Item = &'lli usize;
fn next(&mut self) -> Option<Self::Item> {
if self.count < 2 {
self.count = 2;
Some(self.mapper.calculate(self.count))
} else {
None
}
}
}
No lifetime issues, no disclosure or dependency on implementation details of the TopLevel structs at the other levels.
Implementation on the Rust playground

Cannot modify the field in recursive structure

I have a recursive structure, where field of a structure is a reference to other struct of same type:
use std::collections::HashMap;
pub struct RecursiveStruct<'a> {
outer: Option<Box<&'a RecursiveStruct<'a>>>,
dict: HashMap<u32, String>
}
With this structure I also have couple of methods such as constructor, method which adds (k,v) pair to calee's field and a getter:
impl<'a> RecursiveStruct<'a> {
pub fn new(outer: Option<Box<&'a RecursiveStruct<'a>>>) -> Self {
let dict: HashMap<u32, String> = HashMap::new();
RecursiveStruct { outer, dict }
}
// searches for value corresponding to key in all struct layers
pub fn get(&self, key: u32) -> Result<String, ()> {
let item = self.dict.get(&key);
match item {
Some(x) => Ok(x.clone()),
None => {
match &self.outer {
Some(x) => x.get(key),
None => Err(())
}
}
}
}
// adds (key, val) to "innermost" instance of struct
pub fn add(&mut self, key:u32, val: String) {
self.dict.insert(key, val);
}
}
These methods work fine, but when I try to add a method, which tries to modify dict field in any of the inner layers, I get cannot borrow '***x' as mutable, as it is behind a '&' reference error.
pub fn re_assign(&mut self, key: u32, val: String) {
if self.dict.contains_key(&key) {
self.dict.insert(key, val);
} else {
match &mut self.outer {
Some(x) => x.re_assign(key, val.clone()),
None => println!("Such key couldn't be found!"),
};
}
}
Here is the link to playground.
You are using a &, but want a &mut, rust references are immutable by default:
Playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=be82b8ba01dff60e106af9e59df8228e
use std::collections::HashMap;
pub struct RecursiveStruct<'a> {
outer: Option<Box<&'a mut RecursiveStruct<'a>>>,
dict: HashMap<u32, String>
}
impl<'a> RecursiveStruct<'a> {
pub fn new(outer: Option<Box<&'a mut RecursiveStruct<'a>>>) -> Self {
let dict: HashMap<u32, String> = HashMap::new();
RecursiveStruct { outer, dict }
}
pub fn get(&self, key: u32) -> Result<String, ()> {
let item = self.dict.get(&key);
match item {
Some(x) => Ok(x.clone()),
None => {
match &self.outer {
Some(x) => x.get(key),
None => Err(())
}
}
}
}
pub fn re_assign(&mut self, key: u32, val: String) {
if self.dict.contains_key(&key) {
self.dict.insert(key, val);
} else {
match &mut self.outer {
Some(x) => x.re_assign(key, val.clone()),
None => println!("Such key couldn't be found!"),
};
}
}
pub fn add(&mut self, key:u32, val: String) {
self.dict.insert(key, val);
}
}
fn main() {
// instantiate "outer" struct and set its field
let mut S = RecursiveStruct::new(None);
S.add(42, "Answer to the universe!".to_string());
// instantiate "inner" struct
let mut S1 = RecursiveStruct::new(Some(Box::new(&mut S)));
println!("{}", S1.get(42).unwrap()); // get the field of outer struct
// modify the field of outer struct
S1.re_assign(42, "The answer has been changed.".to_string());
println!("{}", S1.get(42).unwrap());
}

Rust: Modifying to an object contained in a Vec from another object of the same Vec

I have a Vec<dyn MyObj> and the first implementation of MyObj contained in the Vec can contains something that refers to a second implementation of MyObj contained in the same Vec.
I'd like that the first implementation can mutate the second implementation.
Here is my first idea:
use std::{rc::{Rc, Weak}, cell::RefCell};
trait MyObj {
fn f(&mut self);
}
type CRef = Weak<RefCell<Container>>;
struct Container {
list: Vec<Box<dyn MyObj>>,
this: CRef,
}
impl Container {
fn new() -> Rc<RefCell<Self>> {
let res = Rc::new(RefCell::new(Self {
list: vec![],
this: Weak::new(),
}));
{
let this = Rc::downgrade(&res);
let mut ref_on_res = res.borrow_mut();
ref_on_res.this = this;
}
res
}
fn register(&mut self, v: impl MyObj + 'static) -> ObjRef {
let index = self.list.len();
self.list.push(Box::new(v));
ObjRef::new(self.this.clone(), index)
}
fn get(&mut self, index: usize) -> &mut (dyn MyObj + 'static) {
let elt = &mut self.list[index];
Box::as_mut(elt)
}
}
struct ObjRef {
c: CRef,
i: usize,
}
impl ObjRef {
fn new(c: CRef, i: usize) -> Self {
Self { c, i }
}
}
impl MyObj for ObjRef {
fn f(&mut self) {
let i = self.i;
self.c.upgrade().map(|c| c.borrow_mut().get(i).f());
}
}
struct A {
r: ObjRef,
}
// First implementation
impl A {
fn new(r: ObjRef) -> A {
A { r }
}
}
impl MyObj for A {
fn f(&mut self) {
self.r.f();
}
}
// Second implementation
struct B(usize);
impl MyObj for B {
fn f(&mut self) {
self.0 += 1;
println!("B({})", self.0);
}
}
fn main() {
let c = Container::new();
let mut r = {
let b = c.borrow_mut().register(B(100));
c.borrow_mut().register(A::new(b))
};
r.f(); // -> Panic: already borrowed: BorrowMutError
}
Obviously, it panics and i understand why but i have no idea to fix this problem.
Have you any idea to do this kind of modification ?

PyContextProtocol example for pyo3?

In the __enter__ method I want to return an object which is accessible in Rust and Python, so that Rust is able to update values in the object and Python can read the updated values.
I would like to have something like this:
#![feature(specialization)]
use std::thread;
use pyo3::prelude::*;
use pyo3::types::{PyType, PyAny, PyDict};
use pyo3::exceptions::ValueError;
use pyo3::PyContextProtocol;
use pyo3::wrap_pyfunction;
#[pyclass]
#[derive(Debug, Clone)]
pub struct Statistics {
pub files: u32,
pub errors: Vec<String>,
}
fn counter(
root_path: &str,
statistics: &mut Statistics,
) {
statistics.files += 1;
statistics.errors.push(String::from("Foo"));
}
#[pyfunction]
pub fn count(
py: Python,
root_path: &str,
) -> PyResult<PyObject> {
let mut statistics = Statistics {
files: 0,
errors: Vec::new(),
};
let rc: std::result::Result<(), std::io::Error> = py.allow_threads(|| {
counter(root_path, &mut statistics);
Ok(())
});
let pyresult = PyDict::new(py);
match rc {
Err(e) => { pyresult.set_item("error", e.to_string()).unwrap();
return Ok(pyresult.into())
},
_ => ()
}
pyresult.set_item("files", statistics.files).unwrap();
pyresult.set_item("errors", statistics.errors).unwrap();
Ok(pyresult.into())
}
#[pyclass]
#[derive(Debug)]
pub struct Count {
root_path: String,
exit_called: bool,
thr: Option<thread::JoinHandle<()>>,
statistics: Statistics,
}
#[pymethods]
impl Count {
#[new]
fn __new__(
obj: &PyRawObject,
root_path: &str,
) {
obj.init(Count {
root_path: String::from(root_path),
exit_called: false,
thr: None,
statistics: Statistics {
files: 0,
errors: Vec::new(),
},
});
}
#[getter]
fn statistics(&self) -> PyResult<Statistics> {
Ok(Statistics { files: self.statistics.files,
errors: self.statistics.errors.to_vec(), })
}
}
#[pyproto]
impl<'p> PyContextProtocol<'p> for Count {
fn __enter__(&mut self) -> PyResult<Py<Count>> {
let gil = GILGuard::acquire();
self.thr = Some(thread::spawn(|| {
counter(self.root_path.as_ref(), &mut self.statistics)
}));
Ok(PyRefMut::new(gil.python(), *self).unwrap().into())
}
fn __exit__(
&mut self,
ty: Option<&'p PyType>,
_value: Option<&'p PyAny>,
_traceback: Option<&'p PyAny>,
) -> PyResult<bool> {
self.thr.unwrap().join();
let gil = GILGuard::acquire();
self.exit_called = true;
if ty == Some(gil.python().get_type::<ValueError>()) {
Ok(true)
} else {
Ok(false)
}
}
}
#[pymodule(count)]
fn init(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Count>()?;
m.add_wrapped(wrap_pyfunction!(count))?;
Ok(())
}
But I'm getting the following error:
error[E0477]: the type `[closure#src/lib.rs:90:39: 92:10 self:&mut &'p mut Count]` does not fulfill the required lifetime
--> src/lib.rs:90:25
|
90 | self.thr = Some(thread::spawn(|| {
| ^^^^^^^^^^^^^
|
= note: type must satisfy the static lifetime
I've found a solution. The use of a guarded reference does the trick:
#![feature(specialization)]
use std::{thread, time};
use std::sync::{Arc, Mutex};
extern crate crossbeam_channel as channel;
use channel::{Sender, Receiver, TryRecvError};
use pyo3::prelude::*;
use pyo3::types::{PyType, PyAny};
use pyo3::exceptions::ValueError;
use pyo3::PyContextProtocol;
#[pyclass]
#[derive(Debug, Clone)]
pub struct Statistics {
pub files: u32,
pub errors: Vec<String>,
}
pub fn counter(
statistics: Arc<Mutex<Statistics>>,
cancel: &Receiver<()>,
) {
for _ in 1..15 {
thread::sleep(time::Duration::from_millis(100));
{
let mut s = statistics.lock().unwrap();
s.files += 1;
}
match cancel.try_recv() {
Ok(_) | Err(TryRecvError::Disconnected) => {
println!("Terminating.");
break;
}
Err(TryRecvError::Empty) => {}
}
}
{
let mut s = statistics.lock().unwrap();
s.errors.push(String::from("Foo"));
}
}
#[pyclass]
#[derive(Debug)]
pub struct Count {
exit_called: bool,
statistics: Arc<Mutex<Statistics>>,
thr: Option<thread::JoinHandle<()>>,
cancel: Option<Sender<()>>,
}
#[pymethods]
impl Count {
#[new]
fn __new__(obj: &PyRawObject) {
obj.init(Count {
exit_called: false,
statistics: Arc::new(Mutex::new(Statistics {
files: 0,
errors: Vec::new(),
})),
thr: None,
cancel: None,
});
}
#[getter]
fn statistics(&self) -> PyResult<u32> {
let s = Arc::clone(&self.statistics).lock().unwrap().files;
Ok(s)
}
}
#[pyproto]
impl<'p> PyContextProtocol<'p> for Count {
fn __enter__(&'p mut self) -> PyResult<()> {
let statistics = self.statistics.clone();
let (sender, receiver) = channel::bounded(1);
self.cancel = Some(sender);
self.thr = Some(thread::spawn(move || {
counter(statistics, &receiver)
}));
Ok(())
}
fn __exit__(
&mut self,
ty: Option<&'p PyType>,
_value: Option<&'p PyAny>,
_traceback: Option<&'p PyAny>,
) -> PyResult<bool> {
let _ = self.cancel.as_ref().unwrap().send(());
self.thr.take().map(thread::JoinHandle::join);
let gil = GILGuard::acquire();
self.exit_called = true;
if ty == Some(gil.python().get_type::<ValueError>()) {
Ok(true)
} else {
Ok(false)
}
}
}
#[pyproto]
impl pyo3::class::PyObjectProtocol for Count {
fn __str__(&self) -> PyResult<String> {
Ok(format!("{:?}", self))
}
}
#[pymodule(count)]
fn init(_py: Python, m: &PyModule) -> PyResult<()> {
m.add_class::<Count>()?;
Ok(())
}
Now I can run the following code:
import time
import count
c = count.Count()
with c:
for _ in range(5):
print(c.statistics)
time.sleep(0.1)
As the example shows thread cancelling also works, although a maybe nicer solution is using the crate thread_control.

How can I map Result<(), E> into a numeric error code?

I have a bunch of FFI functions that I call using C. The caller expects 1 for success, or -1 on failure.
struct Error;
fn my_rust_function() -> Result<(), Error> {
Ok(())
}
#[allow(non_snake_case)]
pub extern "C" fn Called_From_C() -> i32 {
let result = my_rust_function();
match result {
Ok(_) => 1,
Err(_) => -1,
}
}
Is there a more idiomatic way of converting my Result<(), Error> into the 1 / -1 return code?
I'd create an extension trait:
trait FfiError {
fn as_c_error(&self) -> i32;
}
impl<T, E> FfiError for Result<T, E> {
fn as_c_error(&self) -> i32 {
match self {
Ok(_) => 1,
Err(_) => -1,
}
}
}
Once it's brought into scope, you can call it like any other method:
pub extern "C" fn called_from_c() -> i32 {
my_rust_function().as_c_error()
}
See also:
Is there a way other than traits to add methods to a type I don't own?
You could use repr(transparent) to create a type where you could implement From and that still represent a i32, this allow to compile check that you transform your result correctly assuming you didn't have bug in your from() implementation so maybe add some unit tests.
type MyResult = Result<(), ()>;
fn my_rust_function() -> MyResult {
Ok(())
}
#[repr(transparent)]
pub struct CResult {
code: i32,
// code: libc::c_int, // if your C lib expect a `c_int` and not a `i32`
}
impl From<MyResult> for CResult {
fn from(result: MyResult) -> Self {
let code = match result {
Ok(_) => 1,
Err(_) => -1,
};
Self { code }
}
}
#[allow(non_snake_case)]
pub extern "C" fn Called_From_C() -> CResult {
let result = my_rust_function();
result.into()
}
You could also use enum with repr(i32):
#[repr(i32)]
pub enum CResult {
NoError = 1,
Error = -1,
}
impl From<MyResult> for CResult {
fn from(result: MyResult) -> Self {
match result {
Ok(_) => CResult::NoError,
Err(_) => CResult::Error,
}
}
}
In nightly, you could also implement Try:
#![feature(try_trait)]
use std::ops::Try;
type MyResult = Result<(), ()>;
fn my_rust_function() -> MyResult {
Ok(())
}
#[repr(i32)]
pub enum CResult {
NoError = 1,
Error = -1,
}
impl From<MyResult> for CResult {
fn from(result: MyResult) -> Self {
match result {
Ok(_) => CResult::NoError,
Err(_) => CResult::Error,
}
}
}
impl From<CResult> for MyResult {
fn from(cresult: CResult) -> Self {
match cresult {
CResult::NoError => Ok(()),
CResult::Error => Err(()),
}
}
}
impl Try for CResult {
type Ok = ();
type Error = ();
fn into_result(self) -> MyResult {
self.into()
}
fn from_ok(_: <Self as Try>::Ok) -> Self {
Self::NoError
}
fn from_error(_: <Self as Try>::Error) -> Self {
Self::Error
}
}
#[allow(non_snake_case)]
pub extern "C" fn Called_From_C() -> CResult {
let _ = my_rust_function()?;
CResult::NoError
}
Note: Be careful with the enumeration one, make sure your implementation is compatible. #[repr(libc::c_int)] is what we really want but I don't know any way to express this in Rust. So maybe a structure with repr(transparent) is more safe if the lib expect a c_int.

Resources