I have a simple recursive function that uses borrows in Rust. I want to convert it into a for loop. I have made a trimmed down example.
enum Counter {
Val(usize),
}
fn count(n: &Counter) -> Result<usize, String> {
match n {
Counter::Val(a) => {
match a {
0..=9 => {
let b = Counter::Val(a+1);
Ok(count(&b)? + 1)
},
10 => Ok(0),
_ => Err("Out of range".to_string()),
}
},
}
}
fn main() {
print!("{:?}\n", count(&Counter::Val(8)));
}
If I convert without being careful, I end up with
enum Counter {
Val(usize),
}
fn count(n: &Counter) -> Result<usize, String> {
enum State<'a> {
Start(&'a Counter),
Val0(Counter),
}
let mut stack = vec![State::Start(n)];
let mut ret = None;
loop {
match (ret, stack.pop()) {
(Some(r), None) => return Ok(r),
(None, Some(State::Start(n))) => {
match n {
Counter::Val(a) => {
match a {
0..=9 => {
let b = Counter::Val(a+1);
stack.push(State::Val0(b));
let br = match &stack[stack.len()-1] {
State::Val0(a) => a,
_ => return Err("Wrong state type".to_string()),
};
stack.push(State::Start(br));
ret = None;
},
10 => { ret = Some(0); },
_ => return Err("Out of range".to_string()),
}
},
}
},
(Some(r), Some(State::Val0(_))) => {
ret = Some(r + 1);
},
_ => return Err("Invalid state".to_string()),
}
}
}
fn main() {
print!("{:?}\n", count(&Counter::Val(8)));
}
This is correctly rejected by the borrow checker.
error[E0502]: cannot borrow `stack` as mutable because it is also borrowed as immutable
--> a1.rs:15:17
|
15 | match (ret, stack.pop()) {
| ^^^^^^^^^^^ mutable borrow occurs here
...
23 | stack.push(State::Val0(b));
| -------------------------- immutable borrow later used here
24 | let br = match &stack[stack.len()-1] {
| ----- immutable borrow occurs here
error[E0502]: cannot borrow `stack` as mutable because it is also borrowed as immutable
--> a1.rs:23:17
|
23 | stack.push(State::Val0(b));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
24 | let br = match &stack[stack.len()-1] {
| -----
| |
| immutable borrow occurs here
| immutable borrow later used here
error[E0502]: cannot borrow `stack` as mutable because it is also borrowed as immutable
--> a1.rs:28:17
|
24 | let br = match &stack[stack.len()-1] {
| ----- immutable borrow occurs here
...
28 | stack.push(State::Start(br));
| ^^^^^^----^^^^^^^^^^^^^^^^^^
| | |
| | immutable borrow later used by call
| mutable borrow occurs here
The stack might need to grow, leading to a reallocation, invalidating all of the borrows. So, this program is incorrect. To account for that, I can use Box to
have pointers to the heap, that won't change.
However, this still gets a similar error. To get
it to compile, I need to add an unsafe to
circumvent the borrow checker.
enum Counter {
Val(usize),
}
fn count(n: &Counter) -> Result<usize, String> {
enum State<'a> {
Start(&'a Counter),
Val0(Box<Counter>),
}
let mut stack = vec![State::Start(n)];
let mut ret = None;
loop {
match (ret, stack.pop()) {
(Some(r), None) => return Ok(r),
(None, Some(State::Start(n))) => {
match n {
Counter::Val(a) => {
match a {
0..=9 => {
let b = Counter::Val(a+1);
stack.push(State::Val0(Box::new(b)));
let br = match &stack[stack.len()-1] {
State::Val0(a) => &**a,
_ => return Err("Wrong state type".to_string()),
};
let br0 = unsafe { &*(br as *const Counter) };
stack.push(State::Start(br0));
ret = None;
},
10 => { ret = Some(0); },
_ => return Err("Out of range".to_string()),
}
},
}
},
(Some(r), Some(State::Val0(_))) => {
ret = Some(r + 1);
},
_ => return Err("Invalid state".to_string()),
}
}
}
fn main() {
print!("{:?}\n", count(&Counter::Val(8)));
}
To avoid using unsafe, I could also use a separate data stack.
enum Counter {
Val(usize),
}
fn count(n: &Counter) -> Result<usize, String> {
enum State {
Start(usize),
Val0,
}
let mut data = vec![];
let mut stack = vec![State::Start(0)];
let mut ret = None;
loop {
match (ret, stack.pop()) {
(Some(r), None) => return Ok(r),
(None, Some(State::Start(nr))) => {
let n0 = if nr == 0 {
n
} else {
&data[nr-1]
};
match n0 {
Counter::Val(a) => {
match a {
0..=9 => {
let b = Counter::Val(a+1);
data.push(b);
stack.push(State::Val0);
stack.push(State::Start(data.len()));
ret = None;
},
10 => { ret = Some(0); },
_ => return Err("Out of range".to_string()),
}
},
}
},
(Some(r), Some(State::Val0)) => {
data.pop();
ret = Some(r + 1);
},
_ => return Err("Invalid state".to_string()),
}
}
}
fn main() {
print!("{:?}\n", count(&Counter::Val(8)));
}
This also works.
My questions are:
That the borrow checker works for the recursive form, is there a way to explain the stack form to it (the Box version)?
Is there a reason to prefer the unsafe with the Box form over the data stack form? In both cases, the borrow checker has been circumvented, so they are both equally correct, as far as I can tell.
Is there a way to wrap a data structure around this, perhaps using unsafe internally, but exposing a safe exterior, that could allow writing this function with keeping the benefit of borrow checking?
Is there an idiomatic way to write the for loop form of the function?
Related
I have the following code:
use std::collections::HashMap;
type Elem = i64;
enum Op {
Num(Elem),
Add((String, String)),
}
type Statements = HashMap<String, Op>;
fn main() {
let mut statements = Statements::new();
statements.insert("lhs".to_string(), Op::Num(64));
statements.insert("op".to_string(), Op::Add(("lhs".to_string(), "rhs".to_string())));
statements.insert("rhs".to_string(), Op::Num(64));
let mut stack = Vec::new();
let mut op_name = "op";
loop {
let result: Elem;
match statements.get(op_name).unwrap() {
Op::Num(_) => {
panic!("{}: Did not expect Num", op_name);
},
Op::Add((lhs, rhs)) => {
if let Op::Num(lhs_value) = statements.get(lhs).unwrap() {
if let Op::Num(rhs_value) = statements.get(rhs).unwrap() {
result = lhs_value + rhs_value;
} else {
stack.push(op_name);
op_name = rhs;
continue;
}
} else {
stack.push(op_name);
op_name = lhs;
continue;
}
},
};
if let Some(new_op_name) = stack.pop() {
*(statements.get_mut(op_name).unwrap()) = Op::Num(result);
op_name = new_op_name;
} else {
println!("Result: {}", result);
return;
}
}
}
which gives me the following compiler error:
error[E0502]: cannot borrow `statements` as mutable because it is also borrowed as immutable
--> minimal.rs:43:15
|
22 | match statements.get(op_name).unwrap() {
| ----------------------- immutable borrow occurs here
...
43 | *(statements.get_mut(op_name).unwrap()) = Op::Num(result);
| ^^^^^^^^^^^-------^^^^^^^^^
| | |
| | immutable borrow later used by call
| mutable borrow occurs here
error: aborting due to previous error
For more information about this error, try `rustc --explain E0502`.
I was expecting the borrowing to end with the match statement, since I do not see how I could still access the result of the borrowing after this statement. Why is the value still being borrowed after the match statement?
Is there something I could do to make my code above work?
In my opinion you'll find it difficult to work with strings and strings references. I would rather switch to some type for your HashMap keys that would be small, and copyable, like an integer number. This way you'll not be required to deal with references and mutability - you can just copy your integers with little to no cost.
I resolved your compilation errors, but the overall code becomes more convoluted. Basically now each insert into the hash map will give you a unique integer ID of the operation. You can only query operations from the hash map using this provided ID. It also requires you to push statements in your hash map in the order of appearance. You can't insert add(lhs, rhs) unless you already know IDs of both lhs and rhs
use std::collections::HashMap;
type Elem = i64;
type StatementId = usize;
enum Op {
Num(Elem),
Add((StatementId, StatementId)),
}
#[derive(Default)]
struct StatementsRecord {
record: HashMap<StatementId, (String, Op)>,
next_statement_id: StatementId,
}
impl StatementsRecord {
fn insert(&mut self, name: String, operation: Op) -> StatementId {
self.record.insert(self.next_statement_id, (name, operation));
self.next_statement_id += 1;
self.next_statement_id - 1 // return id of newly stored operation
}
fn get(&self, id: &StatementId) -> Option<&(String, Op)> {
self.record.get(id)
}
fn get_mut(&mut self, id: &StatementId) -> Option<&mut (String, Op)> {
self.record.get_mut(id)
}
fn update(&mut self, id: &StatementId, value: Op) {
self.get_mut(id).unwrap().1 = value;
}
}
fn main() {
let mut statements = StatementsRecord::default();
let lhs_id = statements.insert("lhs".to_string(), Op::Num(64));
let rhs_id = statements.insert("rhs".to_string(), Op::Num(64));
// 64+64 = 128
let mut op_id = statements.insert("op".to_string(), Op::Add((lhs_id, rhs_id)));
// (64+64) + (64+64) = 256
let mut op_id = statements.insert("op".to_string(), Op::Add((op_id, op_id)));
let mut stack = Vec::new();
loop {
let result: Elem;
match statements.get(&op_id).unwrap() {
(op_name, Op::Num(_)) => {
panic!("{}: Did not expect Num", op_name);
},
(op_name, Op::Add((lhs, rhs))) => {
if let (_, Op::Num(lhs_value)) = statements.get(lhs).unwrap() {
if let (_, Op::Num(rhs_value)) = statements.get(rhs).unwrap() {
result = lhs_value + rhs_value;
} else {
stack.push(op_id);
op_id = *rhs;
continue;
}
} else {
stack.push(op_id);
op_id = *lhs;
continue;
}
},
};
if let Some(new_op_id) = stack.pop() {
statements.update(&op_id, Op::Num(result));
op_id = new_op_id;
} else {
println!("Result: {}", result);
return;
}
}
}
Prints
Result: 256
I'm trying to learn Rust after many years of C++. I have a situation where the compiler is complaining about a borrow, and it doesn't seem to matter whether it is mutable or immutable. I don't seem to be able to use self as a parameter inside a loop that start with: for item in self.func.drain(..).I've tried calling appropriate() as a function:
Self::appropriate(&self,&item,index)
and I have tried it as a method:
self.appropriate(&item,index)
but I get the same message in either case:
The function or method appropriate() is intended imply examine the relationship among its parameters and return a bool without modifying anything. How can I call either a function or method on self without violating borrowing rules?This program is a learning exercise from exercism.org and doesn't include a main() so it won't run but should almost compile except for the error in question. Here's the code I have:
use std::collections::HashMap;
pub type Value = i32;
pub type Result = std::result::Result<(), Error>;
pub struct Forth {
v: Vec<Value>,
f: HashMap<String,usize>,
s: Vec<Vec<String>>,
func: Vec<String>
}
#[derive(Debug, PartialEq)]
pub enum Error {
DivisionByZero,
StackUnderflow,
UnknownWord,
InvalidWord,
}
impl Forth {
pub fn new() -> Forth {
let mut temp: Vec<Vec<String>> = Vec::new();
temp.push(Vec::new());
Forth{v: Vec::<Value>::new(), f: HashMap::new(), s: temp, func: Vec::new()}
}
pub fn stack(&self) -> &[Value] {
&self.v
}
pub fn eval(&mut self, input: &str) -> Result {
self.v.clear();
self.s[0].clear();
let mut count = 0;
{
let temp: Vec<&str> = input.split(' ').collect();
let n = temp.len() as i32;
for x in 0..n as usize {
self.s[0].push(String::from(temp[x]));
}
}
let mut collecting = false;
let mut xlist: Vec<(usize,usize)> = Vec::new();
let mut sx: usize = 0;
let mut z: i32 = -1;
let mut x: usize;
let mut n: usize = self.s[0].len();
loop {
count += 1;
if count > 20 {break;}
z += 1;
x = z as usize;
if x >= n {break;}
z = x as i32;
let word = &self.s[sx][x];
if word == ";" {
if collecting {
collecting = false;
let index: usize = self.s.len();
self.s.push(Vec::<String>::new());
for item in self.func.drain(..) {
if self.s[index].len() > 0 &&
Self::appropriate(&self,&item,index)
{
let sx = *self.f.get(&self.s[index][0]).unwrap();
let n = self.s[sx].len();
for x in 1..n as usize {
let symbol = self.s[sx][x].clone();
self.s[index].push(symbol);
}
}
else {
self.s[index].push(item);
}
}
self.f.insert(self.s[index][0].clone(), index);
self.func.clear();
continue;
}
if 0 < xlist.len() {
(x, n) = xlist.pop().unwrap();
continue;
}
return Err(Error::InvalidWord);
}
if collecting {
self.func.push(String::from(word));
continue;
}
if Self::is_op(word) {
if self.v.len() < 2 {
return Err(Error::StackUnderflow);
}
let b = self.v.pop().unwrap();
let a = self.v.pop().unwrap();
let c = match word.as_str() {
"+" => a + b,
"-" => a - b,
"*" => a * b,
"/" => {if b == 0 {return Err(Error::DivisionByZero);} a / b},
_ => 0
};
self.v.push(c);
continue;
}
match word.parse::<Value>() {
Ok(value) => { self.v.push(value); continue;},
_ => {}
}
if word == ":" {
collecting = true;
self.func.clear();
continue;
}
if word == "drop" {
if self.v.len() < 1 {
return Err(Error::StackUnderflow);
}
self.v.pop();
continue;
}
if word == "dup" {
if self.v.len() < 1 {
return Err(Error::StackUnderflow);
}
let temp = self.v[self.v.len() - 1];
self.v.push(temp);
continue;
}
if !self.f.contains_key(word) {
return Err(Error::UnknownWord);
}
xlist.push((sx,n));
sx = *self.f.get(word).unwrap();
n = self.s[sx].len();
z = 0;
}
Ok(())
}
fn is_op(input: &str) -> bool {
match input {"+"|"-"|"*"|"/" => true, _ => false}
}
fn appropriate(&self, item:&str, index:usize) -> bool
{
false
}
fn prev_def_is_short(&self, index: usize) -> bool {
if index >= self.s.len() {
false
}
else {
if let Some(&sx) = self.f.get(&self.func[0]) {
self.s[sx].len() == 2
}
else {
false
}
}
}
}
The error message relates to the call to appropriate(). I haven't even written the body of that function yet; I'd like to get the parameters right first. The compiler's complaint is:
As a subroutine call
error[E0502]: cannot borrow `self` as immutable because it is also borrowed as mutable
--> src/lib.rs:85:47
|
81 | for item in self.func.drain(..) {
| -------------------
| |
| mutable borrow occurs here
| mutable borrow later used here
...
85 | Self::appropriate(&self,&item,index)
| ^^^^^ immutable borrow occurs here
For more information about this error, try `rustc --explain E0502`.
as a method call
error[E0502]: cannot borrow `*self` as immutable because it is also borrowed as mutable
--> src/lib.rs:85:29
|
81 | for item in self.func.drain(..) {
| -------------------
| |
| mutable borrow occurs here
| mutable borrow later used here
...
85 | self.appropriate(&item,index)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ immutable borrow occurs here
For more information about this error, try `rustc --explain E0502`.
Is there any canonical way to deal with this situation?
The problem is that self.func.drain() will consume the elements contained in self.func, thus an exclusive (&mut) access is needed on self.func for the entire for loop.
If during the iteration you need to pass a reference to self globally, then its func member is potentially accessible while the loop holds an exclusive access to it: Rust forbids that.
Since you use drain() in order to consume all the elements inside self.func, I suggest you swap this vector with an empty one just before the loop, then iterate on this other vector that is not anymore part of self.
No copy of the content of the vector is involved here; swap() only deals with pointers.
Here is an over-simplified version of your code, adapted consequently.
struct Forth {
func: Vec<String>,
}
impl Forth {
fn eval(&mut self) {
/*
for item in self.func.drain(..) {
self.appropriate(&self);
}
*/
let mut func = Vec::new();
std::mem::swap(&mut self.func, &mut func);
for item in func.drain(..) {
let b = self.appropriate();
println!("{:?} {:?}", item, b);
}
}
fn appropriate(&self) -> bool {
false
}
}
fn main() {
let mut f = Forth {
func: vec!["aaa".into(), "bbb".into()],
};
f.eval();
}
I'm trying to solve the RPN calculator exercise at exercism but stumbled upon this temporary value dropped while borrowed error that I can't seem to work out.
Here's my code:
#[derive(Debug)]
pub enum CalculatorInput {
Add,
Subtract,
Multiply,
Divide,
Value(i32),
}
pub fn evaluate(inputs: &[CalculatorInput]) -> Option<i32> {
let mut stack = Vec::new();
for input in inputs {
match input {
CalculatorInput::Value(value) => {
stack.push(value);
},
operator => {
if stack.len() < 2 {
return None;
}
let second = stack.pop().unwrap();
let first = stack.pop().unwrap();
let result = match operator {
CalculatorInput::Add => first + second,
CalculatorInput::Subtract => first - second,
CalculatorInput::Multiply => first * second,
CalculatorInput::Divide => first / second,
CalculatorInput::Value(_) => return None,
};
stack.push(&result.clone());
}
}
}
if stack.len() != 1 {
None
} else {
Some(*stack.pop().unwrap())
}
}
And the error I get:
error[E0716]: temporary value dropped while borrowed
--> src/lib.rs:32:29
|
32 | stack.push(&result.clone());
| ^^^^^^^^^^^^^^ - temporary value is freed at the end of this statement
| |
| creates a temporary which is freed while still in use
...
36 | if stack.len() != 1 {
| ----- borrow later used here
|
= note: consider using a `let` binding to create a longer lived value
If I understand correctly, the variable result is no loger live outside of the for loop (outside of the operator match branch indeed), that's why I cloned it, but it still gives me the same error.
How can I make a copy of the result which is owned by the stack Vec (if that's what I should do)?
Just for reference, and in case anybody fins this useful, this is the final solution taking into account all the help received:
use crate::CalculatorInput::{Add,Subtract,Multiply,Divide,Value};
#[derive(Debug)]
pub enum CalculatorInput {
Add,
Subtract,
Multiply,
Divide,
Value(i32),
}
pub fn evaluate(inputs: &[CalculatorInput]) -> Option<i32> {
let mut stack: Vec<i32> = Vec::new();
for input in inputs {
match input {
Value(value) => {
stack.push(*value);
},
operator => {
if stack.len() < 2 {
return None;
}
let second: i32 = stack.pop().unwrap();
let first: i32 = stack.pop().unwrap();
let result: i32 = match operator {
Add => first + second,
Subtract => first - second,
Multiply => first * second,
Divide => first / second,
Value(_) => return None,
};
stack.push(result);
}
}
}
if stack.len() != 1 {
None
} else {
stack.pop()
}
}
No need to clone, because i32 implements the Copy trait.
The problem was that my vec was receiving an &i32 instead of i32, and thus rust infered it to be a Vec<&i32>.
The error is because Rust did not infer the type you expected.
In your code, the type of value is inferred to be &i32 because input is a reference of a element in inputs, and you push a value later, therefore the type of stack is inferred to be Vec<&i32>.
A best fix is to explicitly specify the type of stack:
let mut stack: Vec<i32> = Vec::new();
And because i32 has implemented Copy trait, you should never need to clone a i32 value, if it is a reference, just dereference it.
Fixed code:
#[derive(Debug)]
pub enum CalculatorInput {
Add,
Subtract,
Multiply,
Divide,
Value(i32),
}
pub fn evaluate(inputs: &[CalculatorInput]) -> Option<i32> {
let mut stack: Vec<i32> = Vec::new();
for input in inputs {
match input {
CalculatorInput::Value(value) => {
stack.push(*value);
}
operator => {
if stack.len() < 2 {
return None;
}
let second = stack.pop().unwrap();
let first = stack.pop().unwrap();
let result = match operator {
CalculatorInput::Add => first + second,
CalculatorInput::Subtract => first - second,
CalculatorInput::Multiply => first * second,
CalculatorInput::Divide => first / second,
CalculatorInput::Value(_) => return None,
};
stack.push(result);
}
}
}
if stack.len() != 1 {
None
} else {
Some(stack.pop().unwrap())
}
}
You have the same behavior with this simple exemple
fn main() {
let mut stack = Vec::new();
let a = String::from("test");
stack.push(&a.clone());
//-------- ^
println!("{:?}", stack);
}
and the good way is to not borrow when clone.
fn main() {
let mut stack = Vec::new();
let a = String::from("test");
stack.push(a.clone());
//-------- ^
println!("{:?}", stack);
}
The variable should be used like this stack.push(result.clone()); and change code like this
pub fn evaluate(inputs: &[CalculatorInput]) -> Option<i32> {
let mut stack: Vec<i32> = Vec::new();
//---------------- ^
for input in inputs {
match input {
CalculatorInput::Value(value) => {
stack.push(value.clone());
//----------------- ^
},
operator => {
if stack.len() < 2 {
return None;
}
let second = stack.pop().unwrap();
let first = stack.pop().unwrap();
let result = match operator {
CalculatorInput::Add => first + second,
CalculatorInput::Subtract => first - second,
CalculatorInput::Multiply => first * second,
CalculatorInput::Divide => first / second,
CalculatorInput::Value(_) => return None,
};
stack.push(result.clone());
//-^
}
}
}
if stack.len() != 1 {
None
} else {
Some(stack.pop().unwrap())
//------- ^
}
}
I'm trying to implement a function that steps though two iterators at the same time, calling a function for each pair. This callback can control which of the iterators is advanced in each step by returning a (bool, bool) tuple. Since the iterators take a reference to a buffer in my use case, they can't implement the Iterator trait from the stdlib, but instead are used though a next_ref function, which is identical to Iterator::next, but takes an additional lifetime parameter.
// An iterator-like type, that returns references to itself
// in next_ref
struct RefIter {
value: u64
}
impl RefIter {
fn next_ref<'a>(&'a mut self) -> Option<&'a u64> {
self.value += 1;
Some(&self.value)
}
}
// Iterate over two RefIter simultaneously and call a callback
// for each pair. The callback returns a tuple of bools
// that indicate which iterators should be advanced.
fn each_zipped<F>(mut iter1: RefIter, mut iter2: RefIter, callback: F)
where F: Fn(&Option<&u64>, &Option<&u64>) -> (bool, bool)
{
let mut current1 = iter1.next_ref();
let mut current2 = iter2.next_ref();
loop {
let advance_flags = callback(¤t1, ¤t2);
match advance_flags {
(true, true) => {
current1 = iter1.next_ref();
current2 = iter2.next_ref();
},
(true, false) => {
current1 = iter1.next_ref();
},
(false, true) => {
current2 = iter1.next_ref();
},
(false, false) => {
return
}
}
}
}
fn main() {
let mut iter1 = RefIter { value: 3 };
let mut iter2 = RefIter { value: 4 };
each_zipped(iter1, iter2, |val1, val2| {
let val1 = *val1.unwrap();
let val2 = *val2.unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:28:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
28 | current1 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter2` as mutable more than once at a time
--> src/main.rs:29:28
|
23 | let mut current2 = iter2.next_ref();
| ----- first mutable borrow occurs here
...
29 | current2 = iter2.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:32:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
32 | current1 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
error[E0499]: cannot borrow `iter1` as mutable more than once at a time
--> src/main.rs:35:28
|
22 | let mut current1 = iter1.next_ref();
| ----- first mutable borrow occurs here
...
35 | current2 = iter1.next_ref();
| ^^^^^ second mutable borrow occurs here
...
42 | }
| - first borrow ends here
I understand why it complains, but can't find a way around it. I'd appreciate any help on the subject.
Link to this snippet in the playground.
Since the iterators take a reference to a buffer in my use case, they can't implement the Iterator trait from the stdlib, but instead are used though a next_ref function, which is identical to Iterator::next, but takes an additional lifetime parameter.
You are describing a streaming iterator. There is a crate for this, aptly called streaming_iterator. The documentation describes your problem (emphasis mine):
While the standard Iterator trait's functionality is based off of
the next method, StreamingIterator's functionality is based off of
a pair of methods: advance and get. This essentially splits the
logic of next in half (in fact, StreamingIterator's next method
does nothing but call advance followed by get).
This is required because of Rust's lexical handling of borrows (more
specifically a lack of single entry, multiple exit borrows). If
StreamingIterator was defined like Iterator with just a required
next method, operations like filter would be impossible to define.
The crate does not currently have a zip function, and certainly not the variant you have described. However, it's easy enough to implement:
extern crate streaming_iterator;
use streaming_iterator::StreamingIterator;
fn each_zipped<A, B, F>(mut iter1: A, mut iter2: B, callback: F)
where
A: StreamingIterator,
B: StreamingIterator,
F: for<'a> Fn(Option<&'a A::Item>, Option<&'a B::Item>) -> (bool, bool),
{
iter1.advance();
iter2.advance();
loop {
let advance_flags = callback(iter1.get(), iter2.get());
match advance_flags {
(true, true) => {
iter1.advance();
iter2.advance();
}
(true, false) => {
iter1.advance();
}
(false, true) => {
iter1.advance();
}
(false, false) => return,
}
}
}
struct RefIter {
value: u64
}
impl StreamingIterator for RefIter {
type Item = u64;
fn advance(&mut self) {
self.value += 1;
}
fn get(&self) -> Option<&Self::Item> {
Some(&self.value)
}
}
fn main() {
let iter1 = RefIter { value: 3 };
let iter2 = RefIter { value: 4 };
each_zipped(iter1, iter2, |val1, val2| {
let val1 = *val1.unwrap();
let val2 = *val2.unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
The issue with this code is that RefIter is being used in two ways, which are fundamentally at odds with one another:
Callers of next_ref recieve a reference to the stored value, which is tied to the lifetime of RefIter
RefIter's value needs to be mutable, so that it can be incremented on each call
This perfectly describes mutable aliasing (you're trying to modify 'value' while a reference to it is being held) - something which Rust is explicitly designed to prevent.
In order to make each_zipped work, you'll need to modify RefIter to avoid handing out references to data that you wish to mutate.
I've implemented one possibility below using a combination of RefCell and Rc:
use std::cell::RefCell;
use std::rc::Rc;
// An iterator-like type, that returns references to itself
// in next_ref
struct RefIter {
value: RefCell<Rc<u64>>
}
impl RefIter {
fn next_ref(&self) -> Option<Rc<u64>> {
let new_val = Rc::new(**self.value.borrow() + 1);
*self.value.borrow_mut() = new_val;
Some(Rc::clone(&*self.value.borrow()))
}
}
// Iterate over two RefIter simultaniously and call a callback
// for each pair. The callback returns a tuple of bools
// that indicate which iterators should be advanced.
fn each_zipped<F>(iter1: RefIter, iter2: RefIter, callback: F)
where F: Fn(&Option<Rc<u64>>, &Option<Rc<u64>>) -> (bool, bool)
{
let mut current1 = iter1.next_ref();
let mut current2 = iter2.next_ref();
loop {
let advance_flags = callback(¤t1, ¤t2);
match advance_flags {
(true, true) => {
current1 = iter1.next_ref();
current2 = iter2.next_ref();
},
(true, false) => {
current1 = iter1.next_ref();
},
(false, true) => {
current2 = iter1.next_ref();
},
(false, false) => {
return
}
}
}
}
fn main() {
let iter1 = RefIter { value: RefCell::new(Rc::new(3)) };
let iter2 = RefIter { value: RefCell::new(Rc::new(4)) };
each_zipped(iter1, iter2, |val1, val2| {
// We can't use unwrap() directly, since we're only passed a reference to an Option
let val1 = **val1.iter().next().unwrap();
let val2 = **val2.iter().next().unwrap();
println!("{}, {}", val1, val2);
(val1 < 10, val2 < 10)
});
}
This version of RefIter hands out Rcs to consumers, instead of references. This avoids the issue of mutable aliasing - updating value is done by placing
a new Rc into the outer RefCell. A side effect of this is that consumers are able to hold onto an 'old' reference to the buffer (through the returned Rc), even after RefIter has advanced.
I'm trying to use the code I found here with some problems, basically it's a borrowing error using some macros, the error:
Compiling playground v0.0.1 (file:///playground)
error[E0597]: `body` does not live long enough
--> src/main.rs:12:3
|
6 | let raw_structure = borrow_function(&body);
| ---- borrow occurs here
...
12 | }
| ^ `body` dropped here while still borrowed
...
31 | let body = get_body_as!(&str, "Hello", function1);
| -------------------------------------- in this macro invocation
32 | println!("Hello");
33 | }
| - borrowed value needs to live until here
I manage to create a Minimal, Complete, and Verifiable example, I was thinking that a solution would be to transform the macros into functions, but I'm not completely sure how to do that either (Playground
):
macro_rules! get_body_as {
($structure:ty, $req:expr, $error_fn:ident) => {
{
let body = get_body!($req, $error_fn);
let raw_structure = borrow_function(&body);
match raw_structure {
Ok(structure) => structure,
Err(error) => "Error"
}
}
}
}
macro_rules! get_body {
($req:expr, $error_fn:ident) => {
{
let mut payload = String::new();
payload
}
}
}
fn borrow_function(s: &str) -> Result<&str, &str> {
Ok(s)
}
fn main() {
let function1 = |s: &str| s;
let body = get_body_as!(&str, "Hello", function1);
println!("Hello");
}
The problem is that you are trying to return a reference to a body variable from a block which owns the body variable, but body is to be dropped at the end of that block, so the reference would outlive the data it references.
If you want your example to compile, you can alter your code so that body is declared within the main function using ident parameter added to get_body_as macro:
macro_rules! get_body_as {
($structure:ty, $req:expr, $error_fn:ident, $body: ident) => {
let $body = get_body!($req, $error_fn);
let raw_structure = borrow_function(&$body);
match raw_structure {
Ok(structure) => structure,
Err(error) => "Error"
}
}
}
macro_rules! get_body {
($req:expr, $error_fn:ident) => {
{
let mut payload = String::new();
payload
}
}
}
fn borrow_function(s: &str) -> Result<&str, &str> {
Ok(s)
}
fn main() {
let function1 = |s: &str| s;
get_body_as!(&str, "Hello", function1, body);
println!("Hello");
}
This example compiles, but still has warnings about unused variables, I have made only minimal changes for compilation to succeed.