I'm writing a bunch of assertions that all involve popping a value from a list.
Coming from a Scala background, I naturally did this:
let mut list = List::new();
let assert_pop = |expected| assert_eq!(list.pop(), expected);
So that I could just write assert_pop(None) or assert_pop(Some(3)) instead of having to write assert_eq!(list.pop(), None) or assert_eq!(list.pop(), Some(3)) every time.
Of course the borrow checker doesn't like this one bit because the closure essentially needs to borrow the value for an undisclosed amount of time, while the rest of my code goes around mutating, thus violating the rule of "no aliasing if you're mutating".
The question is: is there a way to get around this? Do I have to write a macro, or is there a funky memory-safe way I can get around this?
Note that I know that I can just define the closure like this:
let_assert_pop = |lst: &mut List, expected| assert_eq!(lst.pop(), expected);
But that would be less DRY, as I'd have to pass in a &mut list as the first argument at every call.
They key is to define the closure as mut, since it needs a mutable reference.
This works:
let mut v = vec![1, 2];
let mut assert_pop = |expected| assert_eq!(v.pop(), expected);
assert_pop(Some(2));
assert_pop(Some(1));
assert_pop(None);
Note that the the pop closure borrows mutably, so you if you want to use the list afterwards, you have to scope it:
let mut v = vec![1,2];
{
let mut assert_pop = |expected| assert_eq!(v.pop(), expected);
assert_pop(Some(2));
v.push(33); // ERROR: v is borrowed mutably...
}
v.push(33); // Works now, since pop is out of scope.
Instead of directly answering your question (which is well-enough answered already), I'll instead address your other points:
just write assert_pop(None) or assert_pop(Some(3))
a memory-safe way solution
don't pass in a &mut list
To solve all that, don't use a closure, just make a new type:
type List<T> = Vec<T>;
struct Thing<T>(List<T>);
impl<T> Thing<T> {
fn assert_pop(&mut self, expected: Option<T>)
where T: PartialEq + std::fmt::Debug,
{
assert_eq!(self.0.pop(), expected);
}
}
fn main() {
let list = List::new();
let mut list = Thing(list);
list.0.push(1);
list.assert_pop(Some(1));
list.assert_pop(None);
// Take it back if we need to
let _list = list.0;
}
Related
My questions seems to be closely related to Rust error "cannot infer an appropriate lifetime for borrow expression" when attempting to mutate state inside a closure returning an Iterator, but I think it's not the same. So, this
use std::iter;
fn example(text: String) -> impl Iterator<Item = Option<String>> {
let mut i = 0;
let mut chunk = None;
iter::from_fn(move || {
if i <= text.len() {
let p_chunk = chunk;
chunk = Some(&text[..i]);
i += 1;
Some(p_chunk.map(|s| String::from(s)))
} else {
None
}
})
}
fn main() {}
does not compile. The compiler says it cannot determine the appropriate lifetime for &text[..i]. This is the smallest example I could come up with. The idea being, there is an internal state, which is a slice of text, and the iterator returns new Strings allocated from that internal state. I'm new to Rust, so maybe it's all obvious, but how would I annotate lifetimes so that this compiles?
Note that this example is different from the linked example, because there point was passed as a reference, while here text is moved. Also, the answer there is one and half years old by now, so maybe there is an easier way.
EDIT: Added p_chunk to emphasize that chunk needs to be persistent across calls to next and so cannot be local to the closure but should be captured by it.
Your code is an example of attempting to create a self-referential struct, where the struct is implicitly created by the closure. Since both text and chunk are moved into the closure, you can think of both as members of a struct. As chunk refers to the contents in text, the result is a self-referential struct, which is not supported by the current borrow checker.
While self-referential structs are unsafe in general due to moves, in this case it would be safe because text is heap-allocated and is not subsequently mutated, nor does it escape the closure. Therefore it is impossible for the contents of text to move, and a sufficiently smart borrow checker could prove that what you're trying to do is safe and allow the closure to compile.
The answer to the [linked question] says that referencing through an Option is possible but the structure cannot be moved afterwards. In my case, the self-reference is created after text and chunk were moved in place, and they are never moved again, so in principle it should work.
Agreed - it should work in principle, but it is well known that the current borrow checker doesn't support it. The support would require multiple new features: the borrow checker should special-case heap-allocated types like Box or String whose moves don't affect references into their content, and in this case also prove that you don't resize or mem::replace() the closed-over String.
In this case the best workaround is the "obvious" one: instead of persisting the chunk slice, persist a pair of usize indices (or a Range) and create the slice when you need it.
If you move the chunk Option into the closure, your code compiles. I can't quite answer why declaring chunk outside the closure results in a lifetime error for the borrow of text inside the closure, but the chunk Option looks superfluous anyways and the following code should be equivalent:
fn example(text: String) -> impl Iterator<Item = Option<String>> {
let mut i = 0;
iter::from_fn(move || {
if i <= text.len() {
let chunk = text[..i].to_string();
i += 1;
Some(Some(chunk))
} else {
None
}
})
}
Additionally, it seems unlikely that you really want an Iterator<Item = Option<String>> here instead of an Iterator<Item<String>>, since the iterator never yields Some(None) anyways.
fn example(text: String) -> impl Iterator<Item = String> {
let mut i = 0;
iter::from_fn(move || {
if i <= text.len() {
let chunk = text[..i].to_string();
i += 1;
Some(chunk)
} else {
None
}
})
}
Note, you can also go about this iterator without allocating a String for each chunk, if you take a &str as an argument and tie the lifetime of the output to the input argument:
fn example<'a>(text: &'a str) -> impl Iterator<Item = &'a str> + 'a {
let mut i = 0;
iter::from_fn(move || {
if i <= text.len() {
let chunk = &text[..i];
i += 1;
Some(chunk)
} else {
None
}
})
}
I have
fn main() {
let x = String::from("12");
fun1(&x);
}
fn fun1(in_fun: &String) {
let mut y = _______;
y.push_str("z");
println!("in fun {}", y);
}
where _____ is the code for declaring y based on the argument in_fun.
At first I tried let mut y = *in_fun; which errors move occurs because '*in_fun' has type 'String', which does not implement the 'Copy' trait and also let mut y = String::from(*in_fun); which gives same error.
The thing that worked was let mut y = String::from(format!("{}", *in_fun));.
Is this the right way to declare a mutable String from &String?
Also I still don't understand why dereferencing &String with * errors? I understood *& dereferencing to returns just the value of the reference.
First of all, the working code:
fn fun1(in_fun: &String) {
let mut y = in_fun.clone();
y.push_str("z");
println!("in fun {}", y);
}
Or, your instincts tell you you have to dereference, so (*in_fun).clone() works just the same, but is a bit redundant. *in_fun.clone() does NOT work because it's equivalent to *(in_fun.clone()) (dereferencing the clone), which isn't what you want. The reason you don't need to dereference the reference before calling clone is because Rust's method resolution allows you to call methods of a type or access properties of a type using a reference to the type, and .clone has an &self receiver.
The reason that let mut y = *in_fun doesn't work is because this attempts to move the string out from underneath the reference, which doesn't work.
&String is an immutable reference. Rust is strict about this and prevents many common mishaps we people tend to run into. Dereferencing &String is not possible as it would break the guarantees of safety in rust, allowing you to modify where you only have read access. See the ownership explanation.
The function should either accept a mutable reference &mut String (then the string can be modified in place) or it needs to .clone() the string from the immutable reference.
Taking a mutable reference is more efficient than cloning, but it restricts the caller from sharing it immutably in parallel.
If the only thing you want to achieve is to print out some additional information, the best way I know of is:
fn fun1<S: std::fmt::Display>(in_fun: S) {
println!("in fun {}z", in_fun);
}
fn main() {
let mut x = String::from("12");
fun1(&x);
fun1(&mut x);
fun1(x);
fun1("12");
}
I use a Display trait so anything that implements will do. See the playground.
On the other hand, if you really need an owned string, then ask for it :)
fn fun1<S: Into<String>>(in_fun: S) {
let mut y = in_fun.into();
y.push('z');
println!("in fun {}", y);
}
fn main() {
let x = String::from("12");
fun1(&x);
fun1(x);
fun1("12");
}
This way you can accept both &str and String and keep efficient, avoiding cloning if possible.
I would like to have a vector with functions. Then I would like to iterate on this vector and execute functions one by one. The functions would mutate an external state. Additionally, I would like to be able to place the same function twice in the vector.
The problems I have are:
I cannot dereference and execute the function from the vector,
Adding the same function to the vector twice fails with, understandable, error that I cannot have two mutable references.
The closest I got is:
fn main() {
let mut c = 0;
{
let mut f = ||{c += 1};
let mut v: Vec<&mut FnMut()> = vec![];
v.push(&mut f);
// How to execute the stored function? The following complains about
// an immutable reference:
// assignment into an immutable reference
// (v[0])();
// How to store the same function twice? The following will fail with:
// cannot borrow `f` as mutable more than once at a time
// v.push(&mut f);
}
println!("c {}", c);
}
For the first problem, I don't really know why no mutable dereference happens here (in my opinion, it should), but there is a simple workaround: just do the dereference and then reference manually:
(&mut *v[0])();
Your second problem is more complex, though. There is no simple solution, because what you're trying to do violates Rust aliasing guarantees, and since you did not describe the purpose of it, I can't suggest alternatives properly. In general, however, you can overcome this error by switching to runtime borrow-checking with Cell/RefCell or Mutex (the latter is when you need concurrent access). With Cell (works nice for primitives):
use std::cell::Cell;
fn main() {
let c = Cell::new(0);
{
let f = || { c.set(c.get() + 1); };
let mut v: Vec<&Fn()> = vec![];
v.push(&f);
v.push(&f);
v[0]();
v[1]();
}
println!("c {}", c.get());
}
With RefCell (works nice for more complex types):
use std::cell::RefCell;
fn main() {
let c = RefCell::new(0);
{
let f = || { *c.borrow_mut() += 1; };
let mut v: Vec<&Fn()> = vec![];
v.push(&f);
v.push(&f);
v[0]();
v[1]();
}
println!("c {}", *c.borrow());
}
As you can see, now you have &Fn() instead of &mut FnMut(), which can be aliased freely, and whose captured environment may also contain aliased references (immutable, of course).
What is the design rationale for supplying an iter_mut function for HashMap but not HashSet in Rust?
Would it be a faux pas to roll one's own (assuming that can even be done)?
Having one could alleviate situations that give rise to
previous borrow of X occurs here; the immutable borrow prevents
subsequent moves or mutable borrows of X until the borrow ends
Example
An extremely convoluted example (Gist) that does not show-case why the parameter passing is the way that it is. Has a short comment explaining the pain-point:
use std::collections::HashSet;
fn derp(v: i32, unprocessed: &mut HashSet<i32>) {
if unprocessed.contains(&v) {
// Pretend that v has been processed
unprocessed.remove(&v);
}
}
fn herp(v: i32) {
let mut unprocessed: HashSet<i32> = HashSet::new();
unprocessed.insert(v);
// I need to iterate over the unprocessed values
while let Some(u) = unprocessed.iter().next() {
// And them pass them mutably to another function
// as I will process the values inside derp and
// remove them from the set.
//
// This is an extremely convoluted example but
// I need for derp to be a separate function
// as I will employ recursion there, as it is
// much more succinct than an iterative version.
derp(*u, &mut unprocessed);
}
}
fn main() {
println!("Hello, world!");
herp(10);
}
The statement
while let Some(u) = unprocessed.iter().next() {
is an immutable borrow, hence
derp(*u, &mut unprocessed);
is impossible as unprocessed cannot be borrowed mutably. The immutable borrow does not end until the end of the while-loop.
I have tried to use this as reference and essentially ended up with trying to fool the borrow checker through various permutations of assignments, enclosing braces, but due to the coupling of the intended expressions the problem remains.
You have to think about what HashSet actually is. The IterMut that you get from HashMap::iter_mut() is only mutable on the value part: (&key, &mut val), ((&'a K, &'a mut V))
HashSet is basically a HashMap<T, ()>, so the actual values are the keys, and if you would modify the keys the hash of them would have to be updated or you get an invalid HashMap.
If your HashSet contains a Copy type, such as i32, you can work on a copy of the value to release the borrow on the HashSet early. To do this, you need to eliminate all borrows from the bindings in the while let expression. In your original code, u is of type &i32, and it keeps borrowing from unprocessed until the end of the loop. If we change the pattern to Some(&u), then u is of type i32, which doesn't borrow from anything, so we're free to use unprocessed as we like.
fn herp(v: i32) {
let mut unprocessed: HashSet<i32> = HashSet::new();
unprocessed.insert(v);
while let Some(&u) = unprocessed.iter().next() {
derp(u, &mut unprocessed);
}
}
If the type is not Copy or is too expensive to copy/clone, you can wrap them in Rc or Arc, and clone them as you iterate on them using cloned() (cloning an Rc or Arc doesn't clone the underlying value, it just clones the Rc pointer and increments the reference counter).
use std::collections::HashSet;
use std::rc::Rc;
fn derp(v: &i32, unprocessed: &mut HashSet<Rc<i32>>) {
if unprocessed.contains(v) {
unprocessed.remove(v);
}
}
fn herp(v: Rc<i32>) {
let mut unprocessed: HashSet<Rc<i32>> = HashSet::new();
unprocessed.insert(v);
while let Some(u) = unprocessed.iter().cloned().next() {
// If you don't use u afterwards,
// you could also pass if by value to derp.
derp(&u, &mut unprocessed);
}
}
fn main() {
println!("Hello, world!");
herp(Rc::new(10));
}
I was under the impression that mutable references (i.e. &mut T) are always moved. That makes perfect sense, since they allow exclusive mutable access.
In the following piece of code I assign a mutable reference to another mutable reference and the original is moved. As a result I cannot use the original any more:
let mut value = 900;
let r_original = &mut value;
let r_new = r_original;
*r_original; // error: use of moved value *r_original
If I have a function like this:
fn make_move(_: &mut i32) {
}
and modify my original example to look like this:
let mut value = 900;
let r_original = &mut value;
make_move(r_original);
*r_original; // no complain
I would expect that the mutable reference r_original is moved when I call the function make_move with it. However that does not happen. I am still able to use the reference after the call.
If I use a generic function make_move_gen:
fn make_move_gen<T>(_: T) {
}
and call it like this:
let mut value = 900;
let r_original = &mut value;
make_move_gen(r_original);
*r_original; // error: use of moved value *r_original
The reference is moved again and therefore the program behaves as I would expect.
Why is the reference not moved when calling the function make_move?
Code example
There might actually be a good reason for this.
&mut T isn't actually a type: all borrows are parametrized by some (potentially inexpressible) lifetime.
When one writes
fn move_try(val: &mut ()) {
{ let new = val; }
*val
}
fn main() {
move_try(&mut ());
}
the type inference engine infers typeof new == typeof val, so they share the original lifetime. This means the borrow from new does not end until the borrow from val does.
This means it's equivalent to
fn move_try<'a>(val: &'a mut ()) {
{ let new: &'a mut _ = val; }
*val
}
fn main() {
move_try(&mut ());
}
However, when you write
fn move_try(val: &mut ()) {
{ let new: &mut _ = val; }
*val
}
fn main() {
move_try(&mut ());
}
a cast happens - the same kind of thing that lets you cast away pointer mutability. This means that the lifetime is some (seemingly unspecifiable) 'b < 'a. This involves a cast, and thus a reborrow, and so the reborrow is able to fall out of scope.
An always-reborrow rule would probably be nicer, but explicit declaration isn't too problematic.
I asked something along those lines here.
It seems that in some (many?) cases, instead of a move, a re-borrow takes place. Memory safety is not violated, only the "moved" value is still around. I could not find any docs on that behavior either.
#Levans opened a github issue here, although I'm not entirely convinced this is just a doc issue: dependably moving out of a &mut reference seems central to Rust's approach of ownership.
It's implicit reborrow. It's a topic not well documented.
This question has already been answered pretty well:
how implicit reborrow works
how reborrow works along with borrow split
If I tweak the generic one a bit, it would not complain either
fn make_move_gen<T>(_: &mut T) {
}
or
let _ = *r_original;