This code
fn broken(xs: &Vec<f32>) -> Vec<f32> {
xs.iter()
.flat_map(|x| xs.iter().map(|y| x - y))
.collect()
}
gives me the error
error[E0373]: closure may outlive the current function, but it borrows `x`, which is owned by the current function
I understand why this is unsafe. However, I do not understand why writing *x fixes the issue. Does this force a copy?
fn working(xs: &Vec<f32>) -> Vec<f32> {
xs.iter()
.flat_map(|x| xs.iter().map(|y| *x - y))
.collect()
}
Related
I just found out that the following code compiles in Rust 21 (used to not compile in 18)
fn get_func (i: &mut i32) -> impl Fn() -> i32 + '_ {
|| *i
}
Is there an implicit move of i involved ? If so, then why does the following code compile too?
fn get_func (i: &mut i32) -> impl Fn() -> i32 + '_ {
let f = || *i;
println!("{:?}", i); // was expecting here to give borrow of moved variable error, `&mut` doesnt implement `Copy` trait
f
}
Or instead is it implicitly moving (copying in this case) the value being pointed to? But then the following code should compile, which doesn't -- indicating it's moving the reference.
fn get_func (i: &mut i32) -> impl Fn() -> i32 {
|| *i
}
It's downgrading your &mut to a & and making as many copies as it needs to.
This is all a byproduct of the fact that &mut has all the permissions of &; in other words it's fine to downgrade from a mutable reference to a non-mutable reference, and it's fine to copy a non-mutable reference as many times as you want, so this is just doing both of those implicitly.
fn get_func (i: &mut i32) -> impl Fn() -> i32 + '_ {
*i += 1; // fine, still mutable
let f = || *i;
println!("{}", *i); // fine, no mutability required
*i += 1; // ERROR: cannot mutate *i while borrowed by f
f
}
It's worth noting that if you actually do try to capture it as mutable, this happens.
fn get_func (i: &mut i32) -> impl FnMut() -> i32 + '_ {
println!("{}", *i); // Fine
*i += 1; // Fine
let f = || {*i += 1; *i};
println!("{}", *i); // ERROR: cannot borrow *i as immutable
f
}
Also the lifetime doesn't refer to the return of the closure, but the closure itself. Because no memory is moved, the closure is only valid so long as the memory at i's address remains valid. That's why removing the lifetime makes the code not compile.
In any case, all you really have to remember is the classic: "one mutable reference OR multiple immutable references, not both".
This code compiles:
fn main() {
let mut s = String::from("some_string");
let n = f1(&s);
s.clear();
println!("n = {}", n);
}
fn f1(s: &String) -> usize {
10
}
fn f2(s: &String) -> &str {
"def"
}
However, replacing the call to f1() by f2() causes a compilation failure. It appears to me that both f1() and f2() do an immutable borrow, and s.clear() does a mutable borrow, so I should get compilation error in both cases. What am I missing?
Here is the minimum code necessary to reproduce the issue:
fn f1(s: &String) -> usize { unimplemented!() }
fn f2(s: &String) -> &str { unimplemented!() }
fn main() {
let mut s = String::from("some_string");
let n = f1(&s);
s.clear();
println!("n = {}", n);
}
Lifetime analysis is performed based on function signatures.
You will note in the above code that I have used unimplemented!() as the body of the functions, and the problem is exactly the same. This is normal.
In the vast majority of cases1, a function signature fully specifies the interface of a function, and it is unnecessary to look at its implementation.
As a corollary, this also means that whether the lifetime in a return type is linked to the lifetime in any of the arguments is fully specified within the signature, and in this case the full signature of f2 is therefore:
fn f2<'a>(s: &'a String) -> &'a str;
Whether the implementation of f2 is "def" (with the 'static lifetime) or &*s (with the 'a lifetime) does not matter; only the signature matters, and the signature uses the same lifetime due to the elision rules.
1 The one exception I know of concerns the -> impl Trait feature and whether the resulting object implements Send or Sync.
In the case of f1, the return type is not linked to the argument, therefore the borrow of the argument ends at the end of the call to f1:
fn main() {
let mut s = String::from("some_string");
let n = {
// Immutable borrow of s starts here.
f1(&s)
// Immutable borrow of s ends here.
};
s.clear();
println!("n = {}", n);
}
In the case of f2, the return type has the same lifetime as the argument, and therefore is considered to extend the borrow. In Rust 2015, the borrow would extend until the return value went out of scope (lexical borrow); with Rust 2018, the borrow extends until the last use of the return value (non-lexical borrow).
In your case, both are basically identical:
fn main() {
let mut s = String::from("some_string");
let n = {
// Immutable borrow of s starts here.
f2(&s)
};
s.clear(); // Conflicting attempt to mutably borrow s.
println!("n = {}", n);
// Immutable borrow of s ends here.
}
You could observe the difference by switching the order of s.clear() and println!.
The Rust reference says:
If there is exactly one lifetime used in the parameters (elided or not), that lifetime is assigned to all elided output lifetimes.
This means that your method
fn f2(s: &String) -> &str {
"def"
}
is interpreted by Rust as:
fn f2<'a>(s: &'a String) -> &'a str {
"def"
}
Since "def" has the lifetime 'static, its lifetime can be shortened to 'a when being returned from the function (so the compiler won't complain here), but when calling the function the compiler cannot infer that the true lifetime of the string was really 'static. To do this, you must explicitly mark it as 'static yourself:
fn f2(s: &String) -> &'static str {
"def"
}
f1 and f2 both take an immutable borrow. However, the lifetime of the borrow from f1 ends at the end of f1, because you're just returning a usize and not anything from the actual string.
However, f2 returns a &str, which is borrowing your underlying String, s. Since n stays alive, the immutable borrow of s continues until n is no longer used. Effectively, this prevents your call to s.clear() from "pulling the rug out from under" your pointer s.
I've got a persistent compile error where Rust complains that I have an immutable borrow while I'm trying to mutably borrow, but the immutable borrow is from another scope, and I'm not bringing anything across from it.
I have some code that checks for a value in a map, and if it's present, returns it, otherwise it needs to mutate the map in various ways. The problem is that I can't seem to find a way to get Rust let me do both, even though the two operations are totally separate.
Here's some nonsensical code that follows the same structure as my code and exhibits the problem:
use std::collections::BTreeMap;
fn do_stuff(map: &mut BTreeMap<i32, i32>, key: i32) -> Option<&i32> {
// extra scope in vain attempt to contain the borrow
{
// borrow immutably
if let Some(key) = map.get(&key) {
return Some(key);
}
}
// now I'm DONE with the immutable borrow, but rustc still thinks it's borrowed
map.insert(0, 0); // borrow mutably, which errors
None
}
This errors out with:
error[E0502]: cannot borrow `*map` as mutable because it is also borrowed as immutable
--> src/lib.rs:14:5
|
3 | fn do_stuff(map: &mut BTreeMap<i32, i32>, key: i32) -> Option<&i32> {
| - let's call the lifetime of this reference `'1`
...
7 | if let Some(key) = map.get(&key) {
| --- immutable borrow occurs here
8 | return Some(key);
| --------- returning this value requires that `*map` is borrowed for `'1`
...
14 | map.insert(0, 0); // borrow mutably, which errors
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here
This doesn't make any sense to me. How does the immutable borrow outlive that scope?! One branch of that match exits the function via return, and the other does nothing and leaves the scope.
I've seen this happen before where I was mistakenly smuggling the borrow out of the scope in some other variable, but that's not the case here!
True, the borrow is escaping the scope via the return statement, but it's ridiculous that that blocks borrowing farther down in the function -- the program cannot possibly return AND keep going! If I return something else there, the error goes away, so I think this is what the borrow checker is getting hung up on. This feels like a bug to me.
Unfortunately, I've been unable to find any way to rewrite this without hitting the same error, so it's a particularly nasty bug if that's the case.
This is a known issue that will be solved by a future iteration of non-lexical lifetimes, but is not currently handled as of Rust 1.57.
If you are inserting to the same key that you are looking up, I'd encourage you to use the entry API instead.
You can add a smidgen of inefficiency to work around this for now. If the inefficiency is unacceptable, there are deeper workarounds.
HashMap
The general idea is to add a boolean that tells you if a value was present or not. This boolean does not hang on to a reference, so there is no borrow:
use std::collections::BTreeMap;
fn do_stuff(map: &mut BTreeMap<i32, i32>, key: i32) -> Option<&i32> {
if map.contains_key(&key) {
return map.get(&key);
}
map.insert(0, 0);
None
}
fn main() {
let mut map = BTreeMap::new();
do_stuff(&mut map, 42);
println!("{:?}", map)
}
Vec
Similar cases can be solved by using the index of the element instead of the reference. Like the case above, this can introduce a bit of inefficiency due to the need to check the slice bounds again.
Instead of
fn find_or_create_five<'a>(container: &'a mut Vec<u8>) -> &'a mut u8 {
match container.iter_mut().find(|e| **e == 5) {
Some(element) => element,
None => {
container.push(5);
container.last_mut().unwrap()
}
}
}
You can write:
fn find_or_create_five<'a>(container: &'a mut Vec<u8>) -> &'a mut u8 {
let idx = container.iter().position(|&e| e == 5).unwrap_or_else(|| {
container.push(5);
container.len() - 1
});
&mut container[idx]
}
Non-Lexical Lifetimes
These types of examples are one of the primary cases in the NLL RFC: Problem case #3: conditional control flow across functions.
Unfortunately, this specific case isn't ready as of Rust 1.57. If you opt in to the experimental -Zpolonius feature in nightly (RUSTFLAGS="-Z polonius" cargo +nightly check), each of these original examples compile as-is:
use std::collections::BTreeMap;
fn do_stuff(map: &mut BTreeMap<i32, i32>, key: i32) -> Option<&i32> {
if let Some(key) = map.get(&key) {
return Some(key);
}
map.insert(0, 0);
None
}
fn find_or_create_five(container: &mut Vec<u8>) -> &mut u8 {
match container.iter_mut().find(|e| **e == 5) {
Some(element) => element,
None => {
container.push(5);
container.last_mut().unwrap()
}
}
}
See also:
How to update-or-insert on a Vec?
This is the same problem without returning the reference, which does work with the implementation of NLL available in Rust 1.32.
Double mutable borrow error in a loop happens even with NLL on
This problem but in a slightly more complicated case.
When is it necessary to circumvent Rust's borrow checker?
The ultimate escape hatch.
I am trying to write a small wrapper around VecDeque.
Specifically I have the code (playground):
use std::collections::VecDeque;
trait VecCircleTraits<T: Eq> {
fn new() -> VecCircle<T>;
fn find_and_remove(&self, _: T) -> Option<T>;
}
#[derive(Debug)]
struct VecCircle<T: Eq>(VecDeque<T>);
impl<T: Eq> VecCircleTraits<T> for VecCircle<T> {
fn new() -> VecCircle<T> {
return VecCircle(VecDeque::<T>::new());
}
fn find_and_remove(&self, key: T) -> Option<T> {
let search_index: Option<usize> = self.0.into_iter().position(|x| x == key); //error 1
if let Some(index) = search_index {
return self.0.remove(index); // error 2
} else {
return None;
}
}
}
Which gives me the following errors:
error: cannot borrow immutable anonymous field `self.0` as mutable
--> <anon>:20:20
|>
20 |> return self.0.remove(index); // error 2
|> ^^^^^^
error: cannot move out of borrowed content [--explain E0507]
--> <anon>:18:44
|>
18 |> let search_index: Option<usize> = self.0.into_iter().position(|x| x == key); //error 1
|> ^^^^ cannot move out of borrowed content
However, I am little confused as who has ownership over self.0? If I am understanding the docs correctly, wouldn't the memory region be bounded to self.0 and therefore giving it the ownership? Sorry for the shallow logic there but I am still trying to understand the ownership system.
In find_and_remove, you specified &self in the parameter list. This means that the method will receive a borrowed pointer to self; i.e. the type of self is &VecCircle<T>. Therefore, the method doesn't have ownership of the VecCircle<T>.
find_and_remove tries to call into_iter on a VecDeque, and into_iter receives its argument by value (self rather than &self or &mut self). Because of this, Rust interprets self.0 as trying to move the VecDeque out of the VecCircle. However, that's not allowed as you can't move anything out of borrowed content, as moving from some location makes that location invalid. But we can't just tell the caller "Hey, I just invalidated self, stop using it!"; if we wanted to do that, we'd have to specify self in the parameter list, rather than &self.
But that's not what you're trying to do here. into_iter would take ownership of the VecDeque and therefore destroy it. There are other ways to obtain an iterator for the VecDeque without destroying it. Here, we should use iter, which takes &self.
Then, find_and_remove tries to call remove. remove takes &mut self, i.e. a mutable reference to a VecDeque. However, we can't borrow self.0 as mutable, because self is not itself a mutable borrow. We can't just upgrade an immutable borrow to a mutable borrow: it is invalid to have both an immutable borrow and a mutable borrow usable at the same time. The solution here is to change &self to &mut self in the parameter list.
use std::collections::VecDeque;
trait VecCircleTraits<T: Eq> {
fn new() -> VecCircle<T>;
fn find_and_remove(&mut self, _: &T) -> Option<T>;
}
#[derive(Debug)]
struct VecCircle<T: Eq>(VecDeque<T>);
impl<T: Eq> VecCircleTraits<T> for VecCircle<T> {
fn new() -> VecCircle<T> {
return VecCircle(VecDeque::<T>::new());
}
fn find_and_remove(&mut self, key: &T) -> Option<T> {
let search_index: Option<usize> = self.0.iter().position(|x| x == key);
if let Some(index) = search_index {
self.0.remove(index)
} else {
None
}
}
}
Note: I also changed the key parameter to &T to solve another error, this time in the closure passed to position. Since iter iterates over references to the items in the VecDeque, position passes references to the closure. Since find_and_remove doesn't actually need to take ownership of the key, it should just receive an immutable borrow to it, so that both x and key are of type &T and thus we can apply == to them.
No, you don't have ownership of the VecCircle inside the find_and_remove method. All you need to know is in the function definition:
impl<T: Eq> VecCircleTraits<T> for VecCircle<T> {
fn find_and_remove(&self, key: T) -> Option<T>
}
This means that you are borrowing a reference to VecCircle. The longer way to write this would be
fn find_and_remove(self: &VecCircle, key: T) -> Option<T>
Perhaps that is more evident?
Since you don't have ownership of self, you cannot have ownership of self.0.
Vec<T> has two methods:
fn push(&mut self, value: T)
fn split_at_mut(&mut self, mid: usize) -> (&mut [T], &mut [T])
They both take a mutable reference to the vector. But the scope of the borrow seems to be different, e.g:
fn works() {
let mut nums: Vec<i64> = vec![1,2,3,4];
nums.push(5);
println!("{}", nums.len());
}
fn doesnt_work() {
let mut nums: Vec<i64> = vec![1,2,3,4];
let (l,r) = nums.split_at_mut(2);
println!("{}", nums.len());
}
fn also_works() {
let mut nums: Vec<i64> = vec![1,2,3,4];
let _ = nums.split_at_mut(2);
println!("{}", nums.len());
}
The doesnt_work function doesn't compile, saying there is already a mutable borrow on nums and that it ends and the end of the function. The problem goes away if I ignore the values returned from split_at_mut.
The borrowing of nums in doesnt_work will last as long as the variables l and r exist because the values in the vector (and the vector itself) have literally been borrowed and are now accessible only through l and r.
You can see this effect by putting the let for l and r in a scope which ends so the borrow also ends. For example this code works fine but if you try to move the println! inside the scope (inside the curly brackets) then it will fail:
fn works() {
let mut nums = vec![1,2,3,4];
{
let (l, r) = nums.split_at_mut(2);
//println!("{}", nums.len()); //println! will fail here
}
println!("{}", nums.len());
}
In your also_works example you don't do anything with the result so the borrow is lost immediately. Basically the compiler can see that there is no way you can access the vector through the result of the method so you're free to access them through the original vector.
Let me answer my own question, since what I was really missing were lifetimes. This code compiles:
fn maybe_use<'a, 'b>(v1: &'a mut Vec<i64>, v2: &'b mut Vec<i64>) -> &'a mut Vec<i64> {
v1
}
fn main() {
let mut nums1: Vec<i64> = vec![1,2,3,4];
let mut nums2: Vec<i64> = vec![1,2,3,4];
let ret = maybe_use(&mut nums1, &mut nums2);
println!("{}", nums2.len());
}
Because the return type of maybe_use makes it clear the reference comes from the first argument. If we change v2 to use 'a lifetime, main stops compiling because both vectors passed to maybe_use are considered borrowed. If we omit the lifetime altogether, compiler emits this error:
this function's return type contains a borrowed value, but the
signature does not say whether it is borrowed from v1 or v2
So what surprised me initially (how does the compiler know split_at_mut returns pointers to the vector?) boils down to the references having the same lifetime.