Context
Link: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=9a9ffa99023735f4fbedec09e1c7ac55
Here's a contrived repro of what I'm running into
fn main() {
let mut s = String::from("Hello World");
example(&mut s);
}
fn example(s: &mut str) -> Option<String> {
other_func(Some(s.to_owned()))
// other random mutable stuff happens
}
fn other_func(s: Option<String>) {
match s {
Some(ref s) => other_func2(*s),
None => panic!()
}
}
fn other_func2(s: String) {
println!("{}", &s)
}
and the error
Compiling playground v0.0.1 (/playground)
error[E0507]: cannot move out of `*s` which is behind a shared reference
--> src/main.rs:12:36
|
12 | Some(ref s) => other_func2(*s),
| ^^ move occurs because `*s` has type `String`, which does not implement the `Copy` trait
Question
In the following code, why can't I deference the &String without having to do some sort of clone/copy? i.e. this doesn't work
fn other_func(s: Option<String>) {
match s {
Some(ref s) => other_func2(*s),
None => panic!()
}
}
but it works if I replace *s with s.to_owned()/s.to_string()/s.clone()
As an aside, I understand this can probably be solved by refactoring to use &str, but I'm specifically interested in turning &String -> String
Why would the compiler allow you to?
s is &String. And you cannot get a String from a &String without cloning. That's obvious.
And the fact that it was created from an owned String? The compiler doesn't care, and it is right. This is not different from the following code:
let s: String = ...;
let r: &String = ...;
let s2: String = *r; // Error
Which is in turn not different from the following code, for instance, as far as the compiler is concerned:
let r: &String = ...;
let s: String = *s;
And we no longer have an owned string at the beginning. In general, the compiler doesn't track data flow. And rightfully so - when it type-checks the move it doesn't even can confirm that this reference isn't aliased. Or that the owned value is not used anymore. References are just references, they give you no right to drop the value.
Changing that will not be feasible in the general case (for example, the compiler will have to track data flow across function calls), and will require some form of manual annotation to say "this value is mine". And you already have such annotation - use an owned value, String, instead of &String: this is exactly what it's about.
Related
I'm trying to write a function which pushes an element onto the end of a sorted vector only if the element is larger than the last element already in the vector, otherwise returns an error with a ref to the largest element. This doesn't seem to violate any borrowing rules as far as I cant tell, but the borrow checker doesn't like it. I don't understand why.
struct MyArray<K, V>(Vec<(K, V)>);
impl<K: Ord, V> MyArray<K, V> {
pub fn insert_largest(&mut self, k: K, v: V) -> Result<(), &K> {
{
match self.0.iter().next_back() {
None => (),
Some(&(ref lk, _)) => {
if lk > &k {
return Err(lk);
}
}
};
}
self.0.push((k, v));
Ok(())
}
}
error[E0502]: cannot borrow `self.0` as mutable because it is also borrowed as immutable
--> src/main.rs:15:9
|
6 | match self.0.iter().next_back() {
| ------ immutable borrow occurs here
...
15 | self.0.push((k, v));
| ^^^^^^ mutable borrow occurs here
16 | Ok(())
17 | }
| - immutable borrow ends here
Why doesn't this work?
In response to Paolo Falabella's answer.
We can translate any function with a return statement into one without a return statement as follows:
fn my_func() -> &MyType {
'inner: {
// Do some stuff
return &x;
}
// And some more stuff
}
Into
fn my_func() -> &MyType {
let res;
'outer: {
'inner: {
// Do some stuff
res = &x;
break 'outer;
}
// And some more stuff
}
res
}
From this, it becomes clear that the borrow outlives the scope of 'inner.
Is there any problem with instead using the following rewrite for the purpose of borrow-checking?
fn my_func() -> &MyType {
'outer: {
'inner: {
// Do some stuff
break 'outer;
}
// And some more stuff
}
panic!()
}
Considering that return statements preclude anything from happening afterwards which might otherwise violate the borrowing rules.
If we name lifetimes explicitly, the signature of insert_largest becomes fn insert_largest<'a>(&'a mut self, k: K, v: V) -> Result<(), &'a K>. So, when you create your return type &K, its lifetime will be the same as the &mut self.
And, in fact, you are taking and returning lk from inside self.
The compiler is seeing that the reference to lk escapes the scope of the match (as it is assigned to the return value of the function, so it must outlive the function itself) and it can't let the borrow end when the match is over.
I think you're saying that the compiler should be smarter and realize that the self.0.push can only ever be reached if lk was not returned. But it is not. And I'm not even sure how hard it would be to teach it that sort of analysis, as it's a bit more sophisticated than the way I understand the borrow checker reasons today.
Today, the compiler sees a reference and basically tries to answer one question ("how long does this live?"). When it sees that your return value is lk, it assigns lk the lifetime it expects for the return value from the fn's signature ('a with the explicit name we gave it above) and calls it a day.
So, in short:
should an early return end the mutable borrow on self? No. As said the borrow should extend outside of the function and follow its return value
is the borrow checker a bit too strict in the code that goes from the early return to the end of the function? Yes, I think so. The part after the early return and before the end of the function is only reachable if the function has NOT returned early, so I think you have a point that the borrow checked might be less strict with borrows in that specific area of code
do I think it's feasible/desirable to change the compiler to enable that pattern? I have no clue. The borrow checker is one of the most complex pieces of the Rust compiler and I'm not qualified to give you an answer on that. This seems related to (and might even be a subset of) the discussion on non-lexical borrow scopes, so I encourage you to look into it and possibly contribute if you're interested in this topic.
For the time being I'd suggest just returning a clone instead of a reference, if possible. I assume returning an Err is not the typical case, so performance should not be a particular worry, but I'm not sure how the K:Clone bound might work with the types you're using.
impl <K, V> MyArray<K, V> where K:Clone + Ord { // 1. now K is also Clone
pub fn insert_largest(&mut self, k: K, v: V) ->
Result<(), K> { // 2. returning K (not &K)
match self.0.iter().next_back() {
None => (),
Some(&(ref lk, _)) => {
if lk > &k {
return Err(lk.clone()); // 3. returning a clone
}
}
};
self.0.push((k, v));
Ok(())
}
}
Why does returning early not finish outstanding borrows?
Because the current implementation of the borrow checker is overly conservative.
Your code works as-is once non-lexical lifetimes are enabled, but only with the experimental "Polonius" implementation. Polonius is what enables conditional tracking of borrows.
I've also simplified your code a bit:
#![feature(nll)]
struct MyArray<K, V>(Vec<(K, V)>);
impl<K: Ord, V> MyArray<K, V> {
pub fn insert_largest(&mut self, k: K, v: V) -> Result<(), &K> {
if let Some((lk, _)) = self.0.iter().next_back() {
if lk > &k {
return Err(lk);
}
}
self.0.push((k, v));
Ok(())
}
}
I have
fn main() {
let x = String::from("12");
fun1(&x);
}
fn fun1(in_fun: &String) {
let mut y = _______;
y.push_str("z");
println!("in fun {}", y);
}
where _____ is the code for declaring y based on the argument in_fun.
At first I tried let mut y = *in_fun; which errors move occurs because '*in_fun' has type 'String', which does not implement the 'Copy' trait and also let mut y = String::from(*in_fun); which gives same error.
The thing that worked was let mut y = String::from(format!("{}", *in_fun));.
Is this the right way to declare a mutable String from &String?
Also I still don't understand why dereferencing &String with * errors? I understood *& dereferencing to returns just the value of the reference.
First of all, the working code:
fn fun1(in_fun: &String) {
let mut y = in_fun.clone();
y.push_str("z");
println!("in fun {}", y);
}
Or, your instincts tell you you have to dereference, so (*in_fun).clone() works just the same, but is a bit redundant. *in_fun.clone() does NOT work because it's equivalent to *(in_fun.clone()) (dereferencing the clone), which isn't what you want. The reason you don't need to dereference the reference before calling clone is because Rust's method resolution allows you to call methods of a type or access properties of a type using a reference to the type, and .clone has an &self receiver.
The reason that let mut y = *in_fun doesn't work is because this attempts to move the string out from underneath the reference, which doesn't work.
&String is an immutable reference. Rust is strict about this and prevents many common mishaps we people tend to run into. Dereferencing &String is not possible as it would break the guarantees of safety in rust, allowing you to modify where you only have read access. See the ownership explanation.
The function should either accept a mutable reference &mut String (then the string can be modified in place) or it needs to .clone() the string from the immutable reference.
Taking a mutable reference is more efficient than cloning, but it restricts the caller from sharing it immutably in parallel.
If the only thing you want to achieve is to print out some additional information, the best way I know of is:
fn fun1<S: std::fmt::Display>(in_fun: S) {
println!("in fun {}z", in_fun);
}
fn main() {
let mut x = String::from("12");
fun1(&x);
fun1(&mut x);
fun1(x);
fun1("12");
}
I use a Display trait so anything that implements will do. See the playground.
On the other hand, if you really need an owned string, then ask for it :)
fn fun1<S: Into<String>>(in_fun: S) {
let mut y = in_fun.into();
y.push('z');
println!("in fun {}", y);
}
fn main() {
let x = String::from("12");
fun1(&x);
fun1(x);
fun1("12");
}
This way you can accept both &str and String and keep efficient, avoiding cloning if possible.
Why is this exception being thrown, and how I can fix it? This is a piece of code that I am working on to get user input. I want to eventually use enums instead of if else statements, but I don't understand how to implement enums well enough yet.
use std::io;
fn main() {
let version = String::from("0.0.1");
let mut input = String::new();
shell(&mut input, &version);
}
fn shell(input: &mut String, version: &String) {
match io::stdin().read_line(&mut input) {
Ok(b) => {
if &input.trim() == &"ver" {
println!("{}", &version);
} else {
println!("Command '{}' Not Recognized", &input);
shell(&mut input, &version);
}
}
Err(err) => panic!("incorrect"),
}
}
The variable input itself is not mutable, it just contains a mutable reference, which is why you can't make a mutable reference to it.
However, since it is already a mutable reference, you can pass it directly to read_line, without referencing it again:
fn shell(input: &mut String, version: &String) {
match io::stdin().read_line(input) {
// already a &mut ref: ^^^^^
You could make the variable mutable, and re-borrow it:
fn shell(mut input: &mut String, version: &String) {
// ^^^
match io::stdin().read_line(&mut input) {
But this shouldn't be necessary. It only works because of Rust's auto-deref rules, which allows for things like &&&&&&T to be treated as &T in some situations. This is there for convenience because a lot of generic functions return references to their inputs and it would get messy having to dereference everything.
Here's a Thing:
struct Thing(i32);
impl Thing {
pub fn increment_self(&mut self) {
self.0 += 1;
println!("incremented: {}", self.0);
}
}
And here's a function that tries to mutate a Thing and returns either true or false, depending on if a Thing is available:
fn try_increment(handle: Option<&mut Thing>) -> bool {
if let Some(t) = handle {
t.increment_self();
true
} else {
println!("warning: increment failed");
false
}
}
Here's a sample of usage:
fn main() {
try_increment(None);
let mut thing = Thing(0);
try_increment(Some(&mut thing));
try_increment(Some(&mut thing));
try_increment(None);
}
As written, above, it works just fine (link to Rust playground). Output below:
warning: increment failed
incremented: 1
incremented: 2
warning: increment failed
The problem arises when I want to write a function that mutates the Thing twice. For example, the following does not work:
fn try_increment_twice(handle: Option<&mut Thing>) {
try_increment(handle);
try_increment(handle);
}
fn main() {
try_increment_twice(None);
let mut thing = Thing(0);
try_increment_twice(Some(&mut thing));
try_increment_twice(None);
}
The error makes perfect sense. The first call to try_increment(handle) gives ownership of handle away and so the second call is illegal. As is often the case, the Rust compiler yields a sensible error message:
|
24 | try_increment(handle);
| ------ value moved here
25 | try_increment(handle);
| ^^^^^^ value used here after move
|
In an attempt to solve this, I thought it would make sense to pass handle by reference. It should be an immutable reference, mind, because I don't want try_increment to be able to change handle itself (assigning None to it, for example) only to be able to call mutations on its value.
My problem is that I couldn't figure out how to do this.
Here is the closest working version that I could get:
struct Thing(i32);
impl Thing {
pub fn increment_self(&mut self) {
self.0 += 1;
println!("incremented: {}", self.0);
}
}
fn try_increment(handle: &mut Option<&mut Thing>) -> bool {
// PROBLEM: this line is allowed!
// (*handle) = None;
if let Some(ref mut t) = handle {
t.increment_self();
true
} else {
println!("warning: increment failed");
false
}
}
fn try_increment_twice(mut handle: Option<&mut Thing>) {
try_increment(&mut handle);
try_increment(&mut handle);
}
fn main() {
try_increment_twice(None);
let mut thing = Thing(0);
try_increment_twice(Some(&mut thing));
try_increment_twice(None);
}
This code runs, as expected, but the Option is now passed about by mutable reference and that is not what I want:
I'm allowed to mutate the Option by reassigning None to it, breaking all following mutations. (Uncomment line 12 ((*handle) = None;) for example.)
It's messy: There are a whole lot of extraneous &mut's lying about.
It's doubly messy: Heaven only knows why I must use ref mut in the if let statement while the convention is to use &mut everywhere else.
It defeats the purpose of having the complicated borrow-checking and mutability checking rules in the compiler.
Is there any way to actually achieve what I want: passing an immutable Option around, by reference, and actually being able to use its contents?
You can't extract a mutable reference from an immutable one, even a reference to its internals. That's kind of the point! Multiple aliases of immutable references are allowed so, if Rust allowed you to do that, you could have a situation where two pieces of code are able to mutate the same data at the same time.
Rust provides several escape hatches for interior mutability, for example the RefCell:
use std::cell::RefCell;
fn try_increment(handle: &Option<RefCell<Thing>>) -> bool {
if let Some(t) = handle {
t.borrow_mut().increment_self();
true
} else {
println!("warning: increment failed");
false
}
}
fn try_increment_twice(handle: Option<RefCell<Thing>>) {
try_increment(&handle);
try_increment(&handle);
}
fn main() {
let mut thing = RefCell::new(Thing(0));
try_increment_twice(Some(thing));
try_increment_twice(None);
}
TL;DR: The answer is No, I can't.
After the discussions with #Peter Hall and #Stargateur, I have come to understand why I need to use &mut Option<&mut Thing> everywhere. RefCell<> would also be a feasible work-around but it is no neater and does not really achieve the pattern I was originally seeking to implement.
The problem is this: if one were allowed to mutate the object for which one has only an immutable reference to an Option<&mut T> one could use this power to break the borrowing rules entirely. Concretely, you could, essentially, have many mutable references to the same object because you could have many such immutable references.
I knew there was only one mutable reference to the Thing (owned by the Option<>) but, as soon as I started taking references to the Option<>, the compiler no longer knew that there weren't many of those.
The best version of the pattern is as follows:
fn try_increment(handle: &mut Option<&mut Thing>) -> bool {
if let Some(ref mut t) = handle {
t.increment_self();
true
}
else {
println!("warning: increment failed");
false
}
}
fn try_increment_twice(mut handle: Option<&mut Thing>) {
try_increment(&mut handle);
try_increment(&mut handle);
}
fn main() {
try_increment_twice(None);
let mut thing = Thing(0);
try_increment_twice(Some(&mut thing));
try_increment_twice(None);
}
Notes:
The Option<> holds the only extant mutable reference to the Thing
try_increment_twice() takes ownership of the Option<>
try_increment() must take the Option<> as &mut so that the compiler knows that it has the only mutable reference to the Option<>, during the call
If the compiler knows that try_increment() has the only mutable reference to the Option<> which holds the unique mutable reference to the Thing, the compiler knows that the borrow rules have not been violated.
Another Experiment
The problem of the mutability of Option<> remains because one can call take() et al. on a mutable Option<>, breaking everything following.
To implement the pattern that I wanted, I need something that is like an Option<> but, even if it is mutable, it cannot be mutated. Something like this:
struct Handle<'a> {
value: Option<&'a mut Thing>,
}
impl<'a> Handle<'a> {
fn new(value: &'a mut Thing) -> Self {
Self {
value: Some(value),
}
}
fn empty() -> Self {
Self {
value: None,
}
}
fn try_mutate<T, F: Fn(&mut Thing) -> T>(&mut self, mutation: F) -> Option<T> {
if let Some(ref mut v) = self.value {
Some(mutation(v))
}
else {
None
}
}
}
Now, I thought, I can pass around &mut Handle's all day long and know that someone who has a Handle can only mutate its contents, not the handle itself. (See Playground)
Unfortunately, even this gains nothing because, if you have a mutable reference, you can always reassign it with the dereferencing operator:
fn try_increment(handle: &mut Handle) -> bool {
if let Some(_) = handle.try_mutate(|t| { t.increment_self() }) {
// This breaks future calls:
(*handle) = Handle::empty();
true
}
else {
println!("warning: increment failed");
false
}
}
Which is all fine and well.
Bottom line conclusion: just use &mut Option<&mut T>
I wish to extract all named groups from a match into a HashMap and I'm running into a "does not live long enough" error while trying to compile this code:
extern crate regex;
use std::collections::HashMap;
use regex::Regex;
pub struct Route {
regex: Regex,
}
pub struct Router<'a> {
pub namespace_seperator: &'a str,
routes: Vec<Route>,
}
impl<'a> Router<'a> {
// ...
pub fn path_to_params(&self, path: &'a str) -> Option<HashMap<&str, &str>> {
for route in &self.routes {
if route.regex.is_match(path) {
let mut hash = HashMap::new();
for cap in route.regex.captures_iter(path) {
for (name, value) in cap.iter_named() {
hash.insert(name, value.unwrap());
}
}
return Some(hash);
}
}
None
}
}
fn main() {}
Here's the error output:
error: `cap` does not live long enough
--> src/main.rs:23:42
|>
23 |> for (name, value) in cap.iter_named() {
|> ^^^
note: reference must be valid for the anonymous lifetime #1 defined on the block at 18:79...
--> src/main.rs:18:80
|>
18 |> pub fn path_to_params(&self, path: &'a str) -> Option<HashMap<&str, &str>> {
|> ^
note: ...but borrowed value is only valid for the for at 22:16
--> src/main.rs:22:17
|>
22 |> for cap in route.regex.captures_iter(path) {
|> ^
Obviously I still have a thing or two to learn about Rust lifetimes.
Let's follow the lifetime lines:
route.regex.captures_iter(path) creates a FindCapture<'r, 't> where the lifetime 'r is that of route.regex and the lifetime 't is that of path
this iterator yields a Captures<'t>, only linked to the lifetime of path
whose method iter_named(&'t self) yields a SubCapture<'t> itself linked to the lifetime of path and the lifetime of the cap
this iterator yields a (&'t str, Option<&'t str>) so that both keys and values of the HashMap are linked to the lifetime of path and the lifetime of the cap
Therefore, it is unfortunately impossible to have the HashMap outlive the cap variable as this variable is used by the code as a "marker" to keep the buffers containing the groups alive.
I am afraid that the only solution without significant re-structuring is to return a HashMap<String, String>, as unsatisfying as it is. It also occurs to me that a single capture group may match multiple times, not sure if you want to bother with this.
Matthieu M. already explained the lifetime situation well. The good news is that the regex crate recognized the problem and there's a fix in the pipeline for 1.0.
As stated in the commit message:
It was always possible to work around this by using indices.
It is also possible to work around this by using Regex::capture_names, although it's a bit more nested this way:
pub fn path_to_params(&self, path: &'a str) -> Option<HashMap<&str, &str>> {
for route in &self.routes {
if let Some(captures) = route.regex.captures(path) {
let mut hash = HashMap::new();
for name in route.regex.capture_names() {
if let Some(name) = name {
if let Some(value) = captures.name(name) {
hash.insert(name, value);
}
}
}
return Some(hash);
}
}
None
}
Note that I also removed the outer is_match — it's inefficient to run the regex once and then again.