Borrow checker error after adding generic parameter to struct - rust

I have code that works, but it stops compiling with a borrow checker error after a change. I don't understand how the change could affect borrow checking.
Common part to both working and non-working code:
/// Some struct that has references inside
#[derive(Debug)]
struct MyValue<'a> {
number: &'a u32,
}
/// There are many structs similar to `MyValue` and there is a
/// trait common to them all that can create them. In this
/// example I use the `From` trait.
impl<'a> From<&'a u32> for MyValue<'a> {
fn from(value: &'a u32) -> Self {
MyValue { number: value }
}
}
/// `Producer` makes objects that hold references into it. So
/// the produced object must be first dropped before any new
/// one can be made.
trait Producer<'a, T: 'a> {
fn make(&'a mut self) -> T;
}
Here is the working code:
struct MyProducer {
number: u32,
}
impl MyProducer {
fn new() -> Self {
Self { number: 0 }
}
}
impl<'a, T: 'a + From<&'a u32>> Producer<'a, T> for MyProducer {
fn make(&'a mut self) -> T {
self.number += 1;
T::from(&self.number)
}
}
fn main() {
let mut producer = MyProducer::new();
println!(
"made this: {:?}",
<MyProducer as Producer<MyValue>>::make(&mut producer)
);
println!(
"made this: {:?}",
<MyProducer as Producer<MyValue>>::make(&mut producer)
);
}
This compiles and prints the expected output:
made this: MyValue { number: 1 }
made this: MyValue { number: 2 }
I don't like that MyProducer actually implements Producer for every T as it makes it impossible to call make directly on it. I would like to have a type that is a MyProducer for a specific T (for example for MyValue).
To achieve this, I want to add a generic parameter to MyProducer. Because the MyProducer does not really use the T, I use PhantomData to prevent the compiler from complaining.
Here is the code after changes:
use std::marker::PhantomData;
struct MyProducer<'a, T: 'a + From<&'a u32>> {
number: u32,
_phantom: PhantomData<&'a T>,
}
impl<'a, T: 'a + From<&'a u32>> MyProducer<'a, T> {
fn new() -> Self {
Self {
number: 0,
_phantom: PhantomData::default(),
}
}
}
impl<'a, T: From<&'a u32>> Producer<'a, T> for MyProducer<'a, T> {
fn make(&'a mut self) -> T {
self.number += 1;
T::from(&self.number)
}
}
fn main() {
let mut producer = MyProducer::<MyValue>::new();
println!("made this: {:?}", producer.make());
println!("made this: {:?}", producer.make());
}
The main function now looks exactly as I would like it to look like. But the code does not compile. This is the error:
error[E0499]: cannot borrow `producer` as mutable more than once at a time
--> src/main.rs:50:33
|
49 | println!("made this: {:?}", producer.make());
| -------- first mutable borrow occurs here
50 | println!("made this: {:?}", producer.make());
| ^^^^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
I don't understand why it no longer works. The produced object is still dropped before the next one is made.
If I call the make function just once, it compiles and works.
I am using edition 2018, so NLL is active.
Rust Playground: working version before change
Rust Playground: broken version after change

I reduced the noise from the code so the following is an even shorter version of the broken case which demonstrates the same problem: (test in the playground)
use std::marker::PhantomData;
#[derive(Debug)]
struct MyValue<'a>(&'a u32);
impl<'a> From<&'a u32> for MyValue<'a> {
fn from(value: &'a u32) -> Self {
MyValue(value)
}
}
struct MyProducer<'a, T>(u32, PhantomData<&'a T>);
impl<'a, T> MyProducer<'a, T>
where
T: From<&'a u32>,
{
fn new() -> Self {
Self(0, PhantomData)
}
fn make(&'a mut self) -> T {
self.0 += 1;
T::from(&self.0)
}
}
fn main() {
let mut producer = MyProducer::<MyValue>::new();
println!("made this: {:?}", producer.make());
println!("made this: {:?}", producer.make());
}
The main problem here is that the mutable borrow's lifetime is the lifetime of MyProducer, that is, the lifetime of the instance called producer is the same as the mutable borrow taken in its make method. Because the producer instance does not go out of scope (if it would then MyValue wouldn't be able to hold a reference to a value stored in it) so the mutable borrow lives until the end of main's scope. The first rule of borrowing is that there can only be a single mutable borrow of a given value in a scope at any time, hence the compiler error.
If you are looking at my solution here, which is actually working and does what I think you wanted it to: (test in the playground):
#[derive(Debug)]
struct MyValue<'a>(&'a u32);
impl<'a> From<&'a u32> for MyValue<'a> {
fn from(value: &'a u32) -> Self {
MyValue(value)
}
}
struct MyProducer(u32);
impl MyProducer {
fn new() -> Self {
Self(0)
}
fn make<'a, T>(&'a mut self) -> T
where
T: From<&'a u32>,
{
self.0 += 1;
T::from(&self.0)
}
}
fn main() {
let mut producer = MyProducer::new();
println!("made this: {:?}", producer.make::<MyValue>());
println!("made this: {:?}", producer.make::<MyValue>());
}
then you can see that the mutable borrow only lives as long as the make method, therefore after the invocation there's no more living mutable borrow to producer in main's scope, thus you can have another one.

Related

Lifetime Confusion for iterator

I am new to Rust and currently have been following Learning Rust With Entirely Too Many Linked Lists examples. Before section IterMut everything makes sense to me. Yet when implementing IterMut(in a differrnt way from the tutorial), I got totally confused by lifetime mechanism in Rust. Here is the case, first I'd just define the stack to be implemented:
pub struct List<T> {
head: Link<T>,
}
type Link<T> = Option<Box<Node<T>>>;
struct Node<T> {
elem: T,
next: Link<T>,
}
Ok then when I try to implement iterator in the following way:
pub struct IterMut<'a, T>{
this: &'a mut Link<T>
}
impl<T> List<T> {
pub fn iter_mut(&mut self) -> IterMut<T> {
IterMut {
this: &mut self.head
}
}
}
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
if let Some(node) = self.this {
Some(&mut node.elem)
} else {
None
}
}
}
It won't compile, the result says:
error: lifetime may not live long enough
impl<'a, T> Iterator for IterMut<'a, T>{
-- lifetime `'a` defined here
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
- let's call the lifetime of this reference `'1`
if let Some(node) = self.this {
Some(&mut node.elem)
^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'a`
Originally, I implement function next in this way, using as_mut and map:
// function body in the fn next
self.this.as_mut().map(|node|{
self.this = &mut node.next;
&mut node.elem
})
This also triggers compilation error (incompatible lifetime).
Therefore I wonder what is going on here, shouldn't self in fn next have the same lifetime as IterMut ('a)? And is there any workaround for this situation?
The principal problem lies in trying to take a reference to the whole head. While that reference lives, you can't hand out mutable references to anything inside it. You only need access to the Node that's inside head, though. So first, we refactor IterMut to only keep a reference into any given Node:
pub struct IterMut<'a, T>{
this: Option<&'a mut Node<T>>
}
Now, to get that out of the head we use the convenience method as_deref_mut() that Option provides. It just gives us a mutable reference to whatever was inside it (if anything):
impl<T> List<T> {
pub fn iter_mut(&mut self) -> IterMut<'_, T> {
IterMut {
this: self.head.as_deref_mut(),
}
}
}
Now, 'a is only tied to that Node and we can do what we want with it, like takeing it:
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
if let Some(node) = self.this.take() {
self.this = node.next.as_deref_mut();
Some(&mut node.elem)
} else {
None
}
}
}
And we can simplify that with a simple map call:
impl<'a, T> Iterator for IterMut<'a, T>{
type Item = &'a mut T;
fn next(&mut self) -> Option<Self::Item> {
self.this.take().map(|node| {
self.this = node.next.as_deref_mut();
&mut node.elem
})
}
}

A struct with a trait field, but optional

Say I have a struct whose implementation writes somewhere, i.e. to something that implements the std::io::Write trait. However, I don't want the struct to own this. The following code works:
fn main() {
let mut out = std::io::stdout();
let mut foo = Foo::new(&mut out);
foo.print_number(2);
}
struct Foo<'a> {
out: &'a mut dyn std::io::Write
}
impl<'a> Foo<'a> {
pub fn new(out: &'a mut dyn std::io::Write) -> Self {
Self {
out
}
}
pub fn print_number(&mut self, i: isize) {
writeln!(self.out, "The number is {}", i).unwrap()
}
}
But, now this writing functionality should be made optional. I thought this sounds easy enough, but now the following doesn't compile:
fn main() {
let mut out = std::io::stdout();
let mut foo = Foo::new(Some(&mut out));
foo.print_number(2);
}
struct Foo<'a> {
out: Option<&'a mut dyn std::io::Write>
}
impl<'a> Foo<'a> {
pub fn new(out: Option<&'a mut dyn std::io::Write>) -> Self {
Self {
out
}
}
pub fn print_number(&mut self, i: isize) {
if self.out.is_some() {
writeln!(self.out.unwrap(), "The number is {}", i).unwrap()
}
}
}
because of:
error[E0507]: cannot move out of `self.out` which is behind a mutable reference
--> src/main.rs:20:26
|
20 | writeln!(self.out.unwrap(), "The number is {}", i).unwrap()
| ^^^^^^^^
| |
| move occurs because `self.out` has type `Option<&mut dyn std::io::Write>`, which does not implement the `Copy` trait
| help: consider borrowing the `Option`'s content: `self.out.as_ref()`
which I'm not sure how to interpret.
I tried following the suggestion by changing the line in question to:
writeln!(self.out.as_ref().unwrap(), "The number is {}", i).unwrap()
but then I get
error[E0596]: cannot borrow data in a `&` reference as mutable
--> src/main.rs:20:26
|
20 | writeln!(self.out.as_ref().unwrap(), "The number is {}", i).unwrap()
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
I'm really not sure how to interpret these error messages and surprisingly I'm not really getting anywhere by just sprinkling &s and muts in random places without really understanding!
(As an aside, I'm not sure if this is a "good" way of going about this anyway? I'm open to completely different approaches of solving this problem, which is basically to optionally pass something to write to into a struct, but without the struct owning it. I read about the Box type which might also be relevant?)
As you already know, based on you already using &mut for out. The issue with using as_ref() is that it returns an immutable reference. Instead you need to use as_mut().
pub fn print_number(&mut self, i: isize) {
if self.out.is_some() {
writeln!(self.out.as_mut().unwrap(), "The number is {}", i).unwrap()
}
}
Alternatively, you can also simplify this and express it more idiomatically using if let:
pub fn print_number(&mut self, i: isize) {
if let Some(out) = &mut self.out {
writeln!(out, "The number is {}", i).unwrap()
}
}
I would also suggest that instead of unwrapping, that you return the io::Result and let the caller handle any potential error.
pub fn print_number(&mut self, i: isize) -> std::io::Result<()> {
if let Some(out) = &mut self.out {
writeln!(out, "The number is {}", i)?;
}
Ok(())
}
You can also simplify your paths, e.g. std::io::Write and std::io::Result<()>, by importing them with a use declaration, e.g. use std::io::{self, Write}; and then changing them to Write and io::Result<()>.

How do I implement move semantics on a struct with a mutable field?

I have a minimal example of code that implements move semantics for some container:
use std::mem;
impl<'a, T: 'a + ?Sized> Drop for Cointainer<'a, T> {
fn drop(&mut self) {}
}
struct Cointainer<'a, T: 'a + ?Sized> {
item: &'a /* mut */ T,
}
impl<'a, T> Cointainer<'a, T> {
fn mv(self) -> Cointainer<'a, T> {
let new = Cointainer { item: /* &mut */ self.item };
mem::forget(self);
new
}
}
fn main() {}
That compiles and works without any problems.
I realized I will need to mutate the value referenced by Cointainer::item, so I've made the reference mut. When I do so, I get:
error[E0505]: cannot move out of `self` because it is borrowed
--> src/main.rs:14:21
|
13 | let new = Cointainer { item: /* &mut */ self.item };
| --------- borrow of `*self.item` occurs here
14 | mem::forget(self);
| ^^^^ move out of `self` occurs here
I need to create a new container and transfer ownership of item there, and drop the old one.
This example is artificial. The actual "move" operation does some other stuff, and doesn't necessarily return the same container type.
The rules of references state:
At any given time, you can have either but not both of:
One mutable reference.
Any number of immutable references.
Your code with immutable references works because those can be freely copied. Your code with mutable references fails because, as far as the compiler can tell, you would need to have have two concurrent mutable references: the saved reference in new and then mem::forget might also need it.
As humans, we recognize that mem::forget will not access the guts of our structure. This is what unsafe code is for: when the compiler cannot guarantee the code we have is truly safe.
A small unsafe block and some casting to a raw pointer and back solves the problem. As with any unsafe code, it should have a big comment block explaining why the compiler doesn't understand it and why it's truly safe.
impl<'a, T: 'a + ?Sized> Drop for Cointainer<'a, T> {
fn drop(&mut self) {
println!("dropping 1: {:p}", self.item)
}
}
struct Cointainer<'a, T: 'a + ?Sized> {
item: &'a mut T,
}
impl<'a, T> Cointainer<'a, T> {
fn into_inner(self) -> &'a mut T {
// I copied this code from Stack Overflow but didn't read
// the warning about explaining why it's safe. Sure hope
// this doesn't cause any bugs!
unsafe {
let x = self.item as *mut _;
std::mem::forget(self);
&mut *x
}
}
fn mv(self) -> Cointainer2<'a, T> {
let item = self.into_inner();
Cointainer2 { item }
}
}
struct Cointainer2<'a, T: 'a + ?Sized> {
item: &'a mut T,
}
impl<'a, T: 'a + ?Sized> Drop for Cointainer2<'a, T> {
fn drop(&mut self) {
println!("dropping 2: {:p}", self.item)
}
}
fn main() {
let mut a = String::new();
let c1 = Cointainer { item: &mut a };
let c2 = c1.mv();
}

How to have a struct field with the same mutability as the parent struct?

I'm trying to wrap a slice in a struct so that I will be able to instantiate the struct mutably or immutably. Here's a minimal example:
use std::ops::{ Index, IndexMut };
struct Test<'a, T: 'a> {
inner: &'a[T]
}
impl<'a, T: 'a> Test<'a, T> {
fn new (inner: &'a[T]) -> Self { Test { inner: inner } }
}
impl<'a, T> Index<usize> for Test<'a, T> {
type Output = T;
fn index (&self, i: usize) -> &T { &self.inner[i] }
}
impl<'a, T> IndexMut<usize> for Test<'a, T> {
fn index_mut (&mut self, i: usize) -> &mut T { &mut self.inner[i] }
}
fn main() {
let store = [0; 3];
let test = Test::new (&store);
println!("{}", test[1]);
let mut mut_store = [0; 3];
let mut mut_test = Test::new (&mut mut_store);
mut_test[1] = 42;
println!("{}", mut_test[1]);
}
This doesn't compile: "cannot borrow immutable indexed content self.inner[..] as mutable".
I could get it to compile by changing the definition of inner to be of type &'a mut[T], but then inner is mutable even when I don't need it to be (in the above example, I must then declare store as mutable too even though test is immutable).
Is there a way to make it so that the mutability of inner follows the mutability of the Test instance?
As well said in the question, this code compiles:
struct Test<'a, A: 'a> {
inner: &'a mut A,
}
fn main() {
let t = Test { inner: &mut 5i32 };
*t.inner = 9;
}
It is indeed possible to mutate a borrowed element, even when the borrowing content is immutable. This is a case where you must choose your guarantees, while keeping in mind that the mutability of a binding is always independent of the borrowed content's mutability.
Right now, I can think of two possible solutions: you can encapsulate the borrowed content over methods that depend on self's mutability (Playground, will no longer compile):
impl<'a, A: 'a> Test<'a, A> {
fn inner(&self) -> &A {
self.inner
}
fn inner_mut(&mut self) -> &mut A {
self.inner
}
}
Although you still need to keep a borrow to mutable content, it can no longer be mutated from an immutable binding of Test. If you also need it to point to immutable content, you should consider having two different structs (Playground):
struct Test<'a, A: 'a> {
inner: &'a A,
}
impl<'a, A: 'a> Test<'a, A> {
fn inner(&self) -> &A {
self.inner
}
}
struct TestMut<'a, A: 'a> {
inner: &'a mut A,
}
impl<'a, A: 'a> TestMut<'a, A> {
fn inner(&self) -> &A {
self.inner
}
fn inner_mut(&mut self) -> &mut A {
self.inner
}
}
There is a third option: to keep both kinds of borrows exclusively with an enum. At this point however, using the borrowed content as mutable requires run-time checks.

Is it possible to move even with immutable borrows?

use std::marker;
use std::ops;
pub struct Shared<'r, T: 'r> {
data: *mut T,
_pd: marker::PhantomData<&'r T>,
}
impl<'r, T> Shared<'r, T> {
pub fn new(value: T) -> Shared<'r, T> {
let boxed = Box::new(value);
Shared {
data: Box::into_raw(boxed),
_pd: marker::PhantomData,
}
}
pub fn as_ref(&self) -> SharedRef<'r, T> {
SharedRef {
data: self.data,
_pd: marker::PhantomData,
}
}
}
impl<'r, T> ops::Deref for Shared<'r, T> {
type Target = T;
fn deref(&self) -> &T {
unsafe { &*self.data }
}
}
pub struct SharedRef<'r, T: 'r> {
data: *mut T,
_pd: marker::PhantomData<&'r T>,
}
impl<'r, T> ops::Deref for SharedRef<'r, T> {
type Target = T;
fn deref(&self) -> &T {
unsafe { &*self.data }
}
}
impl<'r, T> Drop for Shared<'r, T> {
fn drop(&mut self) {
unsafe {
Box::from_raw(self.data);
}
}
}
fn main() {
let s = Shared::new(42);
let s_ref = s.as_ref();
{
let s1 = s;
}
// lifetime should end here
println!("{}", *s_ref);
}
What I wanted to express was a mix between a Box and an Arc. A uniquely owned pointer that is also capable of giving out references.
The problem is that I want to be able to move Shared around even if there are currently immutable borrows to it. It should be legal in this scenario because it is heap allocated.
The problem is that I have no idea how to express this.
fn main() {
let s = Shared::new(42);
let s_ref = s.as_ref();
{
let s1 = s;
}
// lifetime should end here
println!("{}", *s_ref);
}
Here I move s into a scope with "less" lifetime than it had before. But now after I have moved s into s1, s_ref should not be accessible anymore. So what I want to say is that it is okay to move a Shared if the lifetime does not get smaller.
Can this be expressed in Rust?
The reason Rust allows you to move out of the Shared is that you haven't tied the lifetime of the returned SharedRef to it:
pub fn as_ref(&self) -> SharedRef<'r, T> {
SharedRef {
data: self.data,
_pd: marker::PhantomData,
}
}
Annotating the &self fixes that:
pub fn as_ref(&'r self) -> SharedRef<'r, T> { .. }
My current understanding is that the key difference here is that this says that the lifetime of the SharedRef now matches the lifetime of the borrow of self, keeping the borrow alive. Indeed it doesn't have to be the same lifetime ('r) as in the Shared; it works with a new lifetime just for the borrow/return:
pub fn as_ref<'b>(&'b self) -> SharedRef<'b, T> { .. }
This also disallows the move.
As for the bonus part of the question, where you want to allow moving as long as it's to something with a long enough lifetime, I think the answer is no. The only way I know to stop something being moved at all is to borrow it, and that stops any move.

Resources