PhantomData seems not work for dropck in rust - rust

use core::marker::PhantomData;
fn main() {
struct Inspector<'a>(&'a u8);
impl<'a> Drop for Inspector<'a> {
fn drop(&mut self) {
println!("I was only {} days from retirement!", self.0);
}
}
struct Uniqu<T> {
ptr: *mut T,
marker: PhantomData<T>,
}
struct World<'a> {
ptr: Option<Uniqu<Inspector<'a>>>,
days: u8,
}
let mut world = World{ptr: None, days:1};
let mut inspector = Inspector(&world.days);
let mut uniqu = Uniqu::<Inspector>{ptr: &mut inspector, marker: PhantomData};
world.ptr = Some(uniqu);
}
By refer to drop check rule, It can't compiles successfully. But successfully.
And if i use such code:
use core::marker::PhantomData;
fn main() {
struct Inspector<'a>(&'a u8);
impl<'a> Drop for Inspector<'a> {
fn drop(&mut self) {
println!("I was only {} days from retirement!", self.0);
}
}
struct World<'a> {
ptr: Option<Box<Inspector<'a>>>,
days: u8,
}
let mut world = World{ptr: None, days:1};
let mut inspector = Inspector(&world.days);
world.ptr = Some(Box::new(inspector));
}
Just use Box instead of Uniqu In World struct, it compiles unsuccessfully:
error[E0597]: `world.days` does not live long enough
--> src/main.rs:21:35
|
21 | let mut inspector = Inspector(&world.days);
| ^^^^^^^^^^^ borrowed value does not live long enough
...
26 | }
| -
| |
| `world.days` dropped here while still borrowed
| borrow might be used here, when `world` is dropped and runs the destructor for type `World<'_>`
In this situation, drop checker have checked it. This is what I need.
In first situation, drop checker haven't checked it. PhantomData seems not work, why?

In your first example, Uniqu does not implement Drop, so it has no potentially-unsafe drop code to check for.
Box however does implement Drop - so that it can free the memory it has allocated - which is why it does not pass the drop checker.
Adding a trivial Drop implementation to Uniqu in your first code section does cause the compiler to reject the code, as seen in the playground.

Related

Passing both &mut self and &self to the same function

Stripped down to the bare essentials, my problematic code looks as follows:
pub struct Item;
impl Item {
/// Partial copy. Not the same as simple assignment.
pub fn copy_from(&mut self, _other: &Item) {
}
}
pub struct Container {
items: Vec<Item>,
}
impl Container {
pub fn copy_from(&mut self, self_idx: usize, other: &Container, other_idx: usize) {
self.items[self_idx].copy_from(&other.items[other_idx]);
}
}
fn main() {
let mut container = Container { items: vec![Item, Item] };
container.copy_from(0, &container, 1);
}
This is of course rejected by the borrow checker:
error[E0502]: cannot borrow `container` as mutable because it is also borrowed as immutable
--> src/main.rs:21:5
|
21 | container.copy_from(0, &container, 1);
| ^^^^^^^^^^---------^^^^----------^^^^
| | | |
| | | immutable borrow occurs here
| | immutable borrow later used by call
| mutable borrow occurs here
For more information about this error, try `rustc --explain E0502`.
I understand why that happens, but I don't have a good solution.
I've considered adding a dedicated copy_from_self function that callers need to use in cases where self == other:
pub fn copy_from_self(&mut self, to_idx: usize, from_idx: usize) {
if to_idx != from_idx {
unsafe {
let from_item: *const Item = &self.items[from_idx];
self.items[to_idx].copy_from(&*from_item);
}
}
}
But this is un-ergonomic, bloats the API surface, and needs unsafe code inside.
Note that in reality, the internal items data structure is not a simple Vec, so any approach specific to Vec or slice will not work.
Is there an elegant, idiomatic solution to this problem?
If I understand the comments on the question correctly, a general solution seems to be impossible, so this answer is necessarily specific to my actual situation.
As mentioned, the actual data structure is not a Vec. If it were a Vec, we could use split_at_mut to at least implement copy_from_self safely.
But as it happens, my actual data structure is backed by a Vec, so I was able to add a helper function:
/// Returns a pair of mutable references to different items. Useful if you need to pass
/// a reference to one item to a function that takes `&mut self` on another item.
/// Panics if `a == b`.
fn get_mut_2(&mut self, a: usize, b: usize) -> (&mut T, &mut T) {
assert!(a != b);
if a < b {
let (first, second) = self.items.split_at_mut(b);
(&mut first[a], &mut second[0])
} else if a > b {
let (first, second) = self.items.split_at_mut(a);
(&mut second[0], &mut first[b])
} else {
panic!("cannot call get_mut_2 with the same index {} == {}", a, b);
}
}
Now we can implement copy_from_self without unsafe code:
pub fn copy_from_self(&mut self, to_idx: usize, from_idx: usize) {
let (to, from) = self.items.get_mut_2(to_idx, from_idx);
to.unwrap().copy_from(from.unwrap());
}

Parameterized lifetime causing trouble in chains of function

playground link
I'm having trouble trouble understanding why this code can't compile:
struct Ctx<'a> {
n: u64,
phantom: std::marker::PhantomData<&'a u8>,
}
fn process<'a, 'b: 'a>(ctx: &'b mut Ctx<'a>) {
ctx.n += 1;
}
fn main() {
let mut ctx = Ctx { n: 0, phantom: std::marker::PhantomData };
let ref_ctx = &mut ctx;
process(ref_ctx);
process(ref_ctx);
}
The code is silly, but the error is the same as on the "real" code:
Compiling playground v0.0.1 (/playground)
error[E0499]: cannot borrow `*ref_ctx` as mutable more than once at a time
--> src/main.rs:14:13
|
13 | process(ref_ctx);
| ------- first mutable borrow occurs here
14 | process(ref_ctx);
| ^^^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
For more information about this error, try `rustc --explain E0499`.
error: could not compile `playground` due to previous error
If change the process signature to
fn process(ctx: &mut Ctx<'a>) {
Then it compiles, but I need the lifetimes annotations because in practice Ctx contains references that needs to be annotated. I feel like the problem is 'b: 'a because is could be telling that the borrow MUST outlive the references in Ctx, but it feels weird that just calling a function that returns nothing would extend the borrow,
That feels like a beginner's question, but it definitely has me stumped.
edit:
The code above is minimal because I wanted to avoid explaining the whole context (pun intended), but it might be necessary to find a practical solution. Sorry, that's going to be a bit long.
The real Ctx is here, it's struct that wraps a few Varnish C pointers and gets passed to plugins so they can read it and possibly change it, notably manipulating HTTP objects.
An important part of Ctx is the WS contained in it. It's a workspace that can return non-overlapping slices of writable data that will be deleted automatically once a task has been completed. For the compiler, the best we can tell it is "any slice allocated in the WS will survive for as long as the Ctx wil live".
WS can be used in two ways:
either directly by a plugin to request a slice it'll fill in
or by the boilerplate that's currently python-generated to convert Varnish types into C types. Typically, the plugin returns a String and the boilerplate will allocate a WS slice and copy the data for storage. And example of generated code looks like this:
// the C-matching function that needs to exist for the plugin frame work to pick it up
unsafe extern "C" fn vmod_c_unset_hdr(vrt_ctx: * mut varnish_sys::vrt_ctx, arg1: varnish_sys::VCL_STRING) {
// create the rust Ctx from the pointer
let mut _ctx = Ctx::new(vrt_ctx);
// call the plugin function
match vmod::unset_hdr(
&mut _ctx,
&*arg1.into_rust()
).into_result() { // make sure we have a Result by wrapping lone values
Err(ref e) => { _ctx.fail(e); }, // yell if we didn't got and Err
Ok(v) => v.into_vcl(&mut _ctx.ws), // convert the Ok value into a C type, possibly by copying it into the workspace
}
}
Now, so far, the code at the commit I shared works (hurray), but I get into trouble when I try to implement WS_ReserveAll and WS_Release: basically, grab all the free space you can, write what you need, and then tell Varnish how much you used so the rest is reclaimed and can be used by future calls.
Of course, you can only WS_ReserveAll() once before you need to WS_Release so that felt like a perfect job for Rust.
I though I'd create a new type ReservedBuf, by adding this to the code:
impl<'a> struct WS<'a> {
// some lines omitted
pub fn reserve(&'a mut self) -> ReservedBuf<'a> {
let wsp = unsafe { self.raw.as_mut().unwrap() };
assert_eq!(wsp.magic, varnish_sys::WS_MAGIC);
unsafe {
let sz = varnish_sys::WS_ReserveAll(wsp) as usize;
let buf = from_raw_parts_mut(wsp.f as *mut u8, sz as usize);
ReservedBuf {
buf,
wsp: self.raw,
b: wsp.f as *const u8,
len: 0,
}
}
}
}
pub struct ReservedBuf<'a> {
pub buf: &'a mut [u8],
wsp: *mut varnish_sys::ws,
b: *const u8,
len: usize,
}
impl<'a> ReservedBuf<'a> {
pub fn seal(mut self, sz: usize) -> &'a [u8] {
unsafe {
self.len = std::cmp::min(sz, self.buf.as_ptr().add(sz).offset_from(self.b) as usize);
from_raw_parts_mut(self.buf.as_mut_ptr(), self.len)
}
}
}
impl<'a> Drop for ReservedBuf<'a> {
fn drop(&mut self) {
unsafe {
let wsp = self.wsp.as_mut().unwrap();
assert_eq!(wsp.magic, varnish_sys::WS_MAGIC);
varnish_sys::WS_Release(wsp, self.len as u32);
}
}
}
And it appears to work well enough in the limited tests I ran. My plugin function can be this for example:
pub fn ws_reserve<'a, 'b: 'a>(ctx: &'b mut Ctx<'a>, s: &str) -> Result<varnish_sys::VCL_STRING, String> {
let mut rbuf = ctx.ws.reserve();
let s_buf = s.as_bytes();
let vcl_string = rbuf.buf.as_ptr() as *const i8;
rbuf.buf.write(s_buf);
rbuf.buf.write(b" ");
rbuf.buf.write(s_buf);
rbuf.buf.write(b" ");
rbuf.buf.write(s_buf);
rbuf.buf.write(b"\0");
rbuf.seal(0);
Ok(vcl_string)
}
but I need the lifetime annotations, otherwise I get:
--> src/vmod.rs:21:27
|
20 | pub fn ws_reserve(ctx: &mut Ctx, s: &str) -> Result<varnish_sys::VCL_STRING, String> {
| --------
| |
| these two types are declared with different lifetimes...
21 | let mut rbuf = ctx.ws.reserve();
| ^^^^^^^ ...but data from `ctx` flows into `ctx` here
but with the lifetime annotations, the generated-code calling ws_reserve complains:
unsafe extern "C" fn vmod_c_ws_reserve(vrt_ctx: * mut varnish_sys::vrt_ctx, arg1: varnish_sys::VCL_STRING) -> varnish_sys::VCL_STRING {
let mut _ctx = Ctx::new(vrt_ctx);
match vmod::ws_reserve(
&mut _ctx,
&*arg1.into_rust()
).into_result() {
Err(ref e) => { _ctx.fail(e); ptr::null() },
Ok(v) => v.into_vcl(&mut _ctx.ws),
}
}
error[E0499]: cannot borrow `_ctx` as mutable more than once at a time
--> /home/gquintard/project/varnish-rs/vmod_test/target/debug/build/vmod_example-7ff3d57b642f2bfa/out/generated.rs:69:29
|
66 | &mut _ctx,
| --------- first mutable borrow occurs here
...
69 | Err(ref e) => { _ctx.fail(e); ptr::null() },
| ^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
error[E0499]: cannot borrow `_ctx.ws` as mutable more than once at a time
--> /home/gquintard/project/varnish-rs/vmod_test/target/debug/build/vmod_example-7ff3d57b642f2bfa/out/generated.rs:70:33
|
66 | &mut _ctx,
| --------- first mutable borrow occurs here
...
70 | Ok(v) => v.into_vcl(&mut _ctx.ws),
| ^^^^^^^^^^^^
| |
| second mutable borrow occurs here
| first borrow later used here
For more information about this error, try `rustc --explain E0499`.
error: could not compile `vmod_example` due to 2 previous errors
I guess I sort of understand the compiler qualms about the issues, and I might have painted myself into a corner, but I don't see how to solve this.
When we take a closer look at your code we see you write this:
fn process<'a, 'b: 'a>(ctx: &'b mut Ctx<'a>) {
This 'b: 'a reads as 'b outlives 'a. So you're basically writing a method which borrows ctx longer than what ctx is holding internally with 'a.
With that knowledge it seems very logical why this won't work.
let mut ctx = Ctx { n: 0, phantom: std::marker::PhantomData };
let ref_ctx = &mut ctx;
process(ref_ctx); // Borrow ref_ctx for at least the lifetime of `Ctx::<'a>`
process(ref_ctx); // Can't borrow it again since `'a` has not expired yet
What you're trying to do is probably this:
fn process<'a: 'b, 'b>(ctx: &'b mut Ctx<'a>) {
Borrow ctx for the lifetime 'b which is less long than 'a
If that's the case you can just write this:
fn process<'a>(ctx: &mut Ctx<'a>) {
Or even this:
fn process(ctx: &mut Ctx) {
If this isn't what you're trying to do I hope it still makes sense that using the constraint 'b: 'a creates an impossible borrow.

A struct with a trait field, but optional

Say I have a struct whose implementation writes somewhere, i.e. to something that implements the std::io::Write trait. However, I don't want the struct to own this. The following code works:
fn main() {
let mut out = std::io::stdout();
let mut foo = Foo::new(&mut out);
foo.print_number(2);
}
struct Foo<'a> {
out: &'a mut dyn std::io::Write
}
impl<'a> Foo<'a> {
pub fn new(out: &'a mut dyn std::io::Write) -> Self {
Self {
out
}
}
pub fn print_number(&mut self, i: isize) {
writeln!(self.out, "The number is {}", i).unwrap()
}
}
But, now this writing functionality should be made optional. I thought this sounds easy enough, but now the following doesn't compile:
fn main() {
let mut out = std::io::stdout();
let mut foo = Foo::new(Some(&mut out));
foo.print_number(2);
}
struct Foo<'a> {
out: Option<&'a mut dyn std::io::Write>
}
impl<'a> Foo<'a> {
pub fn new(out: Option<&'a mut dyn std::io::Write>) -> Self {
Self {
out
}
}
pub fn print_number(&mut self, i: isize) {
if self.out.is_some() {
writeln!(self.out.unwrap(), "The number is {}", i).unwrap()
}
}
}
because of:
error[E0507]: cannot move out of `self.out` which is behind a mutable reference
--> src/main.rs:20:26
|
20 | writeln!(self.out.unwrap(), "The number is {}", i).unwrap()
| ^^^^^^^^
| |
| move occurs because `self.out` has type `Option<&mut dyn std::io::Write>`, which does not implement the `Copy` trait
| help: consider borrowing the `Option`'s content: `self.out.as_ref()`
which I'm not sure how to interpret.
I tried following the suggestion by changing the line in question to:
writeln!(self.out.as_ref().unwrap(), "The number is {}", i).unwrap()
but then I get
error[E0596]: cannot borrow data in a `&` reference as mutable
--> src/main.rs:20:26
|
20 | writeln!(self.out.as_ref().unwrap(), "The number is {}", i).unwrap()
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
I'm really not sure how to interpret these error messages and surprisingly I'm not really getting anywhere by just sprinkling &s and muts in random places without really understanding!
(As an aside, I'm not sure if this is a "good" way of going about this anyway? I'm open to completely different approaches of solving this problem, which is basically to optionally pass something to write to into a struct, but without the struct owning it. I read about the Box type which might also be relevant?)
As you already know, based on you already using &mut for out. The issue with using as_ref() is that it returns an immutable reference. Instead you need to use as_mut().
pub fn print_number(&mut self, i: isize) {
if self.out.is_some() {
writeln!(self.out.as_mut().unwrap(), "The number is {}", i).unwrap()
}
}
Alternatively, you can also simplify this and express it more idiomatically using if let:
pub fn print_number(&mut self, i: isize) {
if let Some(out) = &mut self.out {
writeln!(out, "The number is {}", i).unwrap()
}
}
I would also suggest that instead of unwrapping, that you return the io::Result and let the caller handle any potential error.
pub fn print_number(&mut self, i: isize) -> std::io::Result<()> {
if let Some(out) = &mut self.out {
writeln!(out, "The number is {}", i)?;
}
Ok(())
}
You can also simplify your paths, e.g. std::io::Write and std::io::Result<()>, by importing them with a use declaration, e.g. use std::io::{self, Write}; and then changing them to Write and io::Result<()>.

"borrowed value does not live long enough" after implementing Drop trait

Being new to rust I wanted to play with some data structures and ended up with something like a node type without payload.
use std::cell::RefCell;
use std::collections::HashMap;
use std::ops::Drop;
#[derive(Debug)]
struct Container<'a> {
next : Option<&'a RefCell<Container<'a>>>,
}
impl<'a> Container<'a> {
fn new() -> Container<'a> {
Container { next: None }
}
fn set(&mut self, next: &'a RefCell<Container<'a>>) {
self.next = Some(next);
}
}
The goal was to have these nodes not own their neighbours, so std::rc::Rc was out of the question.
So I did some testing which went fine:
fn main() {
// items:
let cont_1 = RefCell::new(Container::new());
let cont_2 = RefCell::new(Container::new());
let b_1 = &cont_1;
let b_2 = &cont_2;
(*b_2).borrow_mut().set(b_1);
(*b_1).borrow_mut().set(b_2);
println!("{:?}", b_1.borrow());
}
Since I was playing around I then tried to implement the Drop trait on the Container type
impl<'a> Drop for Container<'a>{
fn drop(&mut self) {}
}
which results in two of (the other one for cont_2)
error[E0597]: `cont_1` does not live long enough
--> src/main.rs:11:15
|
11 | let b_1 = &cont_1;
| ^^^^^^^ borrowed value does not live long enough
...
18 | }
| -
| |
| `cont_1` dropped here while still borrowed
| borrow might be used here, when `cont_1` is dropped and runs the destructor for type `std::cell::RefCell<Container<'_>>`
Now, I believe, that Drop causes the deallocation to be at the end of scopes otherwise it would usually take place after the last use? But either way the complaint is about the value not living long enough... I have tried adding drop(...) in between, but failed. I guess I dont even understand how exactly the order of deallocation changed. I would expect that cont_1 would be deallocated last since it was initialized/declared first meaning that I don't really understand how it could still be borrowed.
What would happen if in your drop() implementation you use self.next.unwrap()...? Since one of your variables will necessarily be dropped before the other, the last one will have a dangling reference, and so undefined behavior. So you code is correct in not to compile.
IMO, the solution is to use some kind of reference counted pointers. If you do not want Rc, because they do not own the neighbors (it will create a reference loop and thus leak your objects), you can use Weak references. Something like this (playground):
use std::cell::RefCell;
use std::ops::Drop;
use std::rc::{Rc, Weak};
#[derive(Debug)]
struct Container {
next : Option<Weak<RefCell<Container>>>,
}
impl Container {
fn new() -> Container {
Container { next: None }
}
fn set(&mut self, next: &Rc<RefCell<Container>>) {
self.next = Some(Rc::downgrade(next));
}
}
impl Drop for Container{
fn drop(&mut self) {}
}
fn main() {
// items:
let cont_1 = Rc::new(RefCell::new(Container::new()));
let cont_2 = Rc::new(RefCell::new(Container::new()));
cont_1.borrow_mut().set(&cont_1);
cont_2.borrow_mut().set(&cont_2);
println!("{:?}", cont_1.borrow());
}

How can I tell the compiler to release a borrow in a struct without dropping the entire struct?

I have the following struct which represents a plan for a numeric computation:
pub struct NfftPlan<'a> {
x: Option<&'a [f64]>,
f_hat: Option<&'a [Complex64]>,
// ...
}
It has a set_f_hat method:
pub fn set_f_hat(&mut self, f_hat: &'a [Complex64]) {
self.f_hat = Some(f_hat);
}
and an execute method:
pub fn execute(&self) -> Vec<Complex64>
which uses f_hat immutably.
I want to use this in the following way:
let mut f_hat = vec![1,2,3,4];
let plan = NfftPlan::new()
plan.set_f_hat(&f_hat);
plan.execute();
f_hat[0] = 3; // Change f_hat to a new value
plan.execute(); //New computation
This fails because I cant mutate f_hat while plan still exists.
Is there a way for plan to release the borrow to f_hat which would allow me to mutate the f_hat vector?
Something like this:
releasedata(&self) {
self.f_hat = None
} //Now the compiler forgets that plan would hold an borrow to f_hat
I know that Rust does not allow me to change the vector while a borrow to it exists, in this case via the f_hat reference in the NfftPlan struct.
I would like a way to tell the compiler to drop the reference to the vector in the NfftPlan struct without dropping the entire struct.
Explanation
How can I tell the compiler to release a borrow
You cannot, period. This isn't something you "tell" the compiler, the compiler knows all. You can only completely stop using the reference.
without dropping the entire struct
Dropping doesn't clear the borrow, only the borrow no longer being used does, which may happen from the drop.
f_hat[0] = 3; // Change f_hat to a new value
plan.execute(); //New computation
This is exactly one of the types of code that Rust tries to prevent. It is not obvious at all that plan.execute() should return a different value because some apparently unrelated value has changed.
Solutions
Encode it in the type system
I'd structure my types to reflect how they need to be used, creating throwaway values that can execute only once everything has been combined together. This means that the structure that borrows f_mut is dropped as soon as it's done; note how this removes the Option completely:
fn main() {
let mut f_hat = 42;
let plan = Plan::default();
plan.set_f_hat(&f_hat).execute();
f_hat = 3;
plan.set_f_hat(&f_hat).execute();
}
#[derive(Debug, Default)]
struct Plan<'a> {
x: Option<&'a i32>,
}
impl<'a> Plan<'a> {
fn set_f_hat(&self, f_hat: &'a i32) -> PlanPlus<'a> {
PlanPlus { x: self.x, f_hat }
}
}
#[derive(Debug)]
struct PlanPlus<'a> {
x: Option<&'a i32>,
f_hat: &'a i32,
}
impl<'a> PlanPlus<'a> {
fn execute(&self) {}
}
Use interior mutability and reference counting
use std::{cell::Cell, rc::Rc};
#[derive(Debug, Default)]
struct Plan<'a> {
x: Option<&'a i32>,
f_hat: Option<Rc<Cell<i32>>>,
}
impl<'a> Plan<'a> {
fn set_f_hat(&mut self, f_hat: Rc<Cell<i32>>) {
self.f_hat = Some(f_hat);
}
fn execute(&self) {}
}
fn main() {
let f_hat = Rc::new(Cell::new(42));
let mut plan = Plan::default();
plan.set_f_hat(f_hat.clone());
plan.execute();
f_hat.set(3);
plan.execute();
}
Recognize that the member is mutable
#[derive(Debug, Default)]
struct Plan<'a> {
x: Option<&'a i32>,
f_hat: Option<&'a mut i32>,
}
impl<'a> Plan<'a> {
fn f_hat(&mut self) -> &mut Option<&'a mut i32> {
&mut self.f_hat
}
fn execute(&self) {}
}
fn main() {
let mut f_hat = 42;
let mut plan = Plan::default();
*plan.f_hat() = Some(&mut f_hat);
plan.execute();
**plan.f_hat().as_mut().unwrap() = 3;
plan.execute();
}
See also:
What are the options to end a mutable borrow in Rust?
Drop a immutable borrow to make a mutable borrow
Moved variable still borrowing after calling `drop`?

Resources