My question is about dereferencing and referencing in rust.
I have the following code:
#[database("pg_db")]
struct PgDbConn(diesel::PgConnection);
fn main() {
rocket::ignite()
.attach(PgDbConn::fairing())
.mount("/", routes![find_one, find_all])
.launch();
}
#[get("/<id>", format = "json")]
fn find_one(conn: PgDbConn, id: i32) -> Result<Json<Person>, NotFound<String>> {
let one: QueryResult<Person> = person.find(id).first(&*conn); // Notice here deref & ref
...
I would like to know how my PgDbConn struct ends up as a connection. Can someone please explain the mechanism in detail?
Let's have a look at (part of) the implementation of the database attribute macro:
Ok(quote! {
//...
impl ::std::ops::Deref for #guard_type {
type Target = #conn_type;
#[inline(always)]
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl ::std::ops::DerefMut for #guard_type {
#[inline(always)]
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.0
}
}
})
#guard_type is PgDbConn and #conn_type is diesel::PgConnection in your example, so the produced code looks like this:
impl ::std::ops::Deref for PgDbConn {
type Target = diesel::PgConnection;
#[inline(always)]
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl ::std::ops::DerefMut for PgDbConn {
#[inline(always)]
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.0
}
}
With Deref and DerefMut, you can implement dereference for your own types, in this case PgDbConn. Now, you can write *conn to get a diesel::PgConnection from your PgDbConn. However, you want a reference to diesel::PgConnection. To get a reference, you have to again reference the dereferenced PgDbConn, so the end result is &*conn.
Related
given this example
struct Outer<T>(*mut T);
impl<T> Outer<T> {
pub fn new(value: &mut T) -> Outer<T> {
Outer(value as *mut T)
}
}
struct Inner(pub i32);
impl Inner {
pub fn do_thing(&self) {
println!("did the thing {}", self.0);
}
}
fn main() {
let outer = Outer::new(Inner(2));
outer.do_thing() // error: do_thing doesnt exist
}
how would i expose the methods of Inner as methods of Outer
im trying to achieve what Box is doing
You have to reborrow the pointer in order to implement Deref (playground):
impl<T: ?Sized> Deref for Outer<T> {
type Target = T;
fn deref(&self) -> &Self::Target {
unsafe { &*self.0 }
}
}
Answer from before the edit below:
The clean way is to have methods that return the inner type:
impl<T> Outer<T> {
pub fn get(&self) -> &T {
&self.0
}
pub fn get_mut(&mut self) -> &mut T {
&mut self.0
}
pub fn into_inner(self) -> T {
self.0
}
}
This is more useful when the inner type isn't pub (you can just do self.0 for the same effect as all three methods), but is a common pattern that users of rust libraries expect.
The other way is to implement Deref and DerefMut. This is messier because it's a very strong API commitment: you now have all the methods of the inner type added to the outer type unconditionally, and gain any future methods. It also means that any namespace collisions between the wrapper and the target are hard to notice, which is why things like Arc::get_mut are associated functions instead of methods. If you use this, consider changing methods on the wrapper to associated functions (don't take a self parameter).
impl<T> Deref for Outer<T> {
type Target = T;
fn deref(&self) -> &Self::Target {
&self.0
}
}
For example, this works:
pub struct SquareVecIter<'a> {
current: f64,
iter: core::slice::Iter<'a, f64>,
}
pub fn square_iter<'a>(vec: &'a Vec<f64>) -> SquareVecIter<'a> {
SquareVecIter {
current: 0.0,
iter: vec.iter(),
}
}
impl<'a> Iterator for SquareVecIter<'a> {
type Item = f64;
fn next(&mut self) -> Option<Self::Item> {
if let Some(next) = self.iter.next() {
self.current = next * next;
Some(self.current)
} else {
None
}
}
}
// switch to test module
#[cfg(test)]
mod tests_2 {
use super::*;
#[test]
fn test_square_vec() {
let vec = vec![1.0, 2.0];
let mut iter = square_iter(&vec);
assert_eq!(iter.next(), Some(1.0));
assert_eq!(iter.next(), Some(4.0));
assert_eq!(iter.next(), None);
}
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=531edc40dcca4a79d11af3cbd29943b7
But if I have to return references to self.current then I can't get the lifetimes to work.
Yes.
An Iterator cannot yield elements that reference itself simply due to how the trait is designed (search "lending iterator" for more info). So even if you wanted to return Some(&self.current) and deal with the implications therein, you could not.
Returning an f64 (non-reference) is perfectly acceptable and would be expected because this is a kind of generative iterator. And you wouldn't need to store current at all:
pub struct SquareVecIter<'a> {
iter: core::slice::Iter<'a, f64>,
}
pub fn square_iter<'a>(vec: &'a Vec<f64>) -> SquareVecIter<'a> {
SquareVecIter {
iter: vec.iter(),
}
}
impl<'a> Iterator for SquareVecIter<'a> {
type Item = f64;
fn next(&mut self) -> Option<Self::Item> {
if let Some(next) = self.iter.next() {
Some(next * next)
} else {
None
}
}
}
For an example of this in the standard library, look at the Chars iterator for getting characters of a string. It keeps a reference to the original str, but it yields owned chars and not references.
What #kmdreko says, of course.
I just wanted to add a couple of nitpicks:
The pattern of the if let Some(...) = ... {Some} else {None} is so common that it made its way into the standard library as the .map function.
Taking a &Vec<f64> is an antipattern. Use &[f64] instead. It is more general without any drawbacks.
All of your lifetime annotations (except of the one in the struct definition) can be derived automatically, so you can simply omit them.
pub struct SquareVecIter<'a> {
iter: core::slice::Iter<'a, f64>,
}
pub fn square_iter(vec: &[f64]) -> SquareVecIter {
SquareVecIter { iter: vec.iter() }
}
impl Iterator for SquareVecIter<'_> {
type Item = f64;
fn next(&mut self) -> Option<Self::Item> {
self.iter.next().map(|next| next * next)
}
}
pub struct ForesterViewModel {
m_tree_lines: Arc<Mutex<Vec<TreeLine>>>,
}
impl ForesterViewModel {
pub fn new() -> ForesterViewModel {
ForesterViewModel {
m_tree_lines: Arc::new(Mutex::new(vec![])),
}
}
pub fn get_the_forest(&mut self) -> &mut Vec<TreeLine> {
???????????????????????????????
}
}
I need help writing the get_the_forest function. I've tried many various things but they all return compilation errors. I need to return a mutable reference to Vec<TreeLine> which is wrapped behind an Arc and a Mutex in self.m_tree_lines.
There is no way of doing this.
You create a concrete MutexGuard object that releases the mutex when it dropped when you call lock; you cannot move a reference out of the scope that contains the guard:
pub fn as_mut(&mut self) -> &Whatever {
let mut guard = self.data.lock().unwrap();
Ok(guard.deref())
drop(guard) // <--- implicitly added here, which would invalidate the ref
}
You also cannot return both the mutex guard and a reference, for more complex reasons (basically rust cannot express that), for the same reason it cannot have a reference and an object in a single structure; see the discussion on Why can't I store a value and a reference to that value in the same struct?
...so basically your best bet is one of two things:
/// Return the mutex guard itself
pub fn get_the_forest(&mut self) -> Result<MutexGuard<Vec<TreeLine>>, TreeLockError> {
Ok(self.m_tree_lines.lock()?)
}
/// Pass a function in, which patches the mutable internal value
pub fn patch_forest(&mut self, patch: impl Fn(&mut Vec<TreeLine>)) -> Result<(), TreeLockError>{
let mut guard = self.m_tree_lines.lock()?;
patch(&mut guard); // <-- patch happens while guard is still alive
Ok(())
}
Full code:
use std::sync::{Arc, Mutex, MutexGuard};
use std::sync::PoisonError;
use std::error::Error;
use std::fmt;
use std::fmt::Formatter;
use std::ops::Deref;
#[derive(Debug, Copy, Clone)]
pub enum TreeLockError {
FailedToLock
}
impl Error for TreeLockError {}
impl fmt::Display for TreeLockError {
fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result {
write!(f, "{:?}", self)
}
}
impl<T> From<PoisonError<T>> for TreeLockError {
fn from(_: PoisonError<T>) -> Self {
TreeLockError::FailedToLock
}
}
// ---
#[derive(Debug)]
pub struct TreeLine {
pub value: &'static str
}
pub struct ForesterViewModel {
m_tree_lines: Arc<Mutex<Vec<TreeLine>>>,
}
impl ForesterViewModel {
pub fn new() -> ForesterViewModel {
ForesterViewModel {
m_tree_lines: Arc::new(Mutex::new(vec![])),
}
}
pub fn get_the_forest(&mut self) -> Result<MutexGuard<Vec<TreeLine>>, TreeLockError> {
Ok(self.m_tree_lines.lock()?)
}
pub fn patch_forest(&mut self, patch: impl Fn(&mut Vec<TreeLine>)) -> Result<(), TreeLockError>{
let mut guard = self.m_tree_lines.lock()?;
patch(&mut guard);
Ok(())
}
}
fn main() -> Result<(), Box<dyn Error>> {
let mut vm = ForesterViewModel::new();
{
let mut trees = vm.get_the_forest()?;
trees.push(TreeLine{ value: "one"});
trees.push(TreeLine{ value: "two"});
} // <--- Drop the mutable reference here so you can get it again later
// Patch
vm.patch_forest(|trees| {
trees.push(TreeLine{ value: "three"});
});
// ...
let trees = vm.get_the_forest()?;
println!("{:?}", trees.deref());
Ok(())
}
use std::marker;
use std::ops;
pub struct Shared<'r, T: 'r> {
data: *mut T,
_pd: marker::PhantomData<&'r T>,
}
impl<'r, T> Shared<'r, T> {
pub fn new(value: T) -> Shared<'r, T> {
let boxed = Box::new(value);
Shared {
data: Box::into_raw(boxed),
_pd: marker::PhantomData,
}
}
pub fn as_ref(&self) -> SharedRef<'r, T> {
SharedRef {
data: self.data,
_pd: marker::PhantomData,
}
}
}
impl<'r, T> ops::Deref for Shared<'r, T> {
type Target = T;
fn deref(&self) -> &T {
unsafe { &*self.data }
}
}
pub struct SharedRef<'r, T: 'r> {
data: *mut T,
_pd: marker::PhantomData<&'r T>,
}
impl<'r, T> ops::Deref for SharedRef<'r, T> {
type Target = T;
fn deref(&self) -> &T {
unsafe { &*self.data }
}
}
impl<'r, T> Drop for Shared<'r, T> {
fn drop(&mut self) {
unsafe {
Box::from_raw(self.data);
}
}
}
fn main() {
let s = Shared::new(42);
let s_ref = s.as_ref();
{
let s1 = s;
}
// lifetime should end here
println!("{}", *s_ref);
}
What I wanted to express was a mix between a Box and an Arc. A uniquely owned pointer that is also capable of giving out references.
The problem is that I want to be able to move Shared around even if there are currently immutable borrows to it. It should be legal in this scenario because it is heap allocated.
The problem is that I have no idea how to express this.
fn main() {
let s = Shared::new(42);
let s_ref = s.as_ref();
{
let s1 = s;
}
// lifetime should end here
println!("{}", *s_ref);
}
Here I move s into a scope with "less" lifetime than it had before. But now after I have moved s into s1, s_ref should not be accessible anymore. So what I want to say is that it is okay to move a Shared if the lifetime does not get smaller.
Can this be expressed in Rust?
The reason Rust allows you to move out of the Shared is that you haven't tied the lifetime of the returned SharedRef to it:
pub fn as_ref(&self) -> SharedRef<'r, T> {
SharedRef {
data: self.data,
_pd: marker::PhantomData,
}
}
Annotating the &self fixes that:
pub fn as_ref(&'r self) -> SharedRef<'r, T> { .. }
My current understanding is that the key difference here is that this says that the lifetime of the SharedRef now matches the lifetime of the borrow of self, keeping the borrow alive. Indeed it doesn't have to be the same lifetime ('r) as in the Shared; it works with a new lifetime just for the borrow/return:
pub fn as_ref<'b>(&'b self) -> SharedRef<'b, T> { .. }
This also disallows the move.
As for the bonus part of the question, where you want to allow moving as long as it's to something with a long enough lifetime, I think the answer is no. The only way I know to stop something being moved at all is to borrow it, and that stops any move.
I'm wrapping a C library that has two structs: one has a pointer to the other.
struct StructA {
void * some_mem;
};
struct StructB {
void * some_mem;
struct StructA * some_struct;
};
Both of these structs own memory, so my wrapper has constructors and destructors for both of them.
struct StructA(*mut c_void);
impl StructA {
fn new() -> Self {
StructA(c_constructor())
}
}
impl Drop for StructA {
fn drop(&mut self) {
let StructA(ptr) = self;
c_destructor(ptr);
}
}
There's also a function that takes a pointer to StructB and returns its pointer to StructA:
const struct StructA * get_struct(const struct StructB * obj);
The user of this function should not free the returned pointer, since it will be freed when the user frees obj.
How can I wrap this function? The problem is that the destructor for StructB frees all its memory, including the one for StructA. So if my wrapping of get_struct returns an object, then the wrapped StructA will be freed twice (right?). It could instead return a reference to an object, but where would that object live?
I could have separate structs for StructA based on whether it's standalone and needs to be freed or if it's a reference, but I'm hoping that's unnecessary.
I could have separate structs for StructA based on whether it's standalone and needs to be freed or if it's a reference, but I'm hoping that's unnecessary.
It's necessary. The difference between an owned StructA * and a borrowed StructA * is precisely the same as the difference between a Box<T> and a &T. They're both "just a pointer", but the semantics are completely different.
Something along these lines is probably what you want:
use std::marker::PhantomData;
struct OwnedA(*mut c_void);
impl Drop for OwnedA {
fn drop(&mut self) { }
}
impl OwnedA {
fn deref(&self) -> RefA { RefA(self.0, PhantomData) }
}
struct RefA<'a>(*mut c_void, PhantomData<&'a u8>);
struct OwnedB(*mut c_void);
impl Drop for OwnedB {
fn drop(&mut self) { }
}
impl OwnedB {
fn get_a(&self) -> RefA { RefA(get_struct(self.0), PhantomData) }
}
In particular, it's worth noting that lifetime parameter on RefA lets the compiler make sure you don't use a RefA after the backing structure has been freed.
I could have separate structs for StructA based on whether it's standalone and needs to be freed or if it's a reference, but I'm hoping that's unnecessary.
I believe this would be the accepted pattern. For backup, I'd point to the fact that this is a normal pattern in the Rust library. &str and String, &[T] and Vec<T>, Path and PathBuf, and probably lots of others I can't think of.
The good news is that you can use similar patterns as these pairs, leveraging Deref or DerefMut to call down to shared implementation:
use std::ops::{Deref, DerefMut};
enum RawFoo {}
fn c_foo_new() -> *const RawFoo { std::ptr::null() }
fn c_foo_free(_f: *const RawFoo) {}
fn c_foo_count(_f: *const RawFoo) -> u8 { 42 }
fn c_foo_make_awesome(_f: *const RawFoo, _v: bool) { }
struct OwnedFoo(Foo);
impl OwnedFoo {
fn new() -> OwnedFoo {
OwnedFoo(Foo(c_foo_new()))
}
}
impl Drop for OwnedFoo {
fn drop(&mut self) { c_foo_free((self.0).0) }
}
impl Deref for OwnedFoo {
type Target = Foo;
fn deref(&self) -> &Self::Target { &self.0 }
}
impl DerefMut for OwnedFoo {
fn deref_mut(&mut self) -> &mut Self::Target { &mut self.0 }
}
struct Foo(*const RawFoo);
impl Foo {
fn count(&self) -> u8 { c_foo_count(self.0) }
fn make_awesome(&mut self, v: bool) { c_foo_make_awesome(self.0, v) }
}
fn main() {
let mut f = OwnedFoo::new();
println!("{}", f.count());
f.make_awesome(true);
}
Then, when you get a borrowed pointer from your other object, just wrap it up in a &Foo:
use std::mem;
fn c_bar_foo_ref() -> *const RawFoo { std::ptr::null() }
// Ignoring boilerplate for wrapping the raw Bar pointer
struct Bar;
impl Bar {
fn new() -> Bar { Bar }
fn foo(&self) -> &Foo {
unsafe { mem::transmute(c_bar_foo_ref()) }
}
fn foo_mut(&mut self) -> &mut Foo {
unsafe { mem::transmute(c_bar_foo_ref()) }
}
}
fn main() {
let mut b = Bar::new();
println!("{}", b.foo().count());
b.foo_mut().make_awesome(true);
// Doesn't work - lifetime constrained to Bar
// let nope = Bar::new().foo();
}