This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
Closed 2 years ago.
I'm having troubles with a managed State with Rocket. This state holds a Database connection and a collection of Cursors on that Database. Each one of theses have a reference on the Database.
One of the operations on that state require to create a new cursor on the Database and keep it for later use. Unfortunatly, I am stuck with a lifetime problem. Usually, I have no problem dealing with thoses, but right now, I'm out of ideas...
I have recreated the problem bellow in a short example.
#![feature(proc_macro_hygiene, decl_macro)]
#[macro_use]
extern crate rocket;
use rocket::State;
struct Database;
impl Database {
fn create_cursor(&self) -> Cursor {
Cursor { database: self }
}
}
struct Cursor<'a> {
database: &'a Database
}
struct Controller<'a> {
database: Database,
cursors: Vec<Cursor<'a>>,
}
impl<'a> Controller<'a> {
fn do_work(&'a mut self) {
let cursor = self.database.create_cursor();
self.cursors.push(cursor)
}
}
fn main() {
let database = Database;
let controller = Controller { database, cursors: Vec::new() };
rocket::ignite()
.manage(controller)
.launch();
}
#[get("/")]
pub fn do_work_route(
mut controller: State<'static, Controller<'static>>
) -> &'static str {
controller.do_work();
"Something..."
}
error[E0621]: explicit lifetime required in the type of `__req`
--> src/main.rs:42:9
|
40 | #[get("/")]
| ----------- help: add explicit lifetime `'static` to the type of `__req`: `&'_b rocket::Request<'static>`
41 | pub fn do_work_route(
42 | mut controller: State<'static, Controller<'static>>
| ^^^^^^^^^^ lifetime `'static` required
Any lead would be appreciated. Meanwhile, I'll continue digging.
Thanks a lot!
Your Controller struct as written is self-referential, which is not possible: https://users.rust-lang.org/t/how-to-write-software-without-self-referential-structs/13819
The reason is that when a Controller gets moved, the references to the database in its cursors member would become invalid, because the memory location of its database member changed.
The best way forward is probably to step back and think about a design that is not self-referential. A possible solution is to make the database a static variable, then your cursors could store &'static Database references.
If that fails, the link above mentions the rental crate, but it doesn't seem to be easy to use.
Related
This question already has answers here:
How to store rusqlite Connection and Statement objects in the same struct in Rust? [duplicate]
(1 answer)
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
Closed 3 years ago.
I am trying to provide a thin abstraction over a database in Rust. Here's a simplified version of the underlying database API as I understand it:
// Connection to a database
struct Connection {}
impl Connection {
fn new() -> Connection {
Connection {}
}
}
// Prepared SQL statement
struct Statement<'a> {
conn: &'a Connection,
}
impl Connection {
// Create a prepared statement
fn prepare(&self) -> Statement {
return Statement { conn: self };
}
}
I want to provide a wrapper which upon construction connects to a database and stores some prepared statements. After constructing the wrapper I don't actually need the raw connection any more.
Here's a simplified version of my code:
struct Wrapper<'a> {
stmt: Statement<'a>,
}
impl<'a> Wrapper<'a> {
fn new() -> Wrapper<'a> {
let conn = Connection::new();
Wrapper {
stmt: conn.prepare(),
}
}
}
fn main() {
let _w = Wrapper::new();
}
Wrapper::new fails to compile because the returned value references conn whose lifetime ends at the end of the function ("error[E0515]: cannot return value referencing local variable conn").
The simplest solution is just to require the caller to construct a Connection; but then this is a rather leaky "wrapper" for the database.
Ideally I'd like to move conn out of the function. Since I'm moving the Wrapper out of the function, I thought I could just move conn into the Wrapper:
struct Wrapper<'a> {
conn: Connection,
stmt: Statement<'a>,
}
impl<'a> Wrapper<'a> {
fn new() -> Wrapper<'a> {
let conn = Connection {};
Wrapper {
stmt: conn.prepare(),
conn,
}
}
}
But this just adds another error: "cannot move out of conn because it is borrowed", even if I try to dodge the chicken-and-egg initialisation of the .stmt and .conn fields with Option & mutation hackery. I've read that you can't really have one struct member reference another in rust, which would explain this.
For a while I thought I could use an Rc, but for this to work I think it would need to be on the Statement's reference to the Connection; which is in library code.
Can anyone see which Rust design pattern I'm missing?
This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
How to return a reference to a sub-value of a value that is under a mutex?
(5 answers)
Returning a RWLockReadGuard independently from a method
(2 answers)
How can I store a Chars iterator in the same struct as the String it is iterating on?
(2 answers)
Closed 3 years ago.
I would like to wrap a low-level third-party API with my own struct and functions to make it more friendly. Unfortunately, the third-party API needs a reference to a socket in its constructor, which means I'd like my Struct to "own" the socket (someone has to own it so that it can be borrowed, right? and I'd like that hidden as an implementation detail in my API).
The third-party API looks something like this:
struct LowLevelApi<'a> {
stream: &'a mut TcpStream,
// ...
}
impl<'a> LowLevelApi<'a> {
pub fn new(socket: &'a mut TcpStream, ... ) -> LowLevelApi<'a> {
// ...
}
}
I would like to make the interface to my function look like:
pub fn new(host: String, port: u16, ...) -> HighLevelApi {
// ...
}
I tried this:
pub struct HighLevelApi<'a> {
stream: TcpStream,
low: LowLevelApi<'a>
}
impl <'a> HighLevelApi<'a> {
pub fn new(host: String, port: u16) -> HighLevelApi<'a> {
// Ignore lack of error checking for now
let mut stream = TcpStream::connect(format!("{}:{}", host, port)).unwrap();
HighLevelApi {
stream,
low: LowLevelApi::new(&mut stream)
}
}
}
Rust is (rightly) angry: It has no way of knowing that I'm not going to do something bad with the low field later. And even worse, I would need to somehow guarantee that when my structure gets dropped, low gets dropped first, and stream second (since by that point, any relationship between the two is lost - or rather, there never is/was a relationship between the two).
(actually, it's worse than that, because the stream local variable gets moved into the new struct, so the local can't possibly be borrowed by LowLevelApi, but I can't think of a way to initialize HighLevelApi with the stream from the struct, since there's no way to get a handle to that from within the struct's initialization, is there? But based on my guess about what would happen in the paragraph above, it doesn't really matter since it still wouldn't do what I wanted)
What are examples of the various techniques that can be used to store a wrap a third-party (not under my control) struct that needs a reference to something?
The Rental crate seems to do what is needed here, albeit with documentation and examples that leave a lot to the imagination (i.e. trial and error).
Here's roughly what solves this
rental! {
pub mod rentals {
#[rental_mut]
pub struct Wrapper {
stream: Box<TcpStream>,
low: LowLevelApi<'stream>
}
}
}
pub struct HighLevelApi {
wrapper: rentals::Wrapper
}
impl HighLevelApi {
pub fn new(host: String, port: u16) -> HighLevelApi {
Api {
// Ignore lack of error checking for now
wrapper: rentals::Wrapper::new(
Box::new(TcpStream::connect(format!("{}:{}", host, port)).unwrap()),
|s| LowLevelApi::new(s)
)
}
}
pub fn do_something(&mut self) {
self.wrapper.rent_mut(|ll| ll.do_something()) // ll is the LowLevelApi
}
}
I noticed two important things that made this work:
The lifetime name on low in the Wrapper struct must match the name of the "owning" field (in this case "'stream")
You never get direct access to the reference - you get it through a callback/closure:
In the auto-generated constructor (new()) the second parameter isn't a LowLevelApi, it's a closure that gets the &mut TcpStream, and that closure is then expected to return a LowLevelApi
When you want to actually use the LowLevelApi you can "rent" it, hence the wrapper.rent_mut(f) where f is the closure that gets passed a LowLevelApi (ll) and can do what it needs
With these facts, the rest of the Rental documentation makes a lot more sense.
I'm writing a game engine. In the engine, I've got a game state which contains the list of entities in the game.
I want to provide a function on my gamestate update which will in turn tell each entity to update. Each entity needs to be able to refer to the gamestate in order to correctly update itself.
Here's a simplified version of what I have so far.
pub struct GameState {
pub entities: Vec<Entity>,
}
impl GameState {
pub fn update(&mut self) {
for mut t in self.entities.iter_mut() {
t.update(self);
}
}
}
pub struct Entity {
pub value: i64,
}
impl Entity {
pub fn update(&mut self, container: &GameState) {
self.value += container.entities.len() as i64;
}
}
fn main() {
let mut c = GameState { entities: vec![] };
c.entities.push(Entity { value: 1 });
c.entities.push(Entity { value: 2 });
c.entities.push(Entity { value: 3 });
c.update();
}
The problem is the borrow checker doesn't like me passing the gamestate to the entity:
error[E0502]: cannot borrow `*self` as immutable because `self.entities` is also borrowed as mutable
--> example.rs:8:22
|
7 | for mut t in self.entities.iter_mut() {
| ------------- mutable borrow occurs here
8 | t.update(self);
| ^^^^ immutable borrow occurs here
9 | }
| - mutable borrow ends here
error: aborting due to previous error
Can anyone give me some suggestions on better ways to design this that fits with Rust better?
Thanks!
First, let's answer the question you didn't ask: Why is this not allowed?
The answer lies around the guarantees that Rust makes about & and &mut pointers. A & pointer is guaranteed to point to an immutable object, i.e. it's impossible for the objects behind the pointer to mutate while you can use that pointer. A &mut pointer is guaranteed to be the only active pointer to an object, i.e. you can be sure that nobody is going to observe or mutate the object while you're mutating it.
Now, let's look at the signature of Entity::update:
impl Entity {
pub fn update(&mut self, container: &GameState) {
// ...
}
}
This method takes two parameters: a &mut Entity and a &GameState. But hold on, we can get another reference to self through the &GameState! For example, suppose that self is the first entity. If we do this:
impl Entity {
pub fn update(&mut self, container: &GameState) {
let self_again = &container.entities[0];
// ...
}
}
then self and self_again alias each other (i.e. they refer to the same thing), which is not allowed as per the rules I mentioned above because one of the pointers is a mutable pointer.
What can you do about this?
One option is to remove an entity from the entities vector before calling update on it, then inserting it back after the call. This solves the aliasing problem because we can't get another alias to the entity from the game state. However, removing the entity from the vector and reinserting it are operations with linear complexity (the vector needs to shift all the following items), and if you do it for each entity, then the main update loop runs in quadratic complexity. You can work around that by using a different data structure; this can be as simple as a Vec<Option<Entity>>, where you simply take the Entity from each Option, though you might want to wrap this into a type that hides all None values to external code. A nice consequence is that when an entity has to interact with other entities, it will automatically skip itself when iterating on the entities vector, since it's no longer there!
A variation on the above is to simply take ownership of the whole vector of entities and temporarily replace the game state's vector of entities with an empty one.
impl GameState {
pub fn update(&mut self) {
let mut entities = std::mem::replace(&mut self.entities, vec![]);
for mut t in entities.iter_mut() {
t.update(self);
}
self.entities = entities;
}
}
This has one major downside: Entity::update will not be able to interact with the other entities.
Another option is to wrap each entity in a RefCell.
use std::cell::RefCell;
pub struct GameState {
pub entities: Vec<RefCell<Entity>>,
}
impl GameState {
pub fn update(&mut self) {
for t in self.entities.iter() {
t.borrow_mut().update(self);
}
}
}
By using RefCell, we can avoid retaining a mutable borrow on self. Here, we can use iter instead of iter_mut to iterate on entities. In return, we now need to call borrow_mut to obtain a mutable pointer to the value wrapped in the RefCell.
RefCell essentially performs borrow checking at runtime. This means that you can end up writing code that compiles fine but panics at runtime. For example, if we write Entity::update like this:
impl Entity {
pub fn update(&mut self, container: &GameState) {
for entity in container.entities.iter() {
self.value += entity.borrow().value;
}
}
}
the program will panic:
thread 'main' panicked at 'already mutably borrowed: BorrowError', ../src/libcore/result.rs:788
That's because we end up calling borrow on the entity that we're currently updating, which is still borrowed by the borrow_mut call done in GameState::update. Entity::update doesn't have enough information to know which entity is self, so you would have to use try_borrow or borrow_state (which are both unstable as of Rust 1.12.1) or pass additional data to Entity::update to avoid panics with this approach.
I'm trying to design a struct to carry around a Postgres connection, transaction, and a bunch of prepared statements, and then execute the prepared statements repeatedly. But I'm running into lifetime problems. Here is what I've got:
extern crate postgres;
use postgres::{Connection, TlsMode};
use postgres::transaction::Transaction;
use postgres::stmt::Statement;
pub struct Db<'a> {
conn: Connection,
tx: Transaction<'a>,
insert_user: Statement<'a>,
}
fn make_db(url: &str) -> Db {
let conn = Connection::connect(url, TlsMode::None).unwrap();
let tx = conn.transaction().unwrap();
let insert_user = tx.prepare("INSERT INTO users VALUES ($1)").unwrap();
Db {
conn: conn,
tx: tx,
insert_user: insert_user,
}
}
pub fn main() {
let db = make_db("postgres://paul#localhost/t");
for u in &["foo", "bar"] {
db.insert_user.execute(&[&u]);
}
db.tx.commit().unwrap();
}
Here is the error I'm getting (on Rust 1.15.0 stable):
error: `conn` does not live long enough
--> src/main.rs:15:14
|
15 | let tx = conn.transaction().unwrap();
| ^^^^ does not live long enough
...
22 | }
| - borrowed value only lives until here
|
note: borrowed value must be valid for the anonymous lifetime #1 defined on the body at 13:28...
--> src/main.rs:13:29
|
13 | fn make_db(url: &str) -> Db {
| ^
I've read the Rust book (I've lost count how many times), but I'm not sure how to make progress here. Any suggestions?
EDIT: Thinking about this some more I still don't understand why in principle I can't tell Rust, "conn lives as long as Db does". The issue is with moving conn, but what if I don't move it? I understand why in C you can't return a pointer to stack-allocated memory, e.g.:
#include <stdio.h>
int *build_array() {
int ar[] = {1,2,3};
return ar;
}
int main() {
int *ar = build_array();
printf("%d\n", ar[1]);
}
And I get how that is similar to in Rust returning a &str or returning a vec slice.
But in Rust you can do this:
#[derive(Debug)]
struct S {
ar: Vec<i32>,
}
fn build_array() -> S {
let v = vec![1, 2, 3];
S { ar: v }
}
fn main() {
let s = build_array();
println!("{:?}", s);
}
And my understanding is that Rust is smart enough so that returning S doesn't actually require a move; essentially it is going straight to the caller's stack frame.
So I don't understand why it can't also put Db (including conn) in the caller's stack frame. Then no moves would be required, and tx would never hold an invalid address. I feel like Rust should be able to figure that out. I tried adding a lifetime hint, like this:
pub struct Db<'a> {
conn: Connection<'a>,
tx: Transaction<'a>,
insert_user: Statement<'a>,
}
But that gives an "unexpected lifetime parameter" error. I can accept that Rust can't follow the logic, but I'm curious if there is a reason why in principle it couldn't.
It does seem that putting conn on the heap should solve my problems, but I can't get this to work either:
pub struct Db<'a> {
conn: Box<Connection>,
tx: Transaction<'a>,
insert_user: Statement<'a>,
}
Even with a let conn = Box::new(Connection::connect(...));, Rust still tells me "conn does not live long enough". Is there some way to make this work with Box, or is that a dead end?
EDIT 2: I tried doing this with macros also, to avoid any extra stack frames:
extern crate postgres;
use postgres::{Connection, TlsMode};
use postgres::transaction::Transaction;
use postgres::stmt::Statement;
pub struct Db<'a> {
conn: Connection,
tx: Transaction<'a>,
insert_user: Statement<'a>,
}
macro_rules! make_db {
( $x:expr ) => {
{
let conn = Connection::connect($x, TlsMode::None).unwrap();
let tx = conn.transaction().unwrap();
let insert_user = tx.prepare("INSERT INTO users VALUES ($1)").unwrap();
Db {
conn: conn,
tx: tx,
insert_user: insert_user,
}
}
}
}
pub fn main() {
let db = make_db!("postgres://paul#localhost/t");
for u in &["foo", "bar"] {
db.insert_user.execute(&[&u]);
}
db.tx.commit().unwrap();
}
But that still tells me that conn does not live long enough. It seems that moving it into the struct should really not require any real RAM changes, but Rust still won't let me do it.
Starting with this function:
fn make_db(url: &str) -> Db {
unimplemented!()
}
Due to lifetime elision, this is equivalent to:
fn make_db<'a>(url: &'a str) -> Db<'a> {
unimplemented!()
}
That is, the lifetimes of all the references inside the Db struct must live as long as the string slice passed in. That only makes sense if the struct is holding on to the string slice.
To "solve" that, we can try to separate the lifetimes:
fn make_db<'a, 'b>(url: &'a str) -> Db<'b> {
unimplemented!()
}
Now this makes even less sense because now we are just making up a lifetime. Where is that 'b coming from? What happens if the caller of make_db decides that the concrete lifetime for the generic lifetime parameter 'b should be 'static? This is further explained in Why can't I store a value and a reference to that value in the same struct?, search for "something is really wrong with our creation function".
We also see the part of the question with "Sometimes, I'm not even taking a reference of the value" in the other question, which says in the answer:
the Child instance contains a reference to the Parent that created it,
If we check out the definition for Connection::transaction:
fn transaction<'a>(&'a self) -> Result<Transaction<'a>>
or the definition if you don't believe the docs:
pub struct Transaction<'conn> {
conn: &'conn Connection,
depth: u32,
savepoint_name: Option<String>,
commit: Cell<bool>,
finished: bool,
}
Yup, a Transaction keeps a reference to its parent Connection. Now that we see that Transaction has a reference to Connection we can return to the other question to see how to solve the problem: split apart the structs so that the nesting mirrors the lifetimes.
This was a very long-winded way of saying: no, you cannot create a single structure that contains a database and a transaction of that database due to the implementation of the postgres crate. Presumably the crate is implemented in this fashion for maximum performance.
I don't see why [returning Db<'b>] makes less sense. Normally when a function returns a thing, the thing lives as long as it is assigned to something. Why can't -> Db work the same way?
The entire point of references is that you don't own the referred-to value. You return Db and the caller of make_db would own that, but what owns the thing that Db is referring to? Where did it come from? You cannot return a reference to something local as that would violate all of Rust's safety rules. If you want to transfer ownership, you just do that.
See also
Is there any way to return a reference to a variable created in a function?
Return local String as a slice (&str)
Using the other answer, I put together working code that lets me bundle up the transaction and all the prepared statements, and pass them around together:
extern crate postgres;
use postgres::{Connection, TlsMode};
use postgres::transaction::Transaction;
use postgres::stmt::Statement;
pub struct Db<'a> {
tx: Transaction<'a>,
insert_user: Statement<'a>,
}
fn make_db(conn: &Connection) -> Db {
let tx = conn.transaction().unwrap();
let insert_user = tx.prepare("INSERT INTO users VALUES ($1)").unwrap();
Db {
tx: tx,
insert_user: insert_user,
}
}
pub fn main() {
let conn = Connection::connect("postgres://paul#localhost/t", TlsMode::None).unwrap();
let db = make_db(&conn);
for u in &["foo", "bar"] {
db.insert_user.execute(&[&u]);
}
db.tx.commit().unwrap();
}
As I understand it, Rust wants to guarantee that conn lives as long as db, so by keeping conn outside of the "constructor", the lexical structure ensures that it won't get removed too early.
My struct still doesn't encapsulate conn, which seems too bad to me, but at least it lets me keep everything else together.
My goal is to have a reference counted struct which is referred as a trait in one context and by its concrete type in another. Best explained in code:
#![feature(box_syntax)]
use std::rc::Rc;
use std::cell::RefCell;
trait Employee {
fn be_managed(&mut self);
}
struct Human;
impl Human {
fn be_human(&mut self) {
println!("I'm just a human who needs a mutable self sometimes");
}
}
impl Employee for Human {
fn be_managed(&mut self) {
println!("Off to the salt mines");
}
}
struct Manager {
my_employee: Rc<RefCell<Box<Employee + 'static>>>, //'
}
fn main() {
let mut human1 = Rc::new(RefCell::new(box Human as Box<Employee>));
let manager1 = Manager {
my_employee: human1.clone(), // This works due to cast above
};
manager1.my_employee.borrow_mut().be_managed();
human1.borrow_mut().be_human(); // But we can't be human anymore
let mut human2 = Rc::new(RefCell::new(box Human));
let manager2 = Manager {
my_employee: human2.clone(), // This doesn't work
};
manager2.my_employee.borrow_mut().be_managed();
human2.borrow_mut().be_human();
}
I want the Manager to be able to have any struct implementing the Employee trait as my_employee, but other references should still be able to call other methods on the original object, ie be_human.
Right now I'm getting the following errors from the above code:
src/main.rs:37:25: 37:35 error: type `core::cell::RefMut<'_, Box<Employee>>` does not implement any method in scope named `be_human`
src/main.rs:37 human1.borrow_mut().be_human(); // But we can't be human anymore
^~~~~~~~~~
src/main.rs:44:22: 44:36 error: mismatched types:
expected `alloc::rc::Rc<core::cell::RefCell<Box<Employee + 'static>>>`,
found `alloc::rc::Rc<core::cell::RefCell<Box<Human>>>`
(expected trait Employee,
found struct `Human`) [E0308]
src/main.rs:44 my_employee: human2.clone(), // This doesn't work
^~~~~~~~~~~~~~
What's the right approach in this situation?
Disclaimer: in this answer I will assume that you are willingly NOT using an enum because you want Employee to be open.
This issue comes up in about every single language that uses dynamic polymorphism, and the traditional answer is the Visitor Pattern.
It is not exactly ideal, though, because of the dependencies it introduces, so if necessary you can use the Acyclic Visitor Pattern; however I advise that you start with a bare bone visitor before delving further.
trait EmployeeVisitor {
fn visit_employee(&self, e: &Employee);
fn visit_human(&self, h: &Human);
}
trait Employee {
fn accept(&self, v: &EmployeeVisitor) {
v.visit_employee(self);
}
}
impl Employee for Human {
fn accept(&self, v: &EmployeeVisitor) {
v.visit_human(self);
}
}
This is the traditional "every problem can be solved with a layer of indirection", and it incurs the traditional issue of bringing another layer of indirection.