Lifetime Issue with Associated Types - rust

I've been pulling my hair out over the last week due to this incredibly annoying issue with lifetimes.
The problem occurs when I try to put a reference to a Buffer inside a DataSource, which is then referenced to a DrawCommand. I'm getting the error: vertex_data_source does not live long enough.
src/main.rs:65:23: 65:41 error:
src/main.rs:65 data_source: &vertex_data_source
^~~~~~~~~~~~~~~~~~
src/main.rs:60:51: 67:2 note: reference must be valid for the block suffix following statement 3 at 60:50...
src/main.rs:60 let vertices = VertexAttributes::new(&buffer);
src/main.rs:61
src/main.rs:62 let vertex_data_source = factory.create_data_source(vertices);
src/main.rs:63
src/main.rs:64 let command: DrawCommand<ResourcesImpl> = DrawCommand {
src/main.rs:65 data_source: &vertex_data_source
...
src/main.rs:62:67: 67:2 note: ...but borrowed value is only valid for the block suffix following statement 4 at 62:66
src/main.rs:62 let vertex_data_source = factory.create_data_source(vertices);
src/main.rs:63
src/main.rs:64 let command: DrawCommand<ResourcesImpl> = DrawCommand {
src/main.rs:65 data_source: &vertex_data_source
src/main.rs:66 };
src/main.rs:67 }
It says vertex_data_source has to be valid for the block suffix following statement 3 at line 60. My interpretation of that error is that vertex_data_source should be defined before line 60. But to create the vertex_data_source in the first place I need access to those VertexAttributes on line 60, so I can't just swap the order around.
I feel like all the 'a lifetimes sprinkled over my code need to be split into 2 or maybe just removed, however I've tried every combination that seemed sensible and I'm not out of ideas.
Below is a greatly simplified example of my code that demonstrates the problem. I would really appreciate a sanity check and hopefully a fresh mind might be able to spot the issue. (every time before a few days of fiddling has produced a fix but this time I'm stumped).
use std::cell::RefCell;
use std::marker::PhantomData;
pub struct DrawCommand<'a, R: Resources<'a>> {
pub data_source: &'a R::DataSource
}
pub trait Resources<'a> {
type DataSource: 'a;
type Buffer: 'a;
}
pub struct DataSource<'a> {
id: u32,
attributes: Vec<VertexAttributes<'a, ResourcesImpl<'a>>>,
current_element_array_buffer_binding: RefCell<Option<Buffer<'a>>>
}
pub struct Buffer<'a> {
context: &'a GraphicsContextImpl
}
pub struct GraphicsContextImpl;
pub struct ResourcesImpl<'a> {
phantom: PhantomData<&'a u32> // 'a is the lifetime of the context reference
}
impl<'a> Resources<'a> for ResourcesImpl<'a> {
type Buffer = Buffer<'a>;
type DataSource = DataSource<'a>;
}
struct Factory<'a> {
context: &'a GraphicsContextImpl
}
impl<'a> Factory<'a> {
/// Creates a buffer
fn create_buffer<T>(&self) -> Buffer<'a> {
Buffer {
context: self.context
}
}
fn create_data_source(&self, attributes: Vec<VertexAttributes<'a, ResourcesImpl<'a>>>) -> DataSource<'a> {
DataSource {
id: 0,
attributes: attributes,
current_element_array_buffer_binding: RefCell::new(None)
}
}
}
fn main() {
let context = GraphicsContextImpl;
let factory = Factory {
context: &context
};
let buffer = factory.create_buffer::<u32>();
let vertices = VertexAttributes::new(&buffer);
let vertex_data_source = factory.create_data_source(vec!(vertices));
let command: DrawCommand<ResourcesImpl> = DrawCommand {
data_source: &vertex_data_source
};
}
pub struct VertexAttributes<'a, R: Resources<'a>> {
pub buffer: &'a R::Buffer,
}
impl<'a, R: Resources<'a>> VertexAttributes<'a, R> {
pub fn new(buffer: &'a R::Buffer) -> VertexAttributes<'a, R> {
VertexAttributes {
buffer: buffer
}
}
}
Thanks very much in advance.
EDIT:
I've updated the code to better reflect my actual implementation.
By the way - replacing this:
let vertex_data_source = factory.create_data_source(vec!(vertices));
With this:
let vertex_data_source = DataSource {
id: 0,
attributes: vec!(vertices),
current_element_array_buffer_binding: RefCell::new(None)
};
Doesn't solve the issue.

This allows your example to compile:
pub struct DrawCommand<'a : 'b, 'b, R: Resources<'a>> {
pub data_source: &'b R::DataSource
}
However, I'm finding it extremely difficult to create a more minimal example. As best I can determine, you have an issue because you are declaring that you will hold a reference to an item that itself has a reference, and that those two references need to have a common lifetime ('a). Through some combination of the other lifetimes, this is actually impossible.
Adding a second lifetime allows the reference to the DataSource to differ from the reference of the DataSource itself.
I'm still going to try to create a more minified example.

Related

How can I add an item to a struct member that is an `Option<Vec<...>>`?

How can I add an item to a struct member that is an Option<Vec<...>>?
In other words, what do I need to to to make add_foo below work?
struct Foo {
value: u32,
}
struct Bar {
foos: Option<Vec<Foo>>,
}
impl Bar {
fn add_foo(&mut self, foo: Foo) {
if self.foos.is_none() {
self.foos = Some(vec![]);
}
self.foos.unwrap().push(foo);
// ^^^^^^^^^ move occurs because `self.foos` has type `Option<Vec<Foo>>`, which does not implement the `Copy` trait
}
}
fn main() {
let mut bar = Bar{foos: None};
bar.add_foo(Foo{value:42});
}
The error message suggests adding as_ref() but that doesn't help:
self.foos.as_ref().unwrap().push(foo);
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cannot borrow as mutable
I though there might be something as as_mutref() but there is no such thing. Since I already hold a mutable reference to the Bar struct, I would expect to be able to change the foos field in the Bar struct.
Apologies if my terminology is off; still getting used to the whole Rust ownership concept.
Something like the following might be what you're looking for:
fn add_foo(&mut self, foo: Foo) {
if let Some(foos) = self.foos.as_mut() {
// foos has type: &mut Vec<Foo>
foos.push(foo);
} else {
self.foos = Some(vec![foo]);
}
}
In general, using an if let or match or some other destructuring syntax is considered more idiomatic than using a is_some() check followed by an unwrap(). At the very least it saves on a comparison, but more importantly is harder to make accidentally panic.

How best to deal with struct field that can change types

I'm working with a library that uses Rust types to keep track of state. As a simplified example, say you have two structs:
struct FirstStruct {}
struct SecondStruct {}
impl FirstStruct {
pub fn new() -> FirstStruct {
FirstStruct {}
}
pub fn second(self) -> SecondStruct {
SecondStruct {}
}
// configuration methods defined in this struct
}
impl SecondStruct {
pub fn print_something(&self) {
println!("something");
}
pub fn first(self) -> FirstStruct {
FirstStruct {}
}
}
And to actually use these structs you usually follow a pattern like so, after printing you may stay in second state or go back to first state depending on how you're using the library:
fn main() {
let first = FirstStruct::new();
let second = first.second(); // consumes first
second.print_something();
// go back to default state
let _first = second.first();
}
I want to create my own struct that handles the state changes internally and simplifies the interface. This also lets me have a single mutable reference around that I can pass to other functions and call the print method. Using it should look something like this:
fn main() {
let mut combined = CombinedStruct::new(FirstStruct::new());
combined.print();
}
I've come up with the following solution that works, at least in this simplified example:
enum StructState {
First(FirstStruct),
Second(SecondStruct),
}
struct CombinedStruct {
state: Option<StructState>,
}
impl CombinedStruct {
pub fn new(first: FirstStruct) -> CombinedStruct {
CombinedStruct {
state: Some(StructState::First(first)),
}
}
pub fn print(&mut self) {
let s = match self.state.take() {
Some(s) => match s {
StructState::First(first) => first.second(),
StructState::Second(second) => second,
},
None => panic!(),
};
s.print_something();
// If I forget to do this, then I lose access to my struct
// and next call will panic
self.state = Some(StructState::First(s.first()));
}
}
I'm still pretty new to Rust but this doesn't look right to me. I'm not sure if there's a concept I'm missing that could simplify this or if this solution could lead to ownership problems as my application gets more complicated. Is there a better way to do this?
Playground link
I once had a similar problem and went basically with your solution, but I avoided the Option.
I.e. I basically kept your
enum StructState {
First(FirstStruct),
Second(SecondStruct),
}
If an operation tries to convert a FirstStruct to a SecondStruct, I introduced a function try_to_second roughly as follows:
impl StructState {
fn try_to_second(self) -> Result<SecondState, StructState> {
/// implementation
}
}
In this case, an Err indicates that the StructState has not been converted to SecondStruct and preserves the status quo, while an Ok value indicates successfull conversion.
As an alternative, you could try to define try_to_second on FirstStruct:
impl FirstStruct {
fn try_to_second(self) -> Result<FirstStruct, SecondStruct> {
/// implementation
}
}
Again, Err/Ok denote failure/success, but in this case, you have more concrete information encoded in the type.

Correct way to implement container-element relationship in idiomatic Rust

I know why Rust doesn't like my code. However, I don't know what would be the idiomatic Rust approach to the problem.
I'm a C# programmer, and while I feel I understand Rust's system, I think my "old" approach to some problems don't work in Rust at all.
This code reproduces the problem I'm having, and it probably doesn't look like idiomatic Rust (or maybe it doesn't even look good in C# as well):
//a "global" container for the elements and some extra data
struct Container {
elements: Vec<Element>,
global_contextual_data: i32,
//... more contextual data fields
}
impl Container {
//this just calculates whatever I need based on the contextual data
fn calculate_contextual_data(&self) -> i32 {
//This function will end up using the elements vector and the other fields as well,
//and will do some wacky maths with it.
//That's why I currently have the elements stored in the container
}
}
struct Element {
element_data: i32,
//other fields
}
impl Element {
//I need to take a mutable reference to update element_data,
//and a reference to the container to calculate something that needs
//this global contextual data... including the other elements, as previously stated
fn update_element_data(&mut self, some_data: i32, container: &Container) {
self.element_data *= some_data + container.calculate_contextual_data() //do whatever maths I need
}
}
fn main(){
//let it be mutable so I can assign the elements later
let mut container = Container {
elements: vec![],
global_contextual_data: 1
};
//build a vector of elements
let elements = vec![
Element {
element_data: 5
},
Element {
element_data: 7
}
];
//this works
container.elements = elements;
//and this works, but container is now borrowed as mutable
for elem in container.elements.iter_mut() {
elem.element_data += 1; //and while this works
let some_data = 2;
//i can't borrow it as immutable here and pass to the other function
elem.update_element_data(some_data, &container);
}
}
I understand why elem.update_element_data(some_data, &container); won't work: I'm already borrowing it as mutable when I call iter_mut. Maybe each element should have a reference to the container? But then wouldn't I have more opportunities to break at borrow-checking?
I don't think it's possible to bring my old approach to this new system. Maybe I need to rewrite the whole thing. Can someone point me to the right direction? I've just started programming in Rust, and while the ownership system is making some sort of sense to me, the code I should write "around" it is still not that clear.
I came across this question:
What's the Rust way to modify a structure within nested loops? which gave me insight into my problem.
I revisited the problem and boiled the problem down to the sharing of the vector by borrowing for writes and reads at the same time. This is just forbidden by Rust. I don't want to circumvent the borrow checker using unsafe. I was wondering, though, how much data should I copy?
My Element, which in reality is the entity of a game (I'm simulating a clicker game) has both mutable and immutable properties, which I broke apart.
struct Entity {
type: EntityType,
starting_price: f64,
...
...
status: Cell<EntityStatus>
}
Every time I need to change the status of an entity, I need to call get and set methods on the status field. EntityStatus derives Clone, Copy.
I could even put the fields directly on the struct and have them all be Cells but then it would be cumbersome to work with them (lots of calls to get and set), so I went for the more aesthetically pleasant approach.
By allowing myself to copy the status, edit and set it back, I could borrow the array immutably twice (.iter() instead of .iter_mut()).
I was afraid that the performance would be bad due to the copying, but in reality it was pretty good once I compiled with opt-level=3. If it gets problematic, I might change the fields to be Cells or come up with another approach.
Just do the computation outside:
#[derive(Debug)]
struct Container {
elements: Vec<Element>
}
impl Container {
fn compute(&self) -> i32 {
return 42;
}
fn len(&self) -> usize {
return self.elements.len();
}
fn at_mut(&mut self, index: usize) -> &mut Element {
return &mut self.elements[index];
}
}
#[derive(Debug)]
struct Element {
data: i32
}
impl Element {
fn update(&mut self, data: i32, computed_data: i32) {
self.data *= data + computed_data;
}
}
fn main() {
let mut container = Container {
elements: vec![Element {data: 1}, Element {data: 3}]
};
println!("{:?}", container);
for i in 0..container.len() {
let computed_data = container.compute();
container.at_mut(i).update(2, computed_data);
}
println!("{:?}", container);
}
Another option is to add an update_element to your container:
#[derive(Debug)]
struct Container {
elements: Vec<Element>
}
impl Container {
fn compute(&self) -> i32 {
let sum = self.elements.iter().map(|e| {e.data}).reduce(|a, b| {a + b});
return sum.unwrap_or(0);
}
fn len(&self) -> usize {
return self.elements.len();
}
fn at_mut(&mut self, index: usize) -> &mut Element {
return &mut self.elements[index];
}
fn update_element(&mut self, index: usize, data: i32) {
let computed_data = self.compute();
self.at_mut(index).update(data, computed_data);
}
}
#[derive(Debug)]
struct Element {
data: i32
}
impl Element {
fn update(&mut self, data: i32, computed_data: i32) {
self.data *= data + computed_data;
}
}
fn main() {
let mut container = Container {
elements: vec![Element {data: 1}, Element {data: 3}]
};
println!("{:?}", container);
for i in 0..container.len() {
let computed_data = container.compute();
container.at_mut(i).update(2, computed_data);
}
println!("{:?}", container);
for i in 0..container.len() {
container.update_element(i, 2);
}
println!("{:?}", container);
}
Try it!

In memory database design

I am trying to create an in-memory database using HashMap. I have a struct Person:
struct Person {
id: i64,
name: String,
}
impl Person {
pub fn new(id: i64, name: &str) -> Person {
Person {
id: id,
name: name.to_string(),
}
}
pub fn set_name(&mut self, name: &str) {
self.name = name.to_string();
}
}
And I have struct Database:
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::Mutex;
struct Database {
db: Arc<Mutex<HashMap<i64, Person>>>,
}
impl Database {
pub fn new() -> Database {
Database {
db: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn add_person(&mut self, id: i64, person: Person) {
self.db.lock().unwrap().insert(id, person);
}
pub fn get_person(&self, id: i64) -> Option<&mut Person> {
self.db.lock().unwrap().get_mut(&id)
}
}
And code to use this database:
let mut db = Database::new();
db.add_person(1, Person::new(1, "Bob"));
I want to change person's name:
let mut person = db.get_person(1).unwrap();
person.set_name("Bill");
The complete code in the Rust playground.
When compiling, I get a problem with Rust lifetimes:
error[E0597]: borrowed value does not live long enough
--> src/main.rs:39:9
|
39 | self.db.lock().unwrap().get_mut(&id)
| ^^^^^^^^^^^^^^^^^^^^^^^ temporary value does not live long enough
40 | }
| - temporary value only lives until here
|
note: borrowed value must be valid for the anonymous lifetime #1 defined on the method body at 38:5...
--> src/main.rs:38:5
|
38 | / pub fn get_person(&self, id: i64) -> Option<&mut Person> {
39 | | self.db.lock().unwrap().get_mut(&id)
40 | | }
| |_____^
How to implement this approach?
The compiler rejects your code because it violates the correctness model enforced by Rust and could cause crashes. For one, if get_person() were allowed to compile, one might call it from two threads and modify the underlying object without the protection of the mutex, causing data races on the String object inside. Worse, one could wreak havoc even in a single-threaded scenario by doing something like:
let mut ref1 = db.get_person(1).unwrap();
let mut ref2 = db.get_person(1).unwrap();
// ERROR - two mutable references to the same object!
let vec: Vec<Person> = vec![];
vec.push(*ref1); // move referenced object to the vector
println!(*ref2); // CRASH - object already moved
To correct the code, you need to adjust your design to satisfy the following constraints:
No reference can be allowed to outlive the referred-to object;
During the lifetime of a mutable reference, no other reference (mutable or immutable) to the object may exist..
The add_person method already complies with both rules because it eats the object you pass it, moving it to the database.
What if we modified get_person() to return an immutable reference?
pub fn get_person(&self, id: i64) -> Option<&Person> {
self.db.lock().unwrap().get(&id)
}
Even this seemingly innocent version still doesn't compile! That is because it violates the first rule. Rust cannot statically prove that the reference will not outlive the database itself, since the database is allocated on the heap and reference-counted, so it can be dropped at any time. But even if it were possible to somehow explicitly declare the lifetime of the reference to one that provably couldn't outlive the database, retaining the reference after unlocking the mutex would allow data races. There is simply no way to implement get_person() and still retain thread safety.
A thread-safe implementation of a read can opt to return a copy of the data. Person can implement the clone() method and get_person() can invoke it like this:
#[derive(Clone)]
struct Person {
id: i64,
name: String
}
// ...
pub fn get_person(&self, id: i64) -> Option<Person> {
self.db.lock().unwrap().get(&id).cloned()
}
This kind of change won't work for the other use case of get_person(), where the method is used for the express purpose of obtaining a mutable reference to change the person in the database. Obtaining a mutable reference to a shared resource violates the second rule and could lead to crashes as shown above. There are several ways to make it safe. One is by providing a proxy in the database for setting each Person field:
pub fn set_person_name(&self, id: i64, new_name: String) -> bool {
match self.db.lock().unwrap().get_mut(&id) {
Some(mut person) => {
person.name = new_name;
true
}
None => false
}
}
As the number of fields on Person grows, this would quickly get tedious. It could also get slow, as a separate mutex lock would have to be acquired for each access.
There is fortunately a better way to implement modification of the entry. Remember that using a mutable reference violates the rules unless Rust can prove that the reference won't "escape" the block where it is being used. This can be ensured by inverting the control - instead of a get_person() that returns the mutable reference, we can introduce a modify_person() that passes the mutable reference to a callable, which can do whatever it likes with it. For example:
pub fn modify_person<F>(&self, id: i64, f: F) where F: FnOnce(Option<&mut Person>) {
f(self.db.lock().unwrap().get_mut(&id))
}
The usage would look like this:
fn main() {
let mut db = Database::new();
db.add_person(1, Person::new(1, "Bob"));
assert!(db.get_person(1).unwrap().name == "Bob");
db.modify_person(1, |person| {
person.unwrap().set_name("Bill");
});
}
Finally, if you're worried about the performance of get_person() cloning Person for the sole reason of inspecting it, it is trivial to create an immutable version of modify_person that serves as a non-copying alternative to get_person():
pub fn read_person<F, R>(&self, id: i64, f: F) -> R
where F: FnOnce(Option<&Person>) -> R {
f(self.db.lock().unwrap().get(&id))
}
Besides taking a shared reference to Person, read_person is also allowing the closure to return a value if it chooses, typically something it picks up from the object it receives. Its usage would be similar to the usage of modify_person, with the added possibility of returning a value:
// if Person had an "age" field, we could obtain it like this:
let person_age = db.read_person(1, |person| person.unwrap().age);
// equivalent to the copying definition of db.get_person():
let person_copy = db.read_person(1, |person| person.cloned());
This post use the pattern cited as "inversion of control" in the well explained answer and just add only sugar for demonstrating another api for an in-memory db.
With a macro rule it is possible to expose a db client api like that:
fn main() {
let db = Database::new();
let person_id = 1234;
// probably not the best design choice to duplicate the person_id,
// for the purpose here is not important
db.add_person(person_id, Person::new(person_id, "Bob"));
db_update!(db #person_id => set_name("Gambadilegno"));
println!("your new name is {}", db.get_person(person_id).unwrap().name);
}
My opinionated macro has the format:
<database_instance> #<object_key> => <method_name>(<args>)
Below the macro implementation and the full demo code:
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::Mutex;
macro_rules! db_update {
($db:ident # $id:expr => $meth:tt($($args:tt)*)) => {
$db.modify_person($id, |person| {
person.unwrap().$meth($($args)*);
});
};
}
#[derive(Clone)]
struct Person {
id: u64,
name: String,
}
impl Person {
pub fn new(id: u64, name: &str) -> Person {
Person {
id: id,
name: name.to_string(),
}
}
fn set_name(&mut self, value: &str) {
self.name = value.to_string();
}
}
struct Database {
db: Arc<Mutex<HashMap<u64, Person>>>, // access from different threads
}
impl Database {
pub fn new() -> Database {
Database {
db: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn add_person(&self, id: u64, person: Person) {
self.db.lock().unwrap().insert(id, person);
}
pub fn modify_person<F>(&self, id: u64, f: F)
where
F: FnOnce(Option<&mut Person>),
{
f(self.db.lock().unwrap().get_mut(&id));
}
pub fn get_person(&self, id: u64) -> Option<Person> {
self.db.lock().unwrap().get(&id).cloned()
}
}

Create and assign reference data from within a struct method

I have abstracted my problem inside the following code:
struct List<'a> {
attr: &'a String
}
impl<'a> List<'a> {
fn new() -> List<'a> {
let my_attr = "Something";
List {
attr: &my_attr.to_string()
}
}
}
fn main() {
List::new();
}
It yields some notices and fails compiling claiming that the borrowed value (my_attr) doesn't live long enough.
It makes sense, so for instance the following does indeed compile:
struct List<'a> {
attr: &'a String
}
impl<'a> List<'a> {
fn new(my_attr: &'a String) -> List<'a> {
List {
attr: my_attr
}
}
}
fn main() {
let my_attr = "Something".to_string();
List::new(&my_attr);
}
However, I like the first form more, especially from an encapsulation stand point.
Is it possible to create and also assign a reference to a value from within the new method (as per the failing example)?
The issue here is that a &'a String is a borrowed value - that is, it is a reference to a value that is stored elsewhere. On your first example, that "elsewhere" is the new function stack, and on the second it's main's stack.
You don't get an error on the second case because main's lifetime is bigger than your List's lifetime - that is, the List will die before main finishes. That is not the case of your first scenario, where the List is returned by new, and thus the reference is invalid.
The solution here is to make List own its String, instead of just taking a reference to it. This way, List's lifetime isn't linked to anything else.
struct List {
attr: String,
}
impl List {
fn new() -> List {
let my_attr = "Something";
List {
attr: my_attr.to_string()
}
}
}

Resources