I have a struct + implementation in Rust that I return to Python. This object can also be passed back to Rust for further work. (In my actual code, I'm using a HashMap<String, MyStruct>, but even just using a struct directly seems to cause the same issues, so my example uses struct Person for simplicity.)
It appears that I need to impl FromPyObject for Person, but Rust can't find PyAny's downcast method
#[pyclass]
struct Person {
name: String,
age: u8,
height_cm: f32,
}
impl pyo3::FromPyObject<'_> for Person {
fn extract(any: &PyAny) -> PyResult<Self> {
Ok(any.downcast().unwrap())
^^^^^^^^ method not found in `&pyo3::types::any::PyAny`
}
}
#[pyfunction]
fn make_person() -> PyResult<Person> {
Ok(Person {
name: "Bilbo Baggins".to_string(),
age: 51,
height_cm: 91.44,
})
}
#[pyfunction]
fn person_info(py:Python, p: PyObject) -> PyResult<()> {
let p : Person = p.extract(py)?;
println!("{} is {} years old", p.name, p.age);
Ok(())
}
Is this the right way to pass a Rust object from Python back into Rust? If so, what's the right way to use PyAny here?
Related
There is a basic struct implementation that gives me this warning. I want to create methods on a struct. As far as I understand (please correct me if I am wrong), the difference between methods and associative functions is the &self parameter. But for some reason, rust is taking those as associative functions:
struct User {
name: String,
date: u64,
}
impl User {
fn display(&self) {
println!("{}", self.name);
}
}
Here is the warning which I am getting:
associated function `display` is never usedrustc(dead_code)
According to this resource (https://doc.rust-lang.org/book/ch05-03-method-syntax.html), I think this is the correct way to write methods.
Methods are associated functions, they are associated to the Type. You can even say var.method() is just syntax sugar for Type::method(&var), as they do the exact same thing.
Also, while there's nothing wrong with your implementation, for common functions like display, it is more idiomatic to implement the Display trait.
struct User {
name: String,
date: u64,
}
use std::fmt::{Display, Formatter};
impl Display for User {
fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), std::fmt::Error> {
write!(f, "{}", self.name)
}
}
Once this trait is implemented, you can print User like this, also you get to_string() for free
let user = User { /* init */ };
println!("{}", user);
let user_s: String = user.to_string();
If you want to know more about traits, checkout the trait section in the rust book
I have an enum with a String:
enum MyLovelyEnum {
Thing(String),
}
For tests, I would like to be able to pass in a &'static str to avoid MyLovelyEnum::Thing("abc".to_string) over and over.
I found that you can do this nicely with structs with a constructor:
// From: https://hermanradtke.com/2015/05/06/creating-a-rust-function-that-accepts-string-or-str.html
struct Person {
name: String,
}
impl Person {
fn new<S: Into<String>>(name: S) -> Person {
Person { name: name.into() }
}
}
fn main() {
let person = Person::new("Herman");
let person = Person::new("Herman".to_string());
}
I know I can use lifetimes or Cow as described in What's the best practice for str/String values in Rust enums? or I can make my own function.
Is there something close to the example in the blog post for enums? e.g.
// this is the kind of thing I am after but this specifically is not correct syntax
enum MyLovelyEnum {
Thing<S: Into<String>>(S)
}
You can create a generic enum:
enum MyLovelyEnum<S>
where
S: Into<String>,
{
Thing(S),
}
MyLovelyEnum::Thing("a");
MyLovelyEnum::Thing("b".to_string());
I likely wouldn't do that in my code, instead opting to create a constructor, much like the blog post you linked:
enum MyLovelyEnum {
Thing(String),
}
impl MyLovelyEnum {
fn thing(s: impl Into<String>) -> Self {
MyLovelyEnum::Thing(s.into())
}
}
MyLovelyEnum::thing("a");
MyLovelyEnum::thing("b".to_string());
This question already has answers here:
How do I return a reference to something inside a RefCell without breaking encapsulation?
(3 answers)
Closed 2 years ago.
I'm creating a lazily loading struct:
pub struct LazyData<T, U:FnOnce() -> T> {
data: RefCell<Option<T>>,
generator: RefCell<Option<U>>,
}
impl<T,U:FnOnce() -> T> LazyData<T,U> {
pub fn new(generator:U) -> LazyData<T,U> {
LazyData {
data: RefCell::new(None),
generator: RefCell::new(Some(generator)),
}
}
...
}
I've tried to implement a get method that will return the lazily loaded data:
pub fn init(&self) {
if self.data.borrow().is_none() {
let generator = self.generator.replace(None).unwrap();
let _none = self.data.replace(Some(generator()));
};
}
pub fn get(&self) -> &T {
self.init();
self.data.borrow().as_ref().unwrap()
}
But the compiler complains that I'm returning a value referencing data owned by the current function. This error makes sense to me, but I'm not sure how to work around it.
Here's a link to a full example.
Found a workaround here! I can use Ref::map to get to the inner data:
pub fn get(&self) -> Ref<T> {
self.init();
Ref::map(self.data.borrow(), |borrow| {
borrow.as_ref().unwrap()
})
}
This method now returns Ref instead of &T, but I think that should be fine.
I am trying to create an in-memory database using HashMap. I have a struct Person:
struct Person {
id: i64,
name: String,
}
impl Person {
pub fn new(id: i64, name: &str) -> Person {
Person {
id: id,
name: name.to_string(),
}
}
pub fn set_name(&mut self, name: &str) {
self.name = name.to_string();
}
}
And I have struct Database:
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::Mutex;
struct Database {
db: Arc<Mutex<HashMap<i64, Person>>>,
}
impl Database {
pub fn new() -> Database {
Database {
db: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn add_person(&mut self, id: i64, person: Person) {
self.db.lock().unwrap().insert(id, person);
}
pub fn get_person(&self, id: i64) -> Option<&mut Person> {
self.db.lock().unwrap().get_mut(&id)
}
}
And code to use this database:
let mut db = Database::new();
db.add_person(1, Person::new(1, "Bob"));
I want to change person's name:
let mut person = db.get_person(1).unwrap();
person.set_name("Bill");
The complete code in the Rust playground.
When compiling, I get a problem with Rust lifetimes:
error[E0597]: borrowed value does not live long enough
--> src/main.rs:39:9
|
39 | self.db.lock().unwrap().get_mut(&id)
| ^^^^^^^^^^^^^^^^^^^^^^^ temporary value does not live long enough
40 | }
| - temporary value only lives until here
|
note: borrowed value must be valid for the anonymous lifetime #1 defined on the method body at 38:5...
--> src/main.rs:38:5
|
38 | / pub fn get_person(&self, id: i64) -> Option<&mut Person> {
39 | | self.db.lock().unwrap().get_mut(&id)
40 | | }
| |_____^
How to implement this approach?
The compiler rejects your code because it violates the correctness model enforced by Rust and could cause crashes. For one, if get_person() were allowed to compile, one might call it from two threads and modify the underlying object without the protection of the mutex, causing data races on the String object inside. Worse, one could wreak havoc even in a single-threaded scenario by doing something like:
let mut ref1 = db.get_person(1).unwrap();
let mut ref2 = db.get_person(1).unwrap();
// ERROR - two mutable references to the same object!
let vec: Vec<Person> = vec![];
vec.push(*ref1); // move referenced object to the vector
println!(*ref2); // CRASH - object already moved
To correct the code, you need to adjust your design to satisfy the following constraints:
No reference can be allowed to outlive the referred-to object;
During the lifetime of a mutable reference, no other reference (mutable or immutable) to the object may exist..
The add_person method already complies with both rules because it eats the object you pass it, moving it to the database.
What if we modified get_person() to return an immutable reference?
pub fn get_person(&self, id: i64) -> Option<&Person> {
self.db.lock().unwrap().get(&id)
}
Even this seemingly innocent version still doesn't compile! That is because it violates the first rule. Rust cannot statically prove that the reference will not outlive the database itself, since the database is allocated on the heap and reference-counted, so it can be dropped at any time. But even if it were possible to somehow explicitly declare the lifetime of the reference to one that provably couldn't outlive the database, retaining the reference after unlocking the mutex would allow data races. There is simply no way to implement get_person() and still retain thread safety.
A thread-safe implementation of a read can opt to return a copy of the data. Person can implement the clone() method and get_person() can invoke it like this:
#[derive(Clone)]
struct Person {
id: i64,
name: String
}
// ...
pub fn get_person(&self, id: i64) -> Option<Person> {
self.db.lock().unwrap().get(&id).cloned()
}
This kind of change won't work for the other use case of get_person(), where the method is used for the express purpose of obtaining a mutable reference to change the person in the database. Obtaining a mutable reference to a shared resource violates the second rule and could lead to crashes as shown above. There are several ways to make it safe. One is by providing a proxy in the database for setting each Person field:
pub fn set_person_name(&self, id: i64, new_name: String) -> bool {
match self.db.lock().unwrap().get_mut(&id) {
Some(mut person) => {
person.name = new_name;
true
}
None => false
}
}
As the number of fields on Person grows, this would quickly get tedious. It could also get slow, as a separate mutex lock would have to be acquired for each access.
There is fortunately a better way to implement modification of the entry. Remember that using a mutable reference violates the rules unless Rust can prove that the reference won't "escape" the block where it is being used. This can be ensured by inverting the control - instead of a get_person() that returns the mutable reference, we can introduce a modify_person() that passes the mutable reference to a callable, which can do whatever it likes with it. For example:
pub fn modify_person<F>(&self, id: i64, f: F) where F: FnOnce(Option<&mut Person>) {
f(self.db.lock().unwrap().get_mut(&id))
}
The usage would look like this:
fn main() {
let mut db = Database::new();
db.add_person(1, Person::new(1, "Bob"));
assert!(db.get_person(1).unwrap().name == "Bob");
db.modify_person(1, |person| {
person.unwrap().set_name("Bill");
});
}
Finally, if you're worried about the performance of get_person() cloning Person for the sole reason of inspecting it, it is trivial to create an immutable version of modify_person that serves as a non-copying alternative to get_person():
pub fn read_person<F, R>(&self, id: i64, f: F) -> R
where F: FnOnce(Option<&Person>) -> R {
f(self.db.lock().unwrap().get(&id))
}
Besides taking a shared reference to Person, read_person is also allowing the closure to return a value if it chooses, typically something it picks up from the object it receives. Its usage would be similar to the usage of modify_person, with the added possibility of returning a value:
// if Person had an "age" field, we could obtain it like this:
let person_age = db.read_person(1, |person| person.unwrap().age);
// equivalent to the copying definition of db.get_person():
let person_copy = db.read_person(1, |person| person.cloned());
This post use the pattern cited as "inversion of control" in the well explained answer and just add only sugar for demonstrating another api for an in-memory db.
With a macro rule it is possible to expose a db client api like that:
fn main() {
let db = Database::new();
let person_id = 1234;
// probably not the best design choice to duplicate the person_id,
// for the purpose here is not important
db.add_person(person_id, Person::new(person_id, "Bob"));
db_update!(db #person_id => set_name("Gambadilegno"));
println!("your new name is {}", db.get_person(person_id).unwrap().name);
}
My opinionated macro has the format:
<database_instance> #<object_key> => <method_name>(<args>)
Below the macro implementation and the full demo code:
use std::collections::HashMap;
use std::sync::Arc;
use std::sync::Mutex;
macro_rules! db_update {
($db:ident # $id:expr => $meth:tt($($args:tt)*)) => {
$db.modify_person($id, |person| {
person.unwrap().$meth($($args)*);
});
};
}
#[derive(Clone)]
struct Person {
id: u64,
name: String,
}
impl Person {
pub fn new(id: u64, name: &str) -> Person {
Person {
id: id,
name: name.to_string(),
}
}
fn set_name(&mut self, value: &str) {
self.name = value.to_string();
}
}
struct Database {
db: Arc<Mutex<HashMap<u64, Person>>>, // access from different threads
}
impl Database {
pub fn new() -> Database {
Database {
db: Arc::new(Mutex::new(HashMap::new())),
}
}
pub fn add_person(&self, id: u64, person: Person) {
self.db.lock().unwrap().insert(id, person);
}
pub fn modify_person<F>(&self, id: u64, f: F)
where
F: FnOnce(Option<&mut Person>),
{
f(self.db.lock().unwrap().get_mut(&id));
}
pub fn get_person(&self, id: u64) -> Option<Person> {
self.db.lock().unwrap().get(&id).cloned()
}
}
I've been pulling my hair out over the last week due to this incredibly annoying issue with lifetimes.
The problem occurs when I try to put a reference to a Buffer inside a DataSource, which is then referenced to a DrawCommand. I'm getting the error: vertex_data_source does not live long enough.
src/main.rs:65:23: 65:41 error:
src/main.rs:65 data_source: &vertex_data_source
^~~~~~~~~~~~~~~~~~
src/main.rs:60:51: 67:2 note: reference must be valid for the block suffix following statement 3 at 60:50...
src/main.rs:60 let vertices = VertexAttributes::new(&buffer);
src/main.rs:61
src/main.rs:62 let vertex_data_source = factory.create_data_source(vertices);
src/main.rs:63
src/main.rs:64 let command: DrawCommand<ResourcesImpl> = DrawCommand {
src/main.rs:65 data_source: &vertex_data_source
...
src/main.rs:62:67: 67:2 note: ...but borrowed value is only valid for the block suffix following statement 4 at 62:66
src/main.rs:62 let vertex_data_source = factory.create_data_source(vertices);
src/main.rs:63
src/main.rs:64 let command: DrawCommand<ResourcesImpl> = DrawCommand {
src/main.rs:65 data_source: &vertex_data_source
src/main.rs:66 };
src/main.rs:67 }
It says vertex_data_source has to be valid for the block suffix following statement 3 at line 60. My interpretation of that error is that vertex_data_source should be defined before line 60. But to create the vertex_data_source in the first place I need access to those VertexAttributes on line 60, so I can't just swap the order around.
I feel like all the 'a lifetimes sprinkled over my code need to be split into 2 or maybe just removed, however I've tried every combination that seemed sensible and I'm not out of ideas.
Below is a greatly simplified example of my code that demonstrates the problem. I would really appreciate a sanity check and hopefully a fresh mind might be able to spot the issue. (every time before a few days of fiddling has produced a fix but this time I'm stumped).
use std::cell::RefCell;
use std::marker::PhantomData;
pub struct DrawCommand<'a, R: Resources<'a>> {
pub data_source: &'a R::DataSource
}
pub trait Resources<'a> {
type DataSource: 'a;
type Buffer: 'a;
}
pub struct DataSource<'a> {
id: u32,
attributes: Vec<VertexAttributes<'a, ResourcesImpl<'a>>>,
current_element_array_buffer_binding: RefCell<Option<Buffer<'a>>>
}
pub struct Buffer<'a> {
context: &'a GraphicsContextImpl
}
pub struct GraphicsContextImpl;
pub struct ResourcesImpl<'a> {
phantom: PhantomData<&'a u32> // 'a is the lifetime of the context reference
}
impl<'a> Resources<'a> for ResourcesImpl<'a> {
type Buffer = Buffer<'a>;
type DataSource = DataSource<'a>;
}
struct Factory<'a> {
context: &'a GraphicsContextImpl
}
impl<'a> Factory<'a> {
/// Creates a buffer
fn create_buffer<T>(&self) -> Buffer<'a> {
Buffer {
context: self.context
}
}
fn create_data_source(&self, attributes: Vec<VertexAttributes<'a, ResourcesImpl<'a>>>) -> DataSource<'a> {
DataSource {
id: 0,
attributes: attributes,
current_element_array_buffer_binding: RefCell::new(None)
}
}
}
fn main() {
let context = GraphicsContextImpl;
let factory = Factory {
context: &context
};
let buffer = factory.create_buffer::<u32>();
let vertices = VertexAttributes::new(&buffer);
let vertex_data_source = factory.create_data_source(vec!(vertices));
let command: DrawCommand<ResourcesImpl> = DrawCommand {
data_source: &vertex_data_source
};
}
pub struct VertexAttributes<'a, R: Resources<'a>> {
pub buffer: &'a R::Buffer,
}
impl<'a, R: Resources<'a>> VertexAttributes<'a, R> {
pub fn new(buffer: &'a R::Buffer) -> VertexAttributes<'a, R> {
VertexAttributes {
buffer: buffer
}
}
}
Thanks very much in advance.
EDIT:
I've updated the code to better reflect my actual implementation.
By the way - replacing this:
let vertex_data_source = factory.create_data_source(vec!(vertices));
With this:
let vertex_data_source = DataSource {
id: 0,
attributes: vec!(vertices),
current_element_array_buffer_binding: RefCell::new(None)
};
Doesn't solve the issue.
This allows your example to compile:
pub struct DrawCommand<'a : 'b, 'b, R: Resources<'a>> {
pub data_source: &'b R::DataSource
}
However, I'm finding it extremely difficult to create a more minimal example. As best I can determine, you have an issue because you are declaring that you will hold a reference to an item that itself has a reference, and that those two references need to have a common lifetime ('a). Through some combination of the other lifetimes, this is actually impossible.
Adding a second lifetime allows the reference to the DataSource to differ from the reference of the DataSource itself.
I'm still going to try to create a more minified example.