I have a struct that contains various Routers. Mostly hashmaps. But for this specific hashmap, the values are not updating after insertion. There is no delete function. Just an insert function(shown below).
This is the main struct
pub struct Router {
....
web_socket_routes: Arc<RwLock<HashMap<String, HashMap<String, (PyFunction, u8)>>>>,
}
This is a getter
#[inline]
pub fn get_web_socket_map(
&self,
) -> &Arc<RwLock<HashMap<String, HashMap<String, (PyFunction, u8)>>>> {
&self.web_socket_routes
}
This is the insert method
pub fn add_websocket_route(
&mut self,
route: &str,
connect_route: (Py<PyAny>, bool, u8),
close_route: (Py<PyAny>, bool, u8),
message_route: (Py<PyAny>, bool, u8),
) {
let table = self.get_web_socket_map();
let (connect_route_function, connect_route_is_async, connect_route_params) = connect_route;
let (close_route_function, close_route_is_async, close_route_params) = close_route;
let (message_route_function, message_route_is_async, message_route_params) = message_route;
let insert_in_router =
|handler: Py<PyAny>, is_async: bool, number_of_params: u8, socket_type: &str| {
let function = if is_async {
PyFunction::CoRoutine(handler)
} else {
PyFunction::SyncFunction(handler)
};
let mut route_map = HashMap::new();
route_map.insert(socket_type.to_string(), (function, number_of_params));
println!("socket type is {:?} {:?}", table, route);
table.write().unwrap().insert(route.to_string(), route_map);
};
insert_in_router(
connect_route_function,
connect_route_is_async,
connect_route_params,
"connect",
);
insert_in_router(
close_route_function,
close_route_is_async,
close_route_params,
"close",
);
insert_in_router(
message_route_function,
message_route_is_async,
message_route_params,
"message",
);
}
After all the 3 insert_in_router calls, web_socket_routes only contains the insertion of the last insert_in_router call?
I have tried changing the Arc<RwLock< for a generic DashMap but I am still facing the same issues.
Why is this happening?
Your closure unconditionally creates a new inner hashmap each time, which it uses as value in the outer hashmap. However, it inserts it into the outer hashmap under the same key (route.to_string()) all three times, which results in each insert overwriting the previous one(s).
You need to implement a logic that will create a new inner hashmap only if one is missing under that key, otherwise look up the existing one. Then it should insert the value into the inner hashmap, either the freshly created one, or the one looked up. In Rust this is conveniently done using the entry API:
table
.write()
.unwrap()
.entry(route.to_string())
.or_default()
.insert(socket_type.to_string(), (function, number_of_params));
Related
I'm a Rust beginner and am working on a small personal project. I'm working on a function that takes a Vec of records and turns them into strings to be used as rows in a CSV file.
My currently incomplete function looks like
pub fn write_csv_records<T: CSVWritable>( file_path: &str, seperator: char, records: Vec<Box<dyn CSVWritable>>, columns: Vec<String> ) -> ()
{
let body = records.iter()
.map(|r| r.to_csv_row() )
.map(|row_values| {
columns.iter()
// this is my problem the closure in unwrap_or_else causes in issue!
.map( |column| row_values.get( column ).unwrap_or_else(|| &String::from("") ).clone() )
.collect::<Vec<String>>()
.join( &seperator.to_string() )
})
.collect::<Vec<String>>()
.join( "\n" );
}
I have had a persistent error that reads "cannot return a reference to data owned by the current function".
I'm at a loss for how to proceed. Is this related to lifetimes? Am I missing how unwrap_or_else works? What is the right way to provide a default value when something is absent in the row_values HashMap?
In your unwrap_or_else() call, you're returning the address of a function that's owned by the closure. This isn't allowed in Rust because that address is no longer valid after the closure finishes and that variable goes out of scope. Since you're just immediately cloning, you should instead return the actual String from the closure, not return the address of the String.
Also, if your CSVWritable trait returns something like a HashMap where the get() method returns a Option<&T> rather than Option<T>, then you need to do some extra work to get the underlying value (because you're collecting into Vector<String> and not Vector<&String>). So you'll want something like this:
pub trait CSVWritable {
fn to_csv_row(&self) -> std::collections::HashMap<String, String>;
}
pub fn write_csv_records(file_path: &str, seperator: char, records: Vec<Box<dyn CSVWritable>>, columns: Vec<String>) {
let body = records
.iter()
.map(|r| r.to_csv_row())
.map(|row_values| {
columns
.iter()
.map(|column| match row_values.get(column) {
Some(s) => s.clone(),
None => String::from(""),
})
.collect::<Vec<String>>()
.join(&seperator.to_string())
})
.collect::<Vec<String>>()
.join("\n");
}
I need a static array of structs and the structs contain a Vec. I can manage the lifetimes of the actual values. I get the following error:
: Mar23 ; cargo test
Compiling smalltalk v0.1.0 (/Users/dmason/git/AST-Smalltalk/rust)
error[E0507]: cannot move out of `dispatchTable[_]` as `dispatchTable` is a static item
--> src/minimal.rs:32:44
|
30 | let old = ManuallyDrop::into_inner(dispatchTable[pos]);
| ^^^^^^^^^^^^^^^^^^ move occurs because `dispatchTable[_]` has type `ManuallyDrop<Option<Box<Dispatch>>>`, which does not implement the `Copy` trait
error: aborting due to previous error
Here is a minimal compilable example:
#[derive(Copy, Clone)]
struct MethodMatch {
hash: i64,
method: Option<bool>,
}
#[derive(Clone)]
pub struct Dispatch {
class: i64,
table: Vec<MethodMatch>,
}
const max_classes : usize = 100;
use std::mem::ManuallyDrop;
const no_dispatch : ManuallyDrop<Option<Box<Dispatch>>> = ManuallyDrop::new(None);
static mut dispatchTable : [ManuallyDrop<Option<Box<Dispatch>>>;max_classes] = [no_dispatch;max_classes];
use std::sync::RwLock;
lazy_static! {
static ref dispatchFree : RwLock<usize> = {RwLock::new(0)};
}
pub fn addClass(c : i64, n : usize) {
let mut index = dispatchFree.write().unwrap();
let pos = *index;
*index += 1;
replaceDispatch(pos,c,n);
}
pub fn replaceDispatch(pos : usize, c : i64, n : usize) -> Option<Box<Dispatch>> {
let mut table = Vec::with_capacity(n);
table.resize(n,MethodMatch{hash:0,method:None});
unsafe {
let old = ManuallyDrop::into_inner(dispatchTable[pos]);
dispatchTable[pos]=ManuallyDrop::new(Some(Box::new(Dispatch{class:c,table:table})));
old
}
}
The idea I had was to have replaceDispatch create a new Dispatch option object, and replace the current value in the array with the new one, returning the original, with the idea that the caller will get the Dispatch option value and be able to use and then drop/deallocate the object.
I found that it will compile if I add .clone() right after the identified error point. But then the original value never gets dropped, so (the into_inner is redundant and) I'm creating a memory leak!. Do I have to manually drop it (if I could figure out how)? I thought that's what ManuallyDrop bought me. In theory, if I created a copy of the fields from the Vec into a copy, that would point to the old data, so when that object got dropped, the memory would get freed. But (a) that seems very dirty, (b) it's a bit of ugly, unnecessary code (I have to handle the Some/None cases, look inside the Vec, etc.), and (c) I can't see how I'd even do it!!!!
As the compiler tells you, you cannot move a value out of a place observable by others. But since you have the replacement at the ready, you can use std::mem::replace:
pub fn replaceDispatch(pos: usize, c: i64, n: usize) -> Option<Box<Dispatch>> {
... table handling omitted ...
unsafe {
let old = std::mem::replace(
&mut dispatchTable[pos],
ManuallyDrop::new(Some(Box::new(Dispatch {
class: c,
table: table,
}))),
);
ManuallyDrop::into_inner(old)
}
}
Playground
In fact, since you're using the Option to manage the lifetime of Dispatch, you don't need ManuallyDrop at all, and you also don't need the Box: playground.
I have a very data driven program which contains different types of entities having very similar structures and differing only in specific places.
For example, every entity has a name which can be changed. Here's two example methods to demonstrate how the methods might be similar:
pub fn rename_blueprint(
&mut self,
ctx: &mut Context,
db_handle: &Transaction,
blueprint_id: Uuid,
new_name: &str,
) -> Result<(), DataError> {
ctx.debug(format!(
"Renaming blueprint {} to {}",
blueprint_id, new_name
));
self.assert_blueprint_exists(ctx, db_handle, blueprint_id)?;
let mut stmt = db_handle
.prepare("UPDATE `blueprints` SET `name` = ? WHERE `id` == ?")
.on_err(|_| ctx.err("Unable to prepare update statement"))?;
let changed_rows = stmt
.execute(params![new_name.to_string(), blueprint_id])
.on_err(|_| ctx.err("Unable to update name in database"))?;
if changed_rows != 1 {
ctx.err(format!("Invalid amount of rows changed: {}", changed_rows));
return Err(DataError::InvalidChangeCount {
changes: changed_rows,
expected_changes: 1,
});
}
ctx.blueprint_renamed(blueprint_id, new_name);
Ok(())
}
pub fn rename_attribute(
&mut self,
ctx: &mut Context,
db_handle: &Transaction,
attribute_id: Uuid,
new_name: &str,
) -> Result<(), DataError> {
ctx.debug(format!(
"Renaming attribute {} to {}",
attribute_id, new_name
));
self.assert_attribute_exists(ctx, db_handle, attribute_id)?;
let mut stmt = db_handle
.prepare("UPDATE `attributes` SET `name` = ? WHERE `id` == ?")
.on_err(|_| ctx.err("Unable to prepare update statement"))?;
let changed_rows = stmt
.execute(params![new_name.to_string(), attribute_id])
.on_err(|_| ctx.err("Unable to update name in database"))?;
if changed_rows != 1 {
ctx.err(format!("Invalid amount of rows changed: {}", changed_rows));
return Err(DataError::InvalidChangeCount {
changes: changed_rows,
expected_changes: 1,
});
}
ctx.attribute_renamed(attribute_id, new_name);
Ok(())
}
The same method with almost identical code now needs to exist for 5-11 more types of entities. I can usually just replace Blueprint with the name of the other entity type, and it will all work. However, that seems like quite a silly solution.
Likewise, writing a helper method which accepts all relevant strings, methods, and such to call it seems similarly silly.
I don't believe I could even avoid this by passing in some "strategy" or other indirection helper (EntityRenamer or something similar), given that the logic would need to be coded there anyway. It'd just be moving the problem one step up.
It should be mentioned that this is one of the shorter methods. Entities can also be moved, deleted, created, etc. all of which have similar code - sometimes 30+ lines long.
How to avoid code duplication of different structs with semantically equal fields/properties? does not solve my issue. That question basically asks "how to do inheritance, when no inheritance exists", whereas my code is struggling with collectivizing very similar logic into the lowest common denominator. Traits or common implementations won't solve my problem, as the code would still exist - it'd only be moved someplace else.
How would you go about deduplicating this code?
I'm more looking for guidelines than someone writing my code for me. A few possible solutions could be:
use macros and then just use something like entity_rename_impl!(args)
use a helper method with a different parameter for each specific thing that can differ from function to function
don't try to abstract the entire method, and instead focus on writing helper functions for smaller things, so that the methods might be duplicating, but it's very little code that is abstracted elsewhere
A MCVE (playground):
#![allow(unused)]
pub struct Transaction {}
impl Transaction {
pub fn execute_sql(&self, sql: &str) -> i32 {
// .. do something in the database
0
}
pub fn bind_id(&self, id: Uuid) {}
}
#[derive(Clone, Copy)]
pub struct Uuid {}
impl std::fmt::Display for Uuid {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "mockup")
}
}
pub fn assert_blueprint_exists(blueprint_id: Uuid) {}
pub fn track_blueprint_rename(id: Uuid, new_name: String) {}
pub fn assert_attribute_exists(blueprint_id: Uuid) {}
pub fn track_attribute_rename(id: Uuid, new_name: String) {}
pub fn rename_blueprint(
db_handle: &Transaction,
blueprint_id: Uuid,
new_name: &str,
) -> Result<(), String> {
println!("Renaming blueprint {} to {}", blueprint_id, new_name);
assert_blueprint_exists(blueprint_id);
db_handle.bind_id(blueprint_id);
let changed_rows = db_handle.execute_sql("UPDATE `blueprints` SET `name` = ? WHERE `id` == ?");
if changed_rows != 1 {
println!("Invalid amount of rows changed: {}", changed_rows);
return Err("Invalid change count in blueprint rename".to_string());
}
track_blueprint_rename(blueprint_id, new_name.to_string());
Ok(())
}
pub fn rename_attribute(
db_handle: &Transaction,
attribute_id: Uuid,
new_name: &str,
) -> Result<(), String> {
println!("Renaming attribute {} to {}", attribute_id, new_name);
assert_attribute_exists(attribute_id);
db_handle.bind_id(attribute_id);
let changed_rows = db_handle.execute_sql("UPDATE `attributes` SET `name` = ? WHERE `id` == ?");
if changed_rows != 1 {
println!("Invalid amount of rows changed: {}", changed_rows);
return Err("Invalid change count in attribute rename".to_string());
}
track_attribute_rename(attribute_id, new_name.to_string());
Ok(())
}
The way I generally solve this kind of problem is by using generics. Let the caller choose the appropriate type.
trait Entity {
fn desc(&self) -> String;
}
impl Entity for Blueprint {
// ...
}
pub fn rename<T>(/* ... */)
where
T: Entity,
{
// ...
}
I want to write a macro that generates varying structs from an integer argument. For example, make_struct!(3) might generate something like this:
pub struct MyStruct3 {
field_0: u32,
field_1: u32,
field_2: u32
}
What's the best way to transform that "3" literal into a number that I can use to generate code? Should I be using macro_rules! or a proc-macro?
You need a procedural attribute macro and quite a bit of pipework. An example implementation is on Github; bear in mind that it is pretty rough around the edges, but works pretty nicely to start with.
The aim is to have the following:
#[derivefields(u32, "field", 3)]
struct MyStruct {
foo: u32
}
transpile to:
struct MyStruct {
pub field_0: u32,
pub field_1: u32,
pub field_2: u32,
foo: u32
}
To do this, first, we're going to establish a couple of things. We're going to need a struct to easily store and retrieve our arguments:
struct MacroInput {
pub field_type: syn::Type,
pub field_name: String,
pub field_count: u64
}
The rest is pipework:
impl Parse for MacroInput {
fn parse(input: ParseStream) -> syn::Result<Self> {
let field_type = input.parse::<syn::Type>()?;
let _comma = input.parse::<syn::token::Comma>()?;
let field_name = input.parse::<syn::LitStr>()?;
let _comma = input.parse::<syn::token::Comma>()?;
let count = input.parse::<syn::LitInt>()?;
Ok(MacroInput {
field_type: field_type,
field_name: field_name.value(),
field_count: count.base10_parse().unwrap()
})
}
}
This defines syn::Parse on our struct and allows us to use syn::parse_macro_input!() to easily parse our arguments.
#[proc_macro_attribute]
pub fn derivefields(attr: TokenStream, item: TokenStream) -> TokenStream {
let input = syn::parse_macro_input!(attr as MacroInput);
let mut found_struct = false; // We actually need a struct
item.into_iter().map(|r| {
match &r {
&proc_macro::TokenTree::Ident(ref ident) if ident.to_string() == "struct" => { // react on keyword "struct" so we don't randomly modify non-structs
found_struct = true;
r
},
&proc_macro::TokenTree::Group(ref group) if group.delimiter() == proc_macro::Delimiter::Brace && found_struct == true => { // Opening brackets for the struct
let mut stream = proc_macro::TokenStream::new();
stream.extend((0..input.field_count).fold(vec![], |mut state:Vec<proc_macro::TokenStream>, i| {
let field_name_str = format!("{}_{}", input.field_name, i);
let field_name = Ident::new(&field_name_str, Span::call_site());
let field_type = input.field_type.clone();
state.push(quote!(pub #field_name: #field_type,
).into());
state
}).into_iter());
stream.extend(group.stream());
proc_macro::TokenTree::Group(
proc_macro::Group::new(
proc_macro::Delimiter::Brace,
stream
)
)
}
_ => r
}
}).collect()
}
The behavior of the modifier creates a new TokenStream and adds our fields first. This is extremely important; assume that the struct provided is struct Foo { bar: u8 }; appending last would cause a parse error due to a missing ,. Prepending allows us to not have to care about this, since a trailing comma in a struct is not a parse error.
Once we have this TokenStream, we successively extend() it with the generated tokens from quote::quote!(); this allows us to not have to build the token fragments ourselves. One gotcha is that the field name needs to be converted to an Ident (it gets quoted otherwise, which isn't something we want).
We then return this modified TokenStream as a TokenTree::Group to signify that this is indeed a block delimited by brackets.
In doing so, we also solved a few problems:
Since structs without named members (pub struct Foo(u32) for example) never actually have an opening bracket, this macro is a no-op for this
It will no-op any item that isn't a struct
It will also no-op structs without a member
I have a struct representing a grid of data, and accessors for the rows and columns. I'm trying to add accessors for the rows and columns which return iterators instead of Vec.
use std::slice::Iter;
#[derive(Debug)]
pub struct Grid<Item : Copy> {
raw : Vec<Vec<Item>>
}
impl <Item : Copy> Grid <Item>
{
pub fn new( data: Vec<Vec<Item>> ) -> Grid<Item> {
Grid{ raw : data }
}
pub fn width( &self ) -> usize {
self.rows()[0].len()
}
pub fn height( &self ) -> usize {
self.rows().len()
}
pub fn rows( &self ) -> Vec<Vec<Item>> {
self.raw.to_owned()
}
pub fn cols( &self ) -> Vec<Vec<Item>> {
let mut cols = Vec::new();
for i in 0..self.height() {
let col = self.rows().iter()
.map( |row| row[i] )
.collect::<Vec<Item>>();
cols.push(col);
}
cols
}
pub fn rows_iter( &self ) -> Iter<Vec<Item>> {
// LIFETIME ERROR HERE
self.rows().iter()
}
pub fn cols_iter( &self ) -> Iter<Vec<Item>> {
// LIFETIME ERROR HERE
self.cols().iter()
}
}
Both functions rows_iter and cols_iter have the same problem: error: borrowed value does not live long enough. I've tried a lot of things, but pared it back to the simplest thing to post here.
You can use the method into_iter which returns std::vec::IntoIter. The function iter usually only borrows the data source iterated over. into_iter has ownership of the data source. Thus the vector will live as long as the actual data.
pub fn cols_iter( &self ) -> std::vec::IntoIter<Vec<Item>> {
self.cols().intoiter()
}
However, I think that the design of your Grid type could be improved a lot. Always cloning a vector is not a good thing (to name one issue).
Iterators only contain borrowed references to the original data structure; they don't take ownership of it. Therefore, a vector must live longer than an iterator on that vector.
rows and cols allocate and return a new Vec. rows_iter and cols_iter are trying to return an iterator on a temporary Vec. This Vec will be deallocated before rows_iter or cols_iter return. That means that an iterator on that Vec must be deallocated before the function returns. However, you're trying to return the iterator from the function, which would make the iterator live longer than the end of the function.
There is simply no way to make rows_iter and cols_iter compile as is. I believe these methods are simply unnecessary, since you already provide the public rows and cols methods.