I have a struct with a field containing references to other structs (that I did not define)
struct HtmlHandlebars {
user_helpers: Vec<(String, Box<HelperDef + 'static>)>,
}
And HtmlHandlebars has to implement a function
fn render(&self, ...) -> &self
And in that function I would need to move the Box to another function. Something like this:
fn render(&self, ...) -> &self {
let mut handlebars = Handlebars::new();
for (name, helper) in self.user_helpers {
handlebars.register_helper(&name, helper);
}
}
But I am kind of stuck because:
I can't move the Box references because I am borrowing self
I can't copy the Box references because that struct does not implement copy
I can't modify &self to &mut self because that causes other problems...
Maybe I am doing it completely wrong.. Is there something else I can do? What are my options?
If you need a more complete overview of the code, you can find it here
PS: I had no idea how to describe the situation in the title, feel free to change it
The code you've written is trying to consume the Vec and its elements. In general, you can iterate over &self.user_helpers which will give you references to the elements rather than consuming them. That is (modulo silly typos in the pattern):
for &(ref name, ref helper) in self.user_helpers {
handlebars.register_helper(name, helper);
}
See also: The Rust Programming Language on iterating vectors.
There is a problem with that though: Handlebars needs ownership of the helpers. It may very well be that you have to create a new Handlebars object every time you render, but in that case you also need to be able to create all the helpers every time you create a new Handlebars. There is no way to take ownership of the boxes without taking at least a &mut reference to the Vec. You need at least mutable access to take the handlers out of the struct. And if you did that, you wouldn't have the handlers around for the next time render() is called. Depending on how configurable the set of handlers is, you could have a function that constructs the Vec<Box<HelperDef + 'static>> out of thin air when you need it, or you could maintain a list of callbacks that construct Box<HelperDef + 'static> for you.
Related
I've written a program to process my financial transactions but it's starting to run a little slower now that I'm adding more data to it. I decided to write it in Rust. I'm fluent in JS, TS, Python, bash/shell scripting. I need to feed the entire history into the application at this time. Currently my program is single-threaded. My thought is that if I use multi-threading strategically I should be able to cut the run-time down.
Here is how I attempted to implement multi-threading:
for row in lines[1..].iter() {
thread::spawn(|| {
process_transaction(row, &rules)
});
}
Rules is a hashmap that looks like this.
type CustomRule = Box<dyn Fn(&Transaction) -> &'static str>;
type RuleHashMap = HashMap<&'static str, CustomRule>;
Row is a Transaction structure w/ stuff and some functions implemented into it. (Regex, gt/lt matching)
The key will be a regex string and the value will be that custom function. This custom function has to be put in a Box because it's size needs to go onto the heap?
The idea is that I should be able to just quickly iterate over a set of regex patterns then run the corresponding additional logic needed to process that transaction. I am not mutating Transaction or Rules in any way and I'm only printing a result. Here's an example rule:
rules.insert(r"(?i)7-ELEVEN|EXXONMOBIL|CIRCLE K|SUNOCO|SHEETZ|A-PLUS|RACEWAY|SHELLSERVICE|Shell SERVICE|QUICK NEASY|QUICK N EASY|FAS MART|BP|ROYAL MART|CITG|gas|wawa", Box::new(|t:&Transaction|{
match t.less_than(15.0) {
true => "expenses:convience",
false => "expenses:business:gas"
}
}));
The compiler suggested I update the type to implement Send
type CustomRule = dyn Fn(&Transaction) + Send + 'static;
I added that but now it says it does not know the size of the HashMap at compile time. I get this message from the compiler
= help: the trait `Sized` is not implemented for `(dyn for<'r> Fn(&'r Transaction) + Send + 'static)`
note: required by a bound in `HashMap`
What is this? I'm new to lower level programming like this and I want to understand what is actually happening versus just copying blindly. Especially when playing w/ threads. Was putting that custom rule function into a Box<> type the wrong move? Am I making this more complicated than needed.
If all of your closures are "pure" functions that don't capture any state, then you can just store fn pointers in your map instead:
let mut rules: HashMap<&'static str, fn(&Transaction) -> &'static str> = HashMap::new();
rules.insert(r"some long regex", |t: &Transaction|{
"expenses:convience"
});
You say
The idea is that I should be able to just quickly iterate over a set of regex patterns then run the corresponding additional logic needed to process that transaction.
If you're actually iterating, why even use a HashMap? Just use a Vec or array of tuples like so:
let mut rules: [(&'static str, fn(&Transaction) -> &'static str); 1] = [
(r"some long regex", |t: &Transaction|{
"expenses:convience"
}),
];
You probably also should compile the Regexs only once, rather than compiling and running them every time. I recommend putting them in statics with once_cell::sync::Lazy.
I have a struct Database { events: Vec<Event> }. I would like to apply some maps and filters to events. What is a good way to do this?
Here's what I tried:
fn update(db: &mut Database) {
db.events = db.events.into_iter().filter(|e| !e.cancelled).collect();
}
This doesn't work:
cannot move out of `db.events` which is behind a mutable reference
...
move occurs because `db.events` has type `Vec<Event>`, which does not implement the `Copy` trait
Is there any way to persuade Rust compiler that I'm taking the field value only temporarily?
The conceptual issue of why this doesn't work is due to panics. If, for example, the filter callback panics, then db.events would have been moved out of, by into_iter, but would not have had a value to replace it with - it would be uninitialized, and therefore unsafe.
Joël Hecht has what you really want to do in your specific instance: Vec::retain lets you filter out elements in place, and also reuses the storage.
Alexey Larionov also has an answer involving Vec::drain, which will leave an empty vector until the replacement happens. It requires a new allocation, though.
However, in the general case, the replace_with and take_mut crates offer functions to help accomplish what you are doing. You provide the function a closure that takes the value and returns its replacement, and the crates will run that closure, and aborting the process if there are panics.
In the case you exposed, the safer way is to use Vec.retain :
fn update(db: &mut Database) {
db.events.retain(|e| !e.cancelled);
}
Alternatively to #Joël Hecht's answer, you can Vec::drain the elements to then recreate the vector. Playground
fn update(db: &mut Database) {
db.events = db.events
.drain(..)
.filter(|e| !e.cancelled)
.collect();
}
Standard Cell struct provides interior mutability but allows only a few mutation methods such as set(), swap() and replace(). All of these methods change the whole content of the Cell.
However, sometimes more specific manipulations are needed, for example, to change only a part of data contained in the Cell.
So I tried to implement some kind of universal Cell, allowing arbitrary data manipulation.
The manipulation is represented by user-defined closure that accepts a single argument - &mut reference to the interior data of the Cell, so the user itself can deside what to do with the Cell interior. The code below demonstrates the idea:
use std::cell::UnsafeCell;
struct MtCell<Data>{
dcell: UnsafeCell<Data>,
}
impl<Data> MtCell<Data>{
fn new(d: Data) -> MtCell<Data> {
return MtCell{dcell: UnsafeCell::new(d)};
}
fn exec<F, RetType>(&self, func: F) -> RetType where
RetType: Copy,
F: Fn(&mut Data) -> RetType
{
let p = self.dcell.get();
let pd: &mut Data;
unsafe{ pd = &mut *p; }
return func(pd);
}
}
// test:
type MyCell = MtCell<usize>;
fn main(){
let c: MyCell = MyCell::new(5);
println!("initial state: {}", c.exec(|pd| {return *pd;}));
println!("state changed to {}", c.exec(|pd| {
*pd += 10; // modify the interior "in place"
return *pd;
}));
}
However, I have some concerns regarding the code.
Is it safe, i.e can some safe but malicious closure break Rust mutability/borrowing/lifetime rules by using this "universal" cell?
I consider it safe since lifetime of the interior reference parameter prohibits its exposition beyond the closure call time. But I still have doubts (I'm new to Rust).
Maybe I'm re-inventing the wheel and there exist some templates or techniques solving the problem?
Note: I posted the question here (not on code review) as it seems more related to the language rather than code itself (which represents just a concept).
[EDIT] I'd want zero cost abstraction without possibility of runtime failures, so RefCell is not perfect solution.
This is a very common pitfall for Rust beginners.
Is it safe, i.e can some safe but malicious closure break Rust mutability/borrowing/lifetime rules by using this "universal" cell? I consider it safe since lifetime of the interior reference parameter prohibits its exposition beyond the closure call time. But I still have doubts (I'm new to Rust).
In a word, no.
Playground
fn main() {
let mt_cell = MtCell::new(123i8);
mt_cell.exec(|ref1: &mut i8| {
mt_cell.exec(|ref2: &mut i8| {
println!("Double mutable ref!: {:?} {:?}", ref1, ref2);
})
})
}
You're absolutely right that the reference cannot be used outside of the closure, but inside the closure, all bets are off! In fact, pretty much any operation (read or write) on the cell within the closure is undefined behavior (UB), and may cause corruption/crashes anywhere in your program.
Maybe I'm re-inventing the wheel and there exist some templates or techniques solving the problem?
Using Cell is often not the best technique, but it's impossible to know what the best solution is without knowing more about the problem.
If you insist on Cell, there are safe ways to do this. The unstable (ie. beta) Cell::update() method is literally implemented with the following code (when T: Copy):
pub fn update<F>(&self, f: F) -> T
where
F: FnOnce(T) -> T,
{
let old = self.get();
let new = f(old);
self.set(new);
new
}
Or you could use Cell::get_mut(), but I guess that defeats the whole purpose of Cell.
However, usually the best way to change only part of a Cell is by breaking it up into separate Cells. For example, instead of Cell<(i8, i8, i8)>, use (Cell<i8>, Cell<i8>, Cell<i8>).
Still, IMO, Cell is rarely the best solution. Interior mutability is a common design in C and many other languages, but it is somewhat more rare in Rust, at least via shared references and Cell, for a number of reasons (e.g. it's not Sync, and in general people don't expect interior mutability without &mut). Ask yourself why you are using Cell and if it is really impossible to reorganize your code to use normal &mut references.
IMO the bottom line is actually about safety: if no matter what you do, the compiler complains and it seems that you need to use unsafe, then I guarantee you that 99% of the time either:
There's a safe (but possibly complex/unintuitive) way to do it, or
It's actually undefined behavior (like in this case).
EDIT: Frxstrem's answer also has better info about when to use Cell/RefCell.
Your code is not safe, since you can call c.exec inside c.exec to get two mutable references to the cell contents, as demonstrated by this snippet containing only safe code:
let c: MyCell = MyCell::new(5);
c.exec(|n| {
// need `RefCell` to access mutable reference from within `Fn` closure
let n = RefCell::new(n);
c.exec(|m| {
let n = &mut *n.borrow_mut();
// now `n` and `m` are mutable references to the same data, despite using
// no unsafe code. this is BAD!
})
})
In fact, this is exactly the reason why we have both Cell and RefCell:
Cell only allows you to get and set a value and does not allow you to get a mutable reference from an immutable one (thus avoiding the above issue), but it does not have any runtime cost.
RefCell allows you to get a mutable reference from an immutable one, but needs to perform checks at runtime to ensure that this is safe.
As far as I know, there's not really any safe way around this, so you need to make a choice in your code between no runtime cost but less flexibility, and more flexibility but with a small runtime cost.
I have a two widgets in a simple GTK app:
extern crate gdk;
extern crate gtk;
use super::desktop_entry::DesktopEntry;
use gdk::enums::key;
use gtk::prelude::*;
pub fn launch_ui(_desktop_entries: Vec<DesktopEntry>) {
gtk::init().unwrap();
let builder = gtk::Builder::new_from_string(include_str!("interface.glade"));
let window: gtk::Window = builder.get_object("main_window").unwrap();
let search_entry: gtk::SearchEntry = builder.get_object("search_entry").unwrap();
let list_box: gtk::ListBox = builder.get_object("list_box").unwrap();
window.show_all();
search_entry.connect_search_changed(move |_se| {
let _a = list_box.get_selected_rows();
});
window.connect_key_press_event(move |_, key| {
match key.get_keyval() {
key::Down => {
list_box.unselect_all();
}
_ => {}
}
gtk::Inhibit(false)
});
gtk::main();
}
I need to change list_box from both events. I have two closures that move, but it is not possible to move list_box to both closures simultaneously as I get the error:
error[E0382]: capture of moved value: `list_box`
What can I do?
As explained in Shepmaster's answer, you can only move a value out of a variable once, and the compiler will prevent you from doing it a second time. I'll try to add a bit of specific context for this use case. Most of this is from my memory of having used GTK from C ages ago, and a few bits I just looked up in the gtk-rs documentation, so I'm sure I got some details wrong, but I think the general gist is accurate.
Let's first take a look at why you need to move the value into the closures in the first place. The methods you call on list_box inside both closures take self by reference, so you don't actually consume the list box in the closures. This means it would be perfectly valid to define the two closures without the move specifiers – you only need read-only references to list_box, you are allowed to have more than one read-only reference at once, and list_box lives at least as long as the closures.
However, while you are allowed to define the two closures without moving list_box into them, you can't pass the closures defined this way to gtk-rs: all functions connecting event handlers only accept "static" functions, e.g.
fn connect_search_changed<F: Fn(&Self) + 'static>(
&self,
f: F
) -> SignalHandlerId
The type F of the handler has the trait bound Fn(&Self) + 'static, which means that the closure either can't hold any references at all, or all references it holds must have static lifetime. If we don't move list_box into the closure, the closure will hold a non-static reference to it. So we need to get rid of the reference before being able to use the function as an event handler.
Why does gtk-rs impose this limitation? The reason is that gtk-rs is a wrapper around a set of C libraries, and a pointer to the callback is eventually passed on to the underlying glib library. Since C does not have any concept of lifetimes, the only way to do this safely is to require that there aren't any references that may become invalid.
We have now established that our closures can't hold any references. We still need to access list_box from the closures, so what are our options? If you only have a single closure, using move does the trick – by moving list_box into the closure, the closure becomes its owner. However, we have seen that this doesn't work for more than one closure, because we can only move list_box once. We need to find a way to have multiple owners for it, and the Rust standard library provides such a way: the reference-counting pointers Rc and Arc. The former is used for values that are only accessed from the current thread, while the latter is safe to move to other threads.
If I remember correctly, glib executes all event handlers in the main thread, and the trait bounds for the closure reflect this: the closure isn't required to be Send or Sync, so we should be able to make do with Rc. Morevoer, we only need read access to list_box in the closures, so we don't need RefCell or Mutex for interior mutability in this case. In summary, all you need is probably this:
use std::rc::Rc;
let list_box: gtk::ListBox = builder.get_object("list_box").unwrap();
let list_box_1 = Rc::new(list_box);
let list_box_2 = list_box_1.clone();
Now you have two "owned" pointers to the same list box, and these pointers can be moved into the two closures.
Disclaimer: I couldn't really test any of this, since your example code isn't self-contained.
You can use cloning on the gtk-rs widgets.
In gtk-rs every object implementing gtk::Widget (so basically every GTK object you can use inside a gtk::Window) must also implement the Clone trait. Calling clone() is very cheap because it's just a pointer copy and a reference counter update.
Knowing this below is valid and cheap:
let list_box_clone = list_box.clone();
search_entry.connect_search_changed(move |_se| {
let _a = list_box.get_selected_rows();
});
But since this solution is verbose and gets very ugly very soon if you have more than one objects to move, the community came up with the following macro:
macro_rules! clone {
(#param _) => ( _ );
(#param $x:ident) => ( $x );
($($n:ident),+ => move || $body:expr) => (
{
$( let $n = $n.clone(); )+
move || $body
}
);
($($n:ident),+ => move |$($p:tt),+| $body:expr) => (
{
$( let $n = $n.clone(); )+
move |$(clone!(#param $p),)+| $body
}
);
}
The usage is very simple:
search_entry.connect_search_changed(clone!(list_box => move |_se| {
let _a = list_box.get_selected_rows();
}));
This macro is capable of cloning any number of objects that are moved into a closure.
For further explanation and examples check out this tutorial from the gtk-rs team: Callbacks and closures
You literally cannot do this. I encourage you to go back and re-read The Rust Programming Language to refresh yourself on ownership. When a non-Copy type is moved, it's gone — this is a giant reason that Rust even exists: to track this so the programmer doesn't have to.
If a type is Copy, the compiler will automatically make the copy for you. If a type is Clone, then you must invoke the clone explicitly.
You will need to change to shared ownership and most likely interior mutability.
Shared ownership allows a single piece of data to be jointly owned by multiple values, creating additional owners via cloning.
Interior mutability is needed because Rust disallows multiple mutable references to one item at the same time.
Wrap your list_box in a Mutex and then an Arc (Arc<Mutex<T>>). Clone the Arc for each handler and move that clone into the handler. You can then lock the list_box and make whatever changes you need.
See also:
What is the right way to share a reference between closures if the value outlives the closures?
How to share an Arc in multiple closures?
I'm pretty new to Rust and I've been slowly following an arcade game tutorial which has been a great help with the concepts it goes through.
In part nine of the tutorial, in which the main menu is created, the author suggests 'homework' for the reader of making the labels on the main menu ("New Game", "Quit") animate their change in size when focused and unfocused, rather than jump to their idle/focused size. This is where I have been having difficulty...
The basic layout of the relevant parts of the code before I started to implement the change is the following:
// equivalent to 'menu option'
struct Action {
/// function executed if action chosen
func: Box<Fn(&mut Phi) -> ViewAction>,
label: &'static str,
idle_sprite: Sprite, // smaller (32)
focus_sprite: Sprite, // larger (38)
// ...
}
impl Action {
fn new(phi: &mut Phi, label: &'static str, func: Box<Fn(&mut Phi) -> ViewAction>) -> Action {
// ...
}
struct MainMenuView {
actions: Vec<Action>,
selected: i8,
// ...
}
impl MainMenuView {
pub fn new(phi: &mut Phi) -> MainMenuView {
// ...
}
}
impl View for MainMenuView {
fn render(&mut self, phi: &mut Phi, elapsed: f64) -> ViewAction {
// ...
for (i, action) in self.actions.iter().enumerate() {
// ...
}
}
}
fn main() {
::phi::spawn("Arcade Shooter", |phi| {
Box::new(::views::main_menu::MainMenuView::new(phi))
});
}
My first thought for the animation was to make it dynamically create a sprite based on an interpolated size between idle_sizeand focus_size using time elapsed since focus change using methods on Action to focus and defocus to change a current_size field that would be used to generate a sprite for a sprite field.
This required a mutable binding of the Action struct, which took me a little while to work out as there was no let binding anywhere, but seemed to be just about possible by changing the constructor: Action::new(...) -> &mut action, and lots of explicitly marking lifetimes (which had its own issues, but this is getting too long as it is). I then realised that the MainMenuView would have to be mutably bound as well, at which point I stopped this path (I hadn't managed to successfully compile since starting it), as this seemed a really inelegant solution that made basically everything mutable, surely defeating the point of rust's immutability default...
I then wondered whether I could just create a new MainMenuView with a new Action with the new sprite, which could probably work (changing view to another MainMenuView), but this seems like a really wasteful way to just change the size of some text and again is pretty inelegant.
After that, I remembered Cell, but when trying this to make the actions for MainMenuView a Vec<Cell<Actions>>, I found Cell only works with Copy types. This might have been ok (I don't have enough experience to know), but the func field of Action does not implement Copy (and I'm not sure if it can?) and so Action cannot #[derive(Copy)]. Dead end without restructuring a large section of the program to not have func in Action?
This is the end of my main question - basically, what do you do when you have structs nested and you want to have a deep field mutate, but can't put a Cell around it (afaik)? And is this a structural issue with the code such that I should be avoiding this issue in the first place?
I also realised that a solution with a Vec<Sprite> in Action with a lot of sprites of different sizes for the transition would eliminate the need for any of the aforementioned to be mutable. This instinctively felt slightly hacky, as it was effectively hardcoding something that shouldn't have to be. I could also see issues in the implementation with properly aligning to frames (I'm new to synchronising things with frame timing as well), and working for a maximum fps - although the number of sprites could be dynamically created based on the max fps when MainMenuView is constructed...
Use RefCell for non-Copy types.
struct Action {
func: Box<Fn(&mut Phi) -> ViewAction>,
label: &'static str,
sprite: RefCell<Sprite>,
// ...
}
If you can mutate a struct (you either own it or have a mutable reference to it) then you can mutably borrow any of its fields. What that means in this case is that if you are ever given the opportunity to mutate MainMenuView, then you can take a moment to mutate any of the actions as well. Using RefCell or Cell on a field also works when you can't mutate a struct, but obscures when the value may be changing. RefCell also runs the risk of runtime borrow panics. You should avoid RefCell if possible!
I don't know how this framework works, which affects how this question can be answered. It looks like phi takes ownership over your MainMenuView, which means phi decides when you get to mutate it from then on. If you're never given the opportunity to mutate the MainMenuView regularly to perform animation, it may still be possible. Another option that avoids RefCell might be to encode the animation when you mutate selected and compute how it should affect the drawing during the draw call. For example, if you store the timestamp when the selection was changed then you can compute at draw time how the sprite should be drawn.
From your tutorial, it looks like the View::render already takes the MainMenuView as a mutable reference. The MainMenuView has ownership of all the Action values through the Vec, which means mutability transfers through to them. This means you didn't actually have to change anything in order to get mutable access to the Action values, except to call iter_mut() instead of iter() in the for loop in the implementation of View::render for MainMenuView.