I would like to pass a FnOnce closure to an object to be used later, but I would like to avoid any heap allocation. I can avoid heap allocation by keeping the closure on the stack. But the problem is that I can't pass a reference to the object because the FnOnce call_once consumes the closure. So I need to pass an owned pointer (e.g. Box) without heap allocation.
Is this possible? What I'd like to do is this:
fn main() {
let mut scheduler = NoHeapScheduler();
// allocate the task on the stack
let task = move ||;
// somehow pass ownership of the closure, while keeping it allocated on the stack.
scheduler.add_task(StaticBox::new(task));
schedule.run();
}
As far as I know this should be safe as long as the scheduler doesn't outlive the task. Is there any way to make this happen?
Can I create an owned pointer to a stack object?
No. This is non-sensical actually, since by definition a stack object is owned by the stack, so it cannot also be owned by something else.
So I need to pass an owned pointer (e.g. Box) without heap allocation.
There are other owned pointers than Box.
At the moment, I know of none without a heap allocation, but there is little reason it cannot be done.
I envision a InlineFnOnceBox<S: Default, R, A> used as InlineFnOnceBox<[u8; 48], (), ()> in this case, which would contain both the array itself, used as backing storage, plus a virtual pointer to the FnOnce<A -> R> v-table for the type instantiated.
It requires some care (and unsafe code) to instantiate, but otherwise seems feasible.
Can I create an owned pointer to a stack object?
No, but you can simply move the stack object into your scheduler. Your scheduler will increase in size with every closure you schedule, but it will be completely self contained an can even be moved around.
The basic idea is that your Scheduler becomes a kind of singly linked list:
pub trait Scheduler: Sized {
fn run(self);
}
pub struct NoHeapScheduler<F: FnOnce(), T: Scheduler> {
inner: T,
f: F,
}
impl<F: FnOnce(), T: Scheduler> Scheduler for NoHeapScheduler<F, T> {
fn run(self) {
self.inner.run();
(self.f)()
}
}
The Scheduler trait is here to break the recursion chain in the NoHeapScheduler (Otherwise we'd need a feature like variadic generics).
To terminate the chain we also implement Scheduler for some no-op type, e.g. ():
impl Scheduler for () {
fn run(self) {}
}
Now the only thing left is a way to add new closures.
impl<F: FnOnce(), T: Scheduler> NoHeapScheduler<F, T> {
fn add_task<F2: FnOnce()>(self, f: F2) -> NoHeapScheduler<F2, Self> {
NoHeapScheduler {
inner: self,
f: f,
}
}
}
This method moves The current scheduler into a new scheduler and adds the scheduled closure.
You can use this function like so:
let scheduler = scheduler.add_task(task);
Fully working example in the playground
As stated, the answer to the question is "no".
If you pass ownership of the closure, you have to by definition move it into the owner (otherwise what you've got is a reference). You can do that if you've only got one callback using a generic type:
pub struct NoHeapScheduler<F:FnOnce()> {
f: Option<F>,
}
impl<F:FnOnce()> NoHeapScheduler<F> {
pub fn add_task(&mut self, f: F) {
self.f = Some(f);
}
pub fn run(&mut self) {
let f = self.f.take().unwrap();
f()
}
}
fn main() {
let mut scheduler = NoHeapScheduler{ f: None };
let task = move || {};
scheduler.add_task(task);
scheduler.run();
}
Playground
However you'd run into problems adding more than one closure, since they all have different types.
If you're willing to allow allocations and an unstable feature on the nightly compiler, you could use FnBox. This is like FnOnce but works with Box:
#![feature(fnbox)]
use std::boxed::FnBox;
pub struct NoHeapScheduler {
v: Vec<Box<FnBox()>>,
}
impl NoHeapScheduler {
pub fn add_task(&mut self, f: Box<FnBox()>) {
self.v.push(f);
}
pub fn run(&mut self) {
for f in self.v.drain(0..) {
f();
}
}
}
fn main() {
let mut scheduler = NoHeapScheduler{ v: Vec::new() };
let task = move || {println!("Hello,"); };
let other_task = move || {println!("world!"); };
scheduler.add_task(Box::new(task));
scheduler.add_task(Box::new(other_task));
scheduler.run();
}
Playground
I can use an Option to do this. I can keep the Option on the stack and pass a mutable reference around, and then when I'm ready to consume the closure I can use Option::take to take ownership of the closure and consume it.
To handle different implementations of FnOnce, I can lift this out into a trait and use trait objects:
trait Callable {
fn call(&mut self);
}
impl<F: FnOnce()> Callable for Option<F> {
fn call(&mut self) {
if let Some(func) = self.take() {
func();
}
}
}
Then I can pass around trait objects that live on the stack but can be consumed by having the reference.
Related
I've got a struct defined that has a function which defines a static lifetime:
impl MyStruct {
pub fn doSomething(&'static self) {
// Some code goes here
}
}
I'm consuming it from main like so:
fn main() {
let obj = MyStruct {};
obj.doSomething();
}
It's intended for the doSomething call to block and execute for the lifetime of the application.
I'm running into issues with the lifetime checks where it's stating that it may outlive the main function, which seems strange to me as once main is complete the application should exit.
Is there a way to achieve this?
The primary way to create a reference that has the 'static lifetime is to make the variable static. A static variable is one that can be created at compile time:
struct MyStruct;
impl MyStruct {
pub fn do_something(&'static self) {}
}
static OBJ: MyStruct = MyStruct;
fn main() {
OBJ.do_something();
}
As Rust's constant evaluation story improves, this will be more common, but it will never allow configuration at runtime.
A far less common method is to deliberately leak memory, producing a reference that will last "forever". This should be discouraged because leaking memory isn't a good thing:
fn main() {
let obj = Box::leak(Box::new(MyStruct));
obj.do_something();
}
There's also the possibility of creating a singleton:
How do I create a global, mutable singleton?
How can you make a safe static singleton in Rust?
as once main is complete the application should exit.
Perhaps, but the compiler doesn't treat main specially for lifetime purposes.
hyper requires a static runtime when running the server and processing each request.
No, it doesn't. It has a bound of : 'static, which means that any references passed in must be 'static, but you don't have to pass in a bare reference at all.
For patterns like this, the most common thing is to pass in something like an Arc. This allows sharing of the underlying resource.
pub fn do_something<F, T>(f: F)
where
F: Fn() -> T + 'static,
T: 'static,
{
// "spawn" 3 threads
f();
f();
f();
}
struct MyStruct;
static OBJ: MyStruct = MyStruct;
fn main() {
// OK
do_something(|| &OBJ);
// Not OK
let another = MyStruct;
do_something(|| &another);
// OK
use std::sync::Arc;
let shared = Arc::new(MyStruct);
do_something(move || shared.clone());
}
You can even use interior mutability if you wanted dynamic reconfiguration.
See also:
How do I use static lifetimes with threads?
The naive way to do this is with a static variable, but it will require unsafe code if you need to actually set the value inside your main function:
static mut OBJ: MyStruct = MyStruct;
fn main() {
unsafe {
OBJ = MyStruct {};
OBJ.doSomething();
}
}
It's also unsafe to do pretty much anything with a mutable static thereafter.
The much better way to do it is to let a library (lazy_static) take care of the unsafe code.
use lazy_static::lazy_static;
fn main() {
lazy_static!{
static ref OBJ: MyStruct = MyStruct {};
}
OBJ.doSomething();
}
Consider such code:
trait OnUpdate {
fn on_update(&mut self, x: &i32);
}
struct Foo {
field: i32,
cbs: Vec<Box<OnUpdate>>,
}
impl Foo {
fn subscribe(&mut self, cb: Box<OnUpdate>) {
self.cbs.push(cb);
}
fn set_x(&mut self, v: i32) {
self.field = v;
//variant 1
//self.call_callbacks(|v| v.on_update(&self.field));
//variant 2
let f_ref = &self.field;
for item in &mut self.cbs {
item.on_update(f_ref);
}
}
fn call_callbacks<CB: FnMut(&mut Box<OnUpdate>)>(&mut self, mut cb: CB) {
for item in &mut self.cbs {
cb(item);
}
}
}
If I comment variant 2 and uncomment variant 1,
it doesn't compiles, because of I need &Foo and &mut Foo at the same time.
But I really need function in this place, because of I need the same
code to call callbacks in several places.
So do I need macros here to call callbacks, or may be another solution?
Side notes: in real code I use big structure instead of i32,
so I can not copy it. Also I have several methods in OnUpdate,
so I need FnMut in call_callbacks.
An important rule of Rust's borrow checker is, mutable access is exclusive access.
In variant 2, this rule is upheld because the reference to self.field and to mut self.cbs never really overlap. The for loop implicitly invokes into_iter on &mut Vec, which returns a std::slice::IterMut object that references the vector, but not the rest of Foo. In other words, the for loop does not really contain a mutable borrow of self.
In variant 1, there is a call_callbacks which does retain a mutable borrow of self, which means it cannot receive (directly on indirectly) another borrow of self. In other words, at the same time:
It accepts a mutable reference to self, which allows it to modify all its fields, including self.field.
It accepts a closure that also refers to self, because it uses the expression self.field.
Letting this compile would allow call_callbacks to mutate self.field without the closure being aware of it. In case of an integer it might not sound like a big deal, but for other data this would lead to bugs that Rust's borrow checker is explicitly designed to prevent. For example, Rust relies on these properties to prevent unsafe iteration over mutating containers or data races in multi-threaded programs.
In your case it is straightforward to avoid the above situation. set_x is in control both of the contents of the closure and of the mutation to self.field. It could be restated to pass a temporary variable to the closure, and then update self.field, like this:
impl Foo {
fn subscribe(&mut self, cb: Box<OnUpdate>) {
self.cbs.push(cb);
}
fn set_x(&mut self, v: i32) {
self.call_callbacks(|cb| cb.on_update(&v));
self.field = v;
}
fn call_callbacks<OP>(&mut self, mut operation: OP)
where OP: FnMut(&mut OnUpdate)
{
for cb in self.cbs.iter_mut() {
operation(&mut **cb);
}
}
}
Rust has no problem with this code, and effect is the same.
As an exercise, it is possible to write a version of call_callbacks that works like variant 2. In that case, it needs to accept an iterator into the cbs Vec, much like the for loop does, and it must not accept &self at all:
fn set_x(&mut self, v: i32) {
self.field = v;
let fref = &self.field;
Foo::call_callbacks(&mut self.cbs.iter_mut(),
|cb| cb.on_update(fref));
}
fn call_callbacks<OP>(it: &mut Iterator<Item=&mut Box<OnUpdate>>,
mut operation: OP)
where OP: FnMut(&mut OnUpdate)
{
for cb in it {
operation(&mut **cb);
}
}
Does it create a new thread and then execute that anonymous function inside the new thread?
I noticed many ownership / borrowing restrictions when I'm working with a closure. For example, if I have Fn(), I cannot pass a mutable variable inside the closure or it needs to be wrapped with a Mutex:
fn helloworld(f: &Fn(f64)) {
f(42f64);
}
pub fn main() {
let mut killer = 2;
helloworld(&|n| {
println!("{}", n);
killer += 1;
});
}
If a closure can be unsafe like that then something asynchronous or parallel is going on behind the scene and that's why Rust compiler doesn't let me to compile such code.
I might just be confused because I'm coming from a JavaScript / Python world and things are completely different there.
There are two layers to this question.
First, a closure in Rust is just an anonymously-defined type that implements one or more "callable" traits. For example, this:
fn main() {
let a = 6;
let closure = |b| {
println!("product is: {}", a * b);
};
closure(7);
}
is de-sugared into something similar to:
fn main() {
let a = 6;
let closure = {
struct Closure<'a> {
a: &'a i32,
}
impl<'a> Fn<(i32,)> for Closure<'a> {
extern "rust-call" fn call(&self, (b,): (i32,)) {
println!("product is: {}", (*self.a) * b);
}
}
impl<'a> FnMut<(i32,)> for Closure<'a> {
extern "rust-call" fn call_mut(&mut self, args: (i32,)) {
self.call(args)
}
}
impl<'a> FnOnce<(i32,)> for Closure<'a> {
type Output = ();
extern "rust-call" fn call_once(self, args: (i32,)) {
self.call(args)
}
}
Closure {
a: &a,
}
};
FnOnce::call_once(closure, (7,));
}
Note: the above code relies on unstable, internal details and will not work on a stable compiler. It is provided for explanation only; you should not use this pattern yourself.
There's no threading involved, and nothing magical is happening. They boil down to a regular function call with an extra initial "context" argument.
This brings us to the second layer, which is why your specific code doesn't work: because you told the compiler to forbid it. One critical concern for callables is how the context is passed to the callable's code. This is represented by the Fn, FnMut and FnOnce traits (which are explained in the answer to the question When does a closure implement Fn, FnMut and FnOnce?). By taking &Fn(f64), you've restricted yourself to only accepting closures which require immutable access to their context.
If you want a closure to be able to mutate its context, you need to use FnMut instead. Or, if you only need to call a closure once, you can use FnOnce (although not as a trait object like you're doing in your example).
I'm trying to implement a command line server application in Rust.
I'd like to have a list (or hash table) of commands that I can iterate over for printing usage and looking up/executing commands.
My problem is that each Command needs mutable references to the thing the command uses to execute (such as inserting a user into a database or something). The borrow checker obviously doesn't like passing multiple mutable references around. Is there a way to do this? I started by having the execute method take in a reference to the things it needed but then I need a separate list for commands that need different things which sounds like it would get unwieldy.
Here's an example of the problem:
struct SomeStruct {
pub some_field: String,
}
impl SomeStruct {
pub fn new(field: String) -> SomeStruct {
let some_struct = SomeStruct {
some_field: field,
};
return some_struct;
}
pub fn change_field(&mut self) {
self.some_field = "Something else".to_string();
}
}
struct SomeCommand<'a> {
pub some_struct: &'a mut SomeStruct,
}
impl<'a> SomeCommand<'a> {
pub fn new(the_struct: &'a mut SomeStruct) -> SomeCommand {
let some_command = SomeCommand {
some_struct: the_struct,
};
return some_command;
}
pub fn execute(&mut self) {
self.some_struct.change_field();
}
}
fn main() {
let mut some_struct = SomeStruct::new("hey".to_string());
let some_command1 = SomeCommand::new(&mut some_struct);
// Compiler complains because I'm making another mutable binding to some_struct
let some_command2 = SomeCommand::new(&mut some_struct);
}
Is there a better way to do this?
I was trying to do the same in Rust, following examples in book Design Patterns. The problem here is that we need to maintain the generic interface of the Command trait, which means we should not design the trait such that it takes specific objects of certain types. The only solution we are left with is to store a mutable reference to the object itself within the concrete instance of object that implements Command trait. However, if we use &'a mut ..., Rust compiler will not like multiple mutable references to a single object, which means we can only ever have one Command instance for any given object we want to execute the command on, and we can only call execute once.
This can be achieved by using RefCell or Arc<Mutex<>>. I have implemented both and they both work very well. The difference is that RefCell is not thread-safe, so you cannot share the same Command object across threads if you choose to implement it in this way; whereas Arc<Mutex<>> is thread-safe.
Below is my implementation:
trait Command {
fn execute(&mut self); // Execute command.
fn is_reversible(&self) -> bool; // Undoable operation?
fn unexecute(&mut self); // Undo command.
}
// ------ Using RefCell ------
struct ChangeFontSizeCommand<'a> {
text: &'a RefCell<dyn Text>,
old_size: Option<usize>,
new_size: usize
}
impl<'a> Command for ChangeFontSizeCommand<'a> {
// Implementation... (many calls to .borrow() and .borrow_mut())
}
impl<'a> ChangeFontSizeCommand<'a> {
pub fn new(text: &'a RefCell<dyn Text>, new_size: usize) -> Self {
// Implementation...
}
}
// ------ Using Arc and Mutex ------
struct ChangeFontColorCommand {
text: Arc<Mutex<dyn Text>>,
old_color: Option<Color>,
new_color: Color
}
impl Command for ChangeFontColorCommand {
// Implementation... (many calls to .lock().unwrap())
}
impl ChangeFontColorCommand {
pub fn new(text: Arc<Mutex<dyn Text>>, new_color: Color) -> Self {
// Implementation...
}
}
Notice that in both examples, instance of RefCell or Arc<Mutex<>> has to be created outside of object initializer, we cannot pass in a mutable reference and create them inside of the command implementation struct, that would violate Rust's borrow checker rules.
I think passing the mutable reference as a parameter of execute() instead of storing it within SomeCommand is the way to go. Don't keep references living longer than they need to live.
But this is a fairly broad question: I can think of a dozen possible solutions -- it's hard to tell what's best in your case, since your example is pretty generic. Maybe we could be more specific after you told us a little more (maybe a small subset of features you want to implement).
Just looking at the title of your post: docopt is used by many Rust project -- AFAIK even cargo uses it. However, I suspect it doesn't help you with your main design problem.
Besides: in your new methods you can delete some code thanks to implicit returns. This:
pub fn new(field: String) -> SomeStruct {
let some_struct = SomeStruct {
some_field: field,
};
return some_struct;
}
... becomes this:
pub fn new(field: String) -> SomeStruct {
SomeStruct {
some_field: field,
}
}
In C++ it would something like struct A is composed of struct B and some function of B takes a pointer to the parent object A. So function of A calling that function of B will simply pass the this pointer to it. I'm trying this in Rust but failing to get it to work - this is what I want to achieve:
struct A<Type: T> {
composition: Type,
value: usize,
}
impl<Type> A<Type> where Type: T {
fn new(obj: Type) -> A<Type> {
A {
composition: obj,
value: 999,
}
}
fn f(&mut self) {
println!("Value: {:?}", self.value);
}
fn call(&mut self) {
self.composition.f(&mut self);
}
}
trait T {
fn f(&mut self, &mut A<Self>);
}
struct B {
value: usize,
}
impl B {
fn new() -> B {
B { value: 0, }
}
}
impl T for B {
fn f(&mut self, parent: &mut A<B>) {
println!("B::f");
parent.f();
}
}
fn main() {
let objA = A::new(B::new());
// I want call sequence -> A::call() -> B::f() -> A::f()
objA.call();
}
Note that i require mutability in all the functions although in example above it might seem that &mut self in most function parameters don't make much sense. How do it do this in Rust?
This cannot work because you're violating mutable aliasing requirements - you're trying to mutably borrow A and its substructure at the same time:
self.composition.f(self);
// roughtly equivalent to:
let c = &mut self.composition; // borrow substructure
c.f(self /* borrow self */);
(I've removed explicit &mut self because it is incorrect (as it gives you &mut &mut A<...>, but it does not change the whole picture at all.)
This is a natural error in Rust framework. Suppose that f implementation on this particular composition X rewrites composition field on the passed object:
impl T for X {
fn f(&mut self, a: &mut A<X>) {
a.composition = create_x_somehow();
}
}
And suddenly the object this method is called on is destroyed, and self is invalidated!
Naturally, the compiler prevents you from doing this even if you know that you don't modify composition, because such kind of knowledge cannot be encoded statically (especially given that this is a trait method which can be implemented by anyone having access to your trait).
You have essentially two choices in such situations:
reformulate the problem so it does not require such architecture anymore, or
use special language/library constructs to work around such static checks.
The second point is about using such things as Cell/RefCell (they are safe, i.e. don't require unsafe blocks, but they can panic at runtime - probably these can work in your case) or, if nothing else helps, dropping to raw pointers and unsafe code. But, frankly, the first option usually is better: if you design your code based on ownership semantics and aliasing rules enforced by the compiler, almost always the resulting architecture would be of much better quality.