I'm designing a little struct that runs closures for me and I can set them to stop:
pub fn run(&self, f: Box<dyn Fn()>) {
let should_continue = self.should_continue.clone();
self.run_thread = Some(std::thread::spawn(move || {
while should_continue.load(Ordering::Relaxed) {
//f should run fast so `should_continue` is readed frequently
f();
}
}));
}
as you can see, I'm passing Fn in a box, which gives me an error about Box not being shareable between threads. Actually, I don't care about fn once I pass it to this function run, so I wanted to move the closure to this function, since I'll not use it anymore. I cannot mark Fn as send because the f that I'm gonna actually pass does not implement Send.
So, how can I move a closure completely?
//move this closure to inside of run
self.run(||{});
Having a buildable reproduction case rather than code with random unprovided dependencies is useful so here's what I understand of your code.
The error I get is that the dyn Fn can not be sent between threads which is very different than shared: while there are many things which can not be shared (Sync) between threads (they can only be used from one thread at a time) there are also things which must remain on their original thread at all time. Rc for instance, is not Send, because it's not a thread-safe reference-counted pointer sending an Rc to a different thread would break its guarantees, therefore that's not allowed.
dyn Fn is opaque and offers no real guarantee as to what it's doing internally except for, well, being callable multiple times. So as far as the compiler is concerned it could close over something which isn't Send (e.g. a reference to a !Sync type, or an Rc, ...), which means the compiler assumes the Fn isn't Send either.
The solution is simple: just define f: Box<dyn Fn() + Send>, this way within run you guarantee that the function can, in fact, be sent between threads; and the caller to run will get an error if they're trying to send a function which can not be sent.
demo
run_ok uses a trivial closure, there is no issue with sending it over. run_not_ok closes over an Rc, and the function therefore doesn't compile (just uncomment it to see). run_ok2 is the same function as run_not_ok using an Arc instead of the Rc, and compiles fine.
Related
Is there a way to mutably borrow (or move a reference to) some value into a closure and continue using it outside, in a cleaner way?
For example, I have this code:
let queue = Arc::new(RefCell::new(Vec::new()));
let cqueue = Arc::clone(&queue);
EntityEventQueue::register_receiver(&entity_equeue, "position-callback",
Box::new( move |e| {
cqueue.borrow_mut().push(e.clone());
}));
// mutate queue
It works, but I heard that RefCell is bad practice outside some specific uses. Is there a way that I can use queue both inside and outside of the closure?
And if there is not, do you know a better way of implementing this? The one requirement is that the queue must be outside of the EntityEventQueue structure
(I created the register_receiver method, so it can be altered. Its signature is pub fn register_receiver(this: &Arc<RefCell<Self>>, name: &str, callback: Box<dyn FnMut(...) + 'a>)
You should use some synchronization mechanism instead of RefCell. For example a Mutex or a RwLock. Depending on your writing needs. Quick tip is:
One writer (at a time) and several readers -> RwLock
Many writers many readers -> Mutex
Those are std but you have some other synchronization libraries and mechanisms available.
In vulkano, to create a CPUAccessibleBuffer you need give it some data and the CPUAccessibleBuffer::from_data function requires the data to have the 'static lifetime.
I have some data in &[u8] array (created at runtime) that I would like to pass to that function.
However, it errors with this message
argument requires that `data` is borrowed for `'static`
So how can I make the lifetime of the data 'static ?
You should use CpuAccessibleBuffer::from_iter instead, it does the same thing but does not require the collection to be Copy or 'static:
let data: &[u8] = todo!();
let _ = CpuAccessibleBuffer::from_iter(
device,
usage,
host_cached,
data.iter().copied(), // <--- pass like so
);
Or if you actually have a Vec<u8>, you can pass it directly:
let data: Vec<u8> = todo!();
let _ = CpuAccessibleBuffer::from_iter(
device,
usage,
host_cached,
data, // <--- pass like so
);
If you really must create the data at runtime, and you really need to last for 'static, then you can use one of the memory leaking methods such as Box::leak or Vec::leak to deliberately leak a heap allocation and ensure it is never freed.
While leaking memory is normally something one avoids, in this case it's actually a sensible thing to do. If the data must live forever then leaking it is actually the correct thing to do, semantically speaking. You don't want the memory to be freed, not ever, which is exactly what happens when memory is leaked.
Example:
fn require_static_data(data: &'static [u8]) {
unimplemented!()
}
fn main() {
let data = vec![1, 2, 3, 4];
require_static_data(data.leak());
}
Playground
That said, really think over the reallys I led with. Make sure you understand why the code you're calling wants 'static data and ask yourself why your data isn't already 'static.
Is it possible to create the data at compile time? Rust has a powerful build time macro system. It's possible, for example, to use include_bytes! to read in a file and do some processing on it before it's embedded into your executable.
Is there another API you can use, another function call you're not seeing that doesn't require 'static?
(These questions aren't for you specifically, but for anyone who comes across this Q&A in the future.)
If the data is created at runtime, it can't have a static lifetime. Static means that data is present for the whole lifetime of the program, which is necessary in some contexts, especially when threading is involved. One way for data to be static is, as Paul already answered, explicitly declaring it as such, i.e.:
static constant_value: i32 = 0;
However, there's no universally applicable way to make arbitrary data static. This type of inference is made at compile-time by the borrow checker, not by the programmer.
Usually if a function requires 'static (type) arguments (as in this case) it means that anything less could potentially be unsafe, and you need to reorganize the way data flows in and out of your program to provide this type of data safely. Unfortunately, that's not something SO can provide within the scope of this question.
Make a constant with static lifetime:
static NUM: i32 = 18;
This question already has an answer here:
How can I pass a reference to a stack variable to a thread?
(1 answer)
Closed 2 years ago.
I'm quite new to Rust, so I've encountered a few things I'm not used to. One issue that's causing me some grief is related to threads.
I would like to spawn a thread that executes a struct's method, but I'm not able to because the method needs to have a 'static lifetime. I'd prefer the method (and by extension the struct) didn't have a 'static lifetime.
If I know for certain that the thread will exit before the struct's instantiated value is dropped, is there a way to communicate this with Rust? In other words, can I tell Rust that I can guarantee the value will not have been dropped until after the thread exits? Or perhaps is there a way to pass a lifetime to a thread?
If none of this is possible, what can be done instead? I've looked into making the code asynchronous instead, but haven't had any success in fixing the issues described above.
If the method and struct must have a 'static lifetime, how might I go about appropriately specifying this?
Here's a simplified example of the problem:
pub struct Thing {
value: i32,
}
impl Thing {
pub fn new(value: i32) -> Thing {
Thing {
value,
}
}
fn in_thread(&self) {
println!("in thread");
// do things that block the thread
}
pub fn spawn_thread(&self) {
std::thread::spawn(move || {
self.in_thread(); // problem occurs here
});
}
}
If none of this is possible, what can be done instead? I've looked into making the code asynchronous instead, but haven't had any success in fixing the issues described above.
I wouldn't recommend to pass data via references to other threads. Instead, try to design your program so that the thread can own the data. You can do this by either move in the data when spawning a thread, alternatively, you may want to pass data via a Channel.
In a mixture of Rust and C code for a microcontroller, I have a Rust data structure that I have to make known to the C part of my program. I got this working.
pub struct SerialPort {
// ... some attributes ...
}
I tried to refactor the Rust code and wanted to generate the Rust structure in a function that also registers the callbacks in my C Code.
pub fn setup_new_port() -> SerialPort {
let port = SerialPort::new();
port.register_with_c_code();
port
}
This doesn't work because Rust will move the content of the memory of my variable of type SerialPort to a different location when the function returns. The addresses registered in the C code point to the (now freed) stackframe of setup_new_port() while it got moved to the stack frame of the caller, so my C code will access invalid addresses.
Is there any trait that I can implement that gets notified when a move happens? I could adjust the addresses registered in my C code in the implementation of the trait.
I know that I could prevent the move from happening by allocating my structure on the heap, but I would like to avoid this as the code runs on a microcontroller and dynamic memory allocation might not always be present.
No, by design.
Rust was specifically designed without so-called move constructors, in Rust a move is a bitwise copy, allowing many optimizations:
such as the use of C memcpy and realloc,
optimizations based on the fact that no panic! can be called during a move,
...
Actually, there are proposals in C++ to retrofit such a design (is_relocatable), and performance is the main driver.
So what?
Do it like Mutex do! That is, Box whatever should not move, and isolate it in an opaque struct so that nobody can move the content out of the box.
If heap allocation is not available... then you may want to look into PinMut, or otherwise borrow the structure. What is borrowed cannot be moved.
The only solution I see (I do not says that it is the only solution) is to have an arena in the stack, or another kind of placeholder:
#[derive(Default)]
pub struct SerialPort {
_dummy: i32,
}
pub fn setup_new_port(placeholder: &mut Option<SerialPort>) -> &SerialPort {
let port = SerialPort::default();
std::mem::replace(placeholder, Some(port));
let port = placeholder.as_ref().unwrap();
println!("address: {:p}", port);
//port.register_with_c_code();
port
}
fn main() {
let mut placeholder = None;
let port = setup_new_port(&mut placeholder);
println!("address: {:p}", port);
}
The two println! lines print the same value.
I searched in the internet to find a crate that implements an arena in the stack but found nothing so far.
I would split it up.
I'd create an instance of the Serialport somewhere (on the stack) and afterwards call register_with_c_code(). Also I would impl Drop for Serialport, so the c_code would be detached, when the Serialport goes out of scope.
For your Serialport interface, I would not accept any move, but only (mutable) references.
I'm currently struggling with lifetimes in Rust (1.0), especially when it comes to passing structs via channels.
How would I get this simple example to compile:
use std::sync::mpsc::{Receiver, Sender};
use std::sync::mpsc;
use std::thread::spawn;
use std::io;
use std::io::prelude::*;
struct Message<'a> {
text: &'a str,
}
fn main() {
let (tx, rx): (Sender<Message>, Receiver<Message>) = mpsc::channel();
let _handle_receive = spawn(move || {
for message in rx.iter() {
println!("{}", message.text);
}
});
let stdin = io::stdin();
for line in stdin.lock().lines() {
let message = Message {
text: &line.unwrap()[..],
};
tx.send(message).unwrap();
}
}
I get:
error[E0597]: borrowed value does not live long enough
--> src/main.rs:23:20
|
23 | text: &line.unwrap()[..],
| ^^^^^^^^^^^^^ does not live long enough
...
26 | }
| - temporary value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
I can see why this is (line only lives for one iteration of for), but I can't figure out what the right way of doing this is.
Should I, as the compiler hints, try to convert the &str into &'static str?
Am I leaking memory if every line would have a 'static lifetime?
When am I supposed to use 'static anyway? Is it something I should try to avoid or is it perfectly OK?
Is there a better way of passing Strings in structs via channels?
I apologize for those naive questions. I've spent quite some time searching already, but I can't quite wrap my head around it. It's probably my dynamic language background getting in the way :)
As an aside: Is &input[..] for converting a String into a &str considered OK? It's the only stable way I could find to do this.
You can't convert &'a T into &'static T except by leaking memory. Luckily, this is not necessary at all. There is no reason to send borrowed pointers to the thread and keep the lines on the main thread. You don't need the lines on the main thread. Just send the lines themselves, i.e. send String.
If access from multiple threads was necessary (and you don't want to clone), use Arc<String> (in the future, Arc<str> may also work). This way the string is shared between threads, properly shared, so that it will be deallocated exactly when no thread uses it any more.
Sending non-'static references between threads is unsafe because you never know how long the other thread will keep using it, so you don't know when the borrow expires and the object can be freed. Note that scoped threads don't have this problem (which aren't in 1.0 but are being redesigned as we speak) do allow this, but regular, spawned threads do.
'static is not something you should avoid, it is perfectly fine for what it does: Denoting that a value lives for the entire duration the program is running. But if that is not what you're trying to convey, of course it is the wrong tool.
Think about it this way: A thread has no syntactical lifetime, i.e. the thread will not be dropped at the end of code block where it was created. Whatever data you send to the thread, you must be sure that it will live as long as the thread does, which means forever. Which means 'static.
What can go wrong in your case, is if the main loop sends a reference to a thread and destroys the string before it has been handled by the thread. The thread would access invalid memory when dealing with the string.
One option would be to put your lines into some statically allocated container but this would mean that you never can destroy those strings. Generally speaking a bad idea. Another option is to think: does the main thread actually need the line once it is read? What if the main thread transfered responsibility for line to the handling thread?
struct Message {
text: String,
}
for line in stdin.lock().lines() {
let message = Message {
text: line.unwrap(),
};
tx.send(message).unwrap();
}
Now you are transferring ownership (move) from the main thread to the handler thread. Because you move your value, no references are involved and no checks for lifetime apply anymore.