I am trying to write a section of code that does a web request every few seconds and updates a struct in rust. I have so far tried using a thread to accomplish this task however I am having some real trouble moving variables into the thread and assigning the results of the web request to a struct once it completes.
I have provided a small code sample below that doesn't include a web request but does demonstrate the problem I am having moving variables around.
Code:
use std::thread;
#[derive(Debug, Clone)]
struct input_holder {
i : String,
}
fn main() {
// pass this string into the thread loop.
let input = String::from("This is a string");
let input_loop = input.clone();
//assign the output of the method to this struct.
let ih = input_holder {
i : input
};
thread::spawn(|| {
let runtime = tokio::runtime::Runtime::new().unwrap();
loop {
let _ = runtime.block_on(runtime.spawn(async move {
let _new_input = match update_string(input_loop.clone(), ih.clone()){
Some(ni) => ni,
None => String::from("None")
};
}));
}
});
}
//This is where I would do the web request. I can test this method outside of the thread and it works as expected.
pub fn update_string(_input: String, mut ih: input_holder) -> Option<String> {
ih.i = String::from("This is an updated string");
Some(String::from("This is an updated string"))
}
Below is the error message I am running into:
error[E0382]: use of moved value: `ih`
--> src/main.rs:64:63
|
64 | let _ = runtime.block_on(runtime.spawn(async move {
| _______________________________________________________________^
65 | | let _new_input = match update_string(input_loop.clone(), ih.clone()){
| | -- use occurs due to use in generator
66 | | Some(ni) => ni,
67 | | None => String::from("None")
68 | | };
69 | | }));
| |_____________^ value moved here, in previous iteration of loop
|
= note: move occurs because `ih` has type `input_holder`, which does not implement the `Copy` trait
There doesn't seem to be anyway for me to pass ih that I am aware of, I don't know what use occurs due to use in generator means and Google does not seem to have much on this error (there are a bunch of options for move occurs due to use in generator but nothing for this error).
I have tried cloning and not cloning, borrowing, removing the move from the loop (I should note here I can't implement the copy trait on the struct). How are you supposed to get variables in and out of a thread loop like this?
I think I might not be understanding something about how you are supposed to move variables around in rust because in general I find myself needing to make lots and lots of copies of any value I plan to pass between methods.
Related
In my application (a compiler), I'd like to create data cyclic data structures of various kinds throughout my program's execution that all have the same lifetime (in my case, lasting until the end of compilation). In addition,
I don't need to worry about multi-threading
I only need to append information - no need to delete or garbage collect
I only need immutable references to my data
This seemed like a good use case for an Arena, but I saw that this would require passing the arena around to every function in my program, which seemed like a large overhead.
So instead I found a macro called thread_local! that I can use to define global data. Using this, I thought I might be able to define a custom type that wraps an index into the array, and implement Deref on that type:
use std::cell::RefCell;
enum Floop {
CaseA,
CaseB,
CaseC(FloopRef),
CaseD(FloopRef),
CaseE(Vec<FloopRef>),
}
thread_local! {
static FLOOP_ARRAY: RefCell<Vec<Box<Floop>>> = RefCell::new(Vec::new());
}
pub struct FloopRef(usize);
impl std::ops::Deref for FloopRef {
type Target = Floop;
fn deref(&self) -> &Self::Target {
return FLOOP_ARRAY.with(|floops| &floops.borrow()[self.0]);
}
}
pub fn main() {
// initialize some data
FLOOP_ARRAY.with(|floops| {
floops.borrow_mut().push(Box::new(Floop::CaseA));
let idx = floops.borrow_mut().len();
floops.borrow_mut().push(Box::new(Floop::CaseC(FloopRef(idx))));
});
}
Unfortunately I run into lifetime errors:
error: lifetime may not live long enough
--> src/main.rs:20:36
|
20 | return FLOOP_ARRAY.with(|floops| &floops.borrow()[self.0]);
| ------- ^^^^^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'2`
| | |
| | return type of closure is &'2 Box<Floop>
| has type `&'1 RefCell<Vec<Box<Floop>>>`
error[E0515]: cannot return value referencing temporary value
--> src/main.rs:20:36
|
20 | return FLOOP_ARRAY.with(|floops| &floops.borrow()[self.0]);
| ^---------------^^^^^^^^
| ||
| |temporary value created here
| returns a value referencing data owned by the current function
What I'd like to tell the compiler is that I promise I'm never going to remove entries from the Array and that I'm not going to share values across threads and that the array will last until the end of the program so that I can in essence just return a &'static reference to a Floop object. But Rust doesn't seem to be convinced this is safe.
Is there any kind of Rust helper library that would let me do something like this? Or are there safety holes even when I guarantee I only append / only use data with a single thread?
If you would have a reference, you could send the data to another thread, then watch it after it has been dropped because the creating thread was finished.
Even if you would solve this problem, this would still require unsafe code, as the compiler can't be convinced that growing the Vec won't invalidate existing references. This is true in this case since you're using Box, but the compiler cannot know that.
If you pinky promise to never touch the data after the creating thread has finished, you can use the following code. Note that this code is technically UB as when the Vec will grow, we will move all Boxes, and at least currently, moving a Box invalidates all references deriven from it:
enum Floop {
CaseA,
CaseB,
CaseC(&'static Floop),
CaseD(&'static Floop),
CaseE(Vec<&'static Floop>),
}
thread_local! {
static FLOOP_ARRAY: RefCell<Vec<Box<Floop>>> = RefCell::new(Vec::new());
}
fn alloc_floop(floop: Floop) -> &'static mut Floop {
FLOOP_ARRAY.with(|floops| {
let mut floops = floops.borrow_mut();
floops.push(Box::new(floop));
let floop = &mut **floops.last_mut().unwrap() as *mut Floop;
// SAFETY: We never access the data after it has been dropped, and we are
// the only who access this `Box` as we access a `Box` only immediately
// after pushing it.
unsafe { &mut *floop }
})
}
fn main() {
let floop_a = alloc_floop(Floop::CaseA);
let floop_b = alloc_floop(Floop::CaseC(floop_a));
}
A better solution would be something like a thread-safe arena that you can use in a static, but sadly, I found no crate that implements that.
My program is processing commands from different users. Each user has its own state that is modified by the commands he issues.
I use a HashMap to store the state of the users:
let mut states = HashMap::new();
A state is a struct:
pub struct State {
// details omitted
}
The program looks like this:
// loop
// current user is identified by 1st char of command
current_user = command[0..1].to_string()
match states.get(¤t_user) {
Some(&state) => {
println!("Switching User...");
current_state = state;
}
_ => {
println!("Creating User...");
current_state = State{...details omitted...};
states.insert(current_user, current_state);
}
}
// execute command in the State of the user
But I get the following error.
error[E0507]: cannot move out of a shared reference
42 | match states.get(¤t_user) {
| ^^^^^^^^^^^^^^^^^^^^^^^^
43 | Some(&state) => {
| ----
| |
| data moved here
| move occurs because `state` has type `Repository`, which does not implement the `Copy` trait
See Playground.
How can the program be fixed ? Do I need to use another structure than a HashMap ?
The trick is to use states.entry(..).or_insert(..) instead of states.get()...
The code is actually simpler (see playground)
current_state = states.entry(current_user)
.or_insert(State{v: String::from("P")});
I have a struct Screen that handles the manipulation of windows, drawing to the screen, and handling the input.
One of the methods, that handles the input, is as follows:
pub fn prompt(&mut self) {
for e in self.stdin.lock().events() {
match e.unwrap() {
Event::Key(Key::Char('\n')) => break,
Event::Key(Key::Char(c)) => {
if c.is_alphanumeric() {
self.overlay.putc(self.cursor_y, self.cursor_x, c);
self.move_cursor(self.cursor_y, self.cursor_x + 1);
}
}
_ => ()
}
}
}
The issue is that self.move_cursor() takes a mutable reference to self, and modifies the cursor position within the Screen.
The code produces the following error:
--> src/screen.rs:190:25
|
184 | for e in self.stdin.lock().events() {
| --------------------------
| |
| immutable borrow occurs here
| immutable borrow later used here
...
190 | self.move_cursor(self.cursor_y, self.cursor_x + 1);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
The methods operate on different members, so it is safe.
I understand the issue and I'm thinking about how to tackle it. One option that I thought of is to wrap every member of the Screen in RefCell and pass an immutable reference &self to every method of Screen. This does not seem to me like a good solution, though. I was also thinking to simply use unsafe {} but was unable to use it properly in this context.
My question is: what would be the best, or idiomatic, way to handle this; or how to reorganize the struct.
For extra context: I'm drawing to the terminal, using the library termion.
This code does not compile...
fn main() {
let data = "hi".to_string();
let wrap = &data;
std::thread::spawn(move || println!("{}", wrap));
}
...because data doesn't live inside the spawned thread:
error[E0597]: `data` does not live long enough
--> src/main.rs:3:16
|
3 | let wrap = &data;
| ^^^^^ borrowed value does not live long enough
4 | std::thread::spawn(move || println!("{}", wrap));
| ------------------------------------------------ argument requires that `data` is borrowed for `'static`
5 | }
| - `data` dropped here while still borrowed
Why doesn't Rust move data like it does wrap? Is there any way to force data to be moved along with with wrap?
My real code looks more like this. I accept a message, parse it, and then send it to a thread for processing.
struct Message {
data: Vec<u8>,
}
let message = Message {
data: "{\"name\":\"Rust\"}".to_string(),
};
#[derive(Deserialize)]
struct Parsed<'a> {
name: &'a str,
}
let parsed: Parsed = serde_json::from_slice(&message.data).unwrap();
std::thread::Builder::new()
.name(parsed.name) // note: need some of the parsed data prior to moving into the thread...so cannot solve with JSON parsing inside thread
.spawn(move || println("{}", parsed.name));
I know I could modify my Parsed struct to use Strings instead of &'a str, but this reduces efficiency when the compiler should be able to move data just fine.
Why doesn't Rust move data like it does wrap
Several reasons - take your pick:
Because wrap is captured, not data, and the closure moves what is captured.
Because the move of data would immediately invalidate both wrap itself and other references to data if they exist. Also, the enclosing scope might itself need data for further access.
Because then there would be no way to have a closure that captures some values by move and some by reference. Currently you can make a closure move but still elect to capture certain values by reference simply by creating an explicit reference and capturing that. The feature you propose would make that impossible.
I have a lazy static struct that I want to be able to set to some random value in the beginning of the execution of the program, and then get later. This little silly snippet can be used as an example:
use lazy_static::lazy_static;
use std::sync::RwLock;
struct Answer(i8);
lazy_static! {
static ref ANSWER: RwLock<Option<Answer>> = RwLock::new(None);
}
fn answer_question() {
*ANSWER.write().unwrap() = Some(Answer(42));
}
fn what_is_the_answer() -> &'static Answer {
ANSWER
.read()
.unwrap()
.as_ref()
.unwrap()
}
This code fails to compile:
error[E0515]: cannot return value referencing temporary value
--> src/lib.rs:15:5
|
15 | ANSWER
| _____^
| |_____|
| ||
16 | || .read()
17 | || .unwrap()
| ||_________________- temporary value created here
18 | | .as_ref()
19 | | .unwrap()
| |__________________^ returns a value referencing data owned by the current function
I know you can not return a reference to a temporary value. But I want to return a reference to ANSWER which is static - the very opposite of temporary! I guess it is the RwLockReadGuard that the first call to unwrap returns that is the problem?
I can get the code to compile by changing the return type:
fn what_is_the_answer() -> RwLockReadGuard<'static, Option<Answer>> {
ANSWER
.read()
.unwrap()
}
But now the calling code becomes very unergonomic - I have to do two extra calls to get to the actual value:
what_is_the_answer().as_ref().unwrap()
Can I somehow return a reference to the static ANSWER from this function? Can I get it to return a RwLockReadGuard<&Answer> maybe by mapping somehow?
once_cell is designed for this: use .set(...).unwrap() in answer_question and .get().unwrap() in what_is_the_answer.
As far as I understand your intention, the value of Answer can't be computed while it is being initialized in the lazy_static but depends on parameters known only when answer_question is called. The following may not be the most elegant solution, yet it allows for having a &'static-reference to a value that depends on parameters only known at runtime.
The basic approach is to use two lazy_static-values, one of which serves as a "proxy" to do the necessary synchronization, the other being the value itself. This avoids having to access multiple layers of locks and unwrapping of Option-values whenever you access ANSWER.
The ANSWER-value is initialized by waiting on a CondVar, which will signal when the value has been computed. The value is then placed in the lazy_static and from then on unmovable. Hence &'static is possible (see get_the_answer()). I have chosen String as the example-type. Notice that accessing ANSWER without calling generate_the_answer() will cause the initialization to wait forever, deadlocking the program.
use std::{sync, thread};
lazy_static::lazy_static! {
// A proxy to synchronize when the value is generated
static ref ANSWER_PROXY: (sync::Mutex<Option<String>>, sync::Condvar) = {
(sync::Mutex::new(None), sync::Condvar::new())
};
// The actual value, which is initialized from the proxy and stays in place
// forever, hence allowing &'static access
static ref ANSWER: String = {
let (lock, cvar) = &*ANSWER_PROXY;
let mut answer = lock.lock().unwrap();
loop {
// As long as the proxy is None, the answer has not been generated
match answer.take() {
None => answer = cvar.wait(answer).unwrap(),
Some(answer) => return answer,
}
}
};
}
// Generate the answer and place it in the proxy. The `param` is just here
// to demonstrate we can move owned values into the proxy
fn generate_the_answer(param: String) {
// We don't need a thread here, yet we can
thread::spawn(move || {
println!("Generating the answer...");
let mut s = String::from("Hello, ");
s.push_str(¶m);
thread::sleep(std::time::Duration::from_secs(1));
let (lock, cvar) = &*ANSWER_PROXY;
*lock.lock().unwrap() = Some(s);
cvar.notify_one();
println!("Answer generated.");
});
}
// Nothing to see here, except that we have a &'static reference to the answer
fn get_the_answer() -> &'static str {
println!("Asking for the answer...");
&ANSWER
}
fn main() {
println!("Hello, world!");
// Accessing `ANSWER` without generating it will deadlock!
//get_the_answer();
generate_the_answer(String::from("John!"));
println!("The answer is \"{}\"", get_the_answer());
// The second time a value is generated, noone is listening.
// This is the flipside of `ANSWER` being a &'static
generate_the_answer(String::from("Peter!"));
println!("The answer is still \"{}\"", get_the_answer());
}