I am new to Rust and trying to make some code async to run a bunch of tasks in parallel. Here is a simplified example:
use futures::future::join_all;
#[tokio::main]
async fn main() {
let mut list = Vec::new();
for i in 1..10 {
let my_str = format!("Value is: {:?}", &i);
let future = do_something(&my_str);
list.push(future);
}
join_all(list).await;
}
async fn do_something(value: &str)
{
println!("value is: {:?}", value);
}
This fails with "Borrowed value does not live long enough" on the do_something(&my_str) call. I can get the code to compile by changing do_something to accept a String instead of an &str. However, it seems a bit strange to require a String when an &str would work. Is there a better pattern to use here? Thanks!
"However, it seems a bit strange to require a String when an &str would work." But an &str can't work here because it only borrows my_str which gets destroyed before the future completes:
for i in 1..10 {
// Create a new `String` and store it in `my_str`
let my_str = format!("Value is: {:?}", &i);
// Create a future that borrows `my_str`. Note that the future is not
// yet started
let future = do_something(&my_str);
// Store the future in `list`
list.push(future);
// Destroy `my_str` since it goes out of scope and wasn't moved.
}
// Run the futures from `list` until they complete. At this point each
// future will try to access the string that they have borrowed, but those
// strings have already been freed!
join_all(list).await;
Instead your do_something should take ownership of the string along with responsibility for freeing it:
use futures::future::join_all;
#[tokio::main]
async fn main() {
let mut list = Vec::new();
for i in 1..10 {
// Create a new `String` and store it in `my_str`
let my_str = format!("Value is: {:?}", &i);
// Create a future and _move_ `my_str` into it.
let future = do_something(my_str);
// Store the future in `list`
list.push(future);
// `my_str` is not destroyed since it was moved into the future.
}
join_all(list).await;
}
async fn do_something(value: String)
{
println!("value is: {:?}", value);
// Destroy `value` since it goes out of scope and wasn't moved.
}
Related
I use Rayons par_iter()to iterate over different variations of the expensive method I need to run. These runs need to access the same set of checked usizes because they all need to add to it and check it from time to time. I also need them all to shutdown when first thread finishes, this is why I have a kill_switch which will force the iterations to exit when its set to true.
let mut checked: HashSet<usize> = HashSet::new();
let mut kill_switch: bool = false;
permutations.par_iter().for_each(|order| {
let board = Board::new(board_map.clone(), order.clone());
let mut bubbles: Vec<(i8, i8)> = Vec::new();
if let Some(bubbles) = board.solve(&mut bubbles, &mut checked, &kill_switch) {
kill_switch = true;
bubbles.into_iter().for_each(|bubble| {
dbg!(bubble);
});
}
})
This is the code I currently have but I get errors for how I'm using checked and kill_switch. How do I make this work?
Errors:
cannot borrow checked as mutable, as it is a captured variable in a Fn closure
cannot borrow as mutable [E0596]
cannot assign to kill_switch, as it is a captured variable in a Fn closure
cannot assign [E0594]
To fix the errors, you will need to use RefCells to wrap the checked and kill_switch variables and use the borrow_mut method to get a mutable reference to them in the closure.
Here is an example of how you can modify your code:
use std::cell::RefCell;
use std::collections::HashSet;
let checked: RefCell<HashSet<usize>> = RefCell::new(HashSet::new());
let kill_switch: RefCell<bool> = RefCell::new(false);
permutations.par_iter().for_each(|order| {
let board = Board::new(board_map.clone(), order.clone());
let mut bubbles: Vec<(i8, i8)> = Vec::new();
if let Some(bubbles) = board.solve(&mut bubbles, &mut checked.borrow_mut(), &mut kill_switch.borrow_mut()) {
*kill_switch.borrow_mut() = true;
bubbles.into_iter().for_each(|bubble| {
dbg!(bubble);
});
}
})
Note that you will also need to add RefCell as a dependency in your project.
I'm new to rust, and am wondering why the following code doesn't result in a:
cannot borrow val as mutable more than once at a time error. It seems like by the time I've reached the second_layer function, I should have three separate references to the same original val variable:
val_ref in the main function body
val_ref2 in the first_layer function body
val_ref3 in the second_layer function body
Any help would be appreciated!
fn first_layer(val_ref2: &mut String)
{
*val_ref2 = String::from("first_layer");
println!("{}", val_ref2);
second_layer(val_ref2);
}
fn second_layer(val_ref3: &mut String)
{
*val_ref3 = String::from("second_layer");
println!("{}", val_ref3);
}
fn main()
{
let mut val = String::from("asdf");
let val_ref: &mut String = &mut val;
first_layer(val_ref);
println!("{}", val_ref);
}
Thanks,
References in a function calling another function temporarily do not exist for the duration of the function call.
Identifiers do not automatically live on in functions called further down the stack. Just because one function calls another does not mean that the callee should have access to the caller's local variables. You would get a not found in this scope if you tried.
What you can (try to) do is create a closure to capture the caller's environment and then reuse one of the caller's variables. But you will find that the compiler complains if you break the borrowing rules.
fn first_layer(val_ref2: &mut String) {
// println!("{}", val); // Cannot use val from main!
*val_ref2 = String::from("first_layer");
println!("{}", val_ref2);
(|val_ref3: &mut String| {
*val_ref3 = String::from("second_layer");
println!("{}", val_ref3);
println!("{}", val_ref2); // This is not allowed
})(val_ref2);
}
fn main() {
let mut val = String::from("asdf");
let val_ref: &mut String = &mut val;
first_layer(val_ref);
println!("{}", val_ref);
}
Another way to think about it is with inlining. If you inline your function second_layer into first_layer you get:
fn first_layer(val_ref2: &mut String)
{
*val_ref2 = String::from("first_layer");
println!("{}", val_ref2);
*val_ref2 = String::from("second_layer");
println!("{}", val_ref2);
}
This is totally fine!
So then the code with the last two lines abstracted to its own function should also be totally fine.
I have simple client/server application. I am receiving message on the server side from client but I want to send that response to the channel from server to other file and I am receiving error "borrowed value does not live long enough".
I have searched in the stack overflow for similar previous questions but not getting enough understanding of lifetime. Is there a good documentation or if simple example available on this topic?
For now if someone can help me to fix this code (may be edit the portion of code which needs to fix) that would be helpful.
Thanks in advance.
Server side:
use std::os::unix::net::UnixDatagram;
use std::path::Path;
fn unlink_socket (path: impl AsRef<Path>) {
let path = path.as_ref();
if Path::new(path).exists() {
let result = std::fs::remove_file(path);
match result {
Err(e) => {
println!("Couldn't remove the file: {:?}", e);
},
_ => {}
}
}
}
pub fn tcp_datagram_server() {
pub static FILE_PATH: &'static str = "/tmp/datagram.sock";
let (tx, rx) = mpsc::channel();
let mut buf = vec![0; 1024];
unlink_socket(FILE_PATH);
let socket = match UnixDatagram::bind(FILE_PATH) {
Ok(socket) => socket,
Err(e) => {
println!("Couldn't bind: {:?}", e);
return;
}
};
println!("Waiting for client to connect...");
loop {
let received_bytes = socket.recv(buf.as_mut_slice()).expect("recv function failed");
println!("Received {:?}", received_bytes);
let received_message = from_utf8(buf.as_slice()).expect("utf-8 convert failed");
tx.clone().send(received_message);
}
}
fn main() {
tcp_datagram_server();
}
client side:
use std::sync::mpsc;
use std::os::unix::net::UnixDatagram;
use std::path::Path;
use std::io::prelude::*;
pub fn tcp_datagram_client() {
pub static FILE_PATH: &'static str = "/tmp/datagram.sock";
let socket = UnixDatagram::unbound().unwrap();
match socket.connect(FILE_PATH) {
Ok(socket) => socket,
Err(e) => {
println!("Couldn't connect: {:?}", e);
return;
}
};
println!("TCP client Connected to TCP Server {:?}", socket);
loop {
socket.send(b"Hello from client to server").expect("recv function failed");
}
}
fn main() {
tcp_datagram_client();
}
Error I am getting
error[E0597]: `buf` does not live long enough
--> src/unix_datagram_server.rs:38:42
|
38 | let received_message = from_utf8(buf.as_slice()).expect("utf-8 convert failed");
| ^^^ borrowed value does not live long enough
...
41 | }
| -
| |
| `buf` dropped here while still borrowed
| borrow might be used here, when `tx` is dropped and runs the `Drop` code for type `std::sync::mpsc::Sender`
|
= note: values in a scope are dropped in the opposite order they are defined
error: aborting due to previous error; 8 warnings emitted
For now if someone can help me to fix this code (may be edit the portion of code which needs to fix) that would be helpful.
Well the message seems rather clear. send does exactly what it says it does, it sends the parameter through the channel. This means the data must live long enough and remain valid "forever" (it needs to be alive and valid in the channel, as well as when fetched from it by the receiver).
That is not the case here. rustc can't understand that the function never returns, and it can panic anyway which will end up the same: the function will terminate, which will invalidate buf. Since received_message borrows buf, that means received_message can't be valid after the function has terminated. But at that point the message would still be in the channel waiting to be read (or retrieved by the receiver doing who knows what).
Therefore your construction is not allowed.
A second issue is that you're overwriting the buffer data on every loop, which has the same effect of breaking the message you sent during the previous iteration, and thus is not correct either. Though Rust won't let you do that either: if you work around the first error it will tell you that there's an outstanding shared borrow (the message sent through the channel) so you can't modify the backing buffer in the following iteration.
The solution is quite simple: have each iteration create an owned string (copying the current iteration's message) and send that through the channel:
tx.clone().send(received_message.to_string());
Also, these are more style / inefficiency remarks but:
The clone() on tx is completely redundant. The point of having a sender that is Clone is being able to send from multiple threads (hence mp in the channel name, that's for multiple producers). Here you have a single thread, the original sender works fine.
.as_slice() and .as_mut_slice() are rarely used unless necessary, which they aren't here: array references coerce to slices, so you can just use &mut buf and &buf. And why are you calling Path::new on something that's already a path? It doesn't do anything but it's not useful either.
It is rather annoying that your snippet is missing multiple imports and thus doesn't even compile as is.
From more of a unixy perspective, errors are usually printed on stderr. In Rust, eprintln does that for you (otherwise working in the same way println does). And I don't understand the purpose of marking a lexically nested static pub. Since the static is inside the function it's not even visible to the function's siblings, to say nothing of external callers. As a result I'd end up with this:
use std::os::unix::net::UnixDatagram;
use std::path::Path;
use std::sync::mpsc;
use std::str::from_utf8;
fn unlink_socket (path: impl AsRef<Path>) {
let path = path.as_ref();
if path.exists() {
if let Err(e) = std::fs::remove_file(path) {
eprintln!("Couldn't remove the file: {:?}", e);
}
}
}
static FILE_PATH: &'static str = "/tmp/datagram.sock";
pub fn tcp_datagram_server() {
unlink_socket(FILE_PATH);
let socket = match UnixDatagram::bind(FILE_PATH) {
Ok(socket) => socket,
Err(e) => {
eprintln!("Couldn't bind: {:?}", e);
return;
}
};
let (tx, _) = mpsc::channel();
let mut buf = vec![0; 1024];
println!("Waiting for client to connect...");
loop {
let received_bytes = socket.recv(&mut buf).expect("recv function failed");
println!("Received {:?}", received_bytes);
let received_message = from_utf8(&buf).expect("utf-8 convert failed");
tx.send(received_message.to_string());
}
}
There's a hint in the compiler message, that values in a scope are dropped in the opposite order they are defined in, and in the example, buf is defined after tx, which means it will be dropped before tx. Since a reference to buf (in the form of received_message) is passed to tx.send(), then buf should live longer that tx, and therefore switching the definition order will fix this particular error (ie. switch lines 19 and 20).
I want to wait for a backend thread (Like this but in my case the backend manages a database which I want to close properly before the application actually exits) to finish (e.g. join it) after application.run() has finished.
My actual non working main.rs (the closure needs to be non-mut)
the thread to wait for
use gio::prelude::*;
use gtk::prelude::*;
use gtk::{ApplicationWindow, Label};
use std::env::args;
use std::thread;
fn main() {
let application = gtk::Application::new(
Some("com.github.gtk-rs.examples.communication_thread"),
Default::default(),
)
.expect("Initialization failed...");
let (thr, mut receiver) = start_communication_thread();
application.connect_activate(move |application| {
build_ui(application, receiver.take().unwrap())
});
application.run(&args().collect::<Vec<_>>());
thr.join();
}
fn build_ui(application: >k::Application, receiver: glib::Receiver<String>) {
let window = ApplicationWindow::new(application);
let label = Label::new(None);
window.add(&label);
spawn_local_handler(label, receiver);
window.show_all();
}
/// Spawn channel receive task on the main event loop.
fn spawn_local_handler(label: gtk::Label, receiver: glib::Receiver<String>) {
receiver.attach(None, move |item| {
label.set_text(&item);
glib::Continue(true)
});
}
/// Spawn separate thread to handle communication.
fn start_communication_thread() -> (thread::JoinHandle<()>, Option<glib::Receiver<String>>) {
let (sender, receiver) = glib::MainContext::channel(glib::PRIORITY_DEFAULT);
let thr = thread::spawn(move || {
let mut counter = 0;
loop {
let data = format!("Counter = {}!", counter);
println!("Thread received data: {}", data);
if sender.send(data).is_err() {
break
}
counter += 1;
thread::sleep(std::time::Duration::from_millis(100));
}
});
(thr, Some(receiver))
}
As mentioned above, the only error remaining is that application.connect_activate() takes an Fn closure, the current implementation is FnMut.
The error message is:
error[E0596]: cannot borrow `receiver` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:17:31
|
17 | build_ui(application, receiver.take().unwrap())
| ^^^^^^^^ cannot borrow as mutable
So you cannot use "receiver" mutably, which is necessary for you to take() its contents.
But if you wrap the receiver inside a Cell, then you can access the immutable Cell's contents mutably. So add this line directly after the line with start_communication_thread():
let receiver = Cell::new(receiver);
There might be some more correct answer as I am only a beginner at Rust, but at least it seems to work.
Please note that this changes the take() call to be called against the Cell instead of Option, whose implementation has the same effect, replacing the Cell's contents with None.
I am reading a string from a file, splitting it by lines into a vector and then I want to do something with the extracted lines in separate threads. Like this:
use std::fs::File;
use std::io::prelude::*;
use std::thread;
fn main() {
match File::open("data") {
Ok(mut result) => {
let mut s = String::new();
result.read_to_string(&mut s);
let k : Vec<_> = s.split("\n").collect();
for line in k {
thread::spawn(move || {
println!("nL: {:?}", line);
});
}
}
Err(err) => {
println!("Error {:?}",err);
}
}
}
Of course this throws an error, because s will go out of scope before the threads are started:
s` does not live long enough
main.rs:9 let k : Vec<_> = s.split("\n").collect();
^
What can I do now? I've tried many things like Box or Arc, but I couldn't get it working. I somehow need to create a copy of s which also lives in the threads. But how do I do that?
The problem, fundamentally, is that line is a borrowed slice into s. There's really nothing you can do here, since there's no way to guarantee that each line will not outlive s itself.
Also, just to be clear: there is absolutely no way in Rust to "extend the lifetime of a variable". It simply cannot be done.
The simplest way around this is to go from line being borrowed to owned. Like so:
use std::thread;
fn main() {
let mut s: String = "One\nTwo\nThree\n".into();
let k : Vec<String> = s.split("\n").map(|s| s.into()).collect();
for line in k {
thread::spawn(move || {
println!("nL: {:?}", line);
});
}
}
The .map(|s| s.into()) converts from &str to String. Since a String owns its contents, it can be safely moved into each thread's closure, and will live independently of the thread that created it.
Note: you could do this in nightly Rust using the new scoped thread API, but that is still unstable.