Rust: How to fix borrowed value does not live long enough - rust

I have simple client/server application. I am receiving message on the server side from client but I want to send that response to the channel from server to other file and I am receiving error "borrowed value does not live long enough".
I have searched in the stack overflow for similar previous questions but not getting enough understanding of lifetime. Is there a good documentation or if simple example available on this topic?
For now if someone can help me to fix this code (may be edit the portion of code which needs to fix) that would be helpful.
Thanks in advance.
Server side:
use std::os::unix::net::UnixDatagram;
use std::path::Path;
fn unlink_socket (path: impl AsRef<Path>) {
let path = path.as_ref();
if Path::new(path).exists() {
let result = std::fs::remove_file(path);
match result {
Err(e) => {
println!("Couldn't remove the file: {:?}", e);
},
_ => {}
}
}
}
pub fn tcp_datagram_server() {
pub static FILE_PATH: &'static str = "/tmp/datagram.sock";
let (tx, rx) = mpsc::channel();
let mut buf = vec![0; 1024];
unlink_socket(FILE_PATH);
let socket = match UnixDatagram::bind(FILE_PATH) {
Ok(socket) => socket,
Err(e) => {
println!("Couldn't bind: {:?}", e);
return;
}
};
println!("Waiting for client to connect...");
loop {
let received_bytes = socket.recv(buf.as_mut_slice()).expect("recv function failed");
println!("Received {:?}", received_bytes);
let received_message = from_utf8(buf.as_slice()).expect("utf-8 convert failed");
tx.clone().send(received_message);
}
}
fn main() {
tcp_datagram_server();
}
client side:
use std::sync::mpsc;
use std::os::unix::net::UnixDatagram;
use std::path::Path;
use std::io::prelude::*;
pub fn tcp_datagram_client() {
pub static FILE_PATH: &'static str = "/tmp/datagram.sock";
let socket = UnixDatagram::unbound().unwrap();
match socket.connect(FILE_PATH) {
Ok(socket) => socket,
Err(e) => {
println!("Couldn't connect: {:?}", e);
return;
}
};
println!("TCP client Connected to TCP Server {:?}", socket);
loop {
socket.send(b"Hello from client to server").expect("recv function failed");
}
}
fn main() {
tcp_datagram_client();
}
Error I am getting
error[E0597]: `buf` does not live long enough
--> src/unix_datagram_server.rs:38:42
|
38 | let received_message = from_utf8(buf.as_slice()).expect("utf-8 convert failed");
| ^^^ borrowed value does not live long enough
...
41 | }
| -
| |
| `buf` dropped here while still borrowed
| borrow might be used here, when `tx` is dropped and runs the `Drop` code for type `std::sync::mpsc::Sender`
|
= note: values in a scope are dropped in the opposite order they are defined
error: aborting due to previous error; 8 warnings emitted

For now if someone can help me to fix this code (may be edit the portion of code which needs to fix) that would be helpful.
Well the message seems rather clear. send does exactly what it says it does, it sends the parameter through the channel. This means the data must live long enough and remain valid "forever" (it needs to be alive and valid in the channel, as well as when fetched from it by the receiver).
That is not the case here. rustc can't understand that the function never returns, and it can panic anyway which will end up the same: the function will terminate, which will invalidate buf. Since received_message borrows buf, that means received_message can't be valid after the function has terminated. But at that point the message would still be in the channel waiting to be read (or retrieved by the receiver doing who knows what).
Therefore your construction is not allowed.
A second issue is that you're overwriting the buffer data on every loop, which has the same effect of breaking the message you sent during the previous iteration, and thus is not correct either. Though Rust won't let you do that either: if you work around the first error it will tell you that there's an outstanding shared borrow (the message sent through the channel) so you can't modify the backing buffer in the following iteration.
The solution is quite simple: have each iteration create an owned string (copying the current iteration's message) and send that through the channel:
tx.clone().send(received_message.to_string());
Also, these are more style / inefficiency remarks but:
The clone() on tx is completely redundant. The point of having a sender that is Clone is being able to send from multiple threads (hence mp in the channel name, that's for multiple producers). Here you have a single thread, the original sender works fine.
.as_slice() and .as_mut_slice() are rarely used unless necessary, which they aren't here: array references coerce to slices, so you can just use &mut buf and &buf. And why are you calling Path::new on something that's already a path? It doesn't do anything but it's not useful either.
It is rather annoying that your snippet is missing multiple imports and thus doesn't even compile as is.
From more of a unixy perspective, errors are usually printed on stderr. In Rust, eprintln does that for you (otherwise working in the same way println does). And I don't understand the purpose of marking a lexically nested static pub. Since the static is inside the function it's not even visible to the function's siblings, to say nothing of external callers. As a result I'd end up with this:
use std::os::unix::net::UnixDatagram;
use std::path::Path;
use std::sync::mpsc;
use std::str::from_utf8;
fn unlink_socket (path: impl AsRef<Path>) {
let path = path.as_ref();
if path.exists() {
if let Err(e) = std::fs::remove_file(path) {
eprintln!("Couldn't remove the file: {:?}", e);
}
}
}
static FILE_PATH: &'static str = "/tmp/datagram.sock";
pub fn tcp_datagram_server() {
unlink_socket(FILE_PATH);
let socket = match UnixDatagram::bind(FILE_PATH) {
Ok(socket) => socket,
Err(e) => {
eprintln!("Couldn't bind: {:?}", e);
return;
}
};
let (tx, _) = mpsc::channel();
let mut buf = vec![0; 1024];
println!("Waiting for client to connect...");
loop {
let received_bytes = socket.recv(&mut buf).expect("recv function failed");
println!("Received {:?}", received_bytes);
let received_message = from_utf8(&buf).expect("utf-8 convert failed");
tx.send(received_message.to_string());
}
}

There's a hint in the compiler message, that values in a scope are dropped in the opposite order they are defined in, and in the example, buf is defined after tx, which means it will be dropped before tx. Since a reference to buf (in the form of received_message) is passed to tx.send(), then buf should live longer that tx, and therefore switching the definition order will fix this particular error (ie. switch lines 19 and 20).

Related

Wait for backend thread to finish after application.run in Rust

I want to wait for a backend thread (Like this but in my case the backend manages a database which I want to close properly before the application actually exits) to finish (e.g. join it) after application.run() has finished.
My actual non working main.rs (the closure needs to be non-mut)
the thread to wait for
use gio::prelude::*;
use gtk::prelude::*;
use gtk::{ApplicationWindow, Label};
use std::env::args;
use std::thread;
fn main() {
let application = gtk::Application::new(
Some("com.github.gtk-rs.examples.communication_thread"),
Default::default(),
)
.expect("Initialization failed...");
let (thr, mut receiver) = start_communication_thread();
application.connect_activate(move |application| {
build_ui(application, receiver.take().unwrap())
});
application.run(&args().collect::<Vec<_>>());
thr.join();
}
fn build_ui(application: &gtk::Application, receiver: glib::Receiver<String>) {
let window = ApplicationWindow::new(application);
let label = Label::new(None);
window.add(&label);
spawn_local_handler(label, receiver);
window.show_all();
}
/// Spawn channel receive task on the main event loop.
fn spawn_local_handler(label: gtk::Label, receiver: glib::Receiver<String>) {
receiver.attach(None, move |item| {
label.set_text(&item);
glib::Continue(true)
});
}
/// Spawn separate thread to handle communication.
fn start_communication_thread() -> (thread::JoinHandle<()>, Option<glib::Receiver<String>>) {
let (sender, receiver) = glib::MainContext::channel(glib::PRIORITY_DEFAULT);
let thr = thread::spawn(move || {
let mut counter = 0;
loop {
let data = format!("Counter = {}!", counter);
println!("Thread received data: {}", data);
if sender.send(data).is_err() {
break
}
counter += 1;
thread::sleep(std::time::Duration::from_millis(100));
}
});
(thr, Some(receiver))
}
As mentioned above, the only error remaining is that application.connect_activate() takes an Fn closure, the current implementation is FnMut.
The error message is:
error[E0596]: cannot borrow `receiver` as mutable, as it is a captured variable in a `Fn` closure
--> src/main.rs:17:31
|
17 | build_ui(application, receiver.take().unwrap())
| ^^^^^^^^ cannot borrow as mutable
So you cannot use "receiver" mutably, which is necessary for you to take() its contents.
But if you wrap the receiver inside a Cell, then you can access the immutable Cell's contents mutably. So add this line directly after the line with start_communication_thread():
let receiver = Cell::new(receiver);
There might be some more correct answer as I am only a beginner at Rust, but at least it seems to work.
Please note that this changes the take() call to be called against the Cell instead of Option, whose implementation has the same effect, replacing the Cell's contents with None.

Split a futures connection into a sink and a stream and use them in two different tasks

I'm experimenting with the futures API using the websocket library. I have this code:
use futures::future::Future;
use futures::future;
use futures::sink::Sink;
use futures::stream::Stream;
use futures::sync::mpsc::channel;
use futures::sync::mpsc::{Sender, Receiver};
use tokio_core::reactor::Core;
use websocket::{ClientBuilder, OwnedMessage};
pub fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let handle_clone = handle.clone();
let (send, recv): (Sender<String>, Receiver<String>) = channel(100);
let f = ClientBuilder::new("wss://...")
.unwrap()
.async_connect(None, &handle_clone)
.map_err(|e| println!("error: {:?}", e))
.map(|(duplex, _)| duplex.split())
.and_then(move |(sink, stream)| {
// this task consumes the channel, writes messages to the websocket
handle_clone.spawn(future::loop_fn(recv, |recv: Receiver<String>| {
sink.send(OwnedMessage::Close(None))
.and_then(|_| future::ok(future::Loop::Break(())))
.map_err(|_| ())
}));
// the main tasks listens the socket
future::loop_fn(stream, |stream| {
stream
.into_future()
.and_then(|_| future::ok(future::Loop::Break(())))
.map_err(|_| ())
})
});
loop {
core.turn(None)
}
}
After connecting to the server, I want to run "listener" and "sender" tasks without one blocking the other one. The problem is I can't use sink in the new task, it fails with:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/slack_conn.rs:29:17
|
25 | .and_then(move |(sink, stream)| {
| ---- captured outer variable
...
29 | sink.send(OwnedMessage::Close(None))
| ^^^^ cannot move out of captured outer variable in an `FnMut` closure
I could directly use duplex to send and receive, but that leads to worse errors.
Any ideas on how to make this work? Indeed, I'd be happy with any futures code that allows me to non-blockingly connect to a server and spawn two async tasks:
one that reads from the connection and takes some action (prints to screen etc.)
one that reads from a mpsc channel and writes to the connection
It's fine if I have to write it in a different style.
SplitSink implements Sink which defines send to take ownership:
fn send(self, item: Self::SinkItem) -> Send<Self>
where
Self: Sized,
On the other hand, loop_fn requires that the closure be able to be called multiple times. These two things are fundamentally incompatible — how can you call something multiple times which requires consuming a value?
Here's a completely untested piece of code that compiles — I don't have rogue WebSocket servers lying about.
#[macro_use]
extern crate quick_error;
extern crate futures;
extern crate tokio_core;
extern crate websocket;
use futures::{Future, Stream, Sink};
use futures::sync::mpsc::channel;
use tokio_core::reactor::Core;
use websocket::ClientBuilder;
pub fn main() {
let mut core = Core::new().unwrap();
let handle = core.handle();
let (send, recv) = channel(100);
let f = ClientBuilder::new("wss://...")
.unwrap()
.async_connect(None, &handle)
.from_err::<Error>()
.map(|(duplex, _)| duplex.split())
.and_then(|(sink, stream)| {
let reader = stream
.for_each(|i| {
println!("Read a {:?}", i);
Ok(())
})
.from_err();
let writer = sink
.sink_from_err()
.send_all(recv.map_err(Error::Receiver))
.map(|_| ());
reader.join(writer)
});
drop(send); // Close the sending channel manually
core.run(f).expect("Unable to run");
}
quick_error! {
#[derive(Debug)]
pub enum Error {
WebSocket(err: websocket::WebSocketError) {
from()
description("websocket error")
display("WebSocket error: {}", err)
cause(err)
}
Receiver(err: ()) {
description("receiver error")
display("Receiver error")
}
}
}
The points that stuck out during implementation were:
everything has to become a Future eventually
it's way easier to define an error type and convert to it
Knowing if the Item and Error associated types were "right" was tricky. I ended up doing a lot of "type assertions" ({ let x: &Future<Item = (), Error = ()> = &reader; }).

Fill a string buffer from a thread [duplicate]

This question already has answers here:
How do I share a mutable object between threads using Arc?
(1 answer)
Lifetime troubles sharing references between threads
(1 answer)
Closed 5 years ago.
With hyper, I need to make an HTTP connection and read the results.
I want to wrap the whole thing in a timeout,
so I start a thread
and use recv_timeout to wait for it.
Wrapping just the send works,
but I want to also wrap read_to_string.
Here is the code:
fn send_request(url: &str) -> Result<Response, MyAppError> {
let mut c = Client::new();
let mut req = c.get(url);
req.send().map_err(|e| MyAppError::TcpError(e))
}
fn get_url(url: &str, mut buf: &mut String) -> Result<u16, MyAppError> {
let mut resp = send_request(url)?;
resp.read_to_string(&mut buf).map_err(|e| MyAppError::ReadError(e))?;
Ok(resp.status.to_u16())
}
fn get_url_with_timeout_2(url: &str, mut buf: &mut String) -> Result<u16, MyAppError> {
let (tx, rx) = mpsc::channel();
let url = url.to_owned();
let t = thread::spawn(move || {
match tx.send(get_url(&url, &mut buf)) {
Ok(()) => {} // everything good
Err(_) => {} // we have been released, no biggie
}
});
match rx.recv_timeout(Duration::from_millis(5000)) {
Ok(resp) => resp,
Err(_) => Err(MyAppError::Timeout),
}
}
Unfortunately I get a compiler error:
error[E0477]: the type `[closure#src/main.rs:53:25: 58:4 tx:std::sync::mpsc::Sender<std::result::Result<u16, MyAppError>>, url:std::string::String, buf:&mut std::string::String]` does not fulfill the required lifetime
--> src/main.rs:53:11
|
53 | let t = thread::spawn(move || {
| ^^^^^^^^^^^^^
|
= note: type must outlive the static lifetime
How can I pass the buffer to the thread,
let it fill it,
and then print out the buffer back on the main thread?
(This is Rust 1.15.1.)
This repository gives a complete main.rs and shows three examples for getting the webpage:
With no timeout.
With a timeout just on send.
With a timeout on the whole thing.
If you take out 3, it all compiles and runs.
What can I change about 3 to make that work too?
By the way, making a web request is really just the "occasion" for this question. I've already seen this question about doing a timeout. My own interest is not the timeout per se, but about how to fill up a buffer on one thread and read it on another.
Share mutable object between threads suggests using Arc and Mutex to safely share data between threads. Here is an attempt at that:
fn get_url_with_timeout_3(url: &str) -> Result<(u16, String), MyAppError> {
let (tx, rx) = mpsc::channel();
let url = url.to_owned();
let shbuf = Arc::new(Mutex::new(String::new()));
let shbuf2 = shbuf.clone();
let t = thread::spawn(move || {
let mut c = Client::new();
let mut req = c.get(&url);
let mut ret = match req.send() {
Ok(mut resp) => {
let mut buf2 = shbuf2.lock().unwrap();
match resp.read_to_string(&mut buf2) {
Ok(_) => Ok(resp.status.to_u16()),
Err(e) => Err(MyAppError::ReadError(e)),
}
}
Err(e) => Err(MyAppError::TcpError(e)),
};
match tx.send(ret) {
Ok(()) => {} // everything good
Err(_) => {} // we have been released, no biggie
}
});
match rx.recv_timeout(Duration::from_millis(5000)) {
Ok(maybe_status_code) => {
let buf2 = shbuf.lock().unwrap();
Ok((maybe_status_code?, *buf2))
}
Err(_) => Err(MyAppError::Timeout),
}
}
This gives me the error cannot move out of borrowed content for trying to return *buf2 (which makes sense, since it would be leaking data out of the Mutex), but I'm not sure how to express this in a structure where Rust can see the pattern.
If the request thread times out, it holds the lock forever — but I never try to read the buffer, so who cares. If the request thread finishes, it releases the lock and goes away, and the main thread will hold the only reference. I can reason that it is safe, but I'm not sure how to convince the compiler.
I'm not allowed to answer this question since it is supposedly a duplicate, but I've added 3 working implementations to my GitHub repository using the ideas from the comments.

Extend lifetime of a variable for thread

I am reading a string from a file, splitting it by lines into a vector and then I want to do something with the extracted lines in separate threads. Like this:
use std::fs::File;
use std::io::prelude::*;
use std::thread;
fn main() {
match File::open("data") {
Ok(mut result) => {
let mut s = String::new();
result.read_to_string(&mut s);
let k : Vec<_> = s.split("\n").collect();
for line in k {
thread::spawn(move || {
println!("nL: {:?}", line);
});
}
}
Err(err) => {
println!("Error {:?}",err);
}
}
}
Of course this throws an error, because s will go out of scope before the threads are started:
s` does not live long enough
main.rs:9 let k : Vec<_> = s.split("\n").collect();
^
What can I do now? I've tried many things like Box or Arc, but I couldn't get it working. I somehow need to create a copy of s which also lives in the threads. But how do I do that?
The problem, fundamentally, is that line is a borrowed slice into s. There's really nothing you can do here, since there's no way to guarantee that each line will not outlive s itself.
Also, just to be clear: there is absolutely no way in Rust to "extend the lifetime of a variable". It simply cannot be done.
The simplest way around this is to go from line being borrowed to owned. Like so:
use std::thread;
fn main() {
let mut s: String = "One\nTwo\nThree\n".into();
let k : Vec<String> = s.split("\n").map(|s| s.into()).collect();
for line in k {
thread::spawn(move || {
println!("nL: {:?}", line);
});
}
}
The .map(|s| s.into()) converts from &str to String. Since a String owns its contents, it can be safely moved into each thread's closure, and will live independently of the thread that created it.
Note: you could do this in nightly Rust using the new scoped thread API, but that is still unstable.

How to resolve this Rust lifetime issue?

I am trying to read the contents of files in a directory in parallel. I'm running into lifetime issues.
My code looks like this:
use std::io::fs;
use std::io;
use std::collections::HashMap;
use std::comm;
use std::io::File;
fn main() {
let (tx, rx) = comm::channel(); // (Sender, Receiver)
let paths = fs::readdir(&Path::new("resources/tests")).unwrap();
for path in paths.iter() {
let task_tx = tx.clone();
spawn(proc() {
match File::open(path).read_to_end() {
Ok(data) => task_tx.send((path.filename_str().unwrap(), data)),
Err(e) => fail!("Could not read one of the files! Error: {}", e)
};
});
}
let mut results = HashMap::new();
for _ in range(0, paths.len()) {
let (filename, data) = rx.recv();
results.insert(filename, data);
}
println!("{}", results);
}
The compilation error I'm getting is:
error: paths does not live long enough
note: reference must be valid for the static lifetime...
note: ...but borrowed value is only valid for the block at 7:19
I also tried to use into_iter() (or move_iter() previously) in the loop without much success.
I'm suspecting it has to do with the spawned tasks remaining alive beyond the entire main() scope, but I don't know how I can fix this situation.
The error message might be a bit confusing but what it's telling you is that you are trying to use a reference path inside of a task.
Because spawn is using proc you can only use data that you can transfer ownership of to that task (Send kind).
To solve that you can do this (you could use a move_iter but then you can't access paths after the loop):
for path in paths.iter() {
let task_tx = tx.clone();
let p = path.clone();
spawn(proc() {
match File::open(&p).read_to_end() {
The second problem is that you are trying to send &str (filename) over a channel. Same as for tasks types used must be of kind Send:
match File::open(&p).read_to_end() {
Ok(data) => task_tx.send((p.filename_str().unwrap().to_string(), data)),
Err(e) => fail!("Could not read one of the files! Error: {}", e)
};

Resources