I want to terminate reading from a tokio::io::lines stream. I merged it with a oneshot future and terminated it, but tokio::run was still working.
use futures::{sync::oneshot, *}; // 0.1.27
use std::{io::BufReader, time::Duration};
use tokio::prelude::*; // 0.1.21
fn main() {
let (tx, rx) = oneshot::channel::<()>();
let lines = tokio::io::lines(BufReader::new(tokio::io::stdin()));
let lines = lines.for_each(|item| {
println!("> {:?}", item);
Ok(())
});
std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(5000));
println!("system shutting down");
let _ = tx.send(());
});
let lines = lines.select2(rx);
tokio::run(lines.map(|_| ()).map_err(|_| ()));
}
How can I stop reading from this?
There's nothing wrong with your strategy, but it will only work with futures that don't execute a blocking operation via Tokio's blocking (the traditional kind of blocking should never be done inside a future).
You can test this by replacing the tokio::io::lines(..) future with a simple interval future:
let lines = Interval::new(Instant::now(), Duration::from_secs(1));
The problem is that tokio::io::Stdin internally uses tokio_threadpool::blocking .
When you use Tokio thread pool blocking (emphasis mine):
NB: The entire task that called blocking is blocked whenever the
supplied closure blocks, even if you have used future combinators such
as select - the other futures in this task will not make progress
until the closure returns. If this is not desired, ensure that
blocking runs in its own task (e.g. using
futures::sync::oneshot::spawn).
Since this will block every other future in the combinator, your Receiver will not be able to get a signal from the Senderuntil the blocking ends.
Please see How can I read non-blocking from stdin? or you can use tokio-stdin-stdout, which creates a channel to consume data from stdin thread. It also has a line-by-line example.
Thank you for your comment and correcting my sentences.
I tried to stop this non-blocking Future and succeeded.
let lines = Interval::new(Instant::now(), Duration::from_secs(1));
My understating is that it would work for this case to wrap the blocking Future with tokio threadpool::blocking.
I'll try it later.
Thank you very much.
Related
How can I create a future that completes upon the termination of a tokio::process::Child without closing stdin. I know there is try_wait for testing if a process has terminated without closing stdin, but I want to have this behavior with future semantics.
I tried to prepare a MRE for this question where my code panics as a result of writing to stdin after calling wait, but what I observe does not match the behavior stated in the documentation for tokio::process::Child's wait method. I would expect to see that the line stdin.write_u8(24).await.unwrap(); crashes with a broken pipe since stdin should have been closed by wait.
use tokio::{time, io::AsyncWriteExt}; // 1.0.1
use std::time::Duration;
#[tokio::main]
pub async fn main() {
let mut child = tokio::process::Command::new("nano")
.stdin(std::process::Stdio::piped())
.spawn()
.unwrap();
let mut stdin = child.stdin.take().unwrap();
let tasklet = tokio::spawn(async move {
child.wait().await
});
// this delay should give tokio::spawn plenty of time to spin up
// and call `wait` on the child (closing stdin)
time::sleep(Duration::from_millis(1000)).await;
// write character 24 (CANcel, ^X) to stdin to close nano
stdin.write_u8(24).await.unwrap();
match tasklet.await {
Ok(exit_result) => match exit_result {
Ok(exit_status) => eprintln!("exit_status: {}", exit_status),
Err(terminate_error) => eprintln!("terminate_error: {}", terminate_error)
}
Err(join_error) => eprintln!("join_error: {}", join_error)
}
}
So the answer to this question is to Option::take ChildStdin out of tokio::process::Child as described in this Github issue. In this case, wait will not close stdin and the programmer is responsible for not causing deadlocks.
The MRE above doesn't fail for two reasons: (i) I took ChildStdin out of tokio::process::Child and (ii) even if I hadn't taken it out, it still would not have been closed due to a bug in the code that will be fixed in this pull request.
Coming from Java, I am used to idioms along the lines of
while (true) {
try {
someBlockingOperation();
} catch (InterruptedException e) {
Thread.currentThread.interrupt(); // re-set the interrupted flag
cleanup(); // whatever is necessary
break;
}
}
This works, as far as I know, across the whole JDK for anything that might block, like reading from files, from sockets, from a queue and even for Thread.sleep().
Reading on how this is done in Rust, I find lots of seemingly special solutions mentioned like mio, tokio. I also find ErrorKind::Interrupted and tried to get this ErrorKind with sending SIGINT to the thread, but the thread seems to die immediately without leaving any (back)trace.
Here is the code I used (note: not very well versed in Rust yet, so it might look a bit strange, but it runs):
use std::io;
use std::io::Read;
use std::thread;
pub fn main() {
let sub_thread = thread::spawn(|| {
let mut buffer = [0; 10];
loop {
let d = io::stdin().read(&mut buffer);
println!("{:?}", d);
let n = d.unwrap();
if n == 0 {
break;
}
println!("-> {:?}", &buffer[0..n]);
}
});
sub_thread.join().unwrap();
}
By "blocking operations", I mean:
sleep
socket IO
file IO
queue IO (not sure yet where the queues are in Rust)
What would be the respective means to signal to a thread, like Thread.interrupt() in Java, that its time to pack up and go home?
There is no such thing. Blocking means blocking.
Instead, you deliberately use tools that are non-blocking. That's where libraries like mio, Tokio, or futures come in — they handle the architecture of sticking all of these non-blocking, asynchronous pieces together.
catch (InterruptedException e)
Rust doesn't have exceptions. If you expect to handle a failure case, that's better represented with a Result.
Thread.interrupt()
This doesn't actually do anything beyond setting a flag in the thread that some code may check and then throw an exception for. You could build the same structure yourself. One simple implementation:
use std::{
sync::{
atomic::{AtomicBool, Ordering},
Arc,
},
thread,
time::Duration,
};
fn main() {
let please_stop = Arc::new(AtomicBool::new(false));
let t = thread::spawn({
let should_i_stop = please_stop.clone();
move || {
while !should_i_stop.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(100));
println!("Sleeping");
}
}
});
thread::sleep(Duration::from_secs(1));
please_stop.store(true, Ordering::SeqCst);
t.join().unwrap();
}
Sleep
No way of interrupting, as far as I know. The documentation even says:
On Unix platforms this function will not return early due to a signal
Socket IO
You put the socket into nonblocking mode using methods like set_nonblocking and then handle ErrorKind::WouldBlock.
See also:
Tokio
async-std
File IO
There isn't really a good cross-platform way of performing asynchronous file IO. Most implementations spin up a thread pool and perform blocking operations there, sending the data over something that does non-blocking.
See also:
Tokio
async-std
Queue IO
Perhaps you mean something like a MPSC channel, in which case you'd use tools like try_recv.
See also:
How to terminate or suspend a Rust thread from another thread?
What is the best approach to encapsulate blocking I/O in future-rs?
What does java.lang.Thread.interrupt() do?
Coming from Java, I am used to idioms along the lines of
while (true) {
try {
someBlockingOperation();
} catch (InterruptedException e) {
Thread.currentThread.interrupt(); // re-set the interrupted flag
cleanup(); // whatever is necessary
break;
}
}
This works, as far as I know, across the whole JDK for anything that might block, like reading from files, from sockets, from a queue and even for Thread.sleep().
Reading on how this is done in Rust, I find lots of seemingly special solutions mentioned like mio, tokio. I also find ErrorKind::Interrupted and tried to get this ErrorKind with sending SIGINT to the thread, but the thread seems to die immediately without leaving any (back)trace.
Here is the code I used (note: not very well versed in Rust yet, so it might look a bit strange, but it runs):
use std::io;
use std::io::Read;
use std::thread;
pub fn main() {
let sub_thread = thread::spawn(|| {
let mut buffer = [0; 10];
loop {
let d = io::stdin().read(&mut buffer);
println!("{:?}", d);
let n = d.unwrap();
if n == 0 {
break;
}
println!("-> {:?}", &buffer[0..n]);
}
});
sub_thread.join().unwrap();
}
By "blocking operations", I mean:
sleep
socket IO
file IO
queue IO (not sure yet where the queues are in Rust)
What would be the respective means to signal to a thread, like Thread.interrupt() in Java, that its time to pack up and go home?
There is no such thing. Blocking means blocking.
Instead, you deliberately use tools that are non-blocking. That's where libraries like mio, Tokio, or futures come in — they handle the architecture of sticking all of these non-blocking, asynchronous pieces together.
catch (InterruptedException e)
Rust doesn't have exceptions. If you expect to handle a failure case, that's better represented with a Result.
Thread.interrupt()
This doesn't actually do anything beyond setting a flag in the thread that some code may check and then throw an exception for. You could build the same structure yourself. One simple implementation:
use std::{
sync::{
atomic::{AtomicBool, Ordering},
Arc,
},
thread,
time::Duration,
};
fn main() {
let please_stop = Arc::new(AtomicBool::new(false));
let t = thread::spawn({
let should_i_stop = please_stop.clone();
move || {
while !should_i_stop.load(Ordering::SeqCst) {
thread::sleep(Duration::from_millis(100));
println!("Sleeping");
}
}
});
thread::sleep(Duration::from_secs(1));
please_stop.store(true, Ordering::SeqCst);
t.join().unwrap();
}
Sleep
No way of interrupting, as far as I know. The documentation even says:
On Unix platforms this function will not return early due to a signal
Socket IO
You put the socket into nonblocking mode using methods like set_nonblocking and then handle ErrorKind::WouldBlock.
See also:
Tokio
async-std
File IO
There isn't really a good cross-platform way of performing asynchronous file IO. Most implementations spin up a thread pool and perform blocking operations there, sending the data over something that does non-blocking.
See also:
Tokio
async-std
Queue IO
Perhaps you mean something like a MPSC channel, in which case you'd use tools like try_recv.
See also:
How to terminate or suspend a Rust thread from another thread?
What is the best approach to encapsulate blocking I/O in future-rs?
What does java.lang.Thread.interrupt() do?
I need to pause the current thread in Rust and notify it from another thread. In Java I would write:
synchronized(myThread) {
myThread.wait();
}
and from the second thread (to resume main thread):
synchronized(myThread){
myThread.notify();
}
Is is possible to do the same in Rust?
Using a channel that sends type () is probably easiest:
use std::sync::mpsc::channel;
use std::thread;
let (tx,rx) = channel();
// Spawn your worker thread, giving it `send` and whatever else it needs
thread::spawn(move|| {
// Do whatever
tx.send(()).expect("Could not send signal on channel.");
// Continue
});
// Do whatever
rx.recv().expect("Could not receive from channel.");
// Continue working
The () type is because it's effectively zero-information, which means it's pretty clear you're only using it as a signal. The fact that it's size zero means it's also potentially faster in some scenarios (but realistically probably not any faster than a normal machine word write).
If you just need to notify the program that a thread is done, you can grab its join guard and wait for it to join.
let guard = thread::spawn( ... ); // This will automatically join when finished computing
guard.join().expect("Could not join thread");
You can use std::thread::park() and std::thread::Thread::unpark() to achieve this.
In the thread you want to wait,
fn worker_thread() {
std::thread::park();
}
in the controlling thread, which has a thread handle already,
fn main_thread(worker_thread: std::thread::Thread) {
worker_thread.unpark();
}
Note that the parking thread can wake up spuriously, which means the thread can sometimes wake up without the any other threads calling unpark on it. You should prepare for this situation in your code, or use something like std::sync::mpsc::channel that is suggested in the accepted answer.
There are multiple ways to achieve this in Rust.
The underlying model in Java is that each object contains both a mutex and a condition variable, if I remember correctly. So using a mutex and condition variable would work...
... however, I would personally switch to using a channel instead:
the "waiting" thread has the receiving end of the channel, and waits for it
the "notifying" thread has the sending end of the channel, and sends a message
It is easier to manipulate than a condition variable, notably because there is no risk to accidentally use a different mutex when locking the variable.
The std::sync::mpsc has two channels (asynchronous and synchronous) depending on your needs. Here, the asynchronous one matches more closely: std::sync::mpsc::channel.
There is a monitor crate that provides this functionality by combining Mutex with Condvar in a convenience structure.
(Full disclosure: I am the author.)
Briefly, it can be used like this:
let mon = Arc::new(Monitor::new(false));
{
let mon = mon.clone();
let _ = thread::spawn(move || {
thread::sleep(Duration::from_millis(1000));
mon.with_lock(|mut done| { // done is a monitor::MonitorGuard<bool>
*done = true;
done.notify_one();
});
});
}
mon.with_lock(|mut done| {
while !*done {
done.wait();
}
println!("finished waiting");
});
Here, mon.with_lock(...) is semantically equivalent to Java's synchronized(mon) {...}.
Is there a way to check whether data is available on stdin in Rust, or to do a read that returns immediately with the currently available data?
My goal is to be able to read the input produced for instance by cursor keys in a shell that is setup to return all read data immediately. For instance with an equivalent to: stty -echo -echok -icanon min 1 time 0.
I suppose one solution would be to use ncurses or similar libraries, but I would like to avoid any kind of large dependencies.
So far, I got only blocking input, which is not what I want:
let mut reader = stdin();
let mut s = String::new();
match reader.read_to_string(&mut s) {...} // this blocks :(
Converting OP's comment into an answer:
You can spawn a thread and send data over a channel. You can then poll that channel in the main thread using try_recv.
use std::io;
use std::sync::mpsc;
use std::sync::mpsc::Receiver;
use std::sync::mpsc::TryRecvError;
use std::{thread, time};
fn main() {
let stdin_channel = spawn_stdin_channel();
loop {
match stdin_channel.try_recv() {
Ok(key) => println!("Received: {}", key),
Err(TryRecvError::Empty) => println!("Channel empty"),
Err(TryRecvError::Disconnected) => panic!("Channel disconnected"),
}
sleep(1000);
}
}
fn spawn_stdin_channel() -> Receiver<String> {
let (tx, rx) = mpsc::channel::<String>();
thread::spawn(move || loop {
let mut buffer = String::new();
io::stdin().read_line(&mut buffer).unwrap();
tx.send(buffer).unwrap();
});
rx
}
fn sleep(millis: u64) {
let duration = time::Duration::from_millis(millis);
thread::sleep(duration);
}
Most operating systems default to work with the standard input and output in a blocking way. No wonder then that the Rust library follows in stead.
To read from a blocking stream in a non-blocking way you might create a separate thread, so that the extra thread blocks instead of the main one. Checking whether a blocking file descriptor produced some input is similar: spawn a thread, make it read the data, check whether it produced any data so far.
Here's a piece of code that I use with a similar goal of processing a pipe output interactively and that can hopefully serve as an example. It sends the data over a channel, which supports the try_recv method - allowing you to check whether the data is available or not.
Someone has told me that mio might be used to read from a pipe in a non-blocking way, so you might want to check it out too. I suspect that passing the stdin file descriptor (0) to Receiver::from_raw_fd should just work.
You could also potentially look at using ncurses (also on crates.io) which would allow you read in raw mode. There are a few examples in the Github repository which show how to do this.