I believe I understand, in general, one way of doing this:
Create a Command
Use Stdio::piped() to create a new pair of output streams
Configure command.stdout(), and command.stderr()
Spawn the process
Create a new thread and pass the stderr and stdout to it <-- ???
In the remote thread, continually poll for input and write it to the output stream.
In the main thread, wait for the process to finish.
Does that sound right?
My two actual questions:
Is there an easier way that doesn't involve a 'read thread' per process?
If there isn't an easier way, Read::read() requires &mut self; how do you pass that into a remote thread?
Please provide specific examples of how to actually stream the output, not just generic advice about how to do it...
To be more specific, here's the default example of using spawn:
use std::process::Command;
let mut child = Command::new("/bin/cat")
.arg("file.txt")
.spawn()
.expect("failed to execute child");
let ecode = child.wait()
.expect("failed to wait on child");
assert!(ecode.success());
How can the above example be changed to stream the output of child to the console, rather than just waiting for an exit code?
Although the accepted answer is correct, it doesn't cover the non-trivial case.
To stream output and handle it manually, use Stdio::piped() and manually handle the .stdout property on the child returned from calling spawn, like this:
use std::process::{Command, Stdio};
use std::path::Path;
use std::io::{BufReader, BufRead};
pub fn exec_stream<P: AsRef<Path>>(binary: P, args: Vec<&'static str>) {
let mut cmd = Command::new(binary.as_ref())
.args(&args)
.stdout(Stdio::piped())
.spawn()
.unwrap();
{
let stdout = cmd.stdout.as_mut().unwrap();
let stdout_reader = BufReader::new(stdout);
let stdout_lines = stdout_reader.lines();
for line in stdout_lines {
println!("Read: {:?}", line);
}
}
cmd.wait().unwrap();
}
#[test]
fn test_long_running_process() {
exec_stream("findstr", vec!("/s", "sql", "C:\\tmp\\*"));
}
See also Merge child process stdout and stderr regarding catching the output from stderr and stdout simultaneously.
I'll happily accept any example of spawning a long running process and streaming output to the console, by whatever means.
It sounds like you want Stdio::inherit:
use std::process::{Command, Stdio};
fn main() {
let mut cmd =
Command::new("cat")
.args(&["/usr/share/dict/web2"])
.stdout(Stdio::inherit())
.stderr(Stdio::inherit())
.spawn()
.unwrap();
// It's streaming here
let status = cmd.wait();
println!("Exited with status {:?}", status);
}
Related
I'm attempting to read from a process that's backed by a PTY in Rust, but once all bytes have been read from the process then reading from the process returns an Input/output error instead of the expected EOF. Is there an obvious reason for this behaviour, and how might it be resolved so that read returns Ok(0) instead of an error, as per the contract for read?
Here is a minimal working example:
use std::io;
use std::io::Read;
use std::io::Write;
use std::fs::File;
use std::os::unix::io::FromRawFd;
use std::process::Command;
use std::process::Stdio;
extern crate nix;
use crate::nix::pty;
use crate::nix::pty::OpenptyResult;
fn main() {
let OpenptyResult{master: controller_fd, slave: follower_fd} =
pty::openpty(None, None)
.expect("couldn't open a new PTY");
let new_follower_stdio = || unsafe { Stdio::from_raw_fd(follower_fd) };
let mut child =
Command::new("ls")
.stdin(new_follower_stdio())
.stdout(new_follower_stdio())
.stderr(new_follower_stdio())
.spawn()
.expect("couldn't spawn the new PTY process");
{
let mut f = unsafe { File::from_raw_fd(controller_fd) };
let mut buf = [0; 0x100];
loop {
let n = f.read(&mut buf[..])
.expect("couldn't read");
if n == 0 {
break;
}
io::stdout().write_all(&buf[..n])
.expect("couldn't write to STDOUT");
}
}
child.kill()
.expect("couldn't kill the PTY process");
child.wait()
.expect("couldn't wait for the PTY process");
}
This gives the following output:
Cargo.lock Cargo.toml build.Dockerfile scripts src target
thread 'main' panicked at 'couldn't read: Os { code: 5, kind: Uncategorized, message: "Input/output error" }', src/main.rs:35:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
I've also tried using nix::unistd::dup to duplicate the follower_fd for stdin, stdout and stderr, but this didn't change the observed behaviour.
For reference, I'm using Rust 1.60.0 with the following Cargo.toml for this MWE:
[package]
name = "mwe"
version = "0.0.0"
[dependencies]
nix = "=0.24.1"
It seems that this error is expected behaviour for PTYs on Linux, and essentially signals EOF. This information is supported by a number of non-authoritative sources, but a good summary is provided by mosvy on the Unix StackExchange:
On Linux, a read() on the master side of a pseudo-tty will return -1 and set ERRNO to EIO when all the handles to its slave side have been closed, but will either block or return EAGAIN before the slave has been first opened.
I don't know if there's any standard spec or rationale for this, but it allows to (crudely) detect when the other side was closed, and simplifies the logic of programs like script which are just creating a pty and running another program inside it.
It is presumed that the EIO described here corresponds to the "Input/output error" returned above.
Is there any way to avoid killing the parent process using the fork library, just like this code, but without killing the parent and leaving it doing its thing until it ends?
use fork::{daemon, Fork};
use std::process::Command;
fn main() {
if let Ok(Fork::Child) = daemon(false, false) {
Command::new("/usr/bin/firefox")
.output()
.expect("failed to execute process");
}
}
I'm trying to write a terminal program that pipes in a CSV file, parses the records, then launches a quiz based on them. My issue is that once I pipe the file into my command line program using io::stdin(), I can not longer use io::stdin().read_line() to get user input because it stops blocking or waiting for user input. Below is the Minimal Reproducible Example:
use std::io;
fn main() {
let stdin = io::stdin();
println!("Please enter input");
let mut user_input = String::new();
stdin.read_line(&mut user_input).expect("Failed to get input");
println!("The input was {}", user_input);
}
Using cargo run causes the normal blocking behavior. Using echo 'Hello World' | cargo run causes read_line() to no longer block anywhere throughout the program.
I assume it's not a bug and just how stdin works. Can anyone explain the fine detail of this behavior and if there is a workaround?
I assume it's not a bug and just how stdin works.
Correct. Assuming you're ok with targeting Unix systems, a workaround is to open /dev/tty explicitly:
use std::io::{BufReader, BufRead};
use std::fs::File;
fn main() {
// read piped stuff from stdin...
// read interactive input from the user
let mut input = BufReader::new(File::open("/dev/tty").unwrap());
println!("Please enter input");
let mut user_input = String::new();
input.read_line(&mut user_input).expect("Failed to get input");
println!("The input was {}", user_input);
}
Note that getting the input from the user this way is not the idiomatic way to write command-line programs, because it cannot be automated. Instead, consider supporting command-line options, which crates like clap make convenient.
I have a UDP socket that is receiving data
pub async fn start() -> Result<(), std::io::Error> {
loop {
let mut data = vec![0; 1024];
socket.recv_from(&mut data).await?;
}
}
This code is currently blocked on the .await when there is no data coming in. I want to gracefully shut down my server from my main thread, so how do I send a signal to this .await that it should stop sleeping and shut down instead?
Note: The Tokio website has a page on graceful shutdown.
If you have more than one task to kill, you should use a broadcast channel to send shutdown messages. You can use it together with tokio::select!.
use tokio::sync::broadcast::Receiver;
// You may want to log errors rather than return them in this function.
pub async fn start(kill: Receiver<()>) -> Result<(), std::io::Error> {
tokio::select! {
output = real_start() => output,
_ = kill.recv() => Err(...),
}
}
pub async fn real_start() -> Result<(), std::io::Error> {
loop {
let mut data = vec![0; 1024];
socket.recv_from(&mut data).await?;
}
}
Then to kill all the tasks, send a message on the channel.
To kill only a single task, you can use the JoinHandle::abort method, which will kill the task as soon as possible. Note that this method is available only in Tokio 1.x and 0.3.x, and to abort a task using Tokio 0.2.x, see the next section below.
let task = tokio::spawn(start());
...
task.abort();
As an alternative to JoinHandle::abort, you can use abortable from the futures crate. When you spawn the task, you do the following:
let (task, handle) = abortable(start());
tokio::spawn(task);
Then later you can kill the task by calling the abort method.
handle.abort();
Of course, a channel with select! can also be used to kill a single task, perhaps combined with an oneshot channel rather than a broadcast channel.
All of these methods guarantee that the real_start method is killed at an .await. It is not possible to kill the task while it is running code between two .awaits. You can read more about why this is here.
The mini-redis project contains an accessible real-world example of graceful shutdown of a server. Additionally, the Tokio tutorial has chapters on both select and channels.
I want to execute another process and normally want to wait until it has finished. Lets say we spawn and wait for the process in thread T1:
let child = Command::new("rustc").spawn().unwrap();
child.wait();
Now, if a special event occurs (which thread T0 is waiting for) I want to kill the spawned process:
if let Ok(event) = special_event_notifier.recv() {
child.kill();
}
But I don't see a way to do it: both kill and wait take a mutable reference to Child and are therefore mutually exclusive. After calling wait no one can have any reference to child anymore.
I've found the wait-timeout crate, but I want to know if there's another way.
If the child subprocess do not close stdout before finishing, it's possible to wait reading stdout. Here is an example
use std::io::Read;
use std::process::*;
use std::thread;
use std::time::Duration;
fn wait_on_output(mut out: ChildStdout) {
while out.read_exact(&mut [0; 1024]).is_ok() { }
}
fn wait_or_kill(cmd: &mut Command, max: Duration) {
let mut child = cmd.stdout(Stdio::piped())
.spawn()
.expect("Cannot spawn child");
let out = child.stdout.take().expect("No stdout on child");
let h = thread::spawn(move || {
thread::sleep(max);
child.kill().expect("Cannot kill child");
println!("{:?}", child.wait());
});
wait_on_output(out);
h.join().expect("join fail");
}
fn main() {
wait_or_kill(Command::new("sleep").arg("1"), Duration::new(2, 0));
wait_or_kill(Command::new("sleep").arg("3"), Duration::new(2, 0));
}
The output of this program on my system is
Ok(ExitStatus(ExitStatus(0)))
Ok(ExitStatus(ExitStatus(9)))
Although not in the docs, killing a finished child returns Ok.
This works because killing a process close the files associated with it. However, if the child spawn new processes, killing the child may not kill these other processes and they may keep the stdout opened.
Obviously, you can just kill the process yourself. The Child::id method gives you the "OS-assigned process identifier" that should be sufficient for that.
The only problem is that killing a process is a platform-dependent action. On UNIX killing a process is handled with the kill function:
#![feature(libc)]
extern crate libc;
use std::env::args;
use std::process::Command;
use std::thread::{spawn, sleep};
use std::time::Duration;
use libc::{kill, SIGTERM};
fn main() {
let mut child = Command::new("/bin/sh").arg("-c").arg("sleep 1; echo foo").spawn().unwrap();
let child_id = child.id();
if args().any(|arg| arg == "--kill") {
spawn(move || {
sleep(Duration::from_millis(100));
unsafe {
kill(child_id as i32, SIGTERM);
}
});
}
child.wait().unwrap();
}
On Windows you might try the OpenProcess and TerminateProcess functions (available with the kernel32-sys crate).