I'm trying to create a command line program with Rust and I will the need the program to be able to store the result of commands in strings
Here is the current program (source):
use std::process::{Command, Stdio};
fn main() {
let output = Command::new("ls")
// Tell the OS to record the command's output
.stdout(Stdio::piped())
// execute the command, wait for it to complete, then capture the output
.output()
// Blow up if the OS was unable to start the program
.unwrap();
// extract the raw bytes that we captured and interpret them as a string
let stdout = String::from_utf8(output.stdout).unwrap();
println!("{}", stdout);
}
This program works on some commands for example ls but others don't. For example if I try with ll or git branch (which is an example of what I'd like to achieve btw) I have this error:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/main.rs:10:10
Why does this happen and how can I correct it? My environment is Linux.
The solution for git branch (for example) is:
use std::process::{Command, Stdio};
fn main() {
let output = Command::new("git").arg("branch")
// Tell the OS to record the command's output
.stdout(Stdio::piped())
// execute the command, wait for it to complete, then capture the output
.output()
// Blow up if the OS was unable to start the program
.unwrap();
// extract the raw bytes that we captured and interpret them as a string
let stdout = String::from_utf8(output.stdout).unwrap();
println!("{}", stdout);
}
Related
I'm attempting to read from a process that's backed by a PTY in Rust, but once all bytes have been read from the process then reading from the process returns an Input/output error instead of the expected EOF. Is there an obvious reason for this behaviour, and how might it be resolved so that read returns Ok(0) instead of an error, as per the contract for read?
Here is a minimal working example:
use std::io;
use std::io::Read;
use std::io::Write;
use std::fs::File;
use std::os::unix::io::FromRawFd;
use std::process::Command;
use std::process::Stdio;
extern crate nix;
use crate::nix::pty;
use crate::nix::pty::OpenptyResult;
fn main() {
let OpenptyResult{master: controller_fd, slave: follower_fd} =
pty::openpty(None, None)
.expect("couldn't open a new PTY");
let new_follower_stdio = || unsafe { Stdio::from_raw_fd(follower_fd) };
let mut child =
Command::new("ls")
.stdin(new_follower_stdio())
.stdout(new_follower_stdio())
.stderr(new_follower_stdio())
.spawn()
.expect("couldn't spawn the new PTY process");
{
let mut f = unsafe { File::from_raw_fd(controller_fd) };
let mut buf = [0; 0x100];
loop {
let n = f.read(&mut buf[..])
.expect("couldn't read");
if n == 0 {
break;
}
io::stdout().write_all(&buf[..n])
.expect("couldn't write to STDOUT");
}
}
child.kill()
.expect("couldn't kill the PTY process");
child.wait()
.expect("couldn't wait for the PTY process");
}
This gives the following output:
Cargo.lock Cargo.toml build.Dockerfile scripts src target
thread 'main' panicked at 'couldn't read: Os { code: 5, kind: Uncategorized, message: "Input/output error" }', src/main.rs:35:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
I've also tried using nix::unistd::dup to duplicate the follower_fd for stdin, stdout and stderr, but this didn't change the observed behaviour.
For reference, I'm using Rust 1.60.0 with the following Cargo.toml for this MWE:
[package]
name = "mwe"
version = "0.0.0"
[dependencies]
nix = "=0.24.1"
It seems that this error is expected behaviour for PTYs on Linux, and essentially signals EOF. This information is supported by a number of non-authoritative sources, but a good summary is provided by mosvy on the Unix StackExchange:
On Linux, a read() on the master side of a pseudo-tty will return -1 and set ERRNO to EIO when all the handles to its slave side have been closed, but will either block or return EAGAIN before the slave has been first opened.
I don't know if there's any standard spec or rationale for this, but it allows to (crudely) detect when the other side was closed, and simplifies the logic of programs like script which are just creating a pty and running another program inside it.
It is presumed that the EIO described here corresponds to the "Input/output error" returned above.
I'm trying to write a terminal program that pipes in a CSV file, parses the records, then launches a quiz based on them. My issue is that once I pipe the file into my command line program using io::stdin(), I can not longer use io::stdin().read_line() to get user input because it stops blocking or waiting for user input. Below is the Minimal Reproducible Example:
use std::io;
fn main() {
let stdin = io::stdin();
println!("Please enter input");
let mut user_input = String::new();
stdin.read_line(&mut user_input).expect("Failed to get input");
println!("The input was {}", user_input);
}
Using cargo run causes the normal blocking behavior. Using echo 'Hello World' | cargo run causes read_line() to no longer block anywhere throughout the program.
I assume it's not a bug and just how stdin works. Can anyone explain the fine detail of this behavior and if there is a workaround?
I assume it's not a bug and just how stdin works.
Correct. Assuming you're ok with targeting Unix systems, a workaround is to open /dev/tty explicitly:
use std::io::{BufReader, BufRead};
use std::fs::File;
fn main() {
// read piped stuff from stdin...
// read interactive input from the user
let mut input = BufReader::new(File::open("/dev/tty").unwrap());
println!("Please enter input");
let mut user_input = String::new();
input.read_line(&mut user_input).expect("Failed to get input");
println!("The input was {}", user_input);
}
Note that getting the input from the user this way is not the idiomatic way to write command-line programs, because it cannot be automated. Instead, consider supporting command-line options, which crates like clap make convenient.
When reading the input stream with read_line(), if you don't follow the statement with a .expect() the compiler will warn you that this `Result` may be an `Err` variant, which should be handled.
What I want to understand is what kind of situation will trigger such error handling.
I've tried piping my program into another one (./my_program | echo "hello") so I wouldn't have the chance to enter any input and would supposedly be able to see the error handling in action. To my surprise it actually led to a panic state:
thread 'main' panicked at 'failed printing to stdout: Broken pipe (os error 32)', src/libstd/io/stdio.rs:792:9
In this code from the book The Rust Programming Language, we specify a string which I believe should be printed when the program isn't able to read the input stream:
use std::io;
fn main() {
println!("Guess the number!");
println!("Please input your guess.");
let mut guess = String::new();
io::stdin().read_line(&mut guess)
.expect("Failed to read line");
println!("You guessed: {}", guess);
}
How can I actually see that behavior in action?
Let's follow the source:
std::io::Stdin::read_line delegates to std::io::StdinLock::read_line.
std::io::StdinLock::read_line delegates to a private append_to_string function.
append_to_string returns an error on non-UTF-8 input.
Let's try this:
$ rustc a.rs # your program
$ echo "\x99" | ./a
Guess the number!
Please input your guess.
thread 'main' panicked at 'Failed to read line: Custom { kind: InvalidData, error: StringError("stream did not contain valid UTF-8") }', src/libcore/result.rs:997:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
When I try to call the find command from a Rust program, either I get a FIND: Invalid switch or a FIND: Parameter format incorrect error.
find works fine from command line.
echo $PATH
/usr/local/bin:/usr/bin:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:.....
The file I am searching for (main.rs) exists.
use std::process::{Stdio,Command};
use std::io::{Write};
fn main() {
let mut cmd_find = Command::new("/cygdrive/c/cygwin64/bin/find.exe")
.arg("/cygdrive/c/cygwin64/home/*")
.stdin(Stdio::piped())
.spawn()
.unwrap_or_else(|e| { panic!("failed to execute process: {}", e)});
if let Some(ref mut stdin) = cmd_find.stdin {
stdin.write_all(b"main.rs").unwrap();
}
let res = cmd_find.wait_with_output().unwrap().stdout;
println!("{}",String::from_utf8_lossy(&res));
}
./find_cmdd.exe
thread '<main>' panicked at 'failed to execute process: The system cannot find the file specified. (os error 2)', find_cmdd.rs:12
I have also tried the following option,
let mut cmd_find = Command::new("find").....
for which I get FIND:Invalid switch error.
I do not have the luxury of renaming/copying the find.exe to another location.
"FIND:Invalid switch error" indicates this is NOT the cygwin find, but you are invoking the Windows one. To double check:
$ find -k
find: unknown predicate `-k'
$ /cygdrive/c/windows/system32/find -k
FIND: Parameter format not correct
Cygwin basically doesn't exist when you are running a program via Command. Executing a process uses the operating system's native functionality; in the case of Windows that's CreateProcessW.
That means that:
The PATH variable set by your cygwin shell may or may not mean anything when starting a process.
The directory structure with /cygdrive/... doesn't actually exist in Windows; that's an artifact.
All that said, you have to use Windows-native paths:
use std::process::{Stdio, Command};
use std::io::Write;
fn main() {
let mut cmd_find = Command::new(r#"\msys32\usr\bin\find.exe"#)
.args(&[r#"\msys32\home"#])
.stdin(Stdio::piped())
.spawn()
.unwrap_or_else(|e| panic!("failed to execute process: {}", e));
if let Some(ref mut stdin) = cmd_find.stdin {
stdin.write_all(b"main.rs").unwrap();
}
let res = cmd_find.wait_with_output().unwrap().stdout;
println!("{}", String::from_utf8_lossy(&res));
}
As a side note, I have no idea what piping standard input to find does; it doesn't seem to have any effect for me on Msys2 or on OS X...
I believe I understand, in general, one way of doing this:
Create a Command
Use Stdio::piped() to create a new pair of output streams
Configure command.stdout(), and command.stderr()
Spawn the process
Create a new thread and pass the stderr and stdout to it <-- ???
In the remote thread, continually poll for input and write it to the output stream.
In the main thread, wait for the process to finish.
Does that sound right?
My two actual questions:
Is there an easier way that doesn't involve a 'read thread' per process?
If there isn't an easier way, Read::read() requires &mut self; how do you pass that into a remote thread?
Please provide specific examples of how to actually stream the output, not just generic advice about how to do it...
To be more specific, here's the default example of using spawn:
use std::process::Command;
let mut child = Command::new("/bin/cat")
.arg("file.txt")
.spawn()
.expect("failed to execute child");
let ecode = child.wait()
.expect("failed to wait on child");
assert!(ecode.success());
How can the above example be changed to stream the output of child to the console, rather than just waiting for an exit code?
Although the accepted answer is correct, it doesn't cover the non-trivial case.
To stream output and handle it manually, use Stdio::piped() and manually handle the .stdout property on the child returned from calling spawn, like this:
use std::process::{Command, Stdio};
use std::path::Path;
use std::io::{BufReader, BufRead};
pub fn exec_stream<P: AsRef<Path>>(binary: P, args: Vec<&'static str>) {
let mut cmd = Command::new(binary.as_ref())
.args(&args)
.stdout(Stdio::piped())
.spawn()
.unwrap();
{
let stdout = cmd.stdout.as_mut().unwrap();
let stdout_reader = BufReader::new(stdout);
let stdout_lines = stdout_reader.lines();
for line in stdout_lines {
println!("Read: {:?}", line);
}
}
cmd.wait().unwrap();
}
#[test]
fn test_long_running_process() {
exec_stream("findstr", vec!("/s", "sql", "C:\\tmp\\*"));
}
See also Merge child process stdout and stderr regarding catching the output from stderr and stdout simultaneously.
I'll happily accept any example of spawning a long running process and streaming output to the console, by whatever means.
It sounds like you want Stdio::inherit:
use std::process::{Command, Stdio};
fn main() {
let mut cmd =
Command::new("cat")
.args(&["/usr/share/dict/web2"])
.stdout(Stdio::inherit())
.stderr(Stdio::inherit())
.spawn()
.unwrap();
// It's streaming here
let status = cmd.wait();
println!("Exited with status {:?}", status);
}