I'm writing a function that has a particular logging requirement. I want to capture the output from a Command::new() call and save it into a file. I'm just using echo here for simplicity's sake.
fn sys_command(id: &str) -> u64 {
let mut cmd = Command::new("echo")
.args(&[id])
.stdout(Stdio::piped())
.spawn()
.expect("failed to echo");
let stdout = cmd.stdout.as_mut().unwrap();
let stdout_reader = BufReader::new(stdout);
// let log_name = format!("./tmp/log/{}.log", id);
// fs::write(log_name, &stdout_reader);
println!("{:?}", stdout_reader.buffer());
cmd.wait().expect("failed to call");
id.parse::<u64>().unwrap()
}
How can I capture the output and save it to a file? I've made a playground here. My println! call returns [].
Even reading to another buffer prints the wrong value. Below, sys_command("10") prints 3. Here's an updated playground.
fn sys_command(id: &str) -> u64 {
let mut buffer = String::new();
let mut cmd = Command::new("echo")
.args(&[id])
.stdout(Stdio::piped())
.spawn()
.expect("failed to echo");
let stdout = cmd.stdout.as_mut().unwrap();
let mut stdout_reader = BufReader::new(stdout);
let result = stdout_reader.read_to_string(&mut buffer);
println!("{:?}", result.unwrap());
cmd.wait().expect("failed to draw");
id.parse::<u64>().unwrap()
}
What am I missing?
Instead of capturing the output in memory and then writing it to a file, you should just redirect the process's output to a file. This is simpler and more efficient.
let log_name = format!("./tmp/log/{}.log", id);
let log = File::create(log_name).expect("failed to open log");
let mut cmd = Command::new("echo")
.args(&[id])
.stdout(log)
.spawn()
.expect("failed to start echo");
cmd.wait().expect("failed to finish echo");
Even reading to another buffer prints the wrong value. Below, sys_command("10") prints 3.
This is unrelated — read_to_string() returns the number of bytes read, not the contents of them, and there are 3 characters: '1' '0' '\n'.
Related
I'm trying to use std::process::Command to run a command and stream its stdout and stderr while also capturing a copy of stdout/stderr. I found I can use spawn.
This code will capture the output, but won't stream it to stdout/stderr while it's happening:
let mut child = command
.envs(env)
.stdout(Stdio::piped()) // <=== Difference here
.spawn()
.unwrap();
let output = child
.wait_with_output().unwrap();
println!("Done {}", std::str::from_utf8(&output.stdout).unwrap());
This code will stream the output but not capture it:
let mut child = command
.envs(env)
.spawn()
.unwrap();
let output = child
.wait_with_output().unwrap();
println!("Done {}", std::str::from_utf8(&output.stdout).unwrap());
Is there a way to capture a command's output while also streaming it to the parent stdout/stderr?
There might be a less verbose way to do this, but this is the solution I came up with.
Spawn the process with a piped io for stdout and stderr. Spawn a thread for stdout and stderr. In each thread read from the pipe and output directly to stdout or stderr then write the contents to a channel.
In the main thread wait for the process to finish, then join the threads and finally read each channel to get the contents of stdout and stderr.
use std::io::BufRead;
let mut child = command
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.unwrap();
let child_stdout = child
.stdout
.take()
.expect("Internal error, could not take stdout");
let child_stderr = child
.stderr
.take()
.expect("Internal error, could not take stderr");
let (stdout_tx, stdout_rx) = std::sync::mpsc::channel();
let (stderr_tx, stderr_rx) = std::sync::mpsc::channel();
let stdout_thread = thread::spawn(move || {
let stdout_lines = BufReader::new(child_stdout).lines();
for line in stdout_lines {
let line = line.unwrap();
println!("{}", line);
stdout_tx.send(line).unwrap();
}
});
let stderr_thread = thread::spawn(move || {
let stderr_lines = BufReader::new(child_stderr).lines();
for line in stderr_lines {
let line = line.unwrap();
eprintln!("{}", line);
stderr_tx.send(line).unwrap();
}
});
let status = child
.wait()
.expect("Internal error, failed to wait on child");
stdout_thread.join().unwrap();
stderr_thread.join().unwrap();
let stdout = stdout_rx.into_iter().collect::<Vec<String>>().join("");
let stderr = stderr_rx.into_iter().collect::<Vec<String>>().join("");
The channel isn't strictly needed. I originally wanted to mutate a string, but I'm new in Rust with threads and couldn't find any examples showing how to mutate a string in a thread and then read it back into main.
I'm accepting the other solution as it really answered my main question. I just wanted to post back to give everyone a fully-featured answer that does exactly what I originally asked
This is similar to how I stream the compilation and execution output on Rust Explorer.
To stream the output you can pipe the stdout and read it line by line using BufReader.
Playground
use std::io::BufRead;
use std::io::BufReader;
use std::process::Command;
use std::process::Stdio;
fn main() {
// Compile code.
let mut child = Command::new("bash")
.args([
"-c",
"echo 'Hello'; sleep 3s; echo 'World'"
])
.stdout(Stdio::piped())
.spawn()
.unwrap();
let stdout = child.stdout.take().unwrap();
// Stream output.
let lines = BufReader::new(stdout).lines();
for line in lines {
println!("{}", line.unwrap());
}
}
In a program I am trying to display the artwork of my currently playing spotify song using rust.
The code only works if I copy and paste the url into the argument, so I tried making a variable called arturl to use in a .arg(arturl). But that makes the code return nothing, and the arturl variable does return the correct value.
My code:
use std::process::{Command, Stdio};
fn main() {
let arturl = Command::new("playerctl")
.arg("metadata")
.arg("mpris:artUrl")
.stdout(Stdio::piped())
.output()
.expect("url failed");
let arturl = String::from_utf8(arturl.stdout).unwrap();
let picture = Command::new("chafa")
.arg("--size=30")
.arg(arturl)
.stdout(Stdio::piped())
.output()
.expect("picture failed");
let picture = String::from_utf8(picture.stdout).unwrap();
println!("{}", picture);
}
You should probably "clean" the string with str::trim:
use std::process::{Command, Stdio};
fn main() {
let arturl = Command::new("playerctl")
.arg("metadata")
.arg("mpris:artUrl")
.stdout(Stdio::piped())
.output()
.expect("url failed");
let arturl = String::from_utf8(arturl.stdout).unwrap().trim();
println!("{:?}", arturl);
let picture = Command::new("chafa")
.arg("--size=30")
.arg(arturl)
.stdout(Stdio::piped())
.output()
.expect("picture failed");
let picture = String::from_utf8(picture.stdout).unwrap();
println!("{}", picture);
}
Managed to fixed this with adding a simple arturl.pop to get rid of the newline at the end of the string
fixed code:
use std::process::{Command, Stdio};
fn main() {
let arturl = Command::new("playerctl")
.arg("metadata")
.arg("mpris:artUrl")
.stdout(Stdio::piped())
.output()
.expect("url failed");
let mut arturl = String::from_utf8(arturl.stdout).unwrap();
arturl.pop();
println!("{:?}", arturl);
let picture = Command::new("chafa")
.arg("--size=30")
.arg(arturl)
.stdout(Stdio::piped())
.output()
.expect("picture failed");
let picture = String::from_utf8(picture.stdout).unwrap();
println!("{}", picture);
}
I'll try to simplify as much as possible what I'm trying to do accomplish but in a nutshell here is my problem:
I am trying to spawn the node shell as a process in Rust. I would like to pass to the process' stdin javascript code and read the nodejs output from stdout of the process. This would be an interactive usage where the node shell is spawned and keeps receiving JS instructions and executing them.
I do not wish to launch the nodejs app using a file argument.
I have read quite a bit about std::process::Command, tokio and why we can't write and read to a piped input using standard library. One of the solutions that I kept seeing online (in order to not block the main thread while reading/writing) is to use a thread for reading the output. Most solutions did not involve a continuous write/read flow.
What I have done is to spawn 2 threads, one that keeps writing to stdin and one that keeps reading from stdout. That way, I thought, I won't be blocking the main thread. However my issue is that only 1 thread can actively be used. When I have a thread for stdin, stdout does not even receive data.
Here is the code, comments should provide more details
pub struct Runner {
handle: Child,
pub input: Arc<Mutex<String>>,
pub output: Arc<Mutex<String>>,
input_thread: JoinHandle<()>,
output_thread: JoinHandle<()>,
}
impl Runner {
pub fn new() -> Runner {
let mut handle = Command::new("node")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.expect("Failed to spawn node process!");
// begin stdout thread part
let mut stdout = handle.stdout.take().unwrap();
let output = Arc::new(Mutex::new(String::new()));
let out_clone = Arc::clone(&output);
let output_thread = spawn(move || loop {
// code here never executes...why ?
let mut buf: [u8; 1] = [0];
let mut output = out_clone.lock().unwrap();
let what_i_read = stdout.read(&mut buf);
println!("reading: {:?}", what_i_read);
match what_i_read {
Err(err) => {
println!("{}] Error reading from stream: {}", line!(), err);
break;
}
Ok(bytes_read) => {
if bytes_read != 0 {
let char = String::from_utf8(buf.to_vec()).unwrap();
output.push_str(char.as_str());
} else if output.len() != 0 {
println!("result: {}", output);
out_clone.lock().unwrap().clear();
}
}
}
});
// begin stdin thread block
let mut stdin = handle.stdin.take().unwrap();
let input = Arc::new(Mutex::new(String::new()));
let input_clone = Arc::clone(&input);
let input_thread = spawn(move || loop {
let mut in_text = input_clone.lock().unwrap();
if in_text.len() != 0 {
println!("writing: {}", in_text);
stdin.write_all(in_text.as_bytes()).expect("!write");
stdin.write_all("\n".as_bytes()).expect("!write");
in_text.clear();
}
});
Runner {
handle,
input,
output,
input_thread,
output_thread,
}
}
// this function should receive commands
pub fn execute(&mut self, str: &str) {
let input = Arc::clone(&self.input);
let mut input = input.lock().unwrap();
input.push_str(str);
}
}
In the main thread I'd like use this as
let mut runner = Runner::new();
runner.execute("console.log('foo'");
println!("{:?}", runner.output);
I am still new to Rust but at least I passed the point where the borrow checker makes me bang my head against the wall, I am starting to find it more pleasing now :)
I'm trying to make some communications with mongo shell. Unfortunately I receive bytes only the first time (after first show dbs\n), and after that the second show dbs\n call hasn't made. Moreover I don't see the long line ------... (I marked It), some infinite loop is observed (the program does not finish).
What am I doing wrong? I did not find such cases on the Internet when people make communications with child processes multiple times. All such examples is all about send via in once and read via out once, but not multiple times.
fn main() -> Result<(), failure::Error> {
let mut mongod = Command::new("mongo")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.expect("mongo was not started");
let mut inp = mongod.stdin.as_mut().unwrap();
let r = inp.write_all(b"show dbs\n")?;
let r = inp.flush()?;
let mut reader = BufReader::new(mongod.stdout.take().unwrap());
let mut s = String::new();
while let Ok(n) = reader.read_line(&mut s) {
println!("{:?}", n);
}
// !don't see it!
println!("{}", "-------------------------------------------");
let r = inp.write_all(b"show dbs\n")?;
let r = inp.flush()?;
while let Ok(n) = reader.read_line(&mut s) {
println!("{:?}", n);
}
}
I know how to read the command line arguments, but I am having difficulties reading the command output from a pipe.
Connect a program (A) that outputs data to my Rust program using a pipe:
A | R
The program should consume the data line by line as they come.
$ pwd | cargo run should print the pwd output.
OR
$ find . | cargo run should output the find command output which is more than 1 line.
Use BufRead::lines on a locked handle to standard input:
use std::io::{self, BufRead};
fn main() {
let stdin = io::stdin();
for line in stdin.lock().lines() {
let line = line.expect("Could not read line from standard in");
println!("{}", line);
}
}
If you wanted to reuse the allocation of the String, you could use the loop form:
use std::io::{self, Read};
fn main() {
let stdin = io::stdin();
let mut stdin = stdin.lock(); // locking is optional
let mut line = String::new();
// Could also `match` on the `Result` if you wanted to handle `Err`
while let Ok(n_bytes) = stdin.read_to_string(&mut line) {
if n_bytes == 0 { break }
println!("{}", line);
line.clear();
}
}
You just need to read from Stdin.
This is based on an example taken from the documentation:
use std::io;
fn main() {
loop {
let mut input = String::new();
match io::stdin().read_line(&mut input) {
Ok(len) => if len == 0 {
return;
} else {
println!("{}", input);
}
Err(error) => {
eprintln!("error: {}", error);
return;
}
}
}
}
It's mostly the docs example wrapped in a loop, breaking out of the loop when there is no more input, or if there is an error.
The other changes is that it's better in your context to write errors to stderr, which is why the error branch uses eprintln!, instead of println!. This macro probably wasn't available when that documentation was written.
use std::io;
fn main() {
loop {
let mut input = String::new();
io::stdin()
.read_line(&mut input)
.expect("failed to read from pipe");
input = input.trim().to_string();
if input == "" {
break;
}
println!("Pipe output: {}", input);
}
}
OUTPUT:
[18:50:29 Abhinickz#wsl -> pipe$ pwd
/mnt/d/Abhinickz/dev_work/learn_rust/pipe
[18:50:46 Abhinickz#wsl -> pipe$ pwd | cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
Running `target/debug/pipe`
Pipe output: /mnt/d/Abhinickz/dev_work/learn_rust/pipe
You can do it in a pretty snazzy and concise way with rust's iterator methods
use std::io::{self, BufRead};
fn main() {
// get piped input
// eg `cat file | ./program`
// ( `cat file | cargo run` also works )
let input = io::stdin().lock().lines().fold("".to_string(), |acc, line| {
acc + &line.unwrap() + "\n"
});
dbg!(input);
}