I'm trying to consume output from a process in Rust. If the process doesn't terminate after a certain time, I want to terminate/kill it. Ideally I'd like to wrap everything in a generator so that I can iterate the output line by line, but I'm not experienced enough for that in Rust yet.
Here's my code (src/main.rs), using the subprocess crate:
use subprocess::{Popen, PopenConfig, Redirection};
use std::io::Read;
use std::time::Duration;
fn main() {
let mut p = Popen::create(&["bash", "-c", "echo \"Hi There!\"; sleep 1000"], PopenConfig {stdout: Redirection::Pipe, ..Default::default()}, ).unwrap();
let mut output = p.stdout.take().unwrap();
let thread = std::thread::spawn(move || {
let three_secs = Duration::from_secs(3);
let one_sec = Duration::from_secs(1);
let r = p.wait_timeout(three_secs).unwrap();
match r {
None => {
println!("Wait timed out, terminating process");
p.terminate().unwrap();
let r = p.wait_timeout(one_sec).unwrap();
match r {
None => {
println!("Termination didn't seem to work, killing");
p.kill().unwrap();
},
Some(_) => {
println!("Terminated process successfully");
},
}
p.wait().unwrap();},
Some(_) => { println!("Process returned");},
}
println!("Everything fine");
});
println!("Starting to read output");
let mut output_str = String::new();
output.read_to_string(&mut output_str).unwrap();
println!("Done reading output");
thread.join().unwrap();
println!("Output: {}", output_str);
println!("Hello, world!");
}
I'd expect the following output:
Starting to read output
Wait timed out, terminating process
Terminated process successfully
Everything fine
Done reading output
Output: Hi There!
Hello, world!
and the process finishing after three seconds. But what I get is
Starting to read output
Wait timed out, terminating process
Terminated process successfully
Everything fine
and the process doesn't terminate.
For completeness, here is my Cargo.toml to go along with the src/main.rs from above:
[package]
name = "subproc"
version = "0.1.0"
authors = ["<snip>"]
edition = "2018"
[dependencies]
subprocess = "0.2.4"
I'd look for a crate to help you do this.
Perhaps something like this:
https://docs.rs/wait-timeout/0.2.0/wait_timeout/
Here's the example code adapted to capture stdout and iterate over it line-by-line:
use std::io::Read;
use std::process::{Command, Stdio};
use std::time::Duration;
use wait_timeout::ChildExt;
fn main() {
let mut child = Command::new("sh")
.arg("-c")
.arg("while true; do date; sleep 1; done")
.stdout(Stdio::piped())
.spawn()
.unwrap();
let secs = Duration::from_secs(5);
let _status_code = match child.wait_timeout(secs).unwrap() {
Some(status) => status.code(),
None => {
child.kill().unwrap();
child.wait().unwrap().code()
}
};
let mut s = String::new();
child.stdout.unwrap().read_to_string(&mut s).unwrap();
for (num, line) in s.split("\n").enumerate() {
println!("{}: {}", num, line);
}
}
Prints:
0: Mon Jun 1 14:42:06 BST 2020
1: Mon Jun 1 14:42:07 BST 2020
2: Mon Jun 1 14:42:08 BST 2020
3: Mon Jun 1 14:42:09 BST 2020
4: Mon Jun 1 14:42:10 BST 2020
5:
If you wanted to do other work whilst the child runs, you'd have to use an event loop or a thread.
With the help of Jack O'Connor, the author of the os_pipe library, I managed to write a solution that will read the process output, and do the timeout waiting and killing in another thread. Be advised that this solution will only kill the launched process, not its children, you'll need more handling if your child process has children itself.
use std::process::{Command, Stdio};
use std::io::{BufRead, BufReader};
use std::thread;
use std::time;
fn main() {
const TIMEOUT : i32 = 5;
let mut cmd = Command::new("bash");
cmd.arg("-c").arg("for ((i=1000; i > 0; i--)); do echo \"$i bottles of beer on the wall\"; sleep 1; done");
cmd.stdout(Stdio::piped());
let mut child = cmd.spawn().unwrap();
let stdout = child.stdout.take().unwrap();
let thread = thread::spawn(move || {
for _ in 0..TIMEOUT {
if let Ok(Some(_)) = child.try_wait() {
return;
}
thread::sleep(time::Duration::from_secs(1));
}
child.kill().unwrap();
});
let reader = BufReader::new(stdout);
for line in reader.lines() {
println!("line: {}", line.unwrap());
}
thread.join().unwrap();
println!("Hello, world!");
}
Related
I'll try to simplify as much as possible what I'm trying to do accomplish but in a nutshell here is my problem:
I am trying to spawn the node shell as a process in Rust. I would like to pass to the process' stdin javascript code and read the nodejs output from stdout of the process. This would be an interactive usage where the node shell is spawned and keeps receiving JS instructions and executing them.
I do not wish to launch the nodejs app using a file argument.
I have read quite a bit about std::process::Command, tokio and why we can't write and read to a piped input using standard library. One of the solutions that I kept seeing online (in order to not block the main thread while reading/writing) is to use a thread for reading the output. Most solutions did not involve a continuous write/read flow.
What I have done is to spawn 2 threads, one that keeps writing to stdin and one that keeps reading from stdout. That way, I thought, I won't be blocking the main thread. However my issue is that only 1 thread can actively be used. When I have a thread for stdin, stdout does not even receive data.
Here is the code, comments should provide more details
pub struct Runner {
handle: Child,
pub input: Arc<Mutex<String>>,
pub output: Arc<Mutex<String>>,
input_thread: JoinHandle<()>,
output_thread: JoinHandle<()>,
}
impl Runner {
pub fn new() -> Runner {
let mut handle = Command::new("node")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.spawn()
.expect("Failed to spawn node process!");
// begin stdout thread part
let mut stdout = handle.stdout.take().unwrap();
let output = Arc::new(Mutex::new(String::new()));
let out_clone = Arc::clone(&output);
let output_thread = spawn(move || loop {
// code here never executes...why ?
let mut buf: [u8; 1] = [0];
let mut output = out_clone.lock().unwrap();
let what_i_read = stdout.read(&mut buf);
println!("reading: {:?}", what_i_read);
match what_i_read {
Err(err) => {
println!("{}] Error reading from stream: {}", line!(), err);
break;
}
Ok(bytes_read) => {
if bytes_read != 0 {
let char = String::from_utf8(buf.to_vec()).unwrap();
output.push_str(char.as_str());
} else if output.len() != 0 {
println!("result: {}", output);
out_clone.lock().unwrap().clear();
}
}
}
});
// begin stdin thread block
let mut stdin = handle.stdin.take().unwrap();
let input = Arc::new(Mutex::new(String::new()));
let input_clone = Arc::clone(&input);
let input_thread = spawn(move || loop {
let mut in_text = input_clone.lock().unwrap();
if in_text.len() != 0 {
println!("writing: {}", in_text);
stdin.write_all(in_text.as_bytes()).expect("!write");
stdin.write_all("\n".as_bytes()).expect("!write");
in_text.clear();
}
});
Runner {
handle,
input,
output,
input_thread,
output_thread,
}
}
// this function should receive commands
pub fn execute(&mut self, str: &str) {
let input = Arc::clone(&self.input);
let mut input = input.lock().unwrap();
input.push_str(str);
}
}
In the main thread I'd like use this as
let mut runner = Runner::new();
runner.execute("console.log('foo'");
println!("{:?}", runner.output);
I am still new to Rust but at least I passed the point where the borrow checker makes me bang my head against the wall, I am starting to find it more pleasing now :)
I have a LoRa module which is connected to a serial port. I want to send a number of argument to configure it, but the connection times out after the first argument. Any ideas what I'm doing wrong?
use std::io::{self, Write};
use std::time::Duration;
fn main() {
let port = serialport::new("/dev/ttyS0", 115_200)
.timeout(Duration::from_millis(6000))
.open();
match port {
Ok(mut port) => {
let mut serial_buf: Vec<u8> = vec![0; 1000];
let config = vec![
"AT+CFG=433500000,5,9,7,1,1,0,0,0,0,3000,8,4\r\n",
"AT+RX\r\n",
"AT+SAVE\r\n",
];
for entry in config.iter() {
port.write(entry.as_bytes()).expect("Write failed!");
Duration::from_millis(1500);
loop {
match port.read(serial_buf.as_mut_slice()) {
Ok(t) => io::stdout().write_all(&serial_buf[..t]).unwrap(),
Err(e) => eprintln!("{:?}", e),
}
}
}
}
Err(e) => {
eprintln!("Error: {}", e);
::std::process::exit(1);
}
}
}
My working Python code:
main.py:
connection = Connection('/dev/ttys013', 115200, 8, 'N', 1, 1)
configuration.config_module(
'AT+CFG=433500000,5,9,7,1,1,0,0,0,0,3000,8,4', 'AT+RX', 'AT+SAVE')
configuration.py:
def config_module(self, *arguments):
self.connection.write_to_mcu(arguments[0])
for i in range(0, 1):
time.sleep(2)
validation = self.connection.read_from_mcu()
print(validation[:-1].decode())
for i in range(1, len(arguments)):
self.connection.write_to_mcu(arguments[i])
time.sleep(2)
validation_from_mcu = self.connection.read_from_mcu()
print(validation_from_mcu[:-1].decode())
connection.py:
def write_to_mcu(self, message):
self.serial_connection.write((message + '\r\n').encode())
def read_from_mcu(self):
return self.serial_connection.readline()
You probably already found the bug, but i'll try to answer the question anyway. The main issue was your infinite loop.
use std::io::{self, Write, BufRead, BufReader};
use std::time::Duration;
fn main() {
let mut port = serialport::new("/dev/ttyS0", 115_200)
.timeout(Duration::from_millis(6000))
.open()
// expect does exactly the same thing as your match did
.expect("Error: ");
// in your python you use readline, to archive this
// in rust the easiest way is to use a BufReader
let mut port = BufReader::new(port);
let mut line_buffer = String::new();
let config = vec![
"AT+CFG=433500000,5,9,7,1,1,0,0,0,0,3000,8,4\r\n",
"AT+RX\r\n",
"AT+SAVE\r\n",
];
for entry in config.iter() {
// get_mut is required because BufReader does not implement
// io::Write
port.get_mut()
// use write_all because write does not guarantee that
// every byte gets written
.write_all(entry.as_bytes()).expect("Write failed!");
// Duration::from_millis only returns a Duration and does not sleep
std::thread::sleep(Duration::from_millis(1500));
// clear the line buffer to use it again
line_buffer.clear();
// previously you never stopped reading
port.read_line(&mut line_buffer).expect("Read failed!");
// maybe you have a reason to use io::stdout().write_all()
// but most times you can just use println or print
print!("{}", line_buffer);
}
}
Docs
Write::write
BufRead::read_line
Duration::from_millis
std::thread::sleep
I know how to read the command line arguments, but I am having difficulties reading the command output from a pipe.
Connect a program (A) that outputs data to my Rust program using a pipe:
A | R
The program should consume the data line by line as they come.
$ pwd | cargo run should print the pwd output.
OR
$ find . | cargo run should output the find command output which is more than 1 line.
Use BufRead::lines on a locked handle to standard input:
use std::io::{self, BufRead};
fn main() {
let stdin = io::stdin();
for line in stdin.lock().lines() {
let line = line.expect("Could not read line from standard in");
println!("{}", line);
}
}
If you wanted to reuse the allocation of the String, you could use the loop form:
use std::io::{self, Read};
fn main() {
let stdin = io::stdin();
let mut stdin = stdin.lock(); // locking is optional
let mut line = String::new();
// Could also `match` on the `Result` if you wanted to handle `Err`
while let Ok(n_bytes) = stdin.read_to_string(&mut line) {
if n_bytes == 0 { break }
println!("{}", line);
line.clear();
}
}
You just need to read from Stdin.
This is based on an example taken from the documentation:
use std::io;
fn main() {
loop {
let mut input = String::new();
match io::stdin().read_line(&mut input) {
Ok(len) => if len == 0 {
return;
} else {
println!("{}", input);
}
Err(error) => {
eprintln!("error: {}", error);
return;
}
}
}
}
It's mostly the docs example wrapped in a loop, breaking out of the loop when there is no more input, or if there is an error.
The other changes is that it's better in your context to write errors to stderr, which is why the error branch uses eprintln!, instead of println!. This macro probably wasn't available when that documentation was written.
use std::io;
fn main() {
loop {
let mut input = String::new();
io::stdin()
.read_line(&mut input)
.expect("failed to read from pipe");
input = input.trim().to_string();
if input == "" {
break;
}
println!("Pipe output: {}", input);
}
}
OUTPUT:
[18:50:29 Abhinickz#wsl -> pipe$ pwd
/mnt/d/Abhinickz/dev_work/learn_rust/pipe
[18:50:46 Abhinickz#wsl -> pipe$ pwd | cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
Running `target/debug/pipe`
Pipe output: /mnt/d/Abhinickz/dev_work/learn_rust/pipe
You can do it in a pretty snazzy and concise way with rust's iterator methods
use std::io::{self, BufRead};
fn main() {
// get piped input
// eg `cat file | ./program`
// ( `cat file | cargo run` also works )
let input = io::stdin().lock().lines().fold("".to_string(), |acc, line| {
acc + &line.unwrap() + "\n"
});
dbg!(input);
}
I'm not sure I understand Rust's concurrency support with Mutexes and condition variables. In the following code, the main thread sets the poll_thread to be idle for two seconds, then to "read a register" for 2 seconds, and then return to "idle":
use std::thread;
use std::sync::{Arc, Mutex, Condvar};
use std::time;
#[derive(PartialEq, Debug)]
enum Command {
Idle,
ReadRegister(u32),
}
fn poll_thread(sync_pair: Arc<(Mutex<Command>, Condvar)>) {
let &(ref mutex, ref cvar) = &*sync_pair;
loop {
let mut flag = mutex.lock().unwrap();
while *flag == Command::Idle {
flag = cvar.wait(flag).unwrap();
}
match *flag {
Command::Idle => {
println!("WHAT IMPOSSIBLE!");
panic!();
}
Command::ReadRegister(i) => {
println!("You want me to read {}?", i);
thread::sleep(time::Duration::from_millis(450));
println!("Ok, here it is: {}", 42);
}
}
}
}
pub fn main() {
let pair = Arc::new((Mutex::new(Command::Idle), Condvar::new()));
let pclone = pair.clone();
let rx_thread = thread::spawn(|| poll_thread(pclone));
let &(ref mutex, ref cvar) = &*pair;
for i in 0..10 {
thread::sleep(time::Duration::from_millis(500));
if i == 4 {
println!("Setting ReadRegister");
let mut flag = mutex.lock().unwrap();
*flag = Command::ReadRegister(5);
println!("flag is = {:?}", *flag);
cvar.notify_one();
} else if i == 8 {
println!("Setting Idle");
let mut flag = mutex.lock().unwrap();
*flag = Command::Idle;
println!("flag is = {:?}", *flag);
cvar.notify_one();
}
}
println!("after notify_one()");
rx_thread.join();
}
This works as expected, but when the line to sleep for 450 milliseconds is uncommented, the code will often remain in the "read" state and not return to waiting on the condition variable cvar.wait(). Sometimes it will return to idle after, say, 15 seconds!
I would think that when poll_thread reaches the bottom of the loop, it would release the lock, allowing main to acquire and set flag = Command::Idle, and within roughly half a second, poll_thread would return to idle, but it appears that isn't happening when poll_thread sleeps. Why?
Editor's note — this example was created before Rust 1.0 and the specific types have changed or been removed since then. The general question and concept remains valid.
I have spawned a thread with an infinite loop and timer inside.
thread::spawn(|| {
let mut timer = Timer::new().unwrap();
let periodic = timer.periodic(Duration::milliseconds(200));
loop {
periodic.recv();
// Do my work here
}
});
After a time based on some conditions, I need to terminate this thread from another part of my program. In other words, I want to exit from the infinite loop. How can I do this correctly? Additionally, how could I to suspend this thread and resume it later?
I tried to use a global unsafe flag to break the loop, but I think this solution does not look nice.
For both terminating and suspending a thread you can use channels.
Terminated externally
On each iteration of a worker loop, we check if someone notified us through a channel. If yes or if the other end of the channel has gone out of scope we break the loop.
use std::io::{self, BufRead};
use std::sync::mpsc::{self, TryRecvError};
use std::thread;
use std::time::Duration;
fn main() {
println!("Press enter to terminate the child thread");
let (tx, rx) = mpsc::channel();
thread::spawn(move || loop {
println!("Working...");
thread::sleep(Duration::from_millis(500));
match rx.try_recv() {
Ok(_) | Err(TryRecvError::Disconnected) => {
println!("Terminating.");
break;
}
Err(TryRecvError::Empty) => {}
}
});
let mut line = String::new();
let stdin = io::stdin();
let _ = stdin.lock().read_line(&mut line);
let _ = tx.send(());
}
Suspending and resuming
We use recv() which suspends the thread until something arrives on the channel. In order to resume the thread, you need to send something through the channel; the unit value () in this case. If the transmitting end of the channel is dropped, recv() will return Err(()) - we use this to exit the loop.
use std::io::{self, BufRead};
use std::sync::mpsc;
use std::thread;
use std::time::Duration;
fn main() {
println!("Press enter to wake up the child thread");
let (tx, rx) = mpsc::channel();
thread::spawn(move || loop {
println!("Suspending...");
match rx.recv() {
Ok(_) => {
println!("Working...");
thread::sleep(Duration::from_millis(500));
}
Err(_) => {
println!("Terminating.");
break;
}
}
});
let mut line = String::new();
let stdin = io::stdin();
for _ in 0..4 {
let _ = stdin.lock().read_line(&mut line);
let _ = tx.send(());
}
}
Other tools
Channels are the easiest and the most natural (IMO) way to do these tasks, but not the most efficient one. There are other concurrency primitives which you can find in the std::sync module. They belong to a lower level than channels but can be more efficient in particular tasks.
The ideal solution would be a Condvar. You can use wait_timeout in the std::sync module, as pointed out by #Vladimir Matveev.
This is the example from the documentation:
use std::sync::{Arc, Mutex, Condvar};
use std::thread;
use std::time::Duration;
let pair = Arc::new((Mutex::new(false), Condvar::new()));
let pair2 = pair.clone();
thread::spawn(move|| {
let &(ref lock, ref cvar) = &*pair2;
let mut started = lock.lock().unwrap();
*started = true;
// We notify the condvar that the value has changed.
cvar.notify_one();
});
// wait for the thread to start up
let &(ref lock, ref cvar) = &*pair;
let mut started = lock.lock().unwrap();
// as long as the value inside the `Mutex` is false, we wait
loop {
let result = cvar.wait_timeout(started, Duration::from_millis(10)).unwrap();
// 10 milliseconds have passed, or maybe the value changed!
started = result.0;
if *started == true {
// We received the notification and the value has been updated, we can leave.
break
}
}
Having been back to this question several times myself, here's what I think addresses OP's intent and others' best practice of getting the thread to stop itself. Building on the accepted answer, Crossbeam is a nice upgrade to mpsc in allowing message endpoints to be cloned and moved. It also has a convenient tick function. The real point here is it has try_recv() which is non-blocking.
I'm not sure how universally useful it'd be to put a message checker in the middle of an operational loop like this. I haven't found that Actix (or previously Akka) could really stop a thread without--as stated above--getting the thread to do it itself. So this is what I'm using for now (wide open to correction here, still learning myself).
// Cargo.toml:
// [dependencies]
// crossbeam-channel = "0.4.4"
use crossbeam_channel::{Sender, Receiver, unbounded, tick};
use std::time::{Duration, Instant};
fn main() {
let (tx, rx):(Sender<String>, Receiver<String>) = unbounded();
let rx2 = rx.clone();
// crossbeam allows clone and move of receiver
std::thread::spawn(move || {
// OP:
// let mut timer = Timer::new().unwrap();
// let periodic = timer.periodic(Duration::milliseconds(200));
let ticker: Receiver<Instant> = tick(std::time::Duration::from_millis(500));
loop {
// OP:
// periodic.recv();
crossbeam_channel::select! {
recv(ticker) -> _ => {
// OP: Do my work here
println!("Hello, work.");
// Comms Check: keep doing work?
// try_recv is non-blocking
// rx, the single consumer is clone-able in crossbeam
let try_result = rx2.try_recv();
match try_result {
Err(_e) => {},
Ok(msg) => {
match msg.as_str() {
"END_THE_WORLD" => {
println!("Ending the world.");
break;
},
_ => {},
}
},
_ => {}
}
}
}
}
});
// let work continue for 10 seconds then tell that thread to end.
std::thread::sleep(std::time::Duration::from_secs(10));
println!("Goodbye, world.");
tx.send("END_THE_WORLD".to_string());
}
Using strings as a message device is a tad cringeworthy--to me. Could do the other suspend and restart stuff there in an enum.