I have a process struct, which holds a process handle:
pub struct Process {
process: Child,
init: bool
}
I have a seperate function where I can 'talk' to the engine.
fn talk_to_engine(&mut self, input: &String) -> String {
let stdin = &mut self.process.stdin.as_mut().unwrap();
stdin.write_all(input.as_bytes()).expect("Failed to write to process.");
let mut s = String::new();
return match self.process.stdout.as_mut().unwrap().read_to_string(&mut s) {
Err(why) => panic!("stdout error: {}", why),
Ok(_) => s
}
}
Yet when I run the function, I get a blinking cursor in the terminal and it does nothing.
EDIT: I call the init_engine function which in turn calls the above mentioned function:
/// Initialize the engine.
pub fn init_engine(&mut self, _protocol: String, depth: String) {
//Stockfish::talk_to_engine(self, &protocol);
let output = Stockfish::talk_to_engine(self, &format!("go depth {}", &depth));
print!("{:?}", output);
self.init = true;
}
if you call init_engine, let's say, like this: struct.init_engine("uci".to_string(), "1".to_string());
Without any information a full reproduction case, or even knowing what the input are and subprocess are it's impossible to know, and hard to guess. Especially as you apparently didn't even try to find what was blocking exactly.
But there are two possible problem points I can see here:
The driver will only reads the output once all input has been consumed, if the subprocess interleaves reading and writing it could fill the entirety of the output pipe's buffer then block on writing to stdout forever, basically deadlocking.
read_to_string reads the entirety of the stream, meaning the subprocess must write everything out and terminate or at least close its stdout, otherwise more output remains possible, and the driver will keep waiting for it.
Related
I am wrapping a C/C++ library in a Rust crate and calling into it using FFI (I am not using a subprocess).
This library logs to stdout/stderr (using, say, printf() or std::cout) but I would like to "catch" this output and use Rust's log crate to control the output.
Is it possible to redirect stdout/stderr of FFI calls to log?
Please find below an example illustrating the different
steps to redirect/restore stderr (file descriptor 2).
The (C-like) style used here was intended in order to keep this
example minimal ; of course, you could probably use the libc
crate and encapsulate properly all of this in a struct.
Note that, in trivial cases, you may repeat the
redirect/invoke/obtain/restore sequence as many times as you want,
provided you keep pipe_fd, saved_fd and log_file open.
However, in non-trivial cases, some kind of complication is implied:
if the C code produces a quite long message, how can we detect
that we have read it all?
we could inject an end-marker into STDERR_FILENO after the
message is produced at the invoke step and then read log_file
until this marker is detected in the obtain step. (this adds
some kind of text processing)
we could recreate the pipe and log_file before each redirect
step, close the PIPE_WRITE end before the invoke step, read
log_file until EOF is reached and close it in the obtain step.
(this adds the overhead of more system-calls)
if the C code produces a very long message, wouldn't it exceed
the pipe's internal buffer capacity (and then block writing)?
we could execute the invoke step in a separate thread and
join() it after the obtain step has completed (end-marker or
EOF is reached), so that the invocation still looks serial
from the application's point of view.
(this adds the overhead of spawning/joining a thread)
an alternative is to put all the logging part of the application
in a separate thread (spawned once for all) and keep all the
invocation steps serial.
(if the logging part of the application does not have to be
perceived as serial this is OK, but else this just reports the
same problem one thread further)
we could fork() to perform the redirect and invoke
steps in a child process (if the application data does not have
to be altered, just read), get rid of the restore step and
wait() the process after the obtain step has completed
(end-marker or EOF is reached), so that the invocation still
looks serial from the application's point of view.
(this adds the overhead of spawning/waiting a process, and
precludes the ability to alter the application data from the
invoked code)
// necessary for the redirection
extern "C" {
fn pipe(fd: *mut i32) -> i32;
fn close(fd: i32) -> i32;
fn dup(fd: i32) -> i32;
fn dup2(
old_fd: i32,
new_fd: i32,
) -> i32;
}
const PIPE_READ: usize = 0;
const PIPE_WRITE: usize = 1;
const STDERR_FILENO: i32 = 2;
fn main() {
//
// duplicate original stderr in order to restore it
//
let saved_stderr = unsafe { dup(STDERR_FILENO) };
if saved_stderr == -1 {
eprintln!("cannot duplicate stderr");
return;
}
//
// create resources (pipe + file reading from it)
//
let mut pipe_fd = [-1; 2];
if unsafe { pipe(&mut pipe_fd[0]) } == -1 {
eprintln!("cannot create pipe");
return;
}
use std::os::unix::io::FromRawFd;
let mut log_file =
unsafe { std::fs::File::from_raw_fd(pipe_fd[PIPE_READ]) };
//
// redirect stderr to pipe/log_file
//
if unsafe { dup2(pipe_fd[PIPE_WRITE], STDERR_FILENO) } == -1 {
eprintln!("cannot redirect stderr to pipe");
return;
}
//
// invoke some C code that should write to stderr
//
extern "C" {
fn perror(txt: *const u8);
}
unsafe {
dup(-1); // invalid syscall in order to set errno (used by perror)
perror(&"something bad happened\0".as_bytes()[0]);
};
//
// obtain the previous message
//
use std::io::Read;
let mut buffer = [0_u8; 100];
if let Ok(sz) = log_file.read(&mut buffer) {
println!(
"message ({} bytes): {:?}",
sz,
std::str::from_utf8(&buffer[0..sz]).unwrap(),
);
}
//
// restore initial stderr
//
unsafe { dup2(saved_stderr, STDERR_FILENO) };
//
// close resources
//
unsafe {
close(saved_stderr);
// pipe_fd[PIPE_READ] will be closed by log_file
close(pipe_fd[PIPE_WRITE]);
};
}
I'm trying to get a parent process and a child process to communicate with each other using a tokio::net::UnixStream. For some reason the child is unable to read whatever the parent writes to the socket, and presumably the other way around.
The function I have is similar to the following:
pub async fn run() -> Result<(), Error> {
let mut socks = UnixStream::pair()?;
match fork() {
Ok(ForkResult::Parent { .. }) => {
socks.0.write_u32(31337).await?;
Ok(())
}
Ok(ForkResult::Child) => {
eprintln!("Reading from master");
let msg = socks.1.read_u32().await?;
eprintln!("Read from master {}", msg);
Ok(())
}
Err(_) => Err(Error),
}
}
The socket doesn't get closed, otherwise I'd get an immediate error trying to read from socks.1. If I move the read into the parent process it works as expected. The first line "Reading from master" gets printed, but the second line never gets called.
I cannot change the communication paradigm, since I'll be using execve to start another binary that expects to be talking to a socketpair.
Any idea what I'm doing wrong here? Is it something to do with the async/await?
When you call the fork() system call:
The child process is created with a single thread—the one that called fork().
The default executor in tokio is a thread pool executor. The child process will only get one of the threads in the pool, so it won't work properly.
I found I was able to make your program work by setting the thread pool to contain only a single thread, like this:
use tokio::prelude::*;
use tokio::net::UnixStream;
use nix::unistd::{fork, ForkResult};
use nix::sys::wait;
use std::io::Error;
use std::io::ErrorKind;
use wait::wait;
// Limit to 1 thread
#[tokio::main(core_threads = 1)]
async fn main() -> Result<(), Error> {
let mut socks = UnixStream::pair()?;
match fork() {
Ok(ForkResult::Parent { .. }) => {
eprintln!("Writing!");
socks.0.write_u32(31337).await?;
eprintln!("Written!");
wait().unwrap();
Ok(())
}
Ok(ForkResult::Child) => {
eprintln!("Reading from master");
let msg = socks.1.read_u32().await?;
eprintln!("Read from master {}", msg);
Ok(())
}
Err(_) => Err(Error::new(ErrorKind::Other, "oh no!")),
}
}
Another change I had to make was to force the parent to wait for the child to complete, by calling wait() - also something you probably do not want to be doing in a real async program.
Most of the advice I have read that if you need to fork from a threaded program, either do it before creating any threads, or call exec_ve() in the child immediately after forking (which is what you plan to do anyway).
I'm creating a terminal text editor in Rust. The editor puts the terminal into raw mode, disabling character echoing and the like, and then restores the original terminal function upon exit.
However, the editor has some bugs, and crashes unexpectedly every now and again due to issues like unsigned variable underflow. When this happens, the cleanup code which would restore the terminal to its original state never runs.
The cleanup function I'd like to run is the following:
fn restore_orig_mode(editor_config: &EditorConfig) -> io::Result<()> {
termios::tcsetattr(STDIN, termios::TCSAFLUSH, &editor_config.orig_termios)
}
In the latest stable Rust, #for1096's answer is the best. In your case, it might be quite simple to apply because your clean-up does not need to use state that is shared with the application code:
use std::panic::catch_unwind;
fn run_editor(){
panic!("Error!");
println!("Running!");
}
fn clean_up(){
println!("Cleaning up!");
}
fn main(){
match catch_unwind(|| run_editor()) {
Ok(_) => println!("Exited successfully"),
Err(_) => clean_up()
}
}
If your clean-up requires accessing shared state with your application, then you will need some additional machinery to convince the compiler that it is safe. For example, if your application looks like this:
// The shared state of your application
struct Editor { /* ... */ }
impl Editor {
fn run(&mut self){
println!("running!");
// panic!("Error!");
}
fn clean_up(&mut self){
println!("cleaning up!");
}
fn new() -> Editor {
Editor { }
}
}
Then, in order to call clean_up, you would have to manage access to the data, something like this:
use std::panic::catch_unwind;
use std::sync::{Arc, Mutex};
fn main() {
let editor = Arc::new(Mutex::new(Editor::new()));
match catch_unwind(|| editor.lock().unwrap().run()) {
Ok(_) => println!("Exited successfully"),
Err(_) => {
println!("Application panicked.");
let mut editor = match editor.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
editor.clean_up();
}
}
}
Prior to Rust 1.9, you can only handle panics that occur in a child thread. This isn't much different except that you need to clone the Arc because the original one will need to be moved into the thread closure.
use std::thread;
use std::sync::{Arc, Mutex};
fn main() {
let editor = Arc::new(Mutex::new(Editor::new()));
// clone before the original is moved into the thread closure
let editor_recovery = editor.clone();
let child = thread::spawn(move || {
editor.lock().unwrap().run();
});
match child.join() {
Ok(_) => println!("Exited successfully"),
Err(_) => {
println!("Application panicked.");
let mut editor = match editor_recovery.lock() {
Ok(guard) => guard,
Err(poisoned) => poisoned.into_inner(),
};
editor.clean_up();
}
}
}
Try catch_unwind. I haven't used it, so I cannot guarantee it works.
A common solution to this problem in Unix applications and using other languages such as C is to fork() and have your parent wait for the child. On an error exit by the child, clean up.
This is really the only reliable way to clean up if the clean up is important. For example, your program might be killed by the Linux OOM kill. It will never be able to run a language specific panic, exception, at_exit or anything like that because the operating system simply destroys it.
By having a separate process watching it, that process can handle any special cleanup of files or shared memory.
This solution does not really require using fork(). The parent could be a shell script or a separate executable.
The second run of foo() will crash without an error message. When I remove this unsafe line, it works ok.
use std::process::{Command, Stdio};
use std::os::unix::io::FromRawFd;
fn foo() {
let mut p = Command::new("ls");
unsafe { p.stdout(Stdio::from_raw_fd(2)) };
let mut child = p.spawn().expect("spawn error");
child.wait().expect("wait error");
println!("process: {:?}", p);
}
fn main() {
foo();
foo();
}
It seems the unsafe code here has some issue. Maybe it's not releasing some resource?
Is there a way to do the stdout -> stderr redirection without using unsafe code?
Stdio::from_raw_fd(2) gives ownership of file descriptor 2 to the newly constructed Stdio object. Stdio's destructor will close the file descriptor. Stdio's destructor will run when the Command is dropped, because the Command owns the Stdio.
Of course, the reason you're not getting any output on the second call to foo is that standard error has been closed!
The simple solution would be to duplicate file descriptor 2 and pass the duplicate to Stdio::from_raw_fd.
Since files and streams are closed automatically when being dropped, but io::stdin() only providing a handle to the underlying stream, I fail to see how to explicitly close stdin or stdout or detect EOF on stdin in my program.
Consider
fn main() {
let mut stdin = io::stdin();
let mut linebuffer = String::new();
loop {
match stdin.read_line(&mut linebuffer) {
Ok(i) if i == 0 => { break; },
Ok(i) => {
println!("{} {}", linebuffer, i);
},
Err(e) => { panic!(e); }
}
linebuffer.clear();
}
}
Checking the number of bytes put into the buffer seems flaky because the pipe might get flushed with zero bytes having being written to it. Reading from a closed stdin should cause an IOError, but it doesn't.
Somewhat related to that: How to explicitly close my own stdout / stderr?
Some time ago there was ErrorKind::EndOfFile enum variant which was emitted upon a read operation when the source stream is closed. It seems that it didn't get to the new I/O library implementation, and instead Read trait has been changed to return 0 read bytes upon EOF. And indeed, this is specified in I/O reform RFC. So yes, checking for zero is a valid way to detect end of stream in the current Rust.
By the way, you can write Ok(0) instead of Ok(i) if i == 0:
match stdin.read_line(&mut buf) {
Ok(0) => break,
...
}
As for how to close stdout()/stderr(), it seems that the current API does not provide a way to do it, unfortunately. It is probably a feature worth an RFC or at least an RFC issue.
Regarding my own sub-question on how to close stdout/stderr: The correct way is to use the wait- or the wait_with_output-method on a process::Child. Both methods close the subprocess's stdin before waiting for it to quit, eliminating the possibility of a deadlock between both processes.