I'm looking for a pure rust solution which either uses Rust's stdlib or doesn't utilize a crate with libc.
One example of a Device Status Report (DSR) is a simple query to find the current size of a terminal in width and height. First a CSI request: ESC + [. Then a command of 19 and finally a termination character of t. For example: \x1b[19t. The terminal will then respond with another CSI: ESC + [, a status identifier, 9 and then the row count followed by the column count and a terminating character, t. For an 80x25 terminal, the response would look like: \x1b[9;25;80t.
In Rust, it is rather straight-forward to send a request to the terminal using a print! macro call:
print!("\x1b[19t");
Alternatively, in a shell, one might run:
$ printf "\x1b[19t"
However, I've been unable to actually capture the response coming back from the terminal. This is not my initial approach, but it is supposedly the one that provides me with a non-blocking stdin read (based on experimentation, it is not non-blocking), which is something I'm expecting that I need.
use std::sync::mpsc::{channel, Receiver, TryRecvError};
use std::{
io::{self, Read},
thread, time,
};
fn main() {
let timeout = time::Duration::from_micros(50);
let rx = spawn_read();
print!("\x1b[19t");
let mut data = Vec::new();
for _ in 0..200 {
thread::sleep(timeout);
match rx.try_recv() {
Ok(value) => data.push(value),
Err(TryRecvError::Empty) => {}
Err(TryRecvError::Disconnected) => break,
}
}
let string = match std::str::from_utf8(&data) {
Ok(value) => value,
Err(_why) => "",
};
println!("Captured: {:?}", string);
}
fn spawn_read() -> Receiver<u8> {
let (tx, rx) = channel::<u8>();
thread::spawn(move || loop {
let mut buf = [0u8];
io::stdin().read_exact(&mut buf).unwrap();
if buf != [0u8] {
tx.send(buf[0]).ok();
}
});
rx
}
Running the above, I see:
Captured: ""
But with the code above, I expect (some flavor of CSI 9 ; <rows> ; <cols> t):
Captured: "\x1b[9;25;80t"
Related
In the following program I use Tokio's mpsc channels. The Sender is moved to a task named input_message and the Receiver is moved to another task named printer. Both tasks are tokio::spawn()-ed in the main function. The input_message task is to read the user's input and send it through a Channel. The printer task recv() on the channel to get the user's input and simply prints it to stdout:
use std::error::Error;
use tokio::sync::mpsc;
use std::io::{BufRead, Write};
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let (tx, mut rx) = mpsc::unbounded_channel::<String>();
let printer = tokio::spawn(async move {
loop {
let res = rx.recv().await; // (11) Comment this ..
// let res = rx.try_recv(); // (12) Uncomment this ,,
if let Some(m) = res { // .. and this
// if let Ok(m) = res { // ,, and this
if m.trim() == "q".to_string() {
break;
}
println!("Received: {}", m.trim());
}
}
println!("Printer exited");
});
let input_message = tokio::spawn(async move {
let stdin = std::io::stdin();
let mut bufr = std::io::BufReader::new(stdin);
let mut buf = String::new();
loop {
// Let the printer thread print the string before asking the user's input.
std::thread::sleep(std::time::Duration::from_millis(1));
print!("Enter input: ");
std::io::stdout().flush().unwrap();
bufr.read_line(&mut buf).unwrap();
if buf.trim() == "q".to_string() {
tx.send(buf).unwrap();
break;
}
tx.send(buf).unwrap();
buf = String::new();
}
println!("InputMessage exited");
});
tokio::join!(input_message, printer);
Ok(())
}
The expected behavior of the program is to:
Ask the user a random input (q to quit)
Print that same input to stdout
Using rx.recv().await as in line 11-13 the program seems to buffer the Strings representing the user's input: the various inputs are not received by the printer task that therefore does not print the strings to stdout. Once the quit message (i.e. q) is sent, the input_message task exits and the messages seems to be flushed out of the channel and the receiver processes them all at once, and so the printer task prints all the inputs at once. Here's an example of wrong output:
Enter input: Hello
Enter input: World
Enter input: q
InputMessage exited
Received: Hello
Received: World
Printer exited
My question here is, how is it possible that the channel buffers the messages and processes them in one go only when the sending thread exits, instead of receiving them as they are sent?
What I tried to do is to use the try_recv() function as in line 12-14 and indeed it fixes the problem. The output is correctly printed, here is an example:
Enter input: Hello
Received: Hello
Enter input: World
Received: World
Enter input: q
InputMessage exited
Printer exited
In light of this, I get confused. I get the difference between the recv().await and the try_recv() functions but I think there's something more in this case that I'm ignoring that makes the latter work and the former not work. Is anybody able to shed some light and elaborate on this? Why does try_recv() work and recv().await not, and why should recv().await not work in this scenario? In terms of efficiency is looping on try_recv() bad or "bad practice" at all?
There are a few things to point out here, but first of all, you are waiting for lines on std::io::stdin() which blocks the thread until a line arrives on that stream. While the thread waiting for input, no other future can be executed on this thread, this blog post is a great resource if you want to dive deeper why you shouldn't do that.
Tokio's io module offers an async handle to stdin(), you can work with this as a quick fix, although the documentation explicitly mentions that you should spin up a dedicated (non-async) thread for interactive user input instead of using the async handle.
Swapping std::io::stdin() for tokio::io::stdin() also entails swapping out the standard library BufReader for tokio's implementation that wraps an R: AsyncRead rather than R: Read.
To prevent interleaved writes between the input task and the output task, you can use a responder channel that signals to the input task when the output has been printed. Instead of sending String over the channel, you could send a Message with these fields:
struct Message {
payload: String,
done_tx: oneshot::Sender<()>,
}
After reading an input line, send the Message over the channel to the printer task. The printer task prints the String and signals through the done_tx that the input task can print the input prompt and wait for a new line.
Putting all that together with some other changes like a while loop to wait for messages, you'd end up with something like this:
use std::error::Error;
use tokio::io::{AsyncBufReadExt, AsyncWriteExt};
use tokio::sync::{mpsc, oneshot};
#[derive(Debug)]
struct Message {
done_tx: oneshot::Sender<()>,
message: String,
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let (tx, mut rx) = mpsc::unbounded_channel::<Message>();
let printer = tokio::spawn(async move {
while let Some(Message {
message: m,
done_tx,
}) = rx.recv().await
{
if m.trim() == "q".to_string() {
break;
}
println!("Received: {}", m.trim());
done_tx.send(()).unwrap();
}
println!("Printer exited");
});
let input_message = tokio::spawn(async move {
let stdin = tokio::io::stdin();
let mut stdout = tokio::io::stdout();
let mut bufr = tokio::io::BufReader::new(stdin);
let mut buf = String::new();
loop {
// Let the printer thread print the string before asking the user's input.
stdout.write(b"Enter input: ").await.unwrap();
stdout.flush().await.unwrap();
bufr.read_line(&mut buf).await.unwrap();
let end = buf.trim() == "q";
let (done_tx, done) = oneshot::channel();
let message = Message {
message: buf,
done_tx,
};
tx.send(message).unwrap();
if end {
break;
}
done.await.unwrap();
buf = String::new();
}
println!("InputMessage exited");
});
tokio::join!(input_message, printer);
Ok(())
}
I am using Rust and Tokio 1.6 to build an app which can interact with an Elgato StreamDeck via hidapi = "1.2". I want to poll the HID device for events (key down / key up) and send those events on an mpsc channel, while watching a separate mpsc channel for incoming commands to update the device state (reset, change brightness, update image, etc). Since the device handle is not thread safe, I need to do both things from a single thread.
major edits below
This is a rewrite of my original question. I've left my interim answer below, but in the interest of a more self contained example, here is a the basic process using device_query = "0.2":
use device_query::{DeviceState, Keycode};
use std::time::Duration;
use tokio;
use tokio::sync::mpsc::{Receiver, Sender};
use tokio::time::timeout;
#[tokio::main]
async fn main() {
// channel for key press events coming from device loop
let (key_tx, mut key_rx) = tokio::sync::mpsc::channel(32);
// channel for commands sent to device loop
let (dev_tx, mut dev_rx) = tokio::sync::mpsc::channel(32);
start_device_loop(60, key_tx, dev_rx);
println!("Waiting for key presses");
while let Some(k) = key_rx.recv().await {
match k {
Some(ch) => match ch {
Keycode::Q => dev_tx.clone().try_send(String::from("Quit!")).expect("Could not send command"),
ch => println!("{}", ch),
},
_ => (),
}
}
println!("Done.")
}
/// Starts a tokio task, polling the supplied device and sending key events
/// on the supplied mpsc sender
pub fn start_device_loop(hz: u32, tx: Sender<Option<Keycode>>, mut rx: Receiver<String>) {
let poll_wait = 1000 / hz;
let poll_wait = Duration::from_millis(poll_wait as u64);
tokio::task::spawn(async move {
let dev = DeviceState::new();
loop {
let mut keys = dev.query_keymap();
match keys.len() {
0 => (),
1 => tx.clone().try_send(Some(keys.remove(0))).unwrap(),
_ => println!("So many keys..."),
}
match timeout(poll_wait, rx.recv()).await {
Ok(cmd) => println!("Command '{}' received.", cmd.unwrap()),
_ => (),
};
// std::thread::sleep(poll_wait);
}
});
}
Note this does not compile - I get an error future created by async block is not 'Send' and within 'impl Future', the trait 'Send' is not implemented for '*mut x11::xlib::_XDisplay'. My understanding of the error is that because device_query is not thread-safe, and awaiting introduces the possibility of scope moving across threads, nothing may be awaited while a non-thread-safe object is in scope. And indeed, if I comment out the block around match timeout... and uncomment the std::thread::sleep everything compiles and runs.
Which brings me back to the original question; how can I both send and receive messages in a single thread without using await or the apparently forbidden fruit of poll_recv()?
After much hunting I found noop_waker in the futures crate which appears to do what I need in combination with poll_recv:
pub fn start_device_loop(hz: u32, tx: Sender<Option<Keycode>>, mut rx: Receiver<String>) {
let poll_wait = 1000 / hz;
let poll_wait = Duration::from_millis(poll_wait as u64);
tokio::task::spawn_blocking(move || {
let dev = DeviceState::new();
let waker = futures::task::noop_waker();
let mut cx = std::task::Context::from_waker(&waker);
loop {
let mut keys = dev.query_keymap();
match keys.len() {
0 => (),
1 => tx.clone().try_send(Some(keys.remove(0))).unwrap(),
_ => println!("So many keys..."),
}
match rx.poll_recv(&mut cx) {
Poll::Ready(cmd) => println!("Command '{}' received.", cmd.unwrap()),
_ => ()
};
std::thread::sleep(poll_wait);
}
});
}
After digging through docs and tokio source more I can't find anything that suggests poll_recv is supposed to be an internal-only function or that using it here would have any obvious side effects. Letting the process run at 125hz I'm not seeing any excess resource usage either.
I'm leaving the above code for posterity, but since asking this question the try_recv method has been added to Receivers, making this all much cleaner.
I'm making a small ncurses application in Rust that needs to communicate with a child process. I already have a prototype written in Common Lisp. I'm trying to rewrite it because CL uses a huge amount of memory for such a small tool.
I'm having some trouble figuring out how to interact with the sub-process.
What I'm currently doing is roughly this:
Create the process:
let mut program = match Command::new(command)
.args(arguments)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
{
Ok(child) => child,
Err(_) => {
println!("Cannot run program '{}'.", command);
return;
}
};
Pass it to an infinite (until user exits) loop, which reads and handles input and listens for output like this (and writes it to the screen):
fn listen_for_output(program: &mut Child, output_viewer: &TextViewer) {
match program.stdout {
Some(ref mut out) => {
let mut buf_string = String::new();
match out.read_to_string(&mut buf_string) {
Ok(_) => output_viewer.append_string(buf_string),
Err(_) => return,
};
}
None => return,
};
}
The call to read_to_string however blocks the program until the process exits. From what I can see read_to_end and read also seem to block. If I try running something like ls which exits right away, it works, but with something that doesn't exit like python or sbcl it only continues once I kill the subprocess manually.
Based on this answer, I changed the code to use BufReader:
fn listen_for_output(program: &mut Child, output_viewer: &TextViewer) {
match program.stdout.as_mut() {
Some(out) => {
let buf_reader = BufReader::new(out);
for line in buf_reader.lines() {
match line {
Ok(l) => {
output_viewer.append_string(l);
}
Err(_) => return,
};
}
}
None => return,
}
}
However, the problem still remains the same. It will read all lines that are available, and then block. Since the tool is supposed to work with any program, there is no way to guess out when the output will end, before trying to read. There doesn't appear to be a way to set a timeout for BufReader either.
Streams are blocking by default. TCP/IP streams, filesystem streams, pipe streams, they are all blocking. When you tell a stream to give you a chunk of bytes it will stop and wait till it has the given amout of bytes or till something else happens (an interrupt, an end of stream, an error).
The operating systems are eager to return the data to the reading process, so if all you want is to wait for the next line and handle it as soon as it comes in then the method suggested by Shepmaster in Unable to pipe to or from spawned child process more than once (and also in his answer here) works.
Though in theory it doesn't have to work, because an operating system is allowed to make the BufReader wait for more data in read, but in practice the operating systems prefer the early "short reads" to waiting.
This simple BufReader-based approach becomes even more dangerous when you need to handle multiple streams (like the stdout and stderr of a child process) or multiple processes. For example, BufReader-based approach might deadlock when a child process waits for you to drain its stderr pipe while your process is blocked waiting on it's empty stdout.
Similarly, you can't use BufReader when you don't want your program to wait on the child process indefinitely. Maybe you want to display a progress bar or a timer while the child is still working and gives you no output.
You can't use BufReader-based approach if your operating system happens not to be eager in returning the data to the process (prefers "full reads" to "short reads") because in that case a few last lines printed by the child process might end up in a gray zone: the operating system got them, but they're not large enough to fill the BufReader's buffer.
BufReader is limited to what the Read interface allows it to do with the stream, it's no less blocking than the underlying stream is. In order to be efficient it will read the input in chunks, telling the operating system to fill as much of its buffer as it has available.
You might be wondering why reading data in chunks is so important here, why can't the BufReader just read the data byte by byte. The problem is that to read the data from a stream we need the operating system's help. On the other hand, we are not the operating system, we work isolated from it, so as not to mess with it if something goes wrong with our process. So in order to call to the operating system there needs to be a transition to "kernel mode" which might also incur a "context switch". That is why calling the operating system to read every single byte is expensive. We want as few OS calls as possible and so we get the stream data in batches.
To wait on a stream without blocking you'd need a non-blocking stream. MIO promises to have the required non-blocking stream support for pipes, most probably with PipeReader, but I haven't checked it out so far.
The non-blocking nature of a stream should make it possible to read data in chunks regardless of whether the operating system prefers the "short reads" or not. Because non-blocking stream never blocks. If there is no data in the stream it simply tells you so.
In the absense of a non-blocking stream you'll have to resort to spawning threads so that the blocking reads would be performed in a separate thread and thus won't block your primary thread. You might also want to read the stream byte by byte in order to react to the line separator immediately in case the operating system does not prefer the "short reads". Here's a working example: https://gist.github.com/ArtemGr/db40ae04b431a95f2b78.
P.S. Here's an example of a function that allows to monitor the standard output of a program via a shared vector of bytes:
use std::io::Read;
use std::process::{Command, Stdio};
use std::sync::{Arc, Mutex};
use std::thread;
/// Pipe streams are blocking, we need separate threads to monitor them without blocking the primary thread.
fn child_stream_to_vec<R>(mut stream: R) -> Arc<Mutex<Vec<u8>>>
where
R: Read + Send + 'static,
{
let out = Arc::new(Mutex::new(Vec::new()));
let vec = out.clone();
thread::Builder::new()
.name("child_stream_to_vec".into())
.spawn(move || loop {
let mut buf = [0];
match stream.read(&mut buf) {
Err(err) => {
println!("{}] Error reading from stream: {}", line!(), err);
break;
}
Ok(got) => {
if got == 0 {
break;
} else if got == 1 {
vec.lock().expect("!lock").push(buf[0])
} else {
println!("{}] Unexpected number of bytes: {}", line!(), got);
break;
}
}
}
})
.expect("!thread");
out
}
fn main() {
let mut cat = Command::new("cat")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("!cat");
let out = child_stream_to_vec(cat.stdout.take().expect("!stdout"));
let err = child_stream_to_vec(cat.stderr.take().expect("!stderr"));
let mut stdin = match cat.stdin.take() {
Some(stdin) => stdin,
None => panic!("!stdin"),
};
}
With a couple of helpers I'm using it to control an SSH session:
try_s! (stdin.write_all (b"echo hello world\n"));
try_s! (wait_forˢ (&out, 0.1, 9., |s| s == "hello world\n"));
P.S. Note that await on a read call in async-std is blocking as well. It's just instead of blocking a system thread it only blocks a chain of futures (a stack-less green thread essentially). The poll_read is the non-blocking interface. In async-std#499 I've asked the developers whether there's a short read guarantee from these APIs.
P.S. There might be a similar concern in Nom: "we would want to tell the IO side to refill according to the parser's result (Incomplete or not)"
P.S. Might be interesting to see how stream reading is implemented in crossterm. For Windows, in poll.rs, they are using the native WaitForMultipleObjects. In unix.rs they are using mio poll.
Tokio's Command
Here is an example of using tokio 0.2:
use std::process::Stdio;
use futures::StreamExt; // 0.3.1
use tokio::{io::BufReader, prelude::*, process::Command}; // 0.2.4, features = ["full"]
#[tokio::main]
async fn main() {
let mut cmd = Command::new("/tmp/slow.bash")
.stdout(Stdio::piped()) // Can do the same for stderr
.spawn()
.expect("cannot spawn");
let stdout = cmd.stdout().take().expect("no stdout");
// Can do the same for stderr
// To print out each line
// BufReader::new(stdout)
// .lines()
// .for_each(|s| async move { println!("> {:?}", s) })
// .await;
// To print out each line *and* collect it all into a Vec
let result: Vec<_> = BufReader::new(stdout)
.lines()
.inspect(|s| println!("> {:?}", s))
.collect()
.await;
println!("All the lines: {:?}", result);
}
Tokio-Threadpool
Here is an example of using tokio 0.1 and tokio-threadpool. We start the process in a thread using the blocking function. We convert that to a stream with stream::poll_fn
use std::process::{Command, Stdio};
use tokio::{prelude::*, runtime::Runtime}; // 0.1.18
use tokio_threadpool; // 0.1.13
fn stream_command_output(
mut command: Command,
) -> impl Stream<Item = Vec<u8>, Error = tokio_threadpool::BlockingError> {
// Ensure that the output is available to read from and start the process
let mut child = command
.stdout(Stdio::piped())
.spawn()
.expect("cannot spawn");
let mut stdout = child.stdout.take().expect("no stdout");
// Create a stream of data
stream::poll_fn(move || {
// Perform blocking IO
tokio_threadpool::blocking(|| {
// Allocate some space to store anything read
let mut data = vec![0; 128];
// Read 1-128 bytes of data
let n_bytes_read = stdout.read(&mut data).expect("cannot read");
if n_bytes_read == 0 {
// Stdout is done
None
} else {
// Only return as many bytes as we read
data.truncate(n_bytes_read);
Some(data)
}
})
})
}
fn main() {
let output_stream = stream_command_output(Command::new("/tmp/slow.bash"));
let mut runtime = Runtime::new().expect("Unable to start the runtime");
let result = runtime.block_on({
output_stream
.map(|d| String::from_utf8(d).expect("Not UTF-8"))
.fold(Vec::new(), |mut v, s| {
print!("> {}", s);
v.push(s);
Ok(v)
})
});
println!("All the lines: {:?}", result);
}
There's numerous possible tradeoffs that can be made here. For example, always allocating 128 bytes isn't ideal, but it's simple to implement.
Support
For reference, here's slow.bash:
#!/usr/bin/env bash
set -eu
val=0
while [[ $val -lt 10 ]]; do
echo $val
val=$(($val + 1))
sleep 1
done
See also:
How do I synchronously return a value calculated in an asynchronous Future in stable Rust?
If Unix support is sufficient, you can also make the two output streams as non-blocking and poll over them as you would do it on TcpStream with the set_nonblocking function.
The ChildStdout and ChildStderr returned by the Command spawn are Stdio (and contain a file descriptor), you can modify directly the read behavior of these handle to make it non-blocking.
Based on the work of jcreekmore/timeout-readwrite-rs and anowell/nonblock-rs, I use this wrapper to modify the stream handles:
extern crate libc;
use std::io::Read;
use std::os::unix::io::AsRawFd;
use libc::{F_GETFL, F_SETFL, fcntl, O_NONBLOCK};
fn set_nonblocking<H>(handle: &H, nonblocking: bool) -> std::io::Result<()>
where
H: Read + AsRawFd,
{
let fd = handle.as_raw_fd();
let flags = unsafe { fcntl(fd, F_GETFL, 0) };
if flags < 0 {
return Err(std::io::Error::last_os_error());
}
let flags = if nonblocking{
flags | O_NONBLOCK
} else {
flags & !O_NONBLOCK
};
let res = unsafe { fcntl(fd, F_SETFL, flags) };
if res != 0 {
return Err(std::io::Error::last_os_error());
}
Ok(())
}
You can manage the two streams as any other non-blocking stream. The following example is based on the polling crate which makes really easy to handle read event and BufReader for line reading:
use std::process::{Command, Stdio};
use std::path::PathBuf;
use std::io::{BufReader, BufRead};
use std::thread;
extern crate polling;
use polling::{Event, Poller};
fn main() -> Result<(), std::io::Error> {
let path = PathBuf::from("./worker.sh").canonicalize()?;
let mut child = Command::new(path)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("Failed to start worker");
let handle = thread::spawn({
let stdout = child.stdout.take().unwrap();
set_nonblocking(&stdout, true)?;
let mut reader_out = BufReader::new(stdout);
let stderr = child.stderr.take().unwrap();
set_nonblocking(&stderr, true)?;
let mut reader_err = BufReader::new(stderr);
move || {
let key_out = 1;
let key_err = 2;
let mut out_closed = false;
let mut err_closed = false;
let poller = Poller::new().unwrap();
poller.add(reader_out.get_ref(), Event::readable(key_out)).unwrap();
poller.add(reader_err.get_ref(), Event::readable(key_err)).unwrap();
let mut line = String::new();
let mut events = Vec::new();
loop {
// Wait for at least one I/O event.
events.clear();
poller.wait(&mut events, None).unwrap();
for ev in &events {
// stdout is ready for reading
if ev.key == key_out {
let len = match reader_out.read_line(&mut line) {
Ok(len) => len,
Err(e) => {
println!("stdout read returned error: {}", e);
0
}
};
if len == 0 {
println!("stdout closed (len is null)");
out_closed = true;
poller.delete(reader_out.get_ref()).unwrap();
} else {
print!("[STDOUT] {}", line);
line.clear();
// reload the poller
poller.modify(reader_out.get_ref(), Event::readable(key_out)).unwrap();
}
}
// stderr is ready for reading
if ev.key == key_err {
let len = match reader_err.read_line(&mut line) {
Ok(len) => len,
Err(e) => {
println!("stderr read returned error: {}", e);
0
}
};
if len == 0 {
println!("stderr closed (len is null)");
err_closed = true;
poller.delete(reader_err.get_ref()).unwrap();
} else {
print!("[STDERR] {}", line);
line.clear();
// reload the poller
poller.modify(reader_err.get_ref(), Event::readable(key_err)).unwrap();
}
}
}
if out_closed && err_closed {
println!("Stream closed, exiting process thread");
break;
}
}
}
});
handle.join().unwrap();
Ok(())
}
Additionally, used with a wrapper over an EventFd, it becomes possible to easily stop the process from another thread without blocking nor active polling and uses and only a single thread.
EDIT: It seems the polling crate sets automatically the polled handles in non-blocking mode following my tests. The set_nonblocking function is still useful in case you want to directly use the nix::poll object.
I have encountered enough use-cases where it was useful to interact with a subprocess over line-delimited text that I wrote a crate for it, interactive_process.
I expect the original problem has long since been solved, but I thought it might be helpful to others.
I'm trying to write a Rust program that gets a separated list of filenames on stdin.
On Windows, I might invoke it from a cmd window with something like:
dir /b /s | findstr .*,v$ | rust-prog -n
On Unix I'd use something like:
find . -name '*,v' -print0 | rust-prog -0
I'm having trouble converting what I receive on stdin into something that can be used by std::path::Path. As I understand it, to get something that will compile on Windows or Unix, I'm going to need to use conditional compilation, and std::os::windows::ffi or std::os::unix::ffi as appropriate.
Furthermore, It seems on Windows I'll need to use kernel32::MultiByteToWideChar using the current code page to create something usable by std::os::windows::ffi::OsStrExt.
Is there an easier way to do this? Does what I'm suggesting even seem workable?
As an example, it's easy to convert a string to a path, so I tried to use the string handling functions of stdin:
use std::io::{self, Read};
fn main() {
let mut buffer = String::new();
match io::stdin().read_line(&mut buffer) {
Ok(n) => println!("{}", buffer),
Err(error) => println!("error: {}", error)
}
}
On Windows, if I have a directory with a single file called ¿.txt (that's 0xbf). and pipe the name into stdin. I get: error: stream did not contain valid UTF-8.
Here's a reasonable looking version for Windows. Convert the console supplied string to a wide string using win32api functions then wrap it in an OsString using OsString::from_wide.
I'm not convinced it uses the correct code page yet. dir seems to use OEM code page, so maybe that should be the default. There's also a distinction between input code page and output code page in a console.
In my Cargo.toml
[dependencies]
winapi = "0.2"
kernel32-sys = "0.2.2"
Code to read a list of filenames piped through stdin on Windows as per the question.
extern crate kernel32;
extern crate winapi;
use std::io::{self, Read};
use std::ptr;
use std::fs::metadata;
use std::ffi::OsString;
use std::os::windows::ffi::OsStringExt;
/// Convert windows console input to wide string that can
/// be used by OS functions
fn wide_from_console_string(bytes: &[u8]) -> Vec<u16> {
assert!(bytes.len() < std::i32::MAX as usize);
let mut wide;
let mut len;
unsafe {
let cp = kernel32::GetConsoleCP();
len = kernel32::MultiByteToWideChar(cp, 0, bytes.as_ptr() as *const i8, bytes.len() as i32, ptr::null_mut(), 0);
wide = Vec::with_capacity(len as usize);
len = kernel32::MultiByteToWideChar(cp, 0, bytes.as_ptr() as *const i8, bytes.len() as i32, wide.as_mut_ptr(), len);
wide.set_len(len as usize);
}
wide
}
/// Extract paths from a list supplied as Cr LF
/// separated wide string
/// Would use a generic split on substring if it existed
fn paths_from_wide(wide: &[u16]) -> Vec<OsString> {
let mut r = Vec::new();
let mut start = 0;
let mut i = start;
let len = wide.len() - 1;
while i < len {
if wide[i] == 13 && wide[i + 1] == 10 {
if i > start {
r.push(OsString::from_wide(&wide[start..i]));
}
start = i + 2;
i = i + 2;
} else {
i = i + 1;
}
}
if i > start {
r.push(OsString::from_wide(&wide[start..i]));
}
r
}
fn main() {
let mut bytes = Vec::new();
if let Ok(_) = io::stdin().read_to_end(&mut bytes) {
let pathlist = wide_from_console_string(&bytes[..]);
let paths = paths_from_wide(&pathlist[..]);
for path in paths {
match metadata(&path) {
Ok(stat) => println!("{:?} is_file: {}", &path, stat.is_file()),
Err(e) => println!("Error: {:?} for {:?}", e, &path)
}
}
}
}
I want to run an executable that blocks on stdin and when a key is pressed that same character is printed immediately without Enter having to be pressed.
How can I read one character from stdin without having to hit Enter? I started with this example:
fn main() {
println!("Type something!");
let mut line = String::new();
let input = std::io::stdin().read_line(&mut line).expect("Failed to read line");
println!("{}", input);
}
I looked through the API and tried replacing read_line() with bytes(), but everything I try requires me to hit Enter before read occurs.
This question was asked for C/C++, but there seems to be no standard way to do it: Capture characters from standard input without waiting for enter to be pressed
It might not be doable in Rust considering it's not simple in C/C++.
While #Jon's solution using ncurses works, ncurses clears the screen by design. I came up with this solution that uses the termios crate for my little project to learn Rust. The idea is to modify ECHO and ICANON flags by accessing tcsetattr through termios bindings.
extern crate termios;
use std::io;
use std::io::Read;
use std::io::Write;
use termios::{Termios, TCSANOW, ECHO, ICANON, tcsetattr};
fn main() {
let stdin = 0; // couldn't get std::os::unix::io::FromRawFd to work
// on /dev/stdin or /dev/tty
let termios = Termios::from_fd(stdin).unwrap();
let mut new_termios = termios.clone(); // make a mutable copy of termios
// that we will modify
new_termios.c_lflag &= !(ICANON | ECHO); // no echo and canonical mode
tcsetattr(stdin, TCSANOW, &mut new_termios).unwrap();
let stdout = io::stdout();
let mut reader = io::stdin();
let mut buffer = [0;1]; // read exactly one byte
print!("Hit a key! ");
stdout.lock().flush().unwrap();
reader.read_exact(&mut buffer).unwrap();
println!("You have hit: {:?}", buffer);
tcsetattr(stdin, TCSANOW, & termios).unwrap(); // reset the stdin to
// original termios data
}
One advantage of reading a single byte is capturing arrow keys, ctrl etc. Extended F-keys are not captured (although ncurses can capture these).
This solution is intended for UNIX-like platforms. I have no experience with Windows, but according to this forum perhaps something similar can be achieved using SetConsoleMode in Windows.
Use one of the 'ncurses' libraries now available, for instance this one.
Add the dependency in Cargo
[dependencies]
ncurses = "5.86.0"
and include in main.rs:
extern crate ncurses;
use ncurses::*; // watch for globs
Follow the examples in the library to initialize ncurses and wait for single character input like this:
initscr();
/* Print to the back buffer. */
printw("Hello, world!");
/* Update the screen. */
refresh();
/* Wait for a key press. */
getch();
/* Terminate ncurses. */
endwin();
You can also use termion, but you will have to enable the raw TTY mode which changes the behavior of stdout as well. See the example below (tested with Rust 1.34.0). Note that internally, it also wraps the termios UNIX API.
Cargo.toml
[dependencies]
termion = "1.5.2"
main.rs
use std::io;
use std::io::Write;
use std::thread;
use std::time;
use termion;
use termion::input::TermRead;
use termion::raw::IntoRawMode;
fn main() {
// Set terminal to raw mode to allow reading stdin one key at a time
let mut stdout = io::stdout().into_raw_mode().unwrap();
// Use asynchronous stdin
let mut stdin = termion::async_stdin().keys();
loop {
// Read input (if any)
let input = stdin.next();
// If a key was pressed
if let Some(Ok(key)) = input {
match key {
// Exit if 'q' is pressed
termion::event::Key::Char('q') => break,
// Else print the pressed key
_ => {
write!(
stdout,
"{}{}Key pressed: {:?}",
termion::clear::All,
termion::cursor::Goto(1, 1),
key
)
.unwrap();
stdout.lock().flush().unwrap();
}
}
}
thread::sleep(time::Duration::from_millis(50));
}
}
Here's a lightweight solution only using the libc crate based some code from the console crate:
fn setup_raw_terminal() -> io::Result<()> {
unsafe {
let tty;
let fd = if libc::isatty(libc::STDIN_FILENO) == 1 {
libc::STDIN_FILENO
} else {
tty = fs::File::open("/dev/tty")?;
tty.as_raw_fd()
};
let mut ptr = core::mem::MaybeUninit::uninit();
if libc::tcgetattr(fd, ptr.as_mut_ptr()) == 0 {
let mut termios = ptr.assume_init();
let c_oflag = termios.c_oflag;
libc::cfmakeraw(&mut termios);
termios.c_oflag = c_oflag;
if libc::tcsetattr(fd, libc::TCSADRAIN, &termios) == 0 {
return Ok(());
}
}
}
Err(io::Error::last_os_error())
}
It needs to be called before reading stdin:
let mut buf = [0u8; 1024];
let mut stdin = io::stdin();
setup_raw_terminal()?;
loop {
let size = stdin.read(&mut buf)?;
let data = &buf[0..size];
println!("stdin data: {}", data);
}