In C, I can use rewind back to the start, but I didn't found a similar way in Rust.
I want to open an existed file, and let the file pointer go back to the start point, write new words to it and cover the old one.
But now I can only write something after the last line of the original file and don't know how to change the file pointer.
I known that rust has a crate libc::rewind, but how to use it, or any other ways?
Use seek.
use std::io::{self, Seek, SeekFrom};
use std::fs::File;
fn main() -> io::Result<()> {
let mut file = File::open("foo.bar")?;
file.seek(SeekFrom::Start(0))?;
Ok(())
}
You can use rewind(). It is a syntactic wrapper around SeekFrom::Start(0):
use std::io::{self, Seek};
use std::fs::File;
fn main() -> io::Result<()> {
let mut file = File::open("foo.bar")?;
file.rewind()?;
Ok(())
}
Related
How to iterate over a gziped file which contains a single text file (csv)?
Searching crates.io I found flate2 which has the following code example for decompression:
extern crate flate2;
use std::io::prelude::*;
use flate2::read::GzDecoder;
fn main() {
let mut d = GzDecoder::new("...".as_bytes()).unwrap();
let mut s = String::new();
d.read_to_string(&mut s).unwrap();
println!("{}", s);
}
How to stream a gzip csv file?
For stream io operations rust has the Read and Write traits. To iterate over input by lines you usually want the BufRead trait, which you can always get by wrapping a Read implementation in BufReader::new.
flate2 already operates with these traits; GzDecoder implements Read, and GzDecoder::new takes anything that implements Read.
Example decoding stdin (doesn't work well on playground of course):
extern crate flate2;
use std::io;
use std::io::prelude::*;
use flate2::read::GzDecoder;
fn main() {
let stdin = io::stdin();
let stdin = stdin.lock(); // or just open any normal file
let d = GzDecoder::new(stdin).expect("couldn't decode gzip stream");
for line in io::BufReader::new(d).lines() {
println!("{}", line.unwrap());
}
}
You can then decode your lines with your usual ("without gzip") logic; perhaps make it generic by taking any input implementing BufRead.
I have two three different versions of a function that mimics the input function from python.
use std::io::{self, BufRead, BufReader, Write};
// Adapted from https://docs.rs/python-input/0.8.0/src/python_input/lib.rs.html#13-23
fn input_1(prompt: &str) -> io::Result<String> {
print!("{}", prompt);
io::stdout().flush()?;
let mut buffer = String::new();
io::stdin().read_line(&mut buffer)?;
Ok(buffer.trim_end().to_string())
}
// https://www.reddit.com/r/rust/comments/6qn3y0/store_user_inputs_in_rust/
fn input_2(prompt: &str) -> io::Result<String> {
print!("{}", prompt);
io::stdout().flush()?;
BufReader::new(io::stdin())
.lines()
.next()
.ok_or_else(|| io::Error::new(io::ErrorKind::Other, "Cannot read stdin"))
.and_then(|inner| inner)
}
// tranzystorek user on Discord (edited for future reference)
fn input_3(prompt: &str) -> io::Result<String> {
print!("{}", prompt);
std::io::stdout().flush()?;
BufReader::new(std::io::stdin().lock())
.lines()
.take(1)
.collect()
}
fn main() {
let name = input_1("What's your name? ").unwrap();
println!("Hello, {}!", name);
let name = input_2("What's your name? ").unwrap();
println!("Hello, {}!", name);
let name = input_3("What's your name? ").unwrap();
println!("Hello, {}!", name);
}
But they seem to be very different aproaches and I don't know if there's any advantage using one over the other. From what I've read, having a function like python's input is not as simple as it seems which is why there's none in the standard library.
What problems could I face using any of the versions written above? Is there another, more idiomatic, way of writing this input function? (2018 edition)
Also, here: How can I read a single line from stdin? some of the answers use the lock() method but I don't get its purpose.
I'm learning Rust coming from python.
This is a question of style mostly - both methods are acceptable. Most of the Rustaceans I know would probably favour the second approach, as it's more functional in style but it really doesn't matter in this case.
The key change I'd make is use of the lock method in your second example.
To understand the lock method, consider the following scenario: if you application was multithreaded, and two threads attempted to read from stdin at the same time, what would happen?
The lock ensures that only one thread can access stdin at a time. You always access stdin through a lock. In fact, if you look at the implementation of Stdin::read_line - the method you call in the first example you'll see it's this very simple one-liner:
self.lock().read_line(buf)
So even when you aren't explicitly calling lock it's still being used behind the scenes.
Secondly .next() won't return None in this case, as it will block until data has been entered, so you can use .unwrap() safely here rather than .ok_or/.and_then.
Lastly you missed out the .trim_end() that you had in input_1 ;).
fn input_2(prompt: &str) -> io::Result<String> {
print!("{}", prompt);
io::stdout().flush()?;
io::stdin()
.lock()
.lines()
.next()
.unwrap()
.map(|x| x.trim_end().to_owned())
}
I want to write some code that can read bytes from:
stdin
files
a string
TCP
and maybe others. What is the best way to do this in Rust?
I thought the std::io::Read trait was the way to go, but it seems to be lacking implementations for string at least (I just needed this particular one for testing - maybe I can use something else)?
You may find it help to use the impl<'_> Read for &' [u8] trait for reading bytes from a string. As the type indicates, you'll have to first convert your nice string into a slice of bytes. Here is a short, dumb example.
use std::fs::File;
use std::io::{Error, Read, BufReader};
fn whoo<T: Read>(mut readable: T) {
let mut buffer = [0; 10];
readable.read(&mut buffer).expect("panic");
println!("{:?}", buffer);
}
fn main() -> Result<(), Error> {
whoo("hello there".as_bytes());
whoo("".as_bytes());
let dict = File::open("/usr/share/dict/words")?;
let reader = BufReader::new(dict);
whoo(reader);
Ok(())
}
I need a completely in-memory object that I can give to BufReader and BufWriter. Something like Python's StringIO. I want to write to and read from such an object using methods ordinarily used with Files.
Is there a way to do this using the standard library?
In fact there is a way: Cursor<T>!
(please also read Shepmaster's answer on why often it's even easier)
In the documentation you can see that there are the following impls:
impl<T> Seek for Cursor<T> where T: AsRef<[u8]>
impl<T> Read for Cursor<T> where T: AsRef<[u8]>
impl Write for Cursor<Vec<u8>>
impl<T> AsRef<[T]> for Vec<T>
From this you can see that you can use the type Cursor<Vec<u8>> just as an ordinary file, because Read, Write and Seek are implemented for that type!
Little example (Playground):
use std::io::{Cursor, Read, Seek, SeekFrom, Write};
// Create fake "file"
let mut c = Cursor::new(Vec::new());
// Write into the "file" and seek to the beginning
c.write_all(&[1, 2, 3, 4, 5]).unwrap();
c.seek(SeekFrom::Start(0)).unwrap();
// Read the "file's" contents into a vector
let mut out = Vec::new();
c.read_to_end(&mut out).unwrap();
println!("{:?}", out);
For a more useful example, check the documentation linked above.
You don't need a Cursor most of the time.
object that I can give to BufReader and BufWriter
BufReader requires a value that implements Read:
impl<R: Read> BufReader<R> {
pub fn new(inner: R) -> BufReader<R>
}
BufWriter requires a value that implements Write:
impl<W: Write> BufWriter<W> {
pub fn new(inner: W) -> BufWriter<W> {}
}
If you view the implementors of Read you will find impl<'a> Read for &'a [u8].
If you view the implementors of Write, you will find impl Write for Vec<u8>.
use std::io::{Read, Write};
fn main() {
// Create fake "file"
let mut file = Vec::new();
// Write into the "file"
file.write_all(&[1, 2, 3, 4, 5]).unwrap();
// Read the "file's" contents into a new vector
let mut out = Vec::new();
let mut c = file.as_slice();
c.read_to_end(&mut out).unwrap();
println!("{:?}", out);
}
Writing to a Vec will always append to the end. We also take a slice to the Vec that we can update. Each read of c will advance the slice further and further until it is empty.
The main differences from Cursor:
Cannot seek the data, so you cannot easily re-read data
Cannot write to anywhere but the end
If you want to use BufReader with an in-memory String, you can use the as_bytes() method:
use std::io::BufRead;
use std::io::BufReader;
use std::io::Read;
fn read_buff<R: Read>(mut buffer: BufReader<R>) {
let mut data = String::new();
let _ = buffer.read_line(&mut data);
println!("read_buff got {}", data);
}
fn main() {
read_buff(BufReader::new("Potato!".as_bytes()));
}
This prints read_buff got Potato!. There is no need to use a cursor for this case.
To use an in-memory String with BufWriter, you can use the as_mut_vec method. Unfortunately it is unsafe and I have not found any other way. I don't like the Cursor approach since it consumes the vector and I have not found a way yet to use the Cursor together with BufWriter.
use std::io::BufWriter;
use std::io::Write;
pub fn write_something<W: Write>(mut buf: BufWriter<W>) {
buf.write("potato".as_bytes());
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::{BufWriter};
#[test]
fn testing_bufwriter_and_string() {
let mut s = String::new();
write_something(unsafe { BufWriter::new(s.as_mut_vec()) });
assert_eq!("potato", &s);
}
}
Is there an idiomatic way to process a file one character at a time in Rust?
This seems to be roughly what I'm after:
let mut f = io::BufReader::new(try!(fs::File::open("input.txt")));
for c in f.chars() {
println!("Character: {}", c.unwrap());
}
But Read::chars is still unstable as of Rust v1.6.0.
I considered using Read::read_to_string, but the file may be large and I don't want to read it all into memory.
Let's compare 4 approaches.
1. Read::chars
You could copy Read::chars implementation, but it is marked unstable with
the semantics of a partial read/write of where errors happen is currently unclear and may change
so some care must be taken. Anyway, this seems to be the best approach.
2. flat_map
The flat_map alternative does not compile:
use std::io::{BufRead, BufReader};
use std::fs::File;
pub fn main() {
let mut f = BufReader::new(File::open("input.txt").expect("open failed"));
for c in f.lines().flat_map(|l| l.expect("lines failed").chars()) {
println!("Character: {}", c);
}
}
The problems is that chars borrows from the string, but l.expect("lines failed") lives only inside the closure, so compiler gives the error borrowed value does not live long enough.
3. Nested for
This code
use std::io::{BufRead, BufReader};
use std::fs::File;
pub fn main() {
let mut f = BufReader::new(File::open("input.txt").expect("open failed"));
for line in f.lines() {
for c in line.expect("lines failed").chars() {
println!("Character: {}", c);
}
}
}
works, but it keeps allocation a string for each line. Besides, if there is no line break on the input file, the whole file would be load to the memory.
4. BufRead::read_until
A memory efficient alternative to approach 3 is to use Read::read_until, and use a single string to read each line:
use std::io::{BufRead, BufReader};
use std::fs::File;
pub fn main() {
let mut f = BufReader::new(File::open("input.txt").expect("open failed"));
let mut buf = Vec::<u8>::new();
while f.read_until(b'\n', &mut buf).expect("read_until failed") != 0 {
// this moves the ownership of the read data to s
// there is no allocation
let s = String::from_utf8(buf).expect("from_utf8 failed");
for c in s.chars() {
println!("Character: {}", c);
}
// this returns the ownership of the read data to buf
// there is no allocation
buf = s.into_bytes();
buf.clear();
}
}
I cannot use lines() because my file could be a single line that is gigabytes in size. This an improvement on #malbarbo's recommendation of copying Read::chars from the an old version of Rust. The utf8-chars crate already adds .chars() to BufRead for you.
Inspecting their repository, it doesn't look like they load more than 4 bytes at a time.
Your code will look the same as it did before Rust removed Read::chars:
use std::io::stdin;
use utf8_chars::BufReadCharsExt;
fn main() {
for c in stdin().lock().chars().map(|x| x.unwrap()) {
println!("{}", c);
}
}
Add the following to your Cargo.toml:
[dependencies]
utf8-chars = "1.0.0"
There are two solutions that make sense here.
First, you could copy the implementation of Read::chars() and use it; that would make it completely trivial to move your code over to the standard library implementation if/when it stabilizes.
On the other hand, you could simply iterate line by line (using f.lines()) and then use line.chars() on each line to get the chars. This is a little more hacky, but it will definitely work.
If you only wanted one loop, you could use flat_map() with a lambda like |line| line.chars().