I've poked the serde-yaml and yaml-rust crates a bit, but I haven't seen any examples.
serde-yaml's documentation has the following 4 functions:
from_reader — Deserialize an instance of type T from an IO stream of YAML.
from_slice — Deserialize an instance of type T from bytes of YAML text.
from_str — Deserialize an instance of type T from a string of YAML text.
from_value — Interpret a serde_yaml::Value as an instance of type T.
Using from_reader as an example:
use serde_yaml; // 0.8.7
fn main() -> Result<(), Box<dyn std::error::Error>> {
let f = std::fs::File::open("something.yaml")?;
let d: String = serde_yaml::from_reader(f)?;
println!("Read YAML string: {}", d);
Ok(())
}
something.yaml:
"I am YAML"
You can deserialize into the looser-typed Value if you don't know your format (String in this example), but be sure to read the Serde guide for full details of how to do type-directed serialization and deserialization instead.
See also:
How do I parse a JSON File?
Deserializing TOML into vector of enum with values
In general, using any Serde format is pretty much the same as all the rest.
This example uses the yaml_rust crate
use std::fs::File;
use std::io::prelude::*;
use yaml_rust::yaml::{Hash, Yaml};
use yaml_rust::YamlLoader;
fn main() {
println!("Hello, Yaml");
let file = "./etc/my_yaml_file.yaml";
load_file(file);
}
fn load_file(file: &str) {
let mut file = File::open(file).expect("Unable to open file");
let mut contents = String::new();
file.read_to_string(&mut contents)
.expect("Unable to read file");
let docs = YamlLoader::load_from_str(&contents).unwrap();
// iterate / process doc[s] ..
}
The answer from Shepmaster is great if you want to do it properly. Here's a complete example to get started with.
data['foo']['bar'].as_str() returns an Option<str>.
fn example() -> Result<String> {
let f = std::fs::File::open("something.yaml")?;
let data: serde_yaml::Value = serde_yaml::from_reader(f)?;
data["foo"]["bar"]
.as_str()
.map(|s| s.to_string())
.ok_or(anyhow!("Could not find key foo.bar in something.yaml"))
}
Related
I'm writing a program that handles a vector which is combination of numbers and letters (hence Vec<String>). I sort it with the .sort() method and am now trying to write it to a file.
Where strvec is my sorted vector that I'm trying to write using std::fs::write;
println!("Save results to file?");
let to_save: String = read!();
match to_save.as_str() {
"y" => {
println!("Enter filename");
let filename: String = read!();
let pwd = current_dir().into();
write("/home/user/dl/results", strvec);
Rust tells me "the trait AsRef<[u8]> is not implemented for Vec<String>". I've also tried using &strvec.
How do I avoid this/fix it?
When it comes to writing objects to the file you might want to consider serialization. Most common library for this in Rust is serde, however in this example where you want to store vector of Strings and if you don't need anything human readable in file (but it comes with small size :P), you can also use bincode:
use std::fs;
use bincode;
fn main() {
let v = vec![String::from("aaa"), String::from("bbb")];
let encoded_v = bincode::serialize(&v).expect("Could not encode vector");
fs::write("file", encoded_v).expect("Could not write file");
let read_v = fs::read("file").expect("Could not read file");
let decoded_v: Vec<String> = bincode::deserialize(&read_v).expect("Could not decode vector");
println!("{:?}", decoded_v);
}
Remember to add bincode = "1.3.3" under dependencies in Cargo.toml
#EDIT:
Actually you can easily save String to the file so simple join() should do:
use std::fs;
fn main() {
let v = vec![
String::from("aaa"),
String::from("bbb"),
String::from("ccc")];
fs::write("file", v.join("\n")).expect("");
}
Rust can't write anything besides a &[u8] to a file. There are too many different ways which data can be interpreted before it gets flattened, so you need to handle all of that ahead of time. For a Vec<String>, it's pretty simple, and you can just use concat to squish everything down to a single String, which can be interpreted as a &[u8] because of its AsRef<u8> trait impl.
Another option would be to use join, in case you wanted to add some sort of delimiter between your strings, like a space, comma, or something.
fn main() {
let strvec = vec![
"hello".to_string(),
"world".to_string(),
];
// "helloworld"
std::fs::write("/tmp/example", strvec.concat()).expect("failed to write to file");
// "hello world"
std::fs::write("/tmp/example", strvec.join(" ")).expect("failed to write to file");
}
You can't get a &[u8] from a Vec<String> without copying since a slice must refer to a contiguous sequence of items. Each String will have its own allocation on the heap somewhere, so while each individual String can be converted to a &[u8], you can't convert the whole vector to a single &[u8].
While you can .collect() the vector into a single String and then get a &[u8] from that, this does some unnecessary copying. Consider instead just iterating the Strings and writing each one to the file. With this helper, it's no more complex than using std::fs::write():
use std::path::Path;
use std::fs::File;
use std::io::Write;
fn write_each(
path: impl AsRef<Path>,
items: impl IntoIterator<Item=impl AsRef<[u8]>>,
) -> std::io::Result<()> {
let mut file = File::create(path)?;
for i in items {
file.write_all(i.as_ref())?;
}
// Surface any I/O errors that could otherwise be swallowed when
// the file is closed implicitly by being dropped.
file.sync_all()
}
The bound impl IntoIterator<Item=impl AsRef<[u8]>> is satisfied by both Vec<String> and by &Vec<String>, so you can call this as either write_each("path/to/output", strvec) (to consume the vector) or write_each("path/to/output", &strvec) (if you need to hold on to the vector for later).
I need to create packet to send to the server. For this purpose I use vector with byteorder crate. When I try to append string, Rust compiler tells I use unsafe function and give me an error.
use byteorder::{LittleEndian, WriteBytesExt};
fn main () {
let login = "test";
let packet_length = 30 + (login.len() as i16);
let mut packet = Vec::new();
packet.write_u8(0x00);
packet.write_i16::<LittleEndian>(packet_length);
packet.append(&mut Vec::from(String::from("game name ").as_bytes_mut()));
// ... rest code
}
The error is:
packet.append(&mut Vec::from(String::from("game name ").as_bytes_mut()));
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ call to unsafe function
This is playground to reproduce: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=381c6d14660d47beaece15d068b3dc6a
What is the correct way to insert some string as bytes into vector ?
The unsafe function called was as_bytes_mut(). This creates a mutable reference with exclusive access to the bytes representing the string, allowing you to modify them. You do not really need a mutable reference in this case, as_bytes() would have sufficed.
However, there is a more idiomatic way. Vec<u8> also functions as a writer (it implements std::io::Write), so you can use one of its methods, or even the write! macro, to write encoded text on it.
use std::io::Write;
use byteorder::{LittleEndian, WriteBytesExt};
fn main () -> Result<(), std::io::Error> {
let login = "test";
let packet_length = 30 + (login.len() as i16);
let mut packet = Vec::new();
packet.write_u8(0x00)?;
packet.write_i16::<LittleEndian>(packet_length)?;
let game_name = String::from("game name");
write!(packet, "{} ", game_name)?;
Ok(())
}
Playground
See also:
Use write! macro with a string instead of a string literal
What's the de-facto way of reading and writing files in Rust 1.x?
You can use .extend() on the Vec and pass in the bytes representation of the String:
use byteorder::{LittleEndian, WriteBytesExt};
fn main() {
let login = "test";
let packet_length = 30 + (login.len() as i16);
let mut packet = Vec::new();
packet.write_u8(0x00);
packet.write_i16::<LittleEndian>(packet_length);
let string = String::from("game name ");
packet.extend(string.as_bytes());
}
Playground
Is it possible to pass a primitive type (such as i32) as a function argument.
For example, I want to read the user input and store it in a vector of my choosing with a function
use std::io;
fn main()
{
get_vectorized_line(i32) ;
}
fn get_vectorized_line(expected_type: type)->Vec<expected_type>{
let mut line_content=String::new();
io::stdin().read_line(&mut line_content).expect("Could not read line");
let vectorized_line: Vec<expected_type> = line_content.trim().split(" ").
map(|s| s.parse().expect("Could not parse")).collect();
return vectorized_line;
}
responds with
expected value, found builtin type i32 when the function is called.
I know it is possible to pass types as arguments in Python, (and not in standard C without using macros).
I believe I could be using generic functions to do what I want. But I wanted clarity on this topic
If you really like the syntax of passing a type as an argument:
macro_rules! gvl {
($t:ty) => {{
let mut content = String::new();
std::io::stdin().read_line(&mut content).unwrap();
content.trim().split(' ').map(
|s| s.parse::<$t>().unwrap()
).collect::<Vec<$t>>()
}}
}
And tah-dah:
let _ = gvl!(i32);
No external type annotations required!
Please don't actually do that
Use first class types with a special syntax that is (almost) what you want:
let _ = gvl::<i32>();
fn gvl<T: FromStr>() -> Vec<T> where <T as FromStr>::Err: Debug {
let mut content=String::new();
stdin().read_line(&mut content).unwrap();
content.trim().split(' ').map(
|s| s.parse().unwrap()
).collect()
}
You can use generics on return types. Here the generic T has to implement FromStr (because it is parsed from str) and must be displayable (for the eventual panic message).
use std::io;
fn get_vectorized_line<T: std::str::FromStr>()->Vec<T> where <T as std::str::FromStr>::Err: std::fmt::Debug {
let mut line_content=String::new();
io::stdin().read_line(&mut line_content).expect("Could not read line");
line_content.trim().split(' ').map(|s| s.parse().unwrap()).collect()
}
fn main()
{
let vectorized_line: Vec<i32> = get_vectorized_line() ;
}
Vecs support std::io::Write, so code can be written that takes a File or Vec, for example. From the API reference, it looks like neither Vec nor slices support std::io::Read.
Is there a convenient way to achieve this? Does it require writing a wrapper struct?
Here is an example of working code, that reads and writes a file, with a single line commented that should read a vector.
use ::std::io;
// Generic IO
fn write_4_bytes<W>(mut file: W) -> Result<usize, io::Error>
where W: io::Write,
{
let len = file.write(b"1234")?;
Ok(len)
}
fn read_4_bytes<R>(mut file: R) -> Result<[u8; 4], io::Error>
where R: io::Read,
{
let mut buf: [u8; 4] = [0; 4];
file.read(&mut buf)?;
Ok(buf)
}
// Type specific
fn write_read_vec() {
let mut vec_as_file: Vec<u8> = Vec::new();
{ // Write
println!("Writing Vec... {}", write_4_bytes(&mut vec_as_file).unwrap());
}
{ // Read
// println!("Reading File... {:?}", read_4_bytes(&vec_as_file).unwrap());
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// Comment this line above to avoid an error!
}
}
fn write_read_file() {
let filepath = "temp.txt";
{ // Write
let mut file_as_file = ::std::fs::File::create(filepath).expect("open failed");
println!("Writing File... {}", write_4_bytes(&mut file_as_file).unwrap());
}
{ // Read
let mut file_as_file = ::std::fs::File::open(filepath).expect("open failed");
println!("Reading File... {:?}", read_4_bytes(&mut file_as_file).unwrap());
}
}
fn main() {
write_read_vec();
write_read_file();
}
This fails with the error:
error[E0277]: the trait bound `std::vec::Vec<u8>: std::io::Read` is not satisfied
--> src/main.rs:29:42
|
29 | println!("Reading File... {:?}", read_4_bytes(&vec_as_file).unwrap());
| ^^^^^^^^^^^^ the trait `std::io::Read` is not implemented for `std::vec::Vec<u8>`
|
= note: required by `read_4_bytes`
I'd like to write tests for a file format encoder/decoder, without having to write to the file-system.
While vectors don't support std::io::Read, slices do.
There is some confusion here caused by Rust being able to coerce a Vec into a slice in some situations but not others.
In this case, an explicit coercion to a slice is needed because at the stage coercions are applied, the compiler doesn't know that Vec<u8> doesn't implement Read.
The code in the question will work when the vector is coerced into a slice using one of the following methods:
read_4_bytes(&*vec_as_file)
read_4_bytes(&vec_as_file[..])
read_4_bytes(vec_as_file.as_slice()).
Note:
When asking the question initially, I was taking &Read instead of Read. This made passing a reference to a slice fail, unless I'd passed in &&*vec_as_file which I didn't think to do.
Recent versions of rust you can also use as_slice() to convert a Vec to a slice.
Thanks to #arete on #rust for finding the solution!
std::io::Cursor
std::io::Cursor is a simple and useful wrapper that implements Read for Vec<u8>, so it allows to use vector as a readable entity.
let mut file = Cursor::new(vector);
read_something(&mut file);
And documentation shows how to use Cursor instead of File to write unit-tests!
Working example:
use std::io::Cursor;
use std::io::Read;
fn read_something(file: &mut impl Read) {
let _ = file.read(&mut [0; 8]);
}
fn main() {
let vector = vec![1, 2, 3, 4];
let mut file = Cursor::new(vector);
read_something(&mut file);
}
From the documentation about std::io::Cursor:
Cursors are typically used with in-memory buffers to allow them to implement Read and/or Write...
The standard library implements some I/O traits on various types which are commonly used as a buffer, like Cursor<Vec<u8>> and Cursor<&[u8]>.
Slice
The example above works for slices as well. In that case it would look like the following:
read_something(&mut &vector[..]);
Working example:
use std::io::Read;
fn read_something(file: &mut impl Read) {
let _ = file.read(&mut [0; 8]);
}
fn main() {
let vector = vec![1, 2, 3, 4];
read_something(&mut &vector[..]);
}
&mut &vector[..] is a "mutable reference to a slice" (a reference to a reference to a part of vector), so I just find the explicit option with Cursor to be more clear and elegant.
Cursor <-> Slice
Even more: if you have a Cursor that owns a buffer, and you need to emulate, for instance, a part of a "file", you can get a slice from the Cursor and pass to the function.
read_something(&mut &file.get_ref()[1..3]);
I need a completely in-memory object that I can give to BufReader and BufWriter. Something like Python's StringIO. I want to write to and read from such an object using methods ordinarily used with Files.
Is there a way to do this using the standard library?
In fact there is a way: Cursor<T>!
(please also read Shepmaster's answer on why often it's even easier)
In the documentation you can see that there are the following impls:
impl<T> Seek for Cursor<T> where T: AsRef<[u8]>
impl<T> Read for Cursor<T> where T: AsRef<[u8]>
impl Write for Cursor<Vec<u8>>
impl<T> AsRef<[T]> for Vec<T>
From this you can see that you can use the type Cursor<Vec<u8>> just as an ordinary file, because Read, Write and Seek are implemented for that type!
Little example (Playground):
use std::io::{Cursor, Read, Seek, SeekFrom, Write};
// Create fake "file"
let mut c = Cursor::new(Vec::new());
// Write into the "file" and seek to the beginning
c.write_all(&[1, 2, 3, 4, 5]).unwrap();
c.seek(SeekFrom::Start(0)).unwrap();
// Read the "file's" contents into a vector
let mut out = Vec::new();
c.read_to_end(&mut out).unwrap();
println!("{:?}", out);
For a more useful example, check the documentation linked above.
You don't need a Cursor most of the time.
object that I can give to BufReader and BufWriter
BufReader requires a value that implements Read:
impl<R: Read> BufReader<R> {
pub fn new(inner: R) -> BufReader<R>
}
BufWriter requires a value that implements Write:
impl<W: Write> BufWriter<W> {
pub fn new(inner: W) -> BufWriter<W> {}
}
If you view the implementors of Read you will find impl<'a> Read for &'a [u8].
If you view the implementors of Write, you will find impl Write for Vec<u8>.
use std::io::{Read, Write};
fn main() {
// Create fake "file"
let mut file = Vec::new();
// Write into the "file"
file.write_all(&[1, 2, 3, 4, 5]).unwrap();
// Read the "file's" contents into a new vector
let mut out = Vec::new();
let mut c = file.as_slice();
c.read_to_end(&mut out).unwrap();
println!("{:?}", out);
}
Writing to a Vec will always append to the end. We also take a slice to the Vec that we can update. Each read of c will advance the slice further and further until it is empty.
The main differences from Cursor:
Cannot seek the data, so you cannot easily re-read data
Cannot write to anywhere but the end
If you want to use BufReader with an in-memory String, you can use the as_bytes() method:
use std::io::BufRead;
use std::io::BufReader;
use std::io::Read;
fn read_buff<R: Read>(mut buffer: BufReader<R>) {
let mut data = String::new();
let _ = buffer.read_line(&mut data);
println!("read_buff got {}", data);
}
fn main() {
read_buff(BufReader::new("Potato!".as_bytes()));
}
This prints read_buff got Potato!. There is no need to use a cursor for this case.
To use an in-memory String with BufWriter, you can use the as_mut_vec method. Unfortunately it is unsafe and I have not found any other way. I don't like the Cursor approach since it consumes the vector and I have not found a way yet to use the Cursor together with BufWriter.
use std::io::BufWriter;
use std::io::Write;
pub fn write_something<W: Write>(mut buf: BufWriter<W>) {
buf.write("potato".as_bytes());
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::{BufWriter};
#[test]
fn testing_bufwriter_and_string() {
let mut s = String::new();
write_something(unsafe { BufWriter::new(s.as_mut_vec()) });
assert_eq!("potato", &s);
}
}