I have this lib.rs file.
use std::io::{ Result, Read };
pub trait ReadExt: Read {
/// Read all bytes until EOF in this source, returning them as a new `Vec`.
///
/// See `read_to_end` for other semantics.
fn read_into_vec(&mut self) -> Result<Vec<u8>> {
let mut buf = Vec::new();
let res = self.read_to_end(&mut buf);
res.map(|_| buf)
}
/// Read all bytes until EOF in this source, returning them as a new buffer.
///
/// See `read_to_string` for other semantics.
fn read_into_string(&mut self) -> Result<String> {
let mut buf = String::new();
let res = self.read_to_string(&mut buf);
res.map(|_| buf)
}
}
impl<T> ReadExt for T where T: Read {}
And now I want to write tests for it in a separate test/lib.rs
extern crate readext;
use std::io::{Read,Cursor};
use readext::ReadExt;
#[test]
fn test () {
let bytes = b"hello";
let mut input = Cursor::new(bytes);
let s = input.read_into_string();
assert_eq!(s, "hello");
}
But Rust keeps telling me
type std::io::cursor::Cursor<&[u8; 5]> does not implement any method in scope named read_into_string
I don't know why. Obviously I'm useing it already. Confused.
The answer is already in the error:
type std::io::cursor::Cursor<&[u8; 5]> does not implement any method
in scope named read_into_string
The problem is, Cursor<&[u8; 5]> does not implement Read because the wrapped type is pointer to a fixed-size array instead of a slice, and so it does not implement your trait either. I guess something along these lines should work:
#[test]
fn test () {
let bytes = b"hello";
let mut input = Cursor::new(bytes as &[u8]);
let s = input.read_into_string();
assert_eq!(s, "hello");
}
This way input is of type Cursor<&[u8]> which implements Read and so should implement your trait too.
Related
The following is only an example. If there's a native solution for this exact problem with reading bytes - cool, but my goal is to learn how to do it by myself, for any other purpose as well.
I'd like to do something like this: (pseudo-code below)
let mut reader = Reader::new(bytesArr);
let int32: i32 = reader.read(); // separate implementation to read 4 bits and convert into int32
let int64: i64 = reader.read(); // separate implementation to read 8 bits and convert into int64
I imagine it looking like this: (pseudo-code again)
impl Reader {
read<T>(&mut self) -> T {
// if T is i32 ... else if ...
}
}
or like this:
impl Reader {
read(&mut self) -> i32 {
// ...
}
read(&mut self) -> i64 {
// ...
}
}
But haven't found anything relatable yet.
(I actually have, for the first case (if T is i32 ...), but it looked really unreadable and inconvenient)
You could do this by having a Readable trait which you implement on i32 and i64, which does the operation. Then on Reader you could have a generic function which takes any type that is Readable and return it, for example:
struct Reader {
n: u8,
}
trait Readable {
fn read_from_reader(reader: &mut Reader) -> Self;
}
impl Readable for i32 {
fn read_from_reader(reader: &mut Reader) -> i32 {
reader.n += 1;
reader.n as i32
}
}
impl Readable for i64 {
fn read_from_reader(reader: &mut Reader) -> i64 {
reader.n += 1;
reader.n as i64
}
}
impl Reader {
fn read<T: Readable>(&mut self) -> T {
T::read_from_reader(self)
}
}
fn main() {
let mut r = Reader { n: 0 };
let int32: i32 = r.read();
let int64: i64 = r.read();
println!("{} {}", int32, int64);
}
You can try it on the playground
After some trials and searches, I found that implementing them in current Rust seems a bit difficult, but not impossible.
Here is the code, I'll explain it afterwards:
#![feature(generic_const_exprs)]
use std::{
mem::{self, MaybeUninit},
ptr,
};
static DATA: [u8; 8] = [
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
];
struct Reader;
impl Reader {
fn read<T: Copy + Sized>(&self) -> T
where
[(); mem::size_of::<T>()]: ,
{
let mut buf = [unsafe { MaybeUninit::uninit().assume_init() }; mem::size_of::<T>()];
unsafe {
ptr::copy_nonoverlapping(DATA.as_ptr(), buf.as_mut_ptr(), buf.len());
mem::transmute_copy(&buf)
}
}
}
fn main() {
let reader = Reader;
let v_u8: u8 = reader.read();
dbg!(v_u8);
let v_u16: u16 = reader.read();
dbg!(v_u16);
let v_u32: u32 = reader.read();
dbg!(v_u32);
let v_u64: u64 = reader.read();
dbg!(v_u64);
}
Suppose the global static variable DATA is the target data you want to read.
In current Rust, we cannot directly use the size of a generic parameter as the length of an array. This does not work:
fn example<T: Copy + Sized>() {
let mut _buf = [0_u8; mem::size_of::<T>()];
}
The compiler gives a weird error:
error: unconstrained generic constant
--> src\main.rs:34:31
|
34 | let mut _buf = [0_u8; mem::size_of::<T>()];
| ^^^^^^^^^^^^^^^^^^^
|
= help: try adding a `where` bound using this expression: `where [(); mem::size_of::<T>()]:`
There is an issue that is tracking it, if you want to go deeper into this error you can take a look.
We just follow the compiler's suggestion to add a where bound. This requires feature generic_const_exprs to be enabled.
Next, unsafe { MaybeUninit::uninit().assume_init() } is optional, which drops the overhead of initializing this array, since we will eventually overwrite it completely. You can replace it with 0_u8 if you don't like it.
Finally, copy the data you need and transmute this array to your generic type, return.
I think you will see the output you expect:
[src\main.rs:38] v_u8 = 255
[src\main.rs:41] v_u16 = 65535
[src\main.rs:44] v_u32 = 4294967295
[src\main.rs:47] v_u64 = 18446744073709551615
I am working on a Linux-PAM module and I implemented this function to get the user.
pub fn get_user(pamh: PamHandleT) -> PamResult<&'static CStr> {
let mut raw_user: *const c_char = ptr::null();
let r = unsafe { pam_get_user(pamh, &mut raw_user, ptr::null()) };
if raw_user.is_null() {
Err(r)
} else {
let user = unsafe {CStr::from_ptr(raw_user)};
Ok(user)
}
}
pam_get_user is a C function from libpam that returns a * const c_char via its second argument. PAM documentation states that I must not free that pointer to allow interoperability with other modules.
By using the 'static lifetime for the return value, I believe this value will not be deallocated, is that correct? Maybe I could copy the value to use it in a more Rust-idiomatic way, how could I do that?
CStr is responsible for handling the value and in contrast to CString it does not allocate or deallocate memory, just like str and String. You just pass a pointer to it and have to ensure its requirements. Make sure to read std::ffi::CStr carefully and understand what you are doing.
Your code looks fine so far, so you should be ready to go.
You, probably, need to make owned value. This would allocate though.
pub fn get_user(pamh: PamHandleT) -> PamResult<CString> {
let mut raw_user: *const c_char = ptr::null();
let r = unsafe { pam_get_user(pamh, &mut raw_user, ptr::null()) };
if raw_user.is_null() {
Err(r)
} else {
let user = unsafe {CStr::from_ptr(raw_user)};
Ok(user.to_owned())
}
}
If you want to avoid allocation, you should create some context object.
"The pam_end function terminates the PAM transaction and is the last function an application should call in the PAM context. Upon return the handle pamh is no longer valid and all memory associated with it will be invalid. "
struct TransactionContext{
pamh: PamHandleT
}
impl Drop for TransactionContext{
fn drop(&mut self){
unsafe {pam_end(pamh);}
}
}
pub fn get_user(pamh: &TransactionContext) -> PamResult<&CStr> {
let mut raw_user: *const c_char = ptr::null();
let r = unsafe { pam_get_user(pamh.pamh, &mut raw_user, ptr::null()) };
if raw_user.is_null() {
Err(r)
} else {
let user = unsafe {CStr::from_ptr(raw_user)};
Ok(user)
}
}
This would make result CStr to have same lifetime as TransactionContext and borrow checker would ensure that you don't use result CStr after your TransactionContext is dropped.
I'm trying to read a file into a vector, then print out a random line from that vector.
What am I doing wrong?
I'm asking here because I know I'm making a big conceptual mistake, but I'm having trouble identifying exactly where it is.
I know the error -
error[E0308]: mismatched types
26 | processor(&lines)
| ^^^^^^ expected &str, found struct std::string::String
And I see that there's a mismatch - but I don't know how to give the right type, or refactor the code for that (very short) function.
My code is below:
use std::{
fs::File,
io::{prelude::*, BufReader},
path::Path,
};
fn lines_from_file(filename: impl AsRef<Path>) -> Vec<String> {
let file = File::open(filename).expect("no such file");
let buf = BufReader::new(file);
buf.lines()
.map(|l| l.expect("Could not parse line"))
.collect()
}
fn processor(vectr: &Vec<&str>) -> () {
let vec = vectr;
let index = (rand::random::<f32>() * vec.len() as f32).floor() as usize;
println!("{}", vectr[index]);
}
fn main() {
let lines = lines_from_file("./example.txt");
for line in lines {
println!("{:?}", line);
}
processor(&lines);
}
While you're calling the processor function you're trying to pass a Vec<String> which is what the lines_from_file returns but the processor is expecting a &Vec<&str>. You can change the processor to match that expectation:
fn processor(vectr: &Vec<String>) -> () {
let vec = vectr;
let index = (rand::random::<f32>() * vec.len() as f32).floor() as usize;
println!("{}", vectr[index]);
}
The main function:
fn main() {
let lines = lines_from_file("./example.txt");
for line in &lines {. // &lines to avoid moving the variable
println!("{:?}", line);
}
processor(&lines);
}
More generally, a String is not the same as a string slice &str, therefore Vec<String> is not the same as Vec<&str>. I'd recommend checking the rust book: https://doc.rust-lang.org/nightly/book/ch04-03-slices.html?highlight=String#string-slices
I'm trying to figure out build a feature which requires reading the contents of a file into a futures::stream::BoxStream but I'm having a tough time figuring out what I need to do.
I have figured out how to read a file byte by byte via Bytes which implements an iterator.
use std::fs::File;
use std::io::prelude::*;
use std::io::{BufReader, Bytes};
// TODO: Convert this to a async Stream
fn async_read() -> Box<dyn Iterator<Item = Result<u8, std::io::Error>>> {
let f = File::open("/dev/random").expect("Could not open file");
let reader = BufReader::new(f);
let iter = reader.bytes().into_iter();
Box::new(iter)
}
fn main() {
ctrlc::set_handler(move || {
println!("received Ctrl+C!");
std::process::exit(0);
})
.expect("Error setting Ctrl-C handler");
for b in async_read().into_iter() {
println!("{:?}", b);
}
}
However, I've been struggling a bunch trying to figure out how I can turn this Box<dyn Iterator<Item = Result<u8, std::io::Error>>> into an Stream.
I would have thought something like this would work:
use futures::stream;
use std::fs::File;
use std::io::prelude::*;
use std::io::{BufReader, Bytes};
// TODO: Convert this to a async Stream
fn async_read() -> stream::BoxStream<'static, dyn Iterator<Item = Result<u8, std::io::Error>>> {
let f = File::open("/dev/random").expect("Could not open file");
let reader = BufReader::new(f);
let iter = reader.bytes().into_iter();
std::pin::Pin::new(Box::new(stream::iter(iter)))
}
fn main() {
ctrlc::set_handler(move || {
println!("received Ctrl+C!");
std::process::exit(0);
})
.expect("Error setting Ctrl-C handler");
while let Some(b) = async_read().poll() {
println!("{:?}", b);
}
}
But I keep getting a ton of compiler errors, I've tried other permutations but generally getting no where.
One of the compiler errors:
std::pin::Pin::new
``` --> src/main.rs:14:24
|
14 | std::pin::Pin::new(Box::new(stream::iter(iter)))
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected trait object `dyn std::iter::Iterator`, found enum `std::result::Result`
Anyone have any advice?
I'm pretty new to Rust, and specifically Streams/lower level stuff so I apologize if I got anything wrong, feel free to correct me.
For some additional background, I'm trying to do this so you can CTRL-C out of a command in nushell
I think you are overcomplicating it a bit, you can just return impl Stream from async_read, there is no need to box or pin (same goes for the original Iterator-based version). Then you need to set up an async runtime in order to poll the stream (in this example I just use the runtime provided by futures::executor::block_on). Then you can call futures::stream::StreamExt::next() on the stream to get a future representing the next item.
Here is one way to do this:
use futures::prelude::*;
use std::{
fs::File,
io::{prelude::*, BufReader},
};
fn async_read() -> impl Stream<Item = Result<u8, std::io::Error>> {
let f = File::open("/dev/random").expect("Could not open file");
let reader = BufReader::new(f);
stream::iter(reader.bytes())
}
async fn async_main() {
while let Some(b) = async_read().next().await {
println!("{:?}", b);
}
}
fn main() {
ctrlc::set_handler(move || {
println!("received Ctrl+C!");
std::process::exit(0);
})
.expect("Error setting Ctrl-C handler");
futures::executor::block_on(async_main());
}
I'd like to have a function use a MemWriter to write some bytes and then return a pointer to the buffer. I'm struggling to understand how to use lifetimes in this case. How would I make the below code work and what should I read to fill my knowledge gap here?
struct Request<T: Encodable> {
id: i16,
e: T
}
impl <T: Encodable> Request<T> {
fn serialize<'s>(&'s self) -> io::IoResult<&'s Vec<u8>> {
let mut writer = io::MemWriter::new();
try!(writer.write_be_i16(0 as i16));
let buf = writer.unwrap();
let size = buf.len();
let result: io::IoResult<&Vec<u8>> = Ok(&buf);
result
}
}
You can't return a reference to a buffer that is stored nowhere
You need to store your buffer internally, or you would try to return a reference to freed memory, which is dangerous and thus forbidden by the lifetime checker.
For example like this :
struct Request<T: Encodable> {
buf: Vec<u8>
}
impl <T: Encodable> Request<T> {
fn serialize<'s>(&'s mut self) -> io::IoResult<&'s Vec<u8>> { //'
let mut writer = io::MemWriter::new();
try!(writer.write_be_i16(0 as i16));
self.buf = writer.unwrap();
let size = self.buf.len();
let result: io::IoResult<&Vec<u8>> = Ok(&self.buf);
result
}
}
Or, as Vladimir Matveev pointed out in the comments, you can simply return the Vec. Vec is already a container safely managing memory on the heap, returning it directly should be good for you in most situations, and this way you avoid any lifetime issues.
impl <T: Encodable> Request<T> {
fn serialize(&mut self) -> io::IoResult<Vec<u8>> {
let mut writer = io::MemWriter::new();
try!(writer.write_be_i16(0 as i16));
let buf = writer.unwrap();
let size = buf.len();
Ok(buf)
}
}