How to define &mut [u8] in Rust - rust

I want to call openssl::sign::Signer::sign, which has the signature:
pub fn sign(&self, buf: &mut [u8]) -> Result<usize, ErrorStack>
I'm trying to hand over buf. How to I make it &mut [u8]?
cargo.toml
[dependencies]
openssl = { version = "0.10", features = ["vendored"] }
src/main.rs
use openssl::{base64, sign::Signer};
fn main() {
let mut buffer = [];
let str = base64::encode_block(signer.sign(&mut buffer));
}
But get an error:
openssl::base64::encode_block(signer.sign(&mut buffer));
^^^^^^^^^^^^^^^^^^^ expected `&[u8]`, found enum `std::result::Result`

I'm not familiar with this crate, but base64::encode_block expects &[u8], while the return value of Signer::sign is a Result that returns the number of bytes written as usize.
I'm not sure if the number of bytes written is what you want to encode?
If you want to encode buffer to base64, you could do:
use openssl;
fn main() {
let mut buf = [104, 101, 108, 108, 111];
let encoded = openssl::base64::encode_block(&buf);
println!("{}", encoded);
}
That should give you:
aGVsbG8=
Which is hello as base64.
Or, as an example, if you want to sign the given buf and encode that as base64:
use openssl::{base64, hash::MessageDigest, pkey::PKey, rsa::Rsa, sign::Signer};
fn main() {
let keypair = Rsa::generate(2048).unwrap();
let keypair = PKey::from_rsa(keypair).unwrap();
let signer = Signer::new(MessageDigest::sha256(), &keypair).unwrap();
let mut buf = [104, 101, 108, 108, 111].repeat(52);
let written = signer.sign(&mut buf).unwrap();
println!("bytes written: {}", written);
let encoded = base64::encode_block(&buf);
println!("base64: {}", encoded);
}
Which returns:
bytes written: 256
base64: wB4lBbyzpfRPInxhwm0XVKI3dQYqcUZWNdKyb4RTGDUmwq1DDDbQjMRmFBobRse3pNOxoMy+QQNSXsgI46b75hPfkar9TUowrIGk4Y+ZWWX/cwNJCJazC1dfanP4uft0fzpYJKMSfJTAxswccu1g4yT+u0V5yq+eHbeGDJ+bF2MMhCPds7wGjJguxO0e4wx+HQdVGbU9jrHQ38oIYTChG92iKLRpciiyB8vrbNEBcNNi4hlw6U0sUdz6scpXceREdPFVA6wr0otY3wSZLfcIeKELoBQkR2KPNTTCROreVJ49tXwiQdGe7Ky0NDeNba2H5tKu3uLAtAiG/hVoKEAJG2VsbG8=

Related

Read large f64 binary file into array

I'm looking for help/examples on how to read a relatively large (>12M) binary file of double precision numbers into a rust array. I have metadata on the number of f64 values in the file.
I've read on this and seen the byteorder crate but did not find the documentation/examples particularly helpful.
This is not something that needs to be BufRead, since that likely won't help performance.
Thank you!
The easiest way to do it is to read 8 bytes and convert it to f64 using one of the f64::from_byte-order_bytes() methods:
from_ne_bytes()
from_be_bytes()
from_le_bytes()
These methods are used like that:
let mut buffer = [0u8; 8]; // the buffer can be reused!
reader.read_exact(&mut buffer) ?;
let float = f64::from_be_bytes(buffer);
So you can either read the file 8 bytes at a time or on some larger chunks:
fn main() -> Result<(), Box<dyn Error>> {
let file = File::open("./path/to/file")?;
let mut reader = BufReader::new(file);
let mut buffer = [0u8; 8];
loop {
if let Err(e) = reader.read_exact(&mut buffer) {
// if you know how many bytes are expected, then it's better not to rely on `UnexpectedEof`!
if e.kind() == ErrorKind::UnexpectedEof {
// nothing more to read
break;
}
return Err(e.into());
}
// or use `from_le_bytes()` depending on the byte-order
let float = f64::from_be_bytes(buffer);
//do something with the f64
println!("{}", float);
}
Ok(())
}
If you don't mind adding an additional dependency to your project, then you can also use the ByteOrder crate which has convenience methods to read whole slices:
use byteorder::{ByteOrder, LittleEndian};
let mut bytes = [0; 32]; // the buffer you've read the file into
let mut numbers_got = [0.0; 4];
LittleEndian::read_f64_into(&bytes, &mut numbers_got);
assert_eq!(numbers_given, numbers_got)

Rust - padded array of bytes from str

This rust does exactly what I want, but I don't do much rust and I get the feeling this could be done much better - like maybe in one line. Can anyone give hints to a more "rust idiomatic" way?
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=1f139ccf6e8f88dbe92f1f1e4d7a487a
fn fill_from_str(bytes: &mut [u8], s: &str) {
let mut i=0;
for b in s.as_bytes() {
bytes[i] = *b;
i=i+1;
}
}
fn main() {
let mut bytes: [u8; 10] = [0; 10];
fill_from_str(&mut bytes,"hello");
println!("{:?}",bytes);
}
This can be done very succinctly via std::io::Write which is implemented for &mut [u8]:
use std::io::Write;
fn fill_from_str(mut bytes: &mut [u8], s: &str) {
bytes.write(s.as_bytes()).unwrap();
}
fn main() {
let mut bytes: [u8; 10] = [0; 10];
fill_from_str(&mut bytes, "hello");
println!("{:?}", bytes);
}
[104, 101, 108, 108, 111, 0, 0, 0, 0, 0]
There is a better way to do this: the copy_from_slice method. If the slice and the string are the same length this is a one-liner:
fn copy_from_str(dest:&mut[u8], src:&str){
dest.copy_from_slice(src.as_bytes());
}
The copy_from_slice method also is just a single call to memcpy so it is faster than your version. If you want to support different sizes a little more code is needed:
fn copy_from_str(dest:&mut [u8],src:&str){
if dest.len() == src.len(){
dest.copy_from_slice(src.as_bytes());
} else if dest.len() > src.len(){
dest[..src.len()].copy_from_slice(src.as_bytes());
} else {
dest.copy_from_slice(&src.as_bytes()[..dest.len()]);
}
}
That function will also not panic if it winds up slicing on the boundary of a multibyte character.
Edit: Added Plaground link

How to write incoming stream into a file in warp?

Goal:
The server should be able to receive a stream of binary data and save it to a file.
I'm getting this error:
mismatched types
expected `&[u8]`, found type parameter `B`
How can I get a &[u8] from generic type B?
use warp::Filter;
use warp::{body};
use futures::stream::Stream;
async fn handle_upload<S, B>(stream: S) -> Result<impl warp::Reply, warp::Rejection>
where
S: Stream<Item = Result<B, warp::Error>>,
S: StreamExt,
B: warp::Buf
{
let mut file = File::create("some_binary_file").unwrap();
let pinnedStream = Box::pin(stream);
while let Some(item) = pinnedStream.next().await {
let data = item.unwrap();
file.write_all(data);
}
Ok(warp::reply())
}
#[tokio::main]
async fn main() {
pretty_env_logger::init();
let upload = warp::put()
.and(warp::path("stream"))
.and(body::stream())
.and_then(handle_upload);
warp::serve(upload).run(([127, 0, 0, 1], 3030)).await;
}
B implements warp::Buf which is re-exported from the bytes crate. It has a .bytes() method that returns a &[u8] which may work.
However, the documentation says that .bytes() may return a shorter slice than what it actually contains. So you can call .bytes() and .advance() the stream while it .has_remaining() OR convert it to Bytes and send that to the file:
let mut data = item.unwrap();
file.write_all(data.to_bytes().as_ref());

How do I correctly pass this vector of file lines to my function in Rust?

I'm trying to read a file into a vector, then print out a random line from that vector.
What am I doing wrong?
I'm asking here because I know I'm making a big conceptual mistake, but I'm having trouble identifying exactly where it is.
I know the error -
error[E0308]: mismatched types
26 | processor(&lines)
| ^^^^^^ expected &str, found struct std::string::String
And I see that there's a mismatch - but I don't know how to give the right type, or refactor the code for that (very short) function.
My code is below:
use std::{
fs::File,
io::{prelude::*, BufReader},
path::Path,
};
fn lines_from_file(filename: impl AsRef<Path>) -> Vec<String> {
let file = File::open(filename).expect("no such file");
let buf = BufReader::new(file);
buf.lines()
.map(|l| l.expect("Could not parse line"))
.collect()
}
fn processor(vectr: &Vec<&str>) -> () {
let vec = vectr;
let index = (rand::random::<f32>() * vec.len() as f32).floor() as usize;
println!("{}", vectr[index]);
}
fn main() {
let lines = lines_from_file("./example.txt");
for line in lines {
println!("{:?}", line);
}
processor(&lines);
}
While you're calling the processor function you're trying to pass a Vec<String> which is what the lines_from_file returns but the processor is expecting a &Vec<&str>. You can change the processor to match that expectation:
fn processor(vectr: &Vec<String>) -> () {
let vec = vectr;
let index = (rand::random::<f32>() * vec.len() as f32).floor() as usize;
println!("{}", vectr[index]);
}
The main function:
fn main() {
let lines = lines_from_file("./example.txt");
for line in &lines {. // &lines to avoid moving the variable
println!("{:?}", line);
}
processor(&lines);
}
More generally, a String is not the same as a string slice &str, therefore Vec<String> is not the same as Vec<&str>. I'd recommend checking the rust book: https://doc.rust-lang.org/nightly/book/ch04-03-slices.html?highlight=String#string-slices

How to parse multipart forms using abonander/multipart with Rocket?

This might be useful for me:
I have no idea how you're meant to go about parsing a multipart form
besides doing it manually using just the raw post-data string as input
I will try to adjust the Hyper example but any help will be much appreciated.
Relevant issues:
Support Multipart Forms.
support rocket
Rocket's primary abstraction for data is the FromData trait. Given the POST data and the request, you can construct a given type:
pub trait FromData<'a>: Sized {
type Error;
type Owned: Borrow<Self::Borrowed>;
type Borrowed: ?Sized;
fn transform(
request: &Request,
data: Data
) -> Transform<Outcome<Self::Owned, Self::Error>>;
fn from_data(
request: &Request,
outcome: Transformed<'a, Self>
) -> Outcome<Self, Self::Error>;
}
Then, it's just a matter of reading the API for multipart and inserting tab A into slot B:
#![feature(proc_macro_hygiene, decl_macro)]
use multipart::server::Multipart; // 0.16.1, default-features = false, features = ["server"]
use rocket::{
data::{Data, FromData, Outcome, Transform, Transformed},
post, routes, Request,
}; // 0.4.2
use std::io::Read;
#[post("/", data = "<upload>")]
fn index(upload: DummyMultipart) -> String {
format!("I read this: {:?}", upload)
}
#[derive(Debug)]
struct DummyMultipart {
alpha: String,
one: i32,
file: Vec<u8>,
}
// All of the errors in these functions should be reported
impl<'a> FromData<'a> for DummyMultipart {
type Owned = Vec<u8>;
type Borrowed = [u8];
type Error = ();
fn transform(_request: &Request, data: Data) -> Transform<Outcome<Self::Owned, Self::Error>> {
let mut d = Vec::new();
data.stream_to(&mut d).expect("Unable to read");
Transform::Owned(Outcome::Success(d))
}
fn from_data(request: &Request, outcome: Transformed<'a, Self>) -> Outcome<Self, Self::Error> {
let d = outcome.owned()?;
let ct = request
.headers()
.get_one("Content-Type")
.expect("no content-type");
let idx = ct.find("boundary=").expect("no boundary");
let boundary = &ct[(idx + "boundary=".len())..];
let mut mp = Multipart::with_body(&d[..], boundary);
// Custom implementation parts
let mut alpha = None;
let mut one = None;
let mut file = None;
mp.foreach_entry(|mut entry| match &*entry.headers.name {
"alpha" => {
let mut t = String::new();
entry.data.read_to_string(&mut t).expect("not text");
alpha = Some(t);
}
"one" => {
let mut t = String::new();
entry.data.read_to_string(&mut t).expect("not text");
let n = t.parse().expect("not number");
one = Some(n);
}
"file" => {
let mut d = Vec::new();
entry.data.read_to_end(&mut d).expect("not file");
file = Some(d);
}
other => panic!("No known key {}", other),
})
.expect("Unable to iterate");
let v = DummyMultipart {
alpha: alpha.expect("alpha not set"),
one: one.expect("one not set"),
file: file.expect("file not set"),
};
// End custom
Outcome::Success(v)
}
}
fn main() {
rocket::ignite().mount("/", routes![index]).launch();
}
I've never used either of these APIs for real, so there's no guarantee that this is a good implementation. In fact, all the panicking on error definitely means it's suboptimal. A production usage would handle all of those cleanly.
However, it does work:
%curl -X POST -F alpha=omega -F one=2 -F file=#hello http://localhost:8000/
I read this: DummyMultipart { alpha: "omega", one: 2, file: [104, 101, 108, 108, 111, 44, 32, 119, 111, 114, 108, 100, 33, 10] }
An advanced implementation would allow for some abstraction between the user-specific data and the generic multipart aspects. Something like Multipart<MyForm> would be nice.
The author of Rocket points out that this solution allows a malicious end user to POST an infinitely sized file, which would cause the machine to run out of memory. Depending on the intended use, you may wish to establish some kind of cap on the number of bytes read, potentially writing to the filesystem at some breakpoint.
Official support for multipart form parsing in Rocket is still being discussed. Until then, take a look at the official example on how to integrate the multipart crate with Rocket: https://github.com/abonander/multipart/blob/master/examples/rocket.rs

Resources