I'm trying to convert a Python lib to a Rust one. In this process I need to extract x509 certs from a pkcs7 (or at least the oids).
In Python I used the lib Cryptography:
from typing import List
import base64
from cryptography import x509
from cryptography.hazmat.primitives.serialization import pkcs7
my_sig: str = "something"
signature_decoded: bytes = base64.b64decode(my_sig)
pkcs_certs: List[x509.Certificate] = pkcs7.load_der_pkcs7_certificates(signature_decoded)
Here my_sig is a detached PKCS #7 signature, Base64 encoded as string.
To do my Rust equivalent I tried to use rust crypto. But from what I saw the pkcs7 format is only a struct and don't implement the loading part.
That said I assume I need to implement a trait from the lib that loads my u8 pkcs7 as a pkcs7 struct but I can't find any simple example.
What I have for now:
asn1_extractor.rs
use der::{
asn1::{Any, ObjectIdentifier},
Decodable, Decoder, Encodable, Sequence
};
#[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub struct AlgorithmIdentifier<'a> {
pub algorithm: ObjectIdentifier,
pub parameters: Option<Any<'a>>
}
impl<'a> Decodable<'a> for AlgorithmIdentifier<'a> {
fn decode(decoder: &mut Decoder<'a>) -> der::Result<Self> {
decoder.sequence(|decoder| {
let algorithm = decoder.decode()?;
let parameters = decoder.decode()?;
Ok(Self { algorithm, parameters })
})
}
}
impl<'a> Sequence<'a> for AlgorithmIdentifier<'a> {
fn fields<F, T>(&self, field_encoder: F) -> der::Result<T>
where
F: FnOnce(&[&dyn Encodable]) -> der::Result<T>,
{
field_encoder(&[&self.algorithm, &self.parameters])
}
}
lib.rs
extern crate core;
use pkcs7;
use der::{Decodable, ValueOrd, Decoder, Sequence};
pub mod asn1_extractor;
pub fn pkcs7_oid_verifier(signature: String) -> () {
let bytes = base64::decode(signature).unwrap();
let mut decoder = Decoder::new(&bytes).expect("bruh it doesn work :(");
let asn1 = asn1_extractor::AlgorithmIdentifier::decode(&mut decoder);
// ===> here I want to have a pkcs7 struct as result of the fn
}
Related
I have a library that is used through its rust interface by rust programs, as well as through C/C++ programs through generated cbindgen bindings, so I implemented a free function to free the string once the ffi function has used the string. However I want rust also to control the memory when it is used as a rust lib. How do I achieve this? is it even possible? or is calling the free function manually in rust the only option?
I also tried implementing drop, but that lead to this:
free(): double free detected in tcache 2
[1] 11097 IOT instruction cargo run
This block allows the string to be freed from C/C++, but the string is not freed in rust (valgrind shows definitely lost block). data is assigned using CString::into_raw()
use std::{ffi::CString, os::raw::c_char};
pub struct SomeData {
pub data: *const c_char
}
impl SomeData {
#[no_mangle] pub extern fn free_shared_string(&mut self) {
if !self.data.is_null() {
unsafe { CString::from_raw(self.data.cast_mut()); }
}
}
}
The docs for from_raw warn against doing exactly this.
Safety
This should only ever be called with a pointer that was earlier obtained by calling CString::into_raw. Other usage (e.g., trying to take ownership of a string that was allocated by foreign code) is likely to lead to undefined behavior or allocator corruption.
So do not use from_raw to pretend that a foreign string was allocated using Rust. If you just need to borrow it and let C free it, you should use the CStr type for borrowed strings. If you want to take ownership, you should copy it into a new string, or wrap it in a custom structure that has a Drop implementation capable of freeing the original memory.
You cannot have two different languages owning that memory. Rust is fundamentally built on a single-ownership model, so every piece of memory has a unique owner. There are some (intra-Rust) workarounds for that like Rc, but none of that will translate to C. So pick an owner, and make that language responsible for freeing the data.
The best solution for me was to have a separate feature, used when building the library to be used through C/C++ applications (ie. .a/.so) vs .rlib which cargo will build when included in a rust project through Cargo.toml.
This lets me use the same API from both possible application languages, call free from C/C++ on my string, and drop will free it in rust.
Note: the null character at the end is because the majority of the time my lib is used with C apps, hence storing with null for faster returns for them.
Add default-features = false when adding in Cargo.toml of a rust app.
lib.rs
use std::{ffi::{c_char, CStr, FromBytesWithNulError, CString}, mem::forget, str::Utf8Error, string::FromUtf8Error};
#[cfg(feature = "c-str")]
#[repr(C)]
pub struct SharedString {
str: *const c_char
}
#[cfg(not(feature = "c-str"))]
pub struct SharedString {
str: Vec<u8>
}
#[cfg(feature = "c-str")]
impl SharedString {
pub fn from_bytes(buf: &[u8]) -> Self {
let mut buf = buf.to_vec();
if let Some(c) = buf.last() {
if *c != 0 {
buf.push(0);
}
}
let s = Self { str: buf.as_ptr().cast() };
forget(buf);
s
}
pub unsafe fn get_string(&self) -> Result<String, SharedStringError> {
Ok(CStr::from_ptr(self.str).to_str()?.to_owned())
}
pub unsafe fn free(&self) {
if !self.str.is_null() {
let _ = CString::from_raw(self.str.cast_mut());
}
}
}
#[cfg(not(feature = "c-str"))]
impl SharedString {
pub fn from_bytes(buf: &[u8]) -> Self {
let mut buf = buf.to_vec();
if let Some(c) = buf.last() {
if *c != 0 {
buf.push(0);
}
}
Self { str: buf }
}
pub fn get_string(&self) -> Result<String, SharedStringError> {
let mut s = self.str.clone();
if let Some(c) = s.last() {
if *c == 0 {
s.pop();
}
}
String::from_utf8(s).map_err(|e| e.into())
}
// do nothing because rust vec will get dropped automatically
pub fn free(&self) {}
}
// Just for proof of concept
#[derive(Debug)]
pub enum SharedStringError {
NullError,
Utf8Error
}
impl From<FromBytesWithNulError> for SharedStringError {
fn from(_: FromBytesWithNulError) -> Self {
Self::NullError
}
}
impl From<Utf8Error> for SharedStringError {
fn from(_: Utf8Error) -> Self {
Self::Utf8Error
}
}
impl From<FromUtf8Error> for SharedStringError {
fn from(_: FromUtf8Error) -> Self {
Self::Utf8Error
}
}
Cargo.toml
[package]
name = "mylib"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
[features]
default = ["c-str"]
c-str = []
I Have a substrate pallet implementation as follows
#[pallet::storage]
#[pallet::getter(fn get_payload)]
pub(super) type Payload<T: Config> = StorageMap<
_,
Blake2_128Concat,
Vec<u8>,
Messages<T>
>;
#[derive(Clone, Decode, Encode, Eq, PartialEq, Debug)]
pub struct Messages<T: Config> {
pub meta: Option<Vec<u8>>,
pub header: Option<Vec<u8>>,
}
#[pallet::weight(0)]
pub fn update(
origin: OriginFor<T>,
key: Vec<u8>,
header: Vec<u8>,
) -> DispatchResultWithPostInfo {
let origin_account = ensure_signed(origin)?;
let mut payload = Payload::<T>::get(key.clone());
match payload {
Some(mut val) => {
println!("{:?}",val.header);
<Payload<T>>::mutate(val, val.header=header)
},
None => println!("Not found")
}
and in the update function, when I tried to mutate it throws the following error
the trait `EncodeLike<Vec<u8>>` is not implemented for `Messages<T>`
The struct which corresponds to the Messages derived from Encode and Decode. Documentation is unclear on how to fix this. How to fix this issue?
The error message is correct, Messages<T> doesn't encode like a Vec<u8> and somewhere you try to use Mesages<T> where it only takes something that encodes like a Vec<u8>.
In <Payload<T>>::mutate call: the first argument must be something that encode like Vec<u8> and you give a variable of type Messages<T, it is an error. instead you probably wants to write <Payload<T>>::mutate(key.clone, ..)
Wondering if there's a "proper" way of converting an Enum to a &str and back.
The problem I'm trying to solve:
In the clap crate, args/subcommands are defined and identified by &strs. (Which I'm assuming don't fully take advantage of the type checker.) I'd like to pass a Command Enum to my application instead of a &str which would be verified by the type-checker and also save me from typing (typo-ing?) strings all over the place.
This is what I came up with from searching StackOverflow and std:
use std::str::FromStr;
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum Command {
EatCake,
MakeCake,
}
impl FromStr for Command {
type Err = ();
fn from_str(s: &str) -> std::result::Result<Self, Self::Err> {
match s.to_ascii_lowercase().as_str() {
"eat-cake" => Ok(Self::EatCake),
"make-cake" => Ok(Self::MakeCake),
_ => Err(()),
}
}
}
impl<'a> From<Command> for &'a str {
fn from(c: Command) -> Self {
match c {
Command::EatCake => "eat-cake",
Command::MakeCake => "make-cake",
}
}
}
fn main() {
let command_from_str: Command = "eat-cake".to_owned().parse().unwrap();
let str_from_command: &str = command_from_str.into();
assert_eq!(command_from_str, Command::EatCake);
assert_eq!(str_from_command, "eat-cake");
}
And here's a working playground:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=b5e9ac450fd6a79b855306e96d4707fa
Here's an abridged version of what I'm running in clap.
let matches = App::new("cake")
.setting(AppSettings::SubcommandRequiredElseHelp)
// ...
.subcommand(
SubCommand::with_name(Command::MakeCake.into())
// ...
)
.subcommand(
SubCommand::with_name(Command::EatCake.into())
// ...
)
.get_matches();
It seems to work, but I'm not sure if I'm missing something / a bigger picture.
Related:
How to use an internal library Enum for Clap Args
How do I return an error within match statement while implementing from_str in rust?
Thanks!
The strum crate may save you some work. Using strum I was able to get the simple main() you have to work without any additional From implementations.
use strum_macros::{Display, EnumString, IntoStaticStr};
#[derive(Debug, Clone, Copy, PartialEq)]
#[derive(Display, EnumString, IntoStaticStr)] // strum macros.
pub enum Command {
#[strum(serialize = "eat-cake")]
EatCake,
#[strum(serialize = "make-cake")]
MakeCake,
}
fn main() {
let command_from_str: Command = "eat-cake".to_owned().parse().unwrap();
let str_from_command: &str = command_from_str.into();
assert_eq!(command_from_str, Command::EatCake);
assert_eq!(str_from_command, "eat-cake");
}
I want to convert multiple env.variables to static struct.
I can do it mannually:
Env {
is_development: env::var("IS_DEVELOPMENT")
.unwrap()
.parse::<bool>()
.unwrap(),
server: Server {
host: env::var("HOST").unwrap(),
port: env::var("PORT")
.unwrap()
.parse::<u16>()
.unwrap(),
},
}
But when there is multiple values, it's became bloated. Is there a way to make generic helper function that will give me value that i specify or panic? Something like this (or another solution):
fn get_env_var<T>(env_var_name: String) -> T {
// panic is ok here
let var = env::var(env_var_name).unwrap();
T::from(var)
}
get_env_var<u16>("PORT") // here i got u16
get_env_var<bool>("IS_DEVELOPMENT") // here is my boolean
Full example
use crate::server::logger::log_raw;
use dotenv::dotenv;
use serde::Deserialize;
use std::env;
#[derive(Deserialize, Debug, Clone)]
pub struct Server {
pub host: String,
pub port: u16,
}
#[derive(Deserialize, Debug, Clone)]
pub struct Env {
pub is_development: bool,
pub server: Server,
}
impl Env {
pub fn init() -> Self {
dotenv().expect(".env loading fail");
// how can i specify what type i expect?
fn get_env_var<T>(env_var_name: String) -> T {
// panic is ok here
let var = env::var(env_var_name).unwrap();
T::from(var)
}
// instead this
Env {
is_development: env::var("IS_DEVELOPMENT")
.unwrap()
.parse::<bool>()
.unwrap(),
server: Server {
host: env::var("HOST").unwrap(),
port: env::var("PORT")
.unwrap()
.parse::<u16>()
.unwrap(),
},
}
// do something like this
/*
Env {
is_development: get_env_var<bool>("IS_DEVELOPMENT"),
server: Server {
host: get_env_var<String>("HOST"),
port: get_env_var<u16>("PORT"),
},
}
*/
}
}
lazy_static! {
pub static ref ENV: Env = Env::init();
}
Like in your manual version, where you use str::parse, you can have the same requirement as str::parse, which is FromStr. So if you include the T: FromStr requirement, then you'll be able to do var.parse::<T>().
use std::env;
use std::fmt::Debug;
use std::str::FromStr;
fn get_env_var<T>(env_var_name: &str) -> T
where
T: FromStr,
T::Err: Debug,
{
let var = env::var(env_var_name).unwrap();
var.parse::<T>().unwrap()
}
Then if you run the following by executing PORT=1234 IS_DEVELOPMENT=true cargo run.
fn main() {
println!("{}", get_env_var::<u16>("PORT"));
println!("{}", get_env_var::<bool>("IS_DEVELOPMENT"));
}
Then it will output:
1234
true
Alternatively, you might want to be able to handle VarError::NotPresent and fallback to a default.
use std::env::{self, VarError};
use std::fmt::Debug;
use std::str::FromStr;
fn get_env_var<T>(env_var_name: &str) -> Result<T, VarError>
where
T: FromStr,
T::Err: Debug,
{
let var = env::var(env_var_name)?;
Ok(var.parse().unwrap())
}
Now if you only executed PORT=1234 cargo run, then it would make it easier to do this:
let is_dev = get_env_var::<bool>("IS_DEVELOPMENT")
.map_err(|err| match err {
VarError::NotPresent => Ok(false),
err => Err(err),
})
.unwrap();
println!("{:?}", is_dev);
If you want to fallback to Default if VarError::NotPresent:
fn get_env_var<T>(env_var_name: &str) -> T
where
T: FromStr,
T::Err: Debug,
T: Default,
{
let var = match env::var(env_var_name) {
Err(VarError::NotPresent) => return T::default(),
res => res.unwrap(),
};
var.parse().unwrap()
}
Rust genericity, inspired by Haskell's works through traits and specifically trait bounds. This means when you write
fn get_env_var<T>(env_var_name: String) -> T
since there is no trait bound on T there are essentially no capabilities for it (this is rather unlike C++).
Therefore, as far as rustc is concerned, pretty much the only thing it can do with a T is... take one as parameter then return it as-is.
Thus to do anything useful with a T (including creating one, whether from something else or de novo) you need to use the correct trait and provide the correct trait bounds.
The From trait is entirely the wrong trait to involve here: it specifies total (never-failing) conversions e.g. converting a u16 to a u32, which can never fail.
Whether it's converting a String to a bool or a u16, the conversion is quite obviously less than total: there is an infinity of string values which are not sequences of decimal digits describing a number below 2^16.
In Rust, the signifier of failabibility tends to be Try. There is a TryFrom trait, however for historical reasons and as it documents in its signature the str::parse method is hooked on the FromStr trait.
This means in order to declare that your T can be created from a string (and use the parse method to create one), you need to bound T to FromStr. And of course indicate that it may fail, and will return whatever error T generates when it can't be parsed from a string:
fn get_env_var<T: FromStr>(env_var_name: String) -> Result<T, T::Err> {
let var = env::var(env_var_name).unwrap();
var.parse()
}
Incidentally, taking a String as input is usually avoided unless you really have to[0]. Usually you'd take an &str, that's a lot more flexible as it can be used e.g. with string literals (which are of type &'static str).
So
fn get_env_var<T: FromStr>(env_var_name: &str) -> Result<T, T::Err> {
let var = env::var(env_var_name).unwrap();
var.parse()
}
[0] or for efficiency purposes sometimes
The code below is the beginnings of a small library I'm writing to talk to a web API. Users of the library will instantiate a client MyClient and access the web API through it. Here, I'm trying to get an access token from the API before making requests to it.
In get_new_access() I'm able to make the request and receive the JSON response. I then try to use serde to turn the response into an Access struct, and this is where the problems start.
I've created a library specific error enum MyError which can represent the JSON deserializing and reqwest errors that could occur within get_new_access(). However, when I go to compile I get the trait serde::Deserialize<'_> is not implemented for MyError. My understanding is that this is happening because in the case that I get one of the aforementioned errors, serde does not know how to deserialize it into an Access struct. Of course, I don't want it to do that at all, so my question is what should I do?
I've looked at various serde deserialize examples, but all of them seem to assume that they are running in a main function that can only return a serde error. If I put #[derive(Deserialize)] above MyError's declaration, then I get the same error, but it shifts to reqwest::Error and serde_json::Error instead.
use std::error;
use std::fmt;
extern crate chrono;
extern crate reqwest;
#[macro_use]
extern crate serde_derive;
extern crate serde;
extern crate serde_json;
use chrono::prelude::*;
use reqwest::Client;
pub struct MyClient {
access: Access,
token_expires: DateTime<Utc>,
}
#[derive(Deserialize, Debug)]
struct Access {
access_token: String,
expires_in: i64,
token_type: String,
}
fn main() {
let sc: MyClient = MyClient::new();
println!("{:?}", &sc.access);
}
impl MyClient {
pub fn new() -> MyClient {
let a: Access = MyClient::get_new_access().expect("Couldn't get Access");
let e: DateTime<Utc> = chrono::Utc::now(); //TODO
MyClient {
access: a,
token_expires: e,
}
}
fn get_new_access() -> Result<Access, MyError> {
let params = ["test"];
let client = Client::new();
let json = client
.post(&[""].concat())
.form(¶ms)
.send()?
.text()
.expect("Couldn't get JSON Response");
println!("{}", &json);
serde_json::from_str(&json)?
//let a = Access {access_token: "Test".to_string(), expires_in: 3600, token_type: "Test".to_string() };
//serde_json::from_str(&json)?
}
}
#[derive(Debug)]
pub enum MyError {
WebRequestError(reqwest::Error),
ParseError(serde_json::Error),
}
impl fmt::Display for MyError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "eRROR")
}
}
impl error::Error for MyError {
fn description(&self) -> &str {
"API internal error"
}
fn cause(&self) -> Option<&error::Error> {
// Generic error, underlying cause isn't tracked.
None
}
}
impl From<serde_json::Error> for MyError {
fn from(e: serde_json::Error) -> Self {
MyError::ParseError(e)
}
}
impl From<reqwest::Error> for MyError {
fn from(e: reqwest::Error) -> Self {
MyError::WebRequestError(e)
}
}
Playground link here.
Your first problem is that your fn get_new_access() -> Result<Access, MyError> expects a Result. But in here:
//...
serde_json::from_str(&json)?
}
because of using ?(try macro), you are trying to return Result's unwrapped value which is a subtype of serde::Deserialize<'_>. The compiler warns you about this Deserialize is not a Result. What you should do is just return the result without unwrapping it:
//...
serde_json::from_str(&json)
}
Or
//...
let access = serde_json::from_str(&json)?; // gets access or propagates error
Ok(access) //if no error return access in a Result
}
Then you will have a second problem because your function expects MyError in the Result while you are returning serde_json::Error with this call serde_json::from_str(&json). Luckily Result has the function map_err which maps the actual error type to your custom error type.
This code will solve your problem:
//...
serde_json::from_str(&json).map_err(MyError::ParseError)
}
For the request in the comment :
For example, if I change the web request line to let json = client.post("").form(¶ms).send().map_err(MyError::WebRequestError)?.text()?;,
is that better practice at all?
Yes but since text() returns a Result you need to map it's error as MyError too. Since both send and text has same error type(reqwest::Error) you can combine the results with and_then :
let json = client
.post(&[""].concat())
.form(¶ms)
.send()
.and_then(Response::text) //use reqwest::Response;
.map_err(MyError::WebRequestError)?;