So I don't have a lot of knowledge on this script, compiling as part of a larger project for ARM. This comes from the zksync library which is designed primarily for x86.
I keep running into a conversion error when compiling:
7 | .try_into()
| ^^^^^^^^ the trait `std::convert::From<std::vec::Vec<u8>>` is not implemented for `[u8; 16]`
I tried using try_from() as many of my searches recommended but wasn't able to get it running, as I am still very new to rust, but open to if anyone can get it working.
I am compiling with the following target and linker below -
target - arm-unknown-linux-musleabi
linker - arm-linux-gnueabihf-ld
If anyone can recommend how to to fix this I'd love to hear, I was thinking usize instead of the u64 vars, but couldn't quite figure out if that is the rabbit hole I should try going down.
Full code:
use serde::{Deserialize, Serialize};
use std::convert::TryInto;
/// Defines time range `[valid_from, valid_until]` for which transaction is valid,
/// time format is the same as Ethereum (UNIX timestamp in seconds)
#[derive(Debug, Clone, Copy, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct TimeRange {
pub valid_from: u64,
pub valid_until: u64,
}
impl TimeRange {
pub fn new(valid_from: u64, valid_until: u64) -> Self {
Self {
valid_from,
valid_until,
}
}
pub fn to_be_bytes(&self) -> [u8; 16] {
[
self.valid_from.to_be_bytes(),
self.valid_until.to_be_bytes(),
]
.concat()
.try_into()
.expect("valid_from and valid_until should be u64")
}
pub fn check_correctness(&self) -> bool {
self.valid_from <= self.valid_until
}
pub fn is_valid(&self, block_timestamp: u64) -> bool {
self.valid_from <= block_timestamp && block_timestamp <= self.valid_until
}
}
impl Default for TimeRange {
fn default() -> Self {
Self {
valid_from: 0,
valid_until: u64::max_value(),
}
}
}
Your code compiles successfully on the current stable release (v1.51).
However, TryFrom<Vec<_>> for slices (which is what allows you to use try_into()) was only added in rust 1.48, so maybe you are running a rust version older that 1.48. If so, updating to 1.48+ should fix your problem.
Related
I'm trying to deserialize data in a simple non-human readable and non-self describing format to Rust structs. I've implemented a custom Deserializer for this format and it works great when I'm deserializing the data into a struct like this for example:
#[derive(Serialize, Deserialize)]
pub struct Position {
x: f32,
z: f32,
y: f32,
}
However, let's say this Position struct had a new field added (could have been removed too) in a new version:
#[derive(Serialize, Deserialize)]
pub struct Position {
x: f32,
z: f32,
y: f32,
is_visible: bool, // This field was added in a new version
}
But I still need to support both data from both versions of Position. The version of the data (known at runtime) can be given to the Deserializer but how can the Deserializer know the version of a field (known at compile time)?
I've looked at #[serde(deserialize_with)] but it didn't work because I cannot get the needed version information.
I 've also looked at implementing Deserialize manually for Position and I can receive the versions of the fields of Position by implementing something like Position::get_version(field_name: &str).
However, I cannot figure how to get the version of the data currently being deserialized because Deserialize::deserialize only has a trait bound Deserializer<'de> and I cannot make that bound stricter by adding another bound (so it doesn't know about my custom Deserializer).
At this point, I'm thinking about giving the version data of each field when instantiating the Deserializer but I'm not sure if that will work or if there is a better way to go.
Multiple structs implementing a shared trait
If you have several different versions with several different types of struct, and you want a more robust way of handling different variants, it might be a better idea to write structs for each possible format. You can then define and implement a trait for shared behavior.
trait Position {
fn x(&self) -> f32;
fn y(&self) -> f32;
fn z(&self) -> f32;
fn version_number(&self) -> usize;
}
struct PositionV0 {
x: f32,
y: f32,
z: f32
}
impl Position for PositionV0 {
fn x(&self) -> f32 {
self.x
}
// You get the idea for the fn y, fn z implementations
fn version_number(&self) -> usize {
0
}
}
struct PositionV1 {
x: f32,
y: f32,
z: f32,
is_visible: bool,
}
impl Position for PositionV1 {
fn x(&self) -> f32 {
self.x
}
// You get the idea for the fn y, fn z implementations
fn version_number(&self) -> usize {
1
}
}
Carson's answer is great when you do not have a lot of versions but for me I am working with data structures that range over 20 different versions.
I went with a solution that while I don't think is the most idiomatic, is capable of handling an arbitrary number of versions.
In short:
we implement a Version trait which gives the necessary version info to the Deserializer
Deserializer has VersionedSeqAccess (implements serde::de::SeqAccess) that sets a flag
When flag is set, we put None for that field and immediately unset the flag
The idea is to implement the following trait for the struct:
pub trait Version {
/// We must specify the name of the struct so that any of the fields that
/// are structs won't confuse the Deserializer
fn name() -> &'static str;
fn version() -> VersionInfo;
}
#[derive(Debug, Clone)]
pub enum VersionInfo {
/// Present in all versions
All,
/// Present in this version
Version([u16; 4]),
/// Represent Versions of structs
Struct(&'static [VersionInfo]),
// we can add other ways of expressing the version like a version range for ex.
}
Here is how it will be implemented for the example struct Position. This type of manual deriving is error prone so this can be improved with a derive macro (see end):
struct Position {
x: f32,
z: f32,
y: f32,
is_visible: Option<bool>, // With this solution versioned field must be wrapped in Option
}
impl Version for Position {
fn version() -> VersionInfo {
VersionInfo::Struct(&[
VersionInfo::All,
VersionInfo::All,
VersionInfo::All,
VersionInfo::Version([1, 13, 0, 0]),
])
}
fn name() -> &'static str {
"Position"
}
}
Now, the deserializer will be instansiated with the version of the data format we are currently parsing:
pub struct Deserializer<'de> {
input: &'de [u8],
/// The version the `Deserializer` expect the data format to be
de_version: [u16; 4],
/// Versions of each field. (only used when deserialzing to a struct)
version_info: VersionInfo,
/// Whether to skip deserialzing current item. This flag is set by `VersionedSeqAccess`.
/// When set, the current item is deserialized to `None`
skip: bool,
/// Name of struct we are deserialzing into. We use this to make sure we call the correct
/// visitor for children of this struct who are also structs
name: &'static str,
}
pub fn from_slice<'a, T>(input: &'a [u8], de_version: [u16; 4]) -> Result<T, Error>
where
T: Deserialize<'a> + Version,
{
let mut deserializer = Deserializer::from_slice(input, de_version, T::version(), T::name());
let t = T::deserialize(&mut deserializer)?;
Ok(t)
}
Now that the deserializer has the all the information it needs, this is how we define deserialize_struct:
fn deserialize_struct<V>(
self, name: &'static str, fields: &'static [&'static str], visitor: V,
) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
if name == self.name {
if let VersionInfo::Struct(version_info) = self.version_info {
assert!(version_info.len() == fields.len()); // Make sure the caller implemented version info somewhat correctly. I use a derive macro to implement version so this is not a problem
visitor.visit_seq(VersionedSeqAccess::new(self, fields.len(), &version_info))
} else {
panic!("Struct must always have version info of `Struct` variant")
}
} else {
// This is for children structs of the main struct. We do not support versioning for those
visitor.visit_seq(SequenceAccess::new(self, fields.len()))
}
}
Here is how serde::de::SeqAccess will be implemented for VersionedSeqAccess:
struct VersionedSeqAccess<'a, 'de: 'a> {
de: &'a mut Deserializer<'de>,
version_info: &'static [VersionInfo],
len: usize,
curr: usize,
}
impl<'de, 'a> SeqAccess<'de> for VersionedSeqAccess<'a, 'de> {
type Error = Error;
fn next_element_seed<T>(&mut self, seed: T) -> Result<Option<T::Value>, Error>
where
T: DeserializeSeed<'de>,
{
if self.curr == self.len {
// We iterated through all fields
Ok(None)
} else {
// Get version of the current field
let version = &self.version_info[self.curr as usize];
self.de.version_info = version.clone();
// Set the flag if the version does not match
if !is_correct_version(&self.de.de_version, &version) {
self.de.skip = true;
}
self.curr += 1;
seed.deserialize(&mut *self.de).map(Some)
}
}
}
The final part of the puzzle is inside deserialize_option. If we are at a field not found in current data format the skip flag will be set here and we will produce None:
fn deserialize_option<V>(self, visitor: V) -> Result<V::Value, Self::Error>
where
V: Visitor<'de>,
{
if self.skip == true {
self.skip = false;
visitor.visit_none()
} else {
visitor.visit_some(self)
}
}
A lengthy solution but it works great for my usecase dealing with a lot of structs with lots of fields from different versions. Please do let me know how I can make this less verbose/better. I also implemented a derive macro (not shown here) for the Version trait to be able to do this:
#[derive(Debug, Clone, EventPrinter, Version)]
pub struct Position {
x: f32,
z: f32,
y: f32,
#[version([1, 13, 0, 0])]
is_visible: Option<bool>,
}
With this derive macro, I find that this solution tends to scale well for my usecase.
I define a struct MyData and implement PartialEq and Hash traits for it manually.
I define a enum which includes Rc<MyData> and Rc<RefCell<MyData>>.
I want derive PartialEq and Hash for the enum, but fails:
The PartialEq and Hash both work for Rc<MyData>;
The PartialEq works for Rc<RefCell<MyData>> too;
But the Hash does not work for Rc<RefCell<MyData>> !
I have 2 questions:
Why? Why only the Hash does not work only for Rc<RefCell<MyData>>?
How to fix it?
I can not implement Hash for Rc<RefCell<MyData>>. After searching around I find a way: defining a new wrapper struct, like struct RRWrapper<T> (Rc<RefCell<T>>), and then implement Hash for this RRWrapper. But this will bring much code. Is there an idiomatic way? I think this's a general usage.
Thank in advance,
Wu
PS: In the real code of my program, there is only Rc<RefCell<MyData>> in the enum but no Rc<MyData>. I put Rc<MyData> here just for comparision.
PS2: In the real code of my program, there are more than one Rc<RefCell<T>> in the enum.
Original source code:
use std::rc::Rc;
use std::cell::RefCell;
use std::hash::{Hash, Hasher};
struct MyData {
i: i64,
}
impl Hash for MyData {
fn hash<H: Hasher>(&self, state: &mut H) {
self.hash(state);
}
}
impl PartialEq for MyData {
fn eq(&self, other: &Self) -> bool {
self == other
}
}
#[derive(PartialEq, Hash)]
enum MyEnum {
INT(i64),
STR(String),
MYDATA1(Rc<MyData>), // OK both
MYDATA2(Rc<RefCell<MyData>>), // OK for PartialEq but not for Hash
}
fn main() {
}
Error:
20 | #[derive(PartialEq, Hash)]
| ---- in this derive macro expansion
...
25 | MYDATA2(Rc<RefCell<MyData>>), // OK for PartialEq but not for Hash
| ^^^^^^^^^^^^^^^^^^^ the trait `Hash` is not implemented for `RefCell<MyData>`
|
= note: required because of the requirements on the impl of `Hash` for `Rc<RefCell<MyData>>`
= note: this error originates in the derive macro `Hash` (in Nightly builds, run with -Z macro-backtrace for more info)
Source code of struct RRWrapper:
#[derive(Debug, PartialEq, Eq)]
pub struct RRWrapper<T: Hash+PartialEq+Eq>(Rc<RefCell<T>>);
impl<T: Hash+PartialEq+Eq> Deref for RRWrapper<T> {
type Target = Rc<RefCell<T>>;
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl<T: Hash+PartialEq+Eq> Hash for RRWrapper<T> {
fn hash<H: Hasher>(&self, state: &mut H) {
self.0.borrow().hash(state);
}
}
impl<T: Hash+PartialEq+Eq> Clone for RRWrapper<T> {
fn clone(&self) -> Self {
RRWrapper(self.0.clone())
}
}
impl<T: Hash+PartialEq+Eq> RRWrapper<T> {
pub fn new(inner: T) -> Self {
RRWrapper(Rc::new(RefCell::new(inner)))
}
}
Why? Why only the Hash does not work only for Rc<RefCell>?
If you think about it it makes sense that Hash is not implemented for RefCell. Since it is an abstraction for interior mutability, what could be the hash of something that may change? In general, mutable things are not suitable to work as Hash objects because of that.
How to fix it?
Exactly as you did. Taking responsibility in what you exactly want. Implementing it for a wrapper.
Using the Substrate blockchain framework, how can I convert between Substrate specific types and Rust primitive types and vice versa?
For example:
Convert a time (T::Moment) to a u64
Convert a u64 to a T::Balance
etc...
For the latest in Substrate master
Substrate has removed As in favor of From/Into. An assumption is made that all types are at least u32.
From the trait SimpleArithmatic, the following are implemented:
From: u8, u16, u32
TryFrom: u64, u128, usize
TryInto: u8, u16, u32, u64, u128, usize
Another trait is also provided to provide ergonomic
infallible conversion when you don't care if the value saturates.
UniqueSaturatedInto: u8, u16, u32, u64, u128
UniqueSaturatedFrom: u64, u128
NOTE on SaturatedConversion from Gav
SaturatedConversion (saturated_into and saturated_from) should not be used unless you know what you're doing, you've thought and considered all options and your use-case implies that saturation is fundamentally correct. The only time I imagine this is the case is deep in runtime arithmetic where you are logically certain it will not overflow, but can't provide a proof because it would depend on consistent pre-existing state.
This means that working from u32 to Substrate specific types should be easy:
pub fn u32_to_balance(input: u32) -> T::Balance {
input.into()
}
For larger types, you need to handle the case where the Balance type for a runtime is smaller than what is available:
pub fn u64_to_balance_option(input: u64) -> Option<T::Balance> {
input.try_into().ok()
}
// Note the warning above about saturated conversions
pub fn u64_to_balance_saturated(input: u64) -> T::Balance {
input.saturated_into()
}
When converting from T::Balance to a rust primitive, you need to also handle conversion between incompatible types:
pub fn balance_to_u64(input: T::Balance) -> Option<u64> {
TryInto::<u64>::try_into(input).ok()
}
// Note the warning above about saturated conversions
pub fn balance_to_u64_saturated(input: T::Balance) -> u64 {
input.saturated_into::<u64>()
}
For Substrate v1.0
Substrate provides pub trait As<T> in the sr-primitives crate:
/// Simple trait similar to `Into`, except that it can be used to convert numerics between each
/// other.
pub trait As<T> {
/// Convert forward (ala `Into::into`).
fn as_(self) -> T;
/// Convert backward (ala `From::from`).
fn sa(_: T) -> Self;
}
Here are some working examples of how it can be used:
impl<T: Trait> Module<T> {
// `as_` will turn T::Balance into a u64
pub fn balance_to_u64(input: T::Balance) -> u64 {
input.as_()
}
// Being explicit, you can convert a `u64` to a T::Balance
// using the `As` trait, with `T: u64`, and then calling `sa`
pub fn u64_to_balance(input: u64) -> T::Balance {
<T::Balance as As<u64>>::sa(input)
}
// You can also let Rust figure out what `T` is
pub fn u64_to_balance_implied(input: u64) -> T::Balance {
<T::Balance as As<_>>::sa(input)
}
// You can also let Rust figure out where `sa` is implemented
pub fn u64_to_balance_implied_more(input: u64) -> T::Balance {
T::Balance::sa(input)
}
}
here are few examples which should help in conversion of number to Balance Type:
//declare following import
use frame_support::sp_runtime::traits::Zero;
use frame_support::sp_runtime::SaturatedConversion;
//then saturated_into can be used to convert number into Balance type as follows
let cost_in_u64: u64 = 250; //or expression like 200 + 50;
let cost: BalanceOf<T> = cost_in_u64.saturated_into::<BalanceOf<T>>();
//convert 1010 of type u32 into Balance
let cost2: BalanceOf<T> = 1010u32.into();
//set zero balance
let cost3 = BalanceOf::<T>::zero();
where BalanceOf is defined in cfg as follows
#[cfg(feature = "std")]
type BalanceOf<T> =
<<T as Config>::Currency as Currency<<T as frame_system::Config>::AccountId>>::Balance;
I was playing around with updating a Rust struct in place using chained methods. I found a way to do this, but I was not sure if my code below was idiomatic Rust versus just a workaround.
In particular, I used .to_owned() at the end of the chained method to return the borrowed struct. The code compiles and works just fine. Here is the minimal example.
//struct.rs
#[derive(Debug, Default, Clone, PartialEq)]
pub struct ModelDataCapture {
run: i32,
year: i32,
}
impl ModelDataCapture {
pub fn new() -> Self {
ModelDataCapture::default()
}
pub fn set_run(&mut self, run: i32) -> &mut ModelDataCapture {
self.run = run;
self
}
pub fn set_year(&mut self, year: i32) -> &mut ModelDataCapture {
self.year = year;
self
}
}
//main.rs
let data_capture = ModelDataCapture::new()
.set_run(0)
.set_year(1)
.to_owned(); // <<< QUESTION
println!("here is the data capture {:?}", data_capture);
Is this the proper way to write this in-place modification of the struct? If I do not include the .to_owned() method at the end of the chain, the compile fails with a message that the temporary variable does not live long enough.
Your code "works" but doesn't make sense to me. It:
Creates a value
Mutates the value
Clones the value
Throws away the original value
See the inefficiency? In addition, all the "in-place mutation" is completely discarded, so there's no benefit to it.
I'd generally introduce a binding to mutate:
let mut data_capture = ModelDataCapture::new();
data_capture.set_run(0).set_year(1);
Or go all the way and create a builder that has some equivalent of finish or build
#[derive(Debug)]
struct ModelDataCapture {
run: i32,
year: i32,
}
#[derive(Debug, Default)]
struct ModelDataCaptureBuilder {
run: i32,
year: i32,
}
impl ModelDataCaptureBuilder {
fn set_run(self, run: i32) -> Self {
ModelDataCaptureBuilder { run, ..self }
}
fn set_year(self, year: i32) -> Self {
ModelDataCaptureBuilder { year, ..self }
}
fn build(self) -> ModelDataCapture {
let ModelDataCaptureBuilder { run, year } = self;
ModelDataCapture { run, year }
}
}
fn main() {
let data_capture = ModelDataCaptureBuilder::default().set_run(0).set_year(1).build();
println!("here is the data capture {:?}", data_capture);
}
See Do Rust builder patterns have to use redundant struct code? for more examples of builders that mirror the built items.
You could take self by-value in the first example, but that's annoying in most cases, as you always have to remember to bind the result.
You could change the function to take ownership over self and return self.
Because each "setter" method returns the ownership of self, this code should work out nicely.
For more information, please checkout the rust book
//struct.rs
#[derive(Debug, Default, Clone, PartialEq)]
pub struct ModelDataCapture {
run: i32,
year: i32,
}
impl ModelDataCapture {
pub fn new() -> Self {
ModelDataCapture::default()
}
pub fn set_run(mut self, run: i32) -> ModelDataCapture {
self.run = run;
self
}
pub fn set_year(mut self, year: i32) -> ModelDataCapture {
self.year = year;
self
}
}
fn main() {
//main.rs
let data_capture = ModelDataCapture::new().set_run(0).set_year(1);
println!("here is the data capture {:?}", data_capture);
}
The situation is (severely simplified) this (playpen):
mod tokentree {
pub struct TokenTree;
mod serialize {
use std::collections::BTreeMap;
use super::TokenTree;
#[derive(Debug)]
pub struct InternStr;
pub trait InternStrsExt {
fn intern_strs(&self) -> BTreeMap<&str, InternStr>;
}
impl InternStrsExt for [TokenTree] {
fn intern_strs(&self) -> BTreeMap<&str, InternStr> { BTreeMap::new() }
}
}
pub use self::serialize::{InternStrsExt, InternStr};
}
use tokentree::*;
fn main() {
println!("{:?}", [TokenTree].intern_strs());
}
I get the following error (both on nightly and beta):
<anon>:20:22: 20:47 error: source trait is inaccessible
<anon>:20 println!("{:?}", [TokenTree].intern_strs());
^~~~~~~~~~~~~~~~~~~~~~~~~
My problem is that I don't even know what this is supposed to mean.
It needs a pub declaration. Also your declarations are all over the place. Recommended form is to stick your pub mod declarations first, then, use.
Here is the working example.
mod tokentree {
pub struct TokenTree;
pub mod serialize {
use std::collections::BTreeMap;
use super::TokenTree;
#[derive(Debug)]
pub struct InternStr;
pub trait InternStrsExt {
fn intern_strs(&self) -> BTreeMap<&str, InternStr>;
}
impl InternStrsExt for [TokenTree] {
fn intern_strs(&self) -> BTreeMap<&str, InternStr> { BTreeMap::new() }
}
}
pub use self::serialize::{InternStrsExt, InternStr};
}
pub use tokentree::*;
fn main() {
println!("{:?}", [TokenTree].intern_strs());
}
(playpen)
What happened here is that you stumbled upon following glitches:
https://github.com/rust-lang/rust/issues/18241
https://github.com/rust-lang/rust/issues/16264
You can't export your traits from a private module. That's why you need to change mod serialize into pub mod serialize. For example this playpen example demonstrates that exporting struct Export works, but un-commenting the println! will make it stop compiling, because we used a trait.
Tip: One thing that helps me with the visibility rules is to generate doc files and see which doc files are visible.