Extending borrow lifetimes in rust - reference

I'm trying to parse a series to tokentrees, but when I try to implement my parsing trait I get an error related to reference lifetimes. I thought creating a boxed version would move around any issues with reference counts or lifetimes. The code is as follows.
impl Parse for TokenTree {
fn parse(&mut self) -> Tree {
match self.clone() {
TtDelimited(_, y) => {
let mut y2 = box (*y).clone();
match y2.delim {
token::DelimToken::Paren => y2.parse(),
_ => panic!("not done yet"),
}
}
TtToken(_, t) => E(t),
_ => panic!("not done yet"),
}
}
}
the errors I get make the issue clear, but I can't find any information on solving this particular issue.
35:51 error: `*y2` does not live long enough
token::DelimToken::Paren => y2.parse(),
^~
42:6 note: reference must be valid for the anonymous lifetime #1 defined on the block at 30:31...
fn parse(&mut self) -> Tree{
match self.clone(){
TtDelimited(_, y) => {
let mut y2 = box () (*y).clone();
match y2.delim{
token::DelimToken::Paren => y2.parse(),
...
38:14 note: ...but borrowed value is only valid for the block at 32:33
TtDelimited(_, y) => {
let mut y2 = box () (*y).clone();
match y2.delim{
token::DelimToken::Paren => y2.parse(),
_ => panic!("not done yet"),
}

In this code:
{
let mut y2 = box (*y).clone();
match y2.delim {
token::DelimToken::Paren => y2.parse(),
_ => panic!("not done yet"),
}
}
You create y2, which will only live until the block exits. You didn't include your trait, but my guess is that parse returns a reference to the object itself, something like:
fn parse(&self) -> &str
The reference can only last as long as the object, otherwise it'd point to invalid data.
Edit: How it could maybe, possibly work
You have to track the lifetime of the string you are tokenizing and tie your tokens lifetime to that input:
enum Token<'a> {
Identifier(&'a str),
Whitespace(&'a str),
}
trait Parser {
// Important! The lifetime of the output is tied to the parameter `input`,
// *not* to the structure that implements this!
fn parse<'a>(&self, input: &'a str) -> Token<'a>;
}
struct BasicParser;
impl Parser for BasicParser {
fn parse<'a>(&self, input: &'a str) -> Token<'a> {
Token::Identifier(input)
}
}

Related

Derive macro generation

I'm making my own Serializable trait, in the context of a client / server system.
My idea was that the messages sent by the system is an enum made by the user of this system, so it can be customize as needed.
Too ease implementing the trait on the enum, I would like to use the #[derive(Serializable)] method, as implementing it is always the same thing.
Here is the trait :
pub trait NetworkSerializable {
fn id(&self) -> usize;
fn size(&self) -> usize;
fn serialize(self) -> Vec<u8>;
fn deserialize(id: usize, data: Vec<u8>) -> Self;
}
Now, I've tried to look at the book (this one too) and this example to try to wrap my head around derive macros, but I'm really struggling to understand them and how to implement them. I've read about token streams and abstract trees, and I think I understand the basics.
Let's take the example of the id() method : it should gives a unique id for each variant of the enum, to allow headers of messages to tell which message is incoming.
let's say I have this enum as a message system :
enum NetworkMessages {
ErrorMessage,
SpawnPlayer(usize, bool, Transform), // player id, is_mine, position
MovePlayer(usize, Transform), // player id, new_position
DestroyPlayer(usize) // player_id
}
Then, the id() function should look like this :
fn id(&self) -> usize {
match &self {
&ErrorMessage => 0,
&SpawnPlayer => 1,
&MovePlayer => 2,
&DestroyPlayer => 3,
}
}
Here was my go with writting this using a derive macro :
#[proc_macro_derive(NetworkSerializable)]
pub fn network_serializable_derive(input: TokenStream) -> TokenStream {
// Construct a representation of Rust code as a syntax tree
// that we can manipulate
let ast = syn::parse(input).unwrap();
// Build the trait implementation
impl_network_serializable_macro(&ast)
}
fn impl_network_serializable_macro(ast: &syn::DeriveInput) -> TokenStream {
// get enum name
let ref name = ast.ident;
let ref data = ast.data;
let (id_func, size_func, serialize_func, deserialize_func) = match data {
// Only if data is an enum, we do parsing
Data::Enum(data_enum) => {
// Iterate over enum variants
let mut id_func_internal = TokenStream2::new();
let mut variant_id: usize = 0;
for variant in &data_enum.variants {
// add the branch for the variant
id_func_internal.extend(quote_spanned!{
variant.span() => &variant_id,
});
variant_id += 1;
}
(id_func_internal, (), (), ())
}
_ => {(TokenStream2::new(), (), (), ())},
};
let expanded = quote! {
impl NetworkSerializable for #name {
// variant_checker_functions gets replaced by all the functions
// that were constructed above
fn size(&self) -> usize {
match &self {
#id_func
}
}
/*
#size_func
#serialize_func
#deserialize_func
*/
}
};
expanded.into()
}
So this is generating quite a lot of errors, with the "proc macro NetworkSerializable not expanded: no proc macro dylib present" being first. So I'm guessing there a lot of misunderstaning from my part in here.

How do I return an error within match statement while implementing from_str in rust?

I'm new to rust.
I'm trying to follow the example for implementing the from_str trait here
https://doc.rust-lang.org/std/str/trait.FromStr.html
But I keep getting this error pointing at 'return Err(Self::Err)'
variant or associated item not found in `black_jack_tools::PlayerDifficulty`
I have an idea of why, Self::Err isn't defined in my enum But I don't get why rust cares in this scenario since I'm returning an Err of my Err object which is inline with the Result<Self,Self::Err> type.
Here's my FromStr is below here's a link to the rust playground with an MRE
impl FromStr for PlayerDifficulty {
type Err = ParseError;
fn from_str(s:&str) -> Result<Self,Self::Err>{
let result = match s {
"Player" => Ok(PlayerDifficulty::Player),
"Dealer" => Ok(PlayerDifficulty::Dealer),
"Normal" => Ok(PlayerDifficulty::Normal),
"Perfect"=> Ok(PlayerDifficulty::Perfect),
"Micky" => Ok(PlayerDifficulty::Micky),
"Elliot" => Ok(PlayerDifficulty::Elliot),
"Cultist"=> Ok(PlayerDifficulty::Cultist),
_ => return Err(Self::Err)
};
}
}
What Am I doing wrong?
Is there a better way to do this?
There are three issues with your code. The first is that you need to use <Self as FromStr>::Err if you want to refer to the Err type in your FromStr implementation:
impl FromStr for PlayerDifficulty {
type Err = ParseError;
fn from_str(s:&str) -> Result<Self,Self::Err>{
let result = match s {
"Player" => Ok(PlayerDifficulty::Player),
/* ... */
_ => return Err(<Self as FromStr>::Err)
};
}
}
Self::Err tries to look for an Err variant in the PlayerDifficulty enum but there is no such variant.
The second issue is that std::string::ParseError is in fact an alias for std::convert::Infallible, which is an error that can never happen and cannot be instantiated. Since your conversion may fail, you need to use an error that can be instantiated or define your own:
struct UnknownDifficultyError;
impl FromStr for PlayerDifficulty {
type Err = UnknownDifficultyError;
fn from_str(s:&str) -> Result<Self,Self::Err>{
let result = match s {
"Player" => Ok(PlayerDifficulty::Player),
/* ... */
_ => return Err(UnknownDifficultyError),
};
}
}
Finally, you need to return the result even when conversion succeeds, by removing the let result = and the semicolon:
struct UnknownDifficultyError;
impl FromStr for PlayerDifficulty {
type Err = UnknownDifficultyError;
fn from_str(s:&str) -> Result<Self,Self::Err>{
match s {
"Player" => Ok(PlayerDifficulty::Player),
/* ... */
_ => return Err(UnknownDifficultyError),
}
}
}
Playground
The function will return it last statement. Remove the last semicolon, and you could also remove the internal return statement, the result of the match statement will be returned.
Is there a better way? It looks like you are parsing a string to a enum, the create enum-utils does that. Instead of implementing the parser with boilerplate code you just derive it.
#[derive(Debug, PartialEq, enum_utils::FromStr)]
enum PlayerDifficulty {
Player,
Dealer,
Cultist,
Normal,
}
fn main() {
let _x:PlayerDifficulty= "Player".parse().unwrap();
}
And in your cargo.toml
[dependencies]
enum-utils = "0.1.2"
You should define a custom error
#[derive(Debug)]
struct PlayerError;
impl std::fmt::Display for PlayerError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "Could not parse player")
}
}
impl std::error::Error for PlayerError{}
Then change the match always return the Result in the same path
use std::str::FromStr;
impl FromStr for PlayerDifficulty {
type Err = PlayerError;
fn from_str(s:&str) -> Result<Self,Self::Err>{
match s {
"Player" => Ok(PlayerDifficulty::Player),
"Dealer" => Ok(PlayerDifficulty::Dealer),
"Normal" => Ok(PlayerDifficulty::Normal),
"Perfect"=> Ok(PlayerDifficulty::Perfect),
"Micky" => Ok(PlayerDifficulty::Micky),
"Elliot" => Ok(PlayerDifficulty::Elliot),
"Cultist"=> Ok(PlayerDifficulty::Cultist),
_ => Err(PlayerError)
}
}
}
And use it with ? to propagate error.
fn main() -> (Result<(),Box<dyn std::error::Error>>) {
let _x = PlayerDifficulty::from_str("Player")?;
let _x = PlayerDifficulty::from_str("PlayerPlayer")?;
Ok(())
}

How to return when ref and ownership transfer both won't work

so, if I return this
self.string_ref.unwrap().as_ref()
compiler will say
error[E0515]: cannot return value referencing temporary value
returns a value referencing data owned by the current function
if I return this
*self.string_ref.unwrap().as_ref()
the compiler will say
error[E0507]: cannot move out of borrowed content
this is just drove me crazy
here is the code: (playground)
use std::ptr::NonNull;
struct A {
string_ref: Option<NonNull<String>>,
}
struct Number {
num: i32
}
impl A {
fn hello() {
}
fn give_me_string(&self) -> String {
unsafe {
*self.string_ref.unwrap().as_ref()
}
}
}
fn main() {
let a = A {
string_ref: NonNull::new(&mut String::from("hello world") as *mut String)
};
let t = a.give_me_string();
println!("{}", t)
}
Stripping your example to the bare minimum:
struct A {
string_ref: Option<NonNull<String>>,
}
impl A {
fn give_me_string(&self) -> String {
unsafe {
*self.string_ref.unwrap().as_ref()
}
}
}
There are a few errors here:
The most obvious one is that you're trying to take ownership of self.string_ref, even though you've only borrowed self.
To solve this you'll want to use a match statement, which allows you to destructure self.string_ref and not consume it:
fn give_me_string(&self) -> String {
unsafe {
match self.string_ref {
Some(x) => x.as_ref(),
None => panic!("Had no `string_ref`!")
}
}
}
as_ref returns &T, so you can't return an owned string, instead you need to either clone it and then return an owned string, or take reference to it:
//Option one: Clone contents
match self.string_ref {
Some(ref x) => x.as_ref().clone(),
_ => //...
}
//Option two: Return reference.
fn give_me_string(&self) -> &str {
unsafe {
match &self.string_ref {
Some(x) => x.as_ref() as _,
_ => //...
}
}
}
To address another problem mentioned in the comments, you have the following statement in your main function:
string_ref: NonNull::new(&mut String::from("hello world") as *mut String)
This will cause UB due to its nature. You are forming a String by using String::from, but are not storing its value anywhere and are instead immediately casting into a pointer. This will free the String at the end of the line, causing UB.
So I basically figured out what's going on, thanks to #Optimistic Peach
fn give_me_string(&self) -> &String {
unsafe {
match self.string_ref {
Some(x) => &*(x.as_ptr() as *const _), //without ref
Some(ref x) => x.as_ptr(), // with ref
None => panic!("hello?")
}
}
}

Lifetime issues with non-copyable owned values when trying to use them after passing them to a function

I'm running into some problems with lifetime of variables in Rust. The x variable in do_stuff is borrowed to try_wrap and can thus not be returned in the None case. Am I thinking about this the wrong way?
struct NonCopyable;
impl NonCopyable {
fn new() -> Self {
NonCopyable
}
}
fn do_stuff() -> NonCopyable {
let x = NonCopyable::new();
match try_wrap(x) {
Some(val) => val,
None => x,
}
}
fn try_wrap(x: NonCopyable) -> Option<NonCopyable> {
None
}
fn main() {}
error[E0382]: use of moved value: `x`
--> src/main.rs:13:17
|
11 | match try_wrap(x) {
| - value moved here
12 | Some(val) => val,
13 | None => x,
| ^ value used here after move
|
= note: move occurs because `x` has type `NonCopyable`, which does not implement the `Copy` trait
with lifetime of variables
There are no lifetimes involved here
The x variable in do_stuff is borrowed
No, it is not.
Am I thinking about this the wrong way?
Yes. Borrowing is indicated by an ampersand & and/or a lifetime parameter 'foo:
&i32 // a borrowed integer
&'a str // a borrowed string slice with a lifetime
Foo<'b> // a type that contains a borrow of some kind
Your try_wrap function takes ownership of x:
fn try_wrap(x: NonCopyable) -> Option<NonCopyable>
That means x is gone and the calling function cannot access it anymore. It has been moved into try_wrap, which is now free to do whatever it wants with the value, including destroy it. That is why the calling function is no longer able to safely access it and why you get the error.
If the type implemented Copy, the compiler would have instead implicitly created a copy of the value and passed it in. If the type implemented Clone, you could have explicitly called .clone() on the argument to try_wrap in order to keep the local value.
As Florian Weimer notes, you can use a type to return either the wrapped value or the original. It's difficult to tell based on your example, but I disagree with using Result unless it's an error. Instead, I'd create my own one-off enum or use something like Either:
extern crate either;
use either::Either;
fn do_stuff() -> NonCopyable {
let x = NonCopyable::new();
match try_wrap(x) {
Either::Left(val) => val,
Either::Right(x) => x,
}
}
fn try_wrap(x: NonCopyable) -> Either<NonCopyable, NonCopyable> {
Either::Right(x)
}
You could also embed the logic of try_wrap back into do_stuff, or split up try_wrap so that the logic doesn't require ownership:
fn do_stuff() -> NonCopyable {
let x = NonCopyable::new();
if should_wrap(&x) { do_wrap(x) } else { x }
}
fn should_wrap(x: &NonCopyable) -> bool { false }
fn do_wrap(x: NonCopyable) -> NonCopyable { x }
Since you are returning the same type, it's also possible you would want to take a mutable reference to the value and just do whatever conditional changes need to happen:
fn do_stuff() -> NonCopyable {
let mut x = NonCopyable::new();
try_wrap(&mut x);
x
}
fn try_wrap(x: &mut NonCopyable) {}
I think Option<NonCopyable> is simply the wrong return type for try_wrap. You need a two-armed sum type here like Result, so that the caller can recover the argument in case of an error, perhaps like this:
fn do_stuff() -> NonCopyable {
let x = NonCopyable::new();
match try_wrap(x) {
Ok(val) => val,
Err(x) => x,
}
}
fn try_wrap(x: NonCopyable) -> Result<NonCopyable, NonCopyable> {
Err(x)
}

How do I collect from multiple iterator types?

I am attempting to implement a new trait for a String that has a function that capitalizes the first letter of each String and un-capitalizes the rest. I am basing the function's interface on to_uppercase() and to_lowercase() in the Rust Standard Library.
use std::io;
trait ToCapitalized {
fn to_capitalized(&self) -> String;
}
impl ToCapitalized for String {
fn to_capitalized(&self) -> String {
self.chars().enumerate().map(|(i, c)| {
match i {
0 => c.to_uppercase(),
_ => c.to_lowercase(),
}
}).collect()
}
}
fn main() {
let mut buffer = String::new();
io::stdin().read_line(&mut buffer).ok().expect("Unable to read from stdin.");
println!("{}", buffer.to_capitalized());
}
This code is based on a suggestion given here, but the code is outdated and causes multiple compilation errors. The only issue I am having with my implementation now is the following error:
src/main.rs:10:13: 13:14 error: match arms have incompatible types [E0308]
src/main.rs:10 match i {
^
src/main.rs:10:13: 13:14 help: run `rustc --explain E0308` to see a detailed explanation
src/main.rs:10:13: 13:14 note: expected type `std::char::ToUppercase`
src/main.rs:10:13: 13:14 note: found type `std::char::ToLowercase`
src/main.rs:12:22: 12:38 note: match arm with an incompatible type
src/main.rs:12 _ => c.to_lowercase(),
So in short, the return values of fn to_uppercase(&self) -> ToUppercase and fn to_lowercase(&self) -> ToLowercase can't be collected together because the map now has multiple return types.
I've attempted trying to cast them to another common Iterator type such as Bytes and Chars, but these iterator types can't be collected to form a String. Any suggestions?
Casting is rarely a good approach to solving type issues in Rust. The correct solution here would be to write (or find a crate that defines) a type that unifies disparate iterator types. But that would require effort, so it's simpler to just throw collect out the window:
trait ToCapitalized {
fn to_capitalized(&self) -> String;
}
impl ToCapitalized for String {
fn to_capitalized(&self) -> String {
let mut r = String::with_capacity(self.len());
for (i, c) in self.chars().enumerate() {
match i {
0 => r.extend(c.to_uppercase()),
_ => r.extend(c.to_lowercase()),
}
}
r
}
}
fn main() {
let buffer = String::from("canberra");
println!("{}", buffer.to_capitalized());
}
This is, more or less, what collect would do anyway if you had some type to represent "either ToUppercase or ToLowercase". In the vast majority of cases, this will also only perform a single allocation.
Here's how I would do it:
trait ToCapitalized {
fn to_capitalized(&self) -> String;
}
impl ToCapitalized for String {
fn to_capitalized(&self) -> String {
match self.chars().next() {
Some(c) => {
c.to_uppercase()
.chain(self.chars().skip(1).flat_map(|c| c.to_lowercase()))
.collect()
}
None => String::new(),
}
}
}
fn main() {
println!("{}", "fOoBaR".to_string().to_capitalized());
}
This will be a little slower than the ideal solution, as it decodes the first char twice, but it's quite readable IMO.
Output:
Foobar
After looking at the implementation for pub fn to_uppercase(&self) -> String here, I devised a solution that is a bit of a hybrid between Dogbert and DK.'s solutions and the implementation given in the standard library. It even works with Unicode!
fn to_capitalized(&self) -> String {
match self.len() {
0 => String::new(),
_ => {
let mut s = String::with_capacity(self.len());
s.extend(self.chars().next().unwrap().to_uppercase());
s.extend(self.chars().skip(1).flat_map(|c| c.to_lowercase()));
return s;
}
}
}
Working Rust Playground Example
Edit: For greater visibility, Shepmaster's simplified and optimized solution:
fn to_capitalized(&self) -> String {
let mut s = String::with_capacity(self.len());
let mut chars = self.chars();
s.extend(chars.by_ref().take(1).flat_map(|c| c.to_uppercase()));
s.extend(chars.flat_map(|c| c.to_lowercase()));
s
}

Resources