This question already has answers here:
How to fix lifetime error when function returns a serde Deserialize type?
(2 answers)
Return local String as a slice (&str)
(7 answers)
Closed 2 years ago.
I am trying to write a wrapper function for read a string that contains Vec[u8] (that are really just MsgPacks) and convert them to native rust structs, my code looks like this
use rmp_serde::{decode, from_slice};
use serde::Deserialize;
#[derive(Debug)]
pub enum MsgPackParseErr {
ParseIntError,
SerdeError,
}
impl From<std::num::ParseIntError> for MsgPackParseErr {
fn from(_e: std::num::ParseIntError) -> Self {
return Self::ParseIntError;
}
}
impl From<decode::Error> for MsgPackParseErr {
fn from(_e: decode::Error) -> Self {
return Self::SerdeError;
}
}
pub fn msgpack_from_byte_string<'b, T>(raw: String) -> Result<T, MsgPackParseErr>
where
T: Deserialize<'b>,
{
let parsing_string: Result<Vec<u8>, _> = raw.split(" ").map(|x| x.parse()).collect();
let parsed_string = parsing_string?;
let parsing_obj = from_slice(&parsed_string);
Ok(parsing_obj?)
}
But I am getting the error
temporary value dropped while borrowed
creates a temporary which is freed while still in use
For lines 23 to 28 i.e.
let parsing_obj = from_slice(&parsed_string);
Ok(parsing_obj?)
I have no idea what I am doing wrong here...
Your error comes from the fact that T: Deserialize<'b> in your code constrains T to live only as long as the lifetime 'b which in turn means it can't outlive whatever the input to from_slice was (otherwise it would be a use after free error).
pub fn msgpack_from_byte_string<'b, T>(raw: String) -> Result<T, MsgPackParseErr>
where
T: Deserialize<'b>
So why can't your serialized object be alive longer than the data associated with it? If possible, serde avoids the copy of the input data and allocation of extra fields by directly referencing the input buffer. This is also explained in more detail in the serde manual chapter about lifetimes.
Note that serde has also other traits that may be more appropriate for your use-case and are not constrained to the lifetime of the input (e.g., DeserializeOwned).
Related
This question already has answers here:
Rust and serde deserializing using generics
(2 answers)
Closed 4 days ago.
I would like to implement a trait function that takes a path and gives an owned value.
use bevy::prelude::*;
use serde::de;
pub trait RonResource<'a, T: de::Deserialize<'a> + Resource> {
fn import_from_ron(path: &str) -> T {
let ron_string = std::fs::read_to_string(path).unwrap();
ron::from_str::<T>(
&ron_string
).expect(("Failed to load: ".to_owned() + path).as_str())
}
}
I implement the 'a lifetime because de::Deserialize needs it.
But the compiled tells me that "ron_string" will be dropped while still borrowed. (at the "&ron_string" line)
This code works fine if I implement it without generics, like this for exemple:
let settings = ron::from_str::<GasParticleSystemSettings>(
&std::fs::read_to_string("assets/settings/gas_ps_settings.ron").unwrap()
).expect("Failed to load settings/gas_ps_settings.ron")
.side_gas;
I don't get why the value need to "survive" the whole function as it will not be needed. Otherwise, the specific code wouldn't work!
The problem is when using Deserialize<'a> the deserialized object might contain references to the original str ie something like this:
struct HasRef<'a> {
s: &'a str,
}
is allowed. It would contain references to ron_string which is dropped at the end of the function.
What's totally save though is to just require DeserializeOwned instead at which point you can't do 0-copy deserialization any more but you don't have to keep the original string around either:
pub trait RonResource<T: de::DeserializeOwned + Resource> {
fn import_from_ron(path: &str) -> T {
let ron_string = std::fs::read_to_string(path).unwrap();
ron::from_str::<T>(
&ron_string
).expect(("Failed to load: ".to_owned() + path).as_str())
}
}
Your second example works presumably because you only access GasParticleSystemSettings.side_gas which I assume either does not contain references or is straight up Copy neither of which would create a lifetime issue.
This question already has answers here:
Why can't I store a value and a reference to that value in the same struct?
(4 answers)
How to return a reference to a sub-value of a value that is under a mutex?
(5 answers)
Returning a RWLockReadGuard independently from a method
(2 answers)
How can I store a Chars iterator in the same struct as the String it is iterating on?
(2 answers)
Closed 3 years ago.
I would like to wrap a low-level third-party API with my own struct and functions to make it more friendly. Unfortunately, the third-party API needs a reference to a socket in its constructor, which means I'd like my Struct to "own" the socket (someone has to own it so that it can be borrowed, right? and I'd like that hidden as an implementation detail in my API).
The third-party API looks something like this:
struct LowLevelApi<'a> {
stream: &'a mut TcpStream,
// ...
}
impl<'a> LowLevelApi<'a> {
pub fn new(socket: &'a mut TcpStream, ... ) -> LowLevelApi<'a> {
// ...
}
}
I would like to make the interface to my function look like:
pub fn new(host: String, port: u16, ...) -> HighLevelApi {
// ...
}
I tried this:
pub struct HighLevelApi<'a> {
stream: TcpStream,
low: LowLevelApi<'a>
}
impl <'a> HighLevelApi<'a> {
pub fn new(host: String, port: u16) -> HighLevelApi<'a> {
// Ignore lack of error checking for now
let mut stream = TcpStream::connect(format!("{}:{}", host, port)).unwrap();
HighLevelApi {
stream,
low: LowLevelApi::new(&mut stream)
}
}
}
Rust is (rightly) angry: It has no way of knowing that I'm not going to do something bad with the low field later. And even worse, I would need to somehow guarantee that when my structure gets dropped, low gets dropped first, and stream second (since by that point, any relationship between the two is lost - or rather, there never is/was a relationship between the two).
(actually, it's worse than that, because the stream local variable gets moved into the new struct, so the local can't possibly be borrowed by LowLevelApi, but I can't think of a way to initialize HighLevelApi with the stream from the struct, since there's no way to get a handle to that from within the struct's initialization, is there? But based on my guess about what would happen in the paragraph above, it doesn't really matter since it still wouldn't do what I wanted)
What are examples of the various techniques that can be used to store a wrap a third-party (not under my control) struct that needs a reference to something?
The Rental crate seems to do what is needed here, albeit with documentation and examples that leave a lot to the imagination (i.e. trial and error).
Here's roughly what solves this
rental! {
pub mod rentals {
#[rental_mut]
pub struct Wrapper {
stream: Box<TcpStream>,
low: LowLevelApi<'stream>
}
}
}
pub struct HighLevelApi {
wrapper: rentals::Wrapper
}
impl HighLevelApi {
pub fn new(host: String, port: u16) -> HighLevelApi {
Api {
// Ignore lack of error checking for now
wrapper: rentals::Wrapper::new(
Box::new(TcpStream::connect(format!("{}:{}", host, port)).unwrap()),
|s| LowLevelApi::new(s)
)
}
}
pub fn do_something(&mut self) {
self.wrapper.rent_mut(|ll| ll.do_something()) // ll is the LowLevelApi
}
}
I noticed two important things that made this work:
The lifetime name on low in the Wrapper struct must match the name of the "owning" field (in this case "'stream")
You never get direct access to the reference - you get it through a callback/closure:
In the auto-generated constructor (new()) the second parameter isn't a LowLevelApi, it's a closure that gets the &mut TcpStream, and that closure is then expected to return a LowLevelApi
When you want to actually use the LowLevelApi you can "rent" it, hence the wrapper.rent_mut(f) where f is the closure that gets passed a LowLevelApi (ll) and can do what it needs
With these facts, the rest of the Rental documentation makes a lot more sense.
I've worked down a real-life example in a web app, which I've solved using unnecessary heap allocation, to the following example:
// Try replacing with (_: &String)
fn make_debug<T>(_: T) -> impl std::fmt::Debug {
42u8
}
fn test() -> impl std::fmt::Debug {
let value = "value".to_string();
// try removing the ampersand to get this to compile
make_debug(&value)
}
pub fn main() {
println!("{:?}", test());
}
As is, compiling this code gives me:
error[E0597]: `value` does not live long enough
--> src/main.rs:9:16
|
5 | fn test() -> impl std::fmt::Debug {
| -------------------- opaque type requires that `value` is borrowed for `'static`
...
9 | make_debug(&value)
| ^^^^^^ borrowed value does not live long enough
10 | }
| - `value` dropped here while still borrowed
I can fix this error in at least two ways:
Instead of passing in a reference to value in test(), pass in value itself
Instead of the parameter T, explicitly state the type of the argument for make_debug as &String or &str
My understanding of what's happening is that, when there is a parameter, the borrow checker is assuming that any lifetime on that parameter affects the output impl Debug value.
Is there a way to keep the code parameterized, continue passing in a reference, and get the borrow checker to accept it?
I think this is due to the rules around how impl trait opaque types capture lifetimes.
If there are lifetimes inside an argument T, then an impl trait has to incorporate them. Additional lifetimes in the type signature follow the normal rules.
For more information please see:
https://github.com/rust-lang/rust/issues/43396#issuecomment-349716967
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#lifetime-parameters
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#assumption-3-there-should-be-an-explicit-marker-when-a-lifetime-could-be-embedded-in-a-return-type
https://github.com/rust-lang/rfcs/blob/master/text/1951-expand-impl-trait.md#scoping-for-type-and-lifetime-parameters
A more complete answer
Original goal: the send_form function takes an input parameter of type &T which is rendered to a binary representation. That binary representation is owned by the resulting impl Future, and no remnant of the original &T remains. Therefore, the lifetime of &T need not outlive the impl Trait. All good.
The problem arises when T itself, additionally, contains references with lifetimes. If we were not using impl Trait, our signature would look something like this:
fn send_form<T>(self, data: &T) -> SendFormFuture;
And by looking at SendFormFuture, we can readily observe that there is no remnant of T in there at all. Therefore, even if T has lifetimes of its own to deal with, we know that all references are used within the body of send_form, and never used again afterward by SendFormFuture.
However, with impl Future as the output, we get no such guarantees. There's no way to know if the concrete implementation of Future in fact holds onto the T.
In the case where T has no references, this still isn't a problem. Either the impl Future references the T, and fully takes ownership of it, or it doesn't reference it, and no lifetime issues arise.
However, if T does have references, you could end up in a situation where the concrete impl Future is holding onto a reference stored in the T. Even though the impl Future has ownership of the T itself, it doesn't have ownership of the values referenced by the T.
This is why the borrow check must be conservative, and insist that any references inside T must have a 'static lifetime.
The only workaround I can see is to bypass impl Future and be explicit in the return type. Then, you can demonstrate to the borrow checker quite easily that the output type does not reference the input T type at all, and any references in it are irrelevant.
The original code in the actix web client for send_form looks like:
https://docs.rs/awc/0.2.1/src/awc/request.rs.html#503-522
pub fn send_form<T: Serialize>(
self,
value: &T,
) -> impl Future<
Item = ClientResponse<impl Stream<Item = Bytes, Error = PayloadError>>,
Error = SendRequestError,
> {
let body = match serde_urlencoded::to_string(value) {
Ok(body) => body,
Err(e) => return Either::A(err(Error::from(e).into())),
};
// set content-type
let slf = self.set_header_if_none(
header::CONTENT_TYPE,
"application/x-www-form-urlencoded",
);
Either::B(slf.send_body(Body::Bytes(Bytes::from(body))))
}
You may need to patch the library or write your own function that does the same thing but with a concrete type. If anyone else knows how to deal with this apparent limitation of impl trait I'd love to hear it.
Here's how far I've gotten on a rewrite of send_form in awc (the actix-web client library):
pub fn send_form_alt<T: Serialize>(
self,
value: &T,
// ) -> impl Future<
// Item = ClientResponse<impl Stream<Item = Bytes, Error = PayloadError>>,
// Error = SendRequestError,
) -> Either<
FutureResult<String, actix_http::error::Error>,
impl Future<
Item = crate::response::ClientResponse<impl futures::stream::Stream>,
Error = SendRequestError,
>,
> {
Some caveats so far:
Either::B is necessarily an opaque impl trait of Future.
The first param of FutureResult might actually be Void or whatever the Void equivalent in Rust is called.
This question already has answers here:
How do I implement a trait I don't own for a type I don't own?
(3 answers)
Closed 7 years ago.
I want to provide an implementation of a trait ToHex (not defined by me, from serialize) for a primitive type u8:
impl ToHex for u8 {
fn to_hex(&self) -> String {
self.to_str_radix(16)
}
}
The problem is I get this compiler error:
error: cannot provide an extension implementation where both trait and type are not defined in this crate
I understand the reason of this error and its logic, this is because both the trait and the primitive type are external to my code. But how can I handle this situation and provide an ToHex implementation for u8? And more generally how do you handle this kind of issue, it seems to me that this problem must be common and it should be possible and easy to extend types like this?
You should use a newtype struct to do this:
pub struct U8(pub u8)
impl ToHex for U8 {
fn to_hex(&self) -> String {
let U8(x) = *self;
x.to_str_radix(16)
}
}
This does mean, however, that you should wrap u8 into U8 where you need to perform this conversion:
let x: u8 = 127u8
// println!("{}", x.to_hex()); // does not compile
println!("{}", U8(x).to_hex());
This is absolutely free in terms of performance.
I realize this is almost a year old, but the answer was never accepted and I think I've found an alternate solution, that I thought would be good to document here.
In order to extend the functionality of the u8 through traits, instead of trying to extend ToHex, why not create a new trait?
trait MyToHex {
fn to_hex(&self) -> String;
}
impl MyToHex for u8 {
fn to_hex(&self) -> String {
format!("{:x}", *self)
}
}
then used like so
fn main() {
println!("{}", (16).to_hex());
}
This has the advantage that you don't have to wrap every u8 variable with a new and superfluous data type.
The disadvantage is that you still can't use a u8 in a external function (i.e std library, or one you have no control over) that requires the ToHex trait (Vladimir Matveev's solution works in this case), but from OP it sounds like all you want to do is extend u8 only inside your code.
This question already has answers here:
How I can mutate a struct's field from a method?
(2 answers)
Cannot modify a struct field from implementation: "cannot borrow immutable borrowed content as mutable"
(1 answer)
Closed 7 years ago.
I wanted to create a structwith a field containing a Vec<ThreadRng>.
So this works fine, I have a list of generators stored in Foo.generators:
extern crate rand;
use std::vec::Vec;
use rand::{Rng, ThreadRng};
struct Foo {
generators: Vec<ThreadRng>,
}
impl Foo {
fn new() -> Foo {
Foo { generators: vec![rand::thread_rng(), rand::thread_rng()]}
}
}
Now I would like to use it, say in a bar method:
impl Foo {
fn bar(&self) -> i32 {
self.generators[0].gen::<i32>()
}
}
But that I can't, and I get a cannot borrow immutable field 'self.generators' as mutable.
As I understand I cannot use the gen method of ThreadRng because it requires a mutable reference to the RNG (gen<T: Rand>(&mut self)) and since the definition of my field Foo.generators "Vec<ThreadRng>" doesn't specify that the ThreadRng should be mutable, I can't do anything with it that requires mutability.
First question: is my understanding of what's happening correct, or am I completely wrong? If so, could someone explain to me what's really happening?
Second question: admitting that my understanding is correct, how am I supposed to allow actions that requires mutability on my ThreadRngs?
Thanks in advance!
You need to borrow self mutably:
impl Foo {
fn bar(&mut self) -> i32 {
self.generators[0].gen::<i32>()
}
}