Noticed the following difference between match and if in Rust regarding lifetimes - let's say we have simple struct:
struct Foo;
impl Foo {
is_valid(&self) -> bool { true }
}
and a mutex that holds it:
let foo = tokio::sync::Mutex::new(Foo);
With the following match the lock exists until the end of the match construct:
match foo.lock().await.is_valid() {
_ => foo.lock().await.is_valid(), // Deadlock.
};
This will not finish as the lock still holds when the second line is trying to acquire it. However using an if will release the first lock right after evaluating the bool:
if foo.lock().await.is_valid() {
foo.lock().await.is_valid(); // This is fine.
}
This code finishes with the lock engaged twice. Does this mean match hangs on the references more than if? What's the explanation?
I believe this issue will give you the information you're looking for.
In short the Drop trait is triggered after everything in the match statement is resolved. That means that by the time the inner lock is called, the outer one hasn't been freed yet. For the if statement, the Drop trait is called immediately after the condition is checked and before the condition block is called.
Interestingly, this seems to also occur with if let expressions too. The following code also deadlocks:
struct Foo;
impl Foo {
fn is_valid(&self) -> bool { true }
}
#[tokio::main]
async fn main() {
let foo = tokio::sync::Mutex::new(Foo);
if let true = foo.lock().await.is_valid() {
foo.lock().await.is_valid();
};
}
Related
I'm tring to replace a value in a mutable borrow; moving part of it into the new value:
enum Foo<T> {
Bar(T),
Baz(T),
}
impl<T> Foo<T> {
fn switch(&mut self) {
*self = match self {
&mut Foo::Bar(val) => Foo::Baz(val),
&mut Foo::Baz(val) => Foo::Bar(val),
}
}
}
The code above doesn't work, and understandibly so, moving the value out of self breaks the integrity of it. But since that value is dropped immediately afterwards, I (if not the compiler) could guarantee it's safety.
Is there some way to achieve this? I feel like this is a job for unsafe code, but I'm not sure how that would work.
mem:uninitialized has been deprecated since Rust 1.39, replaced by MaybeUninit.
However, uninitialized data is not required here. Instead, you can use ptr::read to get the data referred to by self.
At this point, tmp has ownership of the data in the enum, but if we were to drop self, that data would attempt to be read by the destructor, causing memory unsafety.
We then perform our transformation and put the value back, restoring the safety of the type.
use std::ptr;
enum Foo<T> {
Bar(T),
Baz(T),
}
impl<T> Foo<T> {
fn switch(&mut self) {
// I copied this code from Stack Overflow without reading
// the surrounding text that explains why this is safe.
unsafe {
let tmp = ptr::read(self);
// Must not panic before we get to `ptr::write`
let new = match tmp {
Foo::Bar(val) => Foo::Baz(val),
Foo::Baz(val) => Foo::Bar(val),
};
ptr::write(self, new);
}
}
}
More advanced versions of this code would prevent a panic from bubbling out of this code and instead cause the program to abort.
See also:
replace_with, a crate that wraps this logic up.
take_mut, a crate that wraps this logic up.
Change enum variant while moving the field to the new variant
How can I swap in a new value for a field in a mutable reference to a structure?
The code above doesn't work, and understandibly so, moving the value
out of self breaks the integrity of it.
This is not exactly what happens here. For example, same thing with self would work nicely:
impl<T> Foo<T> {
fn switch(self) {
self = match self {
Foo::Bar(val) => Foo::Baz(val),
Foo::Baz(val) => Foo::Bar(val),
}
}
}
Rust is absolutely fine with partial and total moves. The problem here is that you do not own the value you're trying to move - you only have a mutable borrowed reference. You cannot move out of any reference, including mutable ones.
This is in fact one of the frequently requested features - a special kind of reference which would allow moving out of it. It would allow several kinds of useful patterns. You can find more here and here.
In the meantime for some cases you can use std::mem::replace and std::mem::swap. These functions allow you to "take" a value out of mutable reference, provided you give something in exchange.
Okay, I figured out how to do it with a bit of unsafeness and std::mem.
I replace self with an uninitialized temporary value. Since I now "own" what used to be self, I can safely move the value out of it and replace it:
use std::mem;
enum Foo<T> {
Bar(T),
Baz(T),
}
impl<T> Foo<T> {
fn switch(&mut self) {
// This is safe since we will overwrite it without ever reading it.
let tmp = mem::replace(self, unsafe { mem::uninitialized() });
// We absolutely must **never** panic while the uninitialized value is around!
let new = match tmp {
Foo::Bar(val) => Foo::Baz(val),
Foo::Baz(val) => Foo::Bar(val),
};
let uninitialized = mem::replace(self, new);
mem::forget(uninitialized);
}
}
fn main() {}
This question already has answers here:
How can I conditionally provide a default reference without performing unnecessary computation when it isn't used?
(2 answers)
Closed 2 years ago.
I have a function which takes a reference and a condition. If the condition is false, it uses the passed in reference, but when the condition is true, it use a temp reference pointing to the local variable in the function.
The Foo struct is NOT Copy or Clone.
The commented-off code won't compile and I have an ugly way to work around this problem:
#[derive(Debug)]
struct Foo {
a: String,
}
fn f(r: &Foo, cond: bool) {
// let a = if cond { &Foo { a: "456".into() } } else { r };
// use a temp variable `t` can solve this problem in this demo code,
// but what if in production code where build tmp var `t` is expensive?
let mut a = r;
let t = &Foo { a: "456".into() };
if cond {
a = &t;
}
// use the ref here, inside the function.
println!("{:#?}", a);
}
fn main() {
let a = Foo { a: "123".into() };
f(&a, true);
}
Because the use of the reference is limited in the function, is there a way to tell the compiler: do not free the temp var created in the if branch, keep it until the function return and destroy it when destroy the stack frame?
I think there should be a way without using heap memory. On the other hand, when using Box, the type produced in the two if branch will be different, one is &Foo and another is Box<Foo>, so this won't be the right way.
Cow to the rescue! Cow is an enum that represents either (1) a thing that we own or (2) a thing we have borrowed.
#[derive(Debug, Clone)]
struct Foo {
a: String,
}
fn f(r: &Foo, cond: bool) {
use std::borrow::Cow;
let a: Cow<Foo> = if cond {
// (1) a thing that we own, or
Cow::Owned(Foo { a: "456".into() })
} else {
// (2) a thing we have borrowed
Cow::Borrowed(r)
};
// use the ref here, inside the function.
println!("{:#?}", a);
// if a is a Cow::Owned, the owned thing is dropped properly
// if a is a Cow::Borrowed, the reference is dropped.
}
fn main() {
let a = Foo { a: "123".into() };
f(&a, true);
}
Cow requires Clone, however, which is a little bit surplus to what we already have, but it's not a terribly onerous burden.
Consider the following Rust code, slightly modified from examples in The Book.
I'm trying to understand what happens to the value in the second running of function dangle() in the main() function (see comment). I would imagine that because the value isn't assigned to any owner, it gets deallocated, but I've so far failed to find information to confirm that. Otherwise, I would think that calling dangle() repeatedly would constantly allocate more memory without deallocating it. Which is it?
fn main() {
// Ownership of dangle()'s return value is passed to the variable `thingamabob`.
let thingamabob = dangle();
// No ownership specified. Is the return value deallocated here?
dangle();
println!("Ref: {}", thingamabob);
}
fn dangle() -> String {
// Ownership specified.
let s = String::from("hello");
// Ownership is passed to calling function.
s
}
When a value has no owner (is not bound to a variable) it goes out of scope. Values that go out of scope are dropped. Dropping a value frees the resources associated with that value.
Anything less would lead to memory leaks, which would be a poor idea in a programming language.
See also:
Is it possible in Rust to delete an object before the end of scope?
How does Rust know whether to run the destructor during stack unwind?
Does Rust free up the memory of overwritten variables?
In your example, the second call creates an unnamed temporary value whose lifetime ends immediately after that one line of code, so it goes out of scope (and any resources are reclaimed) immediately.
If you bind the value to a name using let, then its lifetime extends until the end of the current lexical scope (closing curly brace).
You can explore some of this yourself by implementing the Drop trait on a simple type to see when its lifetime ends. Here's a small program I made to play with this (playground):
#[derive(Debug)]
struct Thing {
val: i32,
}
impl Thing {
fn new(val: i32) -> Self {
println!("Creating Thing #{}", val);
Thing { val }
}
fn foo(self, val: i32) -> Self {
Thing::new(val)
}
}
impl Drop for Thing {
fn drop(&mut self) {
println!("Dropping {:?}", self);
}
}
pub fn main() {
let _t1 = Thing::new(1);
Thing::new(2); // dropped immediately
{
let t3 = Thing::new(3);
Thing::new(4).foo(5).foo(6); // all are dropped, in order, as the next one is created
println!("Doing something with t3: {:?}", t3);
} // t3 is dropped here
} // _t1 is dropped last
I have some non-copyable type and a function that consumes and (maybe) produces it:
type Foo = Vec<u8>;
fn quux(_: Foo) -> Option<Foo> {
Some(Vec::new())
}
Now consider a type that is somehow conceptually very similar to Box:
struct NotBox<T> {
contents: T
}
We can write a function that temporarily moves out contents of the NotBox and puts something back in before returning it:
fn bar(mut notbox: NotBox<Foo>) -> Option<NotBox<Foo>> {
let foo = notbox.contents; // now `notbox` is "empty"
match quux(foo) {
Some(new_foo) => {
notbox.contents = new_foo; // we put something back in
Some(notbox)
}
None => None
}
}
I want to write an analogous function that works with Boxes but the compiler does not like it:
fn baz(mut abox: Box<Foo>) -> Option<Box<Foo>> {
let foo = *abox; // now `abox` is "empty"
match quux(foo) {
Some(new_foo) => {
*abox = new_foo; // error: use of moved value: `abox`
Some(abox)
}
None => None
}
}
I could return Some(Box::new(new_foo)) instead but that performs unnecessary allocation - I already have some memory at my disposal! Is it possible to avoid that?
I would also like to get rid of the match statements but again the compiler is not happy with it (even for the NotBox version):
fn bar(mut notbox: NotBox<Foo>) -> Option<NotBox<Foo>> {
let foo = notbox.contents;
quux(foo).map(|new_foo| {
notbox.contents = new_foo; // error: capture of partially moved value: `notbox`
notbox
})
}
Is it possible to work around that?
So, moving out of a Box is a special case... now what?
The std::mem module presents a number of safe functions to move values around, without poking holes (!) into the memory safety of Rust. Of interest here are swap and replace:
pub fn replace<T>(dest: &mut T, src: T) -> T
Which we can use like so:
fn baz(mut abox: Box<Foo>) -> Option<Box<Foo>> {
let foo = std::mem::replace(&mut *abox, Foo::default());
match quux(foo) {
Some(new_foo) => {
*abox = new_foo;
Some(abox)
}
None => None
}
}
It also helps in the map case, because it does not borrow the Box:
fn baz(mut abox: Box<Foo>) -> Option<Box<Foo>> {
let foo = std::mem::replace(&mut *abox, Foo::default());
quux(foo).map(|new_foo| { *abox = new_foo; abox })
}
Moving out of boxes is special-cased in the compiler. You can move something out of them, but you can't move something back in, because the act of moving out also deallocates. You can do something silly with std::ptr::write, std::ptr::read, and std::ptr::replace, but it's hard to get it right, because something valid should be inside a Box when it is dropped. I would recommend just accepting the allocation, or switching to a Box<Option<Foo>> instead.
We can write a function that temporarily moves out contents of the NotBox and puts something back in before returning it
That's because you can partially move out from the struct that you take by value. It behaves as if all fields were separate variables. That is not possible though if the struct implements Drop, because drop needs the whole struct to be valid, always (in case of panic).
As for providing workaround, you haven't provided enough information – especially, why baz needs to take Box as an argument and why quux can't? Which functions are yours and which are part of an API you can't change? What is the real type of Foo? Is it big?
The best workaround would be not to use Box at all.
I have a struct:
struct MyData {
x: i32
}
I want to asynchronously start a long operation on this struct.
My first attempt was this:
fn foo(&self) { //should return immediately
std::thread::Thread::spawn(move || {
println!("{:?}",self.x); //consider a very long operation
});
}
Clearly the compiler cannot infer an appropriate lifetime due to conflicting requirements because self may be on the stack frame and thus cannot be guaranteed to exist by the time the operation is running on a different stack frame.
To solve this, I attempted to make a copy of self and provide that copy to the new thread:
fn foo(&self) { //should return immediately
let clone = self.clone();
std::thread::Thread::spawn(move || {
println!("{:?}",clone.x); //consider a very long operation
});
}
I think that does not compile because now clone is on the stack frame which is similar to before. I also tried to do the clone inside the thread, and that does not compile either, I think for similar reasons.
Then I decided maybe I could use a channel to push the copied data into the thread, on the theory that perhaps channel can magically move (copy?) stack-allocated data between threads, which is suggested by this example in the documentation. However the compiler cannot infer a lifetime for this either:
fn foo(&self) { //should return immediately
let (tx, rx) = std::sync::mpsc::channel();
tx.send(self.clone());
std::thread::Thread::spawn(move || {
println!("{:?}",rx.recv().unwrap().x); //consider a very long operation
});
}
Finally, I decided to just copy my struct onto the heap explicitly, and pass an Arc into the thread. But not even here can the compiler figure out a lifetime:
fn foo(&self) { //should return immediately
let arc = std::sync::Arc::new(self.clone());
std::thread::Thread::spawn(move || {
println!("{:?}",arc.clone().x); //consider a very long operation
});
}
Okay borrow checker, I give up. How do I get a copy of self onto my new thread?
I think your issue is simply because your structure does not derive the Clone trait. You can get your second example to compile and run by adding a #[derive(Clone)] before your struct's definition.
What I don't understand in the compiler behaviour here is what .clone() function it tried to use here. Your structure indeed did not implement the Clone trait so should not by default have a .clone() function.
playpen
You may also want to consider in your function taking self by value, and let your caller decide whether it should make a clone, or just a move.
As an alternative solution, you could use thread::scoped and maintain a handle to the thread. This allows the thread to hold a reference, without the need to copy it in:
#![feature(old_io,std_misc)]
use std::thread::{self,JoinGuard};
use std::old_io::timer;
use std::time::duration::Duration;
struct MyData {
x: i32,
}
// returns immediately
impl MyData {
fn foo(&self) -> JoinGuard<()> {
thread::scoped(move || {
timer::sleep(Duration::milliseconds(300));
println!("{:?}", self.x); //consider a very long operation
timer::sleep(Duration::milliseconds(300));
})
}
}
fn main() {
let d = MyData { x: 42 };
let _thread = d.foo();
println!("I'm so fast!");
}