Given
fn greet(peeps: &str) {
println!("Hello, {}", peeps);
}
I can do:
fn main() {
let a = "World";
thread::spawn(move || greet(a)).join().unwrap();
}
The compiler understands that the thread does not outlive the borrowed string, but this is merely a special case when the lifetime of the &str is known to be 'static. When I try to do the same with a function argument, it does not compile:
fn indirect(peeps: &str) {
thread::spawn(move || greet(&peeps)).join().unwrap();
// Does not compile, for fear that the thread may outlive peeps
}
However, to a human reader, it is obviously the case that the thread cannot outlive the borrowed string.
I have found two workarounds:
Make a copy of the string, which can be moved into the thread:
fn indirect(peeps: &str) {
let peeps = peeps.to_string();
thread::spawn(move || greet(&peeps)).join().unwrap();
}
or, make use of the famously deprecated thread::scoped:
#![feature(scoped)]
fn indirect_scoped(peeps: &str) {
thread::scoped(move || greet(&peeps)).join();
}
I don't want to specify 'static lifetime for the function parameter, I'd prefer not to make an unnecessary copy (workaround 1) and I'd prefer not to use deprecated features (workaround 2).
What should I do in this situation?
The approach with scoped() is the correct way when you want to pass borrowed data to child threads. While thread::scoped() itself is deprecated due to its unsoundness, an alternative sound APIs like crossbeam or scoped_threadpool provide a way to do this on stable Rust:
extern crate crossbeam;
fn indirect(peeps: &str) {
crossbeam::scope(|scope| {
scope.spawn(|| greet(peeps));
});
}
Related
Here's a Thing:
struct Thing(i32);
impl Thing {
pub fn increment_self(&mut self) {
self.0 += 1;
println!("incremented: {}", self.0);
}
}
And here's a function that tries to mutate a Thing and returns either true or false, depending on if a Thing is available:
fn try_increment(handle: Option<&mut Thing>) -> bool {
if let Some(t) = handle {
t.increment_self();
true
} else {
println!("warning: increment failed");
false
}
}
Here's a sample of usage:
fn main() {
try_increment(None);
let mut thing = Thing(0);
try_increment(Some(&mut thing));
try_increment(Some(&mut thing));
try_increment(None);
}
As written, above, it works just fine (link to Rust playground). Output below:
warning: increment failed
incremented: 1
incremented: 2
warning: increment failed
The problem arises when I want to write a function that mutates the Thing twice. For example, the following does not work:
fn try_increment_twice(handle: Option<&mut Thing>) {
try_increment(handle);
try_increment(handle);
}
fn main() {
try_increment_twice(None);
let mut thing = Thing(0);
try_increment_twice(Some(&mut thing));
try_increment_twice(None);
}
The error makes perfect sense. The first call to try_increment(handle) gives ownership of handle away and so the second call is illegal. As is often the case, the Rust compiler yields a sensible error message:
|
24 | try_increment(handle);
| ------ value moved here
25 | try_increment(handle);
| ^^^^^^ value used here after move
|
In an attempt to solve this, I thought it would make sense to pass handle by reference. It should be an immutable reference, mind, because I don't want try_increment to be able to change handle itself (assigning None to it, for example) only to be able to call mutations on its value.
My problem is that I couldn't figure out how to do this.
Here is the closest working version that I could get:
struct Thing(i32);
impl Thing {
pub fn increment_self(&mut self) {
self.0 += 1;
println!("incremented: {}", self.0);
}
}
fn try_increment(handle: &mut Option<&mut Thing>) -> bool {
// PROBLEM: this line is allowed!
// (*handle) = None;
if let Some(ref mut t) = handle {
t.increment_self();
true
} else {
println!("warning: increment failed");
false
}
}
fn try_increment_twice(mut handle: Option<&mut Thing>) {
try_increment(&mut handle);
try_increment(&mut handle);
}
fn main() {
try_increment_twice(None);
let mut thing = Thing(0);
try_increment_twice(Some(&mut thing));
try_increment_twice(None);
}
This code runs, as expected, but the Option is now passed about by mutable reference and that is not what I want:
I'm allowed to mutate the Option by reassigning None to it, breaking all following mutations. (Uncomment line 12 ((*handle) = None;) for example.)
It's messy: There are a whole lot of extraneous &mut's lying about.
It's doubly messy: Heaven only knows why I must use ref mut in the if let statement while the convention is to use &mut everywhere else.
It defeats the purpose of having the complicated borrow-checking and mutability checking rules in the compiler.
Is there any way to actually achieve what I want: passing an immutable Option around, by reference, and actually being able to use its contents?
You can't extract a mutable reference from an immutable one, even a reference to its internals. That's kind of the point! Multiple aliases of immutable references are allowed so, if Rust allowed you to do that, you could have a situation where two pieces of code are able to mutate the same data at the same time.
Rust provides several escape hatches for interior mutability, for example the RefCell:
use std::cell::RefCell;
fn try_increment(handle: &Option<RefCell<Thing>>) -> bool {
if let Some(t) = handle {
t.borrow_mut().increment_self();
true
} else {
println!("warning: increment failed");
false
}
}
fn try_increment_twice(handle: Option<RefCell<Thing>>) {
try_increment(&handle);
try_increment(&handle);
}
fn main() {
let mut thing = RefCell::new(Thing(0));
try_increment_twice(Some(thing));
try_increment_twice(None);
}
TL;DR: The answer is No, I can't.
After the discussions with #Peter Hall and #Stargateur, I have come to understand why I need to use &mut Option<&mut Thing> everywhere. RefCell<> would also be a feasible work-around but it is no neater and does not really achieve the pattern I was originally seeking to implement.
The problem is this: if one were allowed to mutate the object for which one has only an immutable reference to an Option<&mut T> one could use this power to break the borrowing rules entirely. Concretely, you could, essentially, have many mutable references to the same object because you could have many such immutable references.
I knew there was only one mutable reference to the Thing (owned by the Option<>) but, as soon as I started taking references to the Option<>, the compiler no longer knew that there weren't many of those.
The best version of the pattern is as follows:
fn try_increment(handle: &mut Option<&mut Thing>) -> bool {
if let Some(ref mut t) = handle {
t.increment_self();
true
}
else {
println!("warning: increment failed");
false
}
}
fn try_increment_twice(mut handle: Option<&mut Thing>) {
try_increment(&mut handle);
try_increment(&mut handle);
}
fn main() {
try_increment_twice(None);
let mut thing = Thing(0);
try_increment_twice(Some(&mut thing));
try_increment_twice(None);
}
Notes:
The Option<> holds the only extant mutable reference to the Thing
try_increment_twice() takes ownership of the Option<>
try_increment() must take the Option<> as &mut so that the compiler knows that it has the only mutable reference to the Option<>, during the call
If the compiler knows that try_increment() has the only mutable reference to the Option<> which holds the unique mutable reference to the Thing, the compiler knows that the borrow rules have not been violated.
Another Experiment
The problem of the mutability of Option<> remains because one can call take() et al. on a mutable Option<>, breaking everything following.
To implement the pattern that I wanted, I need something that is like an Option<> but, even if it is mutable, it cannot be mutated. Something like this:
struct Handle<'a> {
value: Option<&'a mut Thing>,
}
impl<'a> Handle<'a> {
fn new(value: &'a mut Thing) -> Self {
Self {
value: Some(value),
}
}
fn empty() -> Self {
Self {
value: None,
}
}
fn try_mutate<T, F: Fn(&mut Thing) -> T>(&mut self, mutation: F) -> Option<T> {
if let Some(ref mut v) = self.value {
Some(mutation(v))
}
else {
None
}
}
}
Now, I thought, I can pass around &mut Handle's all day long and know that someone who has a Handle can only mutate its contents, not the handle itself. (See Playground)
Unfortunately, even this gains nothing because, if you have a mutable reference, you can always reassign it with the dereferencing operator:
fn try_increment(handle: &mut Handle) -> bool {
if let Some(_) = handle.try_mutate(|t| { t.increment_self() }) {
// This breaks future calls:
(*handle) = Handle::empty();
true
}
else {
println!("warning: increment failed");
false
}
}
Which is all fine and well.
Bottom line conclusion: just use &mut Option<&mut T>
I am trying to fire up a new thread using some heap data in Rust and I am getting a bunch of errors that stem from the need of the data to have 'static lifetime. I've worked my way backwards up my program but hit a problem.
use std::sync::Arc;
use std::thread;
struct ThreadData {
vector_of_strings: Vec<String>,
terms: Vec<&'static str>,
quotient: usize,
}
fn perform_search(slice: &[String], terms: &[&str]) {
/* ... */
}
fn threaded_search(td_arc: &Arc<ThreadData>) {
let no_of_lines = td_arc.vector_of_strings.len();
let new_tda1 = td_arc.clone();
let strings_as_slice1 = new_tda1.vector_of_strings.as_slice();
thread::spawn(move || {
perform_search(&strings_as_slice1[0..td_arc.quotient], &new_tda1.terms);
});
}
fn main() {
let td = ThreadData {
vector_of_strings: Vec::new(),
terms: Vec::new(),
quotient: 0,
};
let td_arc = Arc::new(td);
threaded_search(&td_arc);
}
Error:
error[E0621]: explicit lifetime required in the type of `td_arc`
--> src/main.rs:20:5
|
14 | fn threaded_search(td_arc: &Arc<ThreadData>) {
| ---------------- help: add explicit lifetime `'static` to the type of `td_arc`: `&'static std::sync::Arc<ThreadData>`
...
20 | thread::spawn(move || {
| ^^^^^^^^^^^^^ lifetime `'static` required
The error about 'static is because the new thread created within thread::spawn may outlive the invocation of threaded_search during which the thread is initially created, which means the thread must not be permitted to use any local variables from threaded_search with a lifetime shorter than 'static.
In your code the new thread is referring to strings_as_slice1 and td_arc.
Generally with thread::spawn and Arc you will want to move ownership of one reference count into the thread and have the thread access whatever it needs through that reference counted pointer rather than from the enclosing short-lived scope directly.
fn threaded_search(td_arc: &Arc<ThreadData>) {
// Increment reference count that we can move into the new thread.
let td_arc = td_arc.clone();
thread::spawn(move || {
perform_search(&td_arc.vector_of_strings[0..td_arc.quotient], &td_arc.terms);
});
}
I'm trying to execute a function on chunks of a vector and then send the result back using the message passing library.
However, I get a strange error about the lifetime of the vector that isn't even participating in the thread operations:
src/lib.rs:153:27: 154:25 error: borrowed value does not live long enough
src/lib.rs:153 let extended_segments = (segment_size..max_val)
error: src/lib.rs:154 .collect::<Vec<_>>()borrowed value does not live long enough
note: reference must be valid for the static lifetime...:153
let extended_segments = (segment_size..max_val)
src/lib.rs:153:3: 155:27: 154 .collect::<Vec<_>>()
note: but borrowed value is only valid for the statement at 153:2:
reference must be valid for the static lifetime...
src/lib.rs:
let extended_segments = (segment_size..max_val)
consider using a `let` binding to increase its lifetime
I tried moving around the iterator and adding lifetimes to different places, but I couldn't get the checker to pass and still stay on type.
The offending code is below, based on the concurrency chapter in the Rust book. (Complete code is at github.)
use std::sync::mpsc;
use std::thread;
fn sieve_segment(a: &[usize], b: &[usize]) -> Vec<usize> {
vec![]
}
fn eratosthenes_sieve(val: usize) -> Vec<usize> {
vec![]
}
pub fn segmented_sieve_parallel(max_val: usize, mut segment_size: usize) -> Vec<usize> {
if max_val <= ((2 as i64).pow(16) as usize) {
// early return if the highest value is small enough (empirical)
return eratosthenes_sieve(max_val);
}
if segment_size > ((max_val as f64).sqrt() as usize) {
segment_size = (max_val as f64).sqrt() as usize;
println!("Segment size is larger than √{}. Reducing to {} to keep resource use down.",
max_val,
segment_size);
}
let small_primes = eratosthenes_sieve((max_val as f64).sqrt() as usize);
let mut big_primes = small_primes.clone();
let (tx, rx): (mpsc::Sender<Vec<usize>>, mpsc::Receiver<Vec<usize>>) = mpsc::channel();
let extended_segments = (segment_size..max_val)
.collect::<Vec<_>>()
.chunks(segment_size);
for this_segment in extended_segments.clone() {
let small_primes = small_primes.clone();
let tx = tx.clone();
thread::spawn(move || {
let sieved_segment = sieve_segment(&small_primes, this_segment);
tx.send(sieved_segment).unwrap();
});
}
for _ in 1..extended_segments.count() {
big_primes.extend(&rx.recv().unwrap());
}
big_primes
}
fn main() {}
How do I understand and avoid this error? I'm not sure how to make the lifetime of the thread closure static as in this question and still have the function be reusable (i.e., not main()). I'm not sure how to "consume all things that come into [the closure]" as mentioned in this question. And I'm not sure where to insert .map(|s| s.into()) to ensure that all references become moves, nor am I sure I want to.
When trying to reproduce a problem, I'd encourage you to create a MCVE by removing all irrelevant code. In this case, something like this seems to produce the same error:
fn segmented_sieve_parallel(max_val: usize, segment_size: usize) {
let foo = (segment_size..max_val)
.collect::<Vec<_>>()
.chunks(segment_size);
}
fn main() {}
Let's break that down:
Create an iterator between numbers.
Collect all of them into a Vec<usize>.
Return an iterator that contains references to the vector.
Since the vector isn't bound to any variable, it's dropped at the end of the statement. This would leave the iterator pointing to an invalid region of memory, so that's disallowed.
Check out the definition of slice::chunks:
fn chunks(&self, size: usize) -> Chunks<T>
pub struct Chunks<'a, T> where T: 'a {
// some fields omitted
}
The lifetime marker 'a lets you know that the iterator contains a reference to something. Lifetime elision has removed the 'a from the function, which looks like this, expanded:
fn chunks<'a>(&'a self, size: usize) -> Chunks<'a, T>
Check out this line of the error message:
help: consider using a let binding to increase its lifetime
You can follow that as such:
fn segmented_sieve_parallel(max_val: usize, segment_size: usize) {
let foo = (segment_size..max_val)
.collect::<Vec<_>>();
let bar = foo.chunks(segment_size);
}
fn main() {}
Although I'd write it as
fn segmented_sieve_parallel(max_val: usize, segment_size: usize) {
let foo: Vec<_> = (segment_size..max_val).collect();
let bar = foo.chunks(segment_size);
}
fn main() {}
Re-inserting this code back into your original problem won't solve the problem, but it will be much easier to understand. That's because you are attempting to pass a reference to thread::spawn, which may outlive the current thread. Thus, everything passed to thread::spawn must have the 'static lifetime. There are tons of questions that detail why that must be prevented and a litany of solutions, including scoped threads and cloning the vector.
Cloning the vector is the easiest, but potentially inefficient:
for this_segment in extended_segments.clone() {
let this_segment = this_segment.to_vec();
// ...
}
I am attempting to write a simpler unit test runner for my Rust project. I have created a TestFixture trait that my test fixture structs will implement, similar to inheriting from the unit test base class in other testing frameworks. The trait is fairly simple. This is my test fixture
pub trait TestFixture {
fn setup(&mut self) -> () {}
fn teardown(&mut self) -> () {}
fn before_each(&mut self) -> () {}
fn after_each(&mut self) -> () {}
fn tests(&mut self) -> Vec<Box<Fn(&mut Self)>>
where Self: Sized {
Vec::new()
}
}
My test running function is as follows
pub fn test_fixture_runner<T: TestFixture>(fixture: &mut T) {
fixture.setup();
let _r = fixture.tests().iter().map(|t| {
let handle = thread::spawn(move || {
fixture.before_each();
t(fixture);
fixture.after_each();
});
if let Err(_) = handle.join() {
println!("Test failed!")
}
});
fixture.teardown();
}
I get the error
src/tests.rs:73:22: 73:35 error: the trait `core::marker::Send` is not implemented for the type `T` [E0277]
src/tests.rs:73 let handle = thread::spawn(move || {
^~~~~~~~~~~~~
note: in expansion of closure expansion
src/tests.rs:69:41: 84:6 note: expansion site
src/tests.rs:73:22: 73:35 note: `T` cannot be sent between threads safely
src/tests.rs:73 let handle = thread::spawn(move || {
^~~~~~~~~~~~~
note: in expansion of closure expansion
src/tests.rs:69:41: 84:6 note: expansion site
src/tests.rs:73:22: 73:35 error: the trait `core::marker::Sync` is not implemented for the type `for<'r> core::ops::Fn(&'r mut T)` [E0277]
src/tests.rs:73 let handle = thread::spawn(move || {
^~~~~~~~~~~~~
note: in expansion of closure expansion
src/tests.rs:69:41: 84:6 note: expansion site
src/tests.rs:73:22: 73:35 note: `for<'r> core::ops::Fn(&'r mut T)` cannot be shared between threads safely
src/tests.rs:73 let handle = thread::spawn(move || {
^~~~~~~~~~~~~
note: in expansion of closure expansion
I have tried adding Arcs around the types being sent to the thread, no dice, same error.
pub fn test_fixture_runner<T: TestFixture>(fixture: &mut T) {
fixture.setup();
let fix_arc = Arc::new(Mutex::new(fixture));
let _r = fixture.tests().iter().map(|t| {
let test_arc = Arc::new(Mutex::new(t));
let fix_arc_clone = fix_arc.clone();
let test_arc_clone = test_arc.clone();
let handle = thread::spawn(move || {
let thread_test = test_arc_clone.lock().unwrap();
let thread_fix = fix_arc_clone.lock().unwrap();
(*thread_fix).before_each();
(*thread_test)(*thread_fix);
(*thread_fix).after_each();
});
if let Err(_) = handle.join() {
println!("Test failed!")
}
});
fixture.teardown();
}
A sample test fixture would be something like
struct BuiltinTests {
pwd: PathBuf
}
impl TestFixture for BuiltinTests {
fn setup(&mut self) {
let mut pwd = env::temp_dir();
pwd.push("pwd");
fs::create_dir(&pwd);
self.pwd = pwd;
}
fn teardown(&mut self) {
fs::remove_dir(&self.pwd);
}
fn tests(&mut self) -> Vec<Box<Fn(&mut BuiltinTests)>> {
vec![Box::new(BuiltinTests::cd_with_no_args)]
}
}
impl BuiltinTests {
fn new() -> BuiltinTests {
BuiltinTests {
pwd: PathBuf::new()
}
}
}
fn cd_with_no_args(&mut self) {
let home = String::from("/");
env::set_var("HOME", &home);
let mut cd = Cd::new();
cd.run(&[]);
assert_eq!(env::var("PWD"), Ok(home));
}
#[test]
fn cd_tests() {
let mut builtin_tests = BuiltinTests::new();
test_fixture_runner(&mut builtin_tests);
}
My whole intention of using threads is isolation from the test runner. If a test fails an assertion it causes a panic which kills the runner. Thanks for any insight, I'm willing to change my design if that will fix the panic problem.
There are several problems with your code, I'll show you how to fix them one by one.
The first problem is that you're using map() to iterate over an iterator. It won't work correctly because map() is lazy - unless you consume the iterator, the closure you passed to it won't run. The correct way is to use for loop:
for t in fixture().tests().iter() {
Second, you're iterating the vector of closures by reference:
fixture.tests().iter().map(|t| {
iter() on a Vec<T> returns an iterator yielding items of type &T, so your t will be of type &Box<Fn(&mut Self)>. However, Box<Fn(&mut T)> does not implement Sync by default (it is a trait object which have no information about the underlying type except that you specified explicitly), so &Box<Fn(&mut T)> can't be used across multiple threads. That's what the second error you see is about.
Most likely you don't want to use these closures by reference; you probably want to move them to the spawned thread entirely. For this you need to use into_iter() instead of iter():
for t in fixture.tests().into_iter() {
Now t will be of type Box<Fn(&mut T)>. However, it still can't be sent across threads. Again, it is a trait object, and the compiler does not know if the type contained inside is Send. For this you need to add Send bound to the type of the closure:
fn tests(&mut self) -> Vec<Box<Fn(&mut Self)+Send>>
Now the error about Fn is gone.
The last error is about Send not being implemented for T. We need to add a Send bound on T:
pub fn test_fixture_runner<T: TestFixture+Send>(fixture: &mut T) {
And now the error becomes more comprehensible:
test.rs:18:22: 18:35 error: captured variable `fixture` does not outlive the enclosing closure
test.rs:18 let handle = thread::spawn(move || {
^~~~~~~~~~~~~
note: in expansion of closure expansion
test.rs:18:5: 28:6 note: expansion site
test.rs:15:66: 31:2 note: captured variable is valid for the anonymous lifetime #1 defined on the block at 15:65
test.rs:15 pub fn test_fixture_runner<T: TestFixture+Send>(fixture: &mut T) {
test.rs:16 fixture.setup();
test.rs:17
test.rs:18 for t in fixture.tests().into_iter() {
test.rs:19 let handle = thread::spawn(move || {
test.rs:20 fixture.before_each();
...
note: closure is valid for the static lifetime
This error happens because you're trying to use a reference in a spawn()ed thread. spawn() requires its closure argument to have 'static bound, that is, its captured environment must not contain references with non-'static lifetimes. But that's exactly what happens here - &mut T is not 'static. spawn() design does not prohibit avoiding joining, so it is explicitly written to disallow passing non-'static references to the spawned thread.
Note that while you're using &mut T, this error is unavoidable, even if you put &mut T in Arc, because then the lifetime of &mut T would be "stored" in Arc and so Arc<Mutex<&mut T>> also won't be 'static.
There are two ways to do what you want.
First, you can use the unstable thread::scoped() API. It is unstable because it is shown to allow memory unsafety in safe code, and the plan is to provide some kind of replacement for it in the future. However, you can use it in nightly Rust (it won't cause memory unsafety by itself, only in specifically crafted situations):
pub fn test_fixture_runner<T: TestFixture+Send>(fixture: &mut T) {
fixture.setup();
let tests = fixture.lock().unwrap().tests();
for t in tests.into_iter() {
let f = &mut *fixture;
let handle = thread::scoped(move || {
f.before_each();
t(f);
f.after_each();
});
handle.join();
}
fixture.teardown();
}
This code compiles because scoped() is written in such a way that it guarantees (in most cases) that the thread won't outlive all captured references. I had to reborrow fixture because otherwise (because &mut references aren't copyable) it would be moved into the thread and fixture.teardown() would be prohibited. Also I had to extract tests variable because otherwise the mutex will be locked by the main thread for the duration of the for loop which would naturally disallow locking it in the child threads.
However, with scoped() you can't isolate the panic in the child thread. If the child thread panics, this panic will be rethrown from join() call. This may or may not be a problem in general, but I think it is a problem for your code.
Another way is to refactor your code to hold the fixture in Arc<Mutex<..>> from the beginning:
pub fn test_fixture_runner<T: TestFixture + Send + 'static>(fixture: Arc<Mutex<T>>) {
fixture.lock().unwrap().setup();
for t in fixture.lock().unwrap().tests().into_iter() {
let fixture = fixture.clone();
let handle = thread::spawn(move || {
let mut fixture = fixture.lock().unwrap();
fixture.before_each();
t(&mut *fixture);
fixture.after_each();
});
if let Err(_) = handle.join() {
println!("Test failed!")
}
}
fixture.lock().unwrap().teardown();
}
Note that now T has also to be 'static, again, because otherwise it couldn't be used with thread::spawn() as it requires 'static. fixture inside the inner closure is not &mut T but a MutexGuard<T>, and so it has to be explicitly converted to &mut T in order to pass it to t.
This may seem overly and unnecessarily complex, however, such design of a programming language does prevent you from making many errors in multithreaded programming. Each of the above errors we have seen is valid - each of them would be a potential cause of memory unsafety or data races if it was ignored.
As stated in the Rust HandBook's Concurrency section:
When a type T implements Send, it indicates to the compiler that something of this type is able to have ownership transferred safely between threads.
If you do not implement Send, ownership cannot be transfered between threads.
So I ran into this code snippet showing how to create "unmoveable" types in Rust - moves are prevented because the compiler treats the object as borrowed for its whole lifetime.
use std::cell::Cell;
use std::marker;
struct Unmovable<'a> {
lock: Cell<marker::ContravariantLifetime<'a>>,
marker: marker::NoCopy
}
impl<'a> Unmovable<'a> {
fn new() -> Unmovable<'a> {
Unmovable {
lock: Cell::new(marker::ContravariantLifetime),
marker: marker::NoCopy
}
}
fn lock(&'a self) {
self.lock.set(marker::ContravariantLifetime);
}
fn new_in(self_: &'a mut Option<Unmovable<'a>>) {
*self_ = Some(Unmovable::new());
self_.as_ref().unwrap().lock();
}
}
fn main(){
let x = Unmovable::new();
x.lock();
// error: cannot move out of `x` because it is borrowed
// let z = x;
let mut y = None;
Unmovable::new_in(&mut y);
// error: cannot move out of `y` because it is borrowed
// let z = y;
assert_eq!(std::mem::size_of::<Unmovable>(), 0)
}
I don't yet understand how this works. My guess is that the lifetime of the borrow-pointer argument is forced to match the lifetime of the lock field. The weird thing is, this code continues working in the same way if:
I change ContravariantLifetime<'a> to CovariantLifetime<'a>, or to InvariantLifetime<'a>.
I remove the body of the lock method.
But, if I remove the Cell, and just use lock: marker::ContravariantLifetime<'a> directly, as so:
use std::marker;
struct Unmovable<'a> {
lock: marker::ContravariantLifetime<'a>,
marker: marker::NoCopy
}
impl<'a> Unmovable<'a> {
fn new() -> Unmovable<'a> {
Unmovable {
lock: marker::ContravariantLifetime,
marker: marker::NoCopy
}
}
fn lock(&'a self) {
}
fn new_in(self_: &'a mut Option<Unmovable<'a>>) {
*self_ = Some(Unmovable::new());
self_.as_ref().unwrap().lock();
}
}
fn main(){
let x = Unmovable::new();
x.lock();
// does not error?
let z = x;
let mut y = None;
Unmovable::new_in(&mut y);
// does not error?
let z = y;
assert_eq!(std::mem::size_of::<Unmovable>(), 0)
}
Then the "Unmoveable" object is allowed to move. Why would that be?
The true answer is comprised of a moderately complex consideration of lifetime variancy, with a couple of misleading aspects of the code that need to be sorted out.
For the code below, 'a is an arbitrary lifetime, 'small is an arbitrary lifetime that is smaller than 'a (this can be expressed by the constraint 'a: 'small), and 'static is used as the most common example of a lifetime that is larger than 'a.
Here are the facts and steps to follow in the consideration:
Normally, lifetimes are contravariant; &'a T is contravariant with regards to 'a (as is T<'a> in the absence of any variancy markers), meaning that if you have a &'a T, it’s OK to substitute a longer lifetime than 'a, e.g. you can store in such a place a &'static T and treat it as though it were a &'a T (you’re allowed to shorten the lifetime).
In a few places, lifetimes can be invariant; the most common example is &'a mut T which is invariant with regards to 'a, meaning that if you have a &'a mut T, you cannot store a &'small mut T in it (the borrow doesn’t live long enough), but you also cannot store a &'static mut T in it, because that would cause trouble for the reference being stored as it would be forgotten that it actually lived for longer, and so you could end up with multiple simultaneous mutable references being created.
A Cell contains an UnsafeCell; what isn’t so obvious is that UnsafeCell is magic, being wired to the compiler for special treatment as the language item named “unsafe”. Importantly, UnsafeCell<T> is invariant with regards to T, for similar sorts of reasons to the invariance of &'a mut T with regards to 'a.
Thus, Cell<any lifetime variancy marker> will actually behave the same as Cell<InvariantLifetime<'a>>.
Furthermore, you don’t actually need to use Cell any more; you can just use InvariantLifetime<'a>.
Returning to the example with the Cell wrapping removed and a ContravariantLifetime (actually equivalent to just defining struct Unmovable<'a>;, for contravariance is the default as is no Copy implementation): why does it allow moving the value? … I must confess, I don’t grok this particular case yet and would appreciate some help myself in understanding why it’s allowed. It seems back to front, that covariance would allow the lock to be shortlived but that contravariance and invariance wouldn’t, but in practice it seems that only invariance is performing the desired function.
Anyway, here’s the final result. Cell<ContravariantLifetime<'a>> is changed to InvariantLifetime<'a> and that’s the only functional change, making the lock method function as desired, taking a borrow with an invariant lifetime. (Another solution would be to have lock take &'a mut self, for a mutable reference is, as already discussed, invariant; this is inferior, however, as it requires needless mutability.)
One other thing that needs mentioning: the contents of the lock and new_in methods are completely superfluous. The body of a function will never change the static behaviour of the compiler; only the signature matters. The fact that the lifetime parameter 'a is marked invariant is the key point. So the whole “construct an Unmovable object and call lock on it” part of new_in is completely superfluous. Similarly setting the contents of the cell in lock was a waste of time. (Note that it is again the invariance of 'a in Unmovable<'a> that makes new_in work, not the fact that it is a mutable reference.)
use std::marker;
struct Unmovable<'a> {
lock: marker::InvariantLifetime<'a>,
}
impl<'a> Unmovable<'a> {
fn new() -> Unmovable<'a> {
Unmovable {
lock: marker::InvariantLifetime,
}
}
fn lock(&'a self) { }
fn new_in(_: &'a mut Option<Unmovable<'a>>) { }
}
fn main() {
let x = Unmovable::new();
x.lock();
// This is an error, as desired:
let z = x;
let mut y = None;
Unmovable::new_in(&mut y);
// Yay, this is an error too!
let z = y;
}
An interesting problem! Here's my understanding of it...
Here's another example that doesn't use Cell:
#![feature(core)]
use std::marker::InvariantLifetime;
struct Unmovable<'a> { //'
lock: Option<InvariantLifetime<'a>>, //'
}
impl<'a> Unmovable<'a> {
fn lock_it(&'a mut self) { //'
self.lock = Some(InvariantLifetime)
}
}
fn main() {
let mut u = Unmovable { lock: None };
u.lock_it();
let v = u;
}
(Playpen)
The important trick here is that the structure needs to borrow itself. Once we have done that, it can no longer be moved because any move would invalidate the borrow. This isn't conceptually different from any other kind of borrow:
struct A(u32);
fn main() {
let a = A(42);
let b = &a;
let c = a;
}
The only thing is that you need some way of letting the struct contain its own reference, which isn't possible to do at construction time. My example uses Option, which requires &mut self and the linked example uses Cell, which allows for interior mutability and just &self.
Both examples use a lifetime marker because it allows the typesystem to track the lifetime without needing to worry about a particular instance.
Let's look at your constructor:
fn new() -> Unmovable<'a> { //'
Unmovable {
lock: marker::ContravariantLifetime,
marker: marker::NoCopy
}
}
Here, the lifetime put into lock is chosen by the caller, and it ends up being the normal lifetime of the Unmovable struct. There's no borrow of self.
Let's next look at your lock method:
fn lock(&'a self) {
}
Here, the compiler knows that the lifetime won't change. However, if we make it mutable:
fn lock(&'a mut self) {
}
Bam! It's locked again. This is because the compiler knows that the internal fields could change. We can actually apply this to our Option variant and remove the body of lock_it!