From implementation of pin_mut!, there are
...
// Move the value to ensure that it is owned
let mut $x = $x;
...
Removing this code, the original $x is still shadowed, so the intention of moving $x here is to "Move the value to ensure that it is owned".
However, I wonder why this is necessary? I found the following code will fire if we can't mutablly borrow $x.
...
let mut $x = unsafe {
$crate::core_reexport::pin::Pin::new_unchecked(&mut $x)
};
...
And the definition of this macro won't accept like pin_mut!(&mut x).
So in what condition will let mut $x = $x fire alone?
You can get into trouble without the ownership transfer because you couldn't ensure that the original value is not accessible. Consider the macro that omits it (which would essentially just be obscuring an unsafe call):
use std::pin::Pin;
macro_rules! pin_mut {
($($x:ident),* $(,)?) => { $(
let mut $x = unsafe {
Pin::new_unchecked(&mut $x)
};
)* }
}
The ident parameter of the macro protects against shenanigans like attempting to pin *&mut foo (which the ownership transfer would prevent as well), however that doesn't mean that we have fully shadowed the name $x. This code would be allowed and shouldn't be:
let mut foo = Foo { ... };
{
pin_mut!(foo);
let _: Pin<&mut Foo> = foo;
}
// Woops we now have an unprotected Foo when its supposed to be pinned and
// thus can break the guarantees of Pin::new_unchecked
let foo_ref: &mut Foo = &mut foo;
See it compiling on the playground. This scenario is mentioned in the Pin::new_unchecked docs.
Related
I'll use the CustomSmartPointer from The Book, which is used to explain the Drop trait, to build an example:
struct CustomSmartPointer {
data: String,
}
impl Drop for CustomSmartPointer {
fn drop(&mut self) {
println!("Dropping CustomSmartPointer with data `{}`!", self.data);
}
}
fn main() {
println!("A");
let pointer = CustomSmartPointer {
data: String::from("my stuff"),
};
println!("B");
println!("output: {}", pointer.data);
println!("C");
}
This prints:
A
B
output: my stuff
C
Dropping CustomSmartPointer with data `my stuff`!
However, from what I learned I would have expected the last two lines to be swapped. The local variable pointer isn't used anymore after being printed, so I would have expected its scope to end after the line that prints its contents, before "C" gets printed. For example, the section on references has this example:
let mut s = String::from("hello");
let r1 = &s; // no problem
let r2 = &s; // no problem
println!("{} and {}", r1, r2);
// variables r1 and r2 will not be used after this point
let r3 = &mut s; // no problem
println!("{}", r3);
So it seems like either (1) the Drop trait extends the scope until the end of the current block, or (2) it is only references whose scope ends "ASAP" and everything else already lives until the end of the current block (and that would be a major difference between references and smart pointers), or (3) something entirely else is going on here. Which one is it?
Edit: another example
I realized one thing while reading the answers so far: that (1) and (2) are equivalent if (1) includes cases where a type does not implement the Drop trait but "things happen while dropping", which is the case e.g. for a struct that contains another value that implements Drop. I tried this and it is indeed the case (using the CustomSmartPointer from above):
struct Wrapper {
csp: CustomSmartPointer,
}
fn create_wrapper() -> Wrapper {
let pointer = CustomSmartPointer {
data: String::from("my stuff"),
};
Wrapper {
csp: pointer,
}
}
fn main() {
println!("A");
let wrapper = create_wrapper();
println!("B");
println!("output: {}", wrapper.csp.data);
println!("C");
}
This still prints "Dropping CSP" last, after "C", so even a non-Drop wrapper struct that contains a value with the Drop trait has lexical scope. As hinted above, you could then equivalently say: the Drop-able value inside the struct causes a usage at the end of the block that causes the whole wrapper to be dropped only at the end of the block, or you could say that only references have NLL. The difference between the two statements is only about when a value gets dropped that is deeply free of Drop-trait values, which isn't observable.
Do not look if a type implements Drop, look at the return value of needs_drop::<T>() instead.
That said, it is option (1): a type that needs_drop() has an implicit call to drop() at the end of the lexical scope. It is this call that extends the scope of the value.
So you code is as if:
fn main() {
println!("A");
let wrapper = create_wrapper();
println!("B");
println!("output: {}", wrapper.csp.data);
println!("C");
drop(wrapper); // <- implicitly called, wrapper scope ends here
}
Naturally, you can call drop(wrapper) anywhere to end the scope prematurely. As drop() takes its argument by value, it finishes the scope there.
If the type of a value does not needs_drop(), then it is released at the last usage of that value, that is a non lexical scope (NLL).
The non-lexical scopes affects not only references, but any type that doesn't need drop. The thing is that if a value doesn't need drop and doesn't borrow anything, then its scope does not have any visible effect and nobody cares.
For example, this code has a NLL that is technically not a reference:
use std::marker::PhantomData;
#[derive(Debug)]
struct Foo<'a> { _pd: PhantomData<&'a ()> }
impl<'a> Foo<'a> {
fn new<T>(x: &'a mut T) -> Foo<'a> {
Foo { _pd: PhantomData }
}
}
fn main() {
let mut x = 42;
let f1 = Foo::new(&mut x);
let f2 = Foo::new(&mut x);
//dbg!(&f1); // uncomment this line and it will fail to compile
}
It's option two. Implementing Drop trait means that additional actions will happen when object is dropped. But everything will be at some point dropped, whether it implements Drop or not.
println! is not taking your pointer
playgroud...
However, from what I learned I would have expected the last two lines to be swapped. The local variable pointer isn't used anymore after being printed, so I would have expected its scope to end after the line that prints its contents, before "C" gets printed. For example, the section on references has this example:
let mut s = String::from("hello");
let r1 = &s; // no problem
let r2 = &s; // no problem
println!("{} and {}", r1, r2);
// variables r1 and r2 will not be used after this point
let r3 = &mut s; // no problem
println!("{}", r3);
The thing here is that s is never taked, r1 and r2 are dropped after r3 because it takes a mutable reference to s
playground
Problem:
Im new to Rust, and im trying to implement a macro which simulates sscanf from C.
So far it works with any numeric types, but not with strings, as i am already trying to parse a string.
macro_rules! splitter {
( $string:expr, $sep:expr) => {
let mut iter:Vec<&str> = $string.split($sep).collect();
iter
}
}
macro_rules! scan_to_types {
($buffer:expr,$sep:expr,[$($y:ty),+],$($x:expr),+) => {
let res = splitter!($buffer,$sep);
let mut i = 0;
$(
$x = res[i].parse::<$y>().unwrap_or_default();
i+=1;
)*
};
}
fn main() {
let mut a :u8; let mut b :i32; let mut c :i16; let mut d :f32;
let buffer = "00:98;76,39.6";
let sep = [':',';',','];
scan_to_types!(buffer,sep,[u8,i32,i16,f32],a,b,c,d); // this will work
println!("{} {} {} {}",a,b,c,d);
}
This obviously wont work, because at compile time, it will try to parse a string slice to str:
let a :u8; let b :i32; let c :i16; let d :f32; let e :&str;
let buffer = "02:98;abc,39.6";
let sep = [':',';',','];
scan_to_types!(buffer,sep,[u8,i32,&str,f32],a,b,e,d);
println!("{} {} {} {}",a,b,e,d);
$x = res[i].parse::<$y>().unwrap_or_default();
| ^^^^^ the trait `FromStr` is not implemented for `&str`
What i have tried
I have tried to compare types using TypeId, and a if else condition inside of the macro to skip the parsing, but the same situation happens, because it wont expand to a valid code:
macro_rules! scan_to_types {
($buffer:expr,$sep:expr,[$($y:ty),+],$($x:expr),+) => {
let res = splitter!($buffer,$sep);
let mut i = 0;
$(
if TypeId::of::<$y>() == TypeId::of::<&str>(){
$x = res[i];
}else{
$x = res[i].parse::<$y>().unwrap_or_default();
}
i+=1;
)*
};
}
Is there a way to set conditions or skip a repetition inside of a macro ? Or instead, is there a better aproach to build sscanf using macros ? I have already made functions which parse those strings, but i couldnt pass types as arguments, or make them generic.
Note before the answer: you probably don't want to emulate sscanf() in Rust. There are many very capable parsers in Rust, so you should probably use one of them.
Simple answer: the simplest way to address your problem is to replace the use of &str with String, which makes your macro compile and run. If your code is not performance-critical, that is probably all you need. If you care about performance and about avoiding allocation, read on.
A downside of String is that under the hood it copies the string data from the string you're scanning into a freshly allocated owned string. Your original approach of using an &str should have allowed for your &str to directly point into the data that was scanned, without any copying. Ideally we'd like to write something like this:
trait MyFromStr {
fn my_from_str(s: &str) -> Self;
}
// when called on a type that impls `FromStr`, use `parse()`
impl<T: FromStr + Default> MyFromStr for T {
fn my_from_str(s: &str) -> T {
s.parse().unwrap_or_default()
}
}
// when called on &str, just return it without copying
impl MyFromStr for &str {
fn my_from_str(s: &str) -> &str {
s
}
}
Unfortunately that doesn't compile, complaining of a "conflicting implementation of trait MyFromStr for &str", even though there is no conflict between the two implementations, as &str doesn't implement FromStr. But the way Rust currently works, a blanket implementation of a trait precludes manual implementations of the same trait, even on types not covered by the blanket impl.
In the future this will be resolved by specialization. Specialization is not yet part of stable Rust, and might not come to stable Rust for years, so we have to think of another solution. In case of macro usage, we can just let the compiler "specialize" for us by creating two traits with the same name. (This is similar to the autoref-based specialization invented by David Tolnay, but even simpler because it doesn't require autoref resolution to work, as we have the types provided explicitly.)
We create separate traits for parsed and unparsed values, and implement them as needed:
trait ParseFromStr {
fn my_from_str(s: &str) -> Self;
}
impl<T: FromStr + Default> ParseFromStr for T {
fn my_from_str(s: &str) -> T {
s.parse().unwrap_or_default()
}
}
pub trait StrFromStr {
fn my_from_str(s: &str) -> &str;
}
impl StrFromStr for &str {
fn my_from_str(s: &str) -> &str {
s
}
}
Then in the macro we just call <$y>::my_from_str() and let the compiler generate the correct code. Since macros are untyped, this works because we never need to provide a single "trait bound" that would disambiguate which my_from_str() we want. (Such a trait bound would require specialization.)
macro_rules! scan_to_types {
($buffer:expr,$sep:expr,[$($y:ty),+],$($x:expr),+) => {
#[allow(unused_assignments)]
{
let res = splitter!($buffer,$sep);
let mut i = 0;
$(
$x = <$y>::my_from_str(&res[i]);
i+=1;
)*
}
};
}
Complete example in the playground.
Can someone explain which exact temporary value is dropped and what the recommended way to do this operation is?
fn main() {
let mut a = &mut String::from("Hello Ownership");
a = &mut a.replace("Ownership", "World");
println!("a is {}", a);
}
If you want to keep the &mut references (which are generally not needed in your case, of course), you can do something like this:
fn main() {
let a = &mut String::from("Hello Ownership");
let a = &mut a.replace("Ownership", "World");
println!("a is {}", a);
}
The type of a would by &mut String. In the second line we do what's known as variable shadowing (not that it's needed) and the type is still &mut String.
That doesn't quite answer your question. I don't know why exactly your version doesn't compile, but at least I thought this info might be useful. (see below)
Update
Thanks to Solomon's findings, I wanted to add that apparently in this case:
let a = &mut ...;
let b = &mut ...;
or this one (variable shadowing, basically the same as the above):
let a = &mut ...;
let a = &mut ...;
, the compiler will automatically extend the lifetime of each temporary until the end of the enclosing block. However, in the case of:
let mut a = &mut ...;
a = &mut ...;
, it seems the compiler simply doesn't do such lifetime extension, so that's why the OP's code doesn't compile, even though the code seems to be doing pretty much the same thing.
Why are you using &mut there? Try this:
fn main() {
let mut a = String::from("Hello Ownership");
a = a.replace("Ownership", "World");
println!("a is {}", a);
}
Aha, figured it out!
https://doc.rust-lang.org/nightly/error-index.html#E0716 says:
Temporaries are not always dropped at the end of the enclosing statement. In simple cases where the & expression is immediately stored into a variable, the compiler will automatically extend the lifetime of the temporary until the end of the enclosing block. Therefore, an alternative way to fix the original program is to write let tmp = &foo() and not let tmp = foo():
fn foo() -> i32 { 22 }
fn bar(x: &i32) -> &i32 { x }
let value = &foo();
let p = bar(value);
let q = *p;
Here, we are still borrowing foo(), but as the borrow is assigned directly into a variable, the temporary will not be dropped until the end of the enclosing block. Similar rules apply when temporaries are stored into aggregate structures like a tuple or struct:
// Here, two temporaries are created, but
// as they are stored directly into `value`,
// they are not dropped until the end of the
// enclosing block.
fn foo() -> i32 { 22 }
let value = (&foo(), &foo());
Imagine that we have some object foo: Foo. Some of the time, when we need to do multithreaded computation on foo, we clone and move it. However, there are also times when we don't need to clone it, which may be expensive, and can instead put it in an Arc to share its data between threads. In the question title, "shared" means "inside an Arc", and "isolated" means in only some places, i.e. not program-wide.
I can think of two umbrella ways to do this:
Have the callee consume the object and wrap/unwrap in Arc internally, returning the object along with any other data in a tuple.
Accept Arc<Foo> as the callee, delegating wrap/unwrap responsibility to the caller.
However, neither of these seems particularly clean. (1) has a complicated return type that might complicate or hinder e.g. chained operations, and (2) forces the caller to do work that it shouldn't really need to know about.
I'm curious if there is an idiomatic pattern for handling this sort of situation. For reference, I've included a basic example of the two patterns I describe above.
use std::sync::Arc;
use std::thread::{self, JoinHandle};
use std::time::Duration;
fn main() {
let foo = Foo(0);
// pattern 1
let (foo, _res) = process(foo);
// pattern 2
let foo = Arc::new(foo);
let _res = process2(Arc::clone(&foo));
let foo = Arc::try_unwrap(foo).unwrap();
}
#[derive(Debug)]
struct Foo(u8);
impl Clone for Foo {
/// an expensive clone operation that we want to avoid when possible
fn clone(&self) -> Self {
thread::sleep(Duration::from_secs(10));
Foo(self.0)
}
}
/// complicated return type
fn process(foo: Foo) -> (Foo, u8) {
let foo = Arc::new(foo);
let mut threads = Vec::<JoinHandle<()>>::new();
for _ in 1..=5 {
let f = Arc::clone(&foo);
let t = thread::spawn(move || {
println!("{:?}", f);
});
threads.push(t);
}
threads.into_iter().for_each(|t| t.join().unwrap());
(Arc::try_unwrap(foo).unwrap(), 1)
}
/// forces the caller to do work that should be hidden
fn process2(foo: Arc<Foo>) -> u8 {
let mut threads = Vec::<JoinHandle<()>>::new();
for _ in 1..=5 {
let f = Arc::clone(&foo);
let t = thread::spawn(move || {
println!("{:?}", f);
});
threads.push(t);
}
threads.into_iter().for_each(|t| t.join().unwrap());
1
}
Playground
This is a good use for scoped threads (provided by crossbeam::scope or the higher-level utilities of rayon), which allow you to simply use an &Foo inside the threads. This avoids needing to move the value into and out of an Arc at all.
Also, if you use rayon you can greatly simplify your code by removing all of the explicit thread creation and joining in favor of its parallel iterators.
use rayon::iter::{ParallelIterator, IntoParallelIterator};
fn main() {
let foo = Foo(0);
let _res = process(&foo);
}
#[derive(Debug)]
struct Foo(u8);
fn process(foo: &Foo) -> u8 {
(1..=5).into_par_iter().for_each(|_i: usize| {
println!("{:?}", foo);
});
1
}
I'm in a not-so-great situation and have to fake a lifetime. It looks a little bit like this:
struct Bar<'a> {
cr: &'a mut char,
}
fn foo<D, F>(data: D, f: F)
where
D: 'static, // <-- !!!
F: FnOnce(D),
{ ... }
let mut c = '⚠';
let bar = Bar { cr: &mut c };
foo(???, |c| /* I need access to a `Bar` here! */);
I have to call the strange function foo. In the closure I pass to it, I need to get access to a Bar (with any lifetime) that was passed through foo. (I know in this minimal example I could just access the bar directly as closures have access to their environment, but let's pretend that's not possible here.) Unfortunately, foo requires D: 'static.
Of course, in reality everything is more complicated. I know Bar and foo don't make too much sense, but this is my attempt at breaking my problem into a minimal example.
How do I make this work? I believe it is possible to make this work safely (i.e. without undefined behavior), but I'm sure it requires the unsafe keyword.
The basic idea is to cast the Bar<'not_static> to a Bar<'static> temporarily, then make sure that it does not outlive the original c. I want to know how to best do that. My idea was the following:
let mut c = '⚠';
let bar = Bar { cr: &mut c };
let bar_static: Arc<Bar<'static>> = unsafe {
let extended = mem::transmute::<Bar<'_>, Bar<'static>>(bar);
Arc::new(extended)
};
foo(bar_static.clone(), |bar| println!("{}", bar.cr));
if Arc::strong_count(&bar_static) != 1 && Arc::weak_count(&bar_static) != 0 {
eprintln!("bad!");
std::process::abort();
}
The idea is to dynamically check that no references to c exist anymore (apart from the one we are holding) after calling foo. That should protect against foo storing the data in a static variable or something like that. I don't expect it, but I rather end the whole process instead of having memory unsafety in my program.
With this, I think, I make sure that the reference (which is incorrectly 'static) does not outlive the actual data. But:
Is that reasoning sane? Does it make sense?
What worries me is that rustc doesn't think c is borrowed after the unsafe block. I could (in my function) arbitrarily access c, although there exists a reference pointing to it. Is that a case of "I'm fine as long as I don't actually access c"? Or rather one of those "immediate UB" cases?
Is mem::transmute the right tool for the job or should I use pointer casts or something else?
Any better ideas?
The answer is that there is no safe way to alter the lifetime of your objects. Lifetimes are compiler guarantees that the value will always live for the duration of its use.
The safest route would be to change foo to include the lifetime of data: D.
fn foo<'a, D, F>(data: D, f: F)
where D: 'a,
F: FnOnce(D),
{
f(data);
}
An alternative is to alter Bar and how it's used via Arc<Mutex<char>> and downgrading it to Weak<Mutex<char>>.
struct Bar {
c: Weak<Mutex<char>>,
}
let t = 't';
let t = Arc::new(Mutex::new(t));
let bar = Bar { c: Arc::downgrade(&t) };
foo(bar, |b| {
if let Some(c) = b.c.upgrade() {
let mut c = c.lock().unwrap();
println!("{}", *c);
*c = 'b';
}
});
println!("{}", t.lock().unwrap());
However, if it's not possible for you to change foo or Bar and you are certain that the referenced object will not be dropped, then you can use std::mem::transmute to alter the lifetime of your object.
As mentioned in the doc:
transmute is incredibly unsafe. There are a vast number of ways to cause undefined behavior with this function. transmute should be the absolute last resort.
unsafe fn to_static<'a>(r: Bar<'a>) -> Bar<'static> {
std::mem::transmute::<Bar<'a>, Bar<'static>>(r)
}
let mut t = 't';
let bar = Bar { c: &mut t };
let static_bar = unsafe { to_static(bar) };
foo(static_bar, |b: Bar| {
println!("{}", b.c);
*b.c = 'b';
});
println!("{}", t);
The example above works because t lives within the scope where foo is called and we know t is static so it's perfectly fine to use. However, if foo sends static_bar into another thread, and t is not static, then the result is undefined behaviour.
If you can guarantee that foo will complete before the scope it's called in ends, then you can use Mutex's to do the following.
let t = 't';
let t = Arc::new(Mutex::new(t));
{ // create scope here
let mut tlock = t.lock().unwrap(); // MutexGuard<char>
let t_ref_mut = &mut *tlock; // get &mut char -- lifetime of mutex guard
let bar = Bar { c: t_ref_mut };
let static_bar = unsafe { to_static(bar) };
// foo must be guaranteed to complete before the scope ends.
foo(static_bar, |b| {
println!("{}", b.c);
*b.c = 'b';
});
} // MutexGuard goes out of scope and lock should be released
// sanity check that lock is released and value is altered
println!("{}", t.lock().unwrap());
Full source code can be viewed here