I have this macro:
macro_rules! set_vars {
( $($x:ident),* ) => {
let outer = 42;
$( let $x = outer; )*
}
}
Which expands this invocation:
set_vars!(x, y, z);
into what I expect (from --pretty=expanded):
let outer = 42;
let x = outer;
let y = outer;
let z = outer;
In the subsequent code I can print x, y, and z just fine, but outer seems to be undefined:
error[E0425]: cannot find value `outer` in this scope
--> src/main.rs:11:5
|
11 | outer;
| ^^^^^ not found in this scope
I can access the outer variable if I pass it as an explicit macro parameter.
Is this intentional, something to do with "macro hygiene"? If so, then it would probably make sense to mark such "internal" variables in --pretty=expanded in some special way?
Yes, this is macro hygiene. Identifiers declared within the macro are not available outside of the macro (and vice versa). Rust macros are not C macros (that is, Rust macros are more than glorified text replacement).
See also:
The Little Book of Rust Macros
A Practical Intro to Macros
So, what are hygienic macros anyway?
Related
Let's say I'm using a nested closure to modify a local variable like so:
let mut i = 0;
let x = (0..).flat_map(|_| {
(0..).map(|_| {
let x = i;
i += 1;
x
})
});
This does not compile with the error:
error: captured variable cannot escape `FnMut` closure body
--> src/main.rs:35:9
|
32 | let mut i = 0;
| ----- variable defined here
33 |
34 | let x = (0..).flat_map(|_| {
| - inferred to be a `FnMut` closure
35 | / (0..).map(|_| {
36 | | let x = i;
| | - variable captured here
37 | | i += 1;
38 | | x
39 | | })
| |__________^ returns a reference to a captured variable which escapes the closure body
|
= note: `FnMut` closures only have access to their captured variables while they are executing...
= note: ...therefore, they cannot allow references to captured variables to escape
I experimented with the same code on a non-nested context, and it compiles without an error:
let mut i = 0;
let x = (0..).map(|_| {
let x = i;
i += 1;
x
});
So I guess the error comes from the fact that the closure is nested, but I can't fully figure out why the error is being triggered.
Let's strip out all of the iterator parts and consider just the closures that are being constructed.
let mut i = 0;
let f = || {
|| {
let x = i;
i += 1;
x
}
};
i is a variable being used by the inner closure, and therefore borrowed by it. Now, think about what happens if I run this program:
let c1 = f();
let c2 = f();
[c1(), c2()]
Both c1 and c2 are closures that capture i and mutate it. Therefore, having both of them allows shared mutable state without any synchronization/mutual-exclusion mechanism, which is always prohibited in Rust. Compiling the above code will produce the same error as you had.
Iterator::flat_map() won't actually call the function and keep two of it around like this, but its signature can't communicate that fact to the compiler, so the compiler has to assume the worst could happen.
Now that we know that it would be unsound to permit this pattern, why is the error described the way you saw?
When you refer to a variable in a closure, this (most often) becomes a borrow of that variable. In other words, calling || { i += 1; } constructs a value that contains an &mut i32 pointing to i. This means that || { i += 1; } itself is an expression that needs to be able to mutably borrow i, so a closure which contains || { i += 1; } itself requires a mutable borrow of i.
Whenever a closure contains a mutable borrow (or some other cases), that means that invoking the closure itself requires &mut self (on the same general principle that you can't ever mutate an &mut by way of an &). That's why the closures are inferred to be FnMuts (the type of function that requires &mut self to invoke) and the compiler is telling you about that.
Now, why do FnMut closures “only have access to their captured variables while they are executing”? I'm not sure how to explain this exact description of the problem, but I'm sure that it's fundamentally about the requirement of &mut being unique, and that if it were permitted there would end up being some way to modify the function's state while its result was still borrowed.
Finally, how do you solve your problem? Well, in simple cases, the answer is usually to make the closure a move || { closure. That means there is no longer a borrow, and the closure now owns the i counter, so it can live as long as the closure does. But in your case, this doesn't help, because you're actually trying (I assume) to have a counter which is shared between everything. Therefore you need an interior mutability tool. The simplest one for this situation is Cell, which allows reading and writing a value (as long as it's Copy) without ever borrowing it.
use std::cell::Cell;
let i = Cell::new(0);
let x = (0..).flat_map(|_| {
(0..).map(|_| {
let x: i32 = i.get();
i.set(x + 1);
x
})
});
I'm working on a rust project written a couple of years ago, and have come across this piece of code, which is literally:
let mut values = vec![];
for x in maybe_values {
if let Some(x) = x {
values.push(Arc::new(x));
}
}
I understand that "if let" introduces a pattern-matching if (Which seems to be a poor re-use of the keyword "let", but I will get over that - If anyone can help me with a mental mnemonic to make sense of "let" here, please do!).
But what is the test Some(x) = x doing?
From my testing, it seems to be a trick/idiom to both a) test that the loop variant 'x' is Some(), and also b) end up with the unwrap()ped value in x.
But I can't fully explain it to myself, and can't find reference to this being an idiom anywhere.
Hope you can help my Rust education path. Thanks.
This is a shorthand for using a full match statement when you only care about matching a single use case.
So this block of code:
if let x = y {
foo();
} else {
bar();
}
Is equivalent to using a full match:
match y {
x => {
foo();
}
_ => {
bar();
}
}
For your specific case, it is equivalent to this. The inner x uses the same name as the outer variable which can be confusing, but they are two separate values.
let mut values = vec![];
for x in maybe_values {
match x {
Some(y) => values.push(Arc::new(y)),
_ => {},
}
}
There are two completely different variables in play here. It's equivalent to.
let mut values = vec![];
for x_1 in maybe_values {
if let Some(x_2) = x_1 {
values.push(Arc::new(x_2));
}
}
In Rust, the right-hand side of a let is evaluated with the left-hand variable not in scope, so when the if let is evaluated, the outer x is still in-scope. Then, if it's a Some value, we make a new variable x which contains the inside of the Option. This variable shadows the previous x, making it inaccessible inside the if statement (in the same way that a function argument called x would render a global variable named x inaccessible by shadowing).
I'm having this problem about generators:
use tokio::runtime::Runtime;
use tokio::task::JoinHandle;
use std::sync::Arc;
pub fn run(f: Box<dyn Fn() -> Result<(), ()> + Send>) {
f();
}
fn main() {
let x = Arc::new(0);
run(Box::new(move ||{
let rt = Runtime::new().unwrap();
let _t = rt.block_on(async move {
let y = x;
});
Ok(())
}));
}
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=de28ecf9e5baf6a017cd6a5230d74d7b
Error:
error[E0507]: cannot move out of `x`, a captured variable in an `Fn` closure
--> src/main.rs:13:41
|
10 | let x = Arc::new(0);
| - captured outer variable
...
13 | let _t = rt.block_on(async move {
| _________________________________________^
14 | | let y = x;
| | -
| | |
| | move occurs because `x` has type `Arc<i32>`, which does not implement the `Copy` trait
| | move occurs due to use in generator
15 | | });
| |_________^ move out of `x` occurs here
I don't understand why x is borrowed, if in both the closure and the block, I use move. So x should be moved to rt.block_on's closure. There should be no borrow at all.
Generators are an unstable feature, currently only available in nightly, which can be compared to generators or coroutines in other languages (e.g. Javascript, Go, Python).
Generators are essentially state machines that can pause execution using yield and later be resumed again, with the possibility of passing data in each transition. This pattern is ideal for asynchronous programming and the Rust compiler expands certain async code to use generators, even though you can't explicitly use them yourself without the nightly feature enabled.
It is possible that it is a bug that these messages are not properly feature-gated, or it may just have been too complicated to present different errors for code generated from async desugaring than for code you explicitly wrote yourself.
So let's ignore generators, which are a bit of a red herring for your actual problem.
You are moving x into a closure:
let x = Arc::new(0);
run(Box::new(move ||{
// 'move' closure uses x so x is moved
}));
And then the closure moves x again into an async block. The problem is that the signature of run accepts a Fn closure - a closure that cannot modify its environment but may be called multiple times. However the closure you have provided moves x into the async block the first time it's called, so it would be invalid to call it a second time.
Given you have said that you can't change run to accept an FnOnce, you need to make f callable multiple times by preventing it from moving x. You can do that by cloning it:
fn main() {
let x = Arc::new(0);
run(Box::new(move || {
let rt = Runtime::new().unwrap();
// each time the closure is called, it will pass a clone of the Arc
// into the task, so it can be called more than once
let x = x.clone();
let _t = rt.block_on(async move {
let y = x;
});
Ok(())
}));
}
The whole design of Arc is about cheap cloning. It's cheap because it only copies the pointer to your data and increments the reference count, which is how it knows when it is safe to free the memory of the holding its data. If you are never cloning an Arc then you almost certainly didn't need it in the first place!
The "move occurs due to use in generator" error seems to confuse things a bit here, but the overall issue isn't related to generators.
Your run function accepts Fn() functions, not FnOnce() functions. That means that the function passed to run must be able to execute multiple times. Given that, your code cannot work as written because running your function takes ownership of x when assigning it to y. If the first call to the function takes ownership of x, what is supposed to happen the second time the function runs? Ownership of x was already given to the future the first time it executed, and two things cant own the same value. The same issue would also happen with
run(Box::new(move ||{
let y = x;
Ok(())
}));
Since you have an Arc I'm guessing you had a sense of this already, but it won't happen without a bit more logic. Since you're using an Arc, you need to clone x at the start of the function, so that each time the function runs, it gets its own Arc object to handle and manipulate.
fn main() {
let x = Arc::new(0);
run(Box::new(move ||{
// Make a new Arc that can be given to the promise without needing
// to worry about what future calls to this callback willl do.
let x = Arc::clone(&x);
let rt = Runtime::new().unwrap();
let _t = rt.block_on(async move {
let y = x;
});
Ok(())
}));
}
In the documentation for std::iter::Iterator::filter() it explains that values are passed to the closure by reference, and since many iterators produce references, in that case the values passed are references to references. It offers some advice to improve ergonomics, by using a &x pattern to remove one level of indirection, or a &&x pattern to remove two levels of indirection.
However, I've found that this second pattern does not compile if the item being iterated does not implement Copy:
#[derive(PartialEq)]
struct Foo(i32);
fn main() {
let a = [Foo(0), Foo(1), Foo(2)];
// This works
let _ = a.iter().filter(|&x| *x != Foo(1));
// This also works
let _ = a.iter().filter(|&x| x != &Foo(1));
// This does not compile
let _ = a.iter().filter(|&&x| x != Foo(1));
}
The error you get is:
error[E0507]: cannot move out of a shared reference
--> src/main.rs:14:30
|
14 | let _ = a.iter().filter(|&&x| x != Foo(1));
| ^^-
| | |
| | data moved here
| | move occurs because `x` has type `Foo`, which does not implement the `Copy` trait
| help: consider removing the `&`: `&x`
Does this mean that if I use the &&x destructuring pattern, and the value is Copy, Rust will silently copy every value I am iterating over? If so, why does that happen?
In Rust, function or closure arguments are irrefutable patterns.
In the Rust reference, it says:
By default, identifier patterns bind a variable to a copy of or move
from the matched value depending on whether the matched value
implements Copy.
So, in this case:
let _ = a.iter().filter(|&x| *x != Foo(1));
the closure is being passed a reference to a reference to the item being iterated; therefore x is bound to a copy of a reference to the item. You can always copy a reference (it is basically a no-op) so this always succeeds.
In this case:
let _ = a.iter().filter(|&&x| x != Foo(1));
x is being bound to a copy of the item itself - which fails if the item is not Copy.
The reference also says:
This can be changed to bind to a reference by using the ref keyword,
or to a mutable reference using ref mut.
That's not so useful in this case though: &&ref x results in x being a reference to the item, the same as if you had used &x.
Does this mean that if I use the &&x destructuring pattern, and the value is Copy, Rust will silently copy every value I am iterating over?
Yes, that's correct, although the compiler backend might optimize the copying away.
If so, why does that happen?
Writing & before a pattern dereferences it. This copies the value it refers to, because a reference only borrows its value, so it can't move the value.
Example:
#[derive(Copy, Clone)]
struct Foo(i32);
let foo: Foo = Foo(5); // value is owned by foo
let borrow: &Foo = &foo; // value is borrowed
let bar: Foo = *borrow; // value is copied, the copy is owned by bar
// the original value is still owned by foo
Your first example is a bit special:
let _ = a.iter().filter(|&x| *x != Foo(1));
This seems at first as if it should require the Copy trait because x is dereferenced twice, first in the pattern, and then in the comparison.
However, the comparison is done by the PartialEq trait, which takes its arguments by reference. So the above is desugared to:
let _ = a.iter().filter(|&x| PartialEq::eq(&*x, &Foo(1)));
Which works because &* cancel each other out. Hence, x is only dereferenced once (in the pattern).
I am playing around with Rust references:
fn main() {
let str = String::from("Hallo");
let &x = &str;
}
This produces the following error:
error[E0507]: cannot move out of borrowed content
--> src/main.rs:3:9
|
3 | let &x = &str;
| ^-
| ||
| |hint: to prevent move, use `ref x` or `ref mut x`
| cannot move out of borrowed content
What is going on here?
Adding to wiomoc's answer: depending on what language(s) you've previously known, variable declaration in Rust might be a little different. Whereas in C/C++ you explicitly have to declare that you want a pointer/reference variable:
int *p = &other_int;
In Rust it's enough to just use let, so the above would in Rust be
let p = &other_int;
and when you write
let &s = &string;
It pattern-matches that, so the Rust compiler reads it roughly as "I know I have a reference, and I want to bind whatever it is referring to to the name p". If you're not familiar with pattern-matching, a more obvious example (that works in Rust as well) would be
let point = (23, 42);
let (x, y) = point;
The second line unpacks the right-hand side to match the left-hand side (both are tuples of two values) and binds the variable names on the left to the values at the same position in the structure on the right. In the case above, it's less obvious that it's matching your "structural description".
The result of let &x = &str;, i.e. "I know &str is a reference, please bind whatever it refers to to the variable x" means that you're trying to have x be the same as str, when at that line all you have is a borrowed reference to str. That's why the compiler tells you you can't create an owned value (which x would be, because it's not being created as a reference) from a reference.
You dont need that &at let x
let str = String::from("Hallo");
let x = &str;
Or if you want to declare the type manually
let string = String::from("Hallo");
let x: &str = &string;