Coming from this code template which works:
{
fn f3( _s : &String) {}
fn f( s : &String) -> impl FnMut() {
let s2 = s.clone();
move || f3( &s2)
}
let mut f2 = f( &"123".to_string());
f2();
}
If I modify the code this way:
{
fn f3( _s : &[u8]) {}
fn f( s : &[u8]) -> impl FnMut() {
// let s2 = s.clone(); // don't work
let s2 = Vec::from(s);
move || f3( &s2[..])
}
let mut f2 = f( &vec![1u8][..]);
f2();
}
I cannot use 'let s2 = s.clone();'. This brings the error message:
1169 | fn f( s : &[u8]) -> impl FnMut() {
| ------------ this return type evaluates to the `'static` lifetime...
1170 | let s2 = s.clone();
| ^^^^^ ...but this borrow...
|
note: ...can't outlive the anonymous lifetime #1 defined on the function body at 1169:3
How can clone initiate a borrow?
In your first example, s is a &String, and String implements Clone, so the clone(&self) method is used.
In your second example, s is a &[u8], and [u8] does not implement Clone. So instead you are using the blanket implementation for &T where T is any type; that is, you are cloning the reference, not the thing being referenced. The result is another reference to the same thing, hence it is still a borrow.
The solution in this case is to use a different method than .clone() to create an owned copy of the slice. As you've noticed, Vec::from works, and gives you a Vec<u8>; you can also use Box::from to get a Box<[u8]>. You can't (currently) get a raw [u8] as a local variable, since that is an unsized type, so the use of ::from here is because you need to choose what other type your owned copy will be.
Related
I want to have two types (in my question-fitted examples I will mark them as IdHandler counting IDs, and IdUser using the IDs), where the dependent one of them should not outlive (hence, lifetimes) the dependency, and that's the only requirement. Figured the quickest solution would be using PhantomData<&'...>:
struct IdHandler {
id_counter: u64,
}
struct IdUser<'a> {
_a: PhantomData<&'a IdHandler>,
id: u64,
}
Then, using a mutable reference with a specified lifetime, I made a function in impl IdHandler to create the users:
fn new_id_user<'a>(self: &'a mut IdHandler) -> IdUser<'a> {
let res = IdUser {
_a: PhantomData,
id: self.id_counter,
};
self.id_counter += 1;
res
}
Now, I will make a function that takes in a mutable reference and tries to make a pair of IdUsers:
fn f<'a>(id_handler: &'a mut IdHandler) -> (IdUser<'a>, IdUser<'a>) {
let a1 = id_handler.new_id_user();
let a2 = id_handler.new_id_user();
(a1, a2)
}
And here's an example of how those types and functions may be used:
fn main() {
let mut id_handler = IdHandler { id_counter: 0u64 };
let (a1, a2) = f(&mut id_handler);
assert_eq!(a1.id, 0u64);
assert_eq!(a2.id, 1u64);
assert_eq!(id_handler.id_counter, 2u64);
id_handler.stop(); // fn stop(self) {}
// Ideally here 'a is dead,
// so you can't perform operations on IdUser<'a>
// assert_eq!(a2.id, 1u64);
}
Given the fact that new_id_user(...) only uses the mutability for a moment, and then hopefully "releases" it back (as it is not used any longer than the function call - I do not know the exact rule Rust uses, but it works without _a/'a or when fn f(...) -> ()), I'd expect f(...) to work just fine. However, the compiler has greeted me with following:
error[E0499]: cannot borrow `*id_handler` as mutable more than once at a time
--> src\main.rs:29:14
|
27 | fn f<'a>(id_handler: &'a mut IdHandler) -> (IdUser<'a>, IdUser<'a>) {
| -- lifetime `'a` defined here
28 | let a1 = id_handler.new_id_user();
| ------------------------ first mutable borrow occurs here
29 | let a2 = id_handler.new_id_user();
| ^^^^^^^^^^^^^^^^^^^^^^^^ second mutable borrow occurs here
30 | (a1, a2)
| -------- returning this value requires that `*id_handler` is borrowed for `'a`
Does anyone know whether there is a solution that I'm missing, or am I doomed to allow IdUsers to outlive the IdHandler?
Unfortunately you cannot. Preserving a value with a lifetime will prevent the mutating the value from which it was derived even if you don't actually keep a reference to it. A common workaround if you want to ensure the lifetimes are bound while still being able to mutate is to use interior mutability (via Cell, RefCell, Mutex, etc).
Here's a working example using Cell:
use std::marker::PhantomData;
use std::cell::Cell;
struct IdHandler {
id_counter: Cell<u64>,
}
impl IdHandler {
fn new_id_user<'a>(self: &'a IdHandler) -> IdUser<'a> {
let res = IdUser {
_a: PhantomData,
id: self.id_counter.get(),
};
self.id_counter.set(self.id_counter.get() + 1);
res
}
}
struct IdUser<'a> {
_a: PhantomData<&'a IdHandler>,
id: u64,
}
fn f<'a>(id_handler: &'a IdHandler) -> (IdUser<'a>, IdUser<'a>) {
let a1 = id_handler.new_id_user();
let a2 = id_handler.new_id_user();
(a1, a2)
}
fn main() {
let mut id_handler = IdHandler { id_counter: Cell::new(0u64) };
let (a1, a2) = f(&mut id_handler);
assert_eq!(a1.id, 0u64);
assert_eq!(a2.id, 1u64);
}
I'm trying to share an Arc<Mutex<T>> with a closure. I get the error that closure is FnOncebecause it moves the variableshared_wake_deque_ out of its environment:
use core::task::{Context, Poll, Waker};
use hyper::client::connect::{Connection, Connected};
use std::sync::{Arc, Mutex};
use std::collections::VecDeque;
pub type OnPollRead = Arc<dyn Fn(&mut Context<'_>, &mut tokio::io::ReadBuf<'_>) -> Poll<std::io::Result<()>> + Send + Sync>;
pub type OnPollWrite = Arc<dyn Fn(&mut Context<'_>, &[u8]) -> Poll<core::result::Result<usize, std::io::Error>> + Send + Sync>;
#[derive(Clone)]
pub struct CustomTransporter {
on_poll_read: Option<OnPollRead>,
on_poll_write: Option<OnPollWrite>,
}
pub struct OVPNClient {
}
impl OVPNClient {
pub fn send(&self, buffer: &[u8]) {
}
}
unsafe impl Send for OVPNClient {}
unsafe impl Send for CustomTransporter {}
impl CustomTransporter {
pub fn new(on_poll_read: Option<OnPollRead>, on_poll_write: Option<OnPollWrite>) -> CustomTransporter {
CustomTransporter{
on_poll_read: on_poll_read,
on_poll_write: on_poll_write
}
}
}
fn main () {
let openvpn_client = Arc::new(Mutex::new(OVPNClient{}));
let shared_wake_deque = Arc::new(Mutex::new(VecDeque::<Waker>::new()));
let shared_wake_deque_ = shared_wake_deque.clone();
let on_poll_read = Arc::new(|context: &mut Context, buffer: &mut tokio::io::ReadBuf| -> Poll<std::io::Result<()>> {
let shared_wake_deque__ = shared_wake_deque_;
Poll::Ready(Ok(()))
});
let on_poll_write = Arc::new(|context: &mut Context, buffer: &[u8]| -> Poll<core::result::Result<usize, std::io::Error>>{
openvpn_client.lock().unwrap().send(buffer);
Poll::Ready(Ok(0))
});
let connector = CustomTransporter::new(Some(on_poll_read), Some(on_poll_write));
}
Error:
error[E0525]: expected a closure that implements the `Fn` trait, but this closure only implements `FnOnce`
--> src/main.rs:44:33
|
44 | let on_poll_read = Arc::new(|context: &mut Context, buffer: &mut tokio::io::ReadBuf| -> Poll<std::io::Result<()>> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ this closure implements `FnOnce`, not `Fn`
45 | let shared_wake_deque__ = shared_wake_deque_;
| ------------------ closure is `FnOnce` because it moves the variable `shared_wake_deque_` out of its environment
...
54 | let connector = CustomTransporter::new(Some(on_poll_read), Some(on_poll_write));
| ------------ the requirement to implement `Fn` derives from here
I get that moving can only occur one time. I didn't even made the closure move. As you see, I'm trying to use a clone of the Arc<Mutex<T>> so I'm not exactly moving it. You can see that I tried cloning so I move a variable that is used just on this closure and never anymore, but it didn't work.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=d996163f5d2a6e9322af9a7472581ce6
Because you have
let shared_wake_deque__ = shared_wake_deque_;
as a simple assignment, it is no different from
let foo = vec![];
let foo2 = foo;
where running code transfers ownership of the vec from foo to foo2. Because you have this in a lambda, running the explicitly transfers ownership of the shared_wake_deque_ value to shared_wake_deque__, meaning that shared_wake_deque_ can't be used any more. For that to be possible, it is required that on_poll_read can only run a single time, because otherwise multiple things would be trying to own the same object (both on_poll_read, since it would need to own it in order to use in some future call, and the shared_wake_deque__ from any previous call would also still have to own it). Since that would be a violation of Rust's safety guarantees because something can only be owned by one thing, you are getting this error.
Most likely, what you want is to us a reference, e.g.
let shared_wake_deque__ = &shared_wake_deque_;
By using a reference to shared_wake_deque_, the closure is allowed to run multiple times and give out multiple shared references. With only that alone however, you'll then get
error[E0597]: `shared_wake_deque_` does not live long enough
because you're trying to reference a variable that is declared outside the closure, and the closure could run after shared_wake_deque_ was dropped. To solve that, you need the closure to take ownership of shared_wake_deque_, which is accomplished by using the move keyword, e.g.
let on_poll_read = Arc::new(move |context: &mut Context, buffer: &mut tokio::io::ReadBuf| -> Poll<std::io::Result<()>> {
let shared_wake_deque__ = &shared_wake_deque_;
then the same problem also arises for on_poll_write so that needs to be move too:
let on_poll_write = Arc::new(move |context: &mut Context, buffer: &[u8]| -> Poll<core::result::Result<usize, std::io::Error>>{
(Rust playground)
I'm trying to have a function shared between threads, that can be called with another function on it:
fn main() {
let on_virtual_tun_write = std::sync::Arc::new(|f: &dyn FnOnce(&mut [u8]), size: usize|-> std::result::Result<(),()>{
let mut buffer = vec![0; size];
f(buffer.as_mut_slice());
Ok(())
});
}
Once it's called, it uses the f passed as an argument, to retrieve a buffer. As you can see, I do not intend to copy the function, I just use it as soon as someone calls the closure, and discards it. However, Rust thinks that I want to copy it. What can I do to tell Rust that I just want to use a reference to the function? I thought &dyn was sufficient.
Error:
error[E0161]: cannot move a value of type dyn for<'r> FnOnce(&'r mut [u8]): the size of dyn for<'r> FnOnce(&'r mut [u8]) cannot be statically determined
--> src/main.rs:4:9
|
4 | f(buffer.as_mut_slice());
| ^
error[E0507]: cannot move out of `*f` which is behind a shared reference
--> src/main.rs:4:9
|
4 | f(buffer.as_mut_slice());
| ^ move occurs because `*f` has type `dyn for<'r> FnOnce(&'r mut [u8])`, which does not implement the `Copy` trait
error: aborting due to 2 previous errors; 1 warning emitted
So I think there's a few issues that are coming together to cause you this problem.
I'm going to use a reduced example which generates essentially the same error message.
fn main() {
let ff = |f: &dyn FnOnce()| { f(); };
}
The core of the problem is that rust is stopping bad code where the FnOnce is used more than once. Like this (which doesn't compile):
fn main() {
let f = || { }; // Imagine this is a real FnOnce()
let ff = |f: &dyn FnOnce()| { f(); };
ff(&f); // Use the FnOnce in our wrapper
f(); // Use the FnOnce again - should fail to compile.
}
Firstly FnOnce() when called consumes its self - this is how it ensures it can only be called once. Which means you need to be passing the object, not a reference to it - otherwise the caller can still be holding a reference. This is not the case for Fn() and FnMut(). The first uses only a reference to self, and the second a mutable reference to self. This means the following compile OK:
fn main() {
let ff = |f: &dyn Fn()| { f(); };
let f = || {};
ff(f); // First call is OK
f(); // Second usage is OK - it's a pure function
let ff = |f: &mut dyn FnMut()| { f(); };
let mut i = 0;
let mut f = || { i += 1 };
ff(&mut f); // OK i=1
f(); // OK i=2
println!("i={}", i); // prints 2
}
So if you can reduce the restrictions on your function type so that it is either Fn or FnMut, that should solve it for you, if not ... read on.
You can pass an instance (not a reference) of a FnOnce to a generic function that uses impl. e.g.
fn ff(f: impl FnOnce()) {
f()
}
fn main() {
// Creating a closure that moves q into it ensures we get a FnOnce
let q=vec![1,2,3];
let f = move || {println!("{}", q.len())};
ff(f); // OK.
// f(); // Compile error with "use of moved value: `f`"
}
this is equivalent to
fn<F> ff(f: F)
where F: FnOnce()
{
f()
}
However, we can't mix generics and closures in a way that can help us. Trying to use |f: impl FnOnce()| generates the error : error[E0562]: impl Trait not allowed outside of function and inherent method return types.
The only way I think you can use both the FnOnce and closure is to box up the function and pass it in. That will transfer ownership of the function into the closure and so we call it. For example the following compiles:
fn main() {
let ff = |f: Box<dyn FnOnce()>| { f() };
let q=vec![1,2,3];
let f = Box::new(move || {println!("{}", q.len())});
ff(f); // OK
// f(); // Errors with "use of moved value: `f`"
}
It is possible to coerce &mut T into &T but it doesn't work if the type mismatch happens within a type constructor.
playground
use ndarray::*; // 0.13.0
fn print(a: &ArrayView1<i32>) {
println!("{:?}", a);
}
pub fn test() {
let mut x = array![1i32, 2, 3];
print(&x.view_mut());
}
For the above code I get following error:
|
9 | print(&x.view_mut());
| ^^^^^^^^^^^^^ types differ in mutability
|
= note: expected reference `&ndarray::ArrayBase<ndarray::ViewRepr<&i32>, ndarray::dimension::dim::Dim<[usize; 1]>>`
found reference `&ndarray::ArrayBase<ndarray::ViewRepr<&mut i32>, ndarray::dimension::dim::Dim<[usize; 1]>>`
It is safe to coerce &mut i32 to &i32 so why it is not applied in this situation? Could you provide some examples on how could it possibly backfire?
In general, it's not safe to coerce Type<&mut T> into Type<&T>.
For example, consider this wrapper type, which is implemented without any unsafe code and is therefore sound:
#[derive(Copy, Clone)]
struct Wrapper<T>(T);
impl<T: Deref> Deref for Wrapper<T> {
type Target = T::Target;
fn deref(&self) -> &T::Target { &self.0 }
}
impl<T: DerefMut> DerefMut for Wrapper<T> {
fn deref_mut(&mut self) -> &mut T::Target { &mut self.0 }
}
This type has the property that &Wrapper<&T> automatically dereferences to &T, and &mut Wrapper<&mut T> automatically dereferences to &mut T. In addition, Wrapper<T> is copyable if T is.
Assume that there exists a function that can take a &Wrapper<&mut T> and coerce it into a &Wrapper<&T>:
fn downgrade_wrapper_ref<'a, 'b, T: ?Sized>(w: &'a Wrapper<&'b mut T>) -> &'a Wrapper<&'b T> {
unsafe {
// the internals of this function is not important
}
}
By using this function, it is possible to get a mutable and immutable reference to the same value at the same time:
fn main() {
let mut value: i32 = 0;
let mut x: Wrapper<&mut i32> = Wrapper(&mut value);
let x_ref: &Wrapper<&mut i32> = &x;
let y_ref: &Wrapper<&i32> = downgrade_wrapper_ref(x_ref);
let y: Wrapper<&i32> = *y_ref;
let a: &mut i32 = &mut *x;
let b: &i32 = &*y;
// these two lines will print the same addresses
// meaning the references point to the same value!
println!("a = {:p}", a as &mut i32); // "a = 0x7ffe56ca6ba4"
println!("b = {:p}", b as &i32); // "b = 0x7ffe56ca6ba4"
}
Full playground example
This is not allowed in Rust, leads to undefined behavior and means that the function downgrade_wrapper_ref is unsound in this case. There may be other specific cases where you, as the programmer, can guarantee that this won't happen, but it still requires you to implement it specifically for those case, using unsafe code, to ensure that you take the responsibility of making those guarantees.
Consider this check for an empty string that relies on content staying unchanged for the runtime of the is_empty function (for illustration purposes only, don't use this in production code):
struct Container<T> {
content: T
}
impl<T> Container<T> {
fn new(content: T) -> Self
{
Self { content }
}
}
impl<'a> Container<&'a String> {
fn is_empty(&self, s: &str) -> bool
{
let str = format!("{}{}", self.content, s);
&str == s
}
}
fn main() {
let mut foo : String = "foo".to_owned();
let container : Container<&mut String> = Container::new(&mut foo);
std::thread::spawn(|| {
container.content.replace_range(1..2, "");
});
println!("an empty str is actually empty: {}", container.is_empty(""))
}
(Playground)
This code does not compile since &mut String does not coerce into &String. If it did, however, it would be possible that the newly created thread changed the content after the format! call but before the equal comparison in the is_empty function, thereby invalidating the assumption that the container's content was immutable, which is required for the empty check.
It seems type coercions don't apply to array elements when array is the function parameter type.
playground
I'm following the sine example in the rust-portaudio examples directory (which uses the non-blocking API), and I'm trying to get run() working with a Trait type argument instead of calculating the samples within run() itself.
The trait I've defined is pretty simple:
pub trait Evaluatable {
fn evaluate(&mut self) -> (f32, f32);
}
I changed the run() function signature to the following to accept my trait:
fn run<E: Evaluatable + 'static>(mut generator: E) -> Result<(), pa::Error>
and upated the callback function to call evaluate() on the trait type instead of generating samples itself:
let callback = move |pa::OutputStreamCallbackArgs { buffer, frames, .. }| {
let mut idx = 0;
for _ in 0..frames {
let samples = generator.evaluate();
buffer[idx] = samples.0;
buffer[idx+1] = samples.1;
idx += 2;
}
pa::Continue
};
E has to be 'static due to this callback (see open_non_blocking_stream()), which is the main source of my frustration...
In main, I can create an Evaluatable type and pass it in just fine (aside: I'm surprised this works, do objects in main get 'static lifetime?):
fn main() {
// Implements Evaluatable
// arguments are: sample rate, frequency, amplitude
let mut sine_generator = SineGenerator::new(44100.0, 200.0, 0.3);
run(sine_generator).unwrap()
}
But I want to be able to combine signals, so I made a struct that can combine them:
use evaluatable::Evaluatable;
pub struct Combine<'a> {
first: &'a mut (Evaluatable + 'a),
second: &'a mut (Evaluatable + 'a),
}
impl<'a> Combine<'a> {
pub fn new(first: &'a mut Evaluatable, second: &'a mut Evaluatable) -> Combine<'a> {
Combine {first, second}
}
}
impl<'a> Evaluatable for Combine<'a> {
fn evaluate(&mut self) -> (f32, f32) {
let first_output = self.first.evaluate();
let second_output = self.second.evaluate();
(first_output.0 + second_output.0, first_output.1 + second_output.1)
}
}
And tried to use it in main():
fn main() {
let mut sine_generator1 = SineGenerator::new(44100.0, 200.0, 0.3);
let mut sine_generator2 = SineGenerator::new(44100.0, 250.0, 0.3);
let combine = Combine::new(&mut sine_generator1, &mut sine_generator2);
run(combine).unwrap()
}
Now, Rust seems to have a problem with the lifetimes not being 'static:
Error[E0597]: `sine_generator1` does not live long enough
--> src/main.rs:27:37
|
27 | let combine = Combine::new(&mut sine_generator1, &mut sine_generator2);
| ^^^^^^^^^^^^^^^ does not live long enough
28 | run(combine).unwrap()
29 | }
| - borrowed value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
Why was Rust able to allow me to use SineGenerator from main(), but won't let me use Combine which takes the same objects (with, I assume, the same lifetimes)?
Is there a better way to implement Combine that will allow me to do what I want to do here? I had to take references because Trait types don't have defined size at compile-time.
Why was Rust able to allow me to use SineGenerator from main(), but won't let me use Combine which takes the same objects (with, I assume, the same lifetimes)?
A 'static bound on a type parameter means that the type may not borrow anything that lives shorter than 'static. Your SineGenerator doesn't borrow anything, so it respects that bound. Combine, on the other hand, doesn't respect the bound because it contains borrowed pointers, and you instantiate it with a lifetime that is shorter than 'static by storing references to local variables in the Combine object.
Is there a better way to implement Combine that will allow me to do what I want to do here? I had to take references because Trait types don't have defined size at compile-time.
The typical solution is to use type parameters instead of trait objects.
pub struct Combine<F, S>
where
F: Evaluatable,
S: Evaluatable,
{
first: F,
second: S,
}
fn main() {
let sine_generator1 = SineGenerator::new(44100.0, 200.0, 0.3);
let sine_generator2 = SineGenerator::new(44100.0, 250.0, 0.3);
let combine = Combine::new(sine_generator1, sine_generator2);
run(combine).unwrap()
}
By having Combine own the Evaluatables, it will respect the 'static bound (if both F and S are 'static themselves).