I have the following scenario where I am trying to add futures to peers which directly resolve to A inside the function test:
#![allow(dead_code)]
use core::pin::Pin;
use std::rc::Rc;
use core::fmt::Debug;
use tokio::io::AsyncRead;
use tokio::io::AsyncWrite;
use core::cell::RefCell;
use futures::stream::FuturesUnordered;
#[derive(Debug)]
pub struct State {
foo: String
}
pub async fn test<A: AsyncWrite + AsyncRead + Unpin + Debug + 'static>(
peers: &mut FuturesUnordered<Pin<Box<dyn futures::Future<Output = A>>>>,
) {
let pp: Vec<Rc<RefCell<A>>> = Vec::new();
for p in pp {
let p = Rc::try_unwrap(p).unwrap();
let p: A = p.into_inner();
peers.push(Box::pin(async move {
p
}));
}
}
rust playground
But I want to remove the 'static lifetime requirement for A since I want to dispose of it once a future in peers has been awaited.
If I remove the generic type parameter and replace it with u64 I have the following:
#![allow(dead_code)]
use core::pin::Pin;
use std::rc::Rc;
use core::fmt::Debug;
use tokio::io::AsyncRead;
use tokio::io::AsyncWrite;
use core::cell::RefCell;
use futures::stream::FuturesUnordered;
#[derive(Debug)]
pub struct State {
foo: String
}
pub async fn test(
peers: &mut FuturesUnordered<Pin<Box<dyn futures::Future<Output = u64>>>>,
) {
let pp: Vec<Rc<RefCell<u64>>> = Vec::new();
for p in pp {
let p = Rc::try_unwrap(p).unwrap();
let p: u64 = p.into_inner();
peers.push(Box::pin(async move {
p
}));
}
}
which works fine.
Why do I need to add the 'static lifetime for it to work with generics? Is there a better solution to my problem here?
'static here does not mean that the lifetime of A is the whole program but that A does not contain any references that are shorter than 'static or in other words A is a completely owned struct. It doesn't prevent A from being freed.
Related
use std::cell::RefCell;
use std::rc::Rc;
use wasm_bindgen::prelude::*;
use wasm_bindgen::JsCast;
fn main() {
let closure_cell = Rc::new(RefCell::new(None));
let closure_cell_clone = Rc::clone(&closure_cell);
*closure_cell.borrow_mut() = Some(Closure::wrap(Box::new(move || {
request_animation_frame(closure_cell_clone.borrow().as_ref().unwrap());
})));
}
fn request_animation_frame(f: &Closure<dyn FnMut()>) {
}
In the above code request_animation_frame expects &Closure<dyn FnMut()>and I pass a closure which implements FnMnut. But I am getting an error
expected type `std::option::Option<wasm_bindgen::closure::Closure<dyn std::ops::FnMut()>>`
found type `std::option::Option<wasm_bindgen::closure::Closure<[closure#src/lib.rs:29:62: 39:6 bucket:_, world:_, closure_cell_clone:_]>>`
Arent FnMut default implemented if a closure can implement that trait? Cant a closure which implements FnMut be sent in place of dyn FnMut ? What am I missing here?
This is actually a pretty sneaky problem. Here's why:
While you can coerce a Box<T> to a Box<dyn Trait> if T: Trait, those two are incompatible types, in that you cannot use a variable or field of the type Box<T> as a Box<dyn Trait>.
Here's an example to show why that could lead to undefined behaviour, using Cell<...>s (where that box would be a field somewhere in Cell<...>)
use std::cell::Cell;
trait SomeTrait { }
#[derive(Clone, Copy)]
pub struct A;
impl SomeTrait for A { }
#[derive(Clone, Copy)]
pub struct B;
impl SomeTrait for B { }
fn use_coerced(cell: &Cell<Box<dyn SomeTrait>>) {
cell.set(Box::new(B)); // valid, since `B` implements `SomeTrait`
}
fn main() {
// I'll put what the type inference will come up with as explicit types here
let some_cell: Cell<Box<A>> = Cell::new(Box::new(A));
use_coerced(&some_cell); // If cells would coerce, this would be valid
let value: Box<A> = some_cell.into_inner(); // But we put `Box<B>` in there. So we'd have undefined behaviour.
}
So to solve that problem, you have to tell the compiler to use the dyn FnMut() type instead of the closure type somewhere. I personally prefer explicitely unsizing it after creating the Box:
fn main() {
let closure_cell = Rc::new(RefCell::new(None));
let closure_cell_clone = Rc::clone(&closure_cell);
*closure_cell.borrow_mut() = Some(Closure::wrap(Box::new(move || {
request_animation_frame(closure_cell_clone.borrow().as_ref().unwrap());
}) as Box<dyn FnMut()>));
}
fn request_animation_frame(f: &Closure<dyn FnMut()>) { }
You can, however, also specify the type of the cell:
fn main() {
let closure_cell: Rc<RefCell<Option<Closure<dyn FnMut()>>>> = Rc::new(RefCell::new(None));
let closure_cell_clone = Rc::clone(&closure_cell);
*closure_cell.borrow_mut() = Some(Closure::wrap(Box::new(move || {
request_animation_frame(closure_cell_clone.borrow().as_ref().unwrap());
})));
}
fn request_animation_frame(f: &Closure<dyn FnMut()>) { }
But that is, as you can see, way more wordy, and unnecessarily so.
The tokio::fs::File::open(path: T + 'static) requires a 'static lifetime on its path parameter.
This makes sense because it is handled in runtime threads during the program's execution. I think it would make more sense if you could pass your own lifetimes, because the runtime does not need to run the whole time and so you could throw away some stuff. Do I understand something wrong?
I'd like to stay for 'static at the moment and so my problem is this...
I have a trait TraitN and some struct StructX { path: String, } with a fn new(path: &String) -> Box<TraitN>. The new creates and sets self.path = path.to_string();.
In some impl fn doit(&self) { ... } for StructX, I'd like to call tokio::fs::File::open(&self.path).
How can I pass &self.path with a 'static lifetime?
This is a complete example:
extern crate futures;
extern crate tokio;
#[macro_use]
extern crate error_chain;
use futures::future;
use futures::future::{loop_fn, ok, Future, Loop};
use futures::Stream;
use std::io::BufReader;
use tokio::{fs, io};
mod error {
error_chain!{}
}
use error::*;
type FutureResult<T> = future::FutureResult<T, Error>;
trait HandlerTrait {
fn new(path: &str) -> Box<HandlerTrait>
where
Self: Sized;
fn get_all(&self) -> FutureResult<Vec<String>>;
}
#[derive(Debug)]
pub struct Handler {
path: String,
}
impl HandlerTrait for Handler {
fn new(path: &str) -> Box<HandlerTrait> {
Box::new(Handler {
path: path.to_string(),
})
}
fn get_all(&self) -> FutureResult<Vec<String>> {
let file = fs::File::open(self.path.clone())
.and_then(|file: fs::File| ok(file))
.wait()
.unwrap();
let lines = io::lines(BufReader::new(file));
ok(lines
.filter(|line| line.len() > 80)
.map(|all| all[0..80].to_string())
.collect()
.wait()
.unwrap())
}
}
fn get_handler(path: &str) -> Option<Box<HandlerTrait>> {
Some(Handler::new(path))
}
fn get_path() -> FutureResult<String> {
ok("./somepath/file".to_string())
}
fn start_runtime() -> Result<()> {
let path: &str = get_path().wait().unwrap().as_str();
tokio::run(doit(path.clone()));
Ok(())
}
fn doit(path: &'static str) -> impl Future<Item = (), Error = ()> + 'static {
let n = 0;
loop_fn(n, move |_nr| {
let lh = get_handler(path).unwrap();
lh.get_all()
.or_else(|_| Err(()))
.and_then(|_all| ok(Loop::Break(())))
})
}
#[test]
fn test() {
start_runtime().unwrap();
assert!(true);
}
error[E0597]: borrowed value does not live long enough
--> src/lib.rs:63:22
|
63 | let path: &str = get_path().wait().unwrap().as_str();
| ^^^^^^^^^^^^^^^^^^^^^^^^^^ - temporary value only lives until here
| |
| temporary value does not live long enough
|
= note: borrowed value must be valid for the static lifetime...
playground
TL;DR Use a String instead of a &str. This might change when async / await syntax is stabilized.
Here's the MCVE I made of your original question:
extern crate tokio; // 0.1.11
trait TraitN {}
struct StructX {
path: String,
}
impl TraitN for StructX {}
fn new(path: &str) -> Box<TraitN> {
Box::new(StructX {
path: path.to_string(),
})
}
impl StructX {
fn doit(&self) {
tokio::fs::File::open(self.path.clone());
}
}
To solve this, clone the String and give ownership of it to the function:
impl StructX {
fn doit(&self) {
tokio::fs::File::open(self.path.clone());
}
}
With your example code, there are numerous problems:
fn start_runtime() -> Result<()> {
let path: &str = get_path().wait().unwrap().as_str();
tokio::run(doit(path.clone()));
Ok(())
}
You cannot take a reference to the result of unwrap because nothing will own that value. You cannot have a reference to this kind of temporary.
Cloning a &'a str returns a &'a str, not a String.
It doesn't make sense to call wait on the value because that blocks the thread. Run everything in the reactor loop.
This function should look like
fn start_runtime() -> Result<()> {
tokio::run({
get_path()
.map_err(|e| panic!("{}", e))
.and_then(|path| doit(path))
});
Ok(())
}
Then all of your code should switch to impl Into<String> instead of &str of &'static str. doit also needs to be able to create duplicate Strings:
fn doit(path: impl Into<String> + Clone) -> impl Future<Item = (), Error = ()> + 'static {
let n = 0;
let path = path.into();
loop_fn(n, move |_nr| {
let lh = get_handler(path.clone()).unwrap();
lh.get_all()
.or_else(|_| Err(()))
.and_then(|_all| ok(Loop::Break(())))
})
}
this [...] is config which doesn't change [...] read from a configfile during app init.
In that case, create a singleton which will give you an effectively-static value:
extern crate lazy_static; // 1.1.0
use lazy_static::lazy_static;
lazy_static! {
static ref PATH: String = {
// Should be read from a file.
String::from("/the/path/to/the/thing")
};
}
Then change all of the values to &'static str:
#[derive(Debug)]
pub struct Handler {
path: &'static str,
}
impl HandlerTrait for Handler {
fn new(path: &'static str) -> Box<HandlerTrait> {
Box::new(Handler {
path
})
}
}
And take a reference to the singleton:
fn start_runtime() -> Result<()> {
tokio::run(doit(&PATH));
Ok(())
}
You can couple this with phimuemue's answer to get a &'static MyConfigStruct, which could then have a fn foo(&'static self) that is available.
There must be something wrong with a language if this becomes so difficult and needs mem-io multiple times.
You are partially correct. It's difficult to have maximally performant async code with today's Rust (1.30) because Rust wants to ensure memory safety above all else. This doesn't mean that the code is unperformant, just that there's a bit of room to do better.
Honestly, making clones here is unlikely to be a performance bottleneck, but it is annoying. That's where async and await syntax comes in. This will allow futures to more easily make use of references in an idiomatic Rust manner.
because the runtime does not need to run the whole time [...] Do I understand something wrong?
However, async and await still might not help you, as by default Tokio will run your future on a different thread. That's one of the primary reasons it requires a 'static bound. This prevents a Tokio thread from having a reference to a stack local that goes out of scope, introducing memory unsafety. This isn't a unique problem to Tokio, however.
See also:
How can I pass a reference to a stack variable to a thread?
The compiler suggests I add a 'static lifetime because the parameter type may not live long enough, but I don't think that's what I want
Why is the bound `T: 'a` required in order to store a reference `&'a T`?
Why does the Rust compiler request I constrain a generic type parameter's lifetime (error E0309)?
Other bits
It appears that every single call to wait in this code is a misuse of futures. You may wish to re-read the Tokio docs to better understand understand how you are supposed to chain futures. If there's a wait call, it's usually at the end of everything, and even that is rare when using Tokio.
See also:
How do I synchronously return a value calculated in an asynchronous Future in stable Rust?
You can restrict the lifetime of &self:
impl StructX {
fn doit(&'static self) {
// here, we know that self and its members are 'static
}
}
If you do this, you may actually be better off to have StructX store the 'static borrow of the path in the first place (instead of a string).
I can answer this now by myself:
fn make_static_str<T>(s: T) -> &'static str where T: Into<String>
A solution with Arc<Mutex<String>> - The test fails because there is no file to read on playground.
I'm trying to use a struct created in main() and pass it on to a function that returns a boxed Future. However, I run into lifetime and borrowing issues and can't seem to resolve this cleanly.
Here is my struct and functions:
extern crate futures; // 0.1.21
extern crate tokio_core; // 0.1.17
use futures::{future::ok, Future};
pub struct SomeStruct {
some_val: u32,
}
impl SomeStruct {
pub fn do_something(&self, value: u32) -> u32 {
// Do some work
return self.some_val + value;
}
}
fn main() {
let core = tokio_core::reactor::Core::new().unwrap();
let my_struct = SomeStruct { some_val: 10 };
let future = get_future(&my_struct);
core.run(future);
let future2 = get_future(&my_struct);
core.run(future2);
}
fn get_future(some_struct: &SomeStruct) -> Box<Future<Item = u32, Error = ()>> {
let fut = ok(20).and_then(|val| {
let result = some_struct.do_something(val);
ok(result)
});
Box::new(fut)
}
On compiling, the following error occurs:
error[E0621]: explicit lifetime required in the type of `some_struct`
--> src/main.rs:33:5
|
28 | fn get_future(some_struct: &SomeStruct) -> Box<Future<Item = u32, Error = ()>> {
| ----------- consider changing the type of `some_struct` to `&'static SomeStruct`
...
33 | Box::new(fut)
| ^^^^^^^^^^^^^ lifetime `'static` required
I suppose the error occurs because SomeStruct is used in the Future and might be used outside of main()s scope, hence the compiler asks me to change the lifetime to 'static. Here is what I tried so far (unsuccessfully):
Changing the lifetime to 'static as suggested by the compiler, which creates borrowing issues in main().
Moving val by adding ok(20).and_then(move |val| { as suggested by the compiler, which creates issues in the second invocation of get_future().
Use the lazy_static crate to explicitly initialize SomeStruct as static (as suggested here), however I run into macro errors when trying that.
The whole example can be found here. I have omitted some details to create an minimal example. The issues occur using tokio-core and futures = "0.1". Migrating to version "0.2" is not an option unfortunately, due to a dependency of another library.
Returning a boxed trait object has a 'static bound by default. Do as the compiler suggests and provide an explicit lifetime, but not 'static:
fn get_future<'a>(some_struct: &'a SomeStruct) -> Box<dyn Future<Item = u32, Error = ()> + 'a> {
let fut = future::ok(20).and_then(move |val| {
let result = some_struct.do_something(val);
future::ok(result)
});
Box::new(fut)
}
You also have to use move to transfer ownership of some_struct to the closure and change core to be mutable. You should also handle potential errors resulting from core.run.
For the example provided, you could also return impl Future:
fn get_future<'a>(some_struct: &'a SomeStruct) -> impl Future<Item = u32, Error = ()> +'a {
future::ok(20).and_then(move |val| {
let result = some_struct.do_something(val);
future::ok(result)
})
}
See also:
How do I return a boxed closure from a method that has a reference to the struct?
Why is adding a lifetime to a trait with the plus operator (Iterator<Item = &Foo> + 'a) needed?
What is the correct way to return an Iterator (or any other trait)?
#![feature(rustc_private)]
#![feature(box_syntax)]
extern crate rustc;
extern crate rustc_driver;
use rustc::hir::intravisit as hir_visit;
use rustc::hir;
use rustc_driver::driver::{CompileController, CompileState};
pub struct SomeVisitor<'a, 'tcx: 'a> {
pub map: &'a hir::map::Map<'tcx>,
}
impl<'v, 'tcx: 'v> rustc::hir::intravisit::Visitor<'tcx> for SomeVisitor<'v, 'tcx> {
fn nested_visit_map<'this>(&'this mut self) -> hir_visit::NestedVisitorMap<'this, 'tcx> {
hir_visit::NestedVisitorMap::All(self.map)
}
}
fn hir(s: &mut CompileState) {
let krate = s.hir_crate.unwrap();
let map = s.hir_map.unwrap();
let mut visitor = SomeVisitor { map };
hir_visit::walk_crate(&mut visitor, krate);
}
fn main() {
{
let mut controller = CompileController::basic();
controller.after_hir_lowering.callback = box hir;
}
}
playground
I understand why I am getting the lifetime error and it is very easy to solve it by adding explicit lifetimes for the function hir.
pub fn walk_crate<'v, V: hir_visit::Visitor<'v>>(visitor: &mut V, krate: &'v Crate) {}
Because of this definition the lifetime for the reference needs to live for 'tcx.
fn hir<'v, 'tcx>(s: &'tcx mut CompileState<'v, 'tcx>) {
let krate = s.hir_crate.unwrap();
let map = s.hir_map.unwrap();
let mut visitor = SomeVisitor { map };
hir_visit::walk_crate(&mut visitor, krate);
}
But then the function hir becomes incompatible for the callback. playground
I assume that I may need to use HRTB here?
Update:
My current workaround is to use transmute. (playground). Surely there must be a better way?
hir_visit::walk_crate(&mut visitor, visitor.map.krate());
The solution was to realize that map also contains a krate as a reference, but with the correct lifetime. This means that I don't have to introduce explicit lifetimes.
playground
I have a struct containing unsafe code with the following method:
use std::sync::Arc;
use std::thread;
#[derive(Debug)]
struct Foo<T> {
items: Vec<Box<(T, String)>>,
}
impl<T> Foo<T> {
pub fn add_element(&self, element: T, key: String) {
if !(self.items.iter().any( |i| i.1 == key)) {
let mut items = unsafe {change_mut(&(self.items))};
items.push(Box::new((element,key)));
}
}
}
unsafe fn change_mut<T>(x: &T) -> &mut T { // changes &self to &mut self
&mut *(x as *const T as *mut T)
}
fn main() {
let foo = Arc::new(Foo { items: vec!() });
let clone = foo.clone();
// This should not be possible, as it might lead to UB
thread::spawn(move || clone.add_element(1, String::from("one")));
println!("{:?}", *foo);
}
This struct is completely safe until someone starts using this method while multithreading. However, due to the fact the struct only contains a Vec<Box<T,String>>, Syncis implemented by default, which I would like to prevent.
I have found two ways to do this, both of which are not that great...
Add a struct field which does not implement Sync for example *const u8, this is obviously rather bad, as it ends up resulting in unnecessary and unclear code and does not clearly show my intent.
impl !Sync for Struct {} is not available on stable and will be removed according to this issue.
The corresponding error is telling me to use marker types instead, but the documention does not present a way to solve my problem either.
error: negative trait bounds are not yet fully implemented; use marker types for
now (see issue #13231)
--> src\holder.rs:44:1
|
44 | impl !Sync for Struct {}
| ^^^^^^^^^^^^^^^^^^^^^^^^
Interior mutability in Rust requires1 the use of UnsafeCell as a hint to the compiler that the normal rules do not apply.
Your structure should therefore look such:
#[derive(Debug)]
struct Foo<T> {
items: UnsafeCell<Vec<Box<(T, String)>>>,
}
And then, the implementation of add_element is adjusted to:
impl<T> Foo<T> {
pub fn add_element(&self, element: T, key: String) {
if !(self.items.iter().any( |i| i.1 == key)) {
let mut items = unsafe { &mut *self.items.get() };
// ^~~~~~~~~~~~~~~~~~~~~~
items.push(Box::new((element,key)));
}
}
}
The use of UnsafeCell makes change_mut completely unnecessary: it is the purpose of UnsafeCell, after all, to allow interior mutability. Note how its get method returns a raw pointer, which cannot be dereferenced without an unsafe block.
Since UnsafeCell does not implement Sync, Foo<T> will not implement Sync either, and therefore it becomes unnecessary to use negative implementations or any marker.
1 If you do not use it directly, chances are the abstractions you do use are built on it. It is as special as it could be: it is a lang item, as denoted by its attribute #[lang = "unsafe_cell"].