I am trying to return a struct containing reference to a shared mutex:
struct Test<'a> {
mutex: Arc<Mutex<()>>,
guard: &'a MutexGuard<'a, ()>,
}
impl<'a> Test<'a> {
pub fn new() -> Self {
let mutex = Arc::new(Mutex::new(()));
let guard = &mutex.lock().unwrap();
Self {
mutex,
guard,
}
}
}
The lifetimes seem correct: the mutex exists at least during the lifetime of Test, so MutexGuard does not have a stalled reference to mutex. But Rust gives errors. How to explain to Rust that the lifetime of mutex field is enough long for guard to work well?
cannot return value referencing local variable `mutex`
returns a value referencing data owned by the current function
BTW, I am trying to create a "mutli-mutex" - a set of mutexes for keys (like as in HashMap), to block downloading a file whose name is in hashmap (because it is already downloading).
Related
I have a Rust function like this:
pub fn get_buffer() -> &[u8] {
// returns *mut c_char
let ptr = unsafe { get_buffer_from_c_code() };
// returns &[u8]
let buf = unsafe { core::slice::from_raw_parts(ptr as *const u8, 10) };
buf
}
It generates this error:
pub fn get_buffer() -> &[u8] {
| ^ expected named lifetime parameter
|
= help: this function's return type contains a borrowed value, but there is no value for it to be borrowed from
help: consider using the `'static` lifetime
|
19 | pub fn get_buffer() -> &'static [u8] {
| ~~~~~~~~
I understand the error. It makes sense.
Question: should I take the compiler's suggestion and add a static lifetime specifier?
I'm connecting Rust to a C library that allocates memory internally and returns a pointer to it. Later, the C library takes care of de-allocating that memory on its own. I'd like the Rust code to be able to read what's in the memory, but otherwise leave it alone.
Rust is allocating the slice, though, in its own memory, and the slice itself (the pointer and the length) need to be dropped eventually.
Does a static lifetime do what I need it to do? Will Rust drop the slice, but not try to free the underlying buffer?
Question: should I take the compiler's suggestion and add a static lifetime specifier?
No. If your function return a static reference then you're promising to your caller that it can keep the reference around (and read through it) as long as it likes, which is only true if the buffer is never deallocated and never modified.
I'm connecting Rust to a C library that allocates memory internally and returns a pointer to it. Later, the C library takes care of de-allocating that memory on its own.
The solution to this problem depends entirely on when the deallocation happens. You need to ensure that there is some lifetime of a borrow such that there is no possibility to cause the deallocation until the borrow ends. You wrote in a comment
When my Rust code returns then C deallocates stuff.
That's key to picking the solution. That means that the reference should be obtained when the Rust code is called. That is:
extern "C" wrapper_called_from_c_code() {
let ptr = unsafe { get_buffer_from_c_code() };
let buf = unsafe { core::slice::from_raw_parts(ptr as *const u8, 10) };
// Constrain the lifetime of the slice to be the duration of
// this call by passing it through a lifetime-generic function.
// (<'a> is just for explicitness and could be elided.)
fn shim<'a>(buf: &'a [u8]) {
safe_rust_code(buf);
}
shim(buf);
// Now, after the function call returns, it's safe to proceed with
// allowing the C code to deallocate the buffer.
}
fn safe_rust_code(buf: &[u8]) {
// write whatever you like here
}
safe_rust_code can do whatever it likes in its function body, but the borrow checker will ensure it cannot hang onto the &'a [u8] slice reference longer than is safe.
The shim function exists to ensure that what wrapper_called_from_c_code needs (that the slice reference is being passed to a lifetime-generic function and not one that accepts &'static [u8]) inside wrapper_called_from_c_code rather than to explain it as a constraint on another function. I consider this good practice to keep invariants in the narrowest scope possible, to reduce the chances that they're broken by merely editing safe code without reading the comments.
Wrap the buffer in a struct that frees the buffer when the struct is dropped. The struct can then own the buffer, much like a Vec owns a block of data on the heap. When it hands out references their lifetimes will naturally be tied to the lifetime of the struct.
pub struct Buffer {
ptr: *const u8,
len: usize,
}
impl Buffer {
pub fn new() -> Self {
Self {
ptr: unsafe { get_buffer_from_c_code() },
len: 10,
}
}
}
impl Drop for Buffer {
fn drop(&mut self) {
unsafe {
free_buffer_from_c_code(self.ptr);
}
}
}
impl Deref for Buffer {
type Target = [u8];
fn deref(&self) -> &[u8] {
// SAFETY: The C library must not modify the contents of the buffer
// for the lifetime of the slice.
unsafe { slice::from_raw_parts(self.ptr, self.len) }
}
}
The solution I ended up going with was something like this:
pub fn get_buffer<'a>(foo: u32) -> &'a [u8] {
let ptr = unsafe { get_buffer_from_c_code() };
let buf = unsafe { core::slice::from_raw_parts(ptr as *const u8, 10) };
buf
}
Adding a parameter and then defining a lifetime on the function itself works. Essentially, it says that the returned slice lives as long as the foo parameter.
You can also do this:
pub fn get_buffer(foo: &u32) -> &[u8] {
If foo is a reference then you can elide the lifetime. The system makes the slice live as long as the foo reference. I don't know why it can do this with a reference and not a value parameter, but there it is.
I've got a struct defined that has a function which defines a static lifetime:
impl MyStruct {
pub fn doSomething(&'static self) {
// Some code goes here
}
}
I'm consuming it from main like so:
fn main() {
let obj = MyStruct {};
obj.doSomething();
}
It's intended for the doSomething call to block and execute for the lifetime of the application.
I'm running into issues with the lifetime checks where it's stating that it may outlive the main function, which seems strange to me as once main is complete the application should exit.
Is there a way to achieve this?
The primary way to create a reference that has the 'static lifetime is to make the variable static. A static variable is one that can be created at compile time:
struct MyStruct;
impl MyStruct {
pub fn do_something(&'static self) {}
}
static OBJ: MyStruct = MyStruct;
fn main() {
OBJ.do_something();
}
As Rust's constant evaluation story improves, this will be more common, but it will never allow configuration at runtime.
A far less common method is to deliberately leak memory, producing a reference that will last "forever". This should be discouraged because leaking memory isn't a good thing:
fn main() {
let obj = Box::leak(Box::new(MyStruct));
obj.do_something();
}
There's also the possibility of creating a singleton:
How do I create a global, mutable singleton?
How can you make a safe static singleton in Rust?
as once main is complete the application should exit.
Perhaps, but the compiler doesn't treat main specially for lifetime purposes.
hyper requires a static runtime when running the server and processing each request.
No, it doesn't. It has a bound of : 'static, which means that any references passed in must be 'static, but you don't have to pass in a bare reference at all.
For patterns like this, the most common thing is to pass in something like an Arc. This allows sharing of the underlying resource.
pub fn do_something<F, T>(f: F)
where
F: Fn() -> T + 'static,
T: 'static,
{
// "spawn" 3 threads
f();
f();
f();
}
struct MyStruct;
static OBJ: MyStruct = MyStruct;
fn main() {
// OK
do_something(|| &OBJ);
// Not OK
let another = MyStruct;
do_something(|| &another);
// OK
use std::sync::Arc;
let shared = Arc::new(MyStruct);
do_something(move || shared.clone());
}
You can even use interior mutability if you wanted dynamic reconfiguration.
See also:
How do I use static lifetimes with threads?
The naive way to do this is with a static variable, but it will require unsafe code if you need to actually set the value inside your main function:
static mut OBJ: MyStruct = MyStruct;
fn main() {
unsafe {
OBJ = MyStruct {};
OBJ.doSomething();
}
}
It's also unsafe to do pretty much anything with a mutable static thereafter.
The much better way to do it is to let a library (lazy_static) take care of the unsafe code.
use lazy_static::lazy_static;
fn main() {
lazy_static!{
static ref OBJ: MyStruct = MyStruct {};
}
OBJ.doSomething();
}
I use lazy_static to keep a HashMap in the memory. With two methods I am adding and getting elements, but I am having some problems with lifetimes.
This is my code:
#[macro_use]
extern crate lazy_static;
use std::sync::Mutex;
use std::collections::HashMap;
lazy_static! {
static ref HASHMAP: Mutex<HashMap<String, Foo>> = Mutex::new({
let mut m = HashMap::new();
m.insert("one".to_string(), Foo{param1:"foo1".to_string(), param2:"foo2".to_string()} );
m.insert("two".to_string(), Foo{param1:"bar1".to_string(), param2:"bar2".to_string()});
m
});
}
pub struct Foo{
param1: String,
param2: String,
}
pub fn ins_val(name: String, f: Foo){
HASHMAP.lock().unwrap().insert(name, f);
}
pub fn get_val(k: &str) -> Option<&Foo>{
HASHMAP.lock().unwrap().get(k)
}
And this is the error:
HASHMAP.lock().unwrap().get(k)
^^^^^^^^^^^^^^^^^^^^^^^
reference must be valid for the anonymous lifetime #1 defined on the block
Rust is correctly preventing a bug here.
Your get_val function is trying to return a reference into HASHMAP, but this isn't safe unless you continue holding the lock, as someone else could otherwise come and modify the map under your feet.
You need to either keep the lock until you return the value (by returning the mutex guard along with the value; for example using MutexGuardRef from the owning_ref crate), or copy the value instead of returning a reference:
pub fn get_val(k: &str) -> Option<Foo> {
HASHMAP.lock().unwrap().get(k).cloned()
}
With this implementation Foo would of course need to implement Clone.
I am storing a closure in a struct like this:
#[derive(Clone)]
struct S<'a> {
func: &'a FnOnce() -> u32
}
fn main() {
let s = S { func: &|| 0 };
let val = (s.func)();
println!("{}", val);
}
When I compile, s.func cannot be moved to execute itself. I understand why it cannot be moved (namely that it's only a reference and that its size is not known at compile time), but don't know why it's being moved at all -- is it just because closures are implemented via traits?
Here's the error message:
error[E0161]: cannot move a value of type std::ops::FnOnce() -> u32:
the size of std::ops::FnOnce() -> u32 cannot be statically determined
--> main.rs:8:15
|
8 | let val = (s.func)();
| ^^^^^^^^
error[E0507]: cannot move out of borrowed content
--> main.rs:8:15
|
8 | let val = (s.func)();
| ^^^^^^^^ cannot move out of borrowed content
error: aborting due to 2 previous errors
Is this only way the solve this to store the closure on the heap (via Box<FnOnce() -> u32>)? And why does calling a closure move it? Presumably calling it doesn't mutate the function itself.
The closure is being moved because FnOnce::call_once takes self by value. This contract enforces the guarantee that the function will not be called more than once.
If you will indeed be calling the closure at most once, and you want to use the FnOnce trait, then your struct needs to take ownership of that closure (and you will need to make your struct generic on the closure type). Note that calling the closure will move the closure out of your struct, thereby invalidating the whole struct; you may work around that by wrapping the FnOnce in an Option and take-ing the closure out of the Option before calling it.
If you might be calling the closure more than once, you don't want to take ownership of the closure, or you don't want to make your struct generic on the closure type, then you should use either Fn or FnMut instead. Fn::call takes self by reference and FnMut::call_mut takes self by mutable reference. Since both accept references, you can use trait objects with them.
As explained by Francis, declaring a closure FnOnce tells Rust that you accept the broadest class of closures, including those that exhaust the objects they capture. That such closures are invoked only once is ensured by the compiler by destroying the closure object itself (by moving it into its own call method) when invoked.
It is possible to use FnOnce and still not have S generic on the closure, but it requires some work to set things up so that the closure can't be possibly invoked more than once:
the closure must be stored in an Option, so its contents can be "stolen" and the Option replaced with None (this part ensures that the closure won't be called twice);
invent a trait that knows how to steal the closure from the option and invoke it (or do something else if the closure was already stolen);
store a reference to the trait object in S - this enables the same S type works on different closures without being generic on closure type.
The result looks like this:
trait Callable {
fn call_once_safe(&mut self, default: u32) -> u32;
}
impl<F: FnOnce() -> u32> Callable for Option<F> {
fn call_once_safe(&mut self, default: u32) -> u32 {
if let Some(func) = self.take() {
func()
} else {
default
}
}
}
struct S<'a> {
func: &'a mut Callable
}
impl<'a> S<'a> {
pub fn invoke(&mut self) -> u32 {
self.func.call_once_safe(1)
}
}
fn main() {
let mut s = S { func: &mut Some(|| 0) };
let val1 = s.invoke();
let val2 = s.invoke();
println!("{} {}", val1, val2);
}
The only place that knows details about the closure is the implementation of Callable for the particular Option<F>, generated for each closure and referenced by the vtable of the &mut Callable fat pointer created when initializing the func in S.
I would like to pass a FnOnce closure to an object to be used later, but I would like to avoid any heap allocation. I can avoid heap allocation by keeping the closure on the stack. But the problem is that I can't pass a reference to the object because the FnOnce call_once consumes the closure. So I need to pass an owned pointer (e.g. Box) without heap allocation.
Is this possible? What I'd like to do is this:
fn main() {
let mut scheduler = NoHeapScheduler();
// allocate the task on the stack
let task = move ||;
// somehow pass ownership of the closure, while keeping it allocated on the stack.
scheduler.add_task(StaticBox::new(task));
schedule.run();
}
As far as I know this should be safe as long as the scheduler doesn't outlive the task. Is there any way to make this happen?
Can I create an owned pointer to a stack object?
No. This is non-sensical actually, since by definition a stack object is owned by the stack, so it cannot also be owned by something else.
So I need to pass an owned pointer (e.g. Box) without heap allocation.
There are other owned pointers than Box.
At the moment, I know of none without a heap allocation, but there is little reason it cannot be done.
I envision a InlineFnOnceBox<S: Default, R, A> used as InlineFnOnceBox<[u8; 48], (), ()> in this case, which would contain both the array itself, used as backing storage, plus a virtual pointer to the FnOnce<A -> R> v-table for the type instantiated.
It requires some care (and unsafe code) to instantiate, but otherwise seems feasible.
Can I create an owned pointer to a stack object?
No, but you can simply move the stack object into your scheduler. Your scheduler will increase in size with every closure you schedule, but it will be completely self contained an can even be moved around.
The basic idea is that your Scheduler becomes a kind of singly linked list:
pub trait Scheduler: Sized {
fn run(self);
}
pub struct NoHeapScheduler<F: FnOnce(), T: Scheduler> {
inner: T,
f: F,
}
impl<F: FnOnce(), T: Scheduler> Scheduler for NoHeapScheduler<F, T> {
fn run(self) {
self.inner.run();
(self.f)()
}
}
The Scheduler trait is here to break the recursion chain in the NoHeapScheduler (Otherwise we'd need a feature like variadic generics).
To terminate the chain we also implement Scheduler for some no-op type, e.g. ():
impl Scheduler for () {
fn run(self) {}
}
Now the only thing left is a way to add new closures.
impl<F: FnOnce(), T: Scheduler> NoHeapScheduler<F, T> {
fn add_task<F2: FnOnce()>(self, f: F2) -> NoHeapScheduler<F2, Self> {
NoHeapScheduler {
inner: self,
f: f,
}
}
}
This method moves The current scheduler into a new scheduler and adds the scheduled closure.
You can use this function like so:
let scheduler = scheduler.add_task(task);
Fully working example in the playground
As stated, the answer to the question is "no".
If you pass ownership of the closure, you have to by definition move it into the owner (otherwise what you've got is a reference). You can do that if you've only got one callback using a generic type:
pub struct NoHeapScheduler<F:FnOnce()> {
f: Option<F>,
}
impl<F:FnOnce()> NoHeapScheduler<F> {
pub fn add_task(&mut self, f: F) {
self.f = Some(f);
}
pub fn run(&mut self) {
let f = self.f.take().unwrap();
f()
}
}
fn main() {
let mut scheduler = NoHeapScheduler{ f: None };
let task = move || {};
scheduler.add_task(task);
scheduler.run();
}
Playground
However you'd run into problems adding more than one closure, since they all have different types.
If you're willing to allow allocations and an unstable feature on the nightly compiler, you could use FnBox. This is like FnOnce but works with Box:
#![feature(fnbox)]
use std::boxed::FnBox;
pub struct NoHeapScheduler {
v: Vec<Box<FnBox()>>,
}
impl NoHeapScheduler {
pub fn add_task(&mut self, f: Box<FnBox()>) {
self.v.push(f);
}
pub fn run(&mut self) {
for f in self.v.drain(0..) {
f();
}
}
}
fn main() {
let mut scheduler = NoHeapScheduler{ v: Vec::new() };
let task = move || {println!("Hello,"); };
let other_task = move || {println!("world!"); };
scheduler.add_task(Box::new(task));
scheduler.add_task(Box::new(other_task));
scheduler.run();
}
Playground
I can use an Option to do this. I can keep the Option on the stack and pass a mutable reference around, and then when I'm ready to consume the closure I can use Option::take to take ownership of the closure and consume it.
To handle different implementations of FnOnce, I can lift this out into a trait and use trait objects:
trait Callable {
fn call(&mut self);
}
impl<F: FnOnce()> Callable for Option<F> {
fn call(&mut self) {
if let Some(func) = self.take() {
func();
}
}
}
Then I can pass around trait objects that live on the stack but can be consumed by having the reference.