I looked at this post but couldn't figure out where I was going wrong with my code. I have a vector of structs that each contain an mpsc::Sender and mpsc::Receiver and I'm trying to move each of these structs into a thread and keep getting this compile error (and a symmetrical one for the Sender). Since the Sender and Receiver are part of a struct, I wasn't sure how to clone them.
for obj in objs.iter() {
let handle = thread::spawn(move || {
^^^^^^^^^^^^^ `std::sync::mpsc::Receiver<abc::xyz>` cannot be shared between threads safely
obj.do_something();
...
});
...
}
for obj in objs.iter() {
This means obj is an &T. The move closure means the reference gets moved into the closure but… that's it. It's a reference. Meaning the type being referenced must be Sync (and also you'll have lifetime issues because stdlib threads are not scoped, so the compiler will eventually tell you that the references don't live long enough).
If objs is a vector, removing the .iter() will fix that issue though it may trigger new ones: it'll perform an "owned" iteration, so you will be moving the obj itself into the loop.
If objs is not a vector, then make it a vector somehow because a slice can't hand out owned objects. Or clone everything but I rather doubt that's what you want. Your final option is to put every obj inside an Arc, and probably a Mutex/RwLock if you want to mutate them.
Related
I am working on a RPC client using a tide server with state. I put another client into that state so my endpoints can use it:
let client = MintClient::new(cfg, Arc::new(db), Default::default());
let state = State {
mint_client: Arc::new(client),
};
let mut app = tide::with_state(state);
The endpoints are:
app.at("/info")
.post(|req: tide::Request<State>| async move {
let State { ref mint_client } = req.state();
Body::from_json(&InfoResponse::new(mint_client.coins()))
});
I copied the Arc from existing code, but I don't know how it works and someone told me its unnecessary in my situation.
If I don't use a State, I get the error "this closure implements FnOnce, not Fn". If I remove the move so it borrows the client, I get "may outlive borrowed value client" on the req parameter.
I don't know how to approach this. Should I use functions as the endpoints instead of closures and pass a immutable reference to them and then use generic lifetime? Is it possible to use generic lifetimes on closures too?
with_state requires the state to implement Clone + Send + Sync + 'static (see https://docs.rs/tide/0.16.0/tide/fn.with_state.html ).
If you want to share MintClient, not clone it, you'll need some type of a reference. Since thread-safety is required (Send + Sync), you can't use ordinary references (with lifetimes) or Rc, it has to be Arc.
but I don't know how it works and someone told me its unnecessary in my situation.
Arc owns the object, and can provide a usual immutable reference to you on demand. It implements Clone, so it is possible to create multiple copies of it (that's useful for sharing). It implements Send + Sync, so it is possible to pass safely between threads without data races (Tokio is multi-threaded).
The owned object is kept alive until the last Arc clone drops. In your case it is tied to the app lifetime, so it probably never drops the MintClient while the server is running.
Read more here:
https://doc.rust-lang.org/std/sync/struct.Arc.html
My program uses rusqlite to build a database from another data source. The database builds multiple tables in the same manner, so I thought I'd make a reusable function to do so:
fn download_generic<Inserter>(table_name: &str,
connection: &mut rusqlite::Connection,
inserter: &mut Inserter)
-> Result<(), String>
where Inserter: FnMut(&str, &json::JsonValue) -> ()
{}
inserter is a function that binds the correct values from a previously-prepared statement and does the insertion.
I call it like this:
let mut insert_stmt = connection
.prepare("insert or replace into categories values(?,?);")
.unwrap();
download_generic("categories",
&mut connection,
&mut |uuid, jsonproperties| {
insert_stmt.execute(&[&uuid, &jsonproperties["name"].as_str().unwrap_or("")]);
});
However I can't pass &mut connection to download_generic because it's already being borrowed by the insert_stmt. Putting it into a RefCell makes no sense because I shouldn't need runtime overhead to make this work.
I could try making the insert_stmt generated by a lambda that you pass to download_generic, but then I get overwhelmed by having to add lifetime markers everywhere, and it seems unnatural, anyway.
By design, Rust prevents you from having an immutable borrow and a mutable borrow on the same object active at the same time. This is to prevent dangling pointers and data races.
In rusqlite's API, some methods on Connection require a mutable self, and some methods only require an immutable self. However, some of the methods that only require an immutable self return objects that keep that borrow active; prepare is an example of this. Therefore, as long as one of these objects stays in scope, Rust will not allow you to take a mutable borrow on the Connection.
There's probably a reason why some methods take self by mutable reference. Requiring a mutable reference ensures the callee that it has exclusive access to that object. If you think that might not be the case for the methods you need to use, or you think there could be another way to solve this, you should report an issue to the library's maintainers.
Regarding prepare specifically, you can work around the conflicting borrows by calling prepare_cached from within the closure instead. In order to do that, you'll have to make download_generic pass the connection back as a parameter to the closure, otherwise you'd have two mutable borrows on connection and that's not allowed.
I'm wrapping a Rust object to be used from Lua. I need the object to be destroyed when neither Rust code nor Lua still has a reference to it, so the obvious (to me) solution is to use Rc<T>, stored in Lua-managed memory.
The Lua API (I'm using rust-lua53 for now) lets you allocate a chunk of memory and attach methods and a finalizer to it, so I want to store an Rc<T> into that chunk of memory.
My current attempt looks like. First, creating an object:
/* Allocate a block of uninitialized memory to use */
let p = state.new_userdata(mem::size_of::<Rc<T>>() as size_t) as *mut Rc<T>;
/* Make a ref-counted pointer to a Rust object */
let rc = Rc::<T>::new(...);
/* Store the Rc */
unsafe { ptr::write(p, rc) };
And in the finaliser:
let p: *mut Rc<T> = ...; /* Get a pointer to the item to finalize */
unsafe { ptr::drop_in_place(p) }; /* Release the object */
Now this seems to work (as briefly tested by adding a println!() to the drop method). But is it correct and safe (as long as I make sure it's not accessed after finalization)? I don't feel confident enough in unsafe Rust to be sure that it's ok to ptr::write an Rc<T>.
I'm also wondering about, rather than storing an Rc<T> directly, storing an Option<Rc<T>>; then instead of drop_in_place() I would ptr::swap() it with None. This would make it easy to handle any use after finalization.
Now this seems to work (as briefly tested by adding a println!() to the drop method). But is it correct and safe (as long as I make sure it's not accessed after finalisation)? I don't feel confident enough in unsafe Rust to be sure that it's ok to ptr::write an Rc<T>.
Yes, you may ptr::write any Rust type to any memory location. This "leaks" the Rc<T> object, but writes a bit-equivalent to the target location.
When using it, you need to guarantee that no one modified it outside of Rust code and that you are still in the same thread as the one where it was created. If you want to be able to move across threads, you need to use Arc.
Rust's thread safety cannot protect you here, because you are using raw pointers.
I'm also wondering about, rather than storing an Rc<T> directly, storing an Option<Rc<T>>; then instead of drop_in_place() I would ptr::swap() it with None. This would make it easy to handle any use after finalisation.
The pendant to ptr::write is ptr::read. So if you can guarantee that no one ever tries to ptr::read or drop_in_place() the object, then you can just call ptr::read (which returns the object) and use that object as you would use any other Rc<T> object. You don't need to care about dropping or anything, because now it's back in Rust's control.
You should also be using new_userdata_typed instead of new_userdata, since that takes the memory handling off your hands. There are other convenience wrapper functions ending with the postfix _typed for most userdata needs.
Your code will work; of course, note that the drop_in_place(p) will just decrease the counter of the Rc and only drop the contained T if and only if it was the last reference, which is the correct action.
I'm currently struggling with lifetimes in Rust (1.0), especially when it comes to passing structs via channels.
How would I get this simple example to compile:
use std::sync::mpsc::{Receiver, Sender};
use std::sync::mpsc;
use std::thread::spawn;
use std::io;
use std::io::prelude::*;
struct Message<'a> {
text: &'a str,
}
fn main() {
let (tx, rx): (Sender<Message>, Receiver<Message>) = mpsc::channel();
let _handle_receive = spawn(move || {
for message in rx.iter() {
println!("{}", message.text);
}
});
let stdin = io::stdin();
for line in stdin.lock().lines() {
let message = Message {
text: &line.unwrap()[..],
};
tx.send(message).unwrap();
}
}
I get:
error[E0597]: borrowed value does not live long enough
--> src/main.rs:23:20
|
23 | text: &line.unwrap()[..],
| ^^^^^^^^^^^^^ does not live long enough
...
26 | }
| - temporary value only lives until here
|
= note: borrowed value must be valid for the static lifetime...
I can see why this is (line only lives for one iteration of for), but I can't figure out what the right way of doing this is.
Should I, as the compiler hints, try to convert the &str into &'static str?
Am I leaking memory if every line would have a 'static lifetime?
When am I supposed to use 'static anyway? Is it something I should try to avoid or is it perfectly OK?
Is there a better way of passing Strings in structs via channels?
I apologize for those naive questions. I've spent quite some time searching already, but I can't quite wrap my head around it. It's probably my dynamic language background getting in the way :)
As an aside: Is &input[..] for converting a String into a &str considered OK? It's the only stable way I could find to do this.
You can't convert &'a T into &'static T except by leaking memory. Luckily, this is not necessary at all. There is no reason to send borrowed pointers to the thread and keep the lines on the main thread. You don't need the lines on the main thread. Just send the lines themselves, i.e. send String.
If access from multiple threads was necessary (and you don't want to clone), use Arc<String> (in the future, Arc<str> may also work). This way the string is shared between threads, properly shared, so that it will be deallocated exactly when no thread uses it any more.
Sending non-'static references between threads is unsafe because you never know how long the other thread will keep using it, so you don't know when the borrow expires and the object can be freed. Note that scoped threads don't have this problem (which aren't in 1.0 but are being redesigned as we speak) do allow this, but regular, spawned threads do.
'static is not something you should avoid, it is perfectly fine for what it does: Denoting that a value lives for the entire duration the program is running. But if that is not what you're trying to convey, of course it is the wrong tool.
Think about it this way: A thread has no syntactical lifetime, i.e. the thread will not be dropped at the end of code block where it was created. Whatever data you send to the thread, you must be sure that it will live as long as the thread does, which means forever. Which means 'static.
What can go wrong in your case, is if the main loop sends a reference to a thread and destroys the string before it has been handled by the thread. The thread would access invalid memory when dealing with the string.
One option would be to put your lines into some statically allocated container but this would mean that you never can destroy those strings. Generally speaking a bad idea. Another option is to think: does the main thread actually need the line once it is read? What if the main thread transfered responsibility for line to the handling thread?
struct Message {
text: String,
}
for line in stdin.lock().lines() {
let message = Message {
text: line.unwrap(),
};
tx.send(message).unwrap();
}
Now you are transferring ownership (move) from the main thread to the handler thread. Because you move your value, no references are involved and no checks for lifetime apply anymore.
As I've read all objects in D are fully location independent. How this requirement is achieved?
One thing that comes to my mind, is that all references are not pointers to the objects, but to some proxy, so when you move object (in memory) you just update that proxy, not all references used in program.
But this is just my guess. How it is done in D for real?
edit: bottom line up front, no proxy object, objects are referenced directly through regular pointers. /edit
structs aren't allowed to keep a pointer to themselves, so if they get copied, they should continue to just work. This isn't strictly enforced by the language though:
struct S {
S* lol;
void beBad() {
lol = &this; // this compiler will allow this....
}
}
S pain() {
S s;
s.beBad();
return s;
}
void main() {
S s;
s = pain();
assert(s.lol !is &s); // but it will also move the object without notice!
}
(EDIT: actually, I guess you could use a postblit to update internal pointers, so it isn't quite without notice. If you're careful enough, you could make it work, but then again, if you're careful enough, you can shoot between your toes without hitting your foot too. EDIT2: Actually no, the compiler/runtime is still allowed to move it without even calling the postblit. One example of where this happens is if it copies a stack frame to the heap to make a closure. The struct data is moved to a new address without being informed. So yeah. /edit)
And actually, that assert isn't guaranteed to pass, the compiler might choose to call pain straight on the local object declared in main, so the pointer would work (though I'm not able to force this optimization here for a demo, generally, when you return a struct from a function, it is actually done via a hidden pointer the caller passes - the caller says "put the return value right here" thus avoiding a copy/move in some cases).
But anyway, the point just is that the compiler is free to copy or not to copy a struct at its leisure, so if you do keep the address of this around in it, it may become invalid without notice; keeping that pointer is not a compile error, but it is undefined behavior.
The situation is different with classes. Classes are allowed to keep references to this internally since a class is (in theory, realized by the garbage collector implementation)) an independent object with an infinite lifetime. While it may be moved (such as be a moving GC (not implemented in D today)), if it is moved, all references to it, internal and external, would also be required to be updated.
So classes can't have the memory pulled out from under them like structs can (unless you the programmer take matters into your own hands and bypass the GC...)
The location independent thing I'm pretty sure is referring only to structs and only to the rule that they can't have pointers to themselves. There's no magic done with references or pointers - they indeed work with memory addresses, no proxy objects.