My program uses rusqlite to build a database from another data source. The database builds multiple tables in the same manner, so I thought I'd make a reusable function to do so:
fn download_generic<Inserter>(table_name: &str,
connection: &mut rusqlite::Connection,
inserter: &mut Inserter)
-> Result<(), String>
where Inserter: FnMut(&str, &json::JsonValue) -> ()
{}
inserter is a function that binds the correct values from a previously-prepared statement and does the insertion.
I call it like this:
let mut insert_stmt = connection
.prepare("insert or replace into categories values(?,?);")
.unwrap();
download_generic("categories",
&mut connection,
&mut |uuid, jsonproperties| {
insert_stmt.execute(&[&uuid, &jsonproperties["name"].as_str().unwrap_or("")]);
});
However I can't pass &mut connection to download_generic because it's already being borrowed by the insert_stmt. Putting it into a RefCell makes no sense because I shouldn't need runtime overhead to make this work.
I could try making the insert_stmt generated by a lambda that you pass to download_generic, but then I get overwhelmed by having to add lifetime markers everywhere, and it seems unnatural, anyway.
By design, Rust prevents you from having an immutable borrow and a mutable borrow on the same object active at the same time. This is to prevent dangling pointers and data races.
In rusqlite's API, some methods on Connection require a mutable self, and some methods only require an immutable self. However, some of the methods that only require an immutable self return objects that keep that borrow active; prepare is an example of this. Therefore, as long as one of these objects stays in scope, Rust will not allow you to take a mutable borrow on the Connection.
There's probably a reason why some methods take self by mutable reference. Requiring a mutable reference ensures the callee that it has exclusive access to that object. If you think that might not be the case for the methods you need to use, or you think there could be another way to solve this, you should report an issue to the library's maintainers.
Regarding prepare specifically, you can work around the conflicting borrows by calling prepare_cached from within the closure instead. In order to do that, you'll have to make download_generic pass the connection back as a parameter to the closure, otherwise you'd have two mutable borrows on connection and that's not allowed.
Related
Is there a way to mutably borrow (or move a reference to) some value into a closure and continue using it outside, in a cleaner way?
For example, I have this code:
let queue = Arc::new(RefCell::new(Vec::new()));
let cqueue = Arc::clone(&queue);
EntityEventQueue::register_receiver(&entity_equeue, "position-callback",
Box::new( move |e| {
cqueue.borrow_mut().push(e.clone());
}));
// mutate queue
It works, but I heard that RefCell is bad practice outside some specific uses. Is there a way that I can use queue both inside and outside of the closure?
And if there is not, do you know a better way of implementing this? The one requirement is that the queue must be outside of the EntityEventQueue structure
(I created the register_receiver method, so it can be altered. Its signature is pub fn register_receiver(this: &Arc<RefCell<Self>>, name: &str, callback: Box<dyn FnMut(...) + 'a>)
You should use some synchronization mechanism instead of RefCell. For example a Mutex or a RwLock. Depending on your writing needs. Quick tip is:
One writer (at a time) and several readers -> RwLock
Many writers many readers -> Mutex
Those are std but you have some other synchronization libraries and mechanisms available.
I am working on a RPC client using a tide server with state. I put another client into that state so my endpoints can use it:
let client = MintClient::new(cfg, Arc::new(db), Default::default());
let state = State {
mint_client: Arc::new(client),
};
let mut app = tide::with_state(state);
The endpoints are:
app.at("/info")
.post(|req: tide::Request<State>| async move {
let State { ref mint_client } = req.state();
Body::from_json(&InfoResponse::new(mint_client.coins()))
});
I copied the Arc from existing code, but I don't know how it works and someone told me its unnecessary in my situation.
If I don't use a State, I get the error "this closure implements FnOnce, not Fn". If I remove the move so it borrows the client, I get "may outlive borrowed value client" on the req parameter.
I don't know how to approach this. Should I use functions as the endpoints instead of closures and pass a immutable reference to them and then use generic lifetime? Is it possible to use generic lifetimes on closures too?
with_state requires the state to implement Clone + Send + Sync + 'static (see https://docs.rs/tide/0.16.0/tide/fn.with_state.html ).
If you want to share MintClient, not clone it, you'll need some type of a reference. Since thread-safety is required (Send + Sync), you can't use ordinary references (with lifetimes) or Rc, it has to be Arc.
but I don't know how it works and someone told me its unnecessary in my situation.
Arc owns the object, and can provide a usual immutable reference to you on demand. It implements Clone, so it is possible to create multiple copies of it (that's useful for sharing). It implements Send + Sync, so it is possible to pass safely between threads without data races (Tokio is multi-threaded).
The owned object is kept alive until the last Arc clone drops. In your case it is tied to the app lifetime, so it probably never drops the MintClient while the server is running.
Read more here:
https://doc.rust-lang.org/std/sync/struct.Arc.html
In vulkano, to create a CPUAccessibleBuffer you need give it some data and the CPUAccessibleBuffer::from_data function requires the data to have the 'static lifetime.
I have some data in &[u8] array (created at runtime) that I would like to pass to that function.
However, it errors with this message
argument requires that `data` is borrowed for `'static`
So how can I make the lifetime of the data 'static ?
You should use CpuAccessibleBuffer::from_iter instead, it does the same thing but does not require the collection to be Copy or 'static:
let data: &[u8] = todo!();
let _ = CpuAccessibleBuffer::from_iter(
device,
usage,
host_cached,
data.iter().copied(), // <--- pass like so
);
Or if you actually have a Vec<u8>, you can pass it directly:
let data: Vec<u8> = todo!();
let _ = CpuAccessibleBuffer::from_iter(
device,
usage,
host_cached,
data, // <--- pass like so
);
If you really must create the data at runtime, and you really need to last for 'static, then you can use one of the memory leaking methods such as Box::leak or Vec::leak to deliberately leak a heap allocation and ensure it is never freed.
While leaking memory is normally something one avoids, in this case it's actually a sensible thing to do. If the data must live forever then leaking it is actually the correct thing to do, semantically speaking. You don't want the memory to be freed, not ever, which is exactly what happens when memory is leaked.
Example:
fn require_static_data(data: &'static [u8]) {
unimplemented!()
}
fn main() {
let data = vec![1, 2, 3, 4];
require_static_data(data.leak());
}
Playground
That said, really think over the reallys I led with. Make sure you understand why the code you're calling wants 'static data and ask yourself why your data isn't already 'static.
Is it possible to create the data at compile time? Rust has a powerful build time macro system. It's possible, for example, to use include_bytes! to read in a file and do some processing on it before it's embedded into your executable.
Is there another API you can use, another function call you're not seeing that doesn't require 'static?
(These questions aren't for you specifically, but for anyone who comes across this Q&A in the future.)
If the data is created at runtime, it can't have a static lifetime. Static means that data is present for the whole lifetime of the program, which is necessary in some contexts, especially when threading is involved. One way for data to be static is, as Paul already answered, explicitly declaring it as such, i.e.:
static constant_value: i32 = 0;
However, there's no universally applicable way to make arbitrary data static. This type of inference is made at compile-time by the borrow checker, not by the programmer.
Usually if a function requires 'static (type) arguments (as in this case) it means that anything less could potentially be unsafe, and you need to reorganize the way data flows in and out of your program to provide this type of data safely. Unfortunately, that's not something SO can provide within the scope of this question.
Make a constant with static lifetime:
static NUM: i32 = 18;
I looked at this post but couldn't figure out where I was going wrong with my code. I have a vector of structs that each contain an mpsc::Sender and mpsc::Receiver and I'm trying to move each of these structs into a thread and keep getting this compile error (and a symmetrical one for the Sender). Since the Sender and Receiver are part of a struct, I wasn't sure how to clone them.
for obj in objs.iter() {
let handle = thread::spawn(move || {
^^^^^^^^^^^^^ `std::sync::mpsc::Receiver<abc::xyz>` cannot be shared between threads safely
obj.do_something();
...
});
...
}
for obj in objs.iter() {
This means obj is an &T. The move closure means the reference gets moved into the closure but… that's it. It's a reference. Meaning the type being referenced must be Sync (and also you'll have lifetime issues because stdlib threads are not scoped, so the compiler will eventually tell you that the references don't live long enough).
If objs is a vector, removing the .iter() will fix that issue though it may trigger new ones: it'll perform an "owned" iteration, so you will be moving the obj itself into the loop.
If objs is not a vector, then make it a vector somehow because a slice can't hand out owned objects. Or clone everything but I rather doubt that's what you want. Your final option is to put every obj inside an Arc, and probably a Mutex/RwLock if you want to mutate them.
I have a CPU struct with a load_rom method:
use std::fs::File;
use std::io::{self, Read};
pub struct CPU {
pub mem: [u8; 4096],
V: [u8; 16],
I: u16,
stack: [u16; 16],
opcode: u16,
}
impl CPU {
pub fn new() -> CPU {
CPU {
mem: [0; 4096],
V: [0; 16],
I: 0,
stack: [0; 16],
opcode: 0,
}
}
pub fn load_rom(&self, filepath: &str) {
let mut rom: Vec<u8> = Vec::new();
let mut file = File::open(filepath).unwrap();
file.read_to_end(&mut rom);
for (i, mut byte) in rom.iter().enumerate() {
self.mem[i] = *byte;
}
}
}
fn main() {}
This generates the error:
error: cannot assign to immutable indexed content `self.mem[..]`
--> src/main.rs:28:13
|
28 | self.mem[i] = *byte;
| ^^^^^^^^^^^^^^^^^^^
When I create a CPU with let mut cpu = CPU::new(); and pass &mut self to the load_rom method, everything works just fine.
If I don't use mut on creation, I get the error:
error: cannot borrow immutable local variable `cpu` as mutable
--> src/main.rs:10:2
|
9 | let cpu = CPU::new();
| --- use `mut cpu` here to make mutable
10 | cpu.load_rom("/Users/.../Code/Rust/chip8/src/roms/connect4.ch8");
| ^^^ cannot borrow mutably
It doesn't seem right that I have to make cpu mutable in order for internal functions to modify its own contents. Do I really have to declare cpu as mutable? Or am I missing something?
make cpu mutable in order for internal functions to modify its own contents
(emphasis mine)
Rust is a systems language, which means that it attempts to give you the ability to create fast and efficient code. One of the primary ways that this is done is by providing references to existing data instead of copying it.
Rust is also a safe language, which (among other things) means that accessing an invalid reference should be impossible.
To accomplish both of these goals, there have to be tradeoffs. Some languages move the safety checks to runtime, enforce mandatory synchronization primitives (e.g. a mutex and friends), or some other interesting solution. Some languages avoid the mess entirely and opt to disallow references or not attempt to guarantee safety.
Rust differs from these by checking as many things at compile time as feasible. This implies that the compiler has to be able to reason about when and where a piece of memory might be mutated.
If it didn't know this, then you might get a reference to something within a value and then call a mutating method on that value that invalidates the reference. When you go to use the now-invalid reference... BOOOOOM. Your program crashes at best, or leaks information or creates a backdoor at worst.
&mut self is in indication to the compiler that this method might mutate the values within. It is only valid to get a mutable reference to a value that is already mutable, which is denoted by the mut keyword on a variable binding (mut cpu here).
However, this isn't just useful to the compiler. Knowing that something is being changed is highly valuable to the programmer too. Mutability in a large system adds hard-to-reason-about complexity, and being forced to explicitly list when something is and isn't mutable can be very informative and mentally freeing.
It's also useful to know the rules for borrowing that Rust applies. These restrict you to one or the other of:
* one or more references (`&T`) to a resource,
* exactly one mutable reference (`&mut T`).
Succinctly, this can be summed as "aliasing XOR mutability".
If your mutation is truly internal, then you can also make use of interior mutability, such as by using a RefCell or a Mutex. What you use depends on your needs and what kind of data you want to store.
These constructs are a good mental fit for structures like caches, where you want to "hide" the mutability from the outside. However, there are also limitations to these as the lifetime of references to the data within must be shortened to continue providing the "aliasing XOR mutabilty" guarantee to keep the code safe.
For your specific problem, I agree with the commenters that it makes sense for load_rom to accept a &mut self. It can even be simplified:
pub fn load_rom(&mut self, filepath: &str) {
let mut file = File::open(filepath).unwrap();
file.read_exact(&mut self.mem);
}
You may want to zero out any old data before loading. Otherwise, if you load a second ROM that's smaller than the first, data from the first ROM can leak to the second (an actual bug from older computers / operating systems).
Rust uses a transitive immutability model. This means that if a variable is marked as immutable, the variable may not be mutated, and data accessed through the variable may not be mutated.
Furthermore, if you have a mutable reference to a variable, the type system disallows any immutable references from coexisting; and so data not marked as `mut' is truly unchanging throughout the lifetime of the the immutable reference.
Together this makes it so that by default, it is not possible for there to be two mutable references to the same data at the same time. This is a requirement for efficient memory safety and thread safety; and also makes it simpler to reason about mutation in Rust code.
If you want "interior" mutability you can use Cell<T> or RefCell<T> from the std::cell module. However, this is probably the wrong thing to do for a CPU struct that is meant to represent a CPU that is expected to be run, and have its state change after each operation. Interior mutability should generally be reserved for performing mutation within the implementation of an operation that does not perform any logical (externally visible) mutation of an object. A CPU running operations or loading memory would not be a good candidate for this, as each operation such as "load memory", "run instruction" or whatever will alter the logical state of the CPU.
See the std::cell documentation for further discussion of when you might want interior mutability.