I am trying to build an Application in Rust with gtk-rs. When the user creates a new project the path to the project root needs to be stored in some way so that it is accessible to all of the application.
Things I have tried to solve this:
Create a mutable static String (unsafe and thus probably not a viable solution)
Store the variable into a file which is slow and stupid for obvious reasons. Works though...
Use lazy_static! as suggested in This Post. But it also says that this is generally not the preferred way. So I tried the next thing.
Create a mut project_root: &mut String and pass it to any function that needs it. Now the issue with this is, when I need to call new_proj_menu_item.connect_activate(move |_| new_project_listener::add_listener(&file_tree_view, project_root)); to set up the listener for the menu Item, the compiler tells me: "project_root has an anonymous lifetime '_ but it needs to satisfy a 'static lifetime requirement E0759 ...is captured here... Note: ...and is required to live as long as 'static here" Which I do not fully understand but seems to be related to the first thing I tried (project_root is the String).
Now my question is: How would I go about storing a variable like this so that it is accessible from the entire project? Is there maybe a GTK internal way that I am unaware of?
I don't know why storing the variable into a file is slow and stupid. It's the standard as far as I know, you typically access that file less that once per minute. If you mean communicating between threads or functions, you definitely should redesign the app. Use mpsc.
Related
If I print x.passwd, I will get 234
If I print y.passwd, I will get 234 too, But how is that possible since y = &x (essentially storing the address of x), shouldnt I be dereferencing in order to access passwd like (*y).passwd?
I was solving a leetcode problem and they were accessing a node's val field directly by the reference without dereferencing and that made me more confused about references.
On Left hand size, we have Option<Box> while on the right we have &Option<Box>, How can we perform Some(node) = node
PS: I Hope someone explains with a memory diagram of what is actually happening. And if anyone has good resources to understand references and borrowing, Please Let me know, I have been referring the docs and Lets Get Rusty youtube channel but still references are little confusing for me.
In Rust, the . operator will automatically dereference as necessary to find something with the right name. In y.passwd, y is a reference, but references don't have any named fields, so the compiler tries looking at the type of the referent — Cred — and does find the field named passwd.
The same thing works with methods, but there's some more to it — in addition to dereferencing, the compiler will also try adding an & to find a matching method. That way you don't have to write the awkward (&foo).bar() just to call a method that takes &self, similar to how you've already found that you don't have to write (*y).passwd.
In general, you rarely (but not never) have to worry about whether or not something is a reference, when using ..
When providing callbacks to JavaScript using Closures, what's a better way to deal with avoiding freeing them? The wasm-bindgen guide suggests using .forget, but admits that that is essentially leaking memory.
Normally we'd store the handle to later get dropped at an appropriate time but for now we want it to be a global handler so we use the forget method to drop it without invalidating the closure. Note that this is leaking memory in Rust, so this should be done judiciously!
It hints at storing the closure until a time when it's appropriate to be dropped. In alexcrichton's answer to a previous question, he mentions...
[...] if it's [...] only invoked once, then you can use Rc/RefCell to drop the Closure inside the the closure itself (using some interior mutability shenanigans)
But he doesn't provide an example of this method.
The Closure documentation also gives an example of returning the reference to the closure to the JavaScript context to let it handle when to free the reference.
If we were to drop cb here it would cause an exception to be raised whenever the interval elapses. Instead we return our handle back to JS so JS can decide when to cancel the interval and deallocate the closure.
I'd also imagine there are ways to use features like lifetimes or the #[wasm_bindgen] macro on a public function to avoid this issue as well, but I'm having trouble figuring out how to do it that way.
My question is, what are the alternatives to using .forget with closures that are being passed back to JavaScript from Rust, and can I please see some simple examples of each option in use?
I recently built a small commercial app and was stuck on this for weeks and was really excited when I got this working. I ended up using Closure.once_into_js. However, that also has the caveat that "The only way the FnOnce is deallocated is by calling the JavaScript function. If the JavaScript function is never called then the FnOnce and everything it closes over will leak." So if the callback is called, everything should be fine, but if not, there is still a memory leak. I found the programming style to be pretty nice. I mapped the JavaScript functions to Rust like this:
#[wasm_bindgen]
fn getSomething(details: &JsValue, callback: JsValue);
pub fn get_something(details: &Details, callback: impl Fn(Option<String>) + 'static){
getSomething(&serde_wasm_bindgen::to_value(details).unwrap(), Closure::once_into_js(move |v: JsValue|
callback(serde_wasm_bindgen::from_value(v).unwrap())
));
}
And then I'm able to use it from Rust in my app like so:
let callback = move |id| {
};
get_something(&details, callback);
I defined the callbacks as static impl functions and then move the values in.
I understand that the preferred way to implement something like a global/instance/module variable in Rust is to create said variable in main() or other common entry point and then pass it down to whoever needs it.
It also seems possible to use a lazy_static for an immutable variable, or it can be combined with a mutex to implement a mutable one.
In my case, I am using Rust to create a .so with bindings to Python and I need to have a large amount of mutable state stored within the Rust library (in response to many different function calls invoked by the Python application).
What is the preferred way to store that state?
Is it only via the mutable lazy_static approach since I have no main() (or more generally, any function which does not terminate between function calls from Python), or is there another way to do it?
Bundle it
In general, and absent other requirements, the answer is to bundle your state in some object and hand it over to the client. A popular name is Context.
Then, the client should have to pass the object around in each function call that requires it:
Either by defining the functionality as methods on the object.
Or by requiring the object as parameter of the functions/methods.
This gives full control to the client.
The client may end up creating a global for it, or may actually appreciate the flexibility of being able to juggle multiple instances.
Note: There is no need to provide any access to the inner state of the object; all the client needs is a handle (ref-counted, in Python) to control the lifetime and decide when to use which handle. In C, this would be a void*.
Exceptions
There are cases, such as a cache, where the functionality is not impacted, only the performance.
In this case, while the flexibility could be appreciated, it may be more of a burden than anything. A global, or thread-local, would then make sense.
I'd be tempted to dip into unsafe code here. You cannot use non-static lifetimes, as the lifetime of your state would be determined by the Python code, which Rust can't see. On the other hand, 'static state has other problems:
It necessarily persists until the end of the program, which means there's no way of recovering memory you're no longer using.
'static variables are essentially singletons, making it very difficult to write an application that makes multiple independent usages of your library.
I would go with a solution similar to what #Matthieu M. suggests, but instead of passing the entire data structure back and forth over the interface, allocate it on the heap, unsafely, and then pass some sort of handle (i.e. pointer) back and forth.
You would probably want to write a cleanup function, and document your library to compel users to call the cleanup function when they're done using a particular handle. Effectively, you're explicitly delegating the management of the lifecycle of the data to the calling code.
With this model, if desired, an application could create, use, and cleanup multiple datasets (each represented by their own handle) concurrently and independently. If an application "forgets" to cleanup a handle when finished, you have a memory leak, but that's no worse than storing the data in a 'static variable.
There may be helper crates and libraries to assist with doing this sort of thing. I'm not familiar enough with rust to know.
A recent question was looking for the ability to construct self-referential structures. In discussing possible answers for the question, one potential answer involved using an UnsafeCell for interior mutability and then "discarding" the mutability through a transmute.
Here's a small example of such an idea in action. I'm not deeply interested in the example itself, but it's just enough complication to require a bigger hammer like transmute as opposed to just using UnsafeCell::new and/or UnsafeCell::into_inner:
use std::{
cell::UnsafeCell, mem, rc::{Rc, Weak},
};
// This is our real type.
struct ReallyImmutable {
value: i32,
myself: Weak<ReallyImmutable>,
}
fn initialize() -> Rc<ReallyImmutable> {
// This mirrors ReallyImmutable but we use `UnsafeCell`
// to perform some initial interior mutation.
struct NotReallyImmutable {
value: i32,
myself: Weak<UnsafeCell<NotReallyImmutable>>,
}
let initial = NotReallyImmutable {
value: 42,
myself: Weak::new(),
};
// Without interior mutability, we couldn't update the `myself` field
// after we've created the `Rc`.
let second = Rc::new(UnsafeCell::new(initial));
// Tie the recursive knot
let new_myself = Rc::downgrade(&second);
unsafe {
// Should be safe as there can be no other accesses to this field
(&mut *second.get()).myself = new_myself;
// No one outside of this function needs the interior mutability
// TODO: Is this call safe?
mem::transmute(second)
}
}
fn main() {
let v = initialize();
println!("{} -> {:?}", v.value, v.myself.upgrade().map(|v| v.value))
}
This code appears to print out what I'd expect, but that doesn't mean that it's safe or using defined semantics.
Is transmuting from a UnsafeCell<T> to a T memory safe? Does it invoke undefined behavior? What about transmuting in the opposite direction, from a T to an UnsafeCell<T>?
(I am still new to SO and not sure if "well, maybe" qualifies as an answer, but here you go. ;)
Disclaimer: The rules for these kinds of things are not (yet) set in stone. So, there is no definitive answer yet. I'm going to make some guesses based on (a) what kinds of compiler transformations LLVM does/we will eventually want to do, and (b) what kind of models I have in my head that would define the answer to this.
Also, I see two parts to this: The data layout perspective, and the aliasing perspective. The layout issue is that NotReallyImmutable could, in principle, have a totally different layout than ReallyImmutable. I don't know much about data layout, but with UnsafeCell becoming repr(transparent) and that being the only difference between the two types, I think the intent is for this to work. You are, however, relying on repr(transparent) being "structural" in the sense that it should allow you to replace things in larger types, which I am not sure has been written down explicitly anywhere. Sounds like a proposal for a follow-up RFC that extends the repr(transparent) guarantees appropriately?
As far as aliasing is concerned, the issue is breaking the rules around &T. I'd say that, as long as you never have a live &T around anywhere when writing through the &UnsafeCell<T>, you are good -- but I don't think we can guarantee that quite yet. Let's look in more detail.
Compiler perspective
The relevant optimizations here are the ones that exploit &T being read-only. So if you reordered the last two lines (transmute and the assignment), that code would likely be UB as we may want the compiler to be able to "pre-fetch" the value behind the shared reference and re-use that value later (i.e. after inlining this).
But in your code, we would only emit "read-only" annotations (noalias in LLVM) after the transmute comes back, and the data is indeed read-only starting there. So, this should be good.
Memory models
The "most aggressive" of my memory models essentially asserts that all values are always valid, and I think even that model should be fine with your code. &UnsafeCell is a special case in that model where validity just stops, and nothing is said about what lives behind this reference. The moment the transmute returns, we grab the memory it points to and make it all read-only, and even if we did that "recursively" through the Rc (which my model doesn't, but only because I couldn't figure out a good way to make it do so) you'd be fine as you don't mutate any more after the transmute. (As you may have noticed, this is the same restriction as in the compiler perspective. The point of these models is to allow compiler optimizations, after all. ;)
(As a side-note, I really wish miri was in better shape right now. Seems I have to try and get validation to work again in there, because then I could tell you to just run your code in miri and it'd tell you if that version of my model is okay with what you are doing :D )
I am thinking about other models currently that only check things "on access", but haven't worked out the UnsafeCell story for that model yet. What this example shows is that the model may have to contain ways for a "phase transition" of memory first being UnsafeCell, but later having normal sharing with read-only guarantees. Thanks for bringing this up, that will make for some nice examples to think about!
So, I think I can say that (at least from my side) there is the intent to allow this kind of code, and doing so does not seem to prevent any optimizations. Whether we'll actually manage to find a model that everybody can agree with and that still allows this, I cannot predict.
The opposite direction: T -> UnsafeCell<T>
Now, this is more interesting. The problem is that, as I said above, you must not have a &T live when writing through an UnsafeCell<T>. But what does "live" mean here? That's a hard question! In some of my models, this could be as weak as "a reference of that type exists somewhere and the lifetime is still active", i.e., it could have nothing to do with whether the reference is actually used. (That's useful because it lets us do more optimizations, like moving a load out of a loop even if we cannot prove that the loop ever runs -- which would introduce a use of an otherwise unused reference.) And since &T is Copy, you cannot even really get rid of such a reference either. So, if you have x: &T, then after let y: &UnsafeCell<T> = transmute(x), the old x is still around and its lifetime still active, so writing through y could well be UB.
I think you'd have to somehow restrict the aliasing that &T allows, very carefully making sure that nobody still holds such a reference. I'm not going to say "this is impossible" because people keep surprising me (especially in this community ;) but TBH I cannot think of a way to make this work. I'd be curious if you have an example though where you think this is reasonable.
I'm currently building a HTTP service exposing actions on a unique object.
I already created the central object, with several methods taking immutable &self references, and using internally various efficient synchronization structures to access the data inside (all the code is unsafe-free). My thought was that this would be enough to make it safe to use concurrently.
And then comes the hard part of actually connecting it to a HTTP server.
I'm currently trying to use Iron, but I could switch to Nickel.rs or any other if it makes things easier.
Most HTTP server examples I saw used stateless handlers, without any access to local variables. I now understand why: it's near-impossible to do.
Here is an example of what I'd like to do, using Nickel.rs:
https://gist.github.com/Gyscos/42510a335098ce935848
Here is a similar failed attempt using Iron:
https://gist.github.com/Gyscos/92e56e95baee0ebce78f
The basic idea being that obj only lives for the duration of the scope, but so does server, so it shouldn't be a big deal... right?
Unfortunately each of my attempts failed. When trying to give the server a closure that accesses self.object, I get an error saying that the closure might outlive the reference.
I saw Iron provided a shared memory module with a Read structure. Not only does it look overly complicated for my needs, I also would like to avoid paying the Arc price when I don't need it (I have a clear view of the lifecycle of the object and really don't need to count references).
The current solution I see would be to have a static Object and use that instead of one specific to MyServer, but I'd prefer to avoid this level of ugliness if possible.
I come from golang, where this is not a problem (I could use object-bound methods as handler for that). So my question is: how do you easily access a shared immutable reference from your HTTP handlers in Rust?
Note that I have no prior experience with Nickel or Iron (personally I'm using SCGI so far).
The error I got compiling your Nickel example is:
<nickel macros>:7:9: 7:70 error: captured variable `obj` does not outlive the enclosing closure
The error happens in the following snippet:
server.utilize(router! {
get "/foo" => |_req, _res| {
obj.foo();
}
});
Now, router! is just a fancy macros to wrap your closure in a Middleware, with some additional checks built in. For our investigation we might want to get to the roots and use the Middleware directly.
Unfortunately Nickel explicitly requires the Middleware to have a 'static lifetime. That's a quirk of the Nickel API design and doesn't have much with Rust per se (except for the fact that Rust allows the library to require such things from the user).
I see two options then. First is to use our superior knowledge of the object lifetime (Nickel doesn't know that our object outlives the server but we do) and tell the compiler so. Compiler allows us to show our superiour knowledge with the helpful unsafe and transmute primitives.
Here it is, working: unsafe.rs.
In Rust unsafe means "this piece of code is safe because the programmer said so". Every unsafe piece must satisfy this safety contract between the programmer and the compiler. In this case we know that the object outlives the server so the safety guarantee is maintained.
The second option is to pay the price for tricks that would satisfy the requirements of the Nickel API. Here we use a scoped thread-local storage for this: thread_local.rs.
Bottom line: Nickel API has requirements that make you jump through some hoops to get you where you want. I haven't investigated the Iron API. I have a feeling that you might have a better luck with the lower-level Hyper API.