I'm currently building a HTTP service exposing actions on a unique object.
I already created the central object, with several methods taking immutable &self references, and using internally various efficient synchronization structures to access the data inside (all the code is unsafe-free). My thought was that this would be enough to make it safe to use concurrently.
And then comes the hard part of actually connecting it to a HTTP server.
I'm currently trying to use Iron, but I could switch to Nickel.rs or any other if it makes things easier.
Most HTTP server examples I saw used stateless handlers, without any access to local variables. I now understand why: it's near-impossible to do.
Here is an example of what I'd like to do, using Nickel.rs:
https://gist.github.com/Gyscos/42510a335098ce935848
Here is a similar failed attempt using Iron:
https://gist.github.com/Gyscos/92e56e95baee0ebce78f
The basic idea being that obj only lives for the duration of the scope, but so does server, so it shouldn't be a big deal... right?
Unfortunately each of my attempts failed. When trying to give the server a closure that accesses self.object, I get an error saying that the closure might outlive the reference.
I saw Iron provided a shared memory module with a Read structure. Not only does it look overly complicated for my needs, I also would like to avoid paying the Arc price when I don't need it (I have a clear view of the lifecycle of the object and really don't need to count references).
The current solution I see would be to have a static Object and use that instead of one specific to MyServer, but I'd prefer to avoid this level of ugliness if possible.
I come from golang, where this is not a problem (I could use object-bound methods as handler for that). So my question is: how do you easily access a shared immutable reference from your HTTP handlers in Rust?
Note that I have no prior experience with Nickel or Iron (personally I'm using SCGI so far).
The error I got compiling your Nickel example is:
<nickel macros>:7:9: 7:70 error: captured variable `obj` does not outlive the enclosing closure
The error happens in the following snippet:
server.utilize(router! {
get "/foo" => |_req, _res| {
obj.foo();
}
});
Now, router! is just a fancy macros to wrap your closure in a Middleware, with some additional checks built in. For our investigation we might want to get to the roots and use the Middleware directly.
Unfortunately Nickel explicitly requires the Middleware to have a 'static lifetime. That's a quirk of the Nickel API design and doesn't have much with Rust per se (except for the fact that Rust allows the library to require such things from the user).
I see two options then. First is to use our superior knowledge of the object lifetime (Nickel doesn't know that our object outlives the server but we do) and tell the compiler so. Compiler allows us to show our superiour knowledge with the helpful unsafe and transmute primitives.
Here it is, working: unsafe.rs.
In Rust unsafe means "this piece of code is safe because the programmer said so". Every unsafe piece must satisfy this safety contract between the programmer and the compiler. In this case we know that the object outlives the server so the safety guarantee is maintained.
The second option is to pay the price for tricks that would satisfy the requirements of the Nickel API. Here we use a scoped thread-local storage for this: thread_local.rs.
Bottom line: Nickel API has requirements that make you jump through some hoops to get you where you want. I haven't investigated the Iron API. I have a feeling that you might have a better luck with the lower-level Hyper API.
Related
I am trying to build an Application in Rust with gtk-rs. When the user creates a new project the path to the project root needs to be stored in some way so that it is accessible to all of the application.
Things I have tried to solve this:
Create a mutable static String (unsafe and thus probably not a viable solution)
Store the variable into a file which is slow and stupid for obvious reasons. Works though...
Use lazy_static! as suggested in This Post. But it also says that this is generally not the preferred way. So I tried the next thing.
Create a mut project_root: &mut String and pass it to any function that needs it. Now the issue with this is, when I need to call new_proj_menu_item.connect_activate(move |_| new_project_listener::add_listener(&file_tree_view, project_root)); to set up the listener for the menu Item, the compiler tells me: "project_root has an anonymous lifetime '_ but it needs to satisfy a 'static lifetime requirement E0759 ...is captured here... Note: ...and is required to live as long as 'static here" Which I do not fully understand but seems to be related to the first thing I tried (project_root is the String).
Now my question is: How would I go about storing a variable like this so that it is accessible from the entire project? Is there maybe a GTK internal way that I am unaware of?
I don't know why storing the variable into a file is slow and stupid. It's the standard as far as I know, you typically access that file less that once per minute. If you mean communicating between threads or functions, you definitely should redesign the app. Use mpsc.
I'm pretty new to Rust and have a couple different implementations of a method that includes a closure referencing self. To use the reference in the closure effectively, I've been using Arc<Self> (I am multithreading) and Pin<Arc<Self>>.
I would like to make this method as generally memory efficient as possible. I assume pinning the Arc in memory would help with this. However, (a) I've read that Arcs are pinned and (b) it seems like Pin<Arc<T>> may require additional allocations.
What is Pin<Arc<T>> good for?
Adding Pin around some pointer type does not change the behavior of the program. It only adds a restriction on what further code you can write (and even that, only if the T in Pin<Arc<T>> is not Unpin, which most types are).
Therefore, there is no "memory efficiency" to be gained by adding Pin.
The only use of Pin is to allow working with types that require they be pinned to use them, such as Futures.
I would like to make it a compiler error to allow a type to be dropped, instead it must be forgotten. My use case is for a type the represents a handle of sorts that must be returned to its source for cleanup. This way a user of the API cannot accidentally leak the handle. They would be required to either return the handle to its source or explicitly forget it. In the source, the associated resources would be cleaned up and the handle explicitly forgotten.
The article The Pain Of Real Linear Types in Rust mentions this. Relevant quote:
One extreme option that I've seen is to implement drop() as
abort("this value must be used"). All "proper" consumers then
mem::forget the value, preventing this "destructor bomb" from going
off. This provides a dynamic version of strict must-use values.
Although it's still vulnerable to the few ways destructors can leak,
this isn't a significant concern in practice. Mostly it just stinks
because it's dynamic and Rust users Want Static Verification.
Ultimately, Rust lacks "proper" support for this kind of type.
So, assuming you want static checks, the answer is no.
You could require the user to pass a function object that returns the handle (FnOnce(Handle) -> Handle), as long as there aren't any other ways to create a handle.
I understand that the preferred way to implement something like a global/instance/module variable in Rust is to create said variable in main() or other common entry point and then pass it down to whoever needs it.
It also seems possible to use a lazy_static for an immutable variable, or it can be combined with a mutex to implement a mutable one.
In my case, I am using Rust to create a .so with bindings to Python and I need to have a large amount of mutable state stored within the Rust library (in response to many different function calls invoked by the Python application).
What is the preferred way to store that state?
Is it only via the mutable lazy_static approach since I have no main() (or more generally, any function which does not terminate between function calls from Python), or is there another way to do it?
Bundle it
In general, and absent other requirements, the answer is to bundle your state in some object and hand it over to the client. A popular name is Context.
Then, the client should have to pass the object around in each function call that requires it:
Either by defining the functionality as methods on the object.
Or by requiring the object as parameter of the functions/methods.
This gives full control to the client.
The client may end up creating a global for it, or may actually appreciate the flexibility of being able to juggle multiple instances.
Note: There is no need to provide any access to the inner state of the object; all the client needs is a handle (ref-counted, in Python) to control the lifetime and decide when to use which handle. In C, this would be a void*.
Exceptions
There are cases, such as a cache, where the functionality is not impacted, only the performance.
In this case, while the flexibility could be appreciated, it may be more of a burden than anything. A global, or thread-local, would then make sense.
I'd be tempted to dip into unsafe code here. You cannot use non-static lifetimes, as the lifetime of your state would be determined by the Python code, which Rust can't see. On the other hand, 'static state has other problems:
It necessarily persists until the end of the program, which means there's no way of recovering memory you're no longer using.
'static variables are essentially singletons, making it very difficult to write an application that makes multiple independent usages of your library.
I would go with a solution similar to what #Matthieu M. suggests, but instead of passing the entire data structure back and forth over the interface, allocate it on the heap, unsafely, and then pass some sort of handle (i.e. pointer) back and forth.
You would probably want to write a cleanup function, and document your library to compel users to call the cleanup function when they're done using a particular handle. Effectively, you're explicitly delegating the management of the lifecycle of the data to the calling code.
With this model, if desired, an application could create, use, and cleanup multiple datasets (each represented by their own handle) concurrently and independently. If an application "forgets" to cleanup a handle when finished, you have a memory leak, but that's no worse than storing the data in a 'static variable.
There may be helper crates and libraries to assist with doing this sort of thing. I'm not familiar enough with rust to know.
I am reading the Rust Book and everything was pretty simple to understand (thanks to the book's authors), until the section about lifetimes. I spent all day, reading a lot of articles on lifetimes and still I am very insecure about using them correctly.
What I do understand, though, is that the concept of explicit lifetime specifiers aims to solve the problem of dangling references. I also know that Rust has reference-counting smart pointers (Rc) which I believe is the same as shared_ptr in C++, which has the same purpose: to prevent dangling references.
Given that those lifetimes are so horrendous to me, and smart pointers are very familiar and comfortable for me (I used them in C++ a lot), can I avoid the lifetimes in favor of smart pointers? Or are lifetimes an inevitable thing that I'll have to understand and use in Rust code?
are lifetimes an inevitable thing that I'll have to understand and use in Rust code?
In order to read existing Rust code, you probably don't need to understand lifetimes. The borrow-checker understands them so if it compiles then they are correct and you can just review what the code does.
I am very insecure about using them correctly.
The most important thing to understand about lifetimes annotations is that they do nothing. Rather, they are a way to express to the compiler the relationship between references. For example, if an input and output to a function have the same lifetime, that means that the output contains a reference to the input (or part of it) and therefore is not allowed to live longer than the input. Using them "incorrectly" means that you are telling the compiler something about the lifetime of a reference which it can prove to be untrue - and it will give you an error, so there is nothing to be insecure about!
can I avoid the lifetimes in favor of smart pointers?
You could choose to avoid using references altogether and use Rc everywhere. You would be missing out on one of the big features of Rust: lifetimes and references form one of the most important zero-cost abstractions, which enable Rust to be fast and safe at the same time. There is code written in Rust that nobody would attempt to write in C/C++ because a human could never be absolutely certain that they haven't introduced a memory bug. Avoiding Rust references in favour of smart pointers will mostly result in slower code, because smart pointers have runtime overhead.
Many APIs use references. In order to use those APIs you will need to have at least some grasp of what is going on.
The best way to understand is just to write code and gain an intuition from what works and what doesn't. Rust's error messages are excellent and will help a lot with forming that intuition.