I'm trying to write a library that will connect to remote servers and exchange data. I did this in C++ using Boost::Asio and am trying to do the same with Rust.
One of the problems I have is mapping concepts from Asio, like async_write/read to Tokio, starting with the fact that seemingly all Tokio examples demand that I replace my main() with an async main(), while I would like to encapsulate all my async code in structures and associated implementations.
Is it possible to use Tokio without replacing main()? Is mio perhaps the only way?
You can create a runtime manually using Runtime::new() which is what the tokio main macro is doing under the hood. It's just for an awful lot of apps, especially examples that's just boilerplate. So the macro automates the simple case.
However, depending on the context of your library, it may be more idiomatic to provide a future based API, and then leave the app consumer to set up the runtime.
Related
The documentation for the tokio-tungstenite crate mostly just directs you to the examples section.
The client example hasn't been updated in a few years and I saw some unfamiliar async code here and because Rust's async APIs are still in flux I wanted to check the use futures_util::future::select was still idiomatic in this situation.
In particular, it seems to work fine if I replace it with the more commonly seen tokio::select! macro, which doesn't require pinning the passed futures.
I have good familiarity with tokio's APIs but not so much the lower level futures ones. Is there a reason to manually pin here and use the future::select instead? More generally, in modern idiomatic async Rust code when would one use the latter?
It should be fine to replace any use of futures::future::select() with tokio::select! (or futures::select!). They both do the same thing. select! allow for more than one future, while select() gives you a named future. I prefer select! when I can (I think it is also more performant, but I'm not sure).
My first experience doing a computer system project was building a server using vanilla Java and then a client on an Android phone. Since then, I've found that there are a lot of frameworks to help manage scalability and remove the need to write boilerplate code.
I'm trying to understand what services like Tokio and Rayon enable.
I came across this paragraph on the Tokio tutorial page and I'm having a hard time understanding it
When you write your application in an asynchronous manner, you enable it to scale much better by reducing the cost of doing many things at the same time. However, asynchronous Rust code does not run on its own, so you must choose a runtime to execute it.
I first thought a "runtime" might refer to where the binary can run, but it looks like Tokio just provides functions that are already available in the Rust standard library while Rayon implements functions that aren't in the standard library.
Are the standard implementations for asynchronous functions written poorly in the standard library or am I not understanding what service Tokio is providing?
Rust currently does not provide an async runtime in the standard library. For full details, see Asynchronous Programming in Rust, and particularly the chapter on "The Async Ecosystem."
Rust currently provides only the bare essentials for writing async code. Importantly, executors, tasks, reactors, combinators, and low-level I/O futures and traits are not yet provided in the standard library. In the meantime, community-provided async ecosystems fill in these gaps.
Rust has very strict backward compatibility requirements, and they haven't chosen to lock-in a specific runtime. There are reasons to pick one over another (features versus size for example), and making it part of the standard library would impose certain choices that aren't clearly the right ones for all projects. This may change in the future as the community projects better explore this space and help determine the best mix of choices without the strong backward compatibility promises.
I'm building a library to be used on a no_std platform which allows you to do some common network-related IO, such as making HTTP requests or reading from/writing to Websockets.
Now, I would like this library to be a well-behaved citizen so that it can be easily included in other no_std applications. I hence want to package the library by implementing reasonable traits etc. The library would allow me to not have to use alloc, so supporting non-alloc no_std would be ideal.
These are the options I have looked at:
embedded_hal and nb: These crates are really low level (no generic traits like Read and Write or anything higher level) and the async model doesn't seem to be compatible with async/await
genio/core_io/...: These don't support async IO at all.
embrio: Seems interesting but it seems like using it would tie me to one specific environment, making the library less portable.
tokio v0.2.x: I would love to use it but there is no no_std support at all.
futures::io v0.3.x: Again, would love to use it but there is no no_std support.
Which async IO abstraction should I use in a no_std environment? If there is no good option right now, which one should I bet on/help out with for the future?
Take a look a look at embassy-rs. There is a very active community. Currently, embassy-rs supports;
Hardware abstraction layers
Time
Networking
Bluetooth
Lora
USB
DFU
All built on rust async. There are also some really nice macros to generate static buffers for tasks, so you don't need alloc.
Once you have coroutines you can create pipelines (haskell: pipes, conduits; python: generators) or cooperative event loops (python: curio). Once you have futures, it appears you can do the same; pipelines (rust: futures-rs) and event loops (rust: tokio). Since futures aren't cooperative they require a callback-based (even poll-based futures require callbacks) scheduler to execute blocking tasks within a thread or process pool. What benefits are there to combining futures (library-level) with coroutines (language-level) as these languages do: (python: asyncio), (rust: rfc), (ecmascript 6+). Fundamentally they seem to be conflicting solutions to the same problem.
I'm not looking for a pro/con comparison, and I don't buy the argument that futures are "one-shot" coroutines. Just look at rust, which built an entire state-machine-based event framework using just futures. I want to know why python/asyncio and javascript both require coroutines together with futures. Why rust is planning on adding coroutines to its futures? Does it have to do with composability of events? Or the implicit stack of coroutines versus the explicit stack of continuation-passing futures? Not that I completely understand this argument, as both futures and coroutines are implemented using continuations... Or does it have something to do with direct vs indirect style?
These are all different (though related) ideas with different amounts of power.
A future is an abstraction that lets you begin a process and then yield back to a handler, that is chosen by the original caller, when the process is done.
A generator is more powerful than a future because it can yield multiple times. You can implement futures on top of generators.
A coroutine is more powerful than a generator because it can choose who to yield to, instead of only to the caller. For example it can yield to another coroutine. You can implement generators on top of coroutines.
Why would you use the less powerful tool, when more powerful ones are available? Sometimes the less powerful tool is the right tool for the job. It's useful to statically encode your program's invariants using types, because it can give you certainty about what something can't do.
For example, when making a REST call to a remote server, a future is probably sufficient. If the REST client exposed a generator, you'd have to deal with the possibility that it could yield multiple times, even though you know there is only going to be one result. If it exposed a coroutine, you'd have to consult the documentation to work out exactly how you're supposed to interact with it - even though you actually only need to do one thing, which is obvious when you're dealing with a future.
The lexer/parser file located here is quite large and I'm not sure if it is suitable for just retrieving a list of Rust functions. Perhaps writing my own/using another library would be a better route to take?
The end objective would be to create a kind of execution manager. To contextualise, it would be able to read a list of function calls wrapped in a function. The function calls that are within the function will then be able to be re/ordered from some web interface. Thought it might be nice to manage larger applications this way.
No. I mean, not really. Whether you write your own parser or re-use syntex, you're going to hit a fundamental limitation: macros.
So let's say you go all-out and expand macro_rules!-based macros, including the ones defined in external crates (which means you'll also need to extract rustc's crate metadata loading... which isn't stable). What about procedural macros and custom derive attributes? Those are defined in code and depend on compiler-internal interfaces to function.
The only way this is likely to ever work correctly is if you build on top of the compiler, or duplicate a huge amount of work (which also involves unstable binary interfaces).
You could use syntex to parse the Rust code in a build script.