Saving changed variables between runtime Rust [closed] - rust

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 months ago.
Improve this question
Is there an elegant way to save a variable/s so that a state is reflected between executions. I think reading and updating a config file (yaml) is a pretty good solution but for just a couple of variables it seems a bit of a hastle. Any thoughts?

I think you misuse term runtimes, I think what you mean is executions. But I think I get what you are trying to ask. This is not Rust-specific. Executables in general are stateless, they do not store any information between executions.
So if you want to persist state between executions, you need to store it somewhere. The options are many. The most common ones are files or databases.
If it's really just a couple of variables the easiest way would probably be json via the awesome serde_json library. With it, you can convert normal Rust structs into human readable ASCII text and store it in file.
You could even wrap it in a struct that can load/store the state.
Although you have to be careful with concurrent access then.
Here is a somewhat simple example of a struct that can persists its state in a file:
use serde::{Deserialize, Serialize};
use std::{fs::File, io::BufReader, path::Path};
const STATE_FILENAME: &'static str = "state.json";
#[derive(Serialize, Deserialize)]
struct PersistentState {
counter: u32,
}
impl PersistentState {
fn load() -> Self {
let path = Path::new(STATE_FILENAME);
if path.exists() && path.is_file() {
let file = File::open(path).unwrap();
serde_json::from_reader(BufReader::new(file)).unwrap()
} else {
Self::default()
}
}
fn store(&self) {
let file = File::create(STATE_FILENAME).unwrap();
serde_json::to_writer(file, self).unwrap();
}
}
impl Default for PersistentState {
fn default() -> Self {
Self { counter: 0 }
}
}
fn main() {
let mut state = PersistentState::load();
state.counter += 1;
println!("Counter: {}", state.counter);
state.store();
}
First execution:
Counter: 1
Second execution:
Counter: 2
state.json file:
{"counter":2}
Of course don't just copy-paste the code into your project but instead think about your usecase and adapt it to your needs.
Pitfalls
Not all types are losslessly storable in json. Most are, like strings, integers and booleans, but floats are not. Floats are stored in human readable decimal, but handled in binary, and there is always a small rounding error in the conversion. But unless you are doing scientific calculations, that is most likely irrelevant.
If you don't like json, serde supports a bunch of other types as well.

Related

Stack overflow occuring in my Rust programm expecially when using `tokio` in combination with `crossbeam` channels [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 months ago.
Improve this question
I have encountered a very weird problem, while using crossbeam channels in combination with tokio.
Seemingly without allocating much memory, my program just stack overflows in debug mode.
But when I decided to replace the unbounded channels with bounded ones and thus allocating more memory to the stack everything seems to work, even in debug mode.
On another application I am writing, this is happening too, but without using tokio and I am required to run it in release mode.
I tried to wrap everything that might consume a lot of memory and is stack-allocated in a Box, but without any success.
My question is if I should be worried about this, or just run my program in release mode without any further concerns.
I am writing a server for my game, so stability is very important.
EDIT
I found out that the rust compiler apparently performs "tail-call optimization" which allow an infinite loop in release mode, source
And because I am running 2 infinite loops in both programs this explains, that they only work in release mode I think.
minimal working example
communication.rs
use crate::channel::*;
use std::mem::forget;
use tokio::net::{TcpStream, ToSocketAddrs};
use tokio::runtime;
use tokio::io::AsyncWriteExt;
pub struct Communicator {
event_sender: Sender<[u8; 255]>,
event_receiver: Receiver<[u8; 255]>,
}
impl Communicator {
pub fn init<H: ToSocketAddrs>(host: H) -> Option<Self> {
let rt = runtime::Builder::new_multi_thread()
.enable_io()
.build()
.unwrap();
if let Some((event_sender, event_receiver)) = rt.block_on(async move {
if let Ok(socket) = TcpStream::connect(host).await {
let (mut read, mut write) = socket.into_split();
let (ev_sender_tx, ev_sender_rx): (Sender<[u8; 255]>, Receiver<[u8; 255]>) = channel();
let (ev_receiver_tx, ev_receiver_rx) = channel();
// rx
tokio::spawn(async move {
loop {
let mut buffer: [u8; 255] = [0; 255];
ev_receiver_tx.send(buffer).unwrap();
}
});
// tx
tokio::spawn(async move {
loop {
if let Some(event) = ev_sender_rx.recv() {
write.write_all(&event).await;
}
}
});
Some((ev_sender_tx, ev_receiver_rx))
} else {
None
}
}) {
// not allowed to run destructor, must leak
forget(rt);
Some(Self { event_sender, event_receiver })
} else {
None
}
}
}
channel.rs
use crossbeam::channel;
use crossbeam::channel::SendError;
#[derive(Debug, Clone)]
pub struct Receiver<T> {
inner: channel::Receiver<T>,
}
impl<T> Receiver<T> {
pub fn recv(&self) -> Option<T> {
self.inner.recv().ok()
}
#[allow(dead_code)]
pub fn try_recv(&self) -> Option<T> {
self.inner.try_recv().ok()
}
}
#[derive(Debug, Clone)]
pub struct Sender<T> {
inner: channel::Sender<T>,
}
impl<T> Sender<T> {
pub fn send(&self, data: T) -> Result<(), SendError<T>> {
self.inner.send(data)
}
}
pub fn channel<T>() -> (Sender<T>, Receiver<T>) {
let (s, r) = channel::unbounded();
(
Sender {inner: s},
Receiver {inner: r},
)
}
// will block if full
pub fn bounded_channel<T>(bound: usize) -> (Sender<T>, Receiver<T>) {
let (s, r) = channel::bounded(bound);
(
Sender {inner: s},
Receiver {inner: r},
)
}
loop {
let mut buffer: [u8; 255] = [0; 255];
ev_receiver_tx.send(buffer).unwrap();
}
Unless I'm understanding something wrong, this loop just fills up the channel completely. And for an unbounded channel, I'm not surprised that this results in an infinite amount of memory being used. What's the purpose of this loop, or is that just for the example?
If you just run this loop on its own, with an unbounded channel, you can watch the memory get eaten until the program crashes.
But when I decided to replace the unbounded channels with bounded ones and thus allocating more memory to the stack everything seems to work, even in debug mode.
I don't quite follow ... how does replacing the channels change the amount of memory allocated to the stack? This actually limits the amount of memory a channel could take if flooded with items, which seems to be the root cause of the crash in this case. I'm still a bit uncertain, though, because this should cause an out-of-memory error, not a stack overflow.
// not allowed to run destructor, must leak
forget(rt);
That one is also highly suspicious to me. According to its documentation, dropping the runtime handle is the official way of shutting down the runtime, so calling forget on it sounds to me like a memory leak and incorrect behaviour. forget really is one of those functions that don't serve much of a purpose unless paired with some form of unsafe code.
If you added this line because it complained that it cannot destroy it while it is borrowed by event_sender and event_receiver, then that was definitely the wrong way to 'fix' that problem.
I think the general approach (if I understand it correctly) of spawning an async runtime for every connection is not possible the way you implemented it. Especially a multi_thread runtime will spawn a lot of threads as you get more and more connections, and as you forget the runtime handle, those threads will never go away again, even if the users disconnect.
I think a more useful way would be to have a reusable multi_thread runtime somewhere (which is already capable of using 100% of your cpu) and then to use tokio::spawn to create new tasks for the new connection in your init function, as you already do.
My question is if I should be worried about this, or just run my program in release mode without any further concerns.
Yes, in my opinion, you should definitely be worried about this. Normal healthy Rust programs do not just randomly stack overflow in debug mode. This is a strong indication of a programming error.
Try to use memory profiling tools to find out where the memory usage actually comes from.
I found out that the rust compiler apparently performs "tail-call optimization" which allow an infinite loop in release mode
I think you misunderstood that post. It means:
In recursion, the compiler might reduce the usage of the stack by replacing the recursive function with a loop. Here is a discussion about how to the compiler might achieve this.
This does not and under no circumstance mean that the compiler just randomly decides to deadlock in an infinite loop in release. That would be horrible and would take all credibility from the Rust compiler and the language as a whole.
The only reason this article mentioned "infinite" loops is because it was an infinite unbounded recursion to begin with. That's what the article was actually about, to show how fast a stack overflow happens if you cause one on purpose, not how to prevent it.
Unless you perform recursion, it's almost impossible to cause a stack overflow in Rust (and in most languages, for that matter). Memory leaks almost exclusively happen on the heap, as memory usage on the stack is in almost all circumstances already known at compile time and stays within a constant limit. (again, recursions excluded) Further, almost all data structures that store large amounts of data store the data on the heap. Among other things, this is done to prevent excessive copies; memory on the stack usually gets moved around a lot, while memory on the heap stays at a constant location.
I falsely assumed that sending data through an unbounded channel results in the data that is being sent to be stored on the heap and data that is being sent through an bounded channel to be stored on the stack if the datatype allows that.
What seems to fix this issue was to just wrap the data that is sent in a Box and thus forcing a heap allocation.
The code would now look somewhat like this:
use crate::channel::*;
use std::mem::forget;
use tokio::net::{TcpStream, ToSocketAddrs};
use tokio::runtime;
use tokio::io::AsyncWriteExt;
pub struct Communicator {
event_sender: Sender<Box<[u8; 255]>>,
event_receiver: Receiver<Box<[u8; 255]>>,
}
impl Communicator {
pub fn init<H: ToSocketAddrs>(host: H) -> Option<Self> {
let rt = runtime::Builder::new_multi_thread()
.enable_io()
.build()
.unwrap();
if let Some((event_sender, event_receiver)) = rt.block_on(async move {
if let Ok(socket) = TcpStream::connect(host).await {
let (mut read, mut write) = socket.into_split();
let (ev_sender_tx, ev_sender_rx): (Sender<Box<[u8; 255]>>, Receiver<Box<[u8; 255]>>) = channel();
let (ev_receiver_tx, ev_receiver_rx) = channel();
// rx
tokio::spawn(async move {
loop {
let mut buffer: [u8; 255] = [0; 255];
ev_receiver_tx.send(Box::new(buffer)).unwrap();
}
});
// tx
tokio::spawn(async move {
loop {
if let Some(event) = ev_sender_rx.recv() {
write.write_all(&(*event)).await;
}
}
});
Some((ev_sender_tx, ev_receiver_rx))
} else {
None
}
}) {
// not allowed to run destructor, must leak
forget(rt);
Some(Self { event_sender, event_receiver })
} else {
None
}
}
main.rs
mod communicator;
mod channel;
fn main() {
let comm = com::Communicator::init("0.0.0.0:8000");;
std::thread::park();
}
EDIT
The problem was that I had a data structure somewhere else in the code, which tried to store a big 3-dimensional array.
Just converting [[[u16; 32]; 32]; 32] to Box<[[[u16; 32]; 32]; 32]> worked.

Dealing with lifetimes in global variables

I am currently writing a board support package for an embedded board and would like to set up a serial output over USB. I am basing my design on the hifive BSP.
The process to do this is in three steps:
Set up the USB bus (UsbBusAllocator, which refers to a UsbPeripheral)
Initialize a SerialPort instance, which refers to UsbBusAllocator
Initialize a UsbDevice instance, which refers to UsbBusAllocator
To make my life simpler, I wrapped the SerialPort and UsbDevice in a SerialWrapper struct:
pub struct SerialWrapper<'a> {
port: SerialPort<'a, Usbd<UsbPeripheral<'a>>>,
dev: UsbDevice<'a, Usbd<UsbPeripheral<'a>>>,
}
impl<'a> SerialWrapper<'a> {
pub fn new(bus: &'a UsbPort<'a>) -> Self {
// create the wrapper ...
}
}
I would like a way make the structure created by the SerialWrapper::new global.
I tried to use:
static mut STDOUT: Option<SerialWrapper<'a>> = None;
but I can't use this as lifetime 'a is not declared.
I thought about using MaybeUninit or PhantomData but both will still need to have SerialWrapper<'a> as type parameter and I will get the same issue.
My goal is to be able to use code similar to this:
struct A;
struct B<'a> {
s: &'a A,
}
static mut STDOUT: Option<B> = None;
fn init_stdout() {
let a = A {};
unsafe {
STDOUT = Some(B {s: &a});
}
}
// access_stdout is the important function
// all the rest can be changed without issue
fn access_stdout() -> Result<(), ()> {
unsafe {
if let Some(_stdout) = STDOUT {
// do stuff is system ready
Ok(())
} else {
// do stuff is system not ready
Err(())
}
}
}
fn main() {
init_stdout();
let _ = access_stdout();
}
Do you have any suggestions on how to proceed?
I don't mind having a solution requiring unsafe code, as long as I can have safe functions to access my serial port.
Short answer: whenever you have a lifetime in the type of a static variable, that lifetime needs to be 'static.
Rust does not allow for dangling references, so if you having anything living for shorter than the static lifetime in a static variable, there's the possibility for dangling. I think your code will need a fair amount of refactoring to satisfy this requirement. I would recommend figuring out a way to store the data you need without references since that will make your life much easier. If it is absolutely imperative that you store references you'll need to figure out a way to leak the data to extend its lifetime to 'static.
I am not terribly familiar with embedded development, and I know that static mut has some use-cases there, however usage of that language feature is pretty universally frowned upon. static muts are wildly unsafe and even bypass some borrow checker mechanisms by allowing multiple mutable references at the same time. If you were to encode this properly in Rust's type system you'd probably want to make it just static STDOUT: SyncWrapperForUnsafeCell<Option<T>>, and then provide a safe interface (that might involve locking) for your wrapper with comments explaining why your current environment makes it safe. However if you think that a static mut is the appropriate option there I trust your judgement.

How should doc comments be spread over multiple elements? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 months ago.
Improve this question
I need to document a piece of code that is 3-fold. I have an enum element, an associated constructor and a method that uses the enum to do work. Here is a minimal example:
use regex::{Error, Regex};
pub enum Element {
And {
lhs: Box<Element>,
rhs: Box<Element>,
},
Value {
regex: Regex,
},
}
impl Element {
pub fn new_and(lhs: Element, rhs: Element) -> Self {
Element::And {
lhs: Box::new(lhs),
rhs: Box::new(rhs),
}
}
pub fn new_value(regex: &str) -> Result<Self, Error> {
let regex = Regex::new(regex)?;
Ok(Element::Value { regex })
}
pub fn is_match(&self, values: &Vec<String>) -> bool {
match self {
Element::And { lhs, rhs } => lhs.is_match(values) && rhs.is_match(values),
Element::Value { regex } => {
for value in values {
if regex.is_match(value) {
return true;
}
}
false
}
}
}
}
fn main() {}
There definitely needs to be a comment on new_value() to document the possible error. But where should I put information on how the actual element is evaluated?
Put this also in the comment of new_value() to have all in one place?
Put it in a comment for Element::Value and is_match() only says that elements are
evaluated as documented there?
Put all evaluation information for all enum values in the comment for is_match() and leave Element::Value empty?
Repeat the evaluation information in all 3 locations?
Disclaimer: A lot of this will come down to personal preference and taste, there's not really any hard and fast rules
That being said, I like to look at the standard library's documentation for inspiration.
Generally, if a function returns a Result, some explanation of when it returns an error is probably a good idea. For example: Returns an error if the regex fails to parse. You could also look at the docs for Regex::new to see what they say about it.
When documenting the behaviour of is_match, you should probably put the comment on is_match.
But I'd also suggest putting a more comprehensive comment (preferably with a usage example) either in the module (with //! at the top) or on pub enum Element. Examples are really underrated IMO, and even a simple one can really help a reader to understand the purpose of this type.
For example, the docs for String (https://doc.rust-lang.org/std/string/struct.String.html) have quite a lot of detail on various uses of String as doc comments on the String struct itself.

When to use impl on an empty struct [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 12 months ago.
Improve this question
I saw the following code on the internet:
pub struct Processor;
impl Processor {
pub fn process(
) -> i32 {
// some stuff here
}
}
and used as:
let a = Processor::process();
What's the advantage of having struct here at all? Can the same thing somehow be achieved without it?
Here are a couple of the more common examples you might run into. I'm sure there are others I forgot, but these will hopefully help you when reading rust code in the future.
Creating structs to hold different implementations of a trait
You may find yourself in a situation where it would be nice to be able to use a custom handler, but want to avoid the overhead of storing a function inside of each struct. An easy alternative is to create a trait for it instead and define types for the sole purpose of having different versions of a trait implemented on them.
This way you can use them like compile-time type modifiers which allow for core functionality to be easily swapped or redefined later without any extra overhead or requiring extra information be stored in a struct.
trait Smoothing {
fn smooth(a: i32, b: i32) -> i32;
}
struct LinearStrategy;
impl Smoothing for LinearStrategy {
fn smooth(a: i32, b: i32) -> i32 {
(a + b) / 2
}
}
struct GeometricStrategy;
impl Smoothing for GeometricStrategy {
fn smooth(a: i32, b: i32) -> i32 {
i32::sqrt(a * a + b * b)
}
}
struct ComplexStruct<T> {
/* etc. */
_phantom: PhantomData<T>,
}
// Change how ComplexStruct operates at compile time
impl<T: Smoothing> ComplexStruct<T> {
pub fn sample(&self, x: i32) -> i32 {
T::smooth(self.raw_sample(x - 1), self.raw_sample(x + 1))
}
}
The best example I could find of this is probably Vec. It may not seem like it at first, but Vec has 2 type parameters. In Vec<T, A = Global>, the A is the allocator used by Vec. By default it is set to the global allocator, but in some cases it can be really handy to be able to easily switch it out with something else and still have access to all of the normal functionality of Vec.
Placeholders for FFI
When creating a rust api for a C/other library it may make sense to add types to mirror the C api without containing the same data. Normally that will end up looking like this where a pointer is wrapped with a safe rust alternative and a placeholder lifetime (Since rust does not own this data).
pub struct Foo<'a> {
ptr: *mut sys::Foo,
_phantom: PhantomData<&'a ()>,
}
However, it may also make sense to use a struct in cases where you need to enforce constructors/destructors are called for a resource. Since Rust calls a struct's drop function when it falls out of scope then gets dropped from memory, it is quite easy to enforce these rules. In this case, by having a struct we can enforce some pre-requisite to be filled to gain access to the functions in the struct.
pub struct FooApi;
impl FooApi {
pub fn new() -> Self {
unsafe { sys::init_thread_foo(); }
FooApi
}
/// Some call that is only safe if sys::init_thread_foo() has been called
pub fn do_something(&self) { /* ... */}
}
/// Call FFI function to dispose of this resource once this FooApi is no longer needed
impl Drop for FooApi {
fn drop(&mut self) {
unsafe {
sys::dispose_thread_foo();
}
}
}
Verbosity
Sometimes a struct could probably be replaced with a module. However depending on the developer, they may prefer to use a struct instead in cases where it makes more conceptual sense to think of acting upon an object. Often these cases are reasonably rare and usually indicate that a type previously or will in the future be split into traits and generified. Alternatively it may also be used in cases where there are a couple equivalent, but non-identical alternatives that can be swapped between (Ex: types of CPUs ).
So far those are a couple of the causes that come to mind, but I may come back to add more.

Store immutable instances of Rust struct, but where? [duplicate]

This question already has answers here:
How can you make a safe static singleton in Rust?
(3 answers)
How do I create a global, mutable singleton?
(7 answers)
Closed 4 years ago.
Sometimes, it’s handy to have a few instances of a struct stored and accessible everywhere.
For instance, if I want to store some meta-data about a currency in a struct like this:
struct Currency {
name: &'static str,
iso_symbols: Vec<&'static str>
}
I would then create an instance of Currency for every major currency. As these property don’t change, it could be hard-coded, at least for some currencies.
I tried to use const, which would work without a Vec (the vec! macro does an allocation, which is not allowed in const).
const BTC: Currency = Currency {
name: "Bitcoin",
iso_symbols: vec!["BTC", "XBT"]
};
So what workaround would you suggest to store a bunch of instance of Currency (for EUR, USD, BTC…)?
Here, in the Rust Playground.
EDIT: My question is quite similar to this one. The only difference is that I don’t need a mutable singleton, so the “Non-answer answer” doesn’t apply, right? The lazy_static idea is great though!
It might be interesting to keep this question around since I didn’t searched with the keyword singleton and I may not be alone to miss this way to consider the problem.
As pointed out in the comments, the lazy_static crate works well.
pub struct Currency {
name: &'static str,
iso_symbols: Vec<&'static str>,
}
lazy_static! {
pub static ref BTC: Currency = Currency {
name: "Bitcoin",
iso_symbols: vec!["BTC", "XBT"]
};
}
fn main() {
println!("{} {:?}", BTC.name, BTC.iso_symbols);
}
We have this global (with pub) variable BTC I was looking for.

Resources