I have a big struct (~200Mb) that I deserialize from a large JSON file from Java using serde_json and this deserialization occurs again when new data is available. The struct has Vecs, a HashMap of strings and structs of strings, etc.
While looking at the man page for mallopt(3), I found that environment variable MALLOC_MMAP_THRESHOLD_ can be set to control how much allocation has to be requested for malloc to allocate using mmap. I want to allocate my struct from mmap because the heap is causing memory fragmentation during reloads. I want the old deallocated memory (the one that is replaced with a new deserialized struct) to be returned to the system immediately (and not kept around by the one of the malloc arenas).
Is there a way to achieve this? Should I be using some other data format?
Related
HVM is a functional runtime which represents pointers as 32-bit values. Its allocator reserves a huge (4 GB) buffer preemptively, which it uses to create internal objects. This is not ideal. Instead, I'd like to use the system allocator, but that's not possible, since it returns 64-bit pointers, which may be larger than the space available to store them. Is there any cross-platform way in Rust to allocate a buffer, such that the pointer to the buffer is guaranteed to fit in an u32? In other words, I'm looking for something akin to:
let ptr = Box::new_with_small_ptr(size);
assert!(ptr as u64 + size < u32::MAX);
There isn't, because it's an extremely niche need and requires a lot of care.
It's not as simple as "just returning a low pointer" - you need to actually allocate that space from the OS. Your entry point into that would be mmap. Be prepared to do some low-level work with MAP_FIXED and reading /proc/self/maps, and also implementing an allocator on top of the memory region you get from mmap.
If your concern is just excess memory usage, note that Linux overcommits memory by default - allocating 4GB of memory won't reserve physical memory unless you actually try to use it all.
Taking this signature for a method of the GlobalAllocator:
unsafe fn alloc(&self, layout: Layout) -> *mut u8
and this sentence from the method's documentation:
The allocated block of memory may or may not be initialized.
Suppose that we are going to allocate some chunk of memory for an [i32, 10]. Assuming the size of i32 it's 4 bytes, our example array would need 40 bytes for the requested storage.
Now, the allocator found a memory spot that fits our requirements. Some 40 bytes of a memory region... but... what's there? I always read the term garbage data, and assume that it's just old data already stored there by another process, program... etc.
What's unitialized memory? Just data that is not initialized with zeros of with some default value for the type that we want to store there?
Why not always memory it's initialized before returning the pointer? It's too costly? But the memory must be initialized in order to use it properly and not cause UB. Why then doesn't comes already initialized?
When some resource it's deallocated, things musn't be pointing to that freed memory. That's that place got zeroed? What really happens when you deallocate some piece of memory?
What's unitialized memory? Just data that is not initialized with zeros of with some default value for the type that we want to store there?
It's worse than either of those. Reading from uninitialized memory is undefined behavior, as in you can no longer reason about a program which does so. Practically, compilers often optimize assuming that code paths that would trigger undefined behavior are never executed and their code can be removed. Or not, depending on how aggressive the compiler is.
If you could reliably read from the pointer, it would contain arbitrary data. It may be zeroes, it may be old data structures, it may be parts of old data structures. It may even be things like passwords and encryption keys, which is another reason why reading uninitialized memory is problematic.
Why not always memory it's initialized before returning the pointer? It's too costly? But the memory must be initialized in order to use it properly and not cause UB. Why then doesn't comes already initialized?
Yes, cost is the issue. The first thing that is typically done after allocating a piece of memory is to write to it. Having the allocator "pre-initialize" the memory is wasteful when the caller is going to overwrite it anyway with the values it wants. This is especially significant with large buffers used for IO or other large storage.
When some resource it's deallocated, things musn't be pointing to that freed memory. That's that place got zeroed? What really happens when you deallocate some piece of memory?
It's up to how the memory allocator is implemented. Most don't waste processing power to clear the data that's been deallocated, since it will be overwritten anyway when it's reallocated. Some allocators may write some bookkeeping data to the freed space. GlobalAllocator is an interface to whatever allocator the system comes with, so it can vary depending on the environment.
I always read the term garbage data, and assume that it's just old data already stored there by another process, program... etc.
Worth noting: all modern desktop OSs have memory isolation between processes - your program cannot access the memory of other processes or the kernel (unless you explicitly share it via specialized functionality). The kernel will clear memory before it assigns it to your process, to prevent leaking sensitive data. But you can see old data from your own process, for the reasons described above.
What you are asking are implementation details that can even vary from run to run. From the perspective of the abstract machine and thus the optimizer they don't matter.
Turning contents of uninitialized memory into almost any type (other than MaybeUninit) is immediate undefined behavior.
let mem: *u8 = unsafe { alloc(...) };
let x: u8 = unsafe { ptr::read(mem) };
if x != x {
print!("wtf");
}
May or may not print, crash or delete the contents of your harddrive, possibly even before reaching that alloc call because the optimizer worked backwards and eliminated the entire code block because it could prove that all execution paths are UB.
This may happen due to assumptions the optimizer relies on, i.e. even when the underlying allocator is well-behaved. But real systems may also behave non-deterministically. E.g. theoretically on a freshly booted embedded system memory might be in an uninitialized state that doesn't reliably return 0 or 1. Or on linux madvise(MADV_FREE) can cause allocations to return inconsistent results over time until initialized.
I'm trying to understand how heap allocations work in Serde JSON.
Why does the following code make one heap allocation? I am expecting no allocations as the value of c is a borrowed serde_json::value::RawValue using `#[serde(borrow)].
#[derive(Deserialize, Debug)]
struct MyStruct<'a> {
#[serde(borrow)]
c: &'a serde_json::value::RawValue,
}
fn main() {
let msg = r#"{"c":[{}]}"#;
// One unexpected allocation here.
serde_json::from_str::<MyStruct>(msg).unwrap();
}
Note using {"c":[2, 3]} for example instead of {"c":[{}]} will result in no allocations.
How can I make it so there are zero allocations when deserializing into MyStruct?
Rust playground link.
You can't avoid the allocations. The parser needs to allocate some memory on the heap because JSON objects and arrays can nest arbitrarily deeply, and it needs to keep track of what type of value it's currently parsing.
I modified your program to panic on the first heap allocation that happens during parsing (because I'm too lazy to debug and there's no debugger on the playground). The backtrace shows where the heap allocation is coming from. The key frame is this one:
14: serde_json::de::Deserializer<R>::ignore_value
at ./.cargo/registry/src/github.com-1ecc6299db9ec823/serde_json-1.0.69/src/de.rs:1049:21
You can be sure that your MyStruct won't point to heap allocated memory, because your struct contains a shared reference and not an owned value (and the input is a static string). In order for c to refer to heap allocated memory, serde_json would have to leak it (and it doesn't do that; it would be pretty bad if every parse could leak memory!).
The allocation is internal to serde_json and will be freed before returning. There is an internal scratch area which can require allocations (source).
The README of serde_json also mentions the fact that it relies on alloc support, so allocations are not unexpected:
As long as there is a memory allocator, it is possible to use serde_json without the rest of the Rust standard library.
If you need to work without allocations you can instead try serde-json-core which is also mentioned in that README.
Is there any way to calculate the total stack and heap size of a struct at runtime?
As far as I can tell, std::mem::{size_of, size_of_val} only work for stack-allocated values, but a struct might contain heap-allocated buffers, too (e.g. Vec).
Servo was using the heapsize crate to measure the size of heap allocations during program.
You can call the heap_size_of function to measure the allocated heap size by jemalloc.
Be aware that you can get different results with different allocators.
Regarding Github: "This crate is not maintained and is no longer used by Servo. At the time of writing, Servo uses internal malloc_size_of instead."
You can either use heapsize crate or you can check the implementation details of malloc_size_of as well
I'm writing my own memory allocator. If my notion is right, there is kernel's addres space from zero to 1-GB, and user-space from 1 to 4, that contains different sections like: code, stack, data, heap and others.
Is it right, that heap section's size and position can't be changed during the program execution?
How can I get the size and the position?
All I need to do, after getting heap area, is allocate the memory at my discretion, isn't I?
Why do you worry about this?
If you're writing an allocator a-la libc, use sbrk and/or mmap to reserve memory from the kernel. You do not need to worry about where the heap is placed with neither of those system calls.
If you want to instrument libc's malloc/calloc/realloc, things are even simpler - just wrap them in your allocator functions.
Yes, the allocator effectively manages the heap by requesting memory from the kernel. Typically, as is the case with brk, its position lies after the end of the data segment, and it grows at increasing addresses (or allocates in multiples of page size with mmap, etc.)
The allocator manages the size; the position of the heap is not relevant as far as the allocator is concerned, but it's at the position of knowing it.
The allocator effectively requests memory from the kernel. Once it has that memory, it can distribute it to programs however it deems fit.
It is an allocator which defines a heap. If you have a custom allocator, and it determines that all memory clients have returned all memory, it is perfectly valid for it to delete its heap or create a new one to supply memory requests from.
Since the allocator itself defines the heap it should know its size and position. If what you are talking about is usurping the OS allocator's responsibility with your own allocator, you should only do this by using the OS allocator to obtain a block of memory then use that as the heap for your own allocator.
Again, once your allocator owns the memory block it may be used at your discretion. You cannot simply take memory which is managed by another allocator and in its free pool and use it without serious potential consequences.