I am writing a Rust program using libevdev through FFI (using bindgen). I was trying to write a struct to store some libevdev pointer and free the pointer automatically when the struct is drop. And I find that this cause memory leak, but calling the free function directly does not. This is weird since these two ways should be equivalent.
Here is the code:
pub struct DevicePtr {
pub value: *mut libevdev,
}
impl Drop for DevicePtr {
fn drop(&mut self) {
if !self.value.is_null() {
println!("dropping device ptr");
unsafe {
libevdev_free(self.value);
}
}
}
}
fn main() {
unsafe {
let dev = libevdev_new();
// libevdev_free(dev);
let device_ptr = DevicePtr { value: dev };
}
}
valgrind --leak-check=full reports that there are 51284 bytes leak, but all of them are still reachable. Also, the program does print dropping device ptr so libevdev_free() is indeed called.
If I replace the line let device_ptr = ... with libevdev_free(dev), then the valgrind reports no leak.
So here is a snippet of heap summary output by valgrind --leak-check=full --show-leak-kinds=all:
==889647== HEAP SUMMARY:
==889647== in use at exit: 51,284 bytes in 262 blocks
==889647== total heap usage: 331 allocs, 69 frees, 64,885 bytes allocated
==889647==
==889647== 4 bytes in 1 blocks are still reachable in loss record 1 of 241
==889647== at 0x483F7B5: malloc (vg_replace_malloc.c:381)
==889647== by 0x4988583: ??? (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7
000.2)
==889647== by 0x4988BC9: g_private_get (in /usr/lib/x86_64-linux-gnu/libglib-
2.0.so.0.7000.2)
==889647== by 0x49540EF: g_slice_alloc (in /usr/lib/x86_64-linux-gnu/libglib-
2.0.so.0.7000.2)
==889647== by 0x492251D: g_hash_table_new_full (in /usr/lib/x86_64-linux-gnu/
libglib-2.0.so.0.7000.2)
==889647== by 0x4945DFA: ??? (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7
000.2)
==889647== by 0x48FFE24: ??? (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7
000.2)
==889647== by 0x401000D: call_init.part.0 (dl-init.c:74)
==889647== by 0x40100EF: call_init (dl-init.c:37)
==889647== by 0x40100EF: _dl_init (dl-init.c:121)
==889647== by 0x4001089: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.33.so)
==889647==
==889647== 8 bytes in 1 blocks are still reachable in loss record 2 of 241
==889647== at 0x483F6C5: malloc (vg_replace_malloc.c:380)
==889647== by 0x493BEB7: g_realloc (in /usr/lib/x86_64-linux-gnu/libglib-2.0.
so.0.7000.2)
==889647== by 0x48B6522: ??? (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.
0.7000.2)
==889647== by 0x48BAF1C: g_type_register_static (in /usr/lib/x86_64-linux-gnu
/libgobject-2.0.so.0.7000.2)
==889647== by 0x48BFBFE: g_type_plugin_get_type (in /usr/lib/x86_64-linux-gnu
/libgobject-2.0.so.0.7000.2)
==889647== by 0x48969F1: ??? (in /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.
0.7000.2)
==889647== by 0x401000D: call_init.part.0 (dl-init.c:74)
==889647== by 0x40100EF: call_init (dl-init.c:37)
==889647== by 0x40100EF: _dl_init (dl-init.c:121)
==889647== by 0x4001089: ??? (in /usr/lib/x86_64-linux-gnu/ld-2.33.so)
==889647==
Related
Was looking into packing enums and while doing so I ran following program
enum SizeEnum {
V1(u32, u32),
// V2(u64),
V3(u32, u32),
}
fn main() {
println!("{:?}", std::mem::size_of::<SizeEnum>());
}
Playground link
The output is 12 bytes (96 bits). What I expected was 16 bytes (128 bits). That's what happens when I uncomment V2 variant.
Questions are:
So why does a u32, u32 variant use less space than u64?
And why 12 bytes (96 bits) rather than something like 64+8 (72 bits)? I assume it's something about padding but would appreciate a detailed answer.
Both questions boil down to alignment.
The u32s need to be aligned to 4 bytes. The u64 needs to be aligned to 8 bytes.
Therefore, for the u32 there are 3 bytes padding for the discriminant (so the u32 is at the fourth byte) and for the u64 there are seven.
let vector: Vec<u8> = Vec::new();
Can the vector in the code above cause a stackoverflow if the vector grows too big?
let vector: Vec<Box<u8>> = Vec::new();
How bout this one? Since its elements are on the heap.
let vector: Box<Vec<u8>> = Box::new(Vec::new());
I'm assuming that in the code above no stackoverflow should be possible, am i correct?
No the actual data is on heap. So there will not be stack overflow.
What is on stack is capacity, length and the pointer to the actual data on heap. If regrowth is required then it is done on the heap. If it is moved (not cloned) then what is copied is just the length, capacity and pointer to data (bitwise shallow copy).
Not the actual implementation but if you have to implement Vector then you will start with:
pub struct Vec<T> {
ptr: Unique<T>,
cap: usize,
len: usize,
}
You see the ptr is actually pointing to heap location where the data is. The vector on stack will consists of just few fields like the 3 mentioned above.
You cannot grow an object on stack, as you push objects on stack frame if any object is allowed to grow it will run over other objects. On heap for growth, if contiguous memory is not available, then entire data is moved to another place with newer capacity; if contiguous memory block is available, then growth in capacity is instant.
Is there a way in rust to fill a vec it with a range that has for limit u64::max_value ?
fn main() {
let vals: Vec<u64> = (2..u64::max_value()).collect();
println!("{:?}", vals.len());
}
compiler throws: thread 'main' panicked at 'capacity overflow', src/liballoc/raw_vec.rs:777:5
I suppose that your computer has a 64 bits architecture. That means that it can address at most 2^64 bytes (in practice, that's less).
Now, since the size of an u64 is 8 bytes, you're trying to reserve 8 * 2^64 bytes. Your computer cannot even address such an amount of bytes!
Also, you're trying to allocate several millions of terabytes in the RAM. That's not a reasonable amount of memory.
The line that panics in the std lib is the following:
let alloc_size = capacity.checked_mul(elem_size).unwrap_or_else(|| capacity_overflow());
It verifies that the capacity (the number of items) multiplied by the size of your element (8 bytes as I said) does not overflow. That's the programmatic way to represent my former reasoning.
No.
u64::max_value() is a huge number. You don't have that much memory. No one does.
Consider:
fn main() {
// Prints 8, 8, 16
println!(
"{}, {}, {}",
std::mem::size_of::<Box<i8>>(),
std::mem::size_of::<Box<&[i8]>>(),
std::mem::size_of::<Box<[i8]>>(),
);
}
Why do owned slices take 16 bytes, but referenced slices take only 8?
Box<T> is basically *const T (Actually it's a newtype around Unique<T>, which itself is a NonNull<T> with PhantomData<T> (for dropck), but let's stick to *const T for simplicity).
A pointer in Rust normally has the same size as size_of::<usize>() except when T is a dynamically sized type (DST). Currently, a Box<DST> is 2 * size_of::<usize>() in size (the exact representation is not stable at the time of writing). A pointer to a DST is called FatPtr.
Currently, there are two kinds of DSTs: Slices and traits. A FatPtr to a slice is defined like this:
#[repr(C)]
struct FatPtr<T> {
data: *const T,
len: usize,
}
Note: For a trait pointer, len is replaced by a pointer to the vtable.
With those information, your question can be answered:
Box<i8>: i8 is a sized type => basically the same as *const i8 => 8 bytes in size (with 64 bit pointer width)
Box<[i8]>: [i8] is a DST => basically the same as FatPtr<i8> => 16 bytes in size (with 64 bit pointer width)
Box<&[i8]>: &[i8] is not a DST. It's basically the same as *const FatPtr<i8> => 8 bytes in size (with 64 bit pointer width)
The size of a reference depends on the "sizedness" of the referenced type:
A reference to a sized type is a single pointer to the memory address.
A reference to an unsized type is a pointer to the memory and the size of the pointed datum. That's what is called a fat pointer:
#[repr(C)]
struct FatPtr<T> {
data: *const T,
len: usize,
}
A Box is a special kind of pointer that points to the heap, but it is still a pointer.
Knowing that, you understand that:
Box<i8> is 8 bytes because i8 is sized,
Box<&[i8]> is 8 bytes because a reference is sized,
Box<[i8]> is 16 bytes because a slice is unsized.
Running this:
fn main() {
std::iter::count(1i16, 3).collect::<Vec<i16>>();
}
I get:
thread '' panicked at 'capacity overflow', /home/tshepang/projects/rust/src/libcore/option.rs:329
That's what I'd expect when running this:
fn main() {
std::iter::count(1i8, 3).collect::<Vec<i8>>();
}
But instead, I get this:
Illegal instruction
In addition, syslog displays this line:
Dec 27 08:31:08 thome kernel: [170925.955841] traps: main[30631] trap invalid opcode ip:7f60ab175470 sp:7fffbb116578 error:0 in main[7f60ab15c000+5b000]
This was a fun adventure.
Iter::collect simply calls FromIterator::from_iter
Vec's implementation of FromIterator asks the iterator for its size and then allocates memory:
let (lower, _) = iterator.size_hint();
let mut vector = Vec::with_capacity(lower);
Vec::with_capacity computes the total size of memory and attempts to allocate it:
let size = capacity.checked_mul(mem::size_of::<T>())
.expect("capacity overflow");
let ptr = unsafe { allocate(size, mem::min_align_of::<T>()) };
if ptr.is_null() { ::alloc::oom() } // Important!
In this case, i8 takes 1 byte, and the lower bound of an infinite iterator is std::uint::MAX. Multiplied together, that's still std::uint::MAX. When we allocate that, we get a null pointer back.
alloc::oom is defined to simply abort, which is implemented by an Illegal Instruction!
The reason that an i16 has different behavior is because it triggers the checked_mul expectation - you can't allocate std::uint::MAX * 2 bytes!
In modern Rust, the examples would be written as:
(1i16..).step_by(3).collect::<Vec<_>>();
(1i8..).step_by(3).collect::<Vec<_>>();
Both now fail in the same manner:
memory allocation of 12297829382473034412 bytes failed
memory allocation of 6148914691236517206 bytes failed