I'm fairly new to Rust and am trying to get the following to work:
let hwnd : *mut HWND = window.hwnd().cast();
let swapchain = unsafe { factory.CreateSwapChainForHwnd(&device, *hwnd, &desc, std::ptr::null(), &output)? } ;
where window.hwnd() returns a *mut c_void and I need to cast that to a windows::Windows::Win32::Foundation::HWND , but this example crashes on a access violation. I assume its because I deref a pointer to a HWND whereas the HWND itself should be the void ptr. HWND can be created from an isize, so like HWND(isize) but I'm not sure if that should get the address of the void pointer or something? Any help is appreciated.
You're right that you need to convert the pointer to an isize, so although this clashes with recent developments regarding pointer provenance, I believe the correct way to construct an HWND is as follows:
let hwnd = HWND(window.hwnd() as isize);
isize and usize are defined to be pointer width, so converting a raw pointer to one of these types is zero cost and essentially just erases type information.
Note this works because HWND is just a newtype struct whose single field is a pub isize, so the HWND(val) syntax just initializes the struct with that field set to val. To access that field you can just do my_hwnd.0, and convert to a pointer via my_hwnd.0 as *mut T.
Related
I was reading through the Rust documentation and can't quite seem to be able to wrap my head around what is going on. For example, over here I see the following example:
// This function takes ownership of a box and destroys it
fn eat_box_i32(boxed_i32: Box<i32>) {
println!("Destroying box that contains {}", boxed_i32);
}
// This function borrows an i32
fn borrow_i32(borrowed_i32: &i32) {
println!("This int is: {}", borrowed_i32);
}
fn main() {
// Create a boxed i32, and a stacked i32
let boxed_i32 = Box::new(5_i32);
let stacked_i32 = 6_i32;
// Borrow the contents of the box. Ownership is not taken,
// so the contents can be borrowed again.
borrow_i32(&boxed_i32);
borrow_i32(&stacked_i32);
{
// Take a reference to the data contained inside the box
let _ref_to_i32: &i32 = &boxed_i32;
// Error!
// Can't destroy `boxed_i32` while the inner value is borrowed later in scope.
eat_box_i32(boxed_i32);
// FIXME ^ Comment out this line
// Attempt to borrow `_ref_to_i32` after inner value is destroyed
borrow_i32(_ref_to_i32);
// `_ref_to_i32` goes out of scope and is no longer borrowed.
}
// `boxed_i32` can now give up ownership to `eat_box` and be destroyed
eat_box_i32(boxed_i32);
}
Things I believe:
eat_box_i32 takes a pointer to a Box
this line let boxed_i32 = Box::new(5_i32); makes is so that boxed_i32 now contains a pointer because Box is not a primitive
Things I don't understand:
why do we need to call borrow_i32(&boxed_i32); with the ampersand? Isn't boxed_i32 already a pointer?
on this line: let _ref_to_i32: &i32 = &boxed_i32; why is the ampersand required on the right hand side? Isn't boxed_i32 already an address?
how come borrow_i32 can be called with pointer to Box and pointer to i32 ?
Comment on the term "pointers"
You can skip this part if you'd like, I just figured given the questions you asked, this might be a helpful comment:
In Rust, &i32, &mut i32, *const i32, *mut i32, Box<i32>, Rc<i32>, Arc<i32> are all arguably a "pointer to i32" type. However, Rust will not let you convert between them casually, even between those that are laid out identically in memory.
It can be useful to talk about pointers in general sometimes, but as a rule of thumb, if you're trying to figure out why one piece of Rust code compiles, and another doesn't, I'd recommend keeping track of which kind of pointer you're working with.
Things you believe:
eat_box_i32 takes a pointer to a Box
Actually not quite. eat_box_i32 accepts a Box<i32>, and not a pointer to a Box<i32>. It just so happens that Box<i32> in memory is stored as a pointer to an i32.
this line let boxed_i32 = Box::new(5_i32); makes is so that boxed_i32 now contains a pointer because Box is not a primitive
Yes, boxed_i32 is a pointer.
Things you don't understand:
why do we need to call borrow_i32(&boxed_i32); with the ampersand? Isn't boxed_i32 already a pointer?
Yes, boxed_i32 is already a pointer. However, a boxed pointer still indicates ownership. If you passed boxed_i32 instead of &boxed_i32, you would still be passing a pointer, but Rust will consider that variable "consumed", and you would no longer be able to use boxed_i32 after that function call.
on this line: let _ref_to_i32: &i32 = &boxed_i32; why is the ampersand required on the right hand side? Isn't boxed_i32 already an address?
Yes, boxed_i32 is already an address, but the fact that it's an address is kind of meant to be opaque (like a struct with a single private field). The actual type of &boxed_i32 is &Box<i32>.
Though this is weird right? If &boxed_i32 is &Box<i32>, how can you assign it to a variable of type &i32?
This is actually a shorthand -- If a type T implements the Deref<Target=R> trait, it'll automatically convert values of type &T into values of type &R as needed. And it turns out that the Box<T> type implements Deref<Target=T>.
See https://doc.rust-lang.org/std/ops/trait.Deref.html for more info about Deref.
So if you wrote it out explicitly without that automatic conversion, that line would actually look something like:
let _ref_to_i32: &i32 = Deref::deref(&boxed_i32);
how come borrow_i32 can be called with pointer to Box and pointer to i32 ?
The reason is the same as with (2) above.
borrow_i32 accepts &i32 as its parameter. Passing &i32 is obviously ok because the types match exactly. If you try to pass it &Box<i32>, Rust will automatically convert it to &i32 for you, because Box<i32> implements Deref<i32>.
EDIT: Thanks #kmdreko for pointing out that Deref allows the coercion, and not AsRef
Just to supplement #math4tots, the auto dereferencing is call Deref Coercion. It is explained in the rustbook here: https://doc.rust-lang.org/book/ch15-02-deref.html#implicit-deref-coercions-with-functions-and-methods
I have an Option<&T> and I would like to have a raw *const T which is null if the option was None. I want to wrap an FFI call that takes a pointer to a Rust-allocated object.
Additionally, the FFI interface I am using has borrowing semantics (I allocate something and pass in a pointer to it), not ownership semantics
extern "C" {
// Parameter may be null
fn ffi_call(*const T);
}
fn safe_wrapper(opt: Option<&T>) {
let ptr: *const T = ???;
unsafe { ffi_call(ptr) }
}
I could use a match statement to do this, but that method feels very verbose.
let ptr = match opt {
Some(inner) => inner as *const T,
None => null(),
};
I could also map the reference to a pointer, then use unwrap_or.
let ptr = opt.map(|inner| inner as *const T).unwrap_or(null());
However, I'm worried that the pointer might be invalidated as it passes through the closure. Does Rust make a guarantee that the final pointer will point to the same thing as the original reference? If T is Copy, does this change the semantics in a meaningful way? Is there a better way that I am overlooking?
Yes, this is safe. I'd write it as:
use std::ptr;
fn safe_wrapper(opt: Option<&u8>) {
let p = opt.map_or_else(ptr::null, |x| x);
unsafe { ffi_call(p) }
}
If you find yourself writing this a lot, you could make it into a trait and reduce it down to a single method call.
the pointer might be invalidated as it passes through the closure
It could be, if you invalidate it yourself somehow. Because the function takes a reference, you know for sure that the referred-to value will be valid for the duration of the function call — that's the purpose of Rust's borrow checker.
The only way for the pointer to become invalid is if you change the value of the pointer (e.g. you add an offset to it). Since you don't do that, it's fine.
Does Rust make a guarantee that the final pointer will point to the same thing as the original reference?
It depends what you mean by "final". Converting a reference to a pointer will always result in both values containing the same location in memory. Anything else would be deliberately malicious and no one would ever have used Rust to begin with.
If T is Copy, does this change the semantics in a meaningful way?
No. Besides we are talking about a &T, which is always Copy
See also:
Convert Option<&mut T> to *mut T
Should we use Option or ptr::null to represent a null pointer in Rust?
Is it valid to use ptr::NonNull in FFI?
the FFI interface I am using has borrowing semantics (I allocate something and pass in a pointer to it), not ownership semantics
To be clear, you cannot determine ownership based purely on what the function types are.
This C function takes ownership:
void string_free(char *)
This C function borrows:
size_t string_len(char *)
Both take a pointer. Rust improves on this situation by clearly delineating what is a borrow and what is a transfer of ownership.
extern "C" {
// Parameter may be null
fn ffi_call(*const T);
}
This code is nonsensical; it does not define the generic type T and FFI functions cannot have generic types anyway.
I want to return a vector in a pub extern "C" fn. Since a vector has an arbitrary length, I guess I need to return a struct with
the pointer to the vector, and
the number of elements in the vector
My current code is:
extern crate libc;
use self::libc::{size_t, int32_t, int64_t};
// struct to represent an array and its size
#[repr(C)]
pub struct array_and_size {
values: int64_t, // this is probably not how you denote a pointer, right?
size: int32_t,
}
// The vector I want to return the address of is already in a Boxed struct,
// which I have a pointer to, so I guess the vector is on the heap already.
// Dunno if this changes/simplifies anything?
#[no_mangle]
pub extern "C" fn rle_show_values(ptr: *mut Rle) -> array_and_size {
let rle = unsafe {
assert!(!ptr.is_null());
&mut *ptr
};
// this is the Vec<i32> I want to return
// the address and length of
let values = rle.values;
let length = values.len();
array_and_size {
values: Box::into_raw(Box::new(values)),
size: length as i32,
}
}
#[derive(Debug, PartialEq)]
pub struct Rle {
pub values: Vec<i32>,
}
The error I get is
$ cargo test
Compiling ranges v0.1.0 (file:///Users/users/havpryd/code/rust-ranges)
error[E0308]: mismatched types
--> src/rle.rs:52:17
|
52 | values: Box::into_raw(Box::new(values)),
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected i64, found *-ptr
|
= note: expected type `i64`
= note: found type `*mut std::vec::Vec<i32>`
error: aborting due to previous error
error: Could not compile `ranges`.
To learn more, run the command again with --verbose.
-> exit code: 101
I posted the whole thing because I could not find an example of returning arrays/vectors in the eminently useful Rust FFI Omnibus.
Is this the best way to return a vector of unknown size from Rust? How do I fix my remaining compile error? Thanks!
Bonus q: if the fact that my vector is in a struct changes the answer, perhaps you could also show how to do this if the vector was not in a Boxed struct already (which I think means the vector it owns is on the heap too)? I guess many people looking up this q will not have their vectors boxed already.
Bonus q2: I only return the vector to view its values (in Python), but I do not want to let the calling code change the vector. But I guess there is no way to make the memory read-only and ensure the calling code does not fudge with the vector? const is just for showing intent, right?
Ps: I do not know C or Rust well, so my attempt might be completely WTF.
pub struct array_and_size {
values: int64_t, // this is probably not how you denote a pointer, right?
size: int32_t,
}
First of all, you're correct. The type you want for values is *mut int32_t.
In general, and note that there are a variety of C coding styles, C often doesn't "like" returning ad-hoc sized array structs like this. The more common C API would be
int32_t rle_values_size(RLE *rle);
int32_t *rle_values(RLE *rle);
(Note: many internal programs do in fact use sized array structs, but this is by far the most common for user-facing libraries because it's automatically compatible with the most basic way of representing arrays in C).
In Rust, this would translate to:
extern "C" fn rle_values_size(rle: *mut RLE) -> int32_t
extern "C" fn rle_values(rle: *mut RLE) -> *mut int32_t
The size function is straightforward, to return the array, simply do
extern "C" fn rle_values(rle: *mut RLE) -> *mut int32_t {
unsafe { &mut (*rle).values[0] }
}
This gives a raw pointer to the first element of the Vec's underlying buffer, which is all C-style arrays really are.
If, instead of giving C a reference to your data you want to give C the data, the most common option would be to allow the user to pass in a buffer that you clone the data into:
extern "C" fn rle_values_buf(rle: *mut RLE, buf: *mut int32_t, len: int32_t) {
use std::{slice,ptr}
unsafe {
// Make sure we don't overrun our buffer's length
if len > (*rle).values.len() {
len = (*rle).values.len()
}
ptr::copy_nonoverlapping(&(*rle).values[0], buf, len as usize);
}
}
Which, from C, looks like
void rle_values_buf(RLE *rle, int32_t *buf, int32_t len);
This (shallowly) copies your data into the presumably C-allocated buffer, which the C user is then responsible for destroying. It also prevents multiple mutable copies of your array from floating around at the same time (assuming you don't implement the version that returns a pointer).
Note that you could sort of "move" the array into C as well, but it's not particularly recommended and involves the use mem::forget and expecting the C user to explicitly call a destruction function, as well as requiring both you and the user to obey some discipline that may be difficult to structure the program around.
If you want to receive an array from C, you essentially just ask for both a *mut i32 and i32 corresponding to the buffer start and length. You can assemble this into a slice using the from_raw_parts function, and then use the to_vec function to create an owned Vector containing the values allocated from the Rust side. If you don't plan on needing to own the values, you can simply pass around the slice you produced via from_raw_parts.
However, it is imperative that all values be initialized from either side, typically to zero. Otherwise you invoke legitimately undefined behavior which often results in segmentation faults (which tend to frustratingly disappear when inspected with GDB).
There are multiple ways to pass an array to C.
First of all, while C has the concept of fixed-size arrays (int a[5] has type int[5] and sizeof(a) will return 5 * sizeof(int)), it is not possible to directly pass an array to a function or return an array from it.
On the other hand, it is possible to wrap a fixed size array in a struct and return that struct.
Furthermore, when using an array, all elements must be initialized, otherwise a memcpy technically has undefined behavior (as it is reading from undefined values) and valgrind will definitely report the issue.
Using a dynamic array
A dynamic array is an array whose length is unknown at compile-time.
One may chose to return a dynamic array if no reasonable upper-bound is known, or this bound is deemed too large for passing by value.
There are two ways to handle this situation:
ask C to pass a suitably sized buffer
allocate a buffer and return it to C
They differ in who allocates the memory: the former is simpler, but may require to either have a way to hint at a suitable size or to be able to "rewind" if the size proves unsuitable.
Ask C to pass a suitable sized buffer
// file.h
int rust_func(int32_t* buffer, size_t buffer_length);
// file.rs
#[no_mangle]
pub extern fn rust_func(buffer: *mut libc::int32_t, buffer_length: libc::size_t) -> libc::c_int {
// your code here
}
Note the existence of std::slice::from_raw_parts_mut to transform this pointer + length into a mutable slice (do initialize it with 0s before making it a slice or ask the client to).
Allocate a buffer and return it to C
// file.h
struct DynArray {
int32_t* array;
size_t length;
}
DynArray rust_alloc();
void rust_free(DynArray);
// file.rs
#[repr(C)]
struct DynArray {
array: *mut libc::int32_t,
length: libc::size_t,
}
#[no_mangle]
pub extern fn rust_alloc() -> DynArray {
let mut v: Vec<i32> = vec!(...);
let result = DynArray {
array: v.as_mut_ptr(),
length: v.len() as _,
};
std::mem::forget(v);
result
}
#[no_mangle]
pub extern fn rust_free(array: DynArray) {
if !array.array.is_null() {
unsafe { Box::from_raw(array.array); }
}
}
Using a fixed-size array
Similarly, a struct containing a fixed size array can be used. Note that both in Rust and C all elements should be initialized, even if unused; zeroing them works well.
Similarly to the dynamic case, it can be either passed by mutable pointer or returned by value.
// file.h
struct FixedArray {
int32_t array[32];
};
// file.rs
#[repr(C)]
struct FixedArray {
array: [libc::int32_t; 32],
}
I have tried using raw pointer casts, like my_struct as *const usize, but this results in a non-scalar cast error. Raw pointers seem to work fine when finding the address of primitives, but not custom structs.
You need to use the & operator to get the address of any variable, so you need to write &my_struct as *const _ (where _ can be a literal _, or the type of the value behind the pointer).
See the std::ptr::addr_of macro. This is especially useful when taking a reference (as suggested in the accepted answer) is not possible due to memory alignment issues.
Here is the example given in the docs on std::ptr::addr_of:
use std::ptr;
#[repr(packed)]
struct Packed {
f1: u8,
f2: u16,
}
let packed = Packed { f1: 1, f2: 2 };
// `&packed.f2` would create an unaligned reference, and thus be Undefined Behavior!
let raw_f2 = ptr::addr_of!(packed.f2);
assert_eq!(unsafe { raw_f2.read_unaligned() }, 2);
In Rust, we can use the Box<T> type to allocate things on the heap. This type is used to safely abstract pointers to heap memory. Box<T> is provided by the Rust standard library.
I was curious about how Box<T> allocation is implemented, so I found its source code. Here is the code for Box<T>::new (as of Rust 1.0):
impl<T> Box<T> {
/// Allocates memory on the heap and then moves `x` into it.
/// [...]
#[stable(feature = "rust1", since = "1.0.0")]
#[inline(always)]
pub fn new(x: T) -> Box<T> {
box x
}
}
The only line in the implementation returns the value box x. This box keyword is not explained anywhere in the official documentation; in fact, it is only mentioned briefly on the std::boxed documentation page.
NOTE: This reply is a bit old. Since it talks about internals and unstable features, things have changed a little bit. The basic mechanism remains the same though, so the answer is still capable of explaining the underlying mechanisms of box.
What does box x usually uses to allocate and free memory?
The answer is the functions marked with lang items exchange_malloc for allocation and exchange_free for freeing. You can see the implementation of those in the default standard library at heap.rs#L112 and heap.rs#L125.
In the end the box x syntax depends on the following lang items:
owned_box on a Box struct to encapsulate the allocated pointer. This struct does not need a Drop implementation, it is implemented automatically by the compiler.
exchange_malloc to allocate the memory.
exchange_free to free the previously allocated memory.
This can be effectively seen in the lang items chapter of the unstable rust book using this no_std example:
#![feature(lang_items, box_syntax, start, no_std, libc)]
#![no_std]
extern crate libc;
extern {
fn abort() -> !;
}
#[lang = "owned_box"]
pub struct Box<T>(*mut T);
#[lang = "exchange_malloc"]
unsafe fn allocate(size: usize, _align: usize) -> *mut u8 {
let p = libc::malloc(size as libc::size_t) as *mut u8;
// malloc failed
if p as usize == 0 {
abort();
}
p
}
#[lang = "exchange_free"]
unsafe fn deallocate(ptr: *mut u8, _size: usize, _align: usize) {
libc::free(ptr as *mut libc::c_void)
}
#[start]
fn main(argc: isize, argv: *const *const u8) -> isize {
let x = box 1;
0
}
#[lang = "stack_exhausted"] extern fn stack_exhausted() {}
#[lang = "eh_personality"] extern fn eh_personality() {}
#[lang = "panic_fmt"] fn panic_fmt() -> ! { loop {} }
Notice how Drop was not implemented for the Box struct? Well let's see the LLVM IR generated for main:
define internal i64 #_ZN4main20hbd13b522fdb5b7d4ebaE(i64, i8**) unnamed_addr #1 {
entry-block:
%argc = alloca i64
%argv = alloca i8**
%x = alloca i32*
store i64 %0, i64* %argc, align 8
store i8** %1, i8*** %argv, align 8
%2 = call i8* #_ZN8allocate20hf9df30890c435d76naaE(i64 4, i64 4)
%3 = bitcast i8* %2 to i32*
store i32 1, i32* %3, align 4
store i32* %3, i32** %x, align 8
call void #"_ZN14Box$LT$i32$GT$9drop.103617h8817b938807fc41eE"(i32** %x)
ret i64 0
}
The allocate (_ZN8allocate20hf9df30890c435d76naaE) was called as expected to build the Box, meanwhile... Look! A Drop method for the Box (_ZN14Box$LT$i32$GT$9drop.103617h8817b938807fc41eE)! Let's see the IR for this method:
define internal void #"_ZN14Box$LT$i32$GT$9drop.103617h8817b938807fc41eE"(i32**) unnamed_addr #0 {
entry-block:
%1 = load i32** %0
%2 = ptrtoint i32* %1 to i64
%3 = icmp ne i64 %2, 2097865012304223517
br i1 %3, label %cond, label %next
next: ; preds = %cond, %entry- block
ret void
cond: ; preds = %entry-block
%4 = bitcast i32* %1 to i8*
call void #_ZN10deallocate20he2bff5e01707ad50VaaE(i8* %4, i64 4, i64 4)
br label %next
}
There it is, deallocate (ZN10deallocate20he2bff5e01707ad50VaaE) being called on the compiler generated Drop!
Notice even on the standard library the Drop trait is not implemented by user-code. Indeed Box is a bit of a magical struct.
Before box was marked as unstable, it was used as a shorthand for calling Box::new. However, it's always been intended to be able to allocate arbitrary types, such as Rc, or to use arbitrary allocators. Neither of these have been finalized, so it wasn't marked as stable for the 1.0 release. This is done to prevent supporting a bad decision for all of Rust 1.x.
For further reference, you can read the RFC that changed the "placement new" syntax and also feature gated it.
box does exactly what Box::new() does - it creates an owned box.
I believe that you can't find implementation of box keyword because currently it is hardcoded to work with owned boxes, and Box type is a lang item:
#[lang = "owned_box"]
#[stable(feature = "rust1", since = "1.0.0")]
#[fundamental]
pub struct Box<T>(Unique<T>);
Because it is a lang item, the compiler has special logic to handle its instantiation which it can link with box keyword.
I believe that the compiler delegates box allocation to functions in alloc::heap module.
As for what box keyword does and supposed to do in general, Shepmaster's answer describes perfectly.