The UnsafeCell documentation says
The UnsafeCell<T> type is the only legal way to obtain aliasable data that is considered mutable.
The only construction method is:
pub const fn new(value: T) -> UnsafeCell<T>
However, it is not possible to create a c_void, we can only create *mut c_void or *const c_void.
Is it possible to create UnsafeCell<c_void> from a *mut c_void? With this, we can let the compiler know that the pointer can point to something mutable.
Or is this not necessary? Can we always use *mut c_void even we know some FFI call will mutate the data it points to and we have multiple references to it?
A use case would be:
struct FFIStruct { v: UnsafeCell<c_void>, other_fields: ... }
impl FFIStruct {
// We don't want to require &mut self, as we
// are sure private call_ffi() will always be called
// sequentially, and we don't want to stop
// status() being callable during the call
fn call_ffi(&self){ ffi_function(self.v.get()) }
pub fn status(&self) -> FFIStatus { ... }
}
Now how do we create FFIStruct? Or just use *mut c_void would be OK?
Example code to create &Cell<c_void>
Requires #![feature(as_cell)]:
unsafe fn get_cell<'a>(p: *mut c_void) -> &'a Cell<c_void> {
Cell::from_mut(&mut *p)
}
TL;DR: Just use *mut Foo. Cells of any kind are not needed here.
Disclaimer: there is no formal Rust memory model, yet.
You cannot create this type, period, because you cannot1 create an instance of c_void.
The thing is, you don't need to create such a type. Aliasing is not spatial but temporal. You can have multiple *mut T pointing to the same place and it doesn't matter until you try to access one. This essentially converts it to a reference and the aliasing requirements need to be upheld while that reference is around.
raw pointers fall outside of Rust's safe memory model.
— The Rustonomicon
Different from references and smart pointers, raw pointers:
Are allowed to ignore the borrowing rules by having both immutable and mutable pointers or multiple mutable pointers to the same location
Aren’t guaranteed to point to valid memory
Are allowed to be null
Don’t implement any automatic cleanup
¸— The Rust Programming Language
See also:
Why does modifying a mutable reference's value through a raw pointer not violate Rust's aliasing rules?
What's the Rust idiom to define a field pointing to a C opaque pointer?
Is it undefined behavior to do runtime borrow management with the help of raw pointers in Rust?
Can an FFI function modify a variable that wasn't declared mutable?
1 You technically can, but that's only because of an implementation and backwards compatibility limitation.
After some internal discussion in the Rust forum and a discussion on RFC 1861, I realize c_void is just a common workaround and other options exists like struct Opaque<UnsafeCell<()>>.
So I concluded what I needed here is *const UnsafeCell<c_void>. From its type we know:
This is a raw pointer, so it is suitable to send to FFI immediately;
The raw pointer assumes const, means any casting to *mut T will UB and programmer should avoid it. This protects the memory it points to being modified within Rust (unless through UnsafeCell, of cause);
It contains UnsafeCell, so it implies interior mutability. This justifies the FFI function being able to mutate it;
It is not possible to create a c_void, so as for UnsafeCell<c_void>. They can only be created by FFI.
Demostrate:
use std::cell::UnsafeCell;
use std::os::raw::c_void;
//Let's say all those functions are in an FFI module,
//with the exact same behaviour
fn ffi_create() -> *mut c_void {
Box::into_raw(Box::new(0u8)) as *mut c_void
}
unsafe fn ffi_write_u8(p: *mut c_void, v:u8) {
*(p as *mut u8) = v;
}
unsafe fn ffi_read_u8(p: *mut c_void) -> u8 {
*(p as *mut u8)
}
fn main() {
unsafe {
//let's ignore ffi_destroy() for now
let pointer = ffi_create() as *const UnsafeCell<c_void>;
let ref_pointer = &pointer;
ffi_write_u8((&*pointer).get(), 7);
let integer = ffi_read_u8((&**ref_pointer).get());
assert_eq!(integer, 7);
}
}
It is interesting how easy and ergonomic (yet expressive) to convert between *mut c_void and *const UnsafeCell<c_void>.
Related
What's the difference between pointer's method swap() and std::ptr::swap() ?
The signatures are similar, and they behaves the same as far as I tested.
pub unsafe fn swap(self, with: *mut T)
pub unsafe fn swap<T>(x: *mut T, y: *mut T)
As for std::mem::swap() (for reference rather than pointer), there is a situation where we cannot call std::mem::swap() as it needs two mutable references. In that case, we shall call slice::swap() for example. What about std::ptr::swap()?
There is no difference between ptr.swap() and std::ptr::swap() - the former's implementation just calls into the latter:
pub const unsafe fn swap(self, with: *mut T)
where
T: Sized,
{
// SAFETY: the caller must uphold the safety contract for `swap`.
unsafe { swap(self, with) }
}
As per the docs, the only differences between mem::swap and ptr::swap are:
ptr::swap operates on pointers instead of references.
ptr::swap allows the pointed-to values to overlap.
ptr::swap does not require the pointed-to data to be initialized/meet the requirements for the pointed-to type.
Other than that, their semantics are the same.
I have a Rust function like this:
pub fn get_buffer() -> &[u8] {
// returns *mut c_char
let ptr = unsafe { get_buffer_from_c_code() };
// returns &[u8]
let buf = unsafe { core::slice::from_raw_parts(ptr as *const u8, 10) };
buf
}
It generates this error:
pub fn get_buffer() -> &[u8] {
| ^ expected named lifetime parameter
|
= help: this function's return type contains a borrowed value, but there is no value for it to be borrowed from
help: consider using the `'static` lifetime
|
19 | pub fn get_buffer() -> &'static [u8] {
| ~~~~~~~~
I understand the error. It makes sense.
Question: should I take the compiler's suggestion and add a static lifetime specifier?
I'm connecting Rust to a C library that allocates memory internally and returns a pointer to it. Later, the C library takes care of de-allocating that memory on its own. I'd like the Rust code to be able to read what's in the memory, but otherwise leave it alone.
Rust is allocating the slice, though, in its own memory, and the slice itself (the pointer and the length) need to be dropped eventually.
Does a static lifetime do what I need it to do? Will Rust drop the slice, but not try to free the underlying buffer?
Question: should I take the compiler's suggestion and add a static lifetime specifier?
No. If your function return a static reference then you're promising to your caller that it can keep the reference around (and read through it) as long as it likes, which is only true if the buffer is never deallocated and never modified.
I'm connecting Rust to a C library that allocates memory internally and returns a pointer to it. Later, the C library takes care of de-allocating that memory on its own.
The solution to this problem depends entirely on when the deallocation happens. You need to ensure that there is some lifetime of a borrow such that there is no possibility to cause the deallocation until the borrow ends. You wrote in a comment
When my Rust code returns then C deallocates stuff.
That's key to picking the solution. That means that the reference should be obtained when the Rust code is called. That is:
extern "C" wrapper_called_from_c_code() {
let ptr = unsafe { get_buffer_from_c_code() };
let buf = unsafe { core::slice::from_raw_parts(ptr as *const u8, 10) };
// Constrain the lifetime of the slice to be the duration of
// this call by passing it through a lifetime-generic function.
// (<'a> is just for explicitness and could be elided.)
fn shim<'a>(buf: &'a [u8]) {
safe_rust_code(buf);
}
shim(buf);
// Now, after the function call returns, it's safe to proceed with
// allowing the C code to deallocate the buffer.
}
fn safe_rust_code(buf: &[u8]) {
// write whatever you like here
}
safe_rust_code can do whatever it likes in its function body, but the borrow checker will ensure it cannot hang onto the &'a [u8] slice reference longer than is safe.
The shim function exists to ensure that what wrapper_called_from_c_code needs (that the slice reference is being passed to a lifetime-generic function and not one that accepts &'static [u8]) inside wrapper_called_from_c_code rather than to explain it as a constraint on another function. I consider this good practice to keep invariants in the narrowest scope possible, to reduce the chances that they're broken by merely editing safe code without reading the comments.
Wrap the buffer in a struct that frees the buffer when the struct is dropped. The struct can then own the buffer, much like a Vec owns a block of data on the heap. When it hands out references their lifetimes will naturally be tied to the lifetime of the struct.
pub struct Buffer {
ptr: *const u8,
len: usize,
}
impl Buffer {
pub fn new() -> Self {
Self {
ptr: unsafe { get_buffer_from_c_code() },
len: 10,
}
}
}
impl Drop for Buffer {
fn drop(&mut self) {
unsafe {
free_buffer_from_c_code(self.ptr);
}
}
}
impl Deref for Buffer {
type Target = [u8];
fn deref(&self) -> &[u8] {
// SAFETY: The C library must not modify the contents of the buffer
// for the lifetime of the slice.
unsafe { slice::from_raw_parts(self.ptr, self.len) }
}
}
The solution I ended up going with was something like this:
pub fn get_buffer<'a>(foo: u32) -> &'a [u8] {
let ptr = unsafe { get_buffer_from_c_code() };
let buf = unsafe { core::slice::from_raw_parts(ptr as *const u8, 10) };
buf
}
Adding a parameter and then defining a lifetime on the function itself works. Essentially, it says that the returned slice lives as long as the foo parameter.
You can also do this:
pub fn get_buffer(foo: &u32) -> &[u8] {
If foo is a reference then you can elide the lifetime. The system makes the slice live as long as the foo reference. I don't know why it can do this with a reference and not a value parameter, but there it is.
I am trying to use the newtype pattern to wrap a pre-existing type. That inner type has a modify method which lets us work with a borrowed mutable value in a callback:
struct Val;
struct Inner(Val);
impl Inner {
fn modify<F>(&self, f: F)
where F: FnOnce(&mut Val) -> &mut Val { … }
}
Now I want to provide a very similar method on my newtype Outer, which however should not work on Vals but again a newtype wrapper WrappedVal:
struct Outer(Inner);
struct WrappedVal(Val);
impl Outer {
fn modify<F>(&self, f: F)
where
F: FnOnce(&mut WrappedVal) -> &mut WrappedVal,
{
self.0.modify(|v| f(/* ??? */));
}
}
This code is a reduced example from the original API. I don't know why the reference is returned from the closure, maybe to facilitate chaining, but it shouldn't be necessary. It takes &self because it uses internal mutability - it's a type representing a peripheral register on an embedded system
How do I get a &mut WrappedVal from a &mut Val?
I have tried various things, but all were busted by the borrow-checker. I cannot move the Val out of the mutable reference to construct a proper WrappedVal, and I couldn't get lifetimes to compile either when experimenting around with struct WrappedVal(&'? mut Val) (which I don't really want actually, since they are complicating a trait implementation).
I eventually got it to compile (see Rust playground demo) using the absolute horror of
self.0.modify(|v| unsafe {
(f((v as *mut Val as *mut WrappedVal).as_mut().unwrap()) as *mut WrappedVal as *mut Val)
.as_mut()
.unwrap()
});
but surely there must be a better way?
There is no safe way with your current definition, and your unsafe code is not guaranteed to be safe. There's no contract that the layout of a WrappedVal matches that of a Val, even though that's all it holds.
Solution not using unsafe
Don't do it. Instead, wrap the reference:
struct WrappedVal<'a>(&'a mut Val);
impl Outer {
fn modify<F>(&self, f: F)
where
F: FnOnce(WrappedVal) -> WrappedVal,
{
self.0.modify(|v| f(WrappedVal(v)).0)
}
}
Solution using unsafe
You can state that your type has the same representation as the type it wraps, making the pointers compatible via repr(transparent):
#[repr(transparent)]
struct WrappedVal(given::Val);
impl Outer {
fn modify<F>(&self, f: F)
where
F: FnOnce(&mut WrappedVal) -> &mut WrappedVal,
{
self.0.modify(|v| {
// Insert documentation why **you** think this is safe
// instead of copy-pasting from Stack Overflow
let wv = unsafe { &mut *(v as *mut given::Val as *mut WrappedVal) };
let wv = f(wv);
unsafe { &mut *(wv as *mut WrappedVal as *mut given::Val) }
})
}
}
With repr(transparent) in place, the two pointers are interchangable. I ran a quick test with Miri and your full example and didn't receive any errors, but that's not a silver bullet that I didn't mess something else up.
Using the ref_cast library you can write:
#[derive(RefCast)]
#[repr(transparent)]
struct WrappedVal(Val);
Then you can convert using WrappedVal::ref_cast_mut(v).
After asking how I should go about freeing memory across the FFI boundary, someone on the Rust reddit suggested that rather than wrapping my structs in a Box, I could use Vec::from_raw_parts to construct a vector from the following struct, and that this could be safely dropped:
#[repr(C)]
pub struct Array {
data: *const c_void,
len: libc::size_t,
}
However, from_raw_parts seems to require *mut _ data, so I'm not sure how to proceed…
The very short answer is self.data as *mut u8. But, let's talk more details...
First, words of warning:
Do not use Vec::from_raw_parts unless the pointer came from a Vec originally. There is no guarantee that an arbitrary pointer will be compatible with a Vec and you are likely to create giant holes in your program if you proceed.
Do not free a pointer that you don't own. Doing so leads to double frees, which will blow other large holes in your program.
You need to know the capacity of the vector before you can reconstruct it. Your example struct only contains a len. This is only acceptable if the len and capacity are equal.
Now, let's see if I can follow my own rules...
extern crate libc;
use std::mem;
#[repr(C)]
pub struct Array {
data: *const libc::c_void,
len: libc::size_t,
}
// Note that both of these methods should probably be implementations
// of the `From` trait to allow them to participate in more places.
impl Array {
fn from_vec(mut v: Vec<u8>) -> Array {
v.shrink_to_fit(); // ensure capacity == size
let a = Array {
data: v.as_ptr() as *const libc::c_void,
len: v.len(),
};
mem::forget(v);
a
}
fn into_vec(self) -> Vec<u8> {
unsafe { Vec::from_raw_parts(self.data as *mut u8, self.len, self.len) }
}
}
fn main() {
let v = vec![1, 2, 3];
let a = Array::from_vec(v);
let v = a.into_vec();
println!("{:?}", v);
}
Note that we don't have to do any explicit dropping of the Vec because the normal Drop implementation of Vec comes into play. We just have to make sure that we construct a Vec properly.
In Rust, a &T where T is a trait is a fat reference, which actually corresponds to raw::TraitObject:
pub struct TraitObject {
pub data: *mut (),
pub vtable: *mut (),
}
Using TraitObject, one can de-construct and re-construct a &T at leisure.
However, while obtaining the vtable from de-constructing a &T is easy, what if I never have the &T in the first place, but just a T and S; essentially, something along the lines of:
fn make_vptr<T: ?Sized, S>() -> *mut ();
How could I divine the v-ptr from there? Is there any intrinsic I could use?
Note: the naive implementation of creating a S (or conjuring it from thin-air) and then making a &T reference does not work; the compiler complains that T is not necessarily a trait and therefore that &T is either one pointer or two pointers in size.
A possibility is to use a macro to do the magic job:
#![feature(raw)]
macro_rules! make_vptr(
($S:ty, $T:ty) => ({
let s: &$S = unsafe { ::std::mem::uninitialized() };
let t: &$T = s;
let r: ::std::raw::TraitObject = unsafe { ::std::mem::transmute(t) };
r.vtable
})
);
This code will not compile if T is not a trait (thanks to transmute(..) checking that &T is a fat pointer) or if T is not implemented by S (thanks to the assignment).
Then, it can be used directly:
use std::fmt::Display;
fn main() {
let u32_display_vtable = make_vptr!(u32, Display);
let x = 42u32;
let disp: &Display = unsafe {
::std::mem::transmute(::std::raw::TraitObject {
data: &x as *const _ as *mut _,
vtable: u32_display_vtable,
})
};
println!("{}", disp);
}
I don't believe this is currently possible.
In order for this to work, you'd need to be able to constrain the T generic parameter to only accept traits. You can't do this. As a result, it won't ever let you do anything with &T that depends on it being a trait, such as getting the vtable.