This question already has answers here:
Creating a static C struct containing strings
(3 answers)
Closed 7 years ago.
I'm trying to learn Rust (newbie in low level programming), and want to translate a tiny lv2 amplifier (audio) plugin "amp.c" (C-code) from C to Rust. I actually got it working (here), but when the host terminates, valgrind says that "
64 bytes in 1 blocks are definitely lost". I think I know why this happens, but I don't know how to fix it.
Before you get tired of reading, here is the final question:
How do I statically allocate a struct that contains a C string?
And here is the introduction:
Why it happens (I think):
Host loads the library and calls lv2_descriptor()
const LV2_Descriptor*
lv2_descriptor()
{
return &descriptor;
}
which returns a pointer to a STATICALLY allocated struct of type LV2_Descriptor,
static const LV2_Descriptor descriptor = {
AMP_URI,
...
};
which is defined as
typedef struct _LV2_Descriptor {
const char * URI;
...
} LV2_Descriptor;
Why is it statically allocated? In the amp.c it says:
It is best to define descriptors statically to avoid leaking memory
and non-portable shared library constructors and destructors to clean
up properly.
However, I translated lv2_descriptor() to Rust as:
#[no_mangle]
pub extern fn lv2_descriptor(index:i32) -> *const LV2Descriptor {
let s = "http://example.org/eg-amp_rust";
let cstr = CString::new(s).unwrap();
let ptr = cstr.as_ptr();
mem::forget(cstr);
let mybox = Box::new(LV2Descriptor{amp_uri: ptr}, ...);
let bxptr = &*mybox as *const LV2Descriptor;
mem::forget(mybox);
return bxptr
}
So it's not statically allocated and I never free it, that's I guess why valgrind complains?
How am I trying to solve it?
I'm trying to do the same thing in Rust as the C-code does, i.e. statically allocate the struct (outside of lv2_descriptor()). The goal is to be fully compatible to the lv2 library, i.e "...to avoid leaking memory..." etc., as it says in the quote, right? So I tried something like:
static ptr1: *const u8 = (b"example.org/eg-amp_rust\n\0").as_ptr();
static ptr2: *const libc::c_char = ptr1 as *const libc::c_char;
static desc: LV2Descriptor = LV2Descriptor{amp_uri: ptr2, ...};
But this does not compile, there are error messages like
src/lib.rs:184:26: 184:72 error: the trait `core::marker::Sync` is not implemented for the type `*const u8` [E0277]
src/lib.rs:184 static ptr1: *const u8 = b"http://example.org/eg-amp_rust\n\0".as_ptr();
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/lib.rs:184:26: 184:72 note: `*const u8` cannot be shared between threads safely
src/lib.rs:184 static ptr1: *const u8 = b"http://example.org/eg-amp_rust\n\0".as_ptr();
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/lib.rs:184:26: 184:72 error: static contains unimplemented expression type [E0019]
src/lib.rs:184 static ptr1: *const u8 = b"http://example.org/eg-amp_rust\n\0".as_ptr();
Specific problem/question:
How do I statically allocate a struct that contains a C string?
The short answer is, you don't for now. Future Rust will probably gain this ability.
What you can do, is statically allocate a struct that contains null pointers, and set those null pointers to something useful when you call the function. Rust has static mut. It requires unsafe code, is not threadsafe at all and is (to the best of my knowledge) considered a code smell.
Right here I consider it a workaround to the fact that there is no way to turn a &[T] into a *const T in a static.
static S: &'static [u8] = b"http://example.org/eg-amp_rust\n\0";
static mut desc: LV2Descriptor = LV2Descriptor {
amp_uri: 0 as *const libc::c_char, // ptr::null() isn't const fn (yet)
};
#[no_mangle]
pub extern fn lv2_descriptor(index: i32) -> *const LV2Descriptor {
let ptr = S.as_ptr() as *const libc::c_char;
unsafe {
desc.amp_uri = ptr;
&desc as *const LV2Descriptor
}
}
Related
This question already has an answer here:
How can I call a raw address from Rust?
(1 answer)
Closed 3 years ago.
Hello people of the internet,
I'm struggeling to invoke a function that is stored in a libc::c_void-Pointer. I can't tell Rust that the pointer is invokable and I can't figure out how to.
I want to translate this C++ Code
void * malloc(size_t size) {
static void *(*real_malloc)(size_t) = nullptr;
if (real_malloc == nullptr) {
real_malloc = reinterpret_cast<void *(*)(size_t)> (dlsym(RTLD_NEXT, "malloc"));
}
// do some logging stuff
void * ptr = real_malloc(size);
return ptr;
}
to Rust.
#[no_mangle]
pub extern fn malloc(bytes: usize) {
let c_string = "malloc\0".as_mut_ptr() as *mut i8; // char array for libc
let real_malloc: *mut libc::c_void = libc::dlsym(libc::RTLD_NEXT, c_string);
return real_malloc(bytes);
}
That's my progress so far after 1h of searching on the internet and trying. I'm new to Rust and not yet familiar with Rust/FFI / Rust with libc. I tried a lot with unsafe{}, casts with as but I always stuck at the following problem:
return real_malloc(bytes);
^^^^^^^^^^^^^^^^^^ expected (), found *-ptr
Q1: How can I call the function behind the void-Pointer stored in real_malloc?
Q2: Is my Rust-String to C-String conversion feasible this way?
I figured it out! Perhaps there is a better way but it works.
The trick is to "cast" the void-Pointer to c-function-Type with std::mem::transmute since it won't work with as
type LibCMallocT = fn(usize) -> *mut libc::c_void;
// C-Style string for symbol-name
let c_string = "malloc\0".as_ptr() as *mut i8; // char array for libc
// Void-Pointer to address of symbol
let real_malloc_addr: *mut libc::c_void = unsafe {libc::dlsym(libc::RTLD_NEXT, c_string)};
// transmute: "Reinterprets the bits of a value of one type as another type"
// Transform void-pointer-type to callable C-Function
let real_malloc: LibCMallocT = unsafe { std::mem::transmute(real_malloc_addr) }
When the shared object is build, one can verify that it works like this:
LD_PRELOAD=./target/debug/libmalloc_log_lib.so some-binary
My full code:
extern crate libc;
use std::io::Write;
const MSG: &str = "HELLO WORLD\n";
type LibCMallocT = fn(usize) -> *mut libc::c_void;
#[no_mangle] // then "malloc" is the symbol name so that ELF-Files can find it (if this lib is preloaded)
pub extern fn malloc(bytes: usize) -> *mut libc::c_void {
/// Disable logging aka return immediately the pointer from the real malloc (libc malloc)
static mut RETURN_IMMEDIATELY: bool = false;
// C-Style string for symbol-name
let c_string = "malloc\0".as_ptr() as *mut i8; // char array for libc
// Void-Pointer to address of symbol
let real_malloc_addr: *mut libc::c_void = unsafe {libc::dlsym(libc::RTLD_NEXT, c_string)};
// transmute: "Reinterprets the bits of a value of one type as another type"
// Transform void-pointer-type to callable C-Function
let real_malloc: LibCMallocT = unsafe { std::mem::transmute(real_malloc_addr) };
unsafe {
if !RETURN_IMMEDIATELY {
// let's do logging and other stuff that potentially
// needs malloc() itself
// This Variable prevent infinite loops because 'std::io::stdout().write_all'
// also uses malloc itself
// TODO: Do proper synchronisazion
// (lock whole method? thread_local variable?)
RETURN_IMMEDIATELY = true;
match std::io::stdout().write_all(MSG.as_bytes()) {
_ => ()
};
RETURN_IMMEDIATELY = false
}
}
(real_malloc)(bytes)
}
PS: Thanks to https://stackoverflow.com/a/46134764/2891595 (After I googled a lot more I found the trick with transmute!)
I am trying to embed a version number into a library. Ideally, this should be a static C string that can be read and doesn't need any additional allocation for reading the version number.
On the Rust side, I am using vergen to generate the versioning information like this:
pub static VERSION: &str = env!("VERGEN_SEMVER");
and I would like to end up with something like
#[no_mangle]
pub static VERSION_C: *const u8 = ... ;
There seems to be a way to achieve this using string literals, but I haven't found a way to do this with compile time strings. Creating a new CString seems to be beyond the current capabilities of static variables and tends to end with an error E0015.
A function returning the pointer like this would be acceptable, as long as it does not allocate new memory.
#[no_mangle]
pub extern "C" fn get_version() -> *const u8 {
// ...
}
The final type of the variable (or return type of the function) doesn't have to be based on u8, but should be translatable through cbindgen. If some other FFI type is more appropriate, using that is perfectly fine.
By ensuring that the static string slice is compatible with a C-style string (as in, it ends with the null terminator byte \0), we can safely fetch a pointer to the beginning of the slice and pass that across the boundary.
pub static VERSION: &str = concat!(env!("VERGEN_SEMVER"), "\0");
#[no_mangle]
pub extern "C" fn get_version() -> *const c_char {
VER.as_ptr() as *const c_char
}
Here's an example in the Playground, where I used the package's version as the environment variable to fetch and called the function in Rust.
Here's what I would like to do in C code:
#include <some_lib.h>
int main() {
some_lib_struct_t x;
some_lib_func(&x);
}
How do I make use of the library in Rust? Here's what I've got so far:
extern crate libc; // 0.2.51
struct some_lib_struct_t;
#[link(name = "some_lib")]
extern "C" {
fn some_lib_func(x: *mut some_lib_struct_t);
}
fn main() {
let mut x: some_lib_struct_t;
unsafe {
some_lib_func(&mut x);
}
}
When compiling I get an error:
error[E0381]: borrow of possibly uninitialized variable: `x`
--> src/main.rs:13:23
|
13 | some_lib_func(&mut x);
| ^^^^^^ use of possibly uninitialized `x`
The safest answer is to initialize the struct yourself:
let mut x: some_lib_struct_t = some_lib_struct_t;
unsafe {
some_lib_func(&mut x);
}
The closest analog to the C code is to use MaybeUninit
use std::mem::MaybeUninit;
unsafe {
let mut x = MaybeUninit::uninit();
some_lib_func(x.as_mut_ptr());
}
Before Rust 1.36, you can use mem::uninitialized:
unsafe {
let mut x: some_lib_struct_t = std::mem::uninitialized();
some_lib_func(&mut x);
}
You have to be sure that some_lib_func completely initializes all the members of the struct, otherwise the unsafety will leak outside of the unsafe block.
Speaking of "members of the struct", I can almost guarantee your code won't do what you want. You've defined some_lib_struct_t as having zero size. That means that no stack space will be allocated for it, and a reference to it won't be what your C code is expecting.
You need to mirror the definition of the C struct in Rust so that the appropriate size, padding, and alignment can be allocated. Usually, this means using repr(C).
Many times, C libraries avoid exposing their internal struct representation by always returning a pointer to the opaque type:
What's the Rust idiom to define a field pointing to a C opaque pointer?
In Rust how can I define or import a C struct from a third party library?
Allocating an object for C / FFI library calls
From the documentation for mem::uninitialized():
Deprecated since 1.39.0: use mem::MaybeUninit instead
The new solution would look like this:
use std::mem::MaybeUninit;
let instance = unsafe {
let mut x: MaybeUninit<some_lib_struct_t> = MaybeUninit::uninit();
some_lib_func(x.as_mut_ptr());
x.assume_init()
}
After reading Shepmaster's answer, I looked closer at the header for the library. Just as they said, some_lib_struct_t was just a typedef for a pointer to actual_lib_struct_t. I made the following changes:
extern crate libc;
struct actual_lib_struct_t;
type some_lib_type_t = *mut actual_lib_struct_t;
#[link(name="some_lib")]
extern {
fn some_lib_func(x: *mut some_lib_type_t);
}
fn main() {
let mut x: some_lib_type_t;
unsafe {
x = std::mem::uninitialized();
some_lib_func(&mut x);
}
}
And it works! I do however get the warning found zero-size struct in foreign module, consider adding a member to this struct, #[warn(improper_ctypes)] on by default.
I want to do Rust bindings to a C library which requires a callback, and this callback must return a C-style char* pointer to the C library which will then free it.
The callback must be in some sense exposed to the user of my library (probably using closures), and I want to provide a Rust interface as convenient as possible (meaning accepting a String output if possible).
However, the C library complains when trying to free() a pointer coming from memory allocated by Rust, probably because Rust uses jemalloc and the C library uses malloc.
So currently I can see two workarounds using libc::malloc(), but both of them have disadvantages:
Give the user of the library a slice that he must fill (inconvenient, and imposes length restrictions)
Take his String output, copy it to an array allocated by malloc, and then free the String (useless copy and allocation)
Can anybody see a better solution?
Here is an equivalent of the interface of the C library, and the implementation of the ideal case (if the C library could free a String allocated in Rust)
extern crate libc;
use std::ffi::CString;
use libc::*;
use std::mem;
extern "C" {
// The second parameter of this function gets passed as an argument of my callback
fn need_callback(callback: extern fn(arbitrary_data: *mut c_void) -> *mut c_char,
arbitrary_data: *mut c_void);
}
// This function must return a C-style char[] that will be freed by the C library
extern fn my_callback(arbitrary_data: *mut c_void) -> *mut c_char {
unsafe {
let mut user_callback: *mut &'static mut FnMut() -> String = mem::transmute(arbitrary_data); //'
let user_string = (*user_callback)();
let c_string = CString::new(user_string).unwrap();
let ret: *mut c_char = mem::transmute(c_string.as_ptr());
mem::forget(c_string); // To prevent deallocation by Rust
ret
}
}
pub fn call_callback(mut user_callback: &mut FnMut() -> String) {
unsafe {
need_callback(my_callback, mem::transmute(&mut user_callback));
}
}
The C part would be equivalent to this:
#include <stdlib.h>
typedef char* (*callback)(void *arbitrary_data);
void need_callback(callback cb, void *arbitrary_data) {
char *user_return = cb(arbitrary_data);
free(user_return); // Complains as the pointer has been allocated with jemalloc
}
It might require some annoying work on your part, but what about exposing a type that implements Write, but is backed by memory allocated via malloc? Then, your client can use the write! macro (and friends) instead of allocating a String.
Here's how it currently works with Vec:
let mut v = Vec::new();
write!(&mut v, "hello, world");
You would "just" need to implement the two methods and then you would have a stream-like interface.
I am trying to figure out how to apply Rust lifetimes to add some compile-time enforcement to Erlang NIF modules. NIF modules are shared libraries normally written in C that provide extensions.
A simplified prototype of the callback you would write in C looks like this:
Handle my_nif_function(Heap *heap, Handle handle);
You are provided a handle and a pointer to the heap that owns it. In your callback you may inspect the input handle, create more handles on the heap, and return one of them as the function return. The heap and all its handles become invalid after your callback returns, so you must not store copies of the heap or its handles during the callback. Unfortunately I’ve seen people do exactly this and it eventually results in a mysterious emulator crash. Can Rust enforce these lifetime constraints?
I think the heap can be easily managed by turning it into a reference.
fn my_nif_function(heap: &Heap, handle: Handle) -> Handle
But how can I link the lifetime of the input and output handles to the heap?
Another wrinkle to this is that you can also create your own heaps and handles which are allowed to live outside the scope of a callback invocation. In C++ I would use std::unique_ptr with a custom destructor. What is the Rust equivalent? The [simplified] C API for managing heaps looks like this:
Heap *create_heap();
void destroy_heap(Heap *);
Reference: NIFs are described here: http://www.erlang.org/doc/man/erl_nif.html . The Erlang names for "heaps" and "handles" are "environments" and "terms". I used the names "heaps" and "handles" so that the question would be more broadly understood.
Rust 1.0
The various marker types have been unified into one: PhantomData
use std::ptr;
use std::marker::PhantomData;
struct Heap {
ptr: *const u8,
}
impl Heap {
fn new(c_ptr: *const u8) -> Heap {
Heap {
ptr: c_ptr
}
}
fn wrap_handle<'a>(&'a self, c_handle: *const u8) -> Handle<'a> {
Handle {
ptr: c_handle,
marker: PhantomData,
}
}
}
struct Handle<'a> {
ptr: *const u8,
marker: PhantomData<&'a ()>,
}
fn main() {
let longer_heap = Heap::new(ptr::null());
let handle = {
let shorter_heap = Heap::new(ptr::null());
let longer_handle = longer_heap.wrap_handle(ptr::null());
let shorter_handle = shorter_heap.wrap_handle(ptr::null());
// longer_handle // ok to return
// shorter_handle // error: `shorter_heap` does not live long enough
};
}
Original Answer
Here's an example of using ContravariantLifetime. We wrap the raw heap pointer into a struct and then wrap raw handle pointers in another struct, reusing the lifetime of the heap.
use std::ptr;
use std::marker::ContravariantLifetime;
struct Heap {
ptr: *const u8,
}
impl Heap {
fn new(c_ptr: *const u8) -> Heap {
Heap {
ptr: c_ptr
}
}
fn wrap_handle<'a>(&'a self, c_handle: *const u8) -> Handle<'a> {
Handle {
ptr: c_handle,
marker: ContravariantLifetime,
}
}
}
struct Handle<'a> {
ptr: *const u8,
marker: ContravariantLifetime<'a>,
}
fn main() {
let longer_heap = Heap::new(ptr::null());
let handle = {
let shorter_heap = Heap::new(ptr::null());
let longer_handle = longer_heap.wrap_handle(ptr::null());
let shorter_handle = shorter_heap.wrap_handle(ptr::null());
// longer_handle // ok to return
// shorter_handle // error: `shorter_heap` does not live long enough
};
}
Lifetime markers
There are 3 lifetime markers. I won't attempt to replicate the reasonably good but dense documentation here, but can also point out the dense Wikipedia page, which might be some small assistance. I've listed them in the order that you are most likely to use them:
ContravariantLifetime
InvariantLifetime
CovariantLifetime