I'm trying to recreate a C struct with mixed bitfield members and "normal" members in Rust for FFI.
I've read that the bitflags crate would be the one to go with, unfortunately I find the documentation lacking on the regards how the syntax actually works.
The bitflags crate makes it easier to create bitmasks in a similar style as in C using enums. The bitfield crate claims to create bitfields that can be accessed, however I have no idea how it works.
I have a C structure like this:
struct mixed {
unsigned int flag_1_1 : 1;
unsigned int flag_2_7 : 7;
unsigned int flag_3_8 : 8;
unsigned int some_val1;
unsigned int some_val2;
unsigned int flag_4_16 : 16;
};
I have no clue on how to represent it in Rust, I'd use the crate libc to have access to c_uint, but other than that, I'm currently pretty much out of ideas and finding other code that does this has not proven successful:
#[repr(transparent)] // do I also need repr(C) ?
struct mixed {
flags_1_3: mixed_flags_1_3;
some_val1: c_uint;
some_val2: c_uint;
flags_4: mixed_flags_4;
}
bitfield!(
#[repr(transparent)] // do I also need repr(C), here too?
struct mixed_flags_1_3(u16 /* what about this parameter? */) {
u8; // what does this mean?
/* get/field, set: first bit, last bit; */
flag_1_1, _: 0, 0;
flag_2_7, _: 7, 1;
flag_3_8, _: 15, 8;
}
)
bitfield!(
#[repr(transparent)]
struct mixed_flags_4(u8) {
u8;
flag_4_16, _: 15, 0;
}
)
These are just guesses, how do I create a correct representation?
In cases like this you can look at genearted code by bindgen:
$ bindgen test.h
#[repr(C)]
#[derive(Copy, Clone, Debug, Default, Eq, Hash, Ord, PartialEq, PartialOrd)]
pub struct __BindgenBitfieldUnit<Storage, Align>
where
Storage: AsRef<[u8]> + AsMut<[u8]>,
{
storage: Storage,
align: [Align; 0],
}
//skipped code with access method for bit fields
#[repr(C)]
#[derive(Debug, Copy, Clone)]
pub struct mixed {
pub _bitfield_1: __BindgenBitfieldUnit<[u8; 2usize], u8>,
pub some_val1: ::std::os::raw::c_uint,
pub some_val2: ::std::os::raw::c_uint,
pub _bitfield_2: __BindgenBitfieldUnit<[u8; 2usize], u16>,
pub __bindgen_padding_0: u16,
}
Using rustc -- -Z unstable-options --pretty=expanded I think I could figure out that the macro does, and this seems to yield something that could be correct, however this is probably only compatible when the compiler does not try to pad or reorder the bitfields in the struct.
#[repr(transparent)] // do I also need repr(C) ?
struct mixed {
flags_1_3: mixed_flags_1_3;
some_val1: c_uint;
some_val2: c_uint;
flags_4: mixed_flags_4;
}
bitfield!(
#[repr(transparent)] // do I also need repr(C), here too?
// Results in a "tuple struct", ie. u16 = total size of bitfields
struct mixed_flags_1_3(u16) {
// All the following fields value should be treated as an
// unsigned int when accessed
c_uint;
/* get/field, set: first bit, last bit; */
flag_1_1, _: 0, 0;
flag_2_7, _: 7, 1;
// One could change the type again here, if one wanted to:
// u16
flag_3_8, _: 15, 8;
}
)
bitfield!(
#[repr(transparent)]
struct mixed_flags_4(u16) {
c_uint;
flag_4_16, _: 15, 0;
}
)
But for now at least I think I will just use libclang and bindgen as dependencies and generate my bindings automatically, due to the aforementioned problems with platform compat.
Related
I'm writing some Rust code that interfaces with C. The C defines some structures like
struct foo {
// other fields
uint32_t count;
const struct bar* array;
};
I want to define a Rust structure with this layout, but for safety I want to put count and array in their own structure as private members, like so:
#[repr(C)]
pub struct Foo {
// other fields
pub bars: BarSlice,
}
#[repr(C)]
pub struct BarSlice {
count: u32,
ptr: *const Bar,
}
This is wrong though, since on 64-bit systems BarSlice will be aligned to 8 bytes, and the field immediately before count might be 8 byte aligned and 4 bytes long. How can I tell Rust to align the members of the struct correctly, but not worry about the struct itself?
I would also rather avoid if possible
creating a different slice type for each alignment circumstance
having private fields in Foo
I have a struct returned to C code from Rust. I have no idea if it's a good way to do things, but it does work for rebuilding the struct and freeing memory without leaks.
#[repr(C)]
pub struct s {
// ...
}
#[repr(C)]
#[allow(clippy::box_vec)]
pub struct s_arr {
arr: *const s,
n: i8,
vec: Box<Vec<s>>,
}
/// Frees memory that was returned to C code
pub unsafe extern "C" fn free_s_arr(a: *mut s_arr) {
Box::from_raw(s_arr);
}
/// Generates an array for the C code
pub unsafe extern "C" fn gen_s_arr() -> *mut s_arr {
let many_s: Vec<s> = Vec::new();
// ... logic here
Box::into_raw(Box::new(s_arr {
arr: many_s.as_mut_ptr(),
n: many_s.len() as i8,
vec: many_s,
}))
}
The C header is currently written by hand, but I wanted to try out cbindgen. The manual C definition for s_arr is:
struct s_arr {
struct s *arr;
int8_t n;
void *_;
};
cbindgen generates the following for s_arr:
typedef struct Box_Vec_s Box_Vec_s;
typedef struct s_arr {
const s *arr;
int8_t n;
Box_Vec_s vec;
} s_arr;
This doesn't work since struct Box_Vec_s is not defined. Ideally I would just want to override the cbindgen type generated for vec to make it void * since it requires no code changes and thus no additional testing, but I am open to other suggestions.
I have looked through the cbindgen documentation, though not the examples, and couldn't find anything.
Your question is a bit unclear, but I think that if I understood you right, you're confusing two things and being led down a dark alley as a result.
In C, a dynamically-sized array, as you probably know, is identified by two things:
Its starting position, as a pointer
Its length
Rust follows the same convention - a Vec<_>, below the hood, shares the same structure (well, almost. It has a capacity as well, but that's beside the point).
Passing the boxed vector on top of a pointer is not only overkill, but extremely unwise. FFI bindings may be smart, but they're not smart enough to deal with a boxed complex type most of the time.
To solve this, we're going to simplify your bindings. I've added a single element in struct S to show you how it works. I've also cleaned up your FFI boundary:
#[repr(C)]
#[no_mangle]
pub struct S {
foo: u8
}
#[repr(C)]
pub struct s_arr {
arr: *mut S,
n: usize,
cap: usize
}
// Retrieve the vector back
pub unsafe extern "C" fn recombine_s_arr(ptr: *mut S, n: usize, cap: usize) -> Vec<S> {
Vec::from_raw_parts(ptr, n, cap)
}
#[no_mangle]
pub unsafe extern "C" fn gen_s_arr() -> s_arr {
let mut many_s: Vec<S> = Vec::new();
let output = s_arr {
arr: many_s.as_mut_ptr(),
n: many_s.len(),
cap: many_s.capacity()
};
std::mem::forget(many_s);
output
}
With this, cbindgen returns the expected header definitions:
typedef struct {
uint8_t foo;
} so58311426S;
typedef struct {
so58311426S *arr;
uintptr_t n;
uintptr_t cap;
} so58311426s_arr;
so58311426s_arr gen_s_arr(void);
This allows us to call gen_s_arr() from either C or Rust and retrieve a struct that is usable across both parts of the FFI boundary (so58311426s_arr). This struct contains all we need to be able to modify our array of S (well, so58311426S according to cbindgen).
When passing through FFI, you need to make sure of a few simple things:
You cannot pass raw boxes or non-primitive types; you will almost universally need to convert down to a set of pointers or change your definitions to accomodate (as I have done here)
You most definitely do not pass raw vectors. At most, you pass a slice, as that is a primitive type (see the point above).
You make sure to std::mem::forget() whatever you do not want to deallocate, and make sure to remember to deallocate it or reform it somewhere else.
I will edit this question in an hour; I have a plane to get on to. Let me know if any of this needs clarifications and I'll get to it once I'm in the right country :-)
I am trying to use node-ffi with dynamic lib generated from rust. This is the rust link, https://github.com/petrachor/pairing-ariel. How can I get JavaScript to properly call rust function and return the expected result?
To compile rust first change crate-type(Cargo.toml) to ["dylib"], and cargo build --release
#[repr(C)]
#[derive(Debug)]
pub struct ArrayStruct<T> {
d: *mut T,
len: usize,
}
#[no_mangle]
pub extern "C" fn g2_get_one(g: ArrayStruct<u8>) -> bool {
return panic::catch_unwind(|| {
g2_to_raw(G2Affine::get_generator(), g);
}).is_ok();
}
My node code to call rust via FFI
var ref = require('ref');
var FFI = require('ffi');
var Struct = require('ref-struct');
//var IArrayType = require('ref-array');
var ArrayStruct8 = Struct({
'd': "uchar*",
'len': "int32"
});
var lib = new FFI.Library('target/release/libpairing', { 'g2_get_zero': [ ref.types.bool, [ ArrayStruct8] ]});
var buf = new Buffer.alloc(192);
var a8 = new ArrayStruct8({d: buf, len: 192});
lib.g2_get_zero(a8);
console.dir(a8);
I was expecting a8.b to contain unsigned char* .. When I did console.log(a8.d), I got "#". There is something out there I have not fixed yet, please help me.
This will get edited once I am at home and have my full set of tools available to get the actual, struct-based example to work.
The symptoms across the FFI boundary with the implementation given in the question are as follows:
Pointer integrity is kept (the same pointer surfaces on both sides of the FFI boundary)
Buffer integrity is not kept or updated. Coming from Rust to NodeJS, the buffer should have been updated, but is not
The quick solution when dealing with low-field-count modification structures like this is to pass a fat pointer instead (pointer + length) as follows, for instance:
#[no_mangle]
pub extern "C" fn do_something_with_array(buf: *mut u8, len: u32) -> u32 {
unsafe {
buf.write_bytes(1, 3);
}
3
}
With the corresponding FFI definition across on the nodeJS front:
var lib = new FFI.Library('target/debug/libtest2', {
'do_something_with_array': [ 'int', ['pointer', 'int'] ]
});
var buf = new Buffer.alloc(192);
var new_len = lib.do_something_with_array(buf, 192);
var new_buf = buf.slice(0, new_len);
It seems like ref-struct requires a sync of some sort as the underlying memory has the right content.
While trying to figure out how to get Buffer integrity kept or updated I noticed that I do not really need ref-struct or something complex. I think I am making use of C language internal layout of struct. Just pointer and it's length. For example,
#[repr(C)]
#[derive(Debug)]
pub struct ArrayStruct<T> {
d: *mut T,
len: usize,
}
#[no_mangle]
pub extern "C" fn g2_get_one(g: ArrayStruct<u8>) -> bool {
return panic::catch_unwind(|| {
g2_to_raw(G2Affine::get_generator(), g);
}).is_ok();
}
To call the g2_get_one from Nodejs do the following:
var lib = new FFI.Library('target/debug/libtest2', {
'do_something_with_array': [ 'int', ['pointer', 'int'] ]
});
var buf = new Buffer.alloc(192);
var new_len = lib.do_something_with_array(buf, 192);
As long as one follows this simple rule you can have multiple structs as parameters in Rust and this rule would be obeyed .
I'm wrapping a C library in Rust, and many of its functions take parameters by pointers to structs, which themselves often have pointers to other structs. In the interest of reducing overhead, I'd like to provide the ability to cache the results of marshaling the Rust data into the C structs.
Here's an example of how the C library might expect some parameters:
#[repr(C)]
struct Foo {
x: i32,
y: f32
}
#[repr(C)]
struct Bar {
p_foo: *const Foo,
z: bool
}
And how I'd imagine an owning, "cached" version would look:
struct Cached {
foo: Option<Foo>,
bar: Bar
}
The p_foo field of bar would be constructed to point to Some value within foo, or a null pointer if there was None.
The issue, here, of course, is that if a value of Cached was to be moved, a straight memcpy would be inappropriate and bar.p_foo would additionally need to be redirected. This would be easy to ensure in C++, with its definable move semantics, but does Rust offer a solution besides "don't set bar.p_foo until it's used"? While it would certainly work to do it that way, I don't imagine that these cached values will be moved more than (or even close to the frequency that) they are reused, and there is a bit of work involved to set up these pointers, especially if the nesting/chaining is deep/long. I'd also rather not Box the substructures up on the heap.
To clarify, here's what I can write in C++, which I would like to replicate in Rust:
struct Foo {
int x;
float y;
};
struct Bar {
Foo const*pFoo;
bool z;
};
// bear with me while I conjure up a Maybe for C++
class Cached {
public:
// would have appropriate copy constructor/assignment
Cached(Cached &&other) {
m_foo = other.m_foo;
m_bar = other.m_bar;
if(m_foo.isJust()) {
m_bar.pFoo = &m_foo.value();
} // else already nullptr
}
// similar move assignment
private:
Maybe<Foo> m_foo;
Bar m_bar;
};
The Rust-equivalent would be to not use raw pointers, as raw pointers are there for implementing our safe datastructures, not for implementing normal datastructures.
#[repr(C)]
struct Foo {
x: i32,
y: f32
}
#[repr(C)]
struct Bar {
p_foo: Option<Box<Foo>>,
z: bool
}
An Option<Box<T>> is guaranteed to be exactly equivalent (in bits in memory) to a *const T, as long as T is a type and not a trait. The only difference is that it's safe to use within Rust.
This way you don't even need a Cached struct anymore, but can directly pass around the Bar object.
I'd also rather not Box the substructures up on the heap.
Then I suggest you don't keep a Bar object around, and instead conjure it up whenever you need to pass one to C:
#[repr(C)]
struct Foo {
x: i32,
y: f32
}
#[repr(C)]
struct Bar<'a> {
p_foo: Option<&'a Foo>,
z: bool
}
struct Cached {
foo: Option<Foo>,
z: bool,
}
impl Cached {
fn bar<'a>(&'a self) -> Bar<'a> {
Bar {
p_foo: self.foo.as_ref(),
z: self.z,
}
}
}
there is a bit of work involved to set up these pointers, especially if the nesting/chaining is deep/long.
That sounds a lot like premature optimization. Don't optimize where you haven't benchmarked.
I would like to wrap a Rust struct in a C++ class.
Rust:
#[repr(C)]
pub struct RustStruct {
num: i32,
// other members..
}
pub extern "C" fn update(rust_struct: *mut RustStruct) {
(*rust_struct).num = 1i32;
}
extern "C" {
void update(void*);
}
C++:
class Wrapper {
public:
Wrapper();
// ..
private:
void* rustStruct;
// ..
};
Wrapper::Wrapper() {
update(rustStruct); // crash
}
int main() {
std::cout << "Testing..";
}
I understand why this wouldn't work. My question is: how can I achieve what I'm basically trying to do (wrap a rust struct in a c++ class)?
There is a mix of multiple FFIs concepts in your answer, so first let me recommend that your read the Reference.
There are two ways to achieve what you wish, you can either:
use a POD struct (Plain Old Data), aka C-compatible struct
use an opaque pointer (void* in C)
Mixing them, as you did, does not make sense.
Which to pick?
Both solutions have advantages and disadvantages, it's basically an expressiveness versus performance trade-off.
On the one hand, opaque pointers are more expressive: they can point to any Rust type. However:
they require dynamic memory allocation
they require being manipulated by Rust functions (so always indirectly from C or C++)
On the other hand, POD struct do not require either of those, but they are limited to only a subset of types expressible in Rust.
How to use a POD?
This is the easiest, actually, so let's start with it!
In Rust:
#[repr(C)]
pub struct RustStruct {
num: i32,
// other members, also PODs!
}
In C++
struct RustStruct {
int32_t num;
// other members, also with Standard Layout
// http://en.cppreference.com/w/cpp/types/is_standard_layout
};
class Wrapper {
public:
private:
RustStruct rustStruct;
};
Note that I just got along with your question stricto censu here, you could actually merge the two in a single C++ class:
class RustStruct {
public:
private:
int32_t num;
// other members, also with Standard Layout
// http://en.cppreference.com/w/cpp/types/is_standard_layout
};
Just avoid virtual methods.
How to use an opaque pointer?
This gets trickier:
Only the Rust code may correctly create/copy/destruct the type
Beware of leaking...
So, we need to implement a lot of functions in Rust:
#![feature(box_raw, box_syntax)]
use std::boxed;
pub struct RustStruct {
num: i32,
// other members, anything goes
}
pub extern "C" fn createRustStruct() -> *mut RustStruct {
boxed::into_raw(box RustStruct::new())
}
pub extern "C" fn destroyRustStruct(o: *mut RustStruct) {
boxed::from_raw(o);
}
Alright... now on to C++:
struct RustStruct;
RustStruct* createRustStruct();
void destroyRustStruct(RustStruct*);
class Wrapper {
public:
Wrapper(): rustStruct(RustStructPtr(createRustStruct())) {}
private:
struct Deleter {
void operator()(RustStruct* rs) const {
destroyRustStruct(rs);
}
};
typedef std::unique_ptr<RustStruct, Deleter> RustStructPtr;
RustStructPtr rustStruct;
}; // class Wrapper
So, yes, a bit more involved, and Wrapper is not copyable either (copy has to be delegated to Rust too). Anyway, this should get you started!
Note: if you have a lot of opaque pointers to wrap, a templated C++ class taking the copy/destroy functions as template parameters could alleviate a lot of boiler plate.