I am trying to use node-ffi with dynamic lib generated from rust. This is the rust link, https://github.com/petrachor/pairing-ariel. How can I get JavaScript to properly call rust function and return the expected result?
To compile rust first change crate-type(Cargo.toml) to ["dylib"], and cargo build --release
#[repr(C)]
#[derive(Debug)]
pub struct ArrayStruct<T> {
d: *mut T,
len: usize,
}
#[no_mangle]
pub extern "C" fn g2_get_one(g: ArrayStruct<u8>) -> bool {
return panic::catch_unwind(|| {
g2_to_raw(G2Affine::get_generator(), g);
}).is_ok();
}
My node code to call rust via FFI
var ref = require('ref');
var FFI = require('ffi');
var Struct = require('ref-struct');
//var IArrayType = require('ref-array');
var ArrayStruct8 = Struct({
'd': "uchar*",
'len': "int32"
});
var lib = new FFI.Library('target/release/libpairing', { 'g2_get_zero': [ ref.types.bool, [ ArrayStruct8] ]});
var buf = new Buffer.alloc(192);
var a8 = new ArrayStruct8({d: buf, len: 192});
lib.g2_get_zero(a8);
console.dir(a8);
I was expecting a8.b to contain unsigned char* .. When I did console.log(a8.d), I got "#". There is something out there I have not fixed yet, please help me.
This will get edited once I am at home and have my full set of tools available to get the actual, struct-based example to work.
The symptoms across the FFI boundary with the implementation given in the question are as follows:
Pointer integrity is kept (the same pointer surfaces on both sides of the FFI boundary)
Buffer integrity is not kept or updated. Coming from Rust to NodeJS, the buffer should have been updated, but is not
The quick solution when dealing with low-field-count modification structures like this is to pass a fat pointer instead (pointer + length) as follows, for instance:
#[no_mangle]
pub extern "C" fn do_something_with_array(buf: *mut u8, len: u32) -> u32 {
unsafe {
buf.write_bytes(1, 3);
}
3
}
With the corresponding FFI definition across on the nodeJS front:
var lib = new FFI.Library('target/debug/libtest2', {
'do_something_with_array': [ 'int', ['pointer', 'int'] ]
});
var buf = new Buffer.alloc(192);
var new_len = lib.do_something_with_array(buf, 192);
var new_buf = buf.slice(0, new_len);
It seems like ref-struct requires a sync of some sort as the underlying memory has the right content.
While trying to figure out how to get Buffer integrity kept or updated I noticed that I do not really need ref-struct or something complex. I think I am making use of C language internal layout of struct. Just pointer and it's length. For example,
#[repr(C)]
#[derive(Debug)]
pub struct ArrayStruct<T> {
d: *mut T,
len: usize,
}
#[no_mangle]
pub extern "C" fn g2_get_one(g: ArrayStruct<u8>) -> bool {
return panic::catch_unwind(|| {
g2_to_raw(G2Affine::get_generator(), g);
}).is_ok();
}
To call the g2_get_one from Nodejs do the following:
var lib = new FFI.Library('target/debug/libtest2', {
'do_something_with_array': [ 'int', ['pointer', 'int'] ]
});
var buf = new Buffer.alloc(192);
var new_len = lib.do_something_with_array(buf, 192);
As long as one follows this simple rule you can have multiple structs as parameters in Rust and this rule would be obeyed .
Related
For the library I need to write I have a struct representing a service written in C:
struct RustService {
id: u8, /* it's me setting the id in the first place */
service: *mut c_void,
status: *mut c_void,
value: Option<u32>,
/* various fields here */
}
impl RustService {
fn new() -> /* can be wrapped*/RustService {
/* ... */
}
}
I have several callbacks passed to C library like this:
#[no_mangle]
extern "C" fn callback_handle(service: *mut c_void, size: *mut u32, buffer: *mut u8)
{
let id: u8 = c_library_get_id(service);
let instance: &mut RustService = /* get that from static container based on id */
if size = mem::size_of::<u32>() as u32 {
let value = buffer.cast::<u32>().read();
instance.value = Some(value);
} else {
panic!("Invalid length passed.");
}
}
What am I looking for is a proven/good practice that would apply here that works without causing deadlock, where my callback could refer back to the right instance using the library in C. That library in C can run multiple instances, however it's not that important. Alternatively I can accept a single instance pattern, that would work in this case.
Other methods of RustService would need to call for example a function with the following signature (from bindgen):
extern "C" pub fn clibCoreWork(status: *mut clibStatus, service: *mut clibService) -> u32;
and guts of that function may decide to call these callbacks mentioned above.
The approach I've taken shoots me in the foot as the inner Mutex deadlocks even in the single thread:
lazy_static! {
static ref MY_INSTANCES: Mutex<HashMap<u8, Weak<Mutex<RustService>>>> = Mutex::new(HashMap::new());
}
I dug the internet but fellow Rustaceans didn't seem to post anything about such problem.
Note: I cannot change underlying types in the C library I'm using. In the ideal world it would let me store pointer to user data and I'd brutally convert it to a mutable reference in unsafe code.
I've been diving into FFI with Rust, and while there are some useful posts and documentation on passing in strings and sometimes getting a string back as the return value from a C function, I don't see any way to get an out parameter. To get my feet wet, I am trying to access a C library that exposes a function that is declared as:
unsigned char *some_func(
const unsigned char *key,
unsigned int klen,
const unsigned char *src,
unsigned char *dest,
unsigned int slen);
I think it should be declared roughly as:
use libc;
extern "C" {
fn some_func(
key: *const libc::c_uchar,
klen: libc::c_uint,
src: *const libc::c_uchar,
dest: *mut libc::c_uchar,
slen: libc::c_uint) -> *mut libc::c_uchar;
}
// wrapper
pub fn wrapped_func(key: &str, src: &str) -> String {
unsafe {
let key_len = key.len() as u32;
let key = std::ffi::CString::new(key).unwrap();
let src_len = src.len() as u32;
let src = std::ffi::CString::new(src).unwrap();
let key_ptr = key.as_ptr();
let src_ptr = src.as_ptr();
let mut dest: *mut std::ffi::CStr;
let result = PC1(key_ptr, key_len, src_ptr, &mut dest, src_len);
// ^^^^^^^^^ expected `u8`, found *-ptr
key_ptr = ptr::null();
src_ptr = ptr::null();
let result_string = if let Ok(s) = result.to_str() {
String::from(s)
} else {
String::new()
};
dest = ptr::null();
result_string
}
There are a few issues I'm having with this:
I don't know how to properly declare dest as my out string. What's the right way to get an output string? From Wrapping Unsafe C Libraries in Rust, it looks like I need to treat this as a CStr:
For cases where you’re getting a const char * from a C function and want to convert it to something Rust can use, you’ll need to ensure it’s not null, then convert it into a &CStr with an appropriate lifetime. If you can’t figure out how to express an appropriate lifetime, the safest thing is to immediately convert it into an owned String!
The C code - and therefore my extern function - declares the strings as unsigned char *, but when I try to invoke, I have a type mismatch between u8 and i8. Is it acceptable to just switch the extern to be c_char, or is this dangerous?
Is setting key_ptr = ptr::null() the correct way (and all that's needed) to release the memory safely?
I have a struct returned to C code from Rust. I have no idea if it's a good way to do things, but it does work for rebuilding the struct and freeing memory without leaks.
#[repr(C)]
pub struct s {
// ...
}
#[repr(C)]
#[allow(clippy::box_vec)]
pub struct s_arr {
arr: *const s,
n: i8,
vec: Box<Vec<s>>,
}
/// Frees memory that was returned to C code
pub unsafe extern "C" fn free_s_arr(a: *mut s_arr) {
Box::from_raw(s_arr);
}
/// Generates an array for the C code
pub unsafe extern "C" fn gen_s_arr() -> *mut s_arr {
let many_s: Vec<s> = Vec::new();
// ... logic here
Box::into_raw(Box::new(s_arr {
arr: many_s.as_mut_ptr(),
n: many_s.len() as i8,
vec: many_s,
}))
}
The C header is currently written by hand, but I wanted to try out cbindgen. The manual C definition for s_arr is:
struct s_arr {
struct s *arr;
int8_t n;
void *_;
};
cbindgen generates the following for s_arr:
typedef struct Box_Vec_s Box_Vec_s;
typedef struct s_arr {
const s *arr;
int8_t n;
Box_Vec_s vec;
} s_arr;
This doesn't work since struct Box_Vec_s is not defined. Ideally I would just want to override the cbindgen type generated for vec to make it void * since it requires no code changes and thus no additional testing, but I am open to other suggestions.
I have looked through the cbindgen documentation, though not the examples, and couldn't find anything.
Your question is a bit unclear, but I think that if I understood you right, you're confusing two things and being led down a dark alley as a result.
In C, a dynamically-sized array, as you probably know, is identified by two things:
Its starting position, as a pointer
Its length
Rust follows the same convention - a Vec<_>, below the hood, shares the same structure (well, almost. It has a capacity as well, but that's beside the point).
Passing the boxed vector on top of a pointer is not only overkill, but extremely unwise. FFI bindings may be smart, but they're not smart enough to deal with a boxed complex type most of the time.
To solve this, we're going to simplify your bindings. I've added a single element in struct S to show you how it works. I've also cleaned up your FFI boundary:
#[repr(C)]
#[no_mangle]
pub struct S {
foo: u8
}
#[repr(C)]
pub struct s_arr {
arr: *mut S,
n: usize,
cap: usize
}
// Retrieve the vector back
pub unsafe extern "C" fn recombine_s_arr(ptr: *mut S, n: usize, cap: usize) -> Vec<S> {
Vec::from_raw_parts(ptr, n, cap)
}
#[no_mangle]
pub unsafe extern "C" fn gen_s_arr() -> s_arr {
let mut many_s: Vec<S> = Vec::new();
let output = s_arr {
arr: many_s.as_mut_ptr(),
n: many_s.len(),
cap: many_s.capacity()
};
std::mem::forget(many_s);
output
}
With this, cbindgen returns the expected header definitions:
typedef struct {
uint8_t foo;
} so58311426S;
typedef struct {
so58311426S *arr;
uintptr_t n;
uintptr_t cap;
} so58311426s_arr;
so58311426s_arr gen_s_arr(void);
This allows us to call gen_s_arr() from either C or Rust and retrieve a struct that is usable across both parts of the FFI boundary (so58311426s_arr). This struct contains all we need to be able to modify our array of S (well, so58311426S according to cbindgen).
When passing through FFI, you need to make sure of a few simple things:
You cannot pass raw boxes or non-primitive types; you will almost universally need to convert down to a set of pointers or change your definitions to accomodate (as I have done here)
You most definitely do not pass raw vectors. At most, you pass a slice, as that is a primitive type (see the point above).
You make sure to std::mem::forget() whatever you do not want to deallocate, and make sure to remember to deallocate it or reform it somewhere else.
I will edit this question in an hour; I have a plane to get on to. Let me know if any of this needs clarifications and I'll get to it once I'm in the right country :-)
This question already has an answer here:
How can I call a raw address from Rust?
(1 answer)
Closed 3 years ago.
Hello people of the internet,
I'm struggeling to invoke a function that is stored in a libc::c_void-Pointer. I can't tell Rust that the pointer is invokable and I can't figure out how to.
I want to translate this C++ Code
void * malloc(size_t size) {
static void *(*real_malloc)(size_t) = nullptr;
if (real_malloc == nullptr) {
real_malloc = reinterpret_cast<void *(*)(size_t)> (dlsym(RTLD_NEXT, "malloc"));
}
// do some logging stuff
void * ptr = real_malloc(size);
return ptr;
}
to Rust.
#[no_mangle]
pub extern fn malloc(bytes: usize) {
let c_string = "malloc\0".as_mut_ptr() as *mut i8; // char array for libc
let real_malloc: *mut libc::c_void = libc::dlsym(libc::RTLD_NEXT, c_string);
return real_malloc(bytes);
}
That's my progress so far after 1h of searching on the internet and trying. I'm new to Rust and not yet familiar with Rust/FFI / Rust with libc. I tried a lot with unsafe{}, casts with as but I always stuck at the following problem:
return real_malloc(bytes);
^^^^^^^^^^^^^^^^^^ expected (), found *-ptr
Q1: How can I call the function behind the void-Pointer stored in real_malloc?
Q2: Is my Rust-String to C-String conversion feasible this way?
I figured it out! Perhaps there is a better way but it works.
The trick is to "cast" the void-Pointer to c-function-Type with std::mem::transmute since it won't work with as
type LibCMallocT = fn(usize) -> *mut libc::c_void;
// C-Style string for symbol-name
let c_string = "malloc\0".as_ptr() as *mut i8; // char array for libc
// Void-Pointer to address of symbol
let real_malloc_addr: *mut libc::c_void = unsafe {libc::dlsym(libc::RTLD_NEXT, c_string)};
// transmute: "Reinterprets the bits of a value of one type as another type"
// Transform void-pointer-type to callable C-Function
let real_malloc: LibCMallocT = unsafe { std::mem::transmute(real_malloc_addr) }
When the shared object is build, one can verify that it works like this:
LD_PRELOAD=./target/debug/libmalloc_log_lib.so some-binary
My full code:
extern crate libc;
use std::io::Write;
const MSG: &str = "HELLO WORLD\n";
type LibCMallocT = fn(usize) -> *mut libc::c_void;
#[no_mangle] // then "malloc" is the symbol name so that ELF-Files can find it (if this lib is preloaded)
pub extern fn malloc(bytes: usize) -> *mut libc::c_void {
/// Disable logging aka return immediately the pointer from the real malloc (libc malloc)
static mut RETURN_IMMEDIATELY: bool = false;
// C-Style string for symbol-name
let c_string = "malloc\0".as_ptr() as *mut i8; // char array for libc
// Void-Pointer to address of symbol
let real_malloc_addr: *mut libc::c_void = unsafe {libc::dlsym(libc::RTLD_NEXT, c_string)};
// transmute: "Reinterprets the bits of a value of one type as another type"
// Transform void-pointer-type to callable C-Function
let real_malloc: LibCMallocT = unsafe { std::mem::transmute(real_malloc_addr) };
unsafe {
if !RETURN_IMMEDIATELY {
// let's do logging and other stuff that potentially
// needs malloc() itself
// This Variable prevent infinite loops because 'std::io::stdout().write_all'
// also uses malloc itself
// TODO: Do proper synchronisazion
// (lock whole method? thread_local variable?)
RETURN_IMMEDIATELY = true;
match std::io::stdout().write_all(MSG.as_bytes()) {
_ => ()
};
RETURN_IMMEDIATELY = false
}
}
(real_malloc)(bytes)
}
PS: Thanks to https://stackoverflow.com/a/46134764/2891595 (After I googled a lot more I found the trick with transmute!)
I would like to use the const C inside the impl R6502 without having to specify the scope R6502::
use bit::BitIndex;
pub struct R6502 {
pub sr: u8, // status register
}
impl R6502 {
// status flag indexs
const C: usize = 0;
const Z: usize = 1;
pub fn step(&mut self) {
self.sr.set_bit(R6502::C, false); // this is what I have to do
self.sr.set_bit(C, false); // this is what I want to do
}
}
I tried use self::C and some other combinations of use to only get errors about items not found.
useing of associated constants is not implemented in Rust 1.20. I haven't found an issue for that, so you can create your own issue in Rust GitHub repository.
In the meantime you can use type alias to reduce character count.
type P = R6502;
self.sr.set_bit(P::C, false);