I want to write a FFI wrapper for sn_api library, which contains async functions. It will be used in single-threaded non-async code written in Red.
I found, that the easy way is to use Runtime::new().unwrap().block_on(...) in every exported function, although it involves a lot of creating new Tokio runtimes and seem to be too heavy to be run on every call:
use std::os::raw::c_char;
use std::ffi::{CString, CStr};
use sn_api::{BootstrapConfig, Safe};
use tokio::runtime::Runtime;
#[no_mangle]
pub extern "C" _safe_connect(ptr: *const Safe, bootstrap_contact: *const c_char) {
assert!(!ptr.is_null());
let _safe = unsafe {
&*ptr
};
let bootstrap_contact = unsafe {
CStr::from_ptr(bootstrap_contact)
}
let mut bootstrap_contacts = BootstrapConfig::default();
bootstrap_contacts.insert(bootstrap_contact.parse().expect("Invalid bootstrap address"));
// how to reuse the Runtime in other functions?
Runtime::new().unwrap().block_on(_safe.connect(None, None, Some(bootstrap_contacts)));
}
Is it possible to run all async functions on a common Runtime? I imagine it would require creating some singleton / global, but my library is compiled with crate-type = ["cdylib"], which seems not a good place for globals. What would be the best approach?
I've decided for an approach, where I create a Tokio Runtime, and then pass it to every FFI function call containing async code:
use std::os::raw::c_char;
use std::ffi::{CString, CStr};
use sn_api::{BootstrapConfig, Safe};
use tokio::runtime::Runtime;
#[no_mangle]
pub extern "C" fn init_runtime() -> *mut Runtime {
Box::into_raw(Box::new(Runtime::new().unwrap()))
}
#[no_mangle]
pub extern "C" _safe_connect(rt_ptr: *mut Runtime, safe_ptr: *mut Safe, bootstrap_contact: *const c_char) {
assert!(!safe_ptr.is_null());
assert!(!rt_ptr.is_null());
let bootstrap_contact = unsafe {
CStr::from_ptr(bootstrap_contact)
}
let mut bootstrap_contacts = BootstrapConfig::default();
bootstrap_contacts.insert(bootstrap_contact.parse().expect("Invalid bootstrap address"));
unsafe {
let _safe = &mut *safe_ptr;
let rt = &mut *rt_ptr;
rt.block_on(_safe.connect(None, None, Some(bootstrap_contacts))).unwrap();
}
}
I faced the same issue. Here is my cut: export-tokio-to-lib.
plugin.rs:
use async_ffi::{FfiFuture, FutureExt};
use tokio::runtime::Handle;
/// # Safety
#[no_mangle]
pub unsafe extern "C" fn test(arg: f32, handle: *const Handle) -> FfiFuture<safer_ffi::String> {
let handle = &*handle;
async move {
let _enter = handle.enter();
tokio::time::sleep(std::time::Duration::from_secs_f32(arg)).await;
format!("slept {arg} secs").into()
}
.into_ffi()
}
Try this.
From this:
#[tokio::main]
async fn main() {
println!("hello");
}
Transformed into:
fn main() {
let mut rt = tokio::runtime::Runtime::new().unwrap();
rt.block_on(async {
println!("hello");
})
}
Reference: https://tokio.rs/tokio/tutorial/hello-tokio#async-main-function
Related
I have a library that is used through its rust interface by rust programs, as well as through C/C++ programs through generated cbindgen bindings, so I implemented a free function to free the string once the ffi function has used the string. However I want rust also to control the memory when it is used as a rust lib. How do I achieve this? is it even possible? or is calling the free function manually in rust the only option?
I also tried implementing drop, but that lead to this:
free(): double free detected in tcache 2
[1] 11097 IOT instruction cargo run
This block allows the string to be freed from C/C++, but the string is not freed in rust (valgrind shows definitely lost block). data is assigned using CString::into_raw()
use std::{ffi::CString, os::raw::c_char};
pub struct SomeData {
pub data: *const c_char
}
impl SomeData {
#[no_mangle] pub extern fn free_shared_string(&mut self) {
if !self.data.is_null() {
unsafe { CString::from_raw(self.data.cast_mut()); }
}
}
}
The docs for from_raw warn against doing exactly this.
Safety
This should only ever be called with a pointer that was earlier obtained by calling CString::into_raw. Other usage (e.g., trying to take ownership of a string that was allocated by foreign code) is likely to lead to undefined behavior or allocator corruption.
So do not use from_raw to pretend that a foreign string was allocated using Rust. If you just need to borrow it and let C free it, you should use the CStr type for borrowed strings. If you want to take ownership, you should copy it into a new string, or wrap it in a custom structure that has a Drop implementation capable of freeing the original memory.
You cannot have two different languages owning that memory. Rust is fundamentally built on a single-ownership model, so every piece of memory has a unique owner. There are some (intra-Rust) workarounds for that like Rc, but none of that will translate to C. So pick an owner, and make that language responsible for freeing the data.
The best solution for me was to have a separate feature, used when building the library to be used through C/C++ applications (ie. .a/.so) vs .rlib which cargo will build when included in a rust project through Cargo.toml.
This lets me use the same API from both possible application languages, call free from C/C++ on my string, and drop will free it in rust.
Note: the null character at the end is because the majority of the time my lib is used with C apps, hence storing with null for faster returns for them.
Add default-features = false when adding in Cargo.toml of a rust app.
lib.rs
use std::{ffi::{c_char, CStr, FromBytesWithNulError, CString}, mem::forget, str::Utf8Error, string::FromUtf8Error};
#[cfg(feature = "c-str")]
#[repr(C)]
pub struct SharedString {
str: *const c_char
}
#[cfg(not(feature = "c-str"))]
pub struct SharedString {
str: Vec<u8>
}
#[cfg(feature = "c-str")]
impl SharedString {
pub fn from_bytes(buf: &[u8]) -> Self {
let mut buf = buf.to_vec();
if let Some(c) = buf.last() {
if *c != 0 {
buf.push(0);
}
}
let s = Self { str: buf.as_ptr().cast() };
forget(buf);
s
}
pub unsafe fn get_string(&self) -> Result<String, SharedStringError> {
Ok(CStr::from_ptr(self.str).to_str()?.to_owned())
}
pub unsafe fn free(&self) {
if !self.str.is_null() {
let _ = CString::from_raw(self.str.cast_mut());
}
}
}
#[cfg(not(feature = "c-str"))]
impl SharedString {
pub fn from_bytes(buf: &[u8]) -> Self {
let mut buf = buf.to_vec();
if let Some(c) = buf.last() {
if *c != 0 {
buf.push(0);
}
}
Self { str: buf }
}
pub fn get_string(&self) -> Result<String, SharedStringError> {
let mut s = self.str.clone();
if let Some(c) = s.last() {
if *c == 0 {
s.pop();
}
}
String::from_utf8(s).map_err(|e| e.into())
}
// do nothing because rust vec will get dropped automatically
pub fn free(&self) {}
}
// Just for proof of concept
#[derive(Debug)]
pub enum SharedStringError {
NullError,
Utf8Error
}
impl From<FromBytesWithNulError> for SharedStringError {
fn from(_: FromBytesWithNulError) -> Self {
Self::NullError
}
}
impl From<Utf8Error> for SharedStringError {
fn from(_: Utf8Error) -> Self {
Self::Utf8Error
}
}
impl From<FromUtf8Error> for SharedStringError {
fn from(_: FromUtf8Error) -> Self {
Self::Utf8Error
}
}
Cargo.toml
[package]
name = "mylib"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
[features]
default = ["c-str"]
c-str = []
I'm building an interpreter with a garbage collector. I want a thread-local nursery region, and a shared older region. I am having trouble setting up the nursery. I have:
const NurserySize : usize = 25000;
#[thread_local]
static mut NurseryMemory : [usize;NurserySize] = [0;NurserySize];
thread_local! {
static Nursery: AllocableRegion = AllocableRegion::makeLocal(unsafe{&mut NurseryMemory});
}
#[cfg(test)]
mod testMemory {
use super::*;
#[test]
fn test1() {
Nursery.with(|n| n.allocObject(10));
}
}
First question is why do I need the unsafe - NurseryMemory is thread local, so access can't be unsafe?
Second question is how can I actually use this? The code is at playground, but it doesn't compile and attempts I've made to fix it seem to make it worse.
1. Why is unsafe required to get a reference to a mutable ThreadLocal?
The same reason that you need unsafe for a normal mutable static,
you would be able to create aliasing mut pointers in safe code.
The following incorrectly creates two mutable references to the mutable thread local.
#![feature(thread_local)]
#[thread_local]
static mut SomeValue: Result<&str, usize> = Ok("hello world");
pub fn main() {
let first = unsafe {&mut SomeValue};
let second = unsafe {&mut SomeValue};
if let Ok(string) = first {
*second = Err(0); // should invalidate the string reference, but it doesn't
println!("{}", string) // as first and second are considered to be disjunct
}
}
first wouldn't even need to be a mutable reference for this to be a problem.
2. How to fix the code?
You could use a RefCell around the AllocatableRegion to dynamically enforce the borrowing of the inner value.
const NurserySize : usize = 25000;
#[thread_local]
static mut NurseryMemory : [usize;NurserySize] = [0;NurserySize];
thread_local! {
static Nursery: RefCell<AllocableRegion> = RefCell::new(AllocableRegion::makeLocal(unsafe{&mut NurseryMemory}));
}
#[cfg(test)]
mod testMemory {
use super::*;
#[test]
fn test1() {
Nursery.with(|n| n.borrow_mut().allocObject(10));
}
}
You don't need unsafe to make a mutable thread-local struct in rust. However, Nursery does need to be a RefCell. This is sufficient:
use std::cell::RefCell;
const NURSERY_SIZE: usize = 300;
thread_local! {
static NURSERY: RefCell<[usize; NURSERY_SIZE]> = RefCell::new([0; NURSERY_SIZE]);
}
#[cfg(test)]
mod test_memory {
use super::*;
#[test]
fn test1() {
NURSERY.with(|n| n.borrow_mut()[10] = 20);
}
}
Rust Playground link
I am working on embedded Rust code for LPC82X series controllers from NXP - the exact toolchain does not matter for the question.
These controllers contain peripheral drivers in ROM. I want to use these drivers, which means I need to use unsafe Rust and FFI without linking actual code.
The ROM APIs expose function pointers packed into C structs at specific address locations. If somebody wants the details of this API, chapter 29 of the LPC82X manual describes the API in question.
My Rust playground dummy sketch looks like this, that would be hidden from application code, by a yet unwritten I2C abstraction lib. This compiles.
#![feature(naked_functions)]
const I2C_ROM_API_ADDRESS: usize = 0x1fff_200c;
static mut ROM_I2C_API: Option<&RomI2cApi> = None;
#[repr(C)]
struct RomI2cApi {
// Dummy functions, real ones take arguments, and have different return
// These won't be called directly, only through the struct's implemented methods
// value
master_transmit_poll: extern "C" fn() -> bool,
master_receive_poll: extern "C" fn() -> bool,
}
impl RomI2cApi {
fn api_table() -> &'static RomI2cApi {
unsafe {
match ROM_I2C_API {
None => RomI2cApi::new(),
Some(table) => table,
}
}
}
unsafe fn new() -> &'static RomI2cApi {
ROM_I2C_API = Some(&*(I2C_ROM_API_ADDRESS as *const RomI2cApi));
ROM_I2C_API.unwrap()
}
#[inline]
fn master_transmit_poll(&self) -> bool {
(self.master_transmit_poll)()
}
#[inline]
fn master_receive_poll(&self) -> bool {
(self.master_receive_poll)()
}
}
impl From<usize> for &'static RomI2cApi {
fn from(address: usize) -> &'static RomI2cApi {
unsafe { &*(address as *const RomI2cApi) }
}
}
fn main() {
let rom_api = unsafe { RomI2cApi::api_table() };
println!("ROM I2C API address is: {:p}", rom_api);
// Should be commented out when trying !
rom_api.master_transmit_poll();
}
I cannot declare the function pointer structs as non-mutable static as statics have many restrictions, including not dereferencing pointers in the assignment. Is there a better workaround than Option? Using Option with the api_table function at least guarantees that initialization happens.
You can get around having a static at all:
const ROM_I2C_API: &RomI2cApi = &*(0x1fff_200c as *const RomI2cApi);
Not yet working, but is planned to work in the future. For now use
const ROM_I2C_API: *const RomI2cApi = 0x1fff_200c as *const RomI2cApi;
fn api_table() -> &'static RomI2cApi {
unsafe { &*(ROM_I2C_API) }
}
This creates a &'static RomI2cApi and allows you to access the functions everywhere directly by calling api_table().master_transmit_poll()
mexPrintf, just like printf, accepts a varargs list of arguments, but I don't know what the best way to wrap this is in Rust. There is a RFC for variadic generics, but what can we do today?
In this example, I want to print of the number of inputs and outputs, but the wrapped function just prints garbage. Any idea how to fix this?
#![allow(non_snake_case)]
#![allow(unused_variables)]
extern crate mex_sys;
use mex_sys::mxArray;
use std::ffi::CString;
use std::os::raw::c_int;
use std::os::raw::c_void;
type VarArgs = *mut c_void;
// attempt to wrap mex_sys::mexPrintf
fn mexPrintf(fmt: &str, args: VarArgs) {
let cs = CString::new(fmt).unwrap();
unsafe {
mex_sys::mexPrintf(cs.as_ptr(), args);
}
}
#[no_mangle]
pub extern "system" fn mexFunction(
nlhs: c_int,
plhs: *mut *mut mxArray,
nrhs: c_int,
prhs: *mut *mut mxArray,
) {
let hw = CString::new("hello world\n").unwrap();
unsafe {
mex_sys::mexPrintf(hw.as_ptr());
}
let inout = CString::new("%d inputs and %d outputs\n").unwrap();
unsafe {
mex_sys::mexPrintf(inout.as_ptr(), nrhs, nlhs);
}
mexPrintf("hello world wrapped\n", std::ptr::null_mut());
let n = Box::new(nrhs);
let p = Box::into_raw(n);
mexPrintf("inputs %d\n", p as VarArgs);
let mut v = vec![3];
mexPrintf("vec %d\n", v.as_mut_ptr() as VarArgs);
}
Contrary to popular belief, it is possible to call variadic / vararg functions that were defined in C. That doesn't mean that doing so is very easy, and it's definitely even easier to do something bad because there are even fewer types for the compiler to check your work with.
Here's an example of calling printf. I've hard-coded just about everything:
extern crate libc;
fn my_thing() {
unsafe {
libc::printf(b"Hello, %s (%d)\0".as_ptr() as *const i8, b"world\0".as_ptr(), 42i32);
}
}
fn main() {
my_thing()
}
Note that I have to very explicitly make sure my format string and arguments are all the right types and the strings are NUL-terminated.
Normally, you'll use tools like CString:
extern crate libc;
use std::ffi::CString;
fn my_thing(name: &str, number: i32) {
let fmt = CString::new("Hello, %s (%d)").expect("Invalid format string");
let name = CString::new(name).expect("Invalid name");
unsafe {
libc::printf(fmt.as_ptr(), name.as_ptr(), number);
}
}
fn main() {
my_thing("world", 42)
}
The Rust compiler test suite also has an example of calling a variadic function.
A word of warning specifically for printf-like functions: C compiler-writers realized that people screw up this particular type of variadic function call all the time. To help combat that, they've encoded special logic that parses the format string and attempts to check the argument types against the types the format string expect. The Rust compiler will not check your C-style format strings for you!
I had confused a "variable list of arguments" with a va_list. I'm going to avoid both if I can and in this situation, I'm just going to do the string formatting in Rust before passing it to interop. Here is what worked for me in this case:
#![allow(non_snake_case)]
#![allow(unused_variables)]
extern crate mex_sys;
use mex_sys::mxArray;
use std::ffi::CString;
use std::os::raw::c_int;
// attempt to wrap mex_sys::mexPrintf
fn mexPrintf(text: &str) {
let cs = CString::new(text).expect("Invalid text");
unsafe {
mex_sys::mexPrintf(cs.as_ptr());
}
}
#[no_mangle]
pub extern "C" fn mexFunction(
nlhs: c_int,
plhs: *mut *mut mxArray,
nrhs: c_int,
prhs: *mut *mut mxArray,
) {
mexPrintf(&format!("{} inputs and {} outputs\n", nrhs, nlhs));
}
I've read the documentation for std::net and mio, and I've found some methods like set_nodelay and set_keepalive, but I haven't found a way to set other socket options like SO_REUSEPORT and SO_REUSEADDR on a given socket. How can I do this?
Because SO_REUSEPORT isn't cross-platform, you will need to dip into platform-specific code. In this case, you can get the raw file descriptor from the socket and then use functions, types, and values from the libc crate to set the options you want:
extern crate libc; // 0.2.43
use std::{io, mem, net::TcpListener, os::unix::io::AsRawFd};
fn main() -> Result<(), io::Error> {
let listener = TcpListener::bind("0.0.0.0:8888")?;
unsafe {
let optval: libc::c_int = 1;
let ret = libc::setsockopt(
listener.as_raw_fd(),
libc::SOL_SOCKET,
libc::SO_REUSEPORT,
&optval as *const _ as *const libc::c_void,
mem::size_of_val(&optval) as libc::socklen_t,
);
if ret != 0 {
return Err(io::Error::last_os_error());
}
}
Ok(())
}
I make no guarantee that this is the right place to set this option, or that I haven't screwed up something in the unsafe block, but it does compile and run on macOS 10.12.
A better solution may be to check out the nix crate, which provides nicer wrappers for most *nix-specific code:
extern crate nix; // 0.11.0
use nix::sys::socket::{self, sockopt::ReusePort};
use std::{error::Error, net::TcpListener, os::unix::io::AsRawFd};
fn main() -> Result<(), Box<Error>> {
let listener = TcpListener::bind("0.0.0.0:8888")?;
socket::setsockopt(listener.as_raw_fd(), ReusePort, &true)?;
Ok(())
}
An even better solution may be to check out the net2 crate, which provides higher-level methods aimed specifically at networking-related code:
extern crate net2; // 0.2.33
use net2::{unix::UnixTcpBuilderExt, TcpBuilder};
fn main() -> Result<(), std::io::Error> {
let listener = TcpBuilder::new_v4()?
.reuse_address(true)?
.reuse_port(true)?
.bind("0.0.0.0:8888")?
.listen(42)?;
Ok(())
}