Cannot set OpenCL kernel argument with buffer memory object - rust

I have the following simple OpenCL kernel, that simply copies all entries pointed at a to b
__kernel void mmcopy(__global float* a, __global float* b) {
unsigned pos = get_global_id(0);
b[pos] = a[pos];
}
The following code snippet shows the opencl function calls for creating a buffer memory object out of four floats, and setting the first argument on the kernel with the buffer object.
let mut v = [1f32, 1f32, 1f32, 1f32];
let size = mem::size_of_val(&v) as size_t;
let mut error_buffer = 0 as i32;
let buffer = unsafe {
clCreateBuffer(
context.id.unwrap(),
(CL_MEM_COPY_HOST_PTR | CL_MEM_READ_WRITE) as u64,
size,
v.as_mut_ptr() as *mut c_void,
&mut error_buffer,
)
};
let real_size = mem::size_of::<cl_mem>() as size_t;
let error = unsafe {
clSetKernelArg(
self.id.unwrap(), // here `self` is a wrapper. `id` is of type `cl_kernel`
0 as cl_uint,
real_size,
buffer as *const c_void,
)
};
However, executing the code results in an error CL_INVALID_MEM_OBJECT.
it looks like creating the buffer didn't succeed, but returned without an error.
The spec is also not very precise when it comes to describe the error in more detail:
for an argument declared to be a memory object when the specified arg_value is not a valid memory object.
note: the OpenCL functions, and types have been generated by rust-bindgen.
update 1
To clarify how the opaque types are represented in rust, here is the representation of cl_mem,
pub struct _cl_mem {
_unused: [u8; 0],
}
pub type cl_mem = *mut _cl_mem;
the ffi to clSetKernelArg
extern "C" {
pub fn clSetKernelArg(
kernel: cl_kernel,
arg_index: cl_uint,
arg_size: size_t,
arg_value: *const ::std::os::raw::c_void,
) -> cl_int;
}
and clCreateBuffer
extern "C" {
pub fn clCreateBuffer(
context: cl_context,
flags: cl_mem_flags,
size: size_t,
host_ptr: *mut ::std::os::raw::c_void,
errcode_ret: *mut cl_int,
) -> cl_mem;
}
In my understanding rust(-bindgen) uses zero sized types (ZST) to represent external opaque types. So basically cl_mem is already a pointer.
update 2
According to pmdj's answer the correct way is to pass a pointer to the cl_mem buffer
let error = unsafe {
clSetKernelArg(
self.id.unwrap(), // here `self` is a wrapper. `id` is of type `cl_kernel`
0 as cl_uint,
real_size,
&buffer as *const _ as *const c_void,
)
};
That actually fixes the problem, and set the return value to CL_SUCCESS. The spec for clSetKernelArg also mentions a pointer to data
A pointer to data that should be used as the argument value for argument specified by arg_index. The argument data pointed to by arg_value is copied and the arg_value pointer can therefore be reused by the application after clSetKernelArg returns. The argument value specified is the value used by all API calls that enqueue kernel (clEnqueueNDRangeKernel) until the argument value is changed by a call to clSetKernelArg for kernel [...]

Before I dig in, I'll point out that I'm a relative beginner with Rust and I'm not particularly familiar with what bindgen produces, but I know OpenCL quite well. So please bear with me if my Rust syntax is off.
The most obvious thing that sticks out for me is that passing the buffer to clSetKernelArg using buffer as *const c_void looks suspicious. My understanding is that your code is roughly equivalent to this C:
cl_int error_buffer = 0;
cl_mem buffer = clCreateBuffer(
context.id,
(CL_MEM_COPY_HOST_PTR | CL_MEM_READ_WRITE),
size,
v,
&error_buffer
);
size_t real_size = siezof(buffer);
cl_int error = clSetKernelArg(self.id, 0, real_size, buffer);
However, the last line is incorrect, it should be:
cl_int error = clSetKernelArg(self.id, 0, real_size, &buffer);
// yes, we want a POINTER to the buffer handle-------^
Although cl_mem is defined as a pointer to an opaque struct type, you need to pass the pointer to that pointer as the argument, just as with any other type of kernel argument: conceptually, I find it useful to think of it as clSetKernelArg performing a memcpy(internal_buffer, arg_value, arg_size); internally - so arg_size must always be the size of the object pointed to by arg_value. I find this helps me work out the correct level of indirection.
So in Rust this is probably along the lines of:
let error = unsafe {
clSetKernelArg(
self.id.unwrap(),
0 as cl_uint,
real_size,
&buffer as *const c_void,
)
};
but I haven't run it past rustc so it's probably wrong. You get the drift though.

Related

What is the correct way to drop a mem::forgotten array of strings sent to C via an out pointer and then passed back to Rust for deletion?

I would like to return some strings to C via a Rust FFI call. I also would like to ensure they're cleaned up properly.
I'm creating the strings on the Rust side and turning them into an address of an array of strings.
use core::mem;
use std::ffi::CString;
#[no_mangle]
pub extern "C" fn drop_rust_memory(mem: *mut ::libc::c_void) {
unsafe {
let boxed = Box::from_raw(mem);
mem::drop(boxed);
}
}
#[no_mangle]
pub extern "C" fn string_xfer(strings_out: *mut *mut *mut ::libc::c_char) -> usize {
unsafe {
let s1 = CString::new("String 1").unwrap();
let s2 = CString::new("String 2").unwrap();
let s1 = s1.into_raw();
let s2 = s2.into_raw();
let strs = vec![s1, s2];
let len = strs.len();
let mut boxed_slice = strs.into_boxed_slice();
*strings_out = boxed_slice.as_mut_ptr() as *mut *mut ::libc::c_char;
mem::forget(boxed_slice);
len
}
}
On the C side, I call the Rust FFI function, print the strings and then attempt to delete them via another Rust FFI call.
extern size_t string_xfer(char ***out);
extern void drop_rust_memory(void* mem);
int main() {
char **receiver;
int len = string_xfer(&receiver);
for (int i = 0; i < len; i++) {
printf("<%s>\n", receiver[i]);
}
drop_rust_memory(receiver);
printf("# rust memory dropped\n");
for (int i = 0; i < len; i++) {
printf("<%s>\n", receiver[i]);
}
return 0;
}
This appears to work. For the second printing after the drop, I would expect to get a crash or some undefined behavior, but I get this
<String 1>
<String 2>
# rust memory dropped
<(null)>
<String 2>
which makes me less sure about the entire thing.
First you may want take a look at Catching panic! when Rust called from C FFI, without spawning threads. Because panic will invoke undefined behaviour in this case. So you better catch the panic or avoid have code that can panic.
Secondly, into_boxed_slice() is primary use when you don't need vector feature any more so "A contiguous growable array type". You could also use as_mut_ptr() and forget the vector. That a choice either you want to carry the capacity information into C so you can make the vector grow or you don't want. (I think vector is missing a into_raw() method but I'm sure you can code one (just an example) to avoid critical code repetition). You could also use Box::into_raw() followed with a cast to transform the slice to pointer:
use std::panic;
use std::ffi::CString;
pub unsafe extern "C" fn string_xfer(len: &mut libc::size_t) -> Option<*mut *mut libc::c_char> {
if let Ok(slice) = panic::catch_unwind(move || {
let s1 = CString::new("String 1").unwrap();
let s2 = CString::new("String 2").unwrap();
let strs = vec![s1.into_raw(), s2.into_raw()];
Box::into_raw(strs.into_boxed_slice())
}) {
*len = (*slice).len();
Some(slice as _)
} else {
None
}
}
Third, your drop_rust_memory() only drop a pointer, I think you are doing a total UB here. Rust memory allocation need the real size of the allocation (capacity). And you didn't give the size of your slice, you tell to Rust "free this pointer that contain a pointer to nothing (void so 0)" but that not the good capacity. You need to use from_raw_parts_mut(), your C code must give the size of the slice to the Rust code. Also, you need to properly free your CString you need to call from_raw() to do it (More information here):
use std::ffi::CString;
pub unsafe extern "C" fn drop_rust_memory(mem: *mut *mut libc::c_char, len: libc::size_t) {
let slice = Box::from_raw(std::slice::from_raw_parts_mut(mem, len));
for &x in slice.iter() {
CString::from_raw(x);
} // CString will free resource don't use mem/vec element after
}
To conclude, you should read more about undefined behaviour, it's not about "expect a crash" or "something" should happen. When your program trigger a UB, everything can happen, you go into a random zone, read more about UB on this amazing LLVM blog post
Note about C style prefer return the pointer and not the size because strings_out: *mut *mut *mut ::libc::c_char is a ugly thing so do pub extern fn string_xfer(size: &mut libc::size_t) -> *mut *mut libc::c_char. Also, How to check if function pointer passed from C is non-NULL

Why does a Box pointer passed to C and back to Rust segfault?

Some C code calls into the Rust open call below which returns a pointer. Later the C code passes the exact same pointer back to the close function which tries to drop (free) it. It segfaults in free(3). Why?
use std::os::raw::{c_int, c_void};
struct Handle;
extern "C" fn open(_readonly: c_int) -> *mut c_void {
let h = Handle;
let h = Box::new(h);
return Box::into_raw(h) as *mut c_void;
}
extern "C" fn close(h: *mut c_void) {
let h = unsafe { Box::from_raw(h) };
// XXX This segfaults - why?
drop(h);
}
In close, you end up creating a Box<c_void> instead of a Box<Handle> because you didn't cast the *mut c_void back to *mut Handle before invoking Box::from_raw.
fn close(h: *mut c_void) {
let h = unsafe { Box::from_raw(h as *mut Handle) };
drop(h);
}
By the way, Box doesn't actually allocate any memory for a zero-sized type (such as Handle here) and uses a fixed, non-zero pointer value instead (which, in the current implementation, is the alignment of the type; a zero-sized type has an alignment of 1 by default). The destructor for a boxed zero-sized type knows not to try to deallocate memory at this fictitious memory address, but c_void is not a zero-sized type (it has size 1), so the destructor for Box<c_void> tries to free memory at address 0x1, which causes the segfault.
If Handle wasn't zero-sized, then the code may not crash, but it would still run the wrong destructor (it'd run c_void's destructor, which does nothing), and this may cause memory leaks. A destructor runs Drop::drop for the type if present, then drops the type's fields.
The problem is you didn't cast the pointer back to a Handle pointer while transforming it back to a Box, and got a Box of the wrong type.
This works:
fn close(h: *mut c_void) {
let h = unsafe { Box::from_raw(h as *mut Handle) };
// ^^^^^^^^^^^^^^
drop(h);
}
In your code, h is a std::boxed::Box<std::ffi::c_void>.

Create *mut *mut to a struct

I'm trying to call pthread_join with a pointer to my struct in order that the C thread can fill in the struct to the memory I point it to. (Yes, I'm aware that this is highly unsafe..)
The function signature of pthread_join:
pub unsafe extern fn pthread_join(native: pthread_t,
value: *mut *mut c_void)
-> c_int
I'm doing this as an exercise of porting C code from a book to Rust. The C code:
pthread_t tid1;
struct foo *fp;
err = pthread_create(&tid1, NULL, thr_fn1, NULL);
err = pthread_join(tid1, (void *)&fp);
I came up with this code:
extern crate libc;
use libc::{pthread_t, pthread_join};
struct Foo {}
fn main() {
let tid1:pthread_t = std::mem::uninitialized();
let mut fp:Box<Foo> = std::mem::uninitialized();
let value = &mut fp;
pthread_join(tid1, &mut value);
}
But the error I see is:
error[E0308]: mismatched types
--> src/bin/11-threads/f04-bogus-pthread-exit.rs:51:24
|
51 | pthread_join(tid1, &mut value);
| ^^^^^^^^^^ expected *-ptr, found mutable reference
|
= note: expected type `*mut *mut libc::c_void`
found type `&mut &mut std::boxed::Box<Foo>`
Is it even possible to achieve this just using casts, or do I need to transmute?
There are several issues here:
Box is a pointer to a heap-allocated resource, you can extract the pointer itself using Box::into_raw(some_box),
References are not silently coerced into pointers (even though they have the same representation), you need an explicit cast,
You need to cast from your concrete type to c_void, type inference may be able to do that
You have a reference to a reference to a pointer, you need a pointer to a pointer; you have one too many levels of indirection.
Let's make it work:
// pthread interface, reduced
struct Void;
fn sample(_: *mut *mut Void) {}
// actual code
struct Foo {}
fn main() {
let mut p = Box::into_raw(Box::new(Foo{})) as *mut Void;
sample(&mut p as *mut _);
}
Note that this is leaking memory (as a result of into_raw), normally the memory should be shoved back into a Box with from_raw for the destructor of Foo to be called and the memory to be freed.
The code can't work as written; that is because the C thread doesn't really "fill in the struct" in the memory you point to. It is responsible for allocating its own memory (or receiving it from another thread beforehand) and filling it out. The only thing the C thread "returns" is a single address, and this address is picked up by pthread_join.
This is why pthread_join receives a void **, i.e. the pointer to a void *. That kind of output parameter enables pthread_join to store (return) the void * pointer provided by the freshly finished thread. The thread can provide the pointer either by passing it to pthread_exit or by returning it from the start_routine passed to pthread_create. In Rust, the raw pointer can be received with code like this:
let mut c_result: *mut libc::c_void = ptr::null_mut();
libc::pthread_join(tid1, &mut c_result as *mut _);
// C_RESULT now contains the raw pointer returned by the worker's
// start routine, or passed to pthread_exit()
The contents and size of the memory that the returned pointer points to are a matter of contract between the thread being joined and the thread that is joining it. If the worker thread is implemented in C and designed to be invoked by other C code, then an obvious choice is for it to allocate memory for a result structure, fill it out, and provide a pointer to allocated memory. For example:
struct ThreadResult { ... };
...
ThreadResult *result = malloc(sizeof(struct ThreadResult));
result->field1 = value1;
...
pthread_exit(result);
In that case your Rust code that joins the thread can interpret the result by replicating the C structure and picking up its ownership:
// obtain a raw-pointer c_result through pthread_join as
// shown above:
let mut c_result = ...;
libc::pthread_join(tid1, &mut c_result as *mut _);
#[repr(C)]
struct ThreadResult { ... } // fields copy-pasted from C
unsafe {
// convert the raw pointer to a Rust reference, so that we may
// inspect its contents
let result = &mut *(c_result as *mut ThreadResult);
// ... inspect result.field1, etc ...
// free the memory allocated in the thread
libc::free(c_result);
// RESULT is no longer usable
}

GraphicsMagick FFI issue

As an exercise, I'm attempting to write a GraphicsMagick FFI wrapper in Rust. I'm having an issue replicating some reference C code:
Demo C code:
Image
*image = (Image *) NULL;
ImageInfo
*imageInfo;
ExceptionInfo
exception;
InitializeMagick(NULL);
imageInfo=CloneImageInfo(0);
GetExceptionInfo(&exception);
And here is my (naive) translation to Rust:
let img: *mut ffi::Image;
let img_info: *mut ffi::ImageInfo;
let exception: *mut ffi::ExceptionInfo = ptr::null_mut();
unsafe {
ffi::InitializeMagick(ptr::null_mut());
img_info =
ffi::CloneImageInfo(ptr::null_mut() as *const ffi::ImageInfo);
ffi::GetExceptionInfo(exception);
// ...
}
This compiles just fine, but when I try to run it, I see:
magick/error.c:388: GetExceptionInfo: Assertion `exception != (ExceptionInfo *) ((void *)0)' failed
which is caused by ffi::GetExceptionInfo(exception). The only difference seems to be that the C exception isn't "initialized", but I don't know enough about C to know if there is a difference between a null and an empty/uninitialized pointer.
The difference between your C and Rust code is that the C version allocates an ExceptionInfo instance on the stack and passes into the GetExceptionInfo a pointer referencing that instance.
Your Rust code, on the other hand, passes a NULL pointer.
GetExceptionInfo specifically guards against being passed a NULL pointer, you can see the assertion's code here, in magick/error.c.
I don't know what kind of FFI bindings you use, but if the ExceptionInfo is fully defined in them then you should be able to allocate it on the stack and pass a reference to it just like in the C version:
let mut exception: ffi::ExceptionInfo = unsafe {std::mem::uninitialized()};
unsafe {ffi::GetExceptionInfo (&mut exception);}
The error message states (rewritten a bit):
Assertion exception != NULL failed
That is, you cannot pass NULL to that method. Note the C code:
ExceptionInfo exception;
This is not a pointer. You need to allocate space for it and then pass in a reference to the allocated space.
The documentation shows the definition:
typedef struct _ExceptionInfo
{
char
*reason,
*description;
ExceptionType
severity;
unsigned long
signature;
} ExceptionInfo;
You will need to represent this in Rust. Something like this untested code:
extern crate libc;
#[repr(C)]
struct ExceptionInfo {
reason: *const libc::c_char,
description: *const libc::c_char,
severity: ExceptionType,
signature: libc::c_ulong,
}
#[repr(C)]
enum ExceptionType {
UndefinedException,
WarningException = 300,
// the rest
}
Then you need to allocate it and pass a reference. More untested code:
let img_info;
let mut exception = ffi::ExceptionInfo::new();
unsafe {
ffi::InitializeMagick(ptr::null_mut());
img_info =
ffi::CloneImageInfo(ptr::null_mut() as *const ffi::ImageInfo);
ffi::GetExceptionInfo(&mut exception);
// ...
}
Note that Rust style is 4 space indents.

static struct with C strings for lv2 plugin [duplicate]

This question already has answers here:
Creating a static C struct containing strings
(3 answers)
Closed 7 years ago.
I'm trying to learn Rust (newbie in low level programming), and want to translate a tiny lv2 amplifier (audio) plugin "amp.c" (C-code) from C to Rust. I actually got it working (here), but when the host terminates, valgrind says that "
64 bytes in 1 blocks are definitely lost". I think I know why this happens, but I don't know how to fix it.
Before you get tired of reading, here is the final question:
How do I statically allocate a struct that contains a C string?
And here is the introduction:
Why it happens (I think):
Host loads the library and calls lv2_descriptor()
const LV2_Descriptor*
lv2_descriptor()
{
return &descriptor;
}
which returns a pointer to a STATICALLY allocated struct of type LV2_Descriptor,
static const LV2_Descriptor descriptor = {
AMP_URI,
...
};
which is defined as
typedef struct _LV2_Descriptor {
const char * URI;
...
} LV2_Descriptor;
Why is it statically allocated? In the amp.c it says:
It is best to define descriptors statically to avoid leaking memory
and non-portable shared library constructors and destructors to clean
up properly.
However, I translated lv2_descriptor() to Rust as:
#[no_mangle]
pub extern fn lv2_descriptor(index:i32) -> *const LV2Descriptor {
let s = "http://example.org/eg-amp_rust";
let cstr = CString::new(s).unwrap();
let ptr = cstr.as_ptr();
mem::forget(cstr);
let mybox = Box::new(LV2Descriptor{amp_uri: ptr}, ...);
let bxptr = &*mybox as *const LV2Descriptor;
mem::forget(mybox);
return bxptr
}
So it's not statically allocated and I never free it, that's I guess why valgrind complains?
How am I trying to solve it?
I'm trying to do the same thing in Rust as the C-code does, i.e. statically allocate the struct (outside of lv2_descriptor()). The goal is to be fully compatible to the lv2 library, i.e "...to avoid leaking memory..." etc., as it says in the quote, right? So I tried something like:
static ptr1: *const u8 = (b"example.org/eg-amp_rust\n\0").as_ptr();
static ptr2: *const libc::c_char = ptr1 as *const libc::c_char;
static desc: LV2Descriptor = LV2Descriptor{amp_uri: ptr2, ...};
But this does not compile, there are error messages like
src/lib.rs:184:26: 184:72 error: the trait `core::marker::Sync` is not implemented for the type `*const u8` [E0277]
src/lib.rs:184 static ptr1: *const u8 = b"http://example.org/eg-amp_rust\n\0".as_ptr();
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/lib.rs:184:26: 184:72 note: `*const u8` cannot be shared between threads safely
src/lib.rs:184 static ptr1: *const u8 = b"http://example.org/eg-amp_rust\n\0".as_ptr();
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/lib.rs:184:26: 184:72 error: static contains unimplemented expression type [E0019]
src/lib.rs:184 static ptr1: *const u8 = b"http://example.org/eg-amp_rust\n\0".as_ptr();
Specific problem/question:
How do I statically allocate a struct that contains a C string?
The short answer is, you don't for now. Future Rust will probably gain this ability.
What you can do, is statically allocate a struct that contains null pointers, and set those null pointers to something useful when you call the function. Rust has static mut. It requires unsafe code, is not threadsafe at all and is (to the best of my knowledge) considered a code smell.
Right here I consider it a workaround to the fact that there is no way to turn a &[T] into a *const T in a static.
static S: &'static [u8] = b"http://example.org/eg-amp_rust\n\0";
static mut desc: LV2Descriptor = LV2Descriptor {
amp_uri: 0 as *const libc::c_char, // ptr::null() isn't const fn (yet)
};
#[no_mangle]
pub extern fn lv2_descriptor(index: i32) -> *const LV2Descriptor {
let ptr = S.as_ptr() as *const libc::c_char;
unsafe {
desc.amp_uri = ptr;
&desc as *const LV2Descriptor
}
}

Resources