Assuming I have a struct defined as such:
cdef extern from "blah.h":
struct my_struct:
int a
int b
I need to be able to convert a dict into my_struct, without assuming any knowledge of my_struct's fields. In other words, I need the following conversion to happen:
def do_something(dict_arg):
cdef my_struct s = dict_arg
do_somthing_with_s(s)
The problem is that Cython won't do it: http://docs.cython.org/src/userguide/language_basics.html#automatic-type-conversions
Of course, If I had knowledge on the my_struct field's name, I could do this:
def do_something(dict_arg):
cdef my_struct s = my_struct(a=dict_arg['a'], b=dict_arg['b'])
do_somthing_with_s(s)
Doing this crashes the cython compiler:
def do_something(dict_arg):
cdef my_struct s = my_struct(**dict_arg)
do_somthing_with_s(s)
The reason why I don't know the field's name is because the code is autogenerated, and I don't want to make an ugly hack to handle this case.
How to initialize a struct from a Python dict with Cython ?
You have to set each element of the structure manually. There are no shortcuts.
If your code is auto generated then should be easy to autogenerate also an inline function that
does the conversion from PyDict to each of your structs.
Related
I'm trying to wrap my head around Rust's aliasing rules in the following situation:
Let's assume we have a memory allocation in C. We pass a pointer to this allocation to Rust. The Rust function does something with the allocation, and then calls back to C code (without any parameters), where another rust function is called with the same allocation as parameter. For now, let's assume that only the first Rust function gets a mutable reference.
The call stack would look like:
Some C Code (owns data)
Rust(pointer as &mut)
Some C code (does not get parameters from Rust)
Rust(pointer as &)
As a short example, let's assume the following two files:
test.c
#include <stdio.h>
#include <stdlib.h>
void first_rust_function(int * ints);
void another_rust_function(const int * ints);
int * arr;
void called_from_rust() {
another_rust_function(arr);
}
int main(int argc, char ** argv) {
arr = malloc(3*sizeof(int));
arr[0]=3;
arr[1]=4;
arr[2]=53;
first_rust_function(arr);
free(arr);
}
test.rs
use std::os::raw::c_int;
extern "C" { fn called_from_rust(); }
#[no_mangle]
pub extern "C" fn first_rust_function(ints : &mut [c_int;3]) {
ints[1] = 7;
unsafe { called_from_rust() };
}
#[no_mangle]
pub extern "C" fn another_rust_function(ints : &[c_int;3]) {
println!("Second value: {}", ints[1])
}
(For the sake of completeness: running this code prints "Second value: 7")
Please note that the call back to C from Rust (called_from_rust()) does not have any parameters. The Rust compiler therefore does not have the information that anyone might read from the pointed-to value.
My gut feeling tells me that this is undefined behaviour, but I'm not certain.
I've had a quick look at Stacked Borrows, and that model is violated. In the above example only the Rule (protector) gets broken, but if first_rust_function(ints : &mut [c_int;3]) would still use ints after calling called_from_rust() also additional rules would be violated.
However, I haven't found any official documentation stating that Stacked Borrows are the aliasing model used by the Rust compiler, and that everything considered undefined under Stacked Borrows is actually undefined in Rust. Naively this looks similar enough to coercion of &mut to &, so it might actually be sane, but given that called_from_rust() does not take the reference as parameter, I don't think this reasoning applies.
This brings me to the actual questions:
Is the above code invoking undefined behaviour (and why/why not?)
If it is undefined: Would the behaviour be well-defined if called_from_rust() would have the pointer as parameter and pass it forward: void called_from_rust(const int * i) { another_rust_function(i); }?
What if both Rust functions were using &mut [c_int;3]?
Is the above code invoking undefined behaviour?
Yes, you've broken Rust's pointer aliasing rules. Relying on the Stacked Borrows rules is a bit dubious since, as you've hinted, I don't think it has been officially adopted as Rust's memory access model (even if it was just a formalization on the current semantics). However, something that is a practical and concrete ruling is LLVM's noalias attribute, which the Rust compiler uses on &mut arguments.
noalias
This indicates that memory locations accessed via pointer values based on the argument or return value are not also accessed, during the execution of the function, via pointer values not based on the argument or return value. ...
So since you access ints[1] in another_rust_function from a pointer that is not based based on the ints in first_rust_function during the execution of that function, it is a violation. In light of this undefined behavior, I believe the compiler is well within its rights to make the code print "Second value: 4".
Would the behaviour be well-defined if called_from_rust() would have the pointer as parameter and pass it forward: void called_from_rust(const int * i) { another_rust_function(i); }?
Yes, that will make it well defined. You can see that because the Rust borrow checker can then see that the value can be used in called_from_rust() and will prevent improper use of ints around that call.
What if both Rust functions were using &mut [c_int;3]?
If you used the fix above, where the second borrow is based on the first, then there's no problem. If you didn't though, then it is way worse.
I have this code:
struct MyStruct
text::String
function MyStruct(_text::String)
text = _text
# do other things
end
end
When I wrote this, I realized that Julia is not recognizing text as MyStruct's field. How can I do something like this in Julia like in Python?
self.text = _text
Don't try to imitate Python. Julia is not object oriented.
You could emulate a Python-style constructor by
mutable struct MyStruct
text::String
function MyStruct(text::String)
self = new()
self.text = some_stuff(text)
return self
end
end
but for this to work the struct needs to be mutable. Then you can set up an uninitialize instance with new() and overwrite the fields.
Note that this is more equivalent to a combination of both __init__ and __new__. In Python, the new part is (99% of the time) already done for you, and you just mutate the already created empty object in __init__. In Julia, you have to do both on your own; this especially also requires returning the new value at the end of the constructor!
Having said all this, it's not often useful to write that way. More idiomatic would be just
struct MyStruct
text::String
MyStruct(text::String) = new(some_stuff(text))
end
unless you absolutely need the struct to be mutable (which has consequences with respect to memory layout and possible optimizations).
And also read up on the difference between inner and outer constructors. If you want the above to be the only valid way to construct MyStruct, this is fine. If you want "convenience constructors", e.g. with default arguments or conversions from other types, prefer outer constructors (where you don't have new but recursively call constructors until an inner constructor is reached).
Taking a quick glance at the constructors documentation and trying out the playground, I was able to come up with this:
struct MyStruct
text::String
function MyStruct(_text::String)
s = new(_text * "def")
# do other things
s
end
end
s = MyStruct("abc")
println(s.text) # abcdef
Here is a minimal working example of the problem:
Below example compiles fine (using Cadence Incisive/Xcelium) if I comment out the import "DPI-C" statement and the call to print_object(s);.
So that proves that the struct with dynamic array is a legal SystemVerilog syntax.
But if I try to pass the same struct via DPI-C, I get the error:
xmvlog: *E,UNUSAG (tb.sv,10|62): unsupported element in unpacked struct datatype in formal argument.
program top;
typedef struct {
int scalar_int;
int dyn_arr_int[];
} my_struct_s;
import "DPI-C" function void print_object(input my_struct_s s);
// Cannot pass a dynamic-array containing struct via DPI-C.
// import "DPI-C" function void print_object(input my_struct_s s);
// |
// xmvlog: *E,UNUSAG (tb.sv,10|62): unsupported element in unpacked struct datatype in formal argument.
initial begin
my_struct_s s;
s.scalar_int = 100;
s.dyn_arr_int = '{25, 35, 45};
$display("Value of the struct printed from SV: %p", s);
print_object(s);
$finish;
end
endprogram : top
So the question is:
Is there a workaround to pass structs with dynamic arrays to the C side? Is there a way to pass structs with "pointers" to those dynamic arrays? Or any other way?
One workaround is to pass a struct with a static array of some "max" size and also pass a scalar integer storing the length of actual data that's a subset of that static array. But that's a very ugly workaround, requires overhead in SystemVerilog to calculate the lengths and to populate that static array, and is also probably inefficient compared to passing just the dynamic array.
EDIT: It was pointed out that my example isn't complete enough to be useful. My question is resolved, but if you're interested in looking at the complete code you can see it here.
Given the following code:
#[no_mangle]
pub extern "C" fn create_acceptor() -> Acceptor {
Acceptor {}
}
pub struct Acceptor {}
If I call this from C what does it actually get? Is it a pointer? An id of some sort?
I'm currently using code like the following on the C side:
extern int create_acceptor();
int main(int argc, char **argv) {
int acceptor = create_acceptor();
return 0;
}
It seems to work fine. I'm using it like an opaque type that I pass back to Rust like acceptor_doSomething(acceptor).
But I'm confused because it doesn't seem to matter what type I use for acceptor on the C side. void* and int both behave the same. Also, the documentation seems to indicate that I should be using #[repr(C)], but it seems to work without it. Why is that?
When printing the value received on the C side, it's showing very small integers, so I'm guessing it's an id on the Rust side?
TL;DR: The example isn't a useful one and doesn't show anything.
what does it actually get? Is it a pointer? An id of some sort?
It's the struct, exactly as defined on the Rust side. It's a pointer if you return a pointer (this example does not) or an id if you return some id (this example does not).
Rust's FFI imposes no overhead or abstraction — what you return is what is returned.
the documentation seems to indicate that I should be using #[repr(C)]
Yes, you should. Without it, your code is most likely undefined behavior. The layout of a Rust struct is not yet guaranteed. Without specifying the representation as C, there's no way for the C code to know which fields are where.
but it seems to work without it
That's because your example struct has no fields and thus no size (which AFAIK isn't even valid in C without non-standard extensions). Rust basically never even needs to look at the data (what data?) so it doesn't matter what you pass back.
When printing the value received on the C side, it's showing very small integers
This is probably just junk on the stack laying around. Literally uninitialized data.
Using a struct with fields without #[repr(C)]
This is bad. Don't do it. String is a struct with non-trivial fields and it is not marked #[repr(C)].
Cargo.toml
[package]
name = "shear"
version = "0.1.0"
authors = ["An Devloper"]
[lib]
crate-type = ["cdylib"]
[dependencies]
src/lib.rs
// Don't do this, `String` isn't #[repr(C)]!
#[no_mangle]
pub extern "C" fn create_example() -> String {
String::from("hello")
}
// Don't do this, `String` isn't #[repr(C)]!
#[no_mangle]
pub extern "C" fn use_example(e: String) {
println!("{}", e);
}
main.c
extern int create_example();
extern void use_example(int example);
int main(int argc, char **argv) {
int example = create_example();
use_example(example);
}
Execution
$ cargo build
Compiling shear v0.1.0 (file:///home/ubuntu/shear)
Finished dev [unoptimized + debuginfo] target(s) in 1.02s
$ gcc -o example main.c -L target/debug/ -lshear
$ LD_LIBRARY_PATH=target/debug/ ./example
Segmentation fault
Please read the The Rust FFI Omnibus for details about how to properly transfer values across the FFI boundary. Disclaimer: I am the primary author.
I think I have a similar issue:
I define a struct containing memoryviews:
ctypedef double[:] vector
ctypedef double[:,::1] matrix
ctypedef struct my_struct:
matrix A
vector b
Now I initialize such a struct with:
cdef my_struct *instance = <my_struct*>malloc(sizeof(my_struct))
instance.A = 2*np.eye(3, dtype='double')
instance.b = np.zeros((3,), dtype='double')
print(matmul(instance.A, instance.B))
Ignore matmul for now (it is a well tested cython function to compute matrix multiplication). The error I get is a SIGSEGV (11) at instance.A = ... assignment line. It seems that it should work (qua types). In fact, if A was not inside a struct this assignment would work.