Let's assume we have a simple module called _sample built with pybind11:
/* py_bindings.cpp */
#include <pybind11/pybind11.h>
namespace py = pybind11;
PYBIND11_MODULE(_sample, m) {
m.def("add", [](int a, int b) { return a + b; });
m.def("add", [](const std::string& lhs, const std::string& rhs) { return lhs + rhs; });
}
This produces a dynamic module file _sample.pyd (Windows) or _sample.so (Linux), which we can then import in the actual module sample:
## sample\__init__.py ##
from ._sample import *
So that we can write:
## script.py ##
import sample as s
print(s.add(4, 2)) # 6
print(s.add('AB', 'C')) # ABC
The above code works fine, but the IDE does not know which functions are included in _sample until the code is actually run. And as a result, there are no function suggestions at all (and no function signature suggestions either).
As I would like to help the users of my library, my question is: how do I include function suggestions (or "function hints") in my module?
I've tried including the below code in sample\__init__.py as I thought the ... might work as a "hint". But unfortunately, this overrides the original add function from _sample.
def add(arg0: int, arg1: int) -> int:
...
Are there ways to hint the function signatures to a Python IDE?
Of course, I want to extend this to classes, class functions & module attributes too. I just picked functions as a starting point.
I think what you're looking for is a stub or interface (pyi) file. The IDE can understand the signature of functions and classes from this file.
If you're using pybind11, check out pybind11-stubgen for automatic generation of a stub file.
Related
I would like to disable the default (zero argument) constructor for a C++ class exposed to R using RCPP_MODULE so that calls to new(class) without any further arguments in R give an error. Here are the options I've tried:
1) Specifying a default constructor and throwing an error in the function body: this does what I need it to, but means specifying a default constructor that sets dummy values for all const member variables (which is tedious for my real use case)
2) Specifying that the default constructor is private (without a definition): this means the code won't compile if .constructor() is used in the Rcpp module, but has no effect if .constructor() is not used in the Rcpp module
3) Explicitly using delete on the default constructor: requires C++11 and seems to have the same (lack of) effect as (2)
I'm sure that I am missing something obvious, but can't for the life of me work out what it is. Does anyone have any ideas?
Thanks in advance,
Matt
Minimal example code (run in R):
inc <- '
using namespace Rcpp;
class Foo
{
public:
Foo()
{
stop("Disallowed default constructor");
}
Foo(int arg)
{
Rprintf("Foo OK\\n");
}
};
class Bar
{
private:
Bar();
// Also has no effect:
// Bar() = delete;
public:
Bar(int arg)
{
Rprintf("Bar OK\\n");
}
};
RCPP_MODULE(mod) {
class_<Foo>("Foo")
.constructor("Disallowed default constructor")
.constructor<int>("Intended 1-argument constructor")
;
class_<Bar>("Bar")
// Wont compile unless this line is commented out:
// .constructor("Private default constructor")
.constructor<int>("Intended 1-argument constructor")
;
}
'
library('Rcpp')
library('inline')
fx <- cxxfunction(signature(), plugin="Rcpp", include=inc)
mod <- Module("mod", getDynLib(fx))
# OK as expected:
new(mod$Foo, 1)
# Fails as expected:
new(mod$Foo)
# OK as expected:
new(mod$Bar, 1)
# Unexpectedly succeeds:
new(mod$Bar)
How can I get new(mod$Bar) to fail without resorting to the solution used for Foo?
EDIT
I have discovered that my question is actually a symptom of something else:
#include <Rcpp.h>
class Foo {
public:
int m_Var;
Foo() {Rcpp::stop("Disallowed default constructor"); m_Var=0;}
Foo(int arg) {Rprintf("1-argument constructor\n"); m_Var=1;}
int GetVar() {return m_Var;}
};
RCPP_MODULE(mod) {
Rcpp::class_<Foo>("Foo")
.constructor<int>("Intended 1-argument constructor")
.property("m_Var", &Foo::GetVar, "Get value assigned to m_Var")
;
}
/*** R
# OK as expected:
f1 <- new(Foo, 1)
# Value set in the 1-parameter constructor as expected:
f1$m_Var
# Unexpectedly succeeds without the error message:
tryCatch(f0 <- new(Foo), error = print)
# This is the type of error I was expecting to see:
tryCatch(f2 <- new(Foo, 1, 2), error = print)
# Note that f0 is not viable (and sometimes brings down my R session):
tryCatch(f0$m_Var, error = print)
*/
[With acknowledgements to #RalfStubner for the improved code]
So in fact it seems that new(Foo) is not actually calling any C++ constructor at all for Foo, so my question was somewhat off-base ... sorry.
I guess there is no way to prevent this happening at the C++ level, so maybe it makes most sense to use a wrapper function around the call to new(Foo) at the R level, or continue to specify a default constructor that throws an error. Both of these solutions will work fine - I was just curious as to exactly why my expectation regarding the absent default constructor was wrong :)
As a follow-up question: does anybody know exactly what is happening in f0 <- new(Foo) above? My limited understanding suggests that although f0 is created in R, the associated pointer leads to something that has not been (correctly/fully) allocated in C++?
After a bit of experimentation I have found a simple solution to my problem that is obvious in retrospect...! All I needed to do was use .factory for the default constructor along with a function that takes no arguments and just throws an error. The default constructor for the Class is never actually referenced so doesn't need to be defined, but it obtains the desired behaviour in R (i.e. an error if the user mistakenly calls new with no additional arguments).
Here is an example showing the solution (Foo_A) and a clearer illustration of the problem (Foo_B):
#include <Rcpp.h>
class Foo {
private:
Foo(); // Or for C++11: Foo() = delete;
const int m_Var;
public:
Foo(int arg) : m_Var(arg) {Rcpp::Rcout << "Constructor with value " << m_Var << "\n";}
void ptrAdd() const {Rcpp::Rcout << "Pointer: " << (void*) this << "\n";}
};
Foo* dummy_factory() {Rcpp::stop("Default constructor is disabled for this class"); return 0;}
RCPP_MODULE(mod) {
Rcpp::class_<Foo>("Foo_A")
.factory(dummy_factory) // Disable the default constructor
.constructor<int>("Intended 1-argument constructor")
.method("ptrAdd", &Foo::ptrAdd, "Show the pointer address")
;
Rcpp::class_<Foo>("Foo_B")
.constructor<int>("Intended 1-argument constructor")
.method("ptrAdd", &Foo::ptrAdd, "Show the pointer address")
;
}
/*** R
# OK as expected:
fa1 <- new(Foo_A, 1)
# Error as expected:
tryCatch(fa0 <- new(Foo_A), error = print)
# OK as expected:
fb1 <- new(Foo_B, 1)
# No error:
tryCatch(fb0 <- new(Foo_B), error = print)
# But this terminates R with the following (quite helpful!) message:
# terminating with uncaught exception of type Rcpp::not_initialized: C++ object not initialized. (Missing default constructor?)
tryCatch(fb0$ptrAdd(), error = print)
*/
As was suggested to me in a comment I have started a discussion at https://github.com/RcppCore/Rcpp/issues/970 relating to this.
I am trying to use Collections-C in Cython.
I noticed that some structures are defined in the .c file, and an alias for them is in the .h file. When I try to define those structures in a .pxd file and use them in a .pyx file, gcc throws an error: storage size of ‘[...]’ isn’t known.
I was able to reproduce my issue to a minimum setup that replicates the external library and my application:
testdef.c
/* Note: I can't change this */
struct bogus_s {
int x;
int y;
};
testdef.h
/* Note: I can't change this */
typedef struct bogus_s Bogus;
cytestdef.pxd
# This is my code
cdef extern from 'testdef.h':
struct bogus_s:
int x
int y
ctypedef bogus_s Bogus
cytestdef.pyx
# This is my code
def fn():
cdef Bogus n
n.x = 12
n.y = 23
print(n.x)
If I run cythonize, I get
In function ‘__pyx_pf_7sandbox_9cytestdef_fn’:
cytestdef.c:1106:9: error: storage size of ‘__pyx_v_n’ isn’t known
Bogus __pyx_v_n;
^~~~~~~~~
I also get the same error if I use ctypedef Bogus: [...] notation as indicated in the Cython manual.
What am I doing wrong?
Thanks.
Looking at the documentation for your Collections-C library these are opaque structures that you're supposed to use purely through pointers (don't need to know the size to have a pointer, while you do to allocate on the stack). Allocation of these structures is done in library functions.
To change your example to match this case:
// C file
int bogus_s_new(struct bogus_s** v) {
*v = malloc(sizeof(struct bogus_s));
return (v!=NULL);
}
void free_bogus_s(struct bogus_s* v) {
free(v);
}
Your H file would contain the declarations for those and your pxd file would contain wrappers for the declarations. Then in Cython:
def fn():
cdef Bogus* n
if not bogus_s_new(&n):
return
try:
# you CANNOT access x and y since the type is
# designed to be opaque. Instead you should use
# the acessor functions defined in the header
# n.x = 12
# n.y = 23
finally:
free_bogus_s(n)
I am writing an R package in which one of the functions takes Rcpp::XPtr as an input (as a SEXP). However, the creation of XPtr from Rcpp::Function is something I want to do inside the package (i.e., the user should be able to input Function).
e.g, my package takes input generated as follows, which requires the user to write an additional function (here putFunPtrInXPtr()) and run the function in R to generate the XPtr (here my_ptr).
#include <Rcpp.h>
using namespace Rcpp;
typedef NumericVector (*funcPtr) (NumericVector y);
// [[Rcpp::export]]
NumericVector timesTwo(NumericVector x) {
return x * 2;
}
// [[Rcpp::export]]
XPtr<funcPtr> putFunPtrInXPtr() {
XPtr<funcPtr> testptr(new funcPtr(×Two), false);
return testptr;
}
/*** R
my_xptr <- putFunPtrInXPtr()
*/
How can I write something in which the user provides Function user_fun and I create the XPtr?
I tried
XPtr<funcPtr> package_fun(Function user_fun_input){
XPtr<funcPtr> testptr(new funcPtr(&user_fun_input), false);
}
user_fun_input is the parameter name inside the package function, but I am getting the following error
cannot initialize a new value of type 'funcPtr' (aka 'Vector<14> (*) (Vector<14>') with an rvalue of type 'Rcpp::Function *' (aka 'Function_Impl<PreserveStorage> *')
Also, there is an R step involved in creating the pointer, I am not sure how to implement that in the package (my .cpp file).
I think the creation of XPtr from Function could be confusing to the user, so better to just take Function as input and create the pointer to it, inside the package. I do use the XPtr in my package to gain speed.
Suggestions are most appreciated!
I was wondering if there an established convention to specifying fixed point binary numbers in decimal format (with the use of a macro). I am not sure if this possible in C/C++, but perhaps this is implemented in some language(s) and there is a notational standard like 0x000000,1.2f,1.2d,1l,etc
Take this example for instance:
I am using Q15.16 for instance, but would like to have the convenience of specifying numbers in decimal format, perhaps something like this:
var num:Int32=1.2fp;
Presumably, the easiest way with regards to Haxe macros, numbers can be initialized with a function:
#:macro
fp_from_float(1.2);
But it would be nice to have a shorthand notation.
Have you seen Luca's Fixed Point example with Haxe 3 and Abstracts?
It's here:
https://groups.google.com/forum/?fromgroups=#!topic/haxelang/JsiWvl-c0v4
Summing it up, with the new Haxe 3 abstract types, you can define a type that will be compiled as an Int:
abstract Fixed16(Int)
{
inline function new(x:Int) this = x;
}
You can also define "conversion functions", which will allow you to automatically convert a float into Fixed16:
#:from public static inline function fromf(x:Float) {
#if debug
if (x >= 32768.0 || x < -32768.0) throw "Conversion to Fixed16 will overflow";
#end
return new Fixed16(Std.int(x*65536.0));
}
The secret here is the #:from metadata. With this code, you will already be able to declare fixed types like this:
var x:Fixed16 = 1.2;
Luca's already defined some operators, to make working with them easier, like:
#:op(A+B) public inline static function add(f:Fixed16, g:Fixed16) {
#if debug
var fr:Float = f.raw();
var gr:Float = g.raw();
if (fr+gr >= 2147483648.0 || fr+gr < -2147483648.0) throw "Addition of Fixed16 values will overflow";
#end
return new Fixed16(f.raw()+g.raw());
}
Again, the secret here is in #:op(A+B) metadata, which will annotate that this function may be called when handling addition. The complete GIST code is available at https://gist.github.com/deltaluca/5413225 , and you can learn more about abstracts at http://haxe.org/manual/abstracts
I came across a problem recently.
I have three files, A.h, B.cpp, C.cpp:
A.h
#ifndef __A_H__
#define __A_H__
int M()
{
return 1;
}
#endif // __A_H__
B.cpp
#include "A.h"
C.cpp
#include "A.h"
As I comile the three files by MSVC, there is a error:
C.obj : error LNK2005: "int __cdecl M(void)" (?M##YAHXZ) already defined in B.obj
It is easy understanding, as we know, B.obj has a symbol named "M", also C.obj has a "M".
Here the error comes.
However, if I change M method to a class which contain a method M like this below:
A.h
#ifndef __A_H__
#define __A_H__
class CA
{
public:
int M()
{
return 1;
}
};
#endif // __A_H__
there is no more errors!! Could somebody tell me what is happening?
If B.cpp and C.cpp include A.h, then both are compiled with your definition of M, so both object files will contain code for M. When the linker gathers all the functions, he sees that M is defined in two object files and does not know which one to use. Thus the linker raises an LNK2005.
If you put your function M into a class declaration, then the compiler marks/handles M as an inline function. This information is written into the object file. The linker sees that both object files contain a definition for an inline version of CA::M, so he assumes that both are equal and picks up randomly one of the two definitions.
If you had written
class CA {
public:
int M();
};
int CA::M()
{
return 1;
}
this would have caused the same problems (LNK2005) as your initial version, because then CA::M would not have been inline any more.
So as you might guess by now, there are two solutions for you. If you want M to be inlined, then change your code to
__inline int M()
{
return 1;
}
If you don't care about inlining, then please do it the standard way and put the function declaration into the header file:
extern int M();
And put the function definition into a cpp file (for A.h this would ideally be A.cpp):
int M()
{
return 1;
}
Please note that the extern is not really necessary in the header file.
Another user suggested that you write
static int M()
{
return 1;
}
I'd not recommend this. This would mean that the compiler puts M into both of your object files and marks M as being a function that is only visible in each object file itself. If the linker sees that a function in B.cpp calls M, it finds M in B.obj and in C.obj. Both have M marked as static, so the linker ignores M in C.obj and picks the M from B.obj. Vice versa if a function in C.cpp calls M, the linker picks the M from C.obj. You will end up with multiple definitions of M, all with the same implementation. This is a waste of space.
See http://faculty.cs.niu.edu/~mcmahon/CS241/c241man/node90.html how to do ifdef guards. You have to start with ifndef before the define.
Edit: Ah no, while your guard is wrong that's not the issue. Put static in front of your function to make it work. Classes are different because they define types.
I don't know what's under the hood, but if you don't need a class I guess that the compiler will automatically add the "extern" key to your functions, so you'll get the error including the header 2 times.
You can add the static keyword to M() method so you'll have only one copy of that function in memory and no errors at compile time.
By the way: I see you have a #endif, but not a #ifdef or #ifndef, is it a copy/paste error?