Let's assume we have a simple module called _sample built with pybind11:
/* py_bindings.cpp */
#include <pybind11/pybind11.h>
namespace py = pybind11;
PYBIND11_MODULE(_sample, m) {
m.def("add", [](int a, int b) { return a + b; });
m.def("add", [](const std::string& lhs, const std::string& rhs) { return lhs + rhs; });
}
This produces a dynamic module file _sample.pyd (Windows) or _sample.so (Linux), which we can then import in the actual module sample:
## sample\__init__.py ##
from ._sample import *
So that we can write:
## script.py ##
import sample as s
print(s.add(4, 2)) # 6
print(s.add('AB', 'C')) # ABC
The above code works fine, but the IDE does not know which functions are included in _sample until the code is actually run. And as a result, there are no function suggestions at all (and no function signature suggestions either).
As I would like to help the users of my library, my question is: how do I include function suggestions (or "function hints") in my module?
I've tried including the below code in sample\__init__.py as I thought the ... might work as a "hint". But unfortunately, this overrides the original add function from _sample.
def add(arg0: int, arg1: int) -> int:
...
Are there ways to hint the function signatures to a Python IDE?
Of course, I want to extend this to classes, class functions & module attributes too. I just picked functions as a starting point.
I think what you're looking for is a stub or interface (pyi) file. The IDE can understand the signature of functions and classes from this file.
If you're using pybind11, check out pybind11-stubgen for automatic generation of a stub file.
I am trying to use Collections-C in Cython.
I noticed that some structures are defined in the .c file, and an alias for them is in the .h file. When I try to define those structures in a .pxd file and use them in a .pyx file, gcc throws an error: storage size of ‘[...]’ isn’t known.
I was able to reproduce my issue to a minimum setup that replicates the external library and my application:
testdef.c
/* Note: I can't change this */
struct bogus_s {
int x;
int y;
};
testdef.h
/* Note: I can't change this */
typedef struct bogus_s Bogus;
cytestdef.pxd
# This is my code
cdef extern from 'testdef.h':
struct bogus_s:
int x
int y
ctypedef bogus_s Bogus
cytestdef.pyx
# This is my code
def fn():
cdef Bogus n
n.x = 12
n.y = 23
print(n.x)
If I run cythonize, I get
In function ‘__pyx_pf_7sandbox_9cytestdef_fn’:
cytestdef.c:1106:9: error: storage size of ‘__pyx_v_n’ isn’t known
Bogus __pyx_v_n;
^~~~~~~~~
I also get the same error if I use ctypedef Bogus: [...] notation as indicated in the Cython manual.
What am I doing wrong?
Thanks.
Looking at the documentation for your Collections-C library these are opaque structures that you're supposed to use purely through pointers (don't need to know the size to have a pointer, while you do to allocate on the stack). Allocation of these structures is done in library functions.
To change your example to match this case:
// C file
int bogus_s_new(struct bogus_s** v) {
*v = malloc(sizeof(struct bogus_s));
return (v!=NULL);
}
void free_bogus_s(struct bogus_s* v) {
free(v);
}
Your H file would contain the declarations for those and your pxd file would contain wrappers for the declarations. Then in Cython:
def fn():
cdef Bogus* n
if not bogus_s_new(&n):
return
try:
# you CANNOT access x and y since the type is
# designed to be opaque. Instead you should use
# the acessor functions defined in the header
# n.x = 12
# n.y = 23
finally:
free_bogus_s(n)
I have a functioning Rust program using real doubles (f64) as the underlying type and wish to extend the system such that it can also handle complex values (num::complex::Complex64).
A (cut down example) function takes some configuration struct config, and depending on that input generates a potential value at an index idx:
fn potential(config: &Config, idx: &Index3) -> Result<f64, Error> {
let num = &config.grid.size;
match config.potential {
PotentialType::NoPotential => Ok(0.0),
PotentialType::Cube => {
if (idx.x > num.x / 4 && idx.x <= 3 * num.x / 4) &&
(idx.y > num.y / 4 && idx.y <= 3 * num.y / 4) &&
(idx.z > num.z / 4 && idx.z <= 3 * num.z / 4) {
Ok(-10.0)
} else {
Ok(0.0)
}
}
PotentialType::Coulomb => {
let r = config.grid.dn * (calculate_r2(idx, &config.grid)).sqrt();
if r < config.grid.dn {
Ok(-1. / config.grid.dn)
} else {
Ok(-1. / r)
}
}
}
}
I now wish to add a ComplexCoulomb match which returns a Complex64 value:
PotentialType::ComplexCoulomb => {
let r = config.grid.dn * (calculate_r2(idx, &config.grid)).sqrt();
if r < config.grid.dn {
Ok(Complex64::new(-1. / config.grid.dn, 1.))
} else {
Ok(Complex64::new(-1. / r, 1.))
}
}
This function is an early entry point in my program, which fills an ndarray::Array3; currently I'm operating on a number of variables with the type ndarray::Array3<f64> - so I need to generalise the whole program, not just this function.
How can I extend this program to use both types based on the input from config? This struct comes from parsing a configuration file on disk and will match a number of PotentialType::Complex* values.
I'm aware of two possible options, but am unsure if either fits my criteria.
Use something similar to Either and return Left for real and Right for complex; then use additional logic to treat the values separately in other functions.
Use generic types. This isn't something I've done too much of before and generalisation over many types seems like a fair chunk of complicated alteration of my current code base. Is there a way to reduce the complexity here?
If you have any other suggestions I'd love to hear them!
There might be a lot of code change, but using generic parameters is probably the most flexible approach, and it won't impact performance. Passing around an enum will be less performant, partly because the enum will be bigger (the size of the larger variant plus a tag to discriminate between them) and partly because the enum variant will have to be frequently checked.
One thing that can get cumbersome is the potentially long list of traits that constrain your type parameter. This can be done on the impl level, rather than on each function, to save repetition. There isn't currently a way to alias a set of traits, which would make this more ergonomic, but there is an RFC approved for that.
I made a very similar change in the Euclid library. It was more than a year ago, so much has changed since then, both in Rust and in that library, but a quick look over that commit should still give you an idea of the amount of changes necessary.
This is the current state of the same (renamed) implementation:
impl <T, Src, Dst> TypedTransform3D<T, Src, Dst>
where T: Copy + Clone +
Add<T, Output=T> +
Sub<T, Output=T> +
Mul<T, Output=T> +
Div<T, Output=T> +
Neg<Output=T> +
ApproxEq<T> +
PartialOrd +
Trig +
One + Zero {
// methods of TypedTransform3D defined here...
}
Some of those traits (Trig, One, Zero) are actually defined inside the crate, as they aren't in the standard library.
I was wondering, is there any programming language where you can have function calls like this:
function_name(parameter1)function_name_continued(parameter2);
or
function_name(param1)function_continued(param2)...function_continued(paramN);
For example you could have this function call:
int dist = distanceFrom(cityA)to(cityB);
if you have defined distanceFromto function like this:
int distanceFrom(city A)to(city B)
{
// find distance between city A and city B
// ...
return distance;
}
As far as I know, in C, Java and SML programming languages, this cannot be done.
Are you aware of any programming language that let's you define and call
functions in this way?
It looks an awful lot like Objective-C
- (int)distanceFrom:(City *)cityA to:(City *)cityB {
// woah!
}
Sounds a lot like Smalltalk's syntax, (which would explain Objective-C's syntax - see kubi's answer).
Example:
dist := metric distanceFrom: cityA to: cityB
where #distanceFrom:to: is a method on some object called metric.
So you have "function calls" (they're really message sends) like
'hello world' indexOf: $o startingAt: 6. "$o means 'the character literal o"
EDIT: I'd said "Really, #distanceFrom:to: should be called #distanceTo: on a City class, but anyway." Justice points out that this couples a City to a Metric, which is Bad. There are good reasons why you might want to vary the metric - aeroplanes might use a geodesic while cars might use a shortest path based on the road network.)
For the curious, Agda2 has a similar, very permissive syntax. The following is valid code:
data City : Set where
London : City
Paris : City
data Distance : Set where
_km : ℕ → Distance
from_to_ : City → City → Distance
from London to London = 0 km
from London to Paris = 342 km
from Paris to London = 342 km
from Paris to Paris = 0 km
If
from Paris to London
is evaluated, the result is
342 km
Looks a lot like a fluent interface or method chaining to me.
In Python, you can explicitly pass the name of the arguments you're calling the function with, which lets you pass them in a different order or skip optional arguments:
>>> l = [3,5,1,2,4]
>>> print l.sort.__doc__
L.sort(cmp=None, key=None, reverse=False) -- stable sort *IN PLACE*;
cmp(x, y) -> -1, 0, 1
>>> l.sort (reverse=True)
>>> l
[5, 4, 3, 2, 1]
This looks a lot like what the Objective C syntax is doing, tagging each argument to a function with its name.
C# 4.0's Named and Optional Arguments feature allows you to achieve something pretty similar:
public static int Distance(string from, string to, string via = "")
{
...
}
public static void Main()
{
int distance;
distance = Distance(from: "New York", to: "Tokyo");
distance = Distance(to: "Tokyo", from: "New York");
distance = Distance(from: "New York", via: "Athens", to: "Tokyo");
}
(see my very favourite personal effort - the final C++ approach at the end of this answer)
Language One
Objective-C but the calling syntax is [object message] so would look like:
int dist = [cities distanceFrom:cityA to:cityB];
if you have defined distanceFromto function like this, within a cities object:
- (int)distanceFrom:(City *)cityA to:(City *)cityB
{
// find distance between city A and city B
// ...
return distance;
}
Language Two
I also suspect you could achieve something very close to this in the IO Language but I'm only just looking at it. You may also want to read about it in comparison to other languages in Seven Languages in Seven Weeks which has a free excerpt about IO.
Language Three
There's an idiom ("chaining") in C++ where you return temporary objects or the current object that is used to replace keyword arguments, according to The Design and Evolution of C++ and looks like this:
int dist = distanceFrom(cityA).to(cityB);
if you have defined distanceFrom function like this, with a little helper object. Note that inline functions make this kind of thing compile to very efficient code.
class DistanceCalculator
{
public:
DistanceCalculator(City* from) : fromCity(from) {}
int to(City * toCity)
{
// find distance between fromCity and toCity
// ...
return distance;
}
private:
City* fromCity;
};
inline DistanceCalculator distanceFrom(City* from)
{
return DistanceCalculator(from);
}
Duhh, I was in a hurry earlier, realised I can refactor to just use a temporary object to give the same syntax:
class distanceFrom
{
public:
distanceFrom(City* from) : fromCity(from) {}
int to(City * toCity)
{
// find distance between fromCity and toCity
// ...
return distance;
}
private:
City* fromCity;
};
MY FAVOURITE
and here's an even more inspired C++ version that allows you to write
int dist = distanceFrom cityA to cityB;
or even
int dist = distanceFrom cityA to cityB to cityC;
based on a wonderfully C++ ish combination of #define and classes:
#include <vector>
#include <numeric>
class City;
#define distanceFrom DistanceCalculator() <<
#define to <<
class DistanceCalculator
{
public:
operator int()
{
// find distance between chain of cities
return std::accumulate(cities.begin(), cities.end(), 0);
}
DistanceCalculator& operator<<(City* aCity)
{
cities.push_back(aCity);
return *this;
}
private:
std::vector<City*> cities;
};
NOTE this may look like a useless exercise but in some contexts it can be very useful to give people a domain-specific language in C++ which they compile alongside libraries. We used a similar approach with Python for geo-modeling scientists at the CSIRO.
You can do this in C, albeit unsafely:
struct Arg_s
{
int from;
int to;
};
int distance_f(struct Arg_s args)
{
return args.to - args.from;
}
#define distance(...) distance_f( ((struct Arg_s){__VA_ARGS__}) )
#define from_ .from =
#define to_ .to =
uses compound literals and designated initializers.
printf("5 to 7 = %i\n",distance(from_ 5, to_ 7));
// 5 to 7 = 2
3 of the 4 confederated languages from RemObjects in their Elements Compiler have this capability in precisely the OP's requested syntax (to support Objective-C runtime, but made available to all operating systems).
in Hydrogene (an extended C#)
https://docs.elementscompiler.com/Hydrogene/LanguageExtensions/MultiPartMethodNames
in Iodine (an extended Java)
https://docs.elementscompiler.com/Iodine/LanguageExtensions/MultiPartMethodNames
in Oxygene (an extended ObjectPascal), scroll down to Multi-Part Method Names section
https://docs.elementscompiler.com/Oxygene/Members/Methods
This looks similar to function overloading (C++/C#)/default parameters (VB).
Default Parameters allow the person defining the function to set defaults for the latter parameters:
e.g. c# overloading:
int CalculateDistance(city A, city B, city via1, city via2)
{....}
int CalculateDistance(city A, city B)
{
return CalculateDistance(city A, city B, null, null)
}
You can use a member function for this.
cityA.distance_to(cityB);
That's valid code in C++, C(with a little tweaking), C#, Java. Using method chains, you can do:
cityA.something(cityB).something(cityC).something(cityD).something(cityE);
In SML you could simply make "to" some value (unit, for example), and "distanceFrom" a curried function that takes three parameters. For example:
val to = ()
fun distanceFrom x _ y = (* implementation function body *)
val foo = distanceFrom cityA to cityB
You could also take advantage of the fact that SML doesn't enforce naming conventions on datataype constructors (much to many peoples' annoyance), so if you want to make sure that the type system enforces your custom syntax:
datatype comp = to
fun distanceFrom x to y = (* implementation *)
val foo = distanceFrom cityA to cityB (* works *)
val foo' = distanceFrom cityA cityB (* whoops, forgot 'to' - type error! *)
You could do this in Scheme or LISP using macros.
The form will be something like:
(DISTANCE-FROM city-a TO city-b)
The symbols in uppercase denotes syntax.
You could even do something like 'named parameters':
(DISTANCE TO city-a FROM city-b)
(DISTANCE FROM city-a TO city-b)
Tcl allows you to do something like this:
proc distance {from cityA to cityB} {...}
set distance [distance from "Chicago IL" to "Tulsa OK"]
I'm not sure if that's quite what you are thinking of though.
You can do it in Java, Use Builder pattern that appears in the book Effective Java by Joshua Bosch (this is second time I put this link in SO, I still didn't use that patern, but looks great)
Well, in Felix you can implement this in two steps: first, you write an ordinary function. Then, you can extend the grammar and map some of the new non-terminals to the function.
This is a bit heavyweight compared to what you might want (welcome to help make it easier!!) I think this does what you want and a whole lot more!
I will give a real example because the whole of the Felix language is actually defined by this technique (below x is the non-terminal for expressions, the p in x[p] is a precedence code):
// alternate conditional
x[sdollar_apply_pri] := x[stuple_pri] "unless" x[let_pri]
"then" x[sdollar_apply_pri] =>#
"`(ast_cond ,_sr ((ast_apply ,_sr (lnot ,_3)) ,_1 ,_5))";
Here's a bit more:
// indexes and slices
x[sfactor_pri] := x[sfactor_pri] "." "[" sexpr "]" =>#
"`(ast_apply ,_sr (,(noi 'subscript) (,_1 ,_4)))";
x[sfactor_pri] := x[sfactor_pri] "." "[" sexpr "to" sexpr "]" =>#
"`(ast_apply ,_sr (,(noi 'substring) (,_1 ,_4 ,_6)))";
x[sfactor_pri] := x[sfactor_pri] "." "[" sexpr "to" "]" =>#
"`(ast_apply ,_sr (,(noi 'copyfrom) (,_1 ,_4)))";
x[sfactor_pri] := x[sfactor_pri] "." "[" "to" sexpr "]" =>#
"`(ast_apply ,_sr (,(noi 'copyto) (,_1 ,_5)))";
The Felix grammar is ordinary user code. In the examples the grammar actions are written in Scheme. The grammar is GLR. It allows "context sensitive keywords", that is, identifiers that are keywords in certain contexts only, which makes it easy to invent new constructs without worrying about breaking existing code.
Perhaps you would like to examine Felix Grammar Online.