python 3.x C extension module and submodule - python-3.x

Question

How do I make a C extension for python 3.x when a module has sub-modules? For example, I have a file called pet.c:
#include <Python.h>
PyObject* CatMeow(PyObject* self) {
return PyUnicode_FromString( ">*<" );
}
static PyMethodDef CatFunctions[] = {
{(char*) "meow", (PyCFunction) CatMeow, METH_NOARGS, NULL},
{NULL, NULL, 0, NULL}
};
static PyModuleDef CatDef = {
PyModuleDef_HEAD_INIT, "cat", "cat ext", -1, CatFunctions,
NULL, NULL, NULL, NULL
};
PyMODINIT_FUNC PyInit_cat(void) {
return PyModule_Create(&CatDef);
}
static PyModuleDef PetDef = {
PyModuleDef_HEAD_INIT, "pet", "pet ext", -1, NULL,
NULL, NULL, NULL, NULL
};
PyMODINIT_FUNC PyInit_pet(void) {
PyObject* p = PyModule_Create(&PetDef);
PyObject* c = PyInit_cat();
Py_INCREF(c);
PyModule_AddObject( p, "cat", c );
return p;
}
When I build it with the following setup.py:
from distutils.core import setup, Extension
setup(
name='pet',
version='0.0',
ext_modules=[Extension('pet', ['pet.c'])]
)
I can see
>>> import pet
>>> pet.cat.meow()
'>*<'
or
>>> from pet import cat
>>> cat.meow()
'>*<'
which is as intended, but when I try
>>> from pet.cat import meow
I have a ModuleNotFoundError saying ... No module named 'pet.cat'; 'pet' is not a package, and if I try
>>> from pet import cat
>>> from cat import meow
I have a ModuleNotFoundError saying ... No module named 'cat'. But if I check the type of cat
>>> type(cat)
<class 'module'>
which says it is a module.
How do I make this work? Adding a module object to another module used to work well in python 2.7. Is it not supposed to work in python3 due to absolute import style? Or do I have to work with multi-phase initialisation as described in PEP 489?

Answer 1

Regarding the first error which complains about pet not being a package. If pet is there only to provide a parent for cat, there is an easy way to turn it into a package: remove all the pet related code from pet.c and use ext_package in setup.py
from distutils.core import setup, Extension
setup(
name = 'pet',
version = '0.0',
ext_package = 'pet',
ext_modules = [Extension('cat', ['pet.c'])]
)
Running the above will create a directory called 'pet' and a shared library of which name starts with 'cat'. This effectively creates a namespace package -- there are two types of packages, regular and namespace, and the latter is the one without requiring __init__.py (see PEP 420 for details). From outside of pet, you can do
>>> from pet import cat
>>> from pet.cat import meow
>>> meow()
'>*<'
>>> cat.meow()
'>*<'
The reason you can't do from cat import meow is because the fully qualified name of the module is 'pet.cat' not 'cat', which you can confirm it from cat.__name__. If you are running the interpreter inside of the directory pet, then you can do from cat import meow.