python: ImportError: dynamic module does not define module export function - python-3.x

im trying to install my function written in c (with python3 setup.py install) but python raise ImportError: dynamic module does not define module export function (PyInit_costFunction)
error!
costFunction.c:
static PyObject *costFunction(PyObject *self, PyObject *args)
{
return Py_BuildValue("d", 0); // or anything!
}
static PyMethodDef costFunction_methods[] = {
{"costFunction", (PyCFunction)costFunction, METH_VARARGS, "cost function"},
{NULL, NULL, 0, NULL}
};
static struct PyModuleDef costFunctionmodule = {
PyModuleDef_HEAD_INIT,"costFunction", NULL, -1, costFunction_methods
};
PyMODINIT_FUNC PyInit_costFunction(void)
{
return PyModule_Create(&costFunctionmodule);
}
setup.py:
from distutils.core import setup, Extension
setup(name='costFunction', version='1.0', \
ext_modules=[Extension('costFunction', ['costFunction.c'],include_dirs=['include'])])
external library: tinyexpr
i'm using linux mint 18 with python 3.5.2
EDIT:
python3-dev version is 3.5.1-3

finally i used an dirty trick!
compiled c code(without python.h and any python datatype in C) with:
gcc -fPIC -Wall -O3 costFunction.c -o costFunction.so -shared -fopenmp
and used python ctypes module to load and use it:
dll = ctypes.CDLL("./costFunction.so")
costFunction = dll.cost_function
costFunction.restype = ctypes.c_double
costFunction.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_int]

Related

Nix GLFW Haskell bindings missing library?

I'm trying to compile a Haskell program (https://hackage.haskell.org/package/gloss-export)
However I'm running into the following error:
<command line>: can't load .so/.DLL for: /nix/store/f2730k4icw6biaaw3k81x853sch2ig8k-bindings-GLFW-3.2.1.1/lib/ghc-8.6.5/x86_64-linux-ghc-8.6.5/libHSbindings-GLFW-3.2.1.1-HsSdzRmGNYCJBRDKgdT43y-ghc8.6.5.so (/nix/store/wx1vk75bpdr65g6xwxbj4rw0pk04v5j3-glibc-2.27/lib/libm.so.6: version `GLIBC_2.29' not found (required by /nix/store/f2730k4icw6biaaw3k81x853sch2ig8k-bindings-GLFW-3.2.1.1/lib/ghc-8.6.5/x86_64-linux-ghc-8.6.5/libHSbindings-GLFW-3.2.1.1-HsSdzRmGNYCJBRDKgdT43y-ghc8.6.5.so))
cabal: repl failed for exe:xmonad from my-xmonad-0.2.0.0.
How could I resolve this dependency?
What I've tried:
Adding the bindings-GLFW haskell package but this did not make any
difference.
Adding the glibc package via the following:
#(myHaskellPackages.callCabal2nix "HaskellNixCabalStarter" (./.) {}).overrideAttrs (oldAttrs: {
#buildInputs = (oldAttrs.buildInputs or []) ++ [ pkgs.glibc ];
#})
This seems to give me another error of:
nix build
builder for '/nix/store/kp5m6kfwjfr4w59ypcnqqqn9812q6vpy-my-xmonad-0.2.0.0.drv'
failed with exit code 1; last 10 log lines:
/nix/store/gwwycf3w6cbj0gd2mpgblrdjc24f3cys-binutils-wrapper-2.31.1/bin/ld.gold
No pkg-config found Using runghc version 8.6.5 found on system at:
/nix/store/hg3na12737n7wws1kndxvs95ai88fgn8-ghc-8.6.5/bin/runghc
Using strip version 2.31 found on system at:
/nix/store/ajrrkivdfvp8dp4vdg5hp1h5hblmanc9-binutils-2.31.1/bin/strip
Using tar found on system at:
/nix/store/aawf0q16ql39w2gwv52qyjfzgbg5f22r-gnutar-1.32/bin/tar No
uhc found
*** abort because of serious configure-time warning from Cabal [1 built (1 failed), 0.0 MiB DL] error: build of
'/nix/store/kp5m6kfwjfr4w59ypcnqqqn9812q6vpy-my-xmonad-0.2.0.0.drv'
failed ```
My entire nix expression:
{
nixpkgs ? import <nixpkgs> {}
, compiler ? "ghc865"
, sources ? import ./nix/sources.nix
} :
let
niv = import sources.nixpkgs {
overlays = [
(_ : _ : { niv = import sources.niv {}; })
] ;
config = {};
};
pkgs = niv.pkgs;
myHaskellPackages = pkgs.haskell.packages.${compiler}.override {
overrides = self: super: rec {
gloss-export = pkgs.haskell.lib.dontCheck (import /home/chris/NewProjects/gloss-export/default.nix {});
};
};
in
(myHaskellPackages.callCabal2nix "HaskellNixCabalStarter" (./.) {}).overrideAttrs (oldAttrs: {
buildInputs = (oldAttrs.buildInputs or []) ++ [ pkgs.glibc ];
})

How to link .so library properly in C++?

I have project structure like this,
a.pb.h --- includes --> protobuf.h
b.grpc.pb.h --- includes --> a.pb.h & grpcpp.h
Also there are a.pb.cc and b.grpc.cc files.
A C++ wrapper with extern C which is wrapper.cc and wrapper.h which includes b.grpc.pb.h and grpcpp.h.
The function inside extern C is char* helloWorld(const char*, const char*, const char*);
Creating .o of a.pb.h and b.grpc.pb.h:
g++ -fpic -std=c++11 `pkg-config --cflags protobuf grpc` -c -o a.pb.o a.pb.cc
g++ -fpic -std=c++11 `pkg-config --cflags protobuf grpc` -c -o b.grpc.pb.o b.grpc.pb.cc
Steps to create libcombined.so:
The grpc and protobuf so are already provided under /usr/local/lib.
First created .so of a.pb.o and b.grpc.pb.o to compile wrapper file as:
g++ -shared -o libcombined.so *.o
Compiled wrapper as:
g++ -fpic wrapper.cc -l:./libcombined.so -c -o wrapper.o -std=c++11
.so of a.pb.o, b.grpc.pb.o and wrapper.o as libcombined.so:
g++ -shared -o libcombinedwrapper.so *.o
Compiled main.c as:
gcc main.c -l:./libcombinedwrapper.so -o main -ldl
I am calling helloWorld from my main.c file which is:
#include <stdio.h>
#include <dlfcn.h>
int main(){
char* (*fn)(const char*,const char*,const char*);
void *handle = dlopen("path_to/libcombined.so",RTLD_NOW);
if(handle==NULL){
fprintf(stderr, "Error: %s\n", dlerror());
}
fn = (char* (*)(const char*,const char*,const char*))dlsym(handle, "helloWorld");
if (!fn) {
/* no such symbol */
fprintf(stderr, "Error: %s\n", dlerror());
dlclose(handle);
return 0;
}
char* msg = fn("asd","asdas","asdasd");
printf("%s",msg);
return 0;
}
Error after executing: ./main
Error: path_to/libcombinedwrapper.so: undefined symbol: _ZN6google8protobuf2io20ZeroCopyOutputStream15WriteAliasedRawEPKvi
Error: ./main: undefined symbol: helloWorld
Segmentation fault (core dumped)
The first above error is from symbol from protobuf.h file.
Can someone please suggest what I am doing wrong while linking or is there something I am doing wrong in main.c file?
g++ -shared -o libcombined.so *.o
You need to also link in all dependencies of the objects (libgrpc here).
You can add -Wl,--no-allow-shlib-undefined to verify that libcombined.so is linking everything it needs.
P.S. To avoid core dump, you should exit or return once dlopen fails.
P.P.S. It is generally a very bad idea(TM) to link *.o. Use proper Makefile to avoid unnecessary compilations and explicitly list objects that you are intending to put into libcombined.so.

python 3.x C extension module and submodule

How do I make a C extension for python 3.x when a module has sub-modules? For example, I have a file called pet.c:
#include <Python.h>
PyObject* CatMeow(PyObject* self) {
return PyUnicode_FromString( ">*<" );
}
static PyMethodDef CatFunctions[] = {
{(char*) "meow", (PyCFunction) CatMeow, METH_NOARGS, NULL},
{NULL, NULL, 0, NULL}
};
static PyModuleDef CatDef = {
PyModuleDef_HEAD_INIT, "cat", "cat ext", -1, CatFunctions,
NULL, NULL, NULL, NULL
};
PyMODINIT_FUNC PyInit_cat(void) {
return PyModule_Create(&CatDef);
}
static PyModuleDef PetDef = {
PyModuleDef_HEAD_INIT, "pet", "pet ext", -1, NULL,
NULL, NULL, NULL, NULL
};
PyMODINIT_FUNC PyInit_pet(void) {
PyObject* p = PyModule_Create(&PetDef);
PyObject* c = PyInit_cat();
Py_INCREF(c);
PyModule_AddObject( p, "cat", c );
return p;
}
When I build it with the following setup.py:
from distutils.core import setup, Extension
setup(
name='pet',
version='0.0',
ext_modules=[Extension('pet', ['pet.c'])]
)
I can see
>>> import pet
>>> pet.cat.meow()
'>*<'
or
>>> from pet import cat
>>> cat.meow()
'>*<'
which is as intended, but when I try
>>> from pet.cat import meow
I have a ModuleNotFoundError saying ... No module named 'pet.cat'; 'pet' is not a package, and if I try
>>> from pet import cat
>>> from cat import meow
I have a ModuleNotFoundError saying ... No module named 'cat'. But if I check the type of cat
>>> type(cat)
<class 'module'>
which says it is a module.
How do I make this work? Adding a module object to another module used to work well in python 2.7. Is it not supposed to work in python3 due to absolute import style? Or do I have to work with multi-phase initialisation as described in PEP 489?
Regarding the first error which complains about pet not being a package. If pet is there only to provide a parent for cat, there is an easy way to turn it into a package: remove all the pet related code from pet.c and use ext_package in setup.py
from distutils.core import setup, Extension
setup(
name = 'pet',
version = '0.0',
ext_package = 'pet',
ext_modules = [Extension('cat', ['pet.c'])]
)
Running the above will create a directory called 'pet' and a shared library of which name starts with 'cat'. This effectively creates a namespace package -- there are two types of packages, regular and namespace, and the latter is the one without requiring __init__.py (see PEP 420 for details). From outside of pet, you can do
>>> from pet import cat
>>> from pet.cat import meow
>>> meow()
'>*<'
>>> cat.meow()
'>*<'
The reason you can't do from cat import meow is because the fully qualified name of the module is 'pet.cat' not 'cat', which you can confirm it from cat.__name__. If you are running the interpreter inside of the directory pet, then you can do from cat import meow.

Creating a sandbox to build a package that has a dependency for libxcb

I am trying to build https://github.com/SaschaWillems/Vulkan. I wrote a small default.nix file:
{ stdenv, libxcb, pkgconfig, cmake, vulkan-loader, assimp }:
stdenv.mkDerivation rec {
name = "VulkanExamples";
buildDepends = [ cmake libxcb.dev pkgconfig vulkan-loader assimp ];
}
which I call with
nix-shell -E 'with import <nixpkgs> {}; callPackage ./default.nix {}'
But when I call cmake . I get
-- Could NOT find PkgConfig (missing: PKG_CONFIG_EXECUTABLE)
-- Could NOT find xcb (missing: XCB_INCLUDE_DIR XCB_LIBRARY
What do I need to specify so that the cmake script can find the correct header files?
I finally got it working, it seems I had to use buildInputs
{ stdenv, libxcb, pkgconfig, cmake, vulkan-loader, assimp }:
with import <nixpkgs> {}; {
vulkanEnv = stdenv.mkDerivation {
name = "vulkan";
buildInputs = [ stdenv cmake libxcb pkgconfig vulkan-loader assimp ];
};
}

Building pjSip 2.5 - unable to undefined reference to 'pj_jni_jvm'

Trying to build pjSIP from sources for Android, using following guide - https://trac.pjsip.org/repos/wiki/Getting-Started/Android
Checkout sources of pjsip (2.5)
export ANDROID_NDK_ROOT=/home/dev/android/android-ndk-r9b
export TARGET_ABI=armeabi
export APP_PLATFORM=android-13
./configure-android --use-ndk-cflags --disable-video --disable-l16-codec --disable-gsm-codec --disable-g722-codec --disable-g7221-codec --disable-ssl --disable-ilbc-codec --disable-tls
configuration is completed successfully.
make dep && make clean && make
And I got following error - >
../src/pj/guid_android.c:43: error: undefined reference to 'pj_jni_jvm'
../src/pj/guid_android.c:117: error: undefined reference to 'pj_jni_jvm'
The command invoked by linked which leads to this error
/home/dev/android/android-ndk-r9b/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86/bin/arm-linux-androideabi-gcc -o ../bin/pjlib-test-arm-unknown-linux-androideabi \
output/pjlib-test-arm-unknown-linux-androideabi/main.o output/pjlib-test-arm-unknown-linux-androideabi/activesock.o output/pjlib-test-arm-unknown-linux-androideabi/atomic.o output/pjlib-test-arm-unknown-linux-androideabi/echo_clt.o output/pjlib-test-arm-unknown-linux-androideabi/errno.o output/pjlib-test-arm-unknown-linux-androideabi/exception.o output/pjlib-test-arm-unknown-linux-androideabi/fifobuf.o output/pjlib-test-arm-unknown-linux-androideabi/file.o output/pjlib-test-arm-unknown-linux-androideabi/hash_test.o output/pjlib-test-arm-unknown-linux-androideabi/ioq_perf.o output/pjlib-test-arm-unknown-linux-androideabi/ioq_udp.o output/pjlib-test-arm-unknown-linux-androideabi/ioq_unreg.o output/pjlib-test-arm-unknown-linux-androideabi/ioq_tcp.o output/pjlib-test-arm-unknown-linux-androideabi/list.o output/pjlib-test-arm-unknown-linux-androideabi/mutex.o output/pjlib-test-arm-unknown-linux-androideabi/os.o output/pjlib-test-arm-unknown-linux-androideabi/pool.o output/pjlib-test-arm-unknown-linux-androideabi/pool_perf.o output/pjlib-test-arm-unknown-linux-androideabi/rand.o output/pjlib-test-arm-unknown-linux-androideabi/rbtree.o output/pjlib-test-arm-unknown-linux-androideabi/select.o output/pjlib-test-arm-unknown-linux-androideabi/sleep.o output/pjlib-test-arm-unknown-linux-androideabi/sock.o output/pjlib-test-arm-unknown-linux-androideabi/sock_perf.o output/pjlib-test-arm-unknown-linux-androideabi/ssl_sock.o output/pjlib-test-arm-unknown-linux-androideabi/string.o output/pjlib-test-arm-unknown-linux-androideabi/test.o output/pjlib-test-arm-unknown-linux-androideabi/thread.o output/pjlib-test-arm-unknown-linux-androideabi/timer.o output/pjlib-test-arm-unknown-linux-androideabi/timestamp.o output/pjlib-test-arm-unknown-linux-androideabi/udp_echo_srv_sync.o output/pjlib-test-arm-unknown-linux-androideabi/udp_echo_srv_ioqueue.o output/pjlib-test-arm-unknown-linux-androideabi/util.o -lpj-arm-unknown-linux-androideabi -nostdlib -L/home/dev/android/android-ndk-r9b/platforms/android-13/arch-arm/usr/lib -L/home/dev/android/android-ndk-r9b/sources/cxx-stl/gnu-libstdc++/4.8/libs/armeabi/ -lm /home/dev/android/android-ndk-r9b/platforms/android-13/arch-arm/usr/lib/crtbegin_so.o -lgnustl_static -lc -lgcc -ldl -lOpenSLES -llog -L/home/dev/android/pjproject-2.5/pjlib/lib -L/home/dev/android/pjproject-2.5/pjlib-util/lib -L/home/dev/android/pjproject-2.5/pjnath/lib -L/home/dev/android/pjproject-2.5/pjmedia/lib -L/home/dev/android/pjproject-2.5/pjsip/lib -L/home/dev/android/pjproject-2.5/third_party/lib -nostdlib -L/home/dev/android/android-ndk-r9b/platforms/android-13/arch-arm/usr/lib -L/home/dev/android/android-ndk-r9b/sources/cxx-stl/gnu-libstdc++/4.8/libs/armeabi/
Code of guid_androind.c
#include <pj/guid.h>
#include <pj/log.h>
#include <pj/string.h>
#include <jni.h>
extern JavaVM *pj_jni_jvm;
static pj_bool_t attach_jvm(JNIEnv **jni_env)
{
if ((*pj_jni_jvm)->GetEnv(pj_jni_jvm, (void **)jni_env,
JNI_VERSION_1_4) < 0)
{
if ((*pj_jni_jvm)->AttachCurrentThread(pj_jni_jvm, jni_env, NULL) < 0)
{
jni_env = NULL;
return PJ_FALSE;
}
return PJ_TRUE;
}
return PJ_FALSE;
}
#define detach_jvm(attached) \
if (attached) \
(*pj_jni_jvm)->DetachCurrentThread(pj_jni_jvm);
Struggling with this error for whole day, any clue is highly appreciated. Meanwhile I'm able to build version 2.3 without any errors, but not 2.5
I believe you missed a step.
https://trac.pjsip.org/repos/wiki/Getting-Started/Android#BuildPreparation
Set your config_site.h to the following:
/* Activate Android specific settings in the 'config_site_sample.h' */
#define PJ_CONFIG_ANDROID 1
#include <pj/config_site_sample.h>

Resources