Golang Embedding Python in Go - python-3.x

I've been reading the articles embedding-python-in-go and the original article packaging-python-code, but I have yet to succeed at running Python.h through cgo.
Can this be done on an m1 mac? if not, should I port everything into a Linux docker container?
what am I missing when setting up this type of project.
below is a simple example from article 1.
This leads to the error
./main.go:22:4: could not determine kind of name for C.PyRun_SimpleString
1 package main
2 /*
3 #cgo pkg-config: python3
#include <Python/Python.h>
4 */
5 import "C"
6
7 import (
8 "unsafe"
9 )
10
11 func main() {
12
13 pycodeGo := `
14 import sys
15 for path in sys.path:
16 print(path)
17 `
18
19 defer C.Py_Finalize()
20 C.Py_Initialize()
21 pycodeC := C.CString(pycodeGo)
22 defer C.free(unsafe.Pointer(pycodeC))
23 C.PyRun_SimpleString(pycodeC)
24
25 }
removing lines 20-23 returns error
# command-line-arguments
Undefined symbols for architecture arm64:
"_Py_Finalize", referenced from:
__cgo_672c0788d500_Cfunc_Py_Finalize in _x002.o
(maybe you meant: __cgo_672c0788d500_Cfunc_Py_Finalize)
"_Py_Initialize", referenced from:
__cgo_672c0788d500_Cfunc_Py_Initialize in _x002.o
(maybe you meant: __cgo_672c0788d500_Cfunc_Py_Initialize)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [run] Error 2
kinda at a lost on the architecture error; m1 issue?
importing python.h on mac is #include <Python/Python.h> instead of #include <Python.h> importing Python.h on mac
software specs
$ python3 -V Python 3.9.10
$ go version go version go1.17.8 darwin/arm64
macOS verrsion 11.4, M1 2020.
Python.h is present under/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/Headers/Python.h
reference git repos ardanlabs/python-go go-python3 christian-korneck/python-go

Related

Macbook PRO 2015 OS Big SUR Python Crash

I am trying to launch python 3 on my macbook, it use to work before ... now if I launch on the termina "python3" it crash and I got this message.
I don't know what to do...
> user:~ my_mac$ python3 dyld: Library not loaded:
> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
> Referenced from:
> /Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
> Reason: image not found Abort trap: 6
I Tried to upgrade the python version and got this message:
> ==> python#3.9 Python has been installed as /usr/local/bin/python3
>
> Unversioned symlinks `python`, `python-config`, `pip` etc. pointing to
> `python3`, `python3-config`, `pip3` etc., respectively, have been
> installed into /usr/local/opt/python#3.9/libexec/bin
>
> You can install Python packages with pip3 install <package> They
> will install into the site-package directory
> /usr/local/lib/python3.9/site-packages
Here the error page.
>Process: Python [65598]
>Path: >/Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
>Identifier: Python
>Version: 3.5.2 (3.5.2)
>Code Type: X86-64 (Native)
>Parent Process: bash [48227]
>Responsible: Terminal [48225]
>User ID: 501
>
>Date/Time: 2021-02-06 09:57:55.710 +0100
>OS Version: macOS 11.1 (20C69)
>Report Version: 12
>Anonymous UUID: FB4549E5-EACE-CBB0-1BA6-6767B1DAD93D
>
>Sleep/Wake UUID: 7A0A2DD1-F786-48A4-8641-D98824F7673B
>
>Time Awake Since Boot: 510000 seconds
>Time Since Wake: 5200 seconds
>
>System Integrity Protection: enabled
>
>Crashed Thread: 0
>
>Exception Type: EXC_CRASH (SIGABRT)
>Exception Codes: 0x0000000000000000, 0x0000000000000000
>Exception Note: EXC_CORPSE_NOTIFY
>
>Termination Reason: DYLD, [0x1] Library missing
>
>Application Specific Information:
>dyld: launch, loading dependent libraries
>
>Dyld Error Message:
> dyld: No shared cache present
>Library not loaded: >/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
> Referenced from: >/Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
> Reason: image not found
>
>Binary Images:
> 0x100000000 - 0x100000fff +org.python.python (3.5.2 - 3.5.2) <78B09BD7-3C8E-0E55->F3A1-BA0FF24E8496> >/Library/Frameworks/Python.framework/Versions/3.5/Resources/Python.app/Contents/MacOS/Python
> 0x7fff6679f000 - 0x7fff6683afff dyld (832.7.1) <DEA51514-B4E8-3368-979B-89D0F8397ABC> >/usr/lib/dyld
>
>Model: MacBookPro11,5, BootROM 425.0.0.0.0, 4 processors, Quad-Core Intel Core i7, 2,8 GHz, 16 >GB, SMC 2.30f2

Error when compiling pytorch: 'cstdint' file not found

I was attempting to compile pytorch using NO_CUDA=1 python setup.py install on Mac OS X, but I got these errors:
In file included from /Users/ezyang/Dev/pytorch-tmp/torch/lib/tmp_install/include/THPP/Tensor.hpp:3:
/Users/ezyang/Dev/pytorch-tmp/torch/lib/tmp_install/include/THPP/Storage.hpp:6:10: fatal error:
'cstdint' file not found
#include <cstdint>
^
1 error generated.
In file included from torch/csrc/autograd/functions/init.cpp:2:
In file included from torch/csrc/autograd/functions/batch_normalization.h:4:
In file included from /Users/ezyang/Dev/pytorch-tmp/torch/lib/tmp_install/include/THPP/THPP.h:4:
/Users/ezyang/Dev/pytorch-tmp/torch/lib/tmp_install/include/THPP/Storage.hpp:6:10: fatal error:
'cstdint' file not found
#include <cstdint>
^
1 error generated.
1 error generated.
1 error generated.
1 error generated.
In file included from torch/csrc/autograd/python_hook.cpp:5:
In file included from /Users/ezyang/Dev/pytorch-tmp/torch/csrc/THP.h:32:
/Users/ezyang/Dev/pytorch-tmp/torch/csrc/utils.h:6:10: fatal error:
'type_traits' file not found
#include <type_traits>
^
13 errors generated.
1 error generated.
1 error generated.
error: command 'gcc' failed with exit status 1
You need to setup environment variables to have Python use the correct C compiler on OS X. You should do this instead:
NO_CUDA=1 MACOSX_DEPLOYMENT_TARGET=10.9 CC=clang CXX=clang++ python setup.py install

Cython Hello World Example works on Anacondas Python 3.3, but not 3.4

I am using Anacondas on a 64-bit Windows machine.
I have compiled a hello world Cython example. It is in file hello.pyx, and contains:
def say_hello_to(name):
print("Hello %s!" % name)
I am running it using run_hello.py
import pyximport; pyximport.install()
import hello as hello
hello.say_hello_to('jon')
The setup file is setup.py:
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = 'Hello world app',
ext_modules = cythonize("hello.pyx"),
)
I am then compiling the code in Python 3.3 on Anacondas, using this code:
> activate py33
> python setup.py build_ext --inplace
( please note that py33 is my Python 3.3 environment )
I can then run the example:
python run_hello.py
which prints out "Hello jon!" as expected.
Now if I change my environment to Python 3.4 and compile:
> activate py34
> python setup.py build_ext --inplace
I get no error, and the shell displays
running build_ext
However, if I try to run run_hello.py from the py34 environment with:
python run_hello.py
I get:
Traceback (most recent call last):
File "run_hello.py", line 2, in <module>
import hello as hello
ImportError: DLL load failed: The specified module could not be found.
The error is not very descriptive. What could I do to help me make this work on Python 3.4?
If I delete hello.c and the /build folder from my hard drive, trying to compile from Python 3.4 returns:
Compiling hello.pyx because it changed.
Cythonizing hello.pyx
running build_ext
building 'hello' extension
creating build
creating build\temp.win-amd64-3.4
creating build\temp.win-amd64-3.4\Release
C:\Anaconda\envs\py34\Scripts\gcc.bat -mdll -O -Wall -IC:\Anaconda\envs\py34\include -IC:\Anaconda\envs\py34\include -c hello.c -o build\temp.win-amd64-3.4\Release\hello.o
writing build\temp.win-amd64-3.4\Release\hello.def
C:\Anaconda\envs\py34\Scripts\gcc.bat -shared -s build\temp.win-amd64-3.4\Release\hello.o build\temp.win-amd64-3.4\Release\hello.def -LC:\Anaconda\envs\py34\libs -LC:\Anaconda\envs\py34\PCbuild\amd64
-lpython34 -lmsvcr100 -o c:\Users\Jon\Documents\GitHub\CythonFunctions\example1\hello.pyd
build\temp.win-amd64-3.4\Release\hello.o:hello.c:(.text+0x314): undefined reference to `__imp__PyThreadState_Current'
build\temp.win-amd64-3.4\Release\hello.o:hello.c:(.text+0x493): undefined reference to `__imp__Py_NoneStruct'
build\temp.win-amd64-3.4\Release\hello.o:hello.c:(.text+0x97b): undefined reference to `__imp_PyExc_ImportError'
collect2.exe: error: ld returned 1 exit status
error: command 'C:\\Anaconda\\envs\\py34\\Scripts\\gcc.bat' failed with exit status 1
If I do the same with Python 3.3, I get:
Compiling hello.pyx because it changed.
Cythonizing hello.pyx
running build_ext
building 'hello' extension
creating build
creating build\temp.win-amd64-3.3
creating build\temp.win-amd64-3.3\Release
C:\Anaconda\envs\py33\Scripts\gcc.bat -DMS_WIN64 -mdll -O -Wall -IC:\Anaconda\envs\py33\include -IC:\Anaconda\envs\py33\include -c hello.c -o build\temp.win-amd64-3.3\Release\hello.o
writing build\temp.win-amd64-3.3\Release\hello.def
C:\Anaconda\envs\py33\Scripts\gcc.bat -DMS_WIN64 -shared -s build\temp.win-amd64-3.3\Release\hello.o build\temp.win-amd64-3.3\Release\hello.def -LC:\Anaconda\envs\py33\libs -LC:\Anaconda\envs\py33\PCb
uild\amd64 -lpython33 -lmsvcr100 -o c:\Users\Jon\Documents\GitHub\CythonFunctions\example1\hello.pyd
Some other users who experience the "gcc.bat failed with exit status 1" have found that the problem is due to 32/64 bit conflicts.
In the py33 version of the compilation data, there is -DMS_WIN64 in the gcc.bat parameters, but it is not in the py34 parameters. Could that be what is causing my issue? If so, how do I get py34 to add it?
I had been using TDM-GCC as the 64-bit compiler with Python 2.7 64-bit for several years without any problems, but after installing Python 3.4 64-bit, I did have problems compiling a Cython module.
I believe I fixed this issue with mingw-w64-for-python. (Also take a look at this github issue).
There is a readme (currently mingwpy-2015-04-readme.pdf), so it is straightforward. But briefly you need to do:
Download mingwpy x86-64 toolchain (64 bit) (currently mingw64static-2014-11.tar.xz)
Unpack the file into a directory (e.g., C:\mingw64static)
Add C:\mingw64static\bin in front of C:\Anaconda3\Scripts in your Path
environmental variable.
Follow the instructions in the readme file to download libpython. (I used
libpython-cp34-none-win_amd64.7z for Python 3.4 64 bit)
Unpack, and copy files (libmsvcr100.a, libpython34.dll.a) in the archive into C:\Python\libs\ directory (for Anaconda, it is C:\Anaconda3\libs)
Create a file called distutils.cfg in your C:\Python\Lib\distutils
directory (for Anaconda, it is C:\Anaconda3\Lib\distutils) with the
following contents:
[build]
compiler=mingw32
Now python will correctly use mingw-w64 when you compile your cython module.
It looks like the py33 environment is compiling using Visual Studio (-lmsvcr100). This is probably because it doesn't have the libpython conda package installed in it, which causes distutils to use mingw (gcc) to compile instead of Visual Studio. conda remove libpython will likely solve the issue for your Python 3.4 environment.

test test_urllibnet failed in Linux

I want to install Python. I tried both Python3.3 and Python3.4 I downloaded the file from https://www.python.org/download/releases/ and followed the instructions.
I executed :
./configure
make
successful. But when execute :
make test
it failed for test_urllibnet.
So according the README file if executed for Python3.4:
./python -m test -v test_urllibnet
== CPython 3.4.0 (default, Apr 16 2014, 16:38:06) [GCC 4.8.1]
== Linux-3.11.0-15-generic-x86_64-with-debian-wheezy-sid little-endian
== hash algorithm: siphash24 64bit
== /home/bernard/Python/Python3.4/Python-3.4.0/build/test_python_27532
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_urllibnet
test_urllibnet skipped -- Use of the 'network' resource not enabled
1 test skipped:
test_urllibnet
So now it skipped because the 'network' is not enabled. But how to enable the 'network'? I could not find it.
On a different computer I try to install Python3.3 with all most the same error.
./python -m test -v test_urllibnet
== CPython 3.3.5 (default, Apr 24 2014, 17:18:29) [GCC 4.6.3]
== Linux-3.5.0-38-generic-x86_64-with-debian-wheezy-sid little-endian
== /home/bernard/Downloads/Python-3.3.5/build/test_python_6523
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1)
[1/1] test_urllibnet
test_urllibnet skipped -- Use of the 'network' resource not enabled
1 test skipped:
test_urllibnet
Those skips are all expected on linux.
Notice this should be Ok for Linux. But in Python3.3 the statement
Python 3.3.5 (default, Apr 24 2014, 17:18:29)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.3/ssl.py", line 60, in <module>
import _ssl # if we can't import it, let the error propagate
ImportError: No module named '_ssl'
So it is not Ok. The solution in python3.3.1 ssl module looks quite similar, but does not work for me.
I looked into
Building Python and more on missing modules
but also without result.
Just installing software, here Python, is escalating and gets real complex.
No solution for me, just a work-around. Installing just Python gets to complex and take to much time. Others seems to have the same problem.
The work around is just use the pre-installed Python2.7 which works. I tried to avoid Python2.7 as much as possible, but now I am obliged to use it. This answer may help to prevent people spend to much time in solving a problem which is hard to solve.

Installing octave package in ubuntu

anyone can help me with this error?
octave:4> pkg install signal-1.2.0.tar.gz
error: the following dependencies where unsatisfied:
signal needs optim >= 1.0.0
signal needs specfun >= 0.0.0
signal needs control >= 2.2.3
signal needs general >= 1.3.2
octave:4> pkg install optim-1.2.2.tar.gz
error: the following dependencies where unsatisfied:
optim needs miscellaneous >= 1.0.10
optim needs struct >= 1.0.10
octave:4> pkg install struct-1.0.10.tar.gz
make: /usr/bin/mkoctfile: Command not found
make: *** [fields2cell.oct] Error 127
'make' returned the following error: make: Entering directory `/tmp/oct-fDBs5k/struct-1.0.10/src'
/usr/bin/mkoctfile -s fields2cell.cc
make: Leaving directory `/tmp/oct-fDBs5k/struct-1.0.10/src'
error: called from `pkg>configure_make' in file /usr/share/octave/3.6.2/m/pkg/pkg.m near line 1391, column 9
error: called from:
error: /usr/share/octave/3.6.2/m/pkg/pkg.m at line 834, column 5
error: /usr/share/octave/3.6.2/m/pkg/pkg.m at line 383, column 9
I need to install signal package, but I have to download other to work from I have noticed but it gets stuck when installing struct package.
#edit
The same error when trying to install misc~ package.
octave:5> pkg install miscellaneous-1.2.0.tar.gz
error: the following dependencies where unsatisfied:
miscellaneous needs general >= 1.3.1
octave:5> pkg install general-1.3.2.tar.gz
make: /usr/bin/mkoctfile: Command not found
make: *** [__exit__.oct] Error 127
'make' returned the following error: make: Entering directory `/tmp/oct-CA6o4U/general/src'
/usr/bin/mkoctfile __exit__.cc
make: Leaving directory `/tmp/oct-CA6o4U/general/src'
error: called from `pkg>configure_make' in file /usr/share/octave/3.6.2/m/pkg/pkg.m near line 1391, column 9
error: called from:
error: /usr/share/octave/3.6.2/m/pkg/pkg.m at line 834, column 5
error: /usr/share/octave/3.6.2/m/pkg/pkg.m at line 383, column 9
#edit
additional info:
octave:6> which mkoctfile
`mkoctfile' is a function from the file /usr/share/octave/3.6.2/m/miscellaneous/mkoctfile.m
I have no idea what to do..
You need to install mkoctfile. Depending on your ubuntu version this will be in different packages. It used be in octave-pkg-dev but now is on liboctave-dev.
Ubuntu and Debian place this as separate package because it is only needed if you want to build the packages yourself. They expect you to use the Octave packages that they packaged (sudo apt-get install octave-signal). Their versions will be behind the actual version but the same happens with the Octave version they distribute. That's the price to pay for stability.

Resources