I met the problem "undefined symbol" when using mlpack in Cython. Here is my test case:
cdef extern from "<mlpack/core.hpp>" namespace "arma":
ctypedef unsigned uword
cdef cppclass vec:
vec()
vec(uword)
cdef cppclass mat:
mat()
mat(uword, uword)
void matprint "print" ()
double& operator() (const uword, const uword)
cdef extern from "<mlpack/methods/pca/pca.hpp>" namespace "mlpack::pca":
cdef cppclass ExactSVDPolicy:
ExactSVDPolicy()
cdef cppclass PCA[ExactSVDPolicy]:
PCA()
void Apply(const mat&, mat&, vec&, mat&)
cdef mat m = mat(4, 2)
(<double*>&m(0, 0))[0] = 1.2
(<double*>&m(1, 0))[0] = 1.0
(<double*>&m(2, 0))[0] = 0.8
(<double*>&m(3, 0))[0] = 0.6
(<double*>&m(0, 1))[0] = 0.6
(<double*>&m(1, 1))[0] = 0.8
(<double*>&m(2, 1))[0] = 1.0
(<double*>&m(3, 1))[0] = 1.2
cdef vec eig = vec(2)
cdef mat coeff = mat(4, 2)
cdef PCA[ExactSVDPolicy] pca
m.matprint()
pca.Apply(m, m, eig, coeff)
m.matprint()
Here is the setup file:
from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
setup(ext_modules = cythonize([Extension("pca", ["pca.pyx"], language='c++')]))
Compilation was OK, but when I import the module, python complains that:
undefined symbol: _ZN6mlpack5Timer5StartERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
I looked for the symbol, it is defined in the libmlpack.so. I put it in /usr/local/lib, which is included in LD_LIBRARY_PATH, but it seems Python does not find the symbol during runtime. Is there anyone who can help? Thanks.
The extension must be linked to the library it is using.
setup(ext_modules=cythonize([Extension(
"pca", ["pca.pyx"], language='c++'),
libraries='mlpack',
]))
That all symbols can be found, and libraries linked correctly, can be checked by ldd <.so>.
See Compiling and Linking Cython documentation.
Related
If I create a class such as 'A' below:
class A(object):
a = 1
def __truediv__(self, var):
return self.a / var
and then try to divide an int by A as:
print(3 / A())
python raises a TypeError. However, if I divide an int by this object python prints:
print(A() / 3)
python prints 0.333333.
How can I make the class work so that I can perform mathematical operations in any order?
N.B. Numpy arrays seem to be able to work both ways i.e:
import numpy as np
1 / np.arange(1, 5)
np.arange(1, 5) / 1
runs and works as expected.
Also implement the reflected dunder methods. In your case, that's __rtruediv__()
I have to use the rows of a numpy array sequentially so I am doing that with a loop. I tried adding prange for speed but it ends up freezing. I am most likely doing something wrong, I may be misunderstanding the meaning of a race condition in this context. What can be done to fix the problem or to treat the array rows in parallel fashion correctly? Reproducible code below for IPython cells:
%load_ext cython
import numpy as np
%%cython --compile-args=-fopenmp --link-args=-fopenmp --force
# if on Windows, replace line above with: %%cython --compile-args=/openmp --link-args=/openmp --force
cimport cython
from cython.parallel cimport prange
#cython.boundscheck(False)
#cython.wraparound(False)
cdef double sum_row(double [:] arr) nogil:
cdef int i
cdef int size = arr.shape[0]
cdef double s = 0
for i in range(size):
s += arr[i]
return s
#cython.boundscheck(False)
#cython.wraparound(False)
cpdef double sum_all_rows(double [:,:] arr) nogil:
cdef int i
cdef int n_rows = arr.shape[1] + 1
cdef double s = 0
for i in range(n_rows):
s += sum_row(arr[i])
return s
#cython.boundscheck(False)
#cython.wraparound(False)
cpdef double sum_all_rows_parallel(double [:,:] arr) nogil:
cdef int i
cdef int n_rows = arr.shape[1] + 1
cdef double s = 0
for i in prange(n_rows):
s += sum_row(arr[i])
return s
X = np.array([[3.14, 2.71, 0.002],
[0.5, 1, 4.21],
[0.001, 0.002, 0.003],
[-0.1, -0.11, -0.12]])
%timeit sum_all_rows(X)
754 ns ± 38.3 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)
sum_all_rows_parallel(X)
And it hangs... I stopped it after roughly 5 minutes. I did not observe any increasing memory usage nor CPU usage.
I have used the toy examples of prange before seeing the expected speedups, so I am able to compile properly.
Also, any suggested readings to better understand this kind of issue?
Thank you for your help.
This is because nogil means two slightly different things in different contexts. In
cpdef double sum_all_rows_parallel(double [:,:] arr) nogil:
it means "this function does not require the GIL and can be called without it", while in:
with nogil:
# or
prange(..., nogil=True)
it means "release the GIL".
Therefore the GIL is never actually released in your example (since sum_all_rows_parallel is presumably called from Python, which holds the GIL), but prange is working under the assumption that it has been released.
What you should do is change prange(...) to prange(...,nogil=True). You possibly also need to remove the nogil from the sum_all_rows_parallel function definition.
Arguably it's a bug in Cython that it doesn't warn you about the issue, and should be reported. However, I suspect it may not be easily fixed.
i am trying to use the "subs" function for differential equation
but i get the error: "can't convert expression to float"
i tryed to check the type of the arrays, but they all float
import sympy as sym
from sympy.integrals import inverse_laplace_transform
from sympy.abc import s,t,y
import numpy as np
U = 1
G =(s+1)/(s*(s+2))
Y = G*U
y = inverse_laplace_transform(Y, s, t)
tm = np.linspace(0,2,3)
y_val = np.zeros(len(tm))
for i in range(len(tm)):
y_val[i] = y.subs(t, tm[i])
print(y)
print(y_val)
line 17
y_val[i] = y.subs(t, tm[i])
TypeError: can't convert expression to float
Ths issue here is that, because tm[0] == 0, the evaluated y in the first iteration of your loop is Heaviside(0), which has no defined real value by default (see https://docs.sympy.org/latest/modules/functions/special.html#heaviside). This is because you have
from sympy.functions import exp, Heaviside
assert y == Heaviside(t) / 2 + exp(-2 * t) * Heaviside(t) / 2
The simplest workaround here is defining a linear space excluding 0, for instance
epsilon = 1e-15
tm = np.linspace(epsilon, 2, 3)
Using y_val = np.zeros(len(tm)), the default datatype of array is float. After modifying the code, you find that one of y_val elements is an object, not float. You can use a list object as a placeholder or you can specify the datatype of numpy array as object:
import sympy as sym
from sympy.integrals import inverse_laplace_transform
from sympy.abc import s,t,y
import numpy as np
U = 1
G =(s+1)/(s*(s+2))
Y = G*U
y = inverse_laplace_transform(Y, s, t)
tm = np.linspace(0,2,3)
# y_val = [0 for _ in range(len(tm))]
y_val = np.zeros(len(tm), dtype=object)
for i in range(len(tm)):
y_val[i] = y.subs(t, tm[i])
print(y_val)
result: [Heaviside(0.0) 0.567667641618306 0.509157819444367]
I have similar problem and your answers work for me, but I still need to put the data into graph.. I modified my problem for this question:
import sympy as sym
from sympy.integrals import inverse_laplace_transform
from sympy.abc import s,t,y
import numpy as np
import matplotlib.pyplot as plt
Y = (5*(1 - 5*s))/(s*(4*(s**2) + s + 1))*(1/s)
y = inverse_laplace_transform(Y, s, t)
tm = np.linspace(1e-15, 20, 100)
y_val = np.zeros(len(tm), dtype=object)
for i in range(len(tm)):
y_val[i] = y.subs(t, tm[i])
plt.plot(y_val, tm)
plt.show()
Running this code I got same error:
TypeError: can't convert expression to float
import visa
import numpy as np
from struct import unpack
import pylab
rm = visa.ResourceManager()
rm.list_resources()
inst = rm.open_resource('GPIB0::1::INSTR',write_termination= '\n')
print(inst.query("*IDN?"))
print(rm)
print(inst)
values = np.array(inst.query_ascii_values('CURV?', converter='s'))
#values1=inst.write_ascii_values('WLISt:WAVeform:DATA somename,', values, converter='s')
len(values)
print(values)
Hi all,
I am really new with Python and programming. I am trying to get a waveform from a Tektronix oscilloscope (TDS 460 A). I am using a GPIB (GPIB USB-HS) to transfer data. With the code written above, I am able to connect with the oscilloscope. I have some very basic questions. When I print(values), it is giving me a string. I was wondering what is that string? Is it the same as the address of the instrument? Now as I am connected with the instrument, how can I proceed further? My ultimate aim is to get the trace from multiple channels of the scope.
You could use:
from struct import unpack
import pyvisa as visa
rm = visa.ResourceManager()
print(rm)
def acquire(channel, port):
try:
scope = rm.open_resource(port)
scope.write("DATA:SOURCE " + channel)
scope.write('DATA:WIDTH 1')
scope.write('DATA:ENC RPB')
ymult = float(scope.ask('WFMPRE:YMULT?'))
yzero = float(scope.ask('WFMPRE:YZERO?'))
yoff = float(scope.ask('WFMPRE:YOFF?'))
xincr = float(scope.ask('WFMPRE:XINCR?'))
xdelay = float(scope.query('HORizontal:POSition?'))
scope.write('CURVE?')
data = scope.read_raw()
headerlen = 2 + int(data[1])
header = data[:headerlen]
ADC_wave = data[headerlen:-1]
ADC_wave = np.array(unpack('%sB' % len(ADC_wave),ADC_wave))
Volts = (ADC_wave - yoff) * ymult + yzero
Time = np.arange(0, (xincr * len(Volts)), xincr)-((xincr * len(Volts))/2-xdelay)
return Time,Volts
except IndexError:
return 0,0
I keep on getting this error in my numba code:
Warning 101:0: Unused argument 'self'
My numba code is below. How do I suppress the error message?
#autojit
def initialise_output_data(self, input_data, output_data, params ):
# Unpack Params
#omega = params['omega']
#beta = params['beta']
#gamma = params['gamma']
psi = params['psi']
# Unpack Output Data
mu = output_data['mu']
s2 = output_data['sigma2']
res = output_data['residuals']
res2 = output_data['residuals2']
# Initialise Garch Variables
s2[0] = input_data[ 'sample_var' ]
res[0] = psi[0] / ( 1.0-psi[1] )
res2[0] = res[0]**2
mu[0] = psi[0] + psi[1]*res[0]
You can suppress all numba warnings on a specific function with warn=False. For example:
#numba.autojit(warn=False)
def f(a, b):
return a
f doesn't use b but numba does not issue a warning. This works for #numba.jit also. Just be careful!
As autojit doesn't seem to exist anymore, and numba.jit doesn't accept the argument warn, some imperfect ways to handle this may be:
Disable all Numba messages of level WARNING or lower
import logging;
logger = logging.getLogger("numba");
logger.setLevel(logging.ERROR)
Disable all messages of level WARNING or lower altogether
import logging;
logging.disable(logging.WARNING)