Enabling Parallelism with Cython - multithreading

Question

I am trying to get the prange function of Cython's parallel package to work and it seems like there is no parallelism in effect. To have a MWE, I have taken the example code from the book Cython: A Guide for Python Programmers and modified it a little bit by adding a few print statements. The example code is freely available at github and the code I'm referring to is at: examples/12-parallel-cython/02-prange-parallel-loops/.
The following is my modification of the julia.pyx file.
# distutils: extra_compile_args = -fopenmp
# distutils: extra_link_args = -fopenmp
from cython cimport boundscheck, wraparound
from cython cimport parallel
import numpy as np
cdef inline double norm2(double complex z) nogil:
return z.real * z.real + z.imag * z.imag
cdef int escape(double complex z,
double complex c,
double z_max,
int n_max) nogil:
cdef:
int i = 0
double z_max2 = z_max * z_max
while norm2(z) < z_max2 and i < n_max:
z = z * z + c
i += 1
return i
#boundscheck(False)
#wraparound(False)
def calc_julia(int resolution, double complex c,
double bound=1.5, double z_max=4.0, int n_max=1000):
cdef:
double step = 2.0 * bound / resolution
int i, j
double complex z
double real, imag
int[:, ::1] counts
counts = np.zeros((resolution+1, resolution+1), dtype=np.int32)
for i in parallel.prange(resolution + 1, nogil=True,
schedule='static', chunksize=1):
real = -bound + i * step
for j in range(resolution + 1):
imag = -bound + j * step
z = real + imag * 1j
counts[i,j] = escape(z, c, z_max, n_max)
return np.asarray(counts)
#boundscheck(False)
#wraparound(False)
def julia_fraction(int[:,::1] counts, int maxval=1000):
cdef:
unsigned int thread_id
int total = 0
int i, j, N, M
N = counts.shape[0]; M = counts.shape[1]
print("N = %d" % N)
with nogil:
for i in parallel.prange(N, schedule="static", chunksize=10):
thread_id = parallel.threadid()
with gil:
print("Thread %d." % (thread_id))
for j in range(M):
if counts[i,j] == maxval:
total += 1
return total / float(counts.size)
When I compile using the setup_julia.py given by
from distutils.core import setup
from Cython.Build import cythonize
from distutils.extension import Extension
setup(name="julia",
ext_modules=cythonize(Extension('julia', ['julia.pyx'], extra_compile_args=['-fopenmp'], extra_link_args=['-fopenmp'])))
with the command
python setup_julia.py build_ext --inplace
and run the run_julia.py file, I see that all instances of the for loop only use one thread -- Thread 0. The terminal output looks like below.
poulin8:02-prange-parallel-loops poulingroup$ python run_julia.py
time: 0.892143
julia fraction: N = 1001
Thread 0.
Thread 0.
Thread 0.
Thread 0.
.
.
.
.
Thread 0.
0.236994773458
As I understand, the for loop is simply running in parallel. Could someone guide me on what I must do for initiating the for loop to distribute load amongst many threads?
I have also tried to set the system variable OMP_NUM_THREADS to some number greater than 1 and there is no effect of this.
I am running the tests on a OSX 10.11.6, with Python 2.7.10 and gcc 5.2.0.

Answer 1

I've got the same problem on Windows 7.
It was running serial.
Noticed compilation message:
python setup_julia.py build_ext --inplace
cl : Command line warning D9002 : ignoring unknown option '-fopenmp'
Apparently in Visual Studio it has to be -openmp
# distutils: extra_compile_args = -openmp
# distutils: extra_link_args = -openmp
Now runs in parallel.
As #danny noted you can use fprintf:
from cython.parallel cimport prange, threadid
from libc.stdio cimport stdout, fprintf
def julia_fraction(int[:,::1] counts, int maxval=1000):
...
thread_id = threadid()
fprintf(stdout, "%d\n", <int>thread_id)
...