To begin with, I have followed up the instruction of the installation guide and deploy the new virtual environment with the yml file presented over that page, but I still met with the same problem as I executed the following codes. I have tried out many ways to solve the problem, please assist me to solve the problem.
Plus, I have referred to the issues on the official pymc3 website, but the problem still existed.
The reproducible example:
import pymc3 as pm
import numpy as np
import pandas as pd
import scipy.stats as stats
from datetime import datetime
import theano.tensor as T
early = 10
late = 12
y = np.r_[np.random.poisson(early, 25), np.random.poisson(late, 75)]
niter = 10000
t = range(len(y))
with pm.Model() as change_point:
cp = pm.DiscreteUniform('change_point', lower=0, upper=len(y), testval=len(y)//2)
mu0 = pm.Exponential('mu0', 1/y.mean())
mu1 = pm.Exponential('mu1', 1/y.mean())
mu = T.switch(t < cp, mu0, mu1)
Y_obs = pm.Poisson('Y_obs', mu=mu, observed=y)
trace = pm.sample(niter)
pm.traceplot(trace, varnames=['change_point', 'mu0', 'mu1'])
Here is the error reports:
[You can find the C code in this temporary file: C:\Users\Mick\AppData\Local\Temp\theano_compilation_error_xk8zcr1g
Traceback (most recent call last):
File "C:\....py", line 48, in <module>
mu = T.switch(t < cp, mu0, mu1)
File "C:\...\lib\site-packages\theano\tensor\var.py", line 41, in __gt__
rval = theano.tensor.basic.gt(self, other)
File "C:\...\lib\site-packages\theano\graph\op.py", line 253, in __call__
compute_test_value(node)
File "C:\...\lib\site-packages\theano\graph\op.py", line 126, in compute_test_value
thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
File "C:\...\lib\site-packages\theano\graph\op.py", line 634, in make_thunk
return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
File "C:\...\lib\site-packages\theano\graph\op.py", line 600, in make_c_thunk
outputs = cl.make_thunk(
File "C:\...\lib\site-packages\theano\link\c\basic.py", line 1203, in make_thunk
cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
File "C:\...\lib\site-packages\theano\link\c\basic.py", line 1138, in __compile__
thunk, module = self.cthunk_factory(
File "C:\...\lib\site-packages\theano\link\c\basic.py", line 1634, in cthunk_factory
module = get_module_cache().module_from_key(key=key, lnk=self)
File "C:\...\lib\site-packages\theano\link\c\cmodule.py", line 1191, in module_from_key
module = lnk.compile_cmodule(location)
File "C:\...\lib\site-packages\theano\link\c\basic.py", line 1543, in compile_cmodule
module = c_compiler.compile_str(
File "C:\...\lib\site-packages\theano\link\c\cmodule.py", line 2546, in compile_str
raise Exception(
Exception: ('Compilation failed (return status=1): C:\\...\\AppData\\Local\\Temp\\ccujaONv.s: Assembler messages:\r. C:\\...\\AppData\\Local\\Temp\\ccujaONv.s:89: Error: invalid register for .seh_savexmm\r. ', 'FunctionGraph(Elemwise{gt,no_inplace}(<TensorType(int64, (True,))>, TensorConstant{[ 0 1 2 .. 97 98 99]}))')]
Related
I have created tf.data.Dataset pipeline variable to which I pass a python function which is trying to run sox command through python's subprocess module. The code is running fine on cpu with windows os but is not able to run on Google Colab's GPU which has linux os. Here is the code -
from scipy.io import wavfile
import tensorflow as tf
def tf_func(inp_sample):
def func(inp_sample,<tf.string object>):
try:
fdesc, infile = tempfile.mkstemp(suffix=".wav")
os.close(fdesc)
fdesc, outfile = tempfile.mkstemp(suffix=".wav")
os.close(fdesc)
wavfile.write(infile,<sample rate>,inp_sample.numpy()) //writes audio file to disk
arguments = ['sox',infile,outfile,'-q','compand',
*DRC_PRESET[<tf.string object>.numpy().decode('utf-8')]]
subprocess.check_call(arguments)
finally:
os.unlink(infile)
os.unlink(outfile)
return tf.convert_to_tensor(inp_sample,dtype=tf.float32)
drc = np.random.choice(<lis of string>)
[inp_sample,] = tf.py_function(dynamic_compression,
[inp_sample,tf.constant(drc)],[tf.float32])
inp_sample.set_shape(target_sample_size) //target_size=<some int>
return inp_sample
...
inp=tf.data.Dataset.from_tensor_slices(inp) // inp shape eg: (4, 500)
inp = inp.map(tf_dynamic_compression)
for i in inp:
print(i.numpy())
And the error it throws -
UnknownError: 2 root error(s) found.
(0) Unknown: CalledProcessError: Command 'None' died with <Signals.SIGINT: 2>.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/script_ops.py", line 233, in __call__
return func(device, token, args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/script_ops.py", line 122, in __call__
ret = self._func(*args)
File "/tmp/tmpn_9q_jxm.py", line 24, in dynamic_compression
ag__.converted_call(subprocess.check_call, dynamic_compression_scope.callopts, (arguments,), None, dynamic_compression_scope)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py", line 541, in converted_call
result = converted_f(*effective_args)
File "/tmp/tmpozf4qyav.py", line 50, in tf__check_call
ag__.if_stmt(cond_1, if_true_1, if_false_1, get_state_1, set_state_1, (), ())
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/operators/control_flow.py", line 895, in if_stmt
return _py_if_stmt(cond, body, orelse)
File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/operators/control_flow.py", line 1004, in _py_if_stmt
return body() if cond else orelse()
File "/tmp/tmpozf4qyav.py", line 44, in if_true_1
raise ag__.converted_call(CalledProcessError, check_call_scope.callopts, (retcode, cmd), None, check_call_scope)
How to solve this problem?
The problem was coming due to sox not getting executed by subprocess module. It has been answered earlier. There are two solutions -
First change the arguments line
arguments = " ".join(['sox',infile,outfile,'-q','compand',*DRC_PRESET[<tf.string object>.numpy().decode('utf-8')]])
and then
os.system(arguments)
OR
subprocess.check_call(arguments, shell=True)
For me the second worked.
I wrote a simple code using concurrent.futures.ProcessPoolExecutor() whihc you can see in below. I'm using Python 3.7.4 on Windows 10 (64 bit) on a Core-i7 laptop.
import time
import concurrent.futures
def f(x):
lo = 0
for i in range(x):
lo += i
return(lo)
n = 7
if __name__ == '__main__':
t1 = time.perf_counter()
with concurrent.futures.ProcessPoolExecutor() as executor:
Ans = [executor.submit(f, 10**7-i) for i in range(n)]
for f in concurrent.futures.as_completed(Ans):
print(f.result())
t2 = time.perf_counter()
print('completed at', t2-t1, 'seconds')
The variable n determines how many processes is going to execute. When I set n to 1, 2, 4, 7 everything work fine. For example the output for n=7 is
49999995000000
49999955000010
49999965000006
49999985000001
49999975000003
49999945000015
49999935000021
completed at 2.0607623 seconds
However for n=10 it gives the following error!
49999945000015
49999955000010
49999965000006
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
File "E:\Python37\lib\multiprocessing\queues.py", line 236, in _feed
obj = _ForkingPickler.dumps(obj)
File "E:\Python37\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function f at 0x00000285BFC4E0D8>: it's not the same object as __main__.f
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "e:/Python37/Python files/Parallel struggle/Python_20191219_parallel_3.py", line 23, in <module>
print(f.result())
File "E:\Python37\lib\concurrent\futures\_base.py", line 428, in result
return self.__get_result()
File "E:\Python37\lib\concurrent\futures\_base.py", line 384, in __get_result
raise self._exception
File "E:\Python37\lib\multiprocessing\queues.py", line 236, in _feed
obj = _ForkingPickler.dumps(obj)
File "E:\Python37\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function f at 0x00000285BFC4E0D8>: it's not the same object as __main__.f
Why some of the processes get done if something is wrong with the code? And what has happened that an error comes up? And is it specific to python on windows? Is it about the number of CPUs?
I would like to generate a jpeg or png file using the Pyx module. I found
pyx.canvas.writeGSfile() which I tried to use in the following code:
from pyx import *
def mark(x, y):
return path.circle(x, y, 0.1)
if __name__ == '__main__':
c = canvas.canvas()
# r = 70
f = trafo.scale(0.7, 1)
circle = path.circle(0, 0, 2)
a, b = circle.split([0, 2])
c.stroke(a, [style.linewidth.Thick, f])
d1 = f.apply(*circle.at(0.5*circle.arclen()))
d2 = f.apply(*circle.at(circle.arclen()))
c.fill(mark(*d1))
c.fill(mark(*d2))
c.writeGSfile('test.jpeg')
But it spits back the following error:
Traceback (most recent call last):
File "/Python/pyx/test.py", line 19, in <module>
c.writeGSfile('test.png')
File "\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyx\canvas.py", line 474, in writeGSfile
p = config.Popen(cmd, stdin=config.PIPE)
File "\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pyx\config.py", line 218, in Popen
return subprocess.Popen(cmd, *args, **kwargs)
File "\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I tried to debug but I'm having trouble understanding the processing.py section. Any ideas?
You need to have ghostscript installed (gs) to generate bitmap output.
I am using astroplan to set up some astronomical observations. Previously, when I ran my code using Python 2.7, it plotted the target on the sky properly. Now, I have moved to Python 3.7 and I get an AttributError on the same code.
I took my larger code and stripped out everything that did not seem to trigger the error. Here below is code that will generate the complaint.
from astroplan import Observer, FixedTarget
import astropy.units as u
from astropy.time import Time
import matplotlib.pyplot as plt
from astroplan.plots import plot_sky
import numpy as np
time = Time('2015-06-16 12:00:00')
subaru = Observer.at_site('subaru')
vega = FixedTarget.from_name('Vega')
sunset_tonight = subaru.sun_set_time(time, which='nearest')
vega_rise = subaru.target_rise_time(time, vega) + 5*u.minute
start = np.max([sunset_tonight, vega_rise])
plot_sky(vega, subaru, start)
plt.show()
Expected result was a simple plot of the target, in this case, the star Vega, on the sky as seen by the Subaru telescope in Hawaii. The astroplan docs give a tutorial that shows how it was to look at the very end of this page:
https://astroplan.readthedocs.io/en/latest/tutorials/summer_triangle.html
Instead, I now get the following error:
Traceback (most recent call last):
File "plot_sky.py", line 16, in <module>
plot_sky(vega, subaru, start)
File "/usr1/local/anaconda_py3/ana37/lib/python3.7/site-packages/astropy/utils/decorators.py", line 842, in plot_sky
func = make_function_with_signature(func, name=name, **wrapped_args)
File "/usr1/local/anaconda_py3/ana37/lib/python3.7/site-packages/astropy/units/decorators.py", line 222, in wrapper
return_ = wrapped_function(*func_args, **func_kwargs)
File "/local/data/fugussd/rkbarry/.local/lib/python3.7/site-packages/astroplan/plots/sky.py", line 216, in plot_sky
ax.set_thetagrids(range(0, 360, 45), theta_labels, frac=1.2)
File "/usr1/local/anaconda_py3/ana37/lib/python3.7/site-packages/matplotlib/projections/polar.py", line 1268, in set_thetagrids
t.update(kwargs)
File "/usr1/local/anaconda_py3/ana37/lib/python3.7/site-packages/matplotlib/text.py", line 187, in update
super().update(kwargs)
File "/usr1/local/anaconda_py3/ana37/lib/python3.7/site-packages/matplotlib/artist.py", line 916, in update
ret = [_update_property(self, k, v) for k, v in props.items()]
File "/usr1/local/anaconda_py3/ana37/lib/python3.7/site-packages/matplotlib/artist.py", line 916, in <listcomp>
ret = [_update_property(self, k, v) for k, v in props.items()]
File "/usr1/local/anaconda_py3/ana37/lib/python3.7/site-packages/matplotlib/artist.py", line 912, in _update_property
raise AttributeError('Unknown property %s' % k)
AttributeError: Unknown property frac
Here are a few lines of code:
import pandas as pd
import numpy as np
from mlxtend.frequent_patterns import apriori
from mlxtend.frequent_patterns import association_rules
def encode_c(x):
if x <= 0:
return 0
if x >= 1:
return 1
return 0
def get_product_consequents():
all_transactions_df = get_dataframe()
basket = (all_transactions_df.groupby(['a', 'b'])['c']
.sum().unstack().reset_index().fillna(0)
.set_index('a'))
basket = basket.applymap(encode_c)
frequent_itemsets = apriori(basket, min_support=0.07, use_colnames=True)
rules = association_rules(frequent_itemsets, metric="lift",
min_threshold=1)
rules['antecedants_length'] = rules['antecedents'].str.len()
rules['consequents_length'] = rules['consequents'].str.len()
rules = rules[(rules['lift'] >= 4) & # 6
(rules['confidence'] >= 0.4)] # 0.8
rules = rules[(rules['antecedants_length'] == 1) &
(rules['consequents_length'] == 1)]
rules = (rules.groupby(['antecedants'])['consequents'])
IMAGE 1
IMAGE 2
When I DO: learning apriori from
rules["antecedant_len"] = rules["antecedents"].apply(lambda x: len(x))
I GET:
IMAGE 1
C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\_pydevd_bundle
\pydevd_resolver.py:71:
FutureWarning: Series.strides is deprecated
and will be removed in a future version
return getattr(var, attribute)
I was doing it from: mlxtend association_rules and apriori docs
As the error says: Series.strides are deprecated.
So how can I find the length of each frozenset in series? i.e. do same as above?
MAIN ERROR
IMAGE 2
rules = (rules.groupby(['antecedants'])['consequents'])
After I evaluate above line:
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm
2018.1.4\helpers\pydev\_pydevd_bundle\pydevd_vars.py", line 376, in
evaluate_expression compiled = compile(expression, '<string>', 'eval')
File "<string>", line 1
rules = (rules.groupby(['antecedants'])['consequents'])
^
SyntaxError: invalid syntax
-----WHY Syntax error, though works fine on JUPYTER NOTEBOOK?
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm
2018.1.4\helpers\pydev\_pydevd_bundle\pydevd_comm.py", line 1159, in do_it
result = pydevd_vars.evaluate_expression(self.thread_id, self.frame_id,
self.expression, self.doExec)
File "C:\Program Files\JetBrains\PyCharm
2018.1.4\helpers\pydev\_pydevd_bundle\pydevd_vars.py", line 378, in
evaluate_expression
Exec(expression, updated_globals, frame.f_locals)
File "C:\Program Files\JetBrains\PyCharm
2018.1.4\helpers\pydev\_pydevd_bundle\pydevd_exec2.py", line 3, in Exec
exec(exp, global_vars, local_vars)
File "<string>", line 1, in <module>
File "C:\ProjcetPath\venv\lib\site-
packages\pandas\core\generic.py", line 6659, in groupby
observed=observed, **kwargs)
File "C:\ProjcetPath\venv\lib\site-
packages\pandas\core\groupby\groupby.py", line 2152, in groupby
return klass(obj, by, **kwds)
File "C:\ProjcetPath\venv\lib\site-
packages\pandas\core\groupby\groupby.py", line 599, in __init__
mutated=self.mutated)
File "C:\ProjcetPath\venv\lib\site-
packages\pandas\core\groupby\groupby.py", line 3291, in _get_grouper
raise KeyError(gpr)
KeyError: 'antecedants'
--- At last it says key error, during handling syntax error key error occurred, For sure it is not key error as I can see column in sciView, also am able to access it this was as done in above code lines.
OTHER INFO: I am using Django as well.