Python Deap GP Evaluating individual causes error - python-3.x

I am currently experiencing an issue whenever I try to evaluate an individual using the GP portion of DEAP.
I receive the following error:
Traceback (most recent call last):
File "ImageGP.py", line 297, in <module>
pop, logs = algorithms.eaSimple(pop, toolbox, 0.9, 0.1, 60, stats=mstats, halloffame=hof, verbose=True)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/deap/algorithms.py", line 148, in eaSimple
for ind, fit in zip(invalid_ind, fitnesses):
File "ImageGP.py", line 229, in evalFunc
func = toolbox.compile(expr=individual)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/deap/gp.py", line 451, in compile
return eval(code, pset.context, {})
File "<string>", line 1
lambda oValue,oAvg13,oAvg17,oAvg21,sobelVal(v),sobelVal(h),edgeVal,blotchVal: [[[0, 75, 82.2857142857, 83.0, 82.9090909091, 4, 12, 4, 180], ... Proceed to print out all of my data ... [0, 147, 151.244897959, 150.728395062, 150.73553719, 248, 244, 5, 210]]]
^
SyntaxError: invalid syntax
If anyone has any ideas about what could be causing this problem, then I would really appreciate some advice. My current evaluation function looks like this:
def evalFunc(individual, data, points):
func = toolbox.compile(expr=individual)
total = 1.0
for point in points:
tmp = [float(x) for x in data[point[1]][point[0]][1:9]]
total += int((0 if (func(*tmp)) < 0 else 1) == points[2])
print ("Fitness: " + str(total))
return total,
Where the data contains the data being used (the values for the 8 variables listed in the error) and point specifying the x and y co-ordinates from which to get those 8 values. Thank you for your suggestions!

Related

Double integrate with sipy

I want ask how can calculate this integral with python
enter image description here
from scipy import integrate
Hight_above_ground = 2.5
surface_ground = 81
surface_collector = 65
f = (np.cos(tilt)*np.cos(tilt)) / (PI * s **2 )
(1/surface_ground) * integrate.dblquad(f, 0, surface_ground, 0, surface_collector)
NB : A1 and A2 are 2 surface areas A1 is surface_gound and A2 is surface_collector
and this the error given
Traceback (most recent call last):
File "Bifacial_systems.py", line 34, in <module>
(1/surface_ground)*integrate.dblquad(f, 0, surface_ground, 0, surface_collector)
File "C:\Users\S1-DEV-Manel\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\integrate\quadpack.py", line 602, in dblquad
opts={"epsabs": epsabs, "epsrel": epsrel})
File "C:\Users\S1-DEV-Manel\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\integrate\quadpack.py", line 826, in nquad
return _NQuad(func, ranges, opts, full_output).integrate(*args)
File "C:\Users\S1-DEV-Manel\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\integrate\quadpack.py", line 881, in integrate
**opt)
File "C:\Users\S1-DEV-Manel\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\integrate\quadpack.py", line 352, in quad
points)
File "C:\Users\S1-DEV-Manel\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\integrate\quadpack.py", line 463, in _quad
return _quadpack._qagse(func,a,b,args,full_output,epsabs,epsrel,limit)
File "C:\Users\S1-DEV-Manel\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\integrate\quadpack.py", line 881, in integrate
**opt)
File "C:\Users\S1-DEV-Manel\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\integrate\quadpack.py", line 352, in quad
points)
File "C:\Users\S1-DEV-Manel\AppData\Local\Programs\Python\Python36\lib\site-packages\scipy\integrate\quadpack.py", line 463, in _quad
return _quadpack._qagse(func,a,b,args,full_output,epsabs,epsrel,limit)
ValueError: invalid callable given
You have to rewrite your integrand making expressing expressiong dA1 and dA2 in terms of theta1, theta2, d theta1 and d theta2. Assuming that dA1 dA2 = d theta1 d theta2 the solution would be
from scipy.integrate import dblquad
def integrand(theta1, theta2, S):
return np.cos(theta1)*np.cos(theta2)/S**2
S = 1
dblquad(integrand, 0, np.pi/2, 0, lambda x: np.pi/2-x, args=(S,))
In case you end up with an integral of higher order you can use nquad

KeyError: 0 - Function works initially, but returns error when called on other data

I have a function:
def create_variables(name, probabilities, labels):
print('function called')
model = Metrics(probabilities, labels)
prec_curve = model.precision_curve()
kappa_curve = model.kappa_curve()
tpr_curve = model.tpr_curve()
fpr_curve = model.fpr_curve()
pr_auc = auc(tpr_curve, prec_curve)
roc_auc = auc(fpr_curve, tpr_curve)
auk = auc(fpr_curve, kappa_curve)
return [name, prec_curve, kappa_curve, tpr_curve, fpr_curve, pr_auc, roc_auc, auk]
I have the following variables:
svm = pd.read_csv('SVM.csv')
svm_prob_1 = svm.probability[svm.fold_number == 1]
svm_prob_2 = svm.probability[svm.fold_number == 2]
svm_label_1 = svm.true_label[svm.fold_number == 1]
svm_label_2 = svm.true_label[svm.fold_number == 2]
I want to execute the following lines:
svm1 = create_variables('svm_fold1', svm_prob_1, svm_label_1)
svm2 = create_variables('svm_fold2', svm_prob_2, svm_label_2)
Python works as expected for svm1. However, when it starts processing svm2, I receive the following error:
svm2 = create_variables('svm_fold2', svm_prob_2, svm_label_2)
function called
Traceback (most recent call last):
File "<ipython-input-742-702cfac4d100>", line 1, in <module>
svm2 = create_variables('svm_fold2', svm_prob_2, svm_label_2)
File "<ipython-input-741-b8b5a84f0298>", line 6, in create_variables
prec_curve = model.precision_curve()
File "<ipython-input-734-dd9c309be961>", line 59, in precision_curve
self.tp, self.tn, self.fp, self.fn = self.confusion_matrix(self.preds)
File "<ipython-input-734-dd9c309be961>", line 72, in confusion_matrix
if pred == self.labels[i]:
File "C:\Users\20200016\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\series.py", line 1068, in __getitem__
result = self.index.get_value(self, key)
File "C:\Users\20200016\AppData\Local\Continuum\anaconda3\lib\site-packages\pandas\core\indexes\base.py", line 4730, in get_value
return self._engine.get_value(s, k, tz=getattr(series.dtype, "tz", None))
File "pandas\_libs\index.pyx", line 80, in pandas._libs.index.IndexEngine.get_value
File "pandas\_libs\index.pyx", line 88, in pandas._libs.index.IndexEngine.get_value
File "pandas\_libs\index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc
File "pandas\_libs\hashtable_class_helper.pxi", line 992, in pandas._libs.hashtable.Int64HashTable.get_item
File "pandas\_libs\hashtable_class_helper.pxi", line 998, in pandas._libs.hashtable.Int64HashTable.get_item
KeyError: 0
svm_prob_1 and svm_prob_2 are both of the same shape and contain non-zero values. svm_label_2 contains 0's and 1's and has the same length as svm_prob_2.
Furthermore, the error seems to be in svm_label_1. After changing this variable, the following line does work:
svm2 = create_variables('svm_fold2', svm_prob_2, svm_label_1
Based on the code below, there seems to be no difference between svm_label_1 and svm_label_2 though.
type(svm_label_1)
Out[806]: pandas.core.series.Series
type(svm_label_2)
Out[807]: pandas.core.series.Series
min(svm_label_1)
Out[808]: 0
min(svm_label_2)
Out[809]: 0
max(svm_label_1)
Out[810]: 1
max(svm_label_2)
Out[811]: 1
sum(svm_label_1)
Out[812]: 81
sum(svm_label_2)
Out[813]: 89
len(svm_label_1)
Out[814]: 856
len(svm_label_2)
Out[815]: 856
Does anyone know what's going wrong here?
I don't know why it works, but converting svm_label_2 into a list worked:
svm_label_2 = list(svm.true_label[svm.fold_number == 2])
Since, svm_label_1 and svm_label_2 are of the same type, I don't understand why the latter raised an error and the first one did not. Therefore, I still welcome any explanation to this phenomenon.

division by zero when population size is 1 neat-python

Note: I submitted an issue on their GitHub, but looking at their GitHub, but it does not look like anyone will reply based on other issues submitted.
File "C:\Users\hunty\AppData\Local\RLBotGUIX\venv\lib\site-packages\neat\math_util.py", line 9, in mean
return sum(map(float, values)) / len(values)
ZeroDivisionError: division by zero
When the population size of my model is set to 1, I receive a division by 0 error, when set to a number > 1, I don't get the error. I found a stack overflow page relating to this, but there was no solution.
Here is some of my code:
config file (the portion that matters):
[NEAT]
fitness_criterion = max
fitness_threshold = 9999
pop_size = 2
reset_on_extinction = False
eval/fitness function:
def main(gnomes, config):
global net, ge
net = neat.nn.FeedForwardNetwork(gnomes[0], config)
ge = gnomes[0]
ge.fitness = 0
If you have a solution or need more code, please say so!
Whole Traceback:
****** Running generation 0 ******
Process Process-1:
Traceback (most recent call last):
File "C:\Users\hunty\AppData\Local\RLBotGUIX\Python37\lib\multiprocessing\process.py", line 297, in _bootstrap
self.run()
File "C:\Users\hunty\AppData\Local\RLBotGUIX\Python37\lib\multiprocessing\process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "C:\Users\hunty\AppData\Local\RLBotGUIX\venv\lib\site-packages\rlbot\setup_manager.py", line 655, in run_agent
bm.run()
File "C:\Users\hunty\AppData\Local\RLBotGUIX\venv\lib\site-packages\rlbot\botmanager\bot_manager.py", line 196, in run
self.load_agent()
File "C:\Users\hunty\AppData\Local\RLBotGUIX\venv\lib\site-packages\rlbot\botmanager\bot_manager.py", line 117, in load_agent
self.agent = agent_class(self.name, self.team, self.index)
File "C:\Users\hunty\AppData\Local\RLBotGUIX\MyBots\learnie\src\bot.py", line 40, in __init__
run(config_path)
File "C:\Users\hunty\AppData\Local\RLBotGUIX\MyBots\learnie\src\bot.py", line 30, in run
winner = p.run(main)
File "C:\Users\hunty\AppData\Local\RLBotGUIX\venv\lib\site-packages\neat\population.py", line 89, in run
fitness_function(list(iteritems(self.population)), self.config)
File "C:\Users\hunty\AppData\Local\RLBotGUIX\MyBots\learnie\src\bot.py", line 18, in main
net = neat.nn.FeedForwardNetwork(gnomes[0], config)
TypeError: __init__() missing 1 required positional argument: 'node_evals'
uninitialized_bot error reading header: An existing connection was forcibly closed by the remote host

OverflowError when subclassing rv_continuous

import scipy.stats as st
import numpy as np # generic math functions
# https://scicomp.stackexchange.com/q/1658
class LorentzGen(st.rv_continuous):
"""Lorentz distribution"""
def _pdf(self, x):
gamma = 0.27
return 2 * gamma / (np.pi * (gamma ** 2 + x ** 2))
transverse_fields = LorentzGen(a=0)
gaussian_gen = st.norm()
L = 2
list_of_temps = np.linspace(1, 10, 40)
for T, temp in enumerate(list_of_temps):
print(f"Run {T}")
for t in range(5000):
if t%500==0:
print(f"Trial {t}")
h_x = [[-transverse_fields.rvs(), xx] for xx in range(L)] # OverflowError: (34, 'Result too large')
# h_y = [[-gaussian_gen.rvs(), xx] for xx in range(L)] # Works
In the above code, I have implemented my own probability distribution (essentially a half Lorentzian, x∈[0,∞]), modified from the answer from scicomp.SE, which I call transverse_fields.
I need to generate a whole bunch of values from this transverse_fields, using them in a nested For loop. The issue is that beyond a certain number of runs, here "Run 1 Trial ~3500", I get a bunch of errors:
C:\ProgramData\Anaconda3\lib\site-packages\scipy\integrate\quadpack.py:385: IntegrationWarning: The integral is probably divergent, or slowly convergent.
warnings.warn(msg, IntegrationWarning)
C:\ProgramData\Anaconda3\lib\site-packages\numpy\lib\function_base.py:2831: RuntimeWarning: overflow encountered in ? (vectorized)
outputs = ufunc(*inputs)
Traceback (most recent call last):
File "C:/<redacted>/stackoverflow.py", line 26, in <module>
h_x = [[-transverse_fields.rvs(), xx] for xx in range(L)] # Result Overflow
File "C:/<redacted>/stackoverflow.py", line 26, in <listcomp>
h_x = [[-transverse_fields.rvs(), xx] for xx in range(L)] # Result Overflow
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 954, in rvs
vals = self._rvs(*args)
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 889, in _rvs
Y = self._ppf(U, *args)
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 902, in _ppf
return self._ppfvec(q, *args)
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\lib\function_base.py", line 2755, in __call__
return self._vectorize_call(func=func, args=vargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\lib\function_base.py", line 2831, in _vectorize_call
outputs = ufunc(*inputs)
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 1587, in _ppf_single
while self._ppf_to_solve(right, q, *args) < 0.:
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 1569, in _ppf_to_solve
return self.cdf(*(x, )+args)-q
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 1745, in cdf
place(output, cond, self._cdf(*goodargs))
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 1621, in _cdf
return self._cdfvec(x, *args)
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\lib\function_base.py", line 2755, in __call__
return self._vectorize_call(func=func, args=vargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\lib\function_base.py", line 2831, in _vectorize_call
outputs = ufunc(*inputs)
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 1618, in _cdf_single
return integrate.quad(self._pdf, self.a, x, args=args)[0]
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\integrate\quadpack.py", line 341, in quad
points)
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\integrate\quadpack.py", line 448, in _quad
return _quadpack._qagse(func,a,b,args,full_output,epsabs,epsrel,limit)
File "C:/<redacted>/stackoverflow.py", line 11, in _pdf
return 2 * gamma / (np.pi * (gamma ** 2 + x ** 2))
OverflowError: (34, 'Result too large')
Process finished with exit code 1
Note that the error does not occur if I bump the number of trials, here t to a smaller number like 50, nor does it occur if list_of_temps has less values, e.g. np.linspace(1,10,4). Even though in the original problem, with np.linspace(1,10,40), the error popped up during Run 1.
With the original setup, there is also no overflow error when I use the standard Gaussian distribution function from scipy.stats.
Similar issues on SO that I've seen attribute this to the range over which the for loop is run is too big. But I don't quite see that here? And I don't quite understand how to implement Decimal as suggested in the answer in the linked question, in any case.
How can I fix this?
I'm running Python 3.6.5 with Anaconda on 64bit Windows 10.
It seems to be an issue with the way I'm declaring my probability distribution with transverse_fields/LorentzGen.
My solution was to use the in-built cauchy distribution in scipy.stats, with a modified scale.
Also since I wanted a half-Lorentzian, I just took the absolute np.abs(...) when drawing a random number from transverse_fields.
import scipy.stats as st
import numpy as np # generic math functions
transverse_fields = st.cauchy(scale=0.27)
L = 2
list_of_temps = np.linspace(1, 10, 40)
for T, temp in enumerate(list_of_temps):
print(f"Run {T}")
for t in range(5000):
if t%500==0:
print(f"Trial {t}")
h_x = [[-np.abs(transverse_fields.rvs()), xx] for xx in range(L)] # Now works
This is satisfactory enough for me now, but I would still appreciate someone explaining why my way of subclassing rv_continuous gave me the aforementioned errors.

Can't display patches with mplleaflet

I'm trying to display a circle of fixed size in a leaflet map.
I tried with .add_artist()(see here) and I'm now trying with .add_patch()
import mplleaflet
import matplotlib.pyplot as plt
import matplotlib as mpl
fig, ax = plt.subplots()
x = 2.363561
y = 48.951918
r = 20
circle1 = mpl.patches.Circle((x,y), radius=r)
circle2= plt.Circle((x, y), r, color='r')
#ax.add_patch(circle1)
#ax.add_patch(circle2)
mplleaflet.show(fig=ax.figure)
You can try to uncomment one of the two commented lines, it doesn't matter which line is uncommented, same error pops.
Here is the trace I get :
Traceback (most recent call last):
File "<ipython-input-81-1fdd7f4a9d12>", line 16, in <module>
mplleaflet.show(fig = ax.figure)
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\_display.py", line 180, in show
save_html(fig, fileobj=f, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\_display.py", line 131, in save_html
html = fig_to_html(fig, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\_display.py", line 84, in fig_to_html
exporter.run(fig)
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\mplexporter\exporter.py", line 51, in run
self.crawl_fig(fig)
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\mplexporter\exporter.py", line 118, in crawl_fig
self.crawl_ax(ax)
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\mplexporter\exporter.py", line 138, in crawl_ax
self.draw_patch(ax, patch)
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\mplexporter\exporter.py", line 227, in draw_patch
mplobj=patch)
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\leaflet_renderer.py", line 125, in draw_path
rings = list(iter_rings(data, pathcodes))
File "C:\ProgramData\Anaconda3\lib\site-packages\mplleaflet\utils.py", line 14, in iter_rings
raise ValueError('Unrecognized code: {}'.format(code))
ValueError: Unrecognized code: C
Any idea on how to solve this ?
===========================
EDIT :
Just to see what would happen, I tried doing the same thing than this, which is changing
elif code == 'L': ring.append(point)
to
elif code == 'L' or code == 'Z' or code == 'S' or code == 'C': ring.append(point)
in mplleaflet\utils.py, line 11 and 12. Here is what I got :
===========================
EDIT 2 to answer comment
mpl.patches.Rectangle and plt.Rectangle show up :
rect1 = mpl.patches.Rectangle((x,y), 1, 1, color = 'r')
rect2 = plt.Rectangle((x, y), 1, 1, fc='r')
Which seems to fit with a rectangle of 1 degree in latitude and 1 degree longitude (see here) (maybe ? Seems a bit small).
As for Polygons, well... At least they show up, but I guess there is a projection problem :
regpol1 = mpl.patches.RegularPolygon((x,y), 100, radius=r, color = 'r')
cirpol1 = mpl.patches.CirclePolygon((x +50,y), radius=r, color = 'b')

Resources