Hide RandomizedSearchCV output - scikit-learn

I'm executing pylint in the terminal to clean up my python script a bit. In my script, I also use RandomizedSearchCV. What can I do so that pylint's results do not show the different combinations of the RandomizedSearchCV results? Or how can I suppress the output from RandomizedSearchCV?
Here's a snippet of the code in my .py script that causes this issue, and the screenshots of the start/end of what I see when I execute in the terminal.
LOGGER.info("Fine tune model and fit it (Model 2)")
# with warnings.catch_warnings():
# warnings.filterwarnings("ignore")
new_model = RandomizedSearchCV(lr_alt, parameters, cv=4, n_iter=15)
# with warnings.catch_warnings():
# warnings.filterwarnings("ignore")
new_model.fit(train_features_x, train_y)
Can't load images yet, but here's a snippet of the start code in terminal:
(env-stats404-w20-HW5) Franciscos-MacBook-Pro:FRANCISCO-AVALOS franciscoavalosjr$ pylint main.py
************* Module main
main.py:69:0: C0103: Argument name "Product" doesn't conform to snake_case naming style (invalid-name)
main.py:80:4: R1705: Unnecessary "elif" after "return" (no-else-return)
main.py:89:75: W0108: Lambda may not be necessary (unnecessary-lambda)
Traceback (most recent call last):
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/bin/pylint", line 8, in <module>
sys.exit(run_pylint())
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/lib/python3.7/site-packages/pylint/__init__.py", line 23, in run_pylint
PylintRun(sys.argv[1:])
And here's a snippet of the end of that pylint run:
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/lib/python3.7/site-packages/astroid/decorators.py", line 131, in raise_if_nothing_inferred
yield next(generator)
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/lib/python3.7/site-packages/astroid/decorators.py", line 88, in wrapped
if context.push(node):
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/lib/python3.7/site-packages/astroid/context.py", line 92, in push
self.path.add((node, name))
RecursionError: maximum recursion depth exceeded while calling a Python object
(env-stats404-w20-HW5) Franciscos-MacBook-Pro:FRANCISCO-AVALOS franciscoavalosjr$

Turns out python didn't like how I assigned my new column. The fix came from creating a new variable with the new formed column instead of adding it to my dataframe. Before and after code below:
Original code:
# DF_MAJORITY = SUB_DATA[SUB_DATA['balance'] == 0]
# DF_MINORITY = SUB_DATA[SUB_DATA['balance'] == 1]
# NEW_MAJORITY_NUMBER = ((DF_MINORITY.shape[0]/0.075) - DF_MINORITY.shape[0])
# NEW_MAJORITY_NUMBER = int(round(NEW_MAJORITY_NUMBER))
# DF_MAJORITY_DOWNSAMPLED = resample(DF_MAJORITY, replace=False, n_samples=NEW_MAJORITY_NUMBER,
# random_state=29)
# DF_DOWNSAMPLED = pd.concat([DF_MAJORITY_DOWNSAMPLED, DF_MINORITY])
New Code:
BALANCE = SUB_DATA.loc[:, 'Delivery Status'].apply(lambda x: classify_shipping(x))
BALANCE = pd.DataFrame(BALANCE)
DF_MAJORITY = BALANCE[BALANCE['Delivery Status'] == 0]
DF_MINORITY = BALANCE[BALANCE['Delivery Status'] == 1]
NEW_MAJORITY_NUMBER = ((DF_MINORITY.shape[0]/0.075) - DF_MINORITY.shape[0])
NEW_MAJORITY_NUMBER = int(round(NEW_MAJORITY_NUMBER))
DF_MAJORITY_DOWNSAMPLED = resample(DF_MAJORITY, replace=False,
n_samples=NEW_MAJORITY_NUMBER, random_state=29)
DF_DOWNSAMPLED = pd.concat([DF_MAJORITY_DOWNSAMPLED, DF_MINORITY])

Related

How to use multiple heads option in selfAttention class?

I am playing around with Self-attention model from trax library.
when I set n_heads=1, everything works fine. But when I set n_heads=2, my code breaks.
I use only input activations and one SelfAttention layer.
Here is a minimal code:
import trax
import numpy as np
attention = trax.layers.SelfAttention(n_heads=2)
activations = np.random.randint(0, 10, (1, 100, 1)).astype(np.float32)
input = (activations, )
init = attention.init(input)
output = attention(input)
But I have en error:
File [...]/site-packages/jax/linear_util.py, line 166, in call_wrapped
ans = self.f(*args, **dict(self.params, **kwargs))
File [...]/layers/research/efficient_attention.py, line 1637, in forward_unbatched_h
return forward_unbatched(*i_h, weights=w_h, state=s_h)
File [...]/layers/research/efficient_attention.py, line 1175, in forward_unbatched
q_info = kv_info = np.arange(q.shape[-2], dtype=np.int32)
IndexError: tuple index out of range
What I do wrong?

Theano error when using PyMC3: theano.gof.fg.MissingInputError

I am generating some (noisy) data-points (y) with some known parameters (m,c) that represent the equation of a straight line. Using sampling-based Bayesian methods, I now want to know the true values of parameters (m,c) from the data. Therefore, I am using DE Metropolis (PyMC3) to estimate the true parameters.
I am getting theano error theano.gof.fg.MissingInputError: Input 0 of the graph (indices start from 0), used to compute sigmoid(c_interval__), was not provided and not given a value.
Theano version: 1.0.4
PyMC3 version: 3.9.1
import matplotlib.pyplot as plt
import numpy as np
import arviz as az
import pymc3
import theano.tensor as tt
from theano.compile.ops import as_op
plt.style.use("ggplot")
# define a theano Op for our likelihood function
class LogLike(tt.Op):
itypes = [tt.dvector] # expects a vector of parameter values when called
otypes = [tt.dscalar] # outputs a single scalar value (the log likelihood)
def __init__(self, loglike, data, x, sigma):
# add inputs as class attributes
self.likelihood = loglike
self.data = data
self.x = x
self.sigma = sigma
def perform(self, node, inputs, outputs):
# the method that is used when calling the Op
theta, = inputs # this will contain my variables
# call the log-likelihood function
logl = self.likelihood(theta, self.x, self.data, self.sigma)
outputs[0][0] = np.array(logl) # output the log-likelihood
def my_model(theta, x):
y = theta[0]*x + theta[1]
return y
def my_loglike(theta, x, data, sigma):
model = my_model(theta, x)
ll = -(0.5/sigma**2)*np.sum((data - model)**2)
return ll
# set up our data
N = 10 # number of data points
sigma = 1. # standard deviation of noise
x = np.linspace(0., 9., N)
mtrue = 0.4 # true gradient
ctrue = 3. # true y-intercept
truemodel = my_model([mtrue, ctrue], x)
# make data
np.random.seed(716742) # set random seed, so the data is reproducible each time
data = sigma*np.random.randn(N) + truemodel
print(data)
ndraws = 3000 # number of draws from the distribution
# create our Op
logl = LogLike(my_loglike, data, x, sigma)
# use PyMC3 to sampler from log-likelihood
with pymc3.Model():
# uniform priors on m and c
m = pymc3.Uniform('m', lower=-10., upper=10.)
c = pymc3.Uniform('c', lower=-10., upper=10.)
# convert m and c to a tensor vector
theta = tt.as_tensor_variable([m, c])
# use a DensityDist (use a lamdba function to "call" the Op)
pymc3.DensityDist('likelihood', lambda v: logl(v), observed={'v': theta})
step = pymc3.DEMetropolis()
trace = pymc3.sample(ndraws, step)
# plot the traces
axes = az.plot_trace(trace)
fig = axes.ravel()[0].figure
fig.savefig('./trace_plots.png')
Find the full trace here:
Population sampling (4 chains)
DEMetropolis: [c, m]
Attempting to parallelize chains to all cores. You can turn this off with `pm.sample(cores=1)`.
Population parallelization failed. Falling back to sequential stepping of chains.---------------------| 0.00% [0/4 00:00<00:00]
Sampling 4 chains for 0 tune and 4_000 draw iterations (0 + 16_000 draws total) took 5 seconds.███████| 100.00% [4000/4000 00:04<00:00]
Traceback (most recent call last):
File "test.py", line 75, in <module>
trace = pymc3.sample(ndraws, step)
File "/home/csl_user/.local/lib/python3.7/site-packages/pymc3/sampling.py", line 599, in sample
idata = arviz.from_pymc3(trace, **ikwargs)
File "/home/csl_user/.local/lib/python3.7/site-packages/arviz/data/io_pymc3.py", line 531, in from_pymc3
save_warmup=save_warmup,
File "/home/csl_user/.local/lib/python3.7/site-packages/arviz/data/io_pymc3.py", line 159, in __init__
self.observations, self.multi_observations = self.find_observations()
File "/home/csl_user/.local/lib/python3.7/site-packages/arviz/data/io_pymc3.py", line 172, in find_observations
multi_observations[key] = val.eval() if hasattr(val, "eval") else val
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/gof/graph.py", line 522, in eval
self._fn_cache[inputs] = theano.function(inputs, self)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/function.py", line 317, in function
output_keys=output_keys)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/function_module.py", line 1839, in orig_function
name=name)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/function_module.py", line 1487, in __init__
accept_inplace)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/function_module.py", line 181, in std_fgraph
update_mapping=update_mapping)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/gof/fg.py", line 175, in __init__
self.__import_r__(output, reason="init")
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/gof/fg.py", line 346, in __import_r__
self.__import__(variable.owner, reason=reason)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/gof/fg.py", line 391, in __import__
raise MissingInputError(error_msg, variable=r)
theano.gof.fg.MissingInputError: Input 0 of the graph (indices start from 0), used to compute sigmoid(c_interval__), was not provided and not given a value. Use the Theano flag exception_verbosity='high', for more information on this error.
I've run into the same problem when following the example how to sample from a black box likelihood found here:
https://docs.pymc.io/notebooks/blackbox_external_likelihood.html
This seems to be a version problem. I'm on Manjaro Linux and also ran theano 1.0.4 and pymc3 3.9 using python 3.8. I could solve the issue and make the code work by downgrading to python 3.7 and pymc3 3.8. This seems to be in issue with python 3.8, as simply downgrading pymc3 did not solve the issue for me. I am far from an expert in pymc3 so I don't have a solution how to fix this issue using the newest versions, but for now downgrading makes my simulations run.
Hope this helps.
Edit: The devs seem to be aware of this, there is a an open issue on their github page
https://github.com/pymc-devs/pymc3/issues/4002

Using manim to make a graph like network with slightly moving nodes

trying to recreate that by using valuetrackers
Here is my code and the err is also inside that pastebin
Thank you for all the help since I cannot yet create an animation I thought about using this to make by
linking the dots and lines both to some lists of valuetrackers and then finally storing the updation of these value trackers in an animation list and finally playing them was my goal
class Network3(Scene):
def construct(self):
screen_grid = ScreenGrid()
self.add(screen_grid)
#function to just random the x and y coordinates
def randomize_xy():
for i in range(0,no_of_dots):
x_coord[i]=random.randint(-7,+7)
y_coord[i]=random.randint(-4,+4)
print(x_coord)
print(y_coord)
no_of_dots=20
#make some dots
dots=[]
for i in range(0,no_of_dots):
dots.append(Dot())
#make initial list of coordinates
x_coord=[]
y_coord=[]
for i in range(0,no_of_dots):
x_coord.append(random.randint(-7,7))
y_coord.append(random.randint(-4,4))
# #make sure all the lengths are ok
# print(f"length of dots lenght={len(dots)} of x_coord={len(x_coord)}") error of int base 10 is coming from here
print("length of dots=",end="")
print(str(len(dots))) #print(len(dots)) still causes an error !!!
#add all dots at one point
self.add(*dots)
#make anim list to move everything to their position from origin
animlist=[]
for i in range(0,no_of_dots):
animlist.extend([dots[i].move_to,[x_coord[i],y_coord[i],0]])
#play
self.play(*animlist)
#make a list of value trackers
x_trackers=[]
y_trackers=[]
for i in range(0,no_of_dots):
x_tracker=ValueTracker(x_coord[i])
y_tracker=ValueTracker(y_coord[i])
x_trackers.append(x_tracker)
y_trackers.append(y_tracker)
#making lines
lines=[]
for i in range(0,no_of_dots-1):
t1=[x_coord[i],y_coord[i],0]
t2=[x_coord[i+1],y_coord[i+1],0]
line=Line(t1,t2)
lines.append(line)
def update_func1(obj,i):
temp1=np.array([x_trackers[i],y_trackers[i],0])
temp2=np.array([x_trackers[i+1],y_trackers[i+1],0])
line=Line(start=temp1,end=temp2)
obj.become(line)
#making a link between each line and value tracker
for i in range(0,no_of_dots-1):
lines[i].add_updater(lambda m:update_func1(m,i))
#adding again
self.add(*lines)
#making a link between each dot and value tracker
for i in range(0,no_of_dots):
dots[i].add_updater(lambda m:m.move_to([x_trackers[i],y_trackers[i],0]))
#now somehow have to change all the value tracker at once
animlist2=[]
for i in range(0,no_of_dots):
animlist2.extend([x_trackers[i].set_value,random.randint(-7,+7)])
animlist2.extend([y_trackers[i].set_value,random.randint(-4,+4)])
#now play
self.play(*animlist2)
#end
self.wait(3)
Current error I am getting is:
Traceback (most recent call last):
File "C:\manim\manimlib\extract_scene.py", line 155, in main
scene = SceneClass(**scene_kwargs)
File "C:\manim\manimlib\scene\scene.py", line 53, in __init__
self.construct()
File "network_graphic.py", line 584, in construct
lines[i].add_updater(lambda m:update_func1(m,i))
File "C:\manim\manimlib\mobject\mobject.py", line 192, in add_updater
self.update(0)
File "C:\manim\manimlib\mobject\mobject.py", line 159, in update
updater(self)
File "network_graphic.py", line 584, in <lambda>
lines[i].add_updater(lambda m:update_func1(m,i))
File "network_graphic.py", line 578, in update_func1
line=Line(start=temp1,end=temp2)
File "C:\manim\manimlib\mobject\geometry.py", line 431, in __init__
self.set_start_and_end_attrs(start, end)
File "C:\manim\manimlib\mobject\geometry.py", line 471, in set_start_and_end_attrs
vect = normalize(rough_end - rough_start)
TypeError: unsupported operand type(s) for -: 'ValueTracker' and 'ValueTracker'
I think your best option is to use the functions with "dt", then watch my video if you have any doubts. This is a basic example Grant made in one of his very old videos, but I think you can use it for your idea.
class RandomMove(Scene):
CONFIG = {
"amplitude": 0.4,
"jiggles_per_second": 1,
}
def construct(self):
points = VGroup(*[
Dot(radius=0.2) for _ in range(9)
])
points.arrange_in_grid(buff=1)
for submob in points:
submob.jiggling_direction = rotate_vector(
RIGHT, np.random.random() * TAU *1.5,
)
submob.jiggling_phase = np.random.random() * TAU *1.5
def update_mob(mob, dt):
for submob in mob:
submob.jiggling_phase += dt * self.jiggles_per_second * TAU
submob.shift(
self.amplitude *
submob.jiggling_direction *
np.sin(submob.jiggling_phase) * dt
)
points.add_updater(update_mob)
self.add(points)
self.wait(10)
Result here

How do tf.gradients() work?

I am fairly new to tensorflow, i have seen some tutorials but i dont know how tf.gradients() works. if i give it an input of two 2D matrices, how will it compute the partial derivatives? i am really confused ,please help me if you anyone could, it would be of a great help.
import tensorflow as tf
import numpy as np
X = np.random.rand(3,3)
y = np.random.rand(2,2)
grad = tf.gradients(X,y)
with tf.Session() as sess:
sess.run(grad)
print(grad)
this gives an error:
Traceback (most recent call last):
File "C:/Users/Sandeep IPK/PycharmProjects/tests/samples2.py", line 10, in
sess.run(grad)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 767, in run
run_metadata_ptr)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 952, in _run
fetch_handler = _FetchHandler(self._graph, fetches, feed_dict_string)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 408, in init
self._fetch_mapper = _FetchMapper.for_fetch(fetches)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 230, in for_fetch
return _ListFetchMapper(fetch)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 337, in init
self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 337, in
self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 227, in for_fetch
(fetch, type(fetch)))
TypeError: Fetch argument None has invalid type
Process finished with exit code 1
TensorFlow uses reverse accumulation which is based on the chain rule, to compute the gradient value at point. In order to compute gradient of function with respect to a variable you have to define both. Also you have to specify value at which you want to compute the gradient. In this example you compute gradient of y=x**2+x+1 with respect to x at 2:
#!/usr/bin/env python3
import tensorflow as tf
x = tf.Variable(2.0)
y = x**2 + x - 1
grad = tf.gradients(y, x)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
grad_value = sess.run(grad)
print(grad_value)
# output: [5.0]
It is also possible to compute a gradient in case your variable is a matrix. In such case the gradient will be also a matrix. Here we use a simple case when the function depends on the sum of all matrix elements:
#!/usr/bin/env python3
import tensorflow as tf
X = tf.Variable(tf.random_normal([3, 3]))
X_sum = tf.reduce_sum(X)
y = X_sum**2 + X_sum - 1
grad = tf.gradients(y, X)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
grad_value = sess.run(grad)
print(grad_value)
# output: [array([[ 9.6220665, 9.6220665, 9.6220665],
# [ 9.6220665, 9.6220665, 9.6220665],
# [ 9.6220665, 9.6220665, 9.6220665]], dtype=float32)]

How to use a tflearn trained model in an application?

I am currently trying use a trained model in an application.
I've been using this code to generate US city names with an LSTM model. The code works fine and I do manage to get city names.
Right now, I am trying to save the model so I can load it in a different application without training the model again.
Here is the code of my basic application :
from __future__ import absolute_import, division, print_function
import os
from six import moves
import ssl
import tflearn
from tflearn.data_utils import *
path = "US_cities.txt"
maxlen = 20
X, Y, char_idx = textfile_to_semi_redundant_sequences(
path, seq_maxlen=maxlen, redun_step=3)
# --- Create LSTM model
g = tflearn.input_data(shape=[None, maxlen, len(char_idx)])
g = tflearn.lstm(g, 512, return_seq=True, name="lstm1")
g = tflearn.dropout(g, 0.5, name='dropout1')
g = tflearn.lstm(g, 512, name='lstm2')
g = tflearn.dropout(g, 0.5, name='dropout')
g = tflearn.fully_connected(g, len(char_idx), activation='softmax', name='fc')
g = tflearn.regression(g, optimizer='adam', loss='categorical_crossentropy',
learning_rate=0.001)
# --- Initializing model and loading
model = tflearn.models.generator.SequenceGenerator(g, char_idx)
model.load('myModel.tfl')
print("Model is now loaded !")
#
# Main Application
#
while(True):
user_choice = input("Do you want to generate a U.S. city names ? [y/n]")
if user_choice == 'y':
seed = random_sequence_from_textfile(path, 20)
print("-- Test with temperature of 1.5 --")
model.generate(20, temperature=1.5, seq_seed=seed, display=True)
else:
exit()
And here is what I get as an output :
Do you want to generate a U.S. city names ? [y/n]y
-- Test with temperature of 1.5 --
rk
Orange Park AcresTraceback (most recent call last):
File "App.py", line 46, in <module>
model.generate(20, temperature=1.5, seq_seed=seed, display=True)
File "/usr/local/lib/python3.5/dist-packages/tflearn/models/generator.py", line 216, in generate
preds = self._predict(x)[0]
File "/usr/local/lib/python3.5/dist-packages/tflearn/models/generator.py", line 180, in _predict
return self.predictor.predict(feed_dict)
File "/usr/local/lib/python3.5/dist-packages/tflearn/helpers/evaluator.py", line 69, in predict
o_pred = self.session.run(output, feed_dict=feed_dict).tolist()
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 717, in run
run_metadata_ptr)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 894, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 25, 61) for Tensor 'InputData/X:0', which has shape '(?, 20, 61)'
Unfortunately, I can't see why the shape has changed when using generate() in my app. Could anyone help me solve this problem?
Thank you in advance
William
SOLVED?
One solution would be to simply add "modes" to the python script thanks to the argument parser :
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("mode", help="Train or/and test", nargs='+', choices=["train","test"])
args = parser.parse_args()
And then
if args.mode == "train":
# define your model
# train the model
model.save('my_model.tflearn')
if args.mode == "test":
model.load('my_model.tflearn')
# do whatever you want with your model
I dont really understand why this works and why when you're trying to load a model from a different script it doesn't.
But I guess this should be fine for the moment...

Resources