I am fairly new to tensorflow, i have seen some tutorials but i dont know how tf.gradients() works. if i give it an input of two 2D matrices, how will it compute the partial derivatives? i am really confused ,please help me if you anyone could, it would be of a great help.
import tensorflow as tf
import numpy as np
X = np.random.rand(3,3)
y = np.random.rand(2,2)
grad = tf.gradients(X,y)
with tf.Session() as sess:
sess.run(grad)
print(grad)
this gives an error:
Traceback (most recent call last):
File "C:/Users/Sandeep IPK/PycharmProjects/tests/samples2.py", line 10, in
sess.run(grad)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 767, in run
run_metadata_ptr)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 952, in _run
fetch_handler = _FetchHandler(self._graph, fetches, feed_dict_string)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 408, in init
self._fetch_mapper = _FetchMapper.for_fetch(fetches)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 230, in for_fetch
return _ListFetchMapper(fetch)
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 337, in init
self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 337, in
self._mappers = [_FetchMapper.for_fetch(fetch) for fetch in fetches]
File "C:\Users\Sandeep IPK\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\client\session.py", line 227, in for_fetch
(fetch, type(fetch)))
TypeError: Fetch argument None has invalid type
Process finished with exit code 1
TensorFlow uses reverse accumulation which is based on the chain rule, to compute the gradient value at point. In order to compute gradient of function with respect to a variable you have to define both. Also you have to specify value at which you want to compute the gradient. In this example you compute gradient of y=x**2+x+1 with respect to x at 2:
#!/usr/bin/env python3
import tensorflow as tf
x = tf.Variable(2.0)
y = x**2 + x - 1
grad = tf.gradients(y, x)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
grad_value = sess.run(grad)
print(grad_value)
# output: [5.0]
It is also possible to compute a gradient in case your variable is a matrix. In such case the gradient will be also a matrix. Here we use a simple case when the function depends on the sum of all matrix elements:
#!/usr/bin/env python3
import tensorflow as tf
X = tf.Variable(tf.random_normal([3, 3]))
X_sum = tf.reduce_sum(X)
y = X_sum**2 + X_sum - 1
grad = tf.gradients(y, X)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
grad_value = sess.run(grad)
print(grad_value)
# output: [array([[ 9.6220665, 9.6220665, 9.6220665],
# [ 9.6220665, 9.6220665, 9.6220665],
# [ 9.6220665, 9.6220665, 9.6220665]], dtype=float32)]
Related
With TF 2.0.0, I can train an architecture with one input, I can train an architecture with one input using a custom generator, and I can train an architecture with two inputs. But I can't train an architecture with two inputs using a custom generator.
To keep it minimalist, here's a simple example, with no generator and no multiple inputs to start with:
from tensorflow.keras import layers, models, Model, Input, losses
from numpy import random, array, zeros
input1 = Input(shape=2)
dense1 = layers.Dense(5)(input1)
fullModel = Model(inputs=input1, outputs=dense1)
fullModel.summary()
# Generate random examples:
nbSamples = 21
X_train = random.rand(nbSamples, 2)
Y_train = random.rand(nbSamples, 5)
batchSize = 4
fullModel.compile(loss=losses.LogCosh())
fullModel.fit(X_train, Y_train, epochs=10, batch_size=batchSize)
It's a simple dense layer which takes in input vectors of size 2. The randomly generated dataset contains 21 examples and the batch size is 4. Instead of loading all the data and giving them to model.fit(), we can also give a custom generator in input. The main advantage (for RAM consumption) of this is to load only batch by batch rather that the whole dataset. Here is a simple example with the previous architecture and a custom generator:
import json
# Save the last dataset in a file:
with open("./dataset1input.txt", 'w') as file:
for i in range(nbSamples):
example = {"x": X_train[i].tolist(), "y": Y_train[i].tolist()}
file.write(json.dumps(example) + "\n")
def generator1input(datasetPath, batch_size, inputSize, outputSize):
X_batch = zeros((batch_size, inputSize))
Y_batch = zeros((batch_size, outputSize))
i=0
while True:
with open(datasetPath, 'r') as file:
for line in file:
example = json.loads(line)
X_batch[i] = array(example["x"])
Y_batch[i] = array(example["y"])
i+=1
if i % batch_size == 0:
yield (X_batch, Y_batch)
i=0
fullModel.compile(loss=losses.LogCosh())
my_generator = generator1input("./dataset1input.txt", batchSize, 2, 5)
fullModel.fit(my_generator, epochs=10, steps_per_epoch=int(nbSamples/batchSize))
Here, the generator opens the dataset file, but loads only batch_size examples (not nbSamples examples) each time it is called and slides into the file while looping.
Now, I can build a simple functional architecture with 2 inputs, and no generator:
input1 = Input(shape=2)
dense1 = layers.Dense(5)(input1)
subModel1 = Model(inputs=input1, outputs=dense1)
input2 = Input(shape=3)
dense2 = layers.Dense(5)(input2)
subModel2 = Model(inputs=input2, outputs=dense2)
averageLayer = layers.average([subModel1.output, subModel2.output])
fullModel = Model(inputs=[input1, input2], outputs=averageLayer)
fullModel.summary()
# Generate random examples:
nbSamples = 21
X1 = random.rand(nbSamples, 2)
X2 = random.rand(nbSamples, 3)
Y = random.rand(nbSamples, 5)
fullModel.compile(loss=losses.LogCosh())
fullModel.fit([X1, X2], Y, epochs=10, batch_size=batchSize)
Until here, all models compile and run, but I'm not able to use a generator with the last architecture and its 2 inputs... By trying the following code (which should logically work in my opinion):
# Save data in a file:
with open("./dataset.txt", 'w') as file:
for i in range(nbSamples):
example = {"x1": X1[i].tolist(), "x2": X2[i].tolist(), "y": Y[i].tolist()}
file.write(json.dumps(example) + "\n")
def generator(datasetPath, batch_size, inputSize1, inputSize2, outputSize):
X1_batch = zeros((batch_size, inputSize1))
X2_batch = zeros((batch_size, inputSize2))
Y_batch = zeros((batch_size, outputSize))
i=0
while True:
with open(datasetPath, 'r') as file:
for line in file:
example = json.loads(line)
X1_batch[i] = array(example["x1"])
X2_batch[i] = array(example["x2"])
Y_batch[i] = array(example["y"])
i+=1
if i % batch_size == 0:
yield ([X1_batch, X2_batch], Y_batch)
i=0
fullModel.compile(loss=losses.LogCosh())
my_generator = generator("./dataset.txt", batchSize, 2, 3, 5)
fullModel.fit(my_generator, epochs=10, steps_per_epoch=(nbSamples//batchSize))
I obtain the following error:
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 729, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 224, in fit
distribution_strategy=strategy)
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 547, in _process_training_inputs
use_multiprocessing=use_multiprocessing)
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 606, in _process_inputs
use_multiprocessing=use_multiprocessing)
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\keras\engine\data_adapter.py", line 566, in __init__
reassemble, nested_dtypes, output_shapes=nested_shape)
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\data\ops\dataset_ops.py", line 540, in from_generator
output_types, tensor_shape.as_shape, output_shapes)
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\data\util\nest.py", line 471, in map_structure_up_to
results = [func(*tensors) for tensors in zip(*all_flattened_up_to)]
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\data\util\nest.py", line 471, in <listcomp>
results = [func(*tensors) for tensors in zip(*all_flattened_up_to)]
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py", line 1216, in as_shape
return TensorShape(shape)
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py", line 776, in __init__
self._dims = [as_dimension(d) for d in dims_iter]
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py", line 776, in <listcomp>
self._dims = [as_dimension(d) for d in dims_iter]
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py", line 718, in as_dimension
return Dimension(value)
File "C:\Anaconda\lib\site-packages\tensorflow_core\python\framework\tensor_shape.py", line 193, in __init__
self._value = int(value)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'tuple'
As explain in the doc, x argument of model.fit() can be A generator or keras.utils.Sequence returning (inputs, targets), and The iterator should return a tuple of length 1, 2, or 3, where the optional second and third elements will be used for y and sample_weight respectively. Thus, I think that it can not take in input more than one generator. Perhaps multiple inputs are not possible with custom generator. Please, would you have an explanation? A solution?
(otherwise, it seems possible to go through tf.data.Dataset.from_generator() with a less custom approach, but I have difficulties to understand what to indicate in the output_signature argument)
[EDIT] Thank you for your response #Francis Tang. In fact, it's possible to use a custom generator, but it allowed me to understand that I just had to change the line:
yield ([X1_batch, X2_batch], Y_batch)
To:
yield (X1_batch, X2_batch), Y_batch
Nevertheless, it is indeed perhaps better to use tf.keras.utils.Sequence. But I find it a bit restrictive.
In particular, I understand in the example given (as well as in most of the examples I could find about Sequence) that __init__() is first used to load the full dataset, which is against the interest of the generator.
But maybe it was a particular example about Sequence(), and there is no need to use __init__() like that: you can directly read a file and load the desired batch into the __getitem__().
In this case, it seems to push to browse each time the data file, or else it is necessary to create a file per batch beforehand (not really optimal).
from tensorflow.python.keras.utils.data_utils import Sequence
class generator(Sequence):
def __init__(self,filename,batch_size):
data = pickle.load(open(filename,'rb'))
self.X1 = data['X1']
self.X2 = data['X2']
self.y = data['y']
self.bs = batch_size
def __len__(self):
return (len(self.y) - 1) // self.bs + 1
def __getitem__(self,idx):
start, end = idx * self.bs, (idx+1) * self.bs
return (self.X1[start:end], self.X2[start:end]), self.y[start:end]
You need to write a class using Sequence: https://www.tensorflow.org/api_docs/python/tf/keras/utils/Sequence
I am generating some (noisy) data-points (y) with some known parameters (m,c) that represent the equation of a straight line. Using sampling-based Bayesian methods, I now want to know the true values of parameters (m,c) from the data. Therefore, I am using DE Metropolis (PyMC3) to estimate the true parameters.
I am getting theano error theano.gof.fg.MissingInputError: Input 0 of the graph (indices start from 0), used to compute sigmoid(c_interval__), was not provided and not given a value.
Theano version: 1.0.4
PyMC3 version: 3.9.1
import matplotlib.pyplot as plt
import numpy as np
import arviz as az
import pymc3
import theano.tensor as tt
from theano.compile.ops import as_op
plt.style.use("ggplot")
# define a theano Op for our likelihood function
class LogLike(tt.Op):
itypes = [tt.dvector] # expects a vector of parameter values when called
otypes = [tt.dscalar] # outputs a single scalar value (the log likelihood)
def __init__(self, loglike, data, x, sigma):
# add inputs as class attributes
self.likelihood = loglike
self.data = data
self.x = x
self.sigma = sigma
def perform(self, node, inputs, outputs):
# the method that is used when calling the Op
theta, = inputs # this will contain my variables
# call the log-likelihood function
logl = self.likelihood(theta, self.x, self.data, self.sigma)
outputs[0][0] = np.array(logl) # output the log-likelihood
def my_model(theta, x):
y = theta[0]*x + theta[1]
return y
def my_loglike(theta, x, data, sigma):
model = my_model(theta, x)
ll = -(0.5/sigma**2)*np.sum((data - model)**2)
return ll
# set up our data
N = 10 # number of data points
sigma = 1. # standard deviation of noise
x = np.linspace(0., 9., N)
mtrue = 0.4 # true gradient
ctrue = 3. # true y-intercept
truemodel = my_model([mtrue, ctrue], x)
# make data
np.random.seed(716742) # set random seed, so the data is reproducible each time
data = sigma*np.random.randn(N) + truemodel
print(data)
ndraws = 3000 # number of draws from the distribution
# create our Op
logl = LogLike(my_loglike, data, x, sigma)
# use PyMC3 to sampler from log-likelihood
with pymc3.Model():
# uniform priors on m and c
m = pymc3.Uniform('m', lower=-10., upper=10.)
c = pymc3.Uniform('c', lower=-10., upper=10.)
# convert m and c to a tensor vector
theta = tt.as_tensor_variable([m, c])
# use a DensityDist (use a lamdba function to "call" the Op)
pymc3.DensityDist('likelihood', lambda v: logl(v), observed={'v': theta})
step = pymc3.DEMetropolis()
trace = pymc3.sample(ndraws, step)
# plot the traces
axes = az.plot_trace(trace)
fig = axes.ravel()[0].figure
fig.savefig('./trace_plots.png')
Find the full trace here:
Population sampling (4 chains)
DEMetropolis: [c, m]
Attempting to parallelize chains to all cores. You can turn this off with `pm.sample(cores=1)`.
Population parallelization failed. Falling back to sequential stepping of chains.---------------------| 0.00% [0/4 00:00<00:00]
Sampling 4 chains for 0 tune and 4_000 draw iterations (0 + 16_000 draws total) took 5 seconds.███████| 100.00% [4000/4000 00:04<00:00]
Traceback (most recent call last):
File "test.py", line 75, in <module>
trace = pymc3.sample(ndraws, step)
File "/home/csl_user/.local/lib/python3.7/site-packages/pymc3/sampling.py", line 599, in sample
idata = arviz.from_pymc3(trace, **ikwargs)
File "/home/csl_user/.local/lib/python3.7/site-packages/arviz/data/io_pymc3.py", line 531, in from_pymc3
save_warmup=save_warmup,
File "/home/csl_user/.local/lib/python3.7/site-packages/arviz/data/io_pymc3.py", line 159, in __init__
self.observations, self.multi_observations = self.find_observations()
File "/home/csl_user/.local/lib/python3.7/site-packages/arviz/data/io_pymc3.py", line 172, in find_observations
multi_observations[key] = val.eval() if hasattr(val, "eval") else val
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/gof/graph.py", line 522, in eval
self._fn_cache[inputs] = theano.function(inputs, self)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/function.py", line 317, in function
output_keys=output_keys)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/function_module.py", line 1839, in orig_function
name=name)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/function_module.py", line 1487, in __init__
accept_inplace)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/compile/function_module.py", line 181, in std_fgraph
update_mapping=update_mapping)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/gof/fg.py", line 175, in __init__
self.__import_r__(output, reason="init")
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/gof/fg.py", line 346, in __import_r__
self.__import__(variable.owner, reason=reason)
File "/home/csl_user/.local/lib/python3.7/site-packages/theano/gof/fg.py", line 391, in __import__
raise MissingInputError(error_msg, variable=r)
theano.gof.fg.MissingInputError: Input 0 of the graph (indices start from 0), used to compute sigmoid(c_interval__), was not provided and not given a value. Use the Theano flag exception_verbosity='high', for more information on this error.
I've run into the same problem when following the example how to sample from a black box likelihood found here:
https://docs.pymc.io/notebooks/blackbox_external_likelihood.html
This seems to be a version problem. I'm on Manjaro Linux and also ran theano 1.0.4 and pymc3 3.9 using python 3.8. I could solve the issue and make the code work by downgrading to python 3.7 and pymc3 3.8. This seems to be in issue with python 3.8, as simply downgrading pymc3 did not solve the issue for me. I am far from an expert in pymc3 so I don't have a solution how to fix this issue using the newest versions, but for now downgrading makes my simulations run.
Hope this helps.
Edit: The devs seem to be aware of this, there is a an open issue on their github page
https://github.com/pymc-devs/pymc3/issues/4002
I'm executing pylint in the terminal to clean up my python script a bit. In my script, I also use RandomizedSearchCV. What can I do so that pylint's results do not show the different combinations of the RandomizedSearchCV results? Or how can I suppress the output from RandomizedSearchCV?
Here's a snippet of the code in my .py script that causes this issue, and the screenshots of the start/end of what I see when I execute in the terminal.
LOGGER.info("Fine tune model and fit it (Model 2)")
# with warnings.catch_warnings():
# warnings.filterwarnings("ignore")
new_model = RandomizedSearchCV(lr_alt, parameters, cv=4, n_iter=15)
# with warnings.catch_warnings():
# warnings.filterwarnings("ignore")
new_model.fit(train_features_x, train_y)
Can't load images yet, but here's a snippet of the start code in terminal:
(env-stats404-w20-HW5) Franciscos-MacBook-Pro:FRANCISCO-AVALOS franciscoavalosjr$ pylint main.py
************* Module main
main.py:69:0: C0103: Argument name "Product" doesn't conform to snake_case naming style (invalid-name)
main.py:80:4: R1705: Unnecessary "elif" after "return" (no-else-return)
main.py:89:75: W0108: Lambda may not be necessary (unnecessary-lambda)
Traceback (most recent call last):
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/bin/pylint", line 8, in <module>
sys.exit(run_pylint())
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/lib/python3.7/site-packages/pylint/__init__.py", line 23, in run_pylint
PylintRun(sys.argv[1:])
And here's a snippet of the end of that pylint run:
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/lib/python3.7/site-packages/astroid/decorators.py", line 131, in raise_if_nothing_inferred
yield next(generator)
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/lib/python3.7/site-packages/astroid/decorators.py", line 88, in wrapped
if context.push(node):
File "/Users/franciscoavalosjr/opt/anaconda3/envs/env-stats404-w20-HW5/lib/python3.7/site-packages/astroid/context.py", line 92, in push
self.path.add((node, name))
RecursionError: maximum recursion depth exceeded while calling a Python object
(env-stats404-w20-HW5) Franciscos-MacBook-Pro:FRANCISCO-AVALOS franciscoavalosjr$
Turns out python didn't like how I assigned my new column. The fix came from creating a new variable with the new formed column instead of adding it to my dataframe. Before and after code below:
Original code:
# DF_MAJORITY = SUB_DATA[SUB_DATA['balance'] == 0]
# DF_MINORITY = SUB_DATA[SUB_DATA['balance'] == 1]
# NEW_MAJORITY_NUMBER = ((DF_MINORITY.shape[0]/0.075) - DF_MINORITY.shape[0])
# NEW_MAJORITY_NUMBER = int(round(NEW_MAJORITY_NUMBER))
# DF_MAJORITY_DOWNSAMPLED = resample(DF_MAJORITY, replace=False, n_samples=NEW_MAJORITY_NUMBER,
# random_state=29)
# DF_DOWNSAMPLED = pd.concat([DF_MAJORITY_DOWNSAMPLED, DF_MINORITY])
New Code:
BALANCE = SUB_DATA.loc[:, 'Delivery Status'].apply(lambda x: classify_shipping(x))
BALANCE = pd.DataFrame(BALANCE)
DF_MAJORITY = BALANCE[BALANCE['Delivery Status'] == 0]
DF_MINORITY = BALANCE[BALANCE['Delivery Status'] == 1]
NEW_MAJORITY_NUMBER = ((DF_MINORITY.shape[0]/0.075) - DF_MINORITY.shape[0])
NEW_MAJORITY_NUMBER = int(round(NEW_MAJORITY_NUMBER))
DF_MAJORITY_DOWNSAMPLED = resample(DF_MAJORITY, replace=False,
n_samples=NEW_MAJORITY_NUMBER, random_state=29)
DF_DOWNSAMPLED = pd.concat([DF_MAJORITY_DOWNSAMPLED, DF_MINORITY])
I'm trying to plot a Random Forest visualization for classification purposes with python 3.
Firstly, I read a CSV file where all necesary data is located. Here, Read_CSV() is a method who run correctly, giving three variables, features (vector with all feature names, specifically 45), data (only the data without label column. There are 148000 rows and 45 columns), labels (column of labels in integer format. There are 3 classes to classify as integers 0, 1 or 2. There are also 148000 rows in this vector).
features,data,labels = Read_CSV()
X_train,X_test,Y_train,Y_test = train_test_split(data,labels,test_size=0.35,random_state=0)
X = np.array(X).astype(np.float)
y = np.array(y).astype(np.float)
ax = ax or plt.gca()
ax.scatter(X[:, 0], X[:, 1], c=y, s=30, cmap=cmap,
clim=(y.min(), y.max()), zorder=3)
ax.axis('tight')
ax.axis('off')
xlim = ax.get_xlim()
ylim = ax.get_ylim()
# fit the estimator
model.fit(X, y)
xx, yy = np.meshgrid(np.linspace(*xlim, num=200),
np.linspace(*ylim, num=200))
Z = model.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)
# Create a color plot with the results
n_classes = len(np.unique(y))
contours = ax.contourf(xx, yy, Z, alpha=0.3,
levels=np.arange(n_classes + 1) - 0.5,
cmap=cmap, clim=(y.min(), y.max()),
zorder=1)
ax.set(xlim=xlim, ylim=ylim)
This part of the code showed here is completely dedicated to obtain a plot like this:
enter image description here
When I run this code I obtain the following:
Traceback (most recent call last):
File "C:/Users/Carles/PycharmProjects/Article/main.py", line 441, in <module>
main()
File "C:/Users/Carles/PycharmProjects/Article/main.py", line 388, in main
visualize_classifier(RandomForestClassifier(),X_train, Y_train)
File "C:/Users/Carles/PycharmProjects/Article/main.py", line 353, in visualize_classifier
Z = model.predict(np.c_[xx.ravel(), yy.ravel()]).reshape(xx.shape)
File "C:\Users\Carles\PycharmProjects\Article\venv\lib\site-packages\sklearn\ensemble\forest.py", line 538, in predict
proba = self.predict_proba(X)
File "C:\Users\Carles\PycharmProjects\Article\venv\lib\site-packages\sklearn\ensemble\forest.py", line 578, in predict_proba
X = self._validate_X_predict(X)
File "C:\Users\Carles\PycharmProjects\Article\venv\lib\site-packages\sklearn\ensemble\forest.py", line 357, in _validate_X_predict
return self.estimators_[0]._validate_X_predict(X, check_input=True)
File "C:\Users\Carles\PycharmProjects\Article\venv\lib\site-packages\sklearn\tree\tree.py", line 384, in _validate_X_predict
% (self.n_features_, n_features))
ValueError: Number of features of the model must match the input. Model n_features is 45 and input n_features is 2
At the very start of declaring the Session with tf.Session(), I declare both tf.global_variables_initializer and tf.local_variables_initializer functions and unfortunately, keep receiving error messages detailing the use of "Uninitialized value Variable_1." Why?
I did some searching around and found this StackExchange Question, but the answer doesn't help my situation. So I looked through the TensorFlow API and found an operation that should return any uninitialized variables, tf.report_uninitialized_variables(). I printed the results and received an empty pair of square brackets, which doesn't make any sense considering the description of my error messages. So what's going on? I've been clawing my eyes out for a day now. Any help is appreciated.
import tensorflow as tf
import os
from tqdm import tqdm
#hyperparam
training_iterations = 100
PATH = "C:\\Users\\ratno\\Desktop\\honest chaos\\skin cam\\drive-download-20180205T055458Z-001"
#==================================import training_data=================================
def import_data(image_path):
image_contents = tf.read_file(filename=image_path)
modified_image = tf.image.decode_jpeg(contents=image_contents, channels=1)
image_tensor = tf.cast(tf.reshape(modified_image, [1, 10000]), dtype=tf.float32)
return image_tensor
#========================neural network================================
def neural_network(input_layer):
Weight_net_1 = {'weights': tf.Variable(tf.random_normal(shape=(10000, 16))),
'bias': tf.Variable(tf.random_normal(shape=(1, 1)))}
Weight_net_2 = {'weights': tf.Variable(tf.random_normal(shape=(16, 16))),
'bias': tf.Variable(tf.random_normal(shape=(1, 1)))}
Weight_net_3 = {'weights': tf.Variable(tf.random_normal(shape=(16, 16))),
'bias': tf.Variable(tf.random_normal(shape=(1, 1)))}
Weight_net_4 = {'weights': tf.Variable(tf.random_normal(shape=(16, 1))),
'bias': tf.Variable(tf.random_normal(shape=(1, 1)))}
#Input Layer
hypothesis = input_layer; x = hypothesis
#Hidden Layer 1
hypothesis = tf.nn.relu(tf.matmul(x, Weight_net_1['weights']) + Weight_net_1['bias']); x = hypothesis
#Hidden Layer 2
hypothesis = tf.nn.relu(tf.matmul(x, Weight_net_2['weights']) + Weight_net_2['bias']); x = hypothesis
#Hidden Layer 3
hypothesis = tf.nn.relu(tf.matmul(x, Weight_net_3['weights']) + Weight_net_3['bias']); x = hypothesis
# output cell
hypothesis = tf.nn.relu(tf.matmul(x, Weight_net_4['weights']) + Weight_net_4['bias'])
return hypothesis
#============================training the network=========================
def train(hypothesis):
LOSS = tf.reduce_sum(1 - hypothesis)
tf.train.AdamOptimizer(0.01).minimize(LOSS)
#Session==================================================================
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(tf.local_variables_initializer())
image_list = [os.path.join(PATH, file_name) for file_name in os.listdir(PATH)]
for iteration in tqdm(range(training_iterations), desc="COMPLETION", ncols=80):
for i in image_list:
modified_image_tensor = sess.run(import_data(image_path=i))
hypo = sess.run(neural_network(input_layer=modified_image_tensor))
sess.run(train(hypothesis=hypo))
print("\n\nTraining completed.\nRunning test prediction.\n")
DIRECTORY = input("Directory: ")
test_input = sess.run(import_data(DIRECTORY))
prediction = sess.run(neural_network(input_layer=test_input))
print(prediction)
if prediction >= 0.5:
print ("Acne")
else:
print ("What")
And as for the error message:
Caused by op 'Variable/read', defined at:
File "C:/Users/ratno/Desktop/honest chaos/Hotdog/HDogntoHDog.py", line 75, in <module>
hypo = sess.run(neural_network(input_layer=modified_image_tensor))
File "C:/Users/ratno/Desktop/honest chaos/Hotdog/HDogntoHDog.py", line 23, in neural_network
Weight_net_1 = {'weights': tf.Variable(tf.random_normal(shape=(10000, 16))),
File "C:\Users\ratno\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\ops\variables.py", line 199, in __init__
expected_shape=expected_shape)
File "C:\Users\ratno\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\ops\variables.py", line 330, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "C:\Users\ratno\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\ops\gen_array_ops.py", line 1400, in identity
result = _op_def_lib.apply_op("Identity", input=input, name=name)
File "C:\Users\ratno\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\op_def_library.py", line 767, in apply_op
op_def=op_def)
File "C:\Users\ratno\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\ops.py", line 2630, in create_op
original_op=self._default_original_op, op_def=op_def)
File "C:\Users\ratno\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\framework\ops.py", line 1204, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value Variable
[[Node: Variable/read = Identity[T=DT_FLOAT, _class=["loc:#Variable"], _device="/job:localhost/replica:0/task:0/cpu:0"](Variable)]]
Let's take a look at your main function, starting from with tf.Session() as sess:. This will be the first line executed when you run your program. The next thing that happens is that you are calling the variables_initializers -- but, you have not yet declared any variables! This is because you have not called any of the other functions you have defed. So this is why, when you then call, e.g., neural_network inside a sess.run call, it will create the (uninitialized) variables as neural_networkis called, and then attempt to use them for sess.run. Obviously this will not work since you have not initialized these newly-created variables.
You have to create your network and all necessary variables in the computational graph before calling the initializers. You could try something along these lines:
data = import_data(image_path)
out = neural_network(data)
tr = train(hypothesis=out)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
By the way, your function train also has no return value so it is unlikely it will work as you are expecting it to. Please re-read the tutorials for tensorflow to understand how to operate an optimizer.