How to cascade two pretrained networks? - python-3.x

Very new to deep learning, I was researching into cascading multiple neural networks and was wondering how to actually put it into code. For a lesser example, say I imported a vgg and resnet model trained on imagenet. Is there an easy way to do this?
I have tried playing around with it and concatenating the outputs of 3 of these networks, putting them in a dense layer, and completing the model. But, I'm being thrown a error.
model1 = load_model(save_path1)
model2 = load_model(save_path2)
model1_out = model1.output
model2_out = model2.output
concat = tf.keras.layers.Concatenate()([model1_out, model2_out])
concatout = Dense(10, activation='softmax')(concat)
combined = models.Model(inputs=[model1.input, model2.input], outputs=[concatout])
combined.compile(optimizer='adam',
loss="categorical_crossentropy",
metrics=['accuracy'])
test_scores = combined.evaluate(x_test, y_test, verbose=2)
print('Loss for test dataset:', test_scores[0])
print('Accuracy for test dataset:', test_scores[1])
test_scores = combined.evaluate(x_test, y_test, verbose=2)
print('Loss for test dataset:', test_scores[0])
print('Accuracy for test dataset:', test_scores[1])
Error: File
ile "C:\\pythonProject1\backup.py", line 72, in <module>
test_scores = combined.evaluate(x_test, y_test, verbose=2)
File "C:\Users\\miniconda3\envs\tf1\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\\AppData\Local\Temp\__autograph_generated_file1poeqzyb.py", line 15, in tf__test_function
retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
ValueError: in user code:
File "C:\Users\\miniconda3\envs\tf1\lib\site-packages\keras\engine\training.py", line 1557, in test_function *
return step_function(self, iterator)
File "C:\Users\\miniconda3\envs\tf1\lib\site-packages\keras\engine\training.py", line 1546, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\\miniconda3\envs\tf1\lib\site-packages\keras\engine\training.py", line 1535, in run_step **
outputs = model.test_step(data)
File "C:\Users\\miniconda3\envs\tf1\lib\site-packages\keras\engine\training.py", line 1499, in test_step
y_pred = self(x, training=False)
File "C:\Users\\miniconda3\envs\tf1\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\\miniconda3\envs\tf1\lib\site-packages\keras\engine\input_spec.py", line 200, in assert_input_compatibility
raise ValueError(f'Layer "{layer_name}" expects {len(input_spec)} input(s),'
ValueError: Layer "model" expects 2 input(s), but it received 1 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(None, 48, 48, 3) dtype=float32>]

Related

Error while training with Sequential in Keras: Shapes are incompatible

I am training a neural network (2 conv layers and 1 dense hidden layer) to classify hand-sign images for 24 alphabets (J and Z has no images). Using ImageDataGenerator's flow() function to create training and testing data generators. Using Keras Sequential to create the neural network model. While training, I am getting the following error:
model = create_model()
# Train your model
history = model.fit(train_generator,
epochs=15,
validation_data=validation_generator)
Epoch 1/15
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-46-746fb7255d3f> in <module>()
6 history = model.fit(train_generator,
7 epochs=15,
----> 8 validation_data=validation_generator)
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py in autograph_handler(*args, **kwargs)
1145 except Exception as e: # pylint:disable=broad-except
1146 if hasattr(e, "ag_error_metadata"):
-> 1147 raise e.ag_error_metadata.to_exception(e)
1148 else:
1149 raise
ValueError: in user code:
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1021, in train_function *
return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1010, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1000, in run_step **
outputs = model.train_step(data)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 860, in train_step
loss = self.compute_loss(x, y, y_pred, sample_weight)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 919, in compute_loss
y, y_pred, sample_weight, regularization_losses=self.losses)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/compile_utils.py", line 201, in __call__
loss_value = loss_obj(y_t, y_p, sample_weight=sw)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 141, in __call__
losses = call_fn(y_true, y_pred)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 245, in call **
return ag_fn(y_true, y_pred, **self._fn_kwargs)
File "/usr/local/lib/python3.7/dist-packages/keras/losses.py", line 1790, in categorical_crossentropy
y_true, y_pred, from_logits=from_logits, axis=axis)
File "/usr/local/lib/python3.7/dist-packages/keras/backend.py", line 5083, in categorical_crossentropy
target.shape.assert_is_compatible_with(output.shape)
**ValueError: Shapes (None, 1) and (None, 24) are incompatible**
Here is the link to the colab code: 1
You need to use one hot encoding for the y parameters where you define training and validation generator. So under 'train_val_generators' function change:
y=training_labels
into
y=tf.keras.utils.to_categorical(training_labels, x)
and do same thing for the validation. x is the number of output neurons.

ValueError: Dimensions must be equal, but are 100 and 19 with input shapes: [?,100], [?,100,19]

I have an error in my code, and I've done read the documentation but it still error, How this error can be fixed?
Code:
import tensorflow.keras.backend as K
import tensorflow_addons as tfa
from tensorflow_addons.layers import CRF
from keras_crf import CRFModel
def create_model(): #
max_words=length_long_sentence
MAX_SENTENCE_NUM=100
embedding_size=100
lstm_size=128
learn_rate=0.01
output_size=len(unique_tag_set)
current_input=Input(shape=(MAX_SENTENCE_NUM,max_words,))
emb_current = Embedding(vocab_size, embedding_size, weights=
[embedding_matrix],input_length=max_words, name='current_embed',trainable=False)(current_input)
hidden_vectors=TimeDistributed(Bidirectional(LSTM(units=lstm_size, return_sequences=False)))
(emb_current )
hidden_vectors=Bidirectional(LSTM(units=lstm_size, return_sequences=True))(hidden_vectors )
base = tf.keras.Model(inputs=current_input, outputs=hidden_vectors)
model = CRFModel(base, 19)
opt = tf.keras.optimizers.Adam(learning_rate=learn_rate)
model.compile(optimizer=opt, metrics=['acc'])
print(model.summary())
return model
model_2=create_model()
and here is the model summary:
Here is the code to fit in training data:
history_2=model_2.fit(x_train_split,y_train_split,
epochs=1,batch_size=16,
shuffle = False, verbose = 1,
validation_split=0.2,
sample_weight=sample_weights)
And I got this error:
ValueError: in user code:
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 878, in train_function *
return step_function(self, iterator)
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 867, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 860, in run_step **
outputs = model.train_step(data)
File "/usr/local/lib/python3.7/dist-packages/keras_crf/crf_model.py", line 49, in train_step
crf_loss = -tfa.text.crf_log_likelihood(potentials, y, sequence_length, kernel)[0]
File "/usr/local/lib/python3.7/dist-packages/tensorflow_addons/text/crf.py", line 242, in crf_log_likelihood
inputs, tag_indices, sequence_lengths, transition_params
File "/usr/local/lib/python3.7/dist-packages/tensorflow_addons/text/crf.py", line 104, in crf_sequence_score
return tf.cond(tf.equal(tf.shape(inputs)[1], 1), _single_seq_fn, _multi_seq_fn)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_addons/text/crf.py", line 97, in _multi_seq_fn
unary_scores = crf_unary_score(tag_indices, sequence_lengths, inputs)
File "/usr/local/lib/python3.7/dist-packages/tensorflow_addons/text/crf.py", line 277, in crf_unary_score
flattened_tag_indices = tf.reshape(offsets + tag_indices, [-1])
ValueError: Dimensions must be equal, but are 100 and 19 for '{{node cond/add_1}} = AddV2[T=DT_INT32](cond/add, cond/add_1/Cast)' with input shapes: [?,100], [?,100,19].
This could be because you have 19 classes. But your y vector has digits: 0, ..., 18. Your model is outputting a 19 dimensional vector.
So, try tf.keras.utils.to_categorical. Link: https://www.tensorflow.org/api_docs/python/tf/keras/utils/to_categorical
Essentially:
y_train_split = tf.keras.utils.to_categorical(y_train_split)
# code to fit

GridSearchCV(): ValueError: Input contains NaN, infinity or a value too large for dtype('float64')

I get the ValueError in the title when I try to perform a GridsearchCV on an MLP classifier. Ofcourse I checked if any of np.inf or np.nan exist in my dataset, but they dont:
print(np.any(np.isnan(X)))
returns False
print(np.all(np.isfinite(X)))
Returns True
I also casted all my values to np.float64
X = X.values.astype(np.float64)
Y = Y.values
My scikit-learn version is 0.22.2.post1 (latest)
The code i'm trying to execute:
from scipy.stats import randint as sp_randint
hiddenlayers = [(sp_randint.rvs(100,600,1),sp_randint.rvs(100,600,1),), (sp_randint.rvs(100,600,1),)]
alpha_range = 10.0 ** np.arange(-2, 1)
param_grid_MLP = [{'solver': ['lbfgs'],
'hidden_layer_sizes': hiddenlayers,
'activation': ['identity','tanh', 'relu', 'logistic'],
'alpha': alpha_range
},
{'solver': ['sgd'],
'hidden_layer_sizes': hiddenlayers,
'activation': ['identity','tanh', 'relu', 'logistic'],
'alpha': alpha_range,
'learning_rate':['constant','invscaling','adaptive']
},
{'solver': ['adam'],
'hidden_layer_sizes': hiddenlayers,
'activation': ['identity','tanh', 'relu', 'logistic'],
'alpha': alpha_range
}]
mlp = MLPClassifier(random_state=0)
cross_validation = StratifiedKFold(5)
# scoring = {'AUC': 'roc_auc',
# 'Accuracy': make_scorer(accuracy_score),
# 'Recall':make_scorer(recall_score,pos_label='crafted'),
# 'Precision': make_scorer(precision_score,pos_label='crafted')}
scoring = {'AUC': 'roc_auc',
'Accuracy': make_scorer(accuracy_score),
'Recall':make_scorer(recall_score,pos_label='crafted')}
grid_search_MLP = GridSearchCV(estimator=mlp,
param_grid=param_grid_MLP,
scoring=scoring,cv=cross_validation.split(X_train,y_train),
refit='Recall',
n_jobs=-1,
verbose=True)
grid_search_MLP.fit(X_train,y_train)
print('Best score: {}'.format(grid_search_MLP.best_score_))
print('Best index: {}'.format(grid_search_MLP.best_index_))
print('Best parameters: {}'.format(grid_search_MLP.best_params_))
mlp = grid_search_MLP.best_estimator_
mlp
The full error traceback:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/joblib/externals/loky/process_executor.py", line 418, in _process_worker
r = call_item()
File "/usr/local/lib/python3.7/dist-packages/joblib/externals/loky/process_executor.py", line 272, in __call__
return self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.7/dist-packages/joblib/_parallel_backends.py", line 608, in __call__
return self.func(*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/joblib/parallel.py", line 256, in __call__
for func, args, kwargs in self.items]
File "/usr/local/lib/python3.7/dist-packages/joblib/parallel.py", line 256, in <listcomp>
for func, args, kwargs in self.items]
File "/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py", line 544, in _fit_and_score
test_scores = _score(estimator, X_test, y_test, scorer)
File "/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py", line 591, in _score
scores = scorer(estimator, X_test, y_test)
File "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_scorer.py", line 87, in __call__
*args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_scorer.py", line 332, in _score
return self._sign * self._score_func(y, y_pred, **self._kwargs)
File "/usr/local/lib/python3.7/dist-packages/sklearn/metrics/_ranking.py", line 369, in roc_auc_score
y_score = check_array(y_score, ensure_2d=False)
File "/usr/local/lib/python3.7/dist-packages/sklearn/utils/validation.py", line 578, in check_array
allow_nan=force_all_finite == 'allow-nan')
File "/usr/local/lib/python3.7/dist-packages/sklearn/utils/validation.py", line 60, in _assert_all_finite
msg_dtype if msg_dtype is not None else X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').
It seems to me that you might have a corrupted value in your array, or a non numeric value. Try to check if there are other types in your array, before transforming to float. Try also to find the min and max value in your array, that might help to find the value which raises the error.
try giving verbose a large number, or run that 3 parts of grid one by one. if you realize sgd gives the problem, its probably explained here MLPRegressor error when solver sgd is used

How to fix 'IndexError: tuple index out of range' in model.fit with tensorflow?

I am trying to create a CNN with Tensorflow and keras with Sequential method.
The inputs have a (size, 50, 50, 1) shape and the labels have (size,). Size is the number of data in the dataset.
The problem is, after compilation, when I call the fit method with my model, I get an index error. See the code bellow :
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Conv2D(32, 4, input_shape=(50, 50, 1), activation='relu', name="conv1"))
model.add(tf.keras.layers.Conv2D(64, 3, activation='relu', name="conv2"))
model.add(tf.keras.layers.Conv2D(128, 3, activation='relu', name="conv3"))
model.add(tf.keras.layers.Flatten(name='Flatten'))
model.add(tf.keras.layers.Dense(128, activation='relu', name="d1"))
model.add(tf.keras.layers.Dense(4, activation='softmax', name="output"))
# Compile the model
model.compile(
loss="sparse_categorical_crossentropy",
optimizer="Adam",
metrics=["accuracy"]
)
model.fit(x_trains, y_labels, epochs=5, verbose=2, validation_data=0.33, shuffle=True)
And the error :
2019-09-24 13:59:40.902561: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Traceback (most recent call last):
File "C:/Users/eloim/Documents/Programmation/Python/CNN_tf/face_train_seq.py", line 49, in <module>
model.fit(x_trains, y_labels, epochs=5, verbose=2, validation_data=0.33, shuffle=True)
File "C:\Users\{}\Anaconda3\envs\CNN_tf\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\{}\Anaconda3\envs\CNN_tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 224, in fit
distribution_strategy=strategy)
File "C:\Users\{}\Anaconda3\envs\CNN_tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 564, in _process_training_inputs
distribution_strategy=distribution_strategy)
File "C:\Users\{}\Anaconda3\envs\CNN_tf\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 606, in _process_inputs
use_multiprocessing=use_multiprocessing)
File "C:\Users\{}\Anaconda3\envs\CNN_tf\lib\site-packages\tensorflow_core\python\keras\engine\data_adapter.py", line 479, in __init__
batch_size=batch_size, shuffle=shuffle, **kwargs)
File "C:\Users\{}\Anaconda3\envs\CNN_tf\lib\site-packages\tensorflow_core\python\keras\engine\data_adapter.py", line 238, in __init__
num_samples = set(int(i.shape[0]) for i in nest.flatten(inputs))
File "C:\Users\{}\Anaconda3\envs\CNN_tf\lib\site-packages\tensorflow_core\python\keras\engine\data_adapter.py", line 238, in <genexpr>
num_samples = set(int(i.shape[0]) for i in nest.flatten(inputs))
IndexError: tuple index out of range
At the line :
model.fit(x_trains, y_labels, epochs=5, verbose=2, validation_data=0.33, shuffle=True)
I do not understand the nature of this error. How can I be rid of it ?
Thanks for your help.

InvalidArguementError tensorflow

I am relatively new to tensorflow and I am working on relation classification. I will list down my problem step wise so that it is clear and hope that someone can point out my mistake( which I am sure must be a silly one):
For the word embedding layer I needed to initialize a tf variable with a tensor which was of size more that 2GB. So I followed the solutions provided here and changed my code.
Code snippets before change :
train.py
if FLAGS.model_type == 'cnn':
with tf.Graph().as_default():
session_conf = tf.ConfigProto(
allow_soft_placement=FLAGS.allow_soft_placement,
log_device_placement=FLAGS.log_device_placement)
sess = tf.Session(config=session_conf)
with sess.as_default():
cnn = textCNN(
sequence_length=x_trains[0].shape[1],
num_classes=num_classes,
vocab_size=len(word_embed_vecs),
embedding_size=FLAGS.embedding_dim,
dist_vocab_size=dist_vocab_size,
dist_size=FLAGS.pos_dim,
filter_sizes=list(map(int,
FLAGS.filter_sizes.split(","))),
num_filters=FLAGS.num_filters,
l2_reg_lambda=FLAGS.l2_reg_lambda,
word_vecs=word_embed_vecs, #word_embed_vecs is of shape
#(2451510, 300)
train_emb=FLAGS.train_emb)
text_cnn.py
class textCNN(object):
"""
A CNN for text classification.
Uses an embedding layer, followed by a convolutional, max-pooling and
softmax layer.
"""
def __init__(
self, sequence_length, num_classes, vocab_size,
embedding_size, dist_vocab_size, dist_size, filter_sizes, num_filters,
l2_reg_lambda=0.0, word_vecs=None, train_emb=True):
# Placeholders for input, output and dropout
self.input_x = tf.placeholder(tf.int32, [None, sequence_length],
name="input_x")
self.e1_dist = tf.placeholder(tf.int32, [None, sequence_length],
name="e1_dist")
self.e2_dist = tf.placeholder(tf.int32, [None, sequence_length],
name="e2_dist")
self.input_y = tf.placeholder(tf.float32, [None, num_classes],
name="input_y")
self.dropout_keep_prob = tf.placeholder(tf.float32,
name="dropout_keep_prob")
l2_loss = tf.constant(0.0)
# Embedding layer
with tf.device('/gpu:0'), tf.name_scope("embedding"):
#with tf.name_scope("embedding"):
self.W = tf.Variable(
trainable=train_emb,
initial_value=tf.constant(word_vecs, dtype=tf.float32), name="W")
self.embedded_chars = tf.nn.embedding_lookup(self.W, self.input_x)
self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars,
-1)
Code snippets after change:
train.py
if FLAGS.model_type == 'cnn':
with tf.Graph().as_default():
session_conf = tf.ConfigProto(
allow_soft_placement=FLAGS.allow_soft_placement,
log_device_placement=FLAGS.log_device_placement)
sess = tf.Session(config=session_conf)
with sess.as_default():
cnn = textCNN(
sequence_length=x_trains[0].shape[1],
num_classes=num_classes,
vocab_size=len(word_embed_vecs),
embedding_size=FLAGS.embedding_dim,
dist_vocab_size=dist_vocab_size,
dist_size=FLAGS.pos_dim,
filter_sizes=list(map(int,
FLAGS.filter_sizes.split(","))),
num_filters=FLAGS.num_filters,
l2_reg_lambda=FLAGS.l2_reg_lambda,
**# word_vecs=word_embed_vecs, #word_embed_vecs is of
shape (2451510, 300)**
train_emb=FLAGS.train_emb)
# Generate batches
dev_f1s = []
test_f1s = []
for fi in range(len(x_trains)):
sess.run(tf.global_variables_initializer())
batches = data_helpers.batch_iter(
list(zip(x_trains[fi], train_e1_dists[fi], train_e2_dists[fi],
y_trains[fi])), FLAGS.batch_size, FLAGS.embedding_dim,
FLAGS.num_epochs)
# Training loop. For each batch...
evaluate_every = len(x_trains[fi]) / FLAGS.batch_size
for batch in batches:
x_batch, e1_dist, e2_dist, y_batch = zip(*batch)
# Train
feed_dict = {
cnn.input_x: x_batch,
cnn.e1_dist: e1_dist,
cnn.e2_dist: e2_dist,
cnn.input_y: y_batch,
cnn.dropout_keep_prob: FLAGS.dropout_keep_prob,
cnn.wordvecs: word_embed_vecs
}
_, step, summaries, loss, accuracy = sess.run(
[train_op, global_step, train_summary_op, cnn.loss,
cnn.accuracy], feed_dict)
text_cnn.py
class textCNN(object):
"""
A CNN for text classification.
Uses an embedding layer, followed by a convolutional, max-pooling and
softmax layer.
"""
def __init__(
self, sequence_length, num_classes, vocab_size,
embedding_size, dist_vocab_size, dist_size, filter_sizes, num_filters,
l2_reg_lambda=0.0, train_emb=True):
# Placeholders for input, output and dropout
self.input_x = tf.placeholder(tf.int32, [None, sequence_length],
name="input_x")
self.e1_dist = tf.placeholder(tf.int32, [None, sequence_length],
name="e1_dist")
self.e2_dist = tf.placeholder(tf.int32, [None, sequence_length],
name="e2_dist")
self.input_y = tf.placeholder(tf.float32, [None, num_classes],
name="input_y")
self.dropout_keep_prob = tf.placeholder(tf.float32,
name="dropout_keep_prob")
self.wordvecs = tf.placeholder(tf.float32, shape = (2451510, 300),
name = "wordvecs")
l2_loss = tf.constant(0.0)
# Embedding layer
with tf.device('/gpu:0'), tf.name_scope("embedding"):
#with tf.name_scope("embedding"):
#self.W = tf.Variable(
# trainable=False,
# initial_value=tf.constant(word_vecs, dtype=tf.float32),
# name="W")
self.WordVecs = tf.Variable(trainable = False,
initial_value=self.wordvecs, name="WordVecs")
self.embedded_chars = tf.nn.embedding_lookup(self.WordVecs,
self.input_x)
self.embedded_chars_expanded = tf.expand_dims(self.embedded_chars,
-1)
So basically I added a placeholder of the shape of my embedding vectors and passed it in feeddict while calling sess.run. however I am getting InvalidArguementError. The traceback details are as below:
Traceback (most recent call last):
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/framework/errors_impl.py", line 473, in
__exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must
feed a value for placeholder tensor 'wordvecs' with dtype float and shape
[2451510,300]
[[Node: wordvecs = Placeholder[dtype=DT_FLOAT, shape=[2451510,300],
_device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "train.py", line 470, in <module>
tf.app.run()
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "train.py", line 466, in main
train()
File "train.py", line 407, in train
sess.run(tf.global_variables_initializer())
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must
feed a value for placeholder tensor 'wordvecs' with dtype float and shape
[2451510,300]
[[Node: wordvecs = Placeholder[dtype=DT_FLOAT, shape=[2451510,300],
_device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
Caused by op 'wordvecs', defined at:
File "train.py", line 470, in <module>
tf.app.run()
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "train.py", line 466, in main
train()
File "train.py", line 310, in train
train_emb=FLAGS.train_emb)
File "/home/mishra/Project/RelExtractKBP/text_cnn.py", line 20, in
__init__
self.wordvecs = tf.placeholder(tf.float32, shape=(2451510, 300),
name="wordvecs")
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/ops/array_ops.py", line 1599, in placeholder
return gen_array_ops._placeholder(dtype=dtype, shape=shape, name=name)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/ops/gen_array_ops.py", line 3091, in
_placeholder
"Placeholder", dtype=dtype, shape=shape, name=name)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/framework/op_def_library.py", line 787, in
_apply_op_helper
op_def=op_def)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/mishra/anaconda3/envs/tensorflow/lib/python3.6/site-
packages/tensorflow/python/framework/ops.py", line 1470, in __init__
self._traceback = self._graph._extract_stack() # pylint:
disable=protected-access
InvalidArgumentError (see above for traceback): You must feed a value for
placeholder tensor 'wordvecs' with dtype float and shape [2451510,300]
[[Node: wordvecs = Placeholder[dtype=DT_FLOAT, shape=[2451510,300],
_device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
From what I understand, this error may occur if I do not pass the value to the placeholder while calling sess.run. However, I am not sure what I need to do when calling sess.run(tf.global_variables_initializer()) which is generating this error as per the Traceback.
Any pointers will really help. Thanks a lot.
The placeholder wordvecs needs to be fed.
This can be reproduced by the following example from tf.placeholder example in the official documentation -
x = tf.placeholder(tf.float32, shape=(1024, 1024))
y = tf.matmul(x, x)
with tf.Session() as sess:
print(sess.run(y)) # ERROR: will fail because x was not
fed.
rand_array = np.random.rand(1024, 1024)
print(sess.run(y, feed_dict={x: rand_array})) # Will succeed.
The error is seen at sess.run(tf.global_variables_initializer()) step because, it tries to initialize the following variable:
self.WordVecs = tf.Variable(trainable = False,
initial_value=self.wordvecs, name="WordVecs")
and its initial_value points to a tf.Placeholder, which is yet to be initialized.
self.wordvecs = tf.placeholder(tf.float32, shape = (2451510, 300),
name = "wordvecs")

Resources