I have a keras model and I try to test it with test data and the evaluate_generator method. I have a use case where a callback in this method would come in handy. In the keras docs: evaluate_generator there is a callback argument. However when I test this with following code I get an error.
model = load_model('/models/model.h5')
img_width = 120
img_height = 120
batch_size = 32
data_generator = ImageDataGenerator(rescale=1. / 255.)
test_generator = data_generator.flow_from_directory(
'/data/test',
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='binary')
STEP_SIZE_TEST = test_generator.n // test_generator.batch_size
class TestCallback(Callback):
def on_test_batch_begin(self, batch, logs=None):
print('Evaluating batch: ' + str(batch))
test_callback = TestCallback()
model.evaluate_generator(test_generator, STEP_SIZE_TEST, callbacks=[test_callback])
The error:
Traceback (most recent call last):
File "/test_callback.py", line 34, in <module>
model.evaluate_generator(generator=test_generator, steps=STEP_SIZE_TEST, callbacks=[test_callback])
File "/miniconda3/envs/models/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
TypeError: evaluate_generator() got an unexpected keyword argument 'callbacks'
When I edit the code and leave out the keyword like so:
model.evaluate_generator(test_generator, STEP_SIZE_TEST, [test_callback])
I get following error:
Exception in thread Thread-16:
Traceback (most recent call last):
File "/miniconda3/envs/models/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/miniconda3/envs/models/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/miniconda3/envs/models/lib/python3.6/site-packages/keras/utils/data_utils.py", line 671, in _run
executor.apply_async(next_sample, (self.uid,)), block=True)
File "/miniconda3/envs/models/lib/python3.6/queue.py", line 127, in put
if self.maxsize > 0:
TypeError: '>' not supported between instances of 'list' and 'int'
My keras version is 2.2.4
The documentation you're seeing is for master branch. The 'callbacks' argument is not supported on 2.2.4. In 2.2.4, the third argument is still max_queue_size and hence there is an error when interpreting the [test_callback] list as an int.
Related
Trying to use multiple TensorFlow models in parallel using pathos.multiprocessing.Pool
Error is:
multiprocess.pool.RemoteTraceback:
Traceback (most recent call last):
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\multiprocess\pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\multiprocess\pool.py", line 44, in mapstar
return list(map(*args))
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\pathos\helpers\mp_helper.py", line 15, in <lambda>
func = lambda args: f(*args)
File "c:\Users\Burge\Desktop\SwarmMemory\sim.py", line 38, in run
i.step()
File "c:\Users\Burge\Desktop\SwarmMemory\agent.py", line 240, in step
output = self.ai(np.array(self.internal_log).reshape(-1, 1, 9))
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1012, in __call__
outputs = call_fn(inputs, *args, **kwargs)
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\tensorflow\python\keras\engine\sequential.py", line 375, in call
return super(Sequential, self).call(inputs, training=training, mask=mask)
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\tensorflow\python\keras\engine\functional.py", line 425, in call
inputs, training=training, mask=mask)
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\tensorflow\python\keras\engine\functional.py", line 569, in _run_internal_graph
assert x_id in tensor_dict, 'Could not compute output ' + str(x)
AssertionError: Could not compute output KerasTensor(type_spec=TensorSpec(shape=(None, 1, 4), dtype=tf.float32, name=None), name='dense_1/BiasAdd:0', description="created by layer 'dense_1'")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\Users\Burge\Desktop\SwarmMemory\sim.py", line 78, in <module>
p.map(Sim.run, sims)
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\pathos\multiprocessing.py", line 137, in map
return _pool.map(star(f), zip(*args)) # chunksize
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\multiprocess\pool.py", line 268, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "c:\users\burge\appdata\local\programs\python\python37\lib\site-packages\multiprocess\pool.py", line 657, in get
raise self._value
AssertionError: Could not compute output KerasTensor(type_spec=TensorSpec(shape=(None, 1, 4), dtype=tf.float32, name=None), name='dense_1/BiasAdd:0', description="created by layer 'dense_1'")
The creation of the pool is as follows:
if __name__ == '__main__':
freeze_support()
model = Sequential()
model.add(Input(shape=(1,9)))
model.add(LSTM(10, return_sequences=True))
model.add(Dropout(0.1))
model.add(LSTM(5))
model.add(Dropout(0.1))
model.add(Dense(4))
model.add(Dense(4))
models = []
sims = []
for i in range(6):
models.append(tensorflow.keras.models.clone_model(model))
sims.append(Sim(models[-1]))
p = Pool()
p.map(Sim.run, sims)
Basically, I am running a simulation using the model provided to the class sim. This means after the sim has run I can get use a fitness function on results, and apply a genetic algorithm to the results.
GitHub link for more information, under branch python-ver:
https://github.com/HarryBurge/SwarmMemory
EDIT:
In case anyone needs to know how to do this in the future.
I used keras-pickle-wrapper to be able to pickle the keras model and just pass it to the run method.
models = []
sims = []
for i in range(6):
models.append(KerasPickleWrapper(tensorflow.keras.models.clone_model(model)))
sims.append(Sim())
p = Pool()
p.map(Sim.run, sims, models)
I'm the author of pathos. Whenever you see self._value in the error, what's generally happening is that something you tried to send to another processor failed to serialize. The error and traceback is a bit obtuse, admittedly. However, what you can do is check the serialization with dill, and determine if you need to use one of the serialization variants (like dill.settings['trace'] = True), or whether you need to restructure your code slightly to better accommodate serialization. If the class you are working with is something you can edit, then an easy thing to do is to add a __reduce__ method, or similar, to aid serialization.
I have checked documentation for keras.fit_generator function still not able to find the problem
Libraries are working fine in my laptop
My code:
# train the network
print("training network...")
sys.stdout.flush()
#class_mode ='categorical', # 2D one-hot encoded labels
H = model.fit_generator(aug.flow(Xtrain, trainY,batch_size=BS),
validation_data=(Xval, valY),
steps_per_epoch=len(trainX) // BS,
epochs=EPOCHS, verbose=1)
# save the model to disk
print("Saving model to disk")
sys.stdout.flush()
model.save("/tmp/mymodel")
i am getting following error for my code :
Traceback (most recent call last):File
"C:\Users\user\AppData\Local\conda\conda\envs\my_root\lib\site-
packages\IPython\core\interactiveshell.py", line 3267, in run_code
File "<ipython-input-80-935b20410c11>", line 8, in <module>
epochs=EPOCHS, verbose=1)
File "C:\Users\user\AppData\Local\conda\conda\envs\my_root\lib\site-
packages\keras\legacy\interfaces.py", line 91, in wrapper
File "C:\Users\user\AppData\Local\conda\conda\envs\my_root\lib\site-
packages\keras\engine\training.py", line 1418, in fit_generator
File "C:\Users\user\AppData\Local\conda\conda\envs\my_root\lib\site-
packages\keras\engine\training_generator.py", line 162, in fit_generator
File "C:\Users\user\AppData\Local\conda\conda\envs\my_root\lib\site-
packages\keras\utils\data_utils.py", line 647, in __init__
File "C:\Users\user\AppData\Local\conda\conda\envs\my_root\lib\site-
packages\keras\utils\data_utils.py", line 433, in __init__
File"C:\Users\user\AppData\Local\conda\conda\envs\
my_root\lib\multiprocessing\context.py", line 133, in Value
File "C:\Users\user\AppData\Local\conda\conda\envs\
my_root\lib\multiprocessing\sharedctypes.py", line 182
exec template % ((name,)*7) in d
^
SyntaxError: invalid syntax
I am stuck in a simple looking problem in Tensorflow.
Traceback (most recent call last):
File op_def_library.py, line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File ops.py, line 1040, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File ops.py, line 883, in _TensorTensorConversionFunction
(dtype.name, t.dtype.name, str(t)))
ValueError: Tensor conversion requested dtype int64 for Tensor with dtype float32: 'Tensor("sequence_sparse_softmax_cross_entropy/zeros_like:0", shape=(?, ?, 10004), dtype=float32)'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "red.py", line 281, in <module>
main()
File "red.py", line 99, in main
sequence_length=lengths)
File loss.py, line 225, in sequence_sparse_softmax_cross_entropy
losses = xloss(labels=labels, logits=logits)
File loss.py", line 48, in loss
post = array_ops.where(target_tensor > zeros, target_tensor - sigmoid_p, zeros)
gen_math_ops.py, line 2924, in greater
"Greater", x=x, y=y, name=name)
op_def_library.py, line 546, in _apply_op_helper
inferred_from[input_arg.type_attr]))
TypeError: Input 'y' of 'Greater' Op has type float32 that does not match type int64 of argument 'x'
Using as type also does not work.
I just defined another function to be used. I defined it and tried to use it. What should I do to make it work? I just want to define a function that takes tensors as input just like tf cross entropy function. Please suggest how to do that.
In particular, how can I resolve the error?
I am extremely new to tensorflow and trying to learn how to save and load a previously trained model. I created a simple model using Estimator and trained it.
classifier = tf.estimator.Estimator(model_fn=bag_of_words_model)
# Train
train_input_fn = tf.estimator.inputs.numpy_input_fn(
x={"words": x_train}, # x_train is 2D numpy array of shape (26, 5)
y=y_train, # y_train is 1D panda series of length 26
batch_size=1000,
num_epochs=None,
shuffle=True)
classifier.train(input_fn=train_input_fn, steps=300)
I then try to save the model:
def serving_input_receiver_fn():
serialized_tf_example = tf.placeholder(dtype=tf.int64, shape=(None, 5), name='words')
receiver_tensors = {"predictor_inputs": serialized_tf_example}
features = {"words": tf.placeholder(tf.int64, shape=(None, 5))}
return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)
full_model_dir = classifier.export_savedmodel(export_dir_base="E:/models/", serving_input_receiver_fn=serving_input_receiver_fn)
I have actually copied the serving_input_receiver_fn from this similar question. I don't understand exactly what is going on in that function. But this stores my model in E:/models/<some time stamp>.
I now try to load this saved model:
from tensorflow.contrib import predictor
classifier = predictor.from_saved_model("E:\\models\\<some time stamp>")
The models perfectly loaded. Hereafter, I am struck on how to use this classifier object to get predictions on new data. I have followed a guide here to achieve it but couldn't do it :(. Here is what I did:
predictions = classifier({'predictor_inputs': x_test})["output"] # x_test is 2D numpy array same like x_train in the training part
I get error as follows:
2019-01-10 12:43:38.603506: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
INFO:tensorflow:Restoring parameters from E:\models\1547101005\variables\variables
Traceback (most recent call last):
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call
return fn(*args)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype int64 and shape [?,5]
[[{{node Placeholder}} = Placeholder[dtype=DT_INT64, shape=[?,5], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:/ml_classif/tensorflow_bow_with_prob/load_model.py", line 85, in <module>
predictions = classifier({'predictor_inputs': x_test})["output"]
File "E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\predictor.py", line 77, in __call__
return self._session.run(fetches=self.fetch_tensors, feed_dict=feed_dict)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
run_metadata_ptr)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
run_metadata)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype int64 and shape [?,5]
[[node Placeholder (defined at E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\saved_model_predictor.py:153) = Placeholder[dtype=DT_INT64, shape=[?,5], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Caused by op 'Placeholder', defined at:
File "E:/ml_classif/tensorflow_bow_with_prob/load_model.py", line 82, in <module>
classifier = predictor.from_saved_model("E:\\models\\1547101005")
File "E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\predictor_factories.py", line 153, in from_saved_model
config=config)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\saved_model_predictor.py", line 153, in __init__
loader.load(self._session, tags.split(','), export_dir)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\saved_model\loader_impl.py", line 197, in load
return loader.load(sess, tags, import_scope, **saver_kwargs)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\saved_model\loader_impl.py", line 350, in load
**saver_kwargs)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\saved_model\loader_impl.py", line 278, in load_graph
meta_graph_def, import_scope=import_scope, **saver_kwargs)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\training\saver.py", line 1696, in _import_meta_graph_with_return_elements
**kwargs))
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\meta_graph.py", line 806, in import_scoped_meta_graph_with_return_elements
return_elements=return_elements)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\importer.py", line 442, in import_graph_def
_ProcessNewOps(graph)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\importer.py", line 234, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\ops.py", line 3440, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\ops.py", line 3440, in <listcomp>
for c_op in c_api_util.new_tf_operations(self)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\ops.py", line 3299, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in __init__
self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype int64 and shape [?,5]
[[node Placeholder (defined at E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\saved_model_predictor.py:153) = Placeholder[dtype=DT_INT64, shape=[?,5], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
It says that I have to feed value to the placeholder (I think the one defined in serving_input_receiver_fn). I have no idea how to do it without using a Session object of tensorflow.
Please feel free to ask for more information if required.
After somewhat vague understanding of serving_input_receiver_fn, I figured out that the features must not be a placeholder as it creates 2 placeholders (1 for serialized_tf_example and the other for features). I modified the function as follows (the changes is just for the features variable):
def serving_input_receiver_fn():
serialized_tf_example = tf.placeholder(dtype=tf.int64, shape=(None, 5), name='words')
receiver_tensors = {"predictor_inputs": serialized_tf_example}
features = {"words": tf.tile(serialized_tf_example, multiples=[1, 1])} # Changed this
return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)
When I try to predict the output from the loaded model, I get no error now. It works! Only thing is that the output is incorrect (for which I am posting a new question :) ).
I have been trying to create a chatbot but I keep getting the following error. I am a beginner in TensorFlow.
Traceback (most recent call last):
File "main.py", line 78, in <module>
model.load("model.tflearn")
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tflearn\models\dnn.py", line 308, in load
self.trainer.restore(model_file, weights_only, **optargs)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tflearn\helpers\trainer.py", line 490, in restore
self.restorer.restore(self.session, model_file)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tensorflow\python\training\saver.py", line 1278, in restore
compat.as_text(save_path))
ValueError: The passed save_path is not a valid checkpoint: C:\Users\User\model.tflearn
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "main.py", line 80, in <module>
model.fit(training, output, n_epoch=1000, batch_size=8, show_metric=True)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tflearn\models\dnn.py", line 216, in fit
callbacks=callbacks)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tflearn\helpers\trainer.py", line 339, in fit
show_metric)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tflearn\helpers\trainer.py", line 816, in _train
tflearn.is_training(True, session=self.session)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tflearn\config.py", line 95, in is_training
tf.get_collection('is_training_ops')[0].eval(session=session)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tensorflow\python\framework\ops.py", line 731, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tensorflow\python\framework\ops.py", line 5579, in _eval_using_default_session
return session.run(tensors, feed_dict)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tensorflow\python\client\session.py", line 950, in run
run_metadata_ptr)
File "C:\Users\User\Anaconda3\envs\newbot\lib\site-packages\tensorflow\python\client\session.py", line 1096, in _run
raise RuntimeError('Attempted to use a closed Session.')
RuntimeError: Attempted to use a closed Session.
This is my TensorFlow code:
tensorflow.reset_default_graph()
net = tflearn.input_data(shape=[None, len(training[0])])
net = tflearn.fully_connected(net, 8)
net = tflearn.fully_connected(net, 8)
net = tflearn.fully_connected(net, len(output[0]), activation="softmax")
net = tflearn.regression(net)
model = tflearn.DNN(net)
try:
model.load("model.tflearn")
except:
model.fit(training, output, n_epoch=1000, batch_size=8, show_metric=True)
model.save("model.tflearn")
I am using:
Python 3.6.9
TensorFlow 1.14.0
TFLearn 0.3.2
Thank you in advance!
Change your Tensorflow code to:
try:
model.load('model.tflearn')
except:
tensorflow.reset_default_graph()
net = tflearn.input_data(shape=[None, len(training[0])])
net = tflearn.fully_connected(net, 8)
net = tflearn.fully_connected(net, 8)
net = tflearn.fully_connected(net, len(output[0]), activation='softmax')
net = tflearn.regression(net)
model = tflearn.DNN(net)
model.fit(training, output, n_epoch=1000, batch_size=8, show_metric=True)
model.save("model.tflearn")
I think the problem happens because you are creating and reseting a model and then requesting to load it, and then the framework gets lost.
Firstly, from this error message
ValueError: The passed save_path is not a valid checkpoint: C:\Users\User\model.tflearn
it looks like C:\Users\User\model.tflearn doesn't exist.
Secondly, you have model.fit function in the exception handling block. Is it done intentionally? I would imagine you want to proceed with the fit and save function only if you are able to load the model successfully.