[root#tel-sz01-test-xxx-001 keras]#/appcom/python/anaconda2/bin/python cnn.py
File
"/appcom/python/anaconda2/lib/python2.7/site-packages/Keras-2.0.2-py2.7.egg/keras/backend/theano_backend.py",
line 1131, in function
raise ValueError(msg) ValueError: Invalid argument "class_mode" passed to K.function
try removing class_mode from model.compile(). This is deprecated in the newer versions of Keras.
Related
I am trying to train the model implemented in this repo https://bitbucket.org/VioletPeng/language-model/src/master/ (the second model: title to title-storyline to story model)
The training would go fine for the first epoch, but as soon as it tries to call the train function to start the second epoch everything breaks and I get the following error:
TypeError: cannot assign 'torch.cuda.FloatTensor' as parameter 'weight_hh_l0' (torch.nn.Parameter or None expected)
I don't know what the issue is, I tried looking this error up and tried changing .cuda to .to(device) and using device= inside the tensor initialization when possible.
But none of this seems to be doing anything.
Below is the full exception stack trace:
File "pytorch_src/main.py", line 253, in <module>
train()
File "pytorch_src/main.py", line 209, in train
output, hidden, rnn_hs, dropped_rnn_hs = model(data, hidden, return_h=True)
File "/home/e/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/e/Documents/Amal/language-model/pytorch_src/model.py", line 81, in forward
raw_output, new_h = rnn(raw_output, hidden[l])
File "/home/e/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/e/Documents/Amal/language-model/pytorch_src/weight_drop.py", line 47, in forward
self._setweights()
File "/home/e/Documents/Amal/language-model/pytorch_src/weight_drop.py", line 44, in _setweights
setattr(self.module, name_w, w)
File "/home/e/anaconda3/lib/python3.7/site-packages/torch/nn/modules/rnn.py", line 108, in __setattr__
super(RNNBase, self).__setattr__(attr, value)
File "/home/e/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 801, in __setattr__
.format(torch.typename(value), name))
I downgraded my python to 3.6 and reinstalled all the requirements and it worked.
So probably the issue was an incompatible torch version.
Newer versions of PyTorch require parameters as torch.nn.Parameter.
I think you need to change the code as follows, at least, it helped me with the same error in the code based on the same codebase:
def _setweights(self):
for name_w in self.weights:
raw_w = getattr(self.module, name_w + '_raw')
w = None
w = torch.nn.functional.dropout(raw_w, p=self.dropout, training=self.training)
setattr(self.module, name_w, torch.nn.Parameter(w))
I am using PyTorch version 1.5.0.
When I pass an input torch tensor of size [8,21,400,400] with a target of size [8,400,400], the program raises a TypeError: iteration over a 0-d tensor. However, the dimensions of the arguments are 4 and 3 respectively.
What could be causing this error?
The traceback points to torch\tensor.py's iter function.
Traceback (most recent call last):
File "train.py", line 108, in <module>
loss, accuracy = lossLayer(pred2, targetBatch)
File "C:\Users\PC\anaconda3\lib\site-packages\torch\tensor.py", line 462, in __iter__
raise TypeError('iteration over a 0-d tensor')
TypeError: iteration over a 0-d tensor
You get the error because nn.CrossEntropyLoss just returns one torch.Tensor, not a pair (it doesn't return accuracy). And this tensor is 0-dimensional, i.e. one number (unless you don't override reduction argument to 'none' to get per-element loss). So when you try to assign its value to two variables loss, accuracy python tries to iterate over this tensor variable, hence the error message. Simply use loss = lossLayer(pred2, targetBatch).
System information
OS Platform and Distribution: Ubuntu 16.04 LTS
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 1.12.0
Python version: 3.5.2
CUDA/cuDNN version: release 9.0, V9.0.176
GPU model and memory: Tesla K80, 12 GB
Describe the current behavior
When I try to save my model using model.save() where model is a tf.keras.Model instance, it throws a TypeError: ('Not JSON Serializable:', <tf.Variable 'Variable:0' shape=() dtype=float32>) .
I am using a tf.keras.backend.variable() in loss_weights in model.compile.
Optimizer: tf.keras.optimizers.Adam
Interestingly, when I try to save my model weights only using model.save_weights where model is a tf.keras.Model instance, it works fine, NO ERROR.
Code to reproduce the issue
alpha = tf.keras.backend.variable(0.25)
Code any model using tf.keras
model= get_model()
model.compile(optimizer=optimizer,loss={"softmax1":generalized_dice_loss,"softmax2":generalized_dice_loss}, loss_weights=[1.0,alpha])
Do training using model.fit()-
model.save()
Other info / logs
Traceback (most recent call last):
File "main_latest.py", line 45, in
max_queue_size=10)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 2177, in fit_generator
initial_epoch=initial_epoch)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/training_generator.py", line 216, in fit_generator
callbacks.on_epoch_end(epoch, epoch_logs)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/callbacks.py", line 214, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/callbacks.py", line 601, in on_epoch_end
self.model.save(filepath, overwrite=True)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/network.py", line 1363, in save
save_model(self, filepath, overwrite, include_optimizer)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/saving.py", line 134, in save_model
default=serialization.get_json_type).encode('utf8')
File "/usr/lib/python3.5/json/init.py", line 237, in dumps
**kw).encode(obj)
File "/usr/lib/python3.5/json/encoder.py", line 198, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode
return _iterencode(o, 0)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/util/serialization.py", line 64, in get_json_type
raise TypeError('Not JSON Serializable:', obj)
TypeError: ('Not JSON Serializable:', <tf.Variable 'Variable:0' shape=() dtype=float32>)
Screenshot of Error:
model.save is trying to save a tf.Variable which is not JSON serializable.
model.fit saves everything, not just the model weights. I've seen this problem when my optimizer had a tf.Tensor which cannot be serialized.
Everything points to alpha being your problem in this case. Also the documentation for model.compile states that the loss_weights should be a Python float.
So using alpha.numpy() should solve your problem:
model.compile(optimizer=optimizer,loss={"softmax1":generalized_dice_loss,"softmax2":generalized_dice_loss}, loss_weights=[1.0,alpha.numpy()])
I have seen problems similar to mine here on Stack Overflow, but not exactly the same. I can reshape when using fully-connected NN layers, but not with Conv1D layers. Here's a minimal example. I'm using TF 1.4.0 on Python 3.6.3.
import tensorflow as tf
# fully connected
fc = tf.placeholder(tf.float32, [None,12])
fc = tf.contrib.layers.fully_connected(fc, 12)
fc = tf.contrib.layers.fully_connected(fc, 6)
fc = tf.reshape(fc, [-1,3,2])
# convolutional
con = tf.placeholder(tf.float32, [None,50,4])
con = tf.layers.Conv1D(con, 12, 3, activation=tf.nn.relu)
con = tf.layers.Conv1D(con, 6, 3, activation=tf.nn.relu)
con = tf.reshape(con, [-1,50,3,2])
Here is the output (yes, I'm aware of the RuntimeWarning. The messages I have found which discuss it suggest that it's harmless, but if you know otherwise, please share!):
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_util.py", line 468, in make_tensor_proto
str_values = [compat.as_bytes(x) for x in proto_values]
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_util.py", line 468, in <listcomp>
str_values = [compat.as_bytes(x) for x in proto_values]
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/compat.py", line 65, in as_bytes
(bytes_or_text,))
TypeError: Expected binary or unicode string, got <tensorflow.python.layers.convolutional.Conv1D object at 0x7fa67e0d1a20>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "minimal reshape example.py", line 16, in <module>
con = tf.reshape(con, [-1,width,3,2])
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 3938, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py", line 513, in _apply_op_helper
raise err
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 926, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py", line 229, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/constant_op.py", line 208, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_util.py", line 472, in make_tensor_proto
"supported type." % (type(values), values))
TypeError: Failed to convert object of type <class 'tensorflow.python.layers.convolutional.Conv1D'> to Tensor. Contents: <tensorflow.python.layers.convolutional.Conv1D object at 0x7fa67e0d1a20>. Consider casting elements to a supported type.
My code fails at con = tf.reshape(con, [-1,50,3,2]). Yet the pattern is nearly identical to the pattern that I use for the fully-connected graph, fc.
I made nets very similar to these work in the higher-level API for TensorFlow called TFLearn. However, TFLearn's DNN Estimator object is having trouble managing a tf.Session correctly. After over a month, I have yet to resolve the issue with TFLearn's developers on GitHub.
I don't mind using TensorFlow's native Estimator, but I have to solve this reshape problem to achieve it.
Well, I found the error: tf.layers.Conv1D != tf.layers.conv1d. Changing the former to the latter eliminated the error. Let the TensorFlow / Python user beware!
Even though TensorFlow seems to avoid Python's object model (which is probably necessary, given the possibility of distributed, low-level computation), there are in fact a few genuine classes in the Python API. The class constructors can accept many (all?) of the same arguments as the similarly-named convenience functions.
I'm running Keras with CNTK backend.
I'm using Python 3.5 GPU-1bit-SGD on Windows 10.
Followed the instruction as given here to install and test the set-up. And everything works!
However, when I run my own code, I get this error:
numBins = y_pred.get_shape().as_list()[1]
File "C:\Users\abisw\AppData\Local\Continuum\Anaconda3\envs\cntkkeraspy35\lib\site-packages\cntk\ops\functions.py", line 480, in __getattr__
return getattr(outputs[0], name)
File "C:\Users\abisw\AppData\Local\Continuum\Anaconda3\envs\cntkkeraspy35\lib\site-packages\cntk\cntk_py.py", line 1125, in <lambda>
__getattr__ = lambda self, name: _swig_getattr(self, Variable, name)
File "C:\Users\abisw\AppData\Local\Continuum\Anaconda3\envs\cntkkeraspy35\lib\site-packages\cntk\cntk_py.py", line 83, in _swig_getattr
raise AttributeError("'%s' object has no attribute '%s'" % (class_type.__name__, name))
AttributeError: 'Variable' object has no attribute 'get_shape'
Any idea what's wrong?
you are mixing keras code with tensorflow code, get_shape is a tensorflow grammar, not keras grammar. If you want the variable shape, you should write:
from keras import K
K.int_shape(y_pred)