On Android Studio, I'm not able to view the model metadata, even though I had added the metadata in Python manually. I get the error:
This model is not supported: input tensor 0 does not have a name.
My attempts at fixing:
I added the layer name to the tensorflow input layer, using:
img_input = Input(shape=input_shape, batch_size=1, name="input_image")
I even checked it in Netron, and the input layer showed up as input_image as expected:
I fixed it by copying 1 line from the documentation, I needed to add a specific property to the TensorMetadataT object, input_meta.name = "image". This does not come from the model layer name, but needs to be manually added.
input_meta.name = "image_input"
Weirdly: This wasn't a problem before, but for some reason, my previous code broke and I needed this change.
Related
I am following this notebook:
One of the method:
def init_hidden(self, batch_size):
''' Initializes hidden state '''
# Create two new tensors with sizes n_layers x batch_size x n_hidden,
# initialized to zero, for hidden state and cell state of LSTM
weight = next(self.parameters()).data
if (train_on_gpu):
hidden = (weight.new(self.n_layers, batch_size, self.n_hidden).zero_().cuda(),
weight.new(self.n_layers, batch_size, self.n_hidden).zero_().cuda())
else:
hidden = (weight.new(self.n_layers, batch_size, self.n_hidden).zero_(),
weight.new(self.n_layers, batch_size, self.n_hidden).zero_())
return hidden
I would like to see what type of weight is and how to use new() method, so I was trying to find out the parameters() method as the data attribute comes from paramerters() method.
Surprisingly, I cannot find where it comes from after reading the source code of nn module in PyTorch.
How do you figure out where to see the definition of methods you saw from PyTorch?
so I was trying to find out the parameters() method as the data
attribute comes from paramerters() method.
Surprisingly, I cannot find where it comes from after reading the
source code of nn module in PyTorch.
You can see the module definition under torch/nn/modules/module.py here at line 178.
You can then easily spot the parameters() method here.
How do you guys figure out where to see the definition of methods you
saw from PyTorch?
The easiest way that I myself always use, is to use VSCode's Go to Definition or its Peek -> Peek definition feature.
I believe Pycharm has a similar functionality as well.
You can also check source code directly from PyTorch documentation.
See here for torch.nn.Module.parameters function (just click orange "Source", which gets you here).
Source is linked if it isn't written in C/C++/low level, in this case you have to go through GitHub and get around the project.
In the following article there is an instruction that dataset needs to be divided into train, validation and test folders where the test folder should not contain the labeled subfolders. Instead it should only contain a single folder (i.e. Test_folder).
When I use the following code, I get the output message refering that no image were found.
Ver.1:
test_generator = test_datagen.flow_from_directory(
"dataset\\test\\test_folder\\",
target_size=(IMG_WIDTH, IMG_HEIGHT),
batch_size=1,
class_mode=None,
shuffle=False,
seed=10)
Output message: "Found 0 images belonging to 0 classes.".
Instead, if I use the same folder structure (dataset\test\class_a\test_1.jpg etc) as in the train and validation folders, everything seems to be OK and I manage to evaluate my model.
Ver.2:
test_generator = test_datagen.flow_from_directory(
"dataset\\test\\",
target_size=(IMG_WIDTH, IMG_HEIGHT),
batch_size=32,
class_mode='categorical',
shuffle=False,
seed=10)
Output message: "Found 1500 images belonging to 3 classes.".
I also tried the recommendation where 'classes' attribute is specified but still 0 images were found.
Ver.3:
test_generator = test_datagen.flow_from_directory(
"dataset2\\test\\test_folder\\",
target_size=(IMG_WIDTH, IMG_HEIGHT),
batch_size=1,
classes=['test'],
class_mode=None,
shuffle=False,
seed=10)
Output message: Found 0 images belonging to 1 classes.
Thus, what is the correct way to call flow_from_directory() method and why am I getting the message that no files were found? Is my model not correctly evaluated when I use the Ver.2 solution?
If None, no labels are returned (the generator will only yield batches of image data, which is useful to use with model.predict_generator()). Please note that in case of class_mode None, the data still needs to reside in a subdirectory of directory for it to work correctly.
Note that evaluation needs labels in order to determine how good the model is. Typically one only uses no labels in production, or real life testing. You seem to want to test your model, so you actually want to have labels. This means you should sort your testing data
train/
label1/
...
...
label2/
...
...
val/
label1/
...
...
label2/
...
...
test/
label1/
...
...
label2/
Reading documentation is often more helpful than those articles when you want to learn how to use a framework. Here is the link to flow_from_directory: https://keras.io/preprocessing/image/#flow_from_directory.
On another note, mc.ai is a website full of ripped Medium articles and should be avoided. Many articles are incomplete + it infringes the copyright of the authors.
If you do not want to change the structure of your test directory. Try this
test_data_gen = test_image_generator.flow_from_directory(test_dir,
target_size=(IMG_HEIGHT, IMG_WIDTH),
batch_size= batch_size ,shuffle=False,
class_mode= 'binary',classes=['.'])
classes=['.'] needs to be specified as the flow_from_directory method will search for folders.
In your Ver.3: you specified the class but there is no folder having test inside the directory so you got 0 images and 1 classes.
Another solution can be to set as a directory for the flow from directory function the parent folder of the test directory, and to set the classes attribute to the name of the test folder containing the images (here in the example 'test')
datagen = ImageDataGenerator()
# In ./dataset directory, there is `test` directory.
test_data = datagen.flow_from_directory('./dataset', classes=['test'])
Here's the reference: https://velog.io/#jhcho/Loading-Unlabeled-Images-with-ImageDataGenerator-flowfromdirectory-in-Keras
I'm trying to visualize my model graph on TensorBoard. I'm using Keras 2.1.5 and tensorflow-gpu 1.13.1. My model is a concatenation of convolutional layers and, at the end, a custom layer where I make some operations with tensors.
Everything works fine, although I defined some prints at the end and at the beggining of my custom layer and then check that Python enters two time my code when I just call it once.
I'm running some trainings and check the model graph on TensorBoard and then found out something I haven't seen in any other example on the web:
Graph of custom layer
There is a connected graph of my custom layer (trans2img) and another one unconnected and with empty placeholders as inputs. I don't understand the reason.
Here is a simple example as my code:
def custom_layer(inputs):
with tf.name_scope('trans2img'):
a = inputs[0]
def some_operation(a):
with tf.name_scope('op1'):
b = 2*a
return b
def some_other_op(b, c):
with tf.name_scope('op2'):
d = b/c
return d
b = some_operation(a)
d = some_other_op(b, inputs[1])
return d
After that, in my network definition file, I load this custom layer as from custom_layer import custom_layer, and then use it as a Lambda layer:
net = Lambda(custom_layer)[branch1, branch2]
I don't know if it is because the way I define the inner operations in my custom_layer or the way I call them. I would like to know how to interpret this second unconnected graph I get and if it's an indicator of unnineficientcode. I would appreciate any clue and help.
This issue happens when you are using multiple tf scope.
For each scope it creates a new op with "op_{integer}".
You need to make use of "absolute_name_scope(scope)" to resolve your issue.
Please refer to below link on how to make use of it.
https://github.com/tensorflow/tensorflow/issues/9545
https://github.com/tensorflow/tensorflow/pull/23250/commits/1169eaca048b660905fc5776e617cc824a19b125
I have some experience with writing machine learning programs in python, but I'm new to TensorFlow and am checking it out. My dev environment is a lubuntu 14.04 64-bit virtual machine. I've created a python 3.5 conda environment from miniconda and installed TensorFlow 0.12 and its dependencies. I began trying to run some example code from TensorFlow's tutorials and encountered this warning when calling fit() in the boston.py example for input functions: source.
WARNING:tensorflow:Rank of input Tensor (1) should be the same as
output_rank (2) for column. Will attempt to expand dims. It is highly
recommended that you resize your input, as this behavior may change.
After some searching in Google, I found other people encountered this same warning:
https://github.com/tensorflow/tensorflow/issues/6184
https://github.com/tensorflow/tensorflow/issues/5098
Tensorflow - Boston Housing Data Tutorial Errors
However, they also experienced errors which prevent code execution from completing. In my case, the code executes with the above warning. Unfortunately, I couldn't find a single answer in those links regarding what caused the warning and how to fix the warning. They all focused on the error. How does one remove the warning? Or is the warning safe to ignore?
Cheers!
Extra info, I also see the following warnings when running the aforementioned boston.py example.
WARNING:tensorflow:*******************************************************
WARNING:tensorflow:TensorFlow's V1 checkpoint format has been
deprecated. WARNING:tensorflow:Consider switching to the more
efficient V2 format: WARNING:tensorflow:
'tf.train.Saver(write_version=tf.train.SaverDef.V2)'
WARNING:tensorflow:now on by default.
WARNING:tensorflow:*******************************************************
and
WARNING:tensorflow:From
/home/kade/miniconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/estimators/dnn_linear_combined.py:1053
in predict.: calling BaseEstimator.predict (from
tensorflow.contrib.learn.python.learn.estimators.estimator) with x is
deprecated and will be removed after 2016-12-01. Instructions for
updating: Estimator is decoupled from Scikit Learn interface by moving
into separate class SKCompat. Arguments x, y and batch_size are only
available in the SKCompat class, Estimator will only accept input_fn.
Example conversion: est = Estimator(...) -> est =
SKCompat(Estimator(...))
UPDATE (2016-12-22):
I've tracked the warning to this file:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/layers/python/layers/feature_column_ops.py
and this code block:
except NotImplementedError:
with variable_scope.variable_scope(
None,
default_name=column.name,
values=columns_to_tensors.values()):
tensor = column._to_dense_tensor(transformed_tensor)
tensor = fc._reshape_real_valued_tensor(tensor, 2, column.name)
variable = [
contrib_variables.model_variable(
name='weight',
shape=[tensor.get_shape()[1], num_outputs],
initializer=init_ops.zeros_initializer(),
trainable=trainable,
collections=weight_collections)
]
predictions = math_ops.matmul(tensor, variable[0], name='matmul')
Note the line: tensor = fc._reshape_real_valued_tensor(tensor, 2, column.name)
The method signature is: _reshape_real_valued_tensor(input_tensor, output_rank, column_name=None)
The value 2 is hardcoded as the value of output_rank, but the boston.py example is passing in an input_tensor of rank 1. I will continue to investigate.
If you specify the shape of your tensor explicitly:
tf.constant(df[k].values, shape=[df[k].size, 1])
the warning should go away.
After I specify the shape of the tensor explicitly.
continuous_cols = {k: tf.constant(df[k].values, shape=[df[k].size, 1]) for k in CONTINUOUS_COLUMNS}
It works!
I'm trying to take the last layer in a model (old model) and make a new model of only one layer (new model) that has the exact same parameters as the last layer of the old model. I want to do this in a way that's agnostic to what the last layer of the old model happens to be. I'm trying to do it with this code, but am getting an error.
newModel = Sequential()
newModel.add(type(oldModel.layers[-1])(oldModel.layers[-1].output_shape,
activation=oldModel.layers[-1].activation,
input_shape=oldModel.layers[-1].input_shape))
That yields the following error:
TypeError: __init__() missing 1 required positional argument: 'output_dim'
If I check the last layer in oldModel, it shows me this:
full_model.model.layers[-1]
>>>> <keras.layers.core.Dense at 0x7fe22010e128>
I tried adding output_dim to the list of parameters I'm copying in this way, but that didn't seem to help. It gave me this error instead when I did that:
Exception: Input 0 is incompatible with layer dense_8: expected ndim=2, found ndim=3
Any idea what I'm doing wrong here?
Found the answer myself. If, instead of making the input_shape the same as the input_shape of the last layer of the old model, I make it the output_shape of the penultimate layer of the old model and specify only [1:] of that output array, it works. Code that works is as follows:
newModel.add(type(oldModel.layers[-1])(oldModel.layers[-1].output_shape,
activation=oldModel.layers[-1].activation,
input_shape=oldModel.layers[-2].output_shape[1:]))