how to change CNN model to Wide convolution in pytorch? - pytorch

I am learning CNN, I want to change CNN model with Wide Convolution by pytorch, who can help?
self.conv23 = nn.Conv2d(Ci, len(Ks) * Co, (3, Co), padding=1)
Traceback (most recent call last):
File "E:/workspace/pycharmworkspace/cnn-text-classification-pytorch-update/main.py", line 137, in <module>
train.train(train_iter, dev_iter, cnn, args)
File "E:\workspace\pycharmworkspace\cnn-text-classification-pytorch-update\train.py", line 40, in train
logit = model(feature)
File "C:\Users\bamtercelboo\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "E:\workspace\pycharmworkspace\cnn-text-classification-pytorch-update\model.py", line 206, in forward
x21 = self.conv(x11, self.conv23) #(N,Co)
File "E:\workspace\pycharmworkspace\cnn-text-classification-pytorch-update\model.py", line 91, in conv
x = F.relu(conv(x)).squeeze(3) # (N,Co,W)
File "C:\Users\bamtercelboo\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "C:\Users\bamtercelboo\Anaconda3\lib\site-packages\torch\nn\modules\conv.py", line 237, in forward
self.padding, self.dilation, self.groups)
File "C:\Users\bamtercelboo\Anaconda3\lib\site-packages\torch\nn\functional.py", line 43, in conv2d
return f(input, weight, bias)
RuntimeError: kernel size should be greater than zero, but got kT: 3 kH: 200 kW: 0 at d:\downloads\pytorch-master-1\torch\lib\thnn\generic/VolumetricConvolutionMM.c:23

I think what you might need is adjusting the groups parameter in the Conv2D layers.

Related

IndexError: too many indices for tensor of dimension 3

I am trying to convert Detectron2 Model into an onnx model format (pytorch to onnx). This is the code I am using (note I am using all necessary imports and to my understanding my installations are correct). Here is my code:
cfg = get_cfg()
cfg.MODEL.DEVICE = 'cpu'
cfg.merge_from_file(model_zoo.get_config_file("COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml"))
model = build_model(cfg)
aug = T.ResizeShortestEdge([cfg.INPUT.MIN_SIZE_TEST, cfg.INPUT.MIN_SIZE_TEST],
cfg.INPUT.MAX_SIZE_TEST)
height, width = im.shape[:2]
image = aug.get_transform(im).apply_image(im)
image = torch.as_tensor(image.astype("float32").transpose(2, 0, 1))
inputs2 = {"image": image}
print(image.shape)
output = export_onnx_model(model, image)
print("output:", output)
onnx.save(output, "/home/ecoation/Documents/model/deploy.onnx")
And here is the bug I am getting:
Traceback (most recent call last):
File "/home/ecoation/Documents/detectronCode.py", line 57, in <module>
output = export_onnx_model(model, image)
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/detectron2/export/caffe2_export.py", line 49, in export_onnx_model
torch.onnx.export(
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/torch/onnx/__init__.py", line 272, in export
return utils.export(model, args, f, export_params, verbose, training,
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/torch/onnx/utils.py", line 88, in export
_export(model, args, f, export_params, verbose, training, input_names, output_names,
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/torch/onnx/utils.py", line 694, in _export
_model_to_graph(model, args, verbose, input_names,
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/torch/onnx/utils.py", line 457, in _model_to_graph
graph, params, torch_out, module = _create_jit_graph(model, args,
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/torch/onnx/utils.py", line 420, in _create_jit_graph
graph, torch_out = _trace_and_get_graph_from_model(model, args)
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/torch/jit/_trace.py", line 116, in wrapper
outs.append(self.inner(*trace_inputs))
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 887, in _call_impl
result = self._slow_forward(*input, **kwargs)
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 860, in _slow_forward
result = self.forward(*input, **kwargs)
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 150, in forward
return self.inference(batched_inputs)
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 203, in inference
images = self.preprocess_image(batched_inputs)
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 228, in preprocess_image
images = [self._move_to_current_device(x["image"]) for x in batched_inputs]
File "/home/ecoation/anaconda3/envs/detectron2/lib/python3.8/site-packages/detectron2/modeling/meta_arch/rcnn.py", line 228, in <listcomp>
images = [self._move_to_current_device(x["image"]) for x in batched_inputs]
IndexError: too many indices for tensor of dimension 2
Feel free to take a look, any help in turning this into a onnx format model is much appreciated. If it helps, I am on Ubuntu 18.04 and all I'm trying to do is take a pretrained simple detectron2 model from regular to onnx format. I'm hoping to get it so I can do custom, but any help with pretrained would be much appreciated.
Edit:
Image size is this :
torch.Size([3, 800, 1067])

Input to reshape is a tensor with 788175 values, but the requested shape has 1050900

I am importing in some arrays of data to train on but tensorflow is outputting below error.
inp = open('train.csv',"rb")
X = pickle.load(inp)
X = X/255.0
X = np.array(X)
model = keras.Sequential([
keras.layers.Flatten(input_shape=(113, 75, 3)),
keras.layers.Dense(75, activation=tf.nn.relu),
keras.layers.Dense(50, activation=tf.nn.relu),
keras.layers.Dense(75, activation=tf.nn.relu),
keras.layers.Dense(25425, activation=tf.nn.softmax),
keras.layers.Reshape((113, 75, 4))
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(X, X, epochs=5)
I should be able to create an autoencoder but the program outputs this:
Traceback (most recent call last):
File "C:\Users\dalto\Documents\geo4\train.py", line 24, in <module>
model.fit(X, X, epochs=5)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training.py", line 643, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 664, in fit
steps_name='steps_per_epoch')
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 383, in model_iteration
batch_outs = f(ins_batch)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\backend.py", line 3510, in __call__
outputs = self._graph_fn(*converted_inputs)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\function.py", line 572, in __call__
return self._call_flat(args)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\function.py", line 671, in _call_flat
outputs = self._inference_function.call(ctx, args)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\function.py", line 445, in call
ctx=ctx)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\execute.py", line 67, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 788175 values, but the requested shape has 1050900
[[node reshape/Reshape (defined at C:\Users\dalto\Documents\geo4\train.py:24) ]] [Op:__inference_keras_scratch_graph_922]
Function call stack:
keras_scratch_graph
If I change the Reshape to (113, 75, 3) I get this it doesn't fix the error it just changes it:
Traceback (most recent call last):
File "C:\Users\dalto\Documents\geo4\train.py", line 24, in <module>
model.fit(X, X, epochs=5)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training.py", line 643, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 664, in fit
steps_name='steps_per_epoch')
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 383, in model_iteration
batch_outs = f(ins_batch)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\keras\backend.py", line 3510, in __call__
outputs = self._graph_fn(*converted_inputs)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\function.py", line 572, in __call__
return self._call_flat(args)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\function.py", line 671, in _call_flat
outputs = self._inference_function.call(ctx, args)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\function.py", line 445, in call
ctx=ctx)
File "C:\Users\dalto\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\eager\execute.py", line 67, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible
shapes: [31,113,75] vs. [31,113,75,3]
[[node metrics/accuracy/Equal (defined at
C:\Users\dalto\Documents\geo4\train.py:24) ]] [Op:__inference_keras_scratch_graph_922]
The input and output size after reshape must be same. So, you'll have to use (113, 75, 3) instead of (113, 75, 4).
Now, by using (113, 75, 3), you're getting the unequal error because you're using sparse_categorical_crossentropy as your loss function, you should instead use categorical_crossentropy.
The basic difference between these is that sparse_categorical_crossentropy works when you have direct integers as your label, and categorical_crossentropy works when you have one-hot encoded labels.
Corrected:
inp = open('train.csv',"rb")
X = pickle.load(inp)
X = X/255.0
X = np.array(X)
model = keras.Sequential([
keras.layers.Flatten(input_shape=(113, 75, 3)),
keras.layers.Dense(75, activation=tf.nn.relu),
keras.layers.Dense(50, activation=tf.nn.relu),
keras.layers.Dense(75, activation=tf.nn.relu),
keras.layers.Dense(25425, activation=tf.nn.softmax),
keras.layers.Reshape((113, 75, 4))
])
model.compile(optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit(X, X, epochs=5)

Prediction from tensorflow model fails

I am extremely new to tensorflow and trying to learn how to save and load a previously trained model. I created a simple model using Estimator and trained it.
classifier = tf.estimator.Estimator(model_fn=bag_of_words_model)
# Train
train_input_fn = tf.estimator.inputs.numpy_input_fn(
x={"words": x_train}, # x_train is 2D numpy array of shape (26, 5)
y=y_train, # y_train is 1D panda series of length 26
batch_size=1000,
num_epochs=None,
shuffle=True)
classifier.train(input_fn=train_input_fn, steps=300)
I then try to save the model:
def serving_input_receiver_fn():
serialized_tf_example = tf.placeholder(dtype=tf.int64, shape=(None, 5), name='words')
receiver_tensors = {"predictor_inputs": serialized_tf_example}
features = {"words": tf.placeholder(tf.int64, shape=(None, 5))}
return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)
full_model_dir = classifier.export_savedmodel(export_dir_base="E:/models/", serving_input_receiver_fn=serving_input_receiver_fn)
I have actually copied the serving_input_receiver_fn from this similar question. I don't understand exactly what is going on in that function. But this stores my model in E:/models/<some time stamp>.
I now try to load this saved model:
from tensorflow.contrib import predictor
classifier = predictor.from_saved_model("E:\\models\\<some time stamp>")
The models perfectly loaded. Hereafter, I am struck on how to use this classifier object to get predictions on new data. I have followed a guide here to achieve it but couldn't do it :(. Here is what I did:
predictions = classifier({'predictor_inputs': x_test})["output"] # x_test is 2D numpy array same like x_train in the training part
I get error as follows:
2019-01-10 12:43:38.603506: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
INFO:tensorflow:Restoring parameters from E:\models\1547101005\variables\variables
Traceback (most recent call last):
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call
return fn(*args)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype int64 and shape [?,5]
[[{{node Placeholder}} = Placeholder[dtype=DT_INT64, shape=[?,5], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:/ml_classif/tensorflow_bow_with_prob/load_model.py", line 85, in <module>
predictions = classifier({'predictor_inputs': x_test})["output"]
File "E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\predictor.py", line 77, in __call__
return self._session.run(fetches=self.fetch_tensors, feed_dict=feed_dict)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
run_metadata_ptr)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
run_metadata)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype int64 and shape [?,5]
[[node Placeholder (defined at E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\saved_model_predictor.py:153) = Placeholder[dtype=DT_INT64, shape=[?,5], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Caused by op 'Placeholder', defined at:
File "E:/ml_classif/tensorflow_bow_with_prob/load_model.py", line 82, in <module>
classifier = predictor.from_saved_model("E:\\models\\1547101005")
File "E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\predictor_factories.py", line 153, in from_saved_model
config=config)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\saved_model_predictor.py", line 153, in __init__
loader.load(self._session, tags.split(','), export_dir)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\saved_model\loader_impl.py", line 197, in load
return loader.load(sess, tags, import_scope, **saver_kwargs)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\saved_model\loader_impl.py", line 350, in load
**saver_kwargs)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\saved_model\loader_impl.py", line 278, in load_graph
meta_graph_def, import_scope=import_scope, **saver_kwargs)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\training\saver.py", line 1696, in _import_meta_graph_with_return_elements
**kwargs))
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\meta_graph.py", line 806, in import_scoped_meta_graph_with_return_elements
return_elements=return_elements)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\importer.py", line 442, in import_graph_def
_ProcessNewOps(graph)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\importer.py", line 234, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\ops.py", line 3440, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\ops.py", line 3440, in <listcomp>
for c_op in c_api_util.new_tf_operations(self)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\ops.py", line 3299, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "E:\ml_classif\venv\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in __init__
self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'Placeholder' with dtype int64 and shape [?,5]
[[node Placeholder (defined at E:\ml_classif\venv\lib\site-packages\tensorflow\contrib\predictor\saved_model_predictor.py:153) = Placeholder[dtype=DT_INT64, shape=[?,5], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
It says that I have to feed value to the placeholder (I think the one defined in serving_input_receiver_fn). I have no idea how to do it without using a Session object of tensorflow.
Please feel free to ask for more information if required.
After somewhat vague understanding of serving_input_receiver_fn, I figured out that the features must not be a placeholder as it creates 2 placeholders (1 for serialized_tf_example and the other for features). I modified the function as follows (the changes is just for the features variable):
def serving_input_receiver_fn():
serialized_tf_example = tf.placeholder(dtype=tf.int64, shape=(None, 5), name='words')
receiver_tensors = {"predictor_inputs": serialized_tf_example}
features = {"words": tf.tile(serialized_tf_example, multiples=[1, 1])} # Changed this
return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)
When I try to predict the output from the loaded model, I get no error now. It works! Only thing is that the output is incorrect (for which I am posting a new question :) ).

ValueError: Shapes (?, 83) and (?, 128) are incompatible

I'm getting the following error when trying to create a LSTM using TensorFlow:
ValueError: Shapes (?, 83) and (?, 128) are incompatible
The input to the model have the following shape:
(batch_size, time, features) / (50, 68, 83)
Here is the relevant code for the model:
x_text = tf.placeholder(tf.float32, [None, *text.shape[1:]])
cells = tf.contrib.rnn.MultiRNNCell([tf.contrib.rnn.ResidualWrapper(
tf.contrib.rnn.BasicLSTMCell(num_units=units))
for units in [128, 256]
])
text_outputs, text_state = tf.nn.dynamic_rnn(
cell=cells,
inputs=x_text,
dtype=tf.float32,
)
I've tried for so long to figure out what's wrong, but I can't. I've search the entire internett (no, really!) and no one seems to be having the same problem where shapes (?, a) and (?, b) are the problem, but rather all other combinations where the solutions has not helped.
Oh - and here's the stack trace:
Traceback (most recent call last):
File "residual_hierachical_rnn.py", line 97, in <module>
dtype=tf.float32,
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn.py", line 627, in dynamic_rnn
dtype=dtype)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn.py", line 824, in _dynamic_rnn_loop
swap_memory=swap_memory)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 3224, in while_loop
result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 2956, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 2893, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/control_flow_ops.py", line 3194, in <lambda>
body = lambda i, lv: (i + 1, orig_body(*lv))
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn.py", line 795, in _time_step
(output, new_state) = call_cell()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn.py", line 781, in <lambda>
call_cell = lambda: cell(input_t, state)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 232, in __call__
return super(RNNCell, self).__call__(inputs, state)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/layers/base.py", line 717, in __call__
outputs = self.call(inputs, *args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1292, in call
cur_inp, new_state = cell(cur_inp, cur_state)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1168, in __call__
res_outputs = (self._residual_fn or default_residual_fn)(inputs, outputs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1166, in default_residual_fn
nest.map_structure(assert_shape_match, inputs, outputs)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/nest.py", line 375, in map_structure
structure[0], [func(*x) for x in entries])
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/nest.py", line 375, in <listcomp>
structure[0], [func(*x) for x in entries])
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1163, in assert_shape_match
inp.get_shape().assert_is_compatible_with(out.get_shape())
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/tensor_shape.py", line 844, in assert_is_compatible_with
raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (?, 83) and (?, 128) are incompatible
Thank you so much in advance for any help!!!
As MPKenning says, your LSTM cells should match your input features.
But the fact that you're using a ResidualWrapper forces you to keep the same depth because what it does is summing up the inputs and the outputs of the cells.
If you remove the ResidualWrapper it should work with [83, 256]:
cells = tf.contrib.rnn.MultiRNNCell([
tf.contrib.rnn.BasicLSTMCell(num_units=units)
for units in [83, 256]
])
The unit-size of your LSTM cells should match the number of features. To fix this, use [83, 256].
Also, I'm aware that the connections between the LSTM layers are fully connected, but from what I've been told it's better to keep the unit size in the layers consistent to make things less confusing. In other words, consider using [83, 83] for your unit-sizes.

Pytorch model weight type conversion

I'm trying to do inference on FlowNet2-C model loading from file.
However, I met some data type problem. How can I resolve it?
Source code
FlowNet2-C pre-trained model
$ python main.py
Initializing Datasets
[0.000s] Loading checkpoint '/notebooks/data/model/FlowNet2-C_checkpoint.pth.tar'
[1.293s] Loaded checkpoint '/notebooks/data/model/FlowNet2-C_checkpoint.pth.tar' (at epoch 0)
(1L, 6L, 384L, 512L)
<class 'torch.autograd.variable.Variable'>
[1.642s] Operation failed
Traceback (most recent call last):
File "main.py", line 102, in <module>
main()
File "main.py", line 98, in main
summary(input_size, model)
File "main.py", line 61, in summary
model(x)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 357, in __call__
result = self.forward(*input, **kwargs)
File "/notebooks/data/vinet/FlowNetC.py", line 75, in forward
out_conv1a = self.conv1(x1)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 357, in __call__
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/container.py", line 67, in forward
input = module(input)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 357, in __call__
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/conv.py", line 282, in forward
self.padding, self.dilation, self.groups)
File "/usr/local/lib/python2.7/dist-packages/torch/nn/functional.py", line 90, in conv2d
return f(input, weight, bias)
RuntimeError: Input type (CUDAFloatTensor) and weight type (CPUFloatTensor) should be the same
Maybe that is because your model and input x to the model has different data types. It seems that the model parameters have been moved to GPU, but your input x is on GPU.
You can try to use model.cuda() after line 94, which will put the model on the GPU. Then the error should disappear.

Resources