I want to create an object-detection app based on a retrained ssd_mobilenet model I've retrained like the guy on youtube.
I chose the model ssd_mobilenet_v2_coco from the Tensorflow Model Zoo. After the retraining process I've got the model with the following structure:
- saved_model
- variables (empty folder)
- saved_model.pb
- checkpoint
- frozen_inverence_graph.pb
- model.ckpt.data-00000-of-00001
- model.ckpt.index
- model.ckpt.meta
- pipeline.config
In the same folder, I have the python script with the following code:
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model("saved_model", input_shapes={"image_tensor":[1,300,300,3]})
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
After running this code, I got the following error:
...
2019-05-24 18:46:59.811289: I tensorflow/lite/toco/import_tensorflow.cc:1324] Converting unsupported operation: TensorArrayGatherV3
2019-05-24 18:46:59.811864: I tensorflow/lite/toco/import_tensorflow.cc:1373] Unable to determine output type for op: TensorArrayGatherV3
2019-05-24 18:46:59.908207: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 1792 operators, 3033 arrays (0 quantized)
2019-05-24 18:47:00.089034: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] After Removing unused ops pass 1: 1771 operators, 2979 arrays (0 quantized)
2019-05-24 18:47:00.314681: I tensorflow/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 1771 operators, 2979 arrays (0 quantized)
2019-05-24 18:47:00.453570: F tensorflow/lite/toco/graph_transformations/resolve_constant_slice.cc:59] Check failed: dim_size >= 1 (0 vs. 1)
Is there any solution for the "Check failed: dim_size >= 1 (0 vs. 1)"?
Conversion of MobileNet SSD is a little different due to some Custom ops that are needed in the graph.
Take a look at this Medium post for the end-to-end process of training and exporting the model as a TFLite graph. For conversion, you would need to use the export_tflite_ssd_graph script.
Related
Im using the new Tensorflow object detection API.
I need to replicate training parameters used on a paper but Im a bit confused.
In the paper is stated
When training neural network models, their base confguration is similar to that used to
train on the COCO 2017 dataset. For the unambiguous comparison of the selected models, the total number of
training steps was set to 100 equal to 100′000 iterations of learning.
Inside model_main_tf2.py, which is the script used to start the training, I can read the following:
"""Creates and runs TF2 object detection models.
For local training/evaluation run:
PIPELINE_CONFIG_PATH=path/to/pipeline.config
MODEL_DIR=/tmp/model_outputs
NUM_TRAIN_STEPS=10000
SAMPLE_1_OF_N_EVAL_EXAMPLES=1
python model_main_tf2.py -- \
--model_dir=$MODEL_DIR --num_train_steps=$NUM_TRAIN_STEPS \
--sample_1_of_n_eval_examples=$SAMPLE_1_OF_N_EVAL_EXAMPLES \
--pipeline_config_path=$PIPELINE_CONFIG_PATH \
--alsologtostderr
"""
Also, you can specify the num_steps and total_steps parameters in the pipeline.config file (used by the training script):
train_config: {
batch_size: 1
sync_replicas: true
startup_delay_steps: 0
replicas_to_aggregate: 8
num_steps: 50000
optimizer {
momentum_optimizer: {
learning_rate: {
cosine_decay_learning_rate {
learning_rate_base: .16
total_steps: 50000
warmup_learning_rate: 0
warmup_steps: 2500
}
}
momentum_optimizer_value: 0.9
}
use_moving_average: false
}
So, what Im not understanding is how should I map what is written in the paper with tensorflow parameters.
What is the num steps and total_steps inside the pipeline.config file?
What is the NUM_TRAIN_STEPS argument instead?
Does it overwrite config file steps or its a completely different thing?
If more details are needed feel free to ask.
I want to run inference in C++ using a yolo3 model I trained with pytorch. I am unable to make the conversions using tracing and scripting provided by pytorch. I have this error during conversion
First diverging operator:
Node diff:
- %2 : __torch__.torch.nn.modules.container.ModuleList = prim::GetAttr[name="module_list"](%self.1)
+ %2 : __torch__.torch.nn.modules.container.___torch_mangle_139.ModuleList = prim::GetAttr[name="module_list"](%self.1)
? ++++++++++++++++++++
ERROR: Tensor-valued Constant nodes differed in value across invocations. This often indicates that the tracer has encountered untraceable code.
Node:
%358 : Tensor = prim::Constant[value=<Tensor>](), scope: __module.module_list.16.yolo_16
I made and trained a pytorch v1.4 model that predicts a sin() value (based on an example found on the web). Inference works. I then tried to compile it with TVM v0.8dev0 and llvm 10 on Ubuntu with a x86 cpu. I followed the TVM setup guide and ran some tutorials for onnx that do work.
I mainly used existing tutorials on TVM to figure out the procedure below. Note that I'm not a ML nor DataScience engineer. These were my steps:
import tvm, torch, os
from tvm import relay
state = torch.load("/home/dude/tvm/tst_state.pt") # load the trained pytorch state
import tst
m = tst.Net()
m.load_state_dict(state) # init the model with its trained state
m.eval()
sm = torch.jit.trace(m, torch.tensor([3.1415 / 4])) # convert to a scripted model
# the model only takes 1 input for inference hence [("input0", (1,))]
mod, params = tvm.relay.frontend.from_pytorch(sm, [("input0", (1,))])
mod.astext # outputs some small relay(?) script
with tvm.transform.PassContext(opt_level=1):
lib = relay.build(mod, target="llvm", target_host="llvm", params=params)
The last line gives me this error that I don't know how to solve nor where I went wrong. I hope that someone can pinpoint my mistake ...
... removed some lines here ...
[bt] (3) /home/dude/tvm/build/libtvm.so(TVMFuncCall+0x5f) [0x7f5cd65660af]
[bt] (2) /home/dude/tvm/build/libtvm.so(+0xb4f8a7) [0x7f5cd5f318a7]
[bt] (1) /home/dude/tvm/build/libtvm.so(tvm::GenericFunc::CallPacked(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*) const+0x1ab) [0x7f5cd5f315cb]
[bt] (0) /home/tvm/build/libtvm.so(+0x1180cab) [0x7f5cd6562cab]
File "/home/tvm/python/tvm/_ffi/_ctypes/packed_func.py", line 81, in cfun
rv = local_pyfunc(*pyargs)
File "/home/tvm/python/tvm/relay/op/strategy/x86.py", line 311, in dense_strategy_cpu
m, _ = inputs[0].shape
ValueError: not enough values to unpack (expected 2, got 1)
Update #1 (original question and details below):
As per the suggestion of #MatthijsHollemans below I've tried to run this by removing dynamic_axes from the initial create_onnx step below. This removed both:
Description of image feature 'input_image' has missing or non-positive width 0.
and
Input 'input_image' of layer '63' not found in any of the outputs of the preceeding layers.
Unfortunately this opens up two sub-questions:
I still want to have a functional ONNX model. Is there a more appropriate way to make H and W dynamic? Or should I be saving two versions of the ONNX model, one without dynamic_axes for the CoreML conversion, and one with for use as a valid ONNX model?
Although this solves the compilation error in xcode (specified below) it introduces the following runtime issues:
Finalizing CVPixelBuffer 0x282f4c5a0 while lock count is 1.
[espresso] [Espresso::handle_ex_plan] exception=Invalid X-dimension 1/480 status=-7
[coreml] Error binding image input buffer input_image: -7
[coreml] Failure in bindInputsAndOutputs.
I am calling this the same way I was calling the fixed size model, which does still work fine. The image dimensions are 640 x 480.
As specified below the model should accept any image between 64x64 and higher.
For flexible shape models, do I need to provide an input differently in xcode?
Original Question (parts still relevant)
I have been slowly working on converting a style transfer model from pytorch > onnx > coreml. One of the issues that has been a struggle is flexible/dynamic input + output shape.
This method (besides i/o renaming) has worked well on iOS 12 & 13 when using a static input shape.
I am using the following code to do the onnx > coreml conversion:
def create_coreml(name):
mlmodel = convert(
model="onnx/" + name + ".onnx",
preprocessing_args={'is_bgr': True},
deprocessing_args={'is_bgr': True},
image_input_names=['input_image'],
image_output_names=['stylized_image'],
minimum_ios_deployment_target='13'
)
spec = mlmodel.get_spec()
img_size_ranges = flexible_shape_utils.NeuralNetworkImageSizeRange()
img_size_ranges.add_height_range((64, -1))
img_size_ranges.add_width_range((64, -1))
flexible_shape_utils.update_image_size_range(
spec,
feature_name='input_image',
size_range=img_size_ranges)
flexible_shape_utils.update_image_size_range(
spec,
feature_name='stylized_image',
size_range=img_size_ranges)
mlmodel = coremltools.models.MLModel(spec)
mlmodel.save("mlmodel/" + name + ".mlmodel")
Although the conversion 'succeeds' there are a couple of warnings (spaces added for readability):
Translation to CoreML spec completed. Now compiling the CoreML model.
/usr/local/lib/python3.7/site-packages/coremltools/models/model.py:111:
RuntimeWarning: You will not be able to run predict() on this Core ML model. Underlying exception message was:
Error compiling model:
"Error reading protobuf spec. validator error: Description of image feature 'input_image' has missing or non-positive width 0.".
RuntimeWarning)
Model Compilation done.
/usr/local/lib/python3.7/site-packages/coremltools/models/model.py:111:
RuntimeWarning: You will not be able to run predict() on this Core ML model. Underlying exception message was:
Error compiling model:
"compiler error: Input 'input_image' of layer '63' not found in any of the outputs of the preceeding layers.
".
RuntimeWarning)
If I ignore these warnings and try to compile the model for latest targets (13.0) I get the following error in xcode:
coremlc: Error: compiler error: Input 'input_image' of layer '63' not found in any of the outputs of the preceeding layers.
Here is what the problematic area appears to look like in netron:
My main question is how can I get these two warnings out of the way?
Happy to provide any other details.
Thanks for any advice!
Below is my pytorch > onnx conversion:
def create_onnx(name):
prior = torch.load("pth/" + name + ".pth")
model = transformer.TransformerNetwork()
model.load_state_dict(prior)
dummy_input = torch.zeros(1, 3, 64, 64) # I wasn't sure what I would set the H W to here?
torch.onnx.export(model, dummy_input, "onnx/" + name + ".onnx",
verbose=True,
opset_version=10,
input_names=["input_image"], # These are being renamed from garbled originals.
output_names=["stylized_image"], # ^
dynamic_axes={'input_image':
{2: 'height', 3: 'width'},
'stylized_image':
{2: 'height', 3: 'width'}}
)
onnx.save_model(original_model, "onnx/" + name + ".onnx")
I am using Matlab 2013b and the econometrics toolbox to learn some ARIMA models.
When I want to specify AR lags in the ARIMA model as follows:
%Estimate simple ARMA model
model1 = arima('ARLags',[1 24],'MALags',0,'D',0);
EstMdl1 = estimate(model1,learningSet');
Then everything is fine when I estimate the model
If now I use
%Estimate simple ARMA model
model1 = arima('ARLags',[1 24],'MALags',1,'D',0);
EstMdl1 = estimate(model1,learningSet');
then the following error is issued:
Error using optimset (line 184)
Invalid value for OPTIONS parameter MaxNodes: must be a real non-negative integer.
Error in internal.econ.arma0 (line 195)
options = optimset('lsqlin');
Error in arima/estimate (line 864)
[AR0, MA0, constant, variance] = internal.econ.arma0(I(YData), LagOpAR0, LagOpMA0);
I am a bit puzzled about this and am looking for a workaround, if not an explanation of what is happening