When I have trained a model and save the params as hdf5 file, and then I try to evaluate the performance of the model on test_dataset, but there is something wrong in param metrics of model.compile, if I set metrics as
model.compile(optimizer=adam,
loss=losses.sparse_categorical_crossentropy,
metrics=[cus_acc, miou])
It will occurs an Error as follow:
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [`labels` out of bound] [Condition x < y did not hold element-wise:] [x (metrics/cus_acc/confusion_matrix/control_dependency:0) = ] [107 118 135...] [y (metrics/cus_acc/confusion_matrix/Cast_2:0) = ] [2] [[Node: metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_STRING, DT_INT64, DT_STRING, DT_INT64], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch/_3827, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_0, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_1, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_2, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_1/_3829, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/data_4, metrics/cus_acc/confusion_matrix/assert_less/Assert/AssertGuard/Assert/Switch_2/_3831)]]
Notes that optimizer adam have been defined, and cus_acc and miou are customized metrics. If delete the metrics it works, so I think there must be something wrong in it, miou and cus_acc are both calculate by the confusion_matrix.
My Question is what makes the error occurs and how to use evaluate_generator to evaluate model performance in keras, if you can provide example code that the best~
Any help would be appreciated. Thanks in advance. :D
It has been solved.
The error occurs when creating confusion_matrix, the nb_classes i.e. 0, 1, 2 must match the label in the ground-truth 0, 1, 2. For example, the ground-truth is not preprocessed and the value of pixels are 0, 127, 255. And then the InvalidArgumentError: assertion failed will occur.
What made my code wrong is not due to the customized metrics, it is when I create a data_generator for ground-truth, the param directory was set the same as the images.
Related
I am trying to perform dimensionality reduction using PCA, where outputs is a list of tensors where each tensor has a shape of (1, 3, 32,32). Here is the code:
from sklearn.decomposition import PCA
pca = PCA(10)
pca_result = pca.fit_transform(output)
But I keep getting this error, regardless of whatever I tried:
ValueError: only one element tensors can be converted to Python scalars
I know that the tensors with size(1,3, 32,32) is making the issue, since its looking for 1 element as the error puts it, but do not know how to solve it.
I have tried flattening each tensor with looping over output (don't know if its the right way of solving this issue), using the following code but it leads to error in pca:
new_outputs = []
for i in outputs:
for j in i:
j = j.cpu()
j = j.detach().numpy()
j = j.flatten()
new_outputs.append(j)
pca_result = pca.fit_transform(new_output)
I would appreciate if anybody can help with this error whether the flattening approach I took, is correct.
PS:I have read the existing posts (post1,post2) discussing this error but none of them could solve my problem.
Assuming your Tensors are stored in a matrix with shape like (10, 3, 32, 32) where 10 corresponds to number of Tensors, you should flatten each like that:
import torch
from sklearn.decomposition import PCA
data= torch.rand((10, 3, 32, 32))
pca = PCA(10)
pca_result = pca.fit_transform(data.flatten(start_dim=1))
data.flatten(start_dim=1) makes your data to be in shape (10, 3*32*32)
The error you posted is actually related to one of the post you linked. The PCA estimator expects array-like object with fit() method and you provided a list of Tensors.
I want to generate attack samples via the following steps:
Find a pre-trained CNN classification model, whose input is X and output is P(y|X), and the most possible result of X is y.
I want to input X' and get y_fool, where X' is not far away from X and y_fool is not equal to y
The steps for getting X' is:enter image description here
How can I get the partial derivative described in the image?
Here is my code but I got None: (The model is Vgg16)
x = torch.autograd.Variable(image, requires_grad=True)
output = model(image)
prob = nn.functional.softmax(output[0], dim=0)
prob.backward(torch.ones(prob.size()))
print(x.grad)
How should I modify my codes? Could someone help me? I would be absolutely grateful.
Here, the point is to backpropagate a "false" example through the network, in other words you need to maximize one particular coordinate of your output which does not correspond to the actual label of x.
Let's say for example that your model outputs N-dimensional vectors, that x label should be [1, 0, 0, ...] and that we will try to make the model actually predict [0, 1, 0, 0, ...] (so y_fool actually has its second coordinate set to 1, instead of the first one).
Quick note on the side : Variable is deprecated, just set the requires_grad flag to True. So you get :
x = torch.tensor(image, requires_grad=True)
output = model(x)
# If the model is well trained, prob_vector[1] should be almost 0 at the beginning
prob_vector = nn.functional.softmax(output, dim=0)
# We want to fool the model and maximize this coordinate instead of prob_vector[0]
fool_prob = prob_vector[1]
# fool_prob is a scalar tensor, so we can backward it easy
fool_prob.backward()
# and you should have your gradients :
print(x.grad)
After that, if you want to use an optimizer in your loop to modify x, remember that pytorch optimizer.step method tries to minimize the loss, whereas you want to maximize it. So either you use a negative learning rate or you change the backprop sign :
# Maximizing a scalar is minimizing its opposite
(-fool_prob).backward()
I am trying to implement the UNet architecture in Pytorch. When I print the model using print(model) I get the correct architecture:
but when I try to print the summary using (or any other input size for that matter):
from torchsummary import summary
summary(model, input_size=(13, 572, 572))
I get an error:
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 70 and 71 in dimension 2 at /Users/distiller/project/conda/conda-bld/pytorch_1579022061893/work/aten/src/TH/generic/THTensor.cpp:612
However, it works perfectly if I give the input_size as input_size=(3, 224, 224))( like it worked for this person here). I am so baffled.
Can someone help me what's wrong?
Edit: I have used the model architecture from here.
This UNet architecture you provided doesn't support that shape (unless the depth parameter is <= 3). Ultimately the reason for this is that the size of a downsampling operation isn't invertible since multiple input shapes map to the same output shape. For example consider
>> torch.nn.functional.max_pool2d(torch.zeros(1, 1, 10, 10), 2).shape
torch.Size([1, 1, 5, 5])
>> torch.nn.functional.max_pool2d(torch.zeros(1, 1, 11, 11), 2).shape
torch.Size([1, 1, 5, 5])
So the question is, given only the output shape is 5x5, what was the shape of the input? Was it 10x10 or 11x11? This same phenomenon applies to downsampling via strided convolutions.
The problem is that the UNet class tries to combine features from the downsampling half to the network to the features in the upsampling half. If it "guesses wrong" about the original shape during upsampling then you will receive a dimension mismatch error.
To avoid this issue you'll need to ensure that the height and width of your input data are multiples of 2**(depth-1). So, for the default depth=5 you need the input image height and width to be a multiple of 16 (e.g. 560 or 576). Alternatively, since 572 is divisible by 4 then you could also set depth=3 to make it work.
I have saved more than 1000 models for each item. Now I need to load all these models into memory (a dataframe) to do predictions. If I just use "for" loop to load these models, each loading will be 3 seconds slower than the previous model loading. So I try to use multiprocessing.pool (ThreadPool).
But, strangely, using ThreadPool will cause the prediction "ValueError: Tensor Tensor". If using normal loading, the prediction is fine.
I tried thread also got error msg
#following code will lead to ValueError
from multiprocessing.pool import ThreadPool as Pool
def load_model(stock):
model_pred.at[0, stock] = keras.models.load_model (
'C:/Users/chenp/Documents/rqpro/models/{}_model.h5'.format (stock))
pool = Pool(processes=16)
for stock in trade_stocks['stock']:
pool.map (load_model, (stock,))
#Prediction
for stock in trade_stocks['stock']:
model = model_pred.loc[0, stock]
prediction = model.predict(pred_data)
#Get following msg:
ValueError: Tensor Tensor("dense_9/Softmax:0", shape=(?, 2), dtype=float32) is not an element of this graph.
#Normal code but too low efficient
for stock in trade_stocks['stock']:
model_pred.at[0, stock] = keras.models.load_model(
'C:/Users/chenp/Documents/rqpro/models/{}_model.h5'.format(stock))
#Get following msg:
ValueError: Tensor Tensor("dense_9/Softmax:0", shape=(?, 2), dtype=float32) is not an element of this graph.
This happens as Keras is not thread safe. For solving this problem, please use _make_predict_function() before predicting. For detailed answer, please check
I'm using tensorflow to build a recommendationg system using collaborative filtering algorithm.
Due to memory usage I have to use sparse matrices.
#Arbitrary number of items are rated by arbitrary number of users
ratings = tf.sparse_placeholder(tf.float32, shape=[None, None])
ratings = tf.sparse_reorder(ratings)
With my feed_dict,
ratings.dense_shape == (45776, 60184)
pred = tf.matmul(items_features, user_preferences, name='Prediction') + global_mean
I have hardcoded so pred.shape == (45776, 60184)
So how comes when I try to collect predictions to later calculate the cost
pred_values = tf.gather_nd(pred, ratings.indices)
I get error
InvalidArgumentError (see above for traceback): flat indices[2714679, :] = [48375, 2227] does not index into param (shape: [45776,60184]).
?
It turns out I did mess up with dimensions. I fed indices in the opposite order than I thought, i.e., for shape (60184,45776) instead of needed (45776,60184).
Having an error telling that my flat indices[2714679, :] are indexing [48375, 2227] - shape that does not appear - didn't help.