I am trying to recreate binary image using a UNet. But the classification labels are extremely skewed (~10% are 0s and rest are 1s in the image). So I got the weights for each class using skearn. This is what I do:
wts = np.array([5.76901408, 0.54744721])
class_wts = dict(enumerate(wts))
Now, when I put this in fit_generator:
history_sgd = model.fit_generator(training_generator, validation_data=valid_generator, steps_per_epoch=train_steps, validation_steps=valid_steps, epochs=epochs, verbose=1, class_weight = class_wts)
I get the following error:
ValueError: class_weight not supported for 3+ dimensional targets.
Could this be because of the generator shape? Because that would be of the shape (batch_size, <image_size>).
Moreover, when I use class_weight as numpy array instead of dictionary, the code works. Why does that happen? I checked a lot of resources online but I cannot figure the difference between the two and why the numpy array format works but not the dictionary.
Related
I have a recurrent network (RNN) whose task is to learn to classify vectors (float32) in two classes. My model is really simple so far:
model = Sequential([
SimpleRNN(units=10, input_shape=(None, len_vector)),
Dense(1, activation="relu")
])
model.compile(loss='mse', optimizer='Adam', metrics=['accuracy'])
history = model.fit(X_train, y_train, epochs=30)
To train this network, I create a dataset with 1000 instances of sequences of vectors. When I create sequences with the same length each, the training works perfectly and the dataset has shape:
[<number of sequences>, <number of vectors in each sequence>, <number of floats in each vector>]
The problem is that my model must be able to work on sequence with various length. I don't know how (or even if it is possible) to create a numpy array where one dimension is not constant.
While searching a solution, I saw that setting the array dtype=object made it possible to assign list of different shapes to element of a numpy array, but the keras model will only accept dtype="float32".
Is there a way I can make this numpy array dataset? Or should I change the algorithm to train the model? Or is the only solution to pad sequences with nul vectors to unify their length?
(Thanks for the help. I'm fairly new to deep learning so I apologize if I'm asking for something obvious.)
Use Ragged Tensors, they provide you to make variable length inputs,
import numpy as np
_input = tf.keras.layers.Input(shape=(None, 100))
lstm = tf.keras.layers.LSTM(20,)(_input)
func = tf.keras.backend.function(inputs=_input, outputs=lstm)
rt = tf.ragged.constant([np.random.randn(1,34,100),
np.random.randn(1,55,100) ,
np.random.randn(1,60,100) ,
np.random.randn(1,70,100)])
func(rt[1])
I'm training a network with MRI images and I wanted to use SSIM as loss function. Till now I was using MSE, and everything was working fine. But when I tried to use SSIM (tf.image.ssim), I get a bunch of these warining messages:
/usr/local/lib/python3.7/dist-packages/matplotlib/image.py:397: UserWarning: Warning: converting a masked element to nan.
dv = (np.float64(self.norm.vmax) -
/usr/local/lib/python3.7/dist-packages/matplotlib/image.py:398: UserWarning: Warning: converting a masked element to nan.
np.float64(self.norm.vmin))
/usr/local/lib/python3.7/dist-packages/matplotlib/image.py:405: UserWarning: Warning: converting a masked element to nan.
a_min = np.float64(newmin)
/usr/local/lib/python3.7/dist-packages/matplotlib/image.py:410: UserWarning: Warning: converting a masked element to nan.
a_max = np.float64(newmax)
/usr/local/lib/python3.7/dist-packages/matplotlib/colors.py:933: UserWarning: Warning: converting a masked element to nan.
dtype = np.min_scalar_type(value)
/usr/local/lib/python3.7/dist-packages/numpy/ma/core.py:713: UserWarning: Warning: converting a masked element to nan.
data = np.array(a, copy=False, subok=subok)
I code is running anyway but no figure is being produced. I am not sure what's happening here or where should I look. I am using tensorflow 2.4.0.
I am attaching a summary of my code my code here:
generator = Generator() #An u-net defined in tf.keras
gen_learningrate = 5e-4
generator_optimizer = tf.keras.optimizers.Adam(gen_learningrate, beta_1=0.9, beta_2=0.999, epsilon=1e-8)
# Generator loss
def generator_loss(gen_output, target):
# SSIM loss
loss = - tf.reduce_mean(tf.image.ssim(target, gen_output, 2))
return loss
#tf.function
def train_step(input_image, target, epoch):
with tf.GradientTape() as gen_tape, tf.GradientTape() as disc_tape:
gen_output = generator(input_image, training=True)
loss = generator_loss(gen_output, target)
generator_gradients = gen_tape.gradient(loss, generator.trainable_variables)
generator_optimizer.apply_gradients(zip(generator_gradients,
generator.trainable_variables))
return loss
def fit(train_ds, epochs, test_ds):
for input_image, target in train_ds:
loss = train_step(input_image,target,epoch)
fit(train_dataset, EPOCHS, test_dataset)
I have explored a little bit and noticed most people using tf.image.ssim() as loss function have used tf.train() from tensorflow 1.0 or model.fit() from tf.keras. I suspect the NaN value returned has something to do with GradientTape() function but I'm not sure how.
In my experience this warning is typically related to attempting plotting a point with a coordinate at infinity. Of course you should really show us more code for us to help you effectively.
You may get your network prediction so near with real image, so it give an Infinity (and NaN if you perform some operations with it).
Be careful to use it.
After training model with ImageDataGenerator(1/255.), do I need to rescale image before predicting ?
I thought it is necessary but experiment result said NO.
I trained a Resnet50 model which has 37 class on top layer.
Model was trained with ImageDataGenerator like this.
datagen = ImageDataGenerator(rescale=1./255)
generator=datagen.flow_from_directory(
directory=os.path.join(os.getcwd(), data_folder),
target_size=(224,224),
batch_size=256,
classes=None,
class_mode='categorical')
history = model.fit_generator(generator, steps_per_epoch=generator.n / 256, epochs=10)
Accuracy achieved 98% after 10 epochs on my train dataset.
The problem is, when i tried to predict each image in TRAIN dataset, prediction was wrong ( result is 33 whatever input image was )
img_p = './data/pets/shiba_inu/shiba_inu_27.jpg'
img = cv2.imread(img_p, cv2.IMREAD_COLOR)
img = cv2.resize(img, (224,224))
img_arr = np.zeros((1,224,224,3))
img_arr[0, :, :, :] = img / 255.
pred = model.predict(img_arr)
yhat = np.argmax(pred, axis=1)
yhat is 5, but y is 33
When I replace this line
img_arr[0, :, :, :] = img / 255.
by this
img_arr[0, :, :, :] = img
yhat is exactly 33.
Someone might suggest to use predict_generator() instead of predict(), but I want to understand what I did wrong here.
I knew what's wrong here.
I'm using Imagenet pretrained model, which DO NOT rescale image by divide it to 255. I have to use resnet50.preprocess_input before train/test.
preprocess_input function can be found here.
https://github.com/keras-team/keras-applications/blob/master/keras_applications/imagenet_utils.py
You must do every preprocessing that you do on train data, on each data that you want to feed to your trained network. actually when, for example, you rescale train images and train a network, your network train to get a matrix with entries between 0 and 1 and find the proper category. so if after training phase, you feed an image without rescaling, you feed a matrix with entries between 0 and 255 to your trained network while your network did not learn how treat with such matrix.
If you are following pre-processing exactly same as at the time of training then, you might look at the part of your code where you are predicting class using yhat = np.argmax(pred, axis=1) my hunch is that there might be class mismatch in accordance to indexing, to check how your classes are indexed when you use flow_from_directory use class_map = generator.class_indices this will return you a dictionary which will show you how your classes are mapped against index.
Note: The reason I state this because I've faced similar problem, using Keras flow_from_directory doesn't sort classes and hence it's quite possible that your prediction class 1 lies on the index 10 while np.argmax will return you class 1'.
I am new to Keras and Tensorflow. I want to add a penalty to my categorical cross entropy loss function based on some of the outputs in the network. Specifically, I decompose the outputs of a fully connected layer into 8 partitions want these outputs to be orthogonal. So I append the activations into a list and convert it into a stack by using Keras backend. Here is how I partitioned the activations:
for i in range(8):
x_sub = Lambda(lambda x: x[:,i*128:i*128+128])(x)
features.append(x_sub)
#convert batch of feature lists in to 8x(128*batch_size) keras tensor
outs.append(Lambda(lambda x: K.reshape(K.stack(x, axis=0), (8, -1)))(features))
net = Model(inputs=[net.input], outputs=outs)
And then I define the loss as follows:
def OrthLoss(features):
W = K.l2_normalize(features, axis=1)
diff = K.dot(W, K.transpose(W)) - K.eye(8)
return K.mean(diff)
However, this does not seem to converge. Is this a right way to accomplish this? I first tried to enforce the orthogonality as a regularizer on weights but as far as I understand Keras performs regularization on each layer separately and found no way to define a regularizer on multiple weights.
my question is quite closely related to this question but also goes beyond it.
I am trying to implement the following LSTM in Keras where
the number of timesteps be nb_tsteps=10
the number of input features is nb_feat=40
the number of LSTM cells at each time step is 120
the LSTM layer is followed by TimeDistributedDense layers
From the question referenced above I understand that I have to present the input data as
nb_samples, 10, 40
where I get nb_samples by rolling a window of length nb_tsteps=10 across the original timeseries of shape (5932720, 40). The code is hence
model = Sequential()
model.add(LSTM(120, input_shape=(X_train.shape[1], X_train.shape[2]),
return_sequences=True, consume_less='gpu'))
model.add(TimeDistributed(Dense(50, activation='relu')))
model.add(Dropout(0.2))
model.add(TimeDistributed(Dense(20, activation='relu')))
model.add(Dropout(0.2))
model.add(TimeDistributed(Dense(10, activation='relu')))
model.add(Dropout(0.2))
model.add(TimeDistributed(Dense(3, activation='relu')))
model.add(TimeDistributed(Dense(1, activation='sigmoid')))
Now to my question (assuming the above is correct so far):
The binary responses (0/1) are heavily imbalanced and I need to pass a class_weight dictionary like cw = {0: 1, 1: 25} to model.fit(). However I get an exception class_weight not supported for 3+ dimensional targets. This is because I present the response data as (nb_samples, 1, 1). If I reshape it into a 2D array (nb_samples, 1) I get the exception Error when checking model target: expected timedistributed_5 to have 3 dimensions, but got array with shape (5932720, 1).
Thanks a lot for any help!
I think you should use sample_weight with sample_weight_mode='temporal'.
From the Keras docs:
sample_weight: Numpy array of weights for the training samples, used
for scaling the loss function (during training only). You can either
pass a flat (1D) Numpy array with the same length as the input samples
(1:1 mapping between weights and samples), or in the case of temporal
data, you can pass a 2D array with shape (samples, sequence_length),
to apply a different weight to every timestep of every sample. In this
case you should make sure to specify sample_weight_mode="temporal" in
compile().
In your case you would need to supply a 2D array with the same shape as your labels.
If this is still an issue.. I think the TimeDistributed Layer expects and returns a 3D array (kind of similar to if you have return_sequences=True in the regular LSTM layer). Try adding a Flatten() layer or another LSTM layer at the end before the prediction layer.
d = TimeDistributed(Dense(10))(input_from_previous_layer)
lstm_out = Bidirectional(LSTM(10))(d)
output = Dense(1, activation='sigmoid')(lstm_out)
Using temporal is a workaround. Check out this stack. The issue is also documented on github.