I am trying to implement a keras LSTM. I am getting an error inside keras.model.fit. I am not understanding what this error means. My code is given below -
print(x_train.shape)
print(y_train.shape)
word_input = Input(shape=(mxlen,), dtype="int32", name="word_input")
x1 = Embedding(len(vocab), 100, input_length=mxlen, weights=[embeddings], trainable=False)(word_input)
x1 = LSTM(100)(x1)
y = Dense(6, activation="softmax", name="main_output")(x1)
model = Model(inputs=[word_input], outputs=[y])
adam = optimizers.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, amsgrad=False)
model.compile(optimizer=adam,loss='categorical_crossentropy',metrics=['categorical_accuracy']) # have to look into it
model.fit(x_train, y_train, epochs=30, batch_size=40, verbose=1)
x_train and y_train has the following shape - (10240, 198) and (10240,).
I am getting the following error:
Your output layer has 6 outputs (probably 6 classes are given), but the target labels in y_train are given as integers, apparently (array is flat).
You need to convert y_train to a one-hot encoded array first, otherwise, the cross-entropy loss cannot be computed. Input and output tensors must always be compatible with the inputs and outputs of the model.
Given that there are 6 classes in y_train, encoded as:
0,1,2,3,4,5
you require a conversion:
0 -> 1.,0.,0.,0.,0.,0.
1 -> 0.,1.,0.,0.,0.,0.
etc.
Try it like this:
import tf.keras.backend as K
...
num_classes = 6
y_one_hot = K.one_hot( y_train, num_classes )
....
model.fit(x_train, y_one_hot, epochs=30, batch_size=40, verbose=1)
Related
I need help to understand Keras prediction class labeling
I am building the binary classification model with Keras. I use 3 'relu' layers plus sigmoid output, the loss function is binary_crossentropy.
The point is that I am not able to understand simple thing:
- for my y_train/y_test I use two values: False / True. When I predict on the X_test I am getting he probability or class predicted, fine. But unfortunately the model has in 50% of cases the accuracy of 0,1 and in other 50% of cases 0,9. This comes from the fact that the distribution of the False / True is uneven - 90% of False and 10% of True. ... and then, what happens is that it looks like the class label for False is 0 in half of the cases and 1 in other half of cases. I am unable to figure out why.
I found some posts saying Keras would number the classes in alphabetical order, so I assumed False would always get 0 and True 1, but this does not seem to be the case.
Please find my code below and advise what I do wrong.
# ####################### Fetch data #################################
#df = pd.read_csv(os.path.join(data_folder, data_file))
df = pd.read_csv('\\\...\\train_data_with_anomalous_column.csv', delimiter=',')
x = df.drop('ANOMALOUS', axis=1)
# x = x.drop('TIMESTAMP', axis=1)
y = df['ANOMALOUS'].copy()
x = x.as_matrix()
y = y.as_matrix()
# y.astype(int)
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.1)
# ####################### get rid of TIMESTAMP columns ###############
# ###### first, copy the TIMESTAMP column into ananother np-array
# ###### in order to be able to join it later
X_test_TST = X_test[:, [0]]
X_train = np.delete(X_train, 0, 1)
X_test = np.delete(X_test, 0, 1)
class_weight = {False: 1.,
True: 1.
}
training_set_size = X_train.shape[0]
n_inputs = 22
n_h1 = args.n_hidden_1
n_h2 = args.n_hidden_2
n_h3 = args.n_hidden_3
n_outputs = 1
n_epochs = 2
model_validation_split = 0.1
batch_size = args.batch_size
learning_rate = args.learning_rate
print(X_train.shape, y_train.shape, X_test.shape, y_test.shape, sep='\n')
# Build a simple MLP model
model = Sequential()
# first hidden layer
model.add(Dense(n_h1, activation='relu', input_shape=(n_inputs,)))
# second hidden layer
model.add(Dense(n_h2, activation='relu'))
# second hidden layer
model.add(Dense(n_h3, activation='relu'))
# output layer
model.add(Dense(n_outputs, activation='sigmoid'))
model.summary()
model.compile(loss='binary_crossentropy',
optimizer=optimizer,
metrics=['accuracy']
)
# start an Azure ML run
run = Run.get_context()
class LogRunMetrics(Callback):
# callback at the end of every epoch
def on_epoch_end(self, epoch, log):
# log a value repeated which creates a list
run.log('Loss', log['loss'])
run.log('Accuracy', log['acc'])
history = model.fit(X_train, y_train,
batch_size=batch_size,
epochs=n_epochs,
verbose=2,
class_weight=class_weight,
callbacks=[LogRunMetrics()])
score = model.evaluate(X_test, y_test, verbose=1)
# #### get accuracy, F1, precision and recall
# Get the class predictions
y_classes = model.predict_classes(X_test, batch_size=batch_size, verbose=1)
print(y_classes
print(classification_report(y_test, y_classes))
...
I have a Sequential model that is trained and compiles successfully, but when running prediction raises the error as stated in the title.
np.seterr(divide='ignore')
# Numpy arrays
X_train, X_test, Y_train, Y_test = Prepr_data.process()
model = Sequential([
Dense(32, input_shape=(84,)), Activation('relu'),
Dense(11), Activation('softmax')
])
model.compile(optimizer='rmsprop',
loss='categorical_crossentropy',
metrics=['accuracy'])
model.fit(X_train, Y_train, epochs=15, batch_size=91)
pred = model.predict(X_test, Y_test)
print(pred)
X-arrays contain independent variables, while Y-arrays contain the dependent variable. That is, Y-arrays contain the labels I am trying to predict, and X contain the rest of the data. All four arrays are numpy arrays of equal length. All data present are one-hot encoded (totalling 84 columns in X, 11 in Y). All data are of dtype float64.
I am quite new to ML so there might just be a very trivial issue I am overlooking ... any help appreciated!
I've been trying to figure out how to use Keras when reading data from a .csv[1]. I have the following code:
dataframe = pd.read_csv("iris.csv", header=None)
dataset = dataframe.values
X = dataset[:, 0:4].astype(float)
Y = dataset[:, 4]
# encode class values as integers
encoder = LabelEncoder()
encoder.fit(Y)
encoded_Y = encoder.transform(Y)
# convert integers to dummy variables (i.e. one hot encoded)
one_hot_y = keras.utils.to_categorical(encoded_Y)
X_train = X[:100]
X_test = X[50:]
Y_train = one_hot_y[:100]
Y_test = one_hot_y[50:]
print(X_train.shape)
# define baseline model
def baseline_model():
# create model
model = keras.models.Sequential()
model.add(keras.layers.Dense(8, input_shape=(4, ), activation='relu'))
model.add(keras.layers.Dense(3, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
model = baseline_model()
history = model.fit(X_train, X_test, epochs=5)
test_loss, test_acc = model.evaluate(Y_train, Y_test)
print('Test accuracy:', test_acc)
However, when I run this, I get the error:
ValueError: Error when checking target: expected dense_1 to have shape (3,) but got array with shape (4,)
I find this odd as I was sure to set input_shape=(4, ). Any help with this would be appreciated.
[1] The CSV looks as follows:
5.1,3.5,1.4,0.2,Iris-setosa
...
You have only 3 output neurons, but the data you are using obviously has 4 classes, so you need to change this line:
model.add(keras.layers.Dense(3, activation='softmax'))
from 3 output classes to 4 output classes:
model.add(keras.layers.Dense(4, activation='softmax'))
I am doing multi-class classification of 5 classes. I am using Tensorflow with Keras. My code is like this:
# load dataset
dataframe = pandas.read_csv("Data5Class.csv", header=None)
dataset = dataframe.values
# split into input (X) and output (Y) variables
X = dataset[:,0:47].astype(float)
Y = dataset[:,47]
print("Load Data.....")
encoder= to_categorical(Y)
def create_larger():
model = Sequential()
print("Create Dense Ip & HL 1 Model ......")
model.add(Dense(47, input_dim=47, kernel_initializer='normal', activation='relu'))
print("Add Dense HL 2 Model ......")
model.add(Dense(40, kernel_initializer='normal', activation='relu'))
print("Add Dense output Model ......")
model.add(Dense(5, kernel_initializer='normal', activation='sigmoid'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
estimators = []
estimators.append(('rnn', KerasClassifier(build_fn=create_larger, epochs=60, batch_size=10, verbose=0)))
pipeline = Pipeline(estimators)
kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=seed)
results = cross_val_score(pipeline, X, encoder, cv=kfold)
print("Accuracy: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
The CSV file I have taken as an input contains the data with labels. The labels are like this 0, 1, 2, 3, 4 which represent 5 different classes.
Then, as the labels are already in integer form, do I need to use
the LabelEncoder() function in my code?
Also, I have used to_categorical(Y) function. Should I use it or I should just pass the Y variable containing these labels to the classifier for training?
I got the error like this:
Supported target types are: ('binary', 'multiclass'). Got 'multilabel-indicator' instead.
This error occurred when I used encoder variable in the code
results = cross_val_score(pipeline, X, encoder, cv=kfold) where encoder variable represents the to_categorical(Y) data. How to solve this error?
As mentioned on the Keras documentation here:
Note: when using the categorical_crossentropy loss, your targets
should be in categorical format (e.g. if you have 10 classes, the
target for each sample should be a 10-dimensional vector that is
all-zeros except for a 1 at the index corresponding to the class of
the sample). In order to convert integer targets into categorical
targets, you can use the Keras utility to_categorical:
from keras.utils.np_utils import to_categorical
categorical_labels = to_categorical(int_labels, num_classes=None)
So this means that you need to use the to_categorical() method on your y before training. But no need to use LabelEncoder if y is already in integer type.
I think the code will speak for itself, but i trained a model, that i now wanna use to predict on some new input data. The new input data seems to be the wrong dimensions though. Below you can see the code and error messages for both the model and the predicting (attempted)
tokenizer = Tokenizer(num_words=10000)
df = pd.read_csv('/home/paperspace/Sentiment Analysis Dataset.csv', index_col = 0,
error_bad_lines = False)
y = list(df['Sentiment'])
tokenizer.fit_on_texts(list(df['SentimentText']))
X = tokenizer.texts_to_sequences(list(df['SentimentText']))
X = pad_sequences(X)
print("Done, fitting on texts.")
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15, shuffle = True)
model = Sequential()
#Creates the wordembeddings.
embedding_vector_dim = 32
model.add(Embedding(10000, embedding_vector_dim, input_length=X.shape[1]))
model.add(Dropout(0.2))
model.add(LSTM(128))
model.add(Dropout(0.2))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
model.summary()
model.fit(numpy.array(X_train), numpy.array(y_train),
batch_size=128,
epochs=1,
validation_data=(numpy.array(X_test), numpy.array(y_test)))
score, acc = model.evaluate(numpy.array(X_test),numpy.array(y_test),
batch_size=128)
model.save('./sentiment_seq.h5')
print('Test score:', score)
print('Test accuracy:', acc)
Now for the trying to predict and error message.
text = "this is actually a very bad movie."
tokenizer = Tokenizer()
tokenizer.fit_on_texts(list(text))
X = tokenizer.texts_to_sequences(list(text))
X = pad_sequences(X)
X_flat = np.array([X.flatten()])
model = load_model('sentiment_test.h5')
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
print(model.predict(X, batch_size = 1, verbose = 1))
ValueError: Error when checking : expected embedding_1_input to have shape (None, 116) but got array with shape (1, 38)
So basically why am i getting this error, when preprocessing is the same when training and predicting, and how can i know what the expected input should be before seeing the error message?
If you're not working with a fixed input length, you should not define an input_length in the embedding layer.