For my NLP project I used CountVectorizer to Extract Features from a dataset using vectorizer = CountVectorizer(stop_words='english') and all_features = vectorizer.fit_transform(data.Text) and i also wrote a Simple RNN model using keras but I am not sure how to do the padding and the tokeniser step and get the data be trained on the model.
my code for RNN is:
model.add(keras.layers.recurrent.SimpleRNN(units = 1000, activation='relu',
use_bias=True))
model.add(keras.layers.Dense(units=1000, input_dim = 2000, activation='sigmoid'))
model.add(keras.layers.Dense(units=500, input_dim=1000, activation='relu'))
model.add(keras.layers.Dense(units=2, input_dim=500,activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
can someone please give me some advice on this?
Thank you
add ensemble - you don't count vectorize, you use ensemble
https://github.com/dnishimoto/python-deep-learning/blob/master/UFO%20.ipynb
docs=ufo_df["summary"] #text
LABELS=['Egg', 'Cross','Sphere', 'Triangle','Disk','Oval','Rectangle','Teardrop']
#LABELS=['Triangle']
target=ufo_df[LABELS]
#print([len(d) for d in docs])
encoded_docs=[one_hot(d,vocab_size) for d in docs]
#print([np.max(d) for d in encoded_docs])
padded_docs = pad_sequences(encoded_docs, maxlen=max_length, padding='post')
#print([d for d in padded_docs])
model=Sequential()
model.add(Embedding(vocab_size, 8, input_length=max_length))
model.add(Flatten())
model.add(Dense(8, activation='softmax'))
#model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
model.fit(padded_docs, target, epochs=50, verbose=0)
Related
I don't understand this error here is my model
L_branch = Sequential()
L_branch.add(Embedding(vocab_size, output_dim=15, input_length=3000, trainable=True))
L_branch.add(Conv1D(50, activation='relu', kernel_size=70, input_shape=(3000, )))
L_branch.add(MaxPooling1D(15))
L_branch.add(Flatten())
# second model
R_branch = Sequential()
R_branch.add(Dense(14, input_shape=(14,), activation='relu'))
R_branch.add(Flatten())
merged = Concatenate()([L_branch.output, R_branch.output])
out = Dense(70, activation='softmax')(merged)
final_model = Model([L_branch.input, R_branch.input], out)
final_model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
final_model.summary()
final_model.fit(
[input1, input2],
Y_train,
batch_size=200,
epochs=1,
verbose=1,
validation_split=0.1
)
where input 1 has the shape (5039, 3000)
and input 2 (5039, 14)
so why is the flatten dense asking for a third dimension? if dense do not change the number of dimensions like embedding layer or convolution?
remove Flatten layer... no need to use it. here the full structure
L_branch = Sequential()
L_branch.add(Embedding(vocab_size, output_dim=15, input_length=3000, trainable=True))
L_branch.add(Conv1D(50, activation='relu', kernel_size=70, input_shape=(3000, )))
L_branch.add(MaxPooling1D(15))
L_branch.add(Flatten())
# second model
R_branch = Sequential()
R_branch.add(Dense(14, input_shape=(14,), activation='relu'))
merged = Concatenate()([L_branch.output, R_branch.output])
out = Dense(70, activation='softmax')(merged)
final_model = Model([L_branch.input, R_branch.input], out)
final_model.compile(
loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
final_model.summary()
Dear all, I have trained word2vec in gensim using Wikipedia data and saved using following program.
model = Word2Vec(LineSentence(inp), size=300, window=5, min_count=5, max_final_vocab=500000,
workers=multiprocessing.cpu_count())
model.save("outp1")
I want use this model in keras for multi-class Text Classification, What changes I need to do in the following code
model = Sequential()
model.add(Embedding(MAX_NB_WORDS, EMBEDDING_DIM, input_length=X.shape[1]))
model.add(SpatialDropout1D(0.2))
model.add(LSTM(100, dropout=0.2, recurrent_dropout=0.2))
model.add(Dense(10, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
epochs = 5
batch_size = 64
history = model.fit(X_train, Y_train, epochs=epochs,
batch_size=batch_size,validation_split=0.1,callbacks=[EarlyStopping(monitor='val_loss', patience=3, min_delta=0.0001)])
accr = model.evaluate(X_test,Y_test)
Actually I am new and trying to learn.
Is keras model layer ferature labels are same to the orginal labels
model.add(Flatten())
model.add(Dense(380,name = 'dense_1'))
model.add(Activation('relu'))
model.add(Dropout(0.1))
model.add(Dense(classes_num ))
model.add(Activation('softmax'))
model.compile(optimizer='rmsprop', loss='categorical_crossentropy',
metrics=['accuracy',mean_pred,recall,precision,fmeasure, matthews_correlation,kullback_leibler_divergence,
binary_crossentropy])
model.summary()
print('model complied!!')
print('starting training....')
history = model.fit(X_train, Y_train, epochs=epochs, batch_size=64,validation_data=(X_test, Y_test))
extract =Model(model.input,[model.get_layer("dense_1").output,model.output])
test_feature,test_labels= extract.predict(X_test)
Is the test_labels and y_test are same or not.If i want to use the layer features what labels i should use
test_label is a decimals that shows probability of membership in each class and it is not same as y_test. if you get index of maximum value in output of softmax layer it shows the class that your network determine according to inputs.
I trained a neural network with 37 Inputs. It has around 85% accuracy. Is it possible for me to find out which Input has the most effect. I tried this code but I cannot figure out how to find most important Input
weights = model.layers[0].get_weights()[0]
biases = model.layers[0].get_weights()[1]
One possible solution is to wrap your model with keras.wrappers.scikit_learn and then use Recursive Feature elimination in scikit-learn:
def create_model():
# create model
model = Sequential()
model.add(Dense(512, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dense(10, activation='softmax'))
# Compile model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
return model
model = KerasClassifier(build_fn=create_model, epochs=100, batch_size=128, verbose=0)
rfe = RFE(estimator=model, n_features_to_select=1, step=1)
rfe.fit(X, y)
ranking = rfe.ranking_.reshape(digits.images[0].shape)
# Plot pixel ranking
plt.matshow(ranking, cmap=plt.cm.Blues)
plt.colorbar()
plt.title("Ranking of pixels with RFE")
plt.show()
If you need to visualize weights see here.
I am building a RNN to predict a many to one question.
#Input_X:
[
[1,2,3,4,5,6,7,8,9,10],
[2,3,4,5,6,7,8,9,10,11]
]
#Input_Y:
[
11,
12
]
#Each number represent a category
X=np.reshape(Input_X,(len(Input_X), 10, 1))
y=np.utils.to_catgeorical(Input_Y) #one hot encode,
My model setup:
#####This works
model=Sequential()
model.add(LSTM(256, input_shape(X.shape[1], X.shape[2])))
model.add(Dense(y.shape[1], activation='softmax'))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)
I want to try the TimeDistributed(Dense) layer instead, for example: https://keras.io/layers/wrappers/. So I changed above to below:
model=Sequential()
model.add(LSTM(256, input_shape(X.shape[1], X.shape[2])))
model.add(TimeDistributed(Dense(y.shape[1], activation='softmax')))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)
I am getting a AssertionError. Which report the matrix size is not what expected.
What steps I missed?
I think you need to add return_sequences=True to the LSTM cell
```
model=Sequential()
model.add(LSTM(256, return_sequences=True, input_shape(X.shape[1], X.shape[2])))
model.add(TimeDistributed(Dense(y.shape[1], activation='softmax')))
model.compile(loss='categorical_crossentrophy', optimizer='adam', metrics=['accuracy'])
model.fit(X,y, ........)
```
return_sequences=True works for me.
In the OP's question, y.shape is (2,1) which has 2 samples and only 1 feature , so it's not suited for Many to Many model.