Keras, convolution neural network val_loss always the same for the same data - conv-neural-network

Question

I am trying to build a neural network that learn over DNA sequence data. The neural network has to be a classifier that say yes or no. The first thing that I have done is to transform a sequence e.g. ATTTGGCAAT with the oneHot encoding using the following function :
class hot_dna:
def init(self,fasta):
#check for and grab sequence name
if re.search(">",fasta):
name = re.split("\n",fasta)[0]
sequence = re.split("\n",fasta)[1]
else :
name = 'unknown_sequence'
sequence = fasta
#get sequence into an array
seq_array = array(list(sequence))
#integer encode the sequence
label_encoder = LabelEncoder()
integer_encoded_seq = label_encoder.fit_transform(seq_array)
#one hot the sequence
onehot_encoder = OneHotEncoder(sparse=False)
#reshape because that's what OneHotEncoder likes
integer_encoded_seq = integer_encoded_seq.reshape(len(integer_encoded_seq), 1)
onehot_encoded_seq = onehot_encoder.fit_transform(integer_encoded_seq)
#add the attributes to self
self.name = name
self.sequence = fasta
self.integer = integer_encoded_seq
self.onehot = onehot_encoded_seq
I already prepared the data in the following categories :
trainNegative used for the training and has to be assigned to no
TrainPositive used for the training and has to be assigned to yes
TrainTTNegative used for the test and has to be assigned to no
TrainTTPositive used for the test and has to be assigned to yes
TestNegative this will never be seen from the neural network and will be used to evaluate true positive, false positive, and has to be assigned to no 6) testPositive -> this will never be seen from the neural network and will be used to evaluate true positive, false positive, and has to be assigned to yes.
In the following step i will just read the fasta file (file containing the sequence for each sample) ans stack them together according to Train, TrainT and test. Also during this loading matrix i will create a vector that has 1 or 0 according to the positive or negative class (yes or no). There is also a control on the shape since all the sequence has to have same dimension.
a=list()
trainT=[]
fasta_sequences = SeqIO.parse(open("/scratch/trainTNegative.fa"),'fasta')
for fasta in fasta_sequences:
f=hot_dna(str(fasta.seq))
if (f.sequence).count("N") == 0:
if f.onehot.shape == (5000,4):
a.append(f.onehot)
trainT.append(0)
fasta_sequences = SeqIO.parse(open("/scratch/trainTPositive.fa"),'fasta')
for fasta in fasta_sequences:
f=hot_dna(str(fasta.seq))
if (f.sequence).count("N") == 0:
if f.onehot.shape == (5000,4):
a.append(f.onehot)
trainT.append(1)
train=np.stack(a)
#train=train.reshape(train.shape[0], 1,train.shape[1],train.shape[2] )
train=train.reshape(train.shape[0],train.shape[1],train.shape[2],1 )
trainT=to_categorical(trainT)
#TESTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
a=list()
testT=[]
fasta_sequences = SeqIO.parse(open("/scratch/trainTTNegative.fa"),'fasta')
for fasta in fasta_sequences:
f=hot_dna(str(fasta.seq))
if (f.sequence).count("N") == 0:
if f.onehot.shape == (5000,4):
a.append(f.onehot)
testT.append(0)
fasta_sequences = SeqIO.parse(open("/scratch/trainTTPositive.fa"),'fasta')
for fasta in fasta_sequences:
f=hot_dna(str(fasta.seq))
if (f.sequence).count("N") == 0:
if f.onehot.shape == (5000,4):
a.append(f.onehot)
testT.append(1)
test=np.stack(a)
#test=test.reshape(test.shape[0], 1,test.shape[1],test.shape[2] )
test=test.reshape(test.shape[0],test.shape[1],test.shape[2],1 )
testT=to_categorical(testT)
#NEVERSEEEEEEEEEEEEEEEEEEEEEEEEENN
a=list()
neverT=[]
fasta_sequences = SeqIO.parse(open("/scratch/testNegative.fa"),'fasta')
for fasta in fasta_sequences:
f=hot_dna(str(fasta.seq))
if (f.sequence).count("N") == 0:
if f.onehot.shape == (5000,4):
a.append(f.onehot)
neverT.append(0)
fasta_sequences = SeqIO.parse(open("/scratch/testPositive.fa"),'fasta')
for fasta in fasta_sequences:
f=hot_dna(str(fasta.seq))
if (f.sequence).count("N") == 0:
if f.onehot.shape == (5000,4):
a.append(f.onehot)
neverT.append(1)
never=np.stack(a)
never=never.reshape(never.shape[0],never.shape[1],never.shape[2],1 )
neverT=to_categorical(neverT)
Now I set up the neural network, the idea is to have 3 convolutional neural network with a window (4,10), since the columns of the one-Hot encoding are 4 and i would like a sliding window of ten.
model = Sequential()
model.add(Conv2D(32, (4, 10), padding="same", activation="relu",input_shape= [test.shape[1],test.shape[2],test.shape[3]]))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (4, 10), padding="same", activation="relu"))
model.add(MaxPooling2D(pool_size=(4, 2)))
model.add(Conv2D(128, (4, 10), padding="same", activation="relu"))
model.add(MaxPooling2D(pool_size=(8, 1)))
model.add(Flatten())
model.add(Dense(32, activation='relu'))
model.add(keras.layers.Dropout(rate=0.5, noise_shape=None, seed=None))
model.add(Dense(16, activation='relu'))
model.add(keras.layers.Dropout(rate=0.5, noise_shape=None, seed=None))
model.add(Dense(8, activation='relu'))
model.add(keras.layers.Dropout(rate=0.5, noise_shape=None, seed=None))
model.add(Dense(2, activation='sigmoid'))
model.compile(optimizer=keras.optimizers.Adam(lr=0.001), loss="categorical_crossentropy",metrics=['accuracy'])
print(model.summary())
Model: "sequential_4"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_10 (Conv2D) (None, 5000, 4, 32) 3232
_________________________________________________________________
max_pooling2d_10 (MaxPooling (None, 2500, 2, 32) 0
_________________________________________________________________
conv2d_11 (Conv2D) (None, 2500, 2, 64) 81984
_________________________________________________________________
max_pooling2d_11 (MaxPooling (None, 625, 1, 64) 0
_________________________________________________________________
conv2d_12 (Conv2D) (None, 625, 1, 128) 327808
_________________________________________________________________
max_pooling2d_12 (MaxPooling (None, 78, 1, 128) 0
_________________________________________________________________
flatten_4 (Flatten) (None, 9984) 0
_________________________________________________________________
dense_13 (Dense) (None, 32) 319520
_________________________________________________________________
dropout_10 (Dropout) (None, 32) 0
_________________________________________________________________
dense_14 (Dense) (None, 16) 528
_________________________________________________________________
dropout_11 (Dropout) (None, 16) 0
_________________________________________________________________
dense_15 (Dense) (None, 8) 136
_________________________________________________________________
dropout_12 (Dropout) (None, 8) 0
_________________________________________________________________
dense_16 (Dense) (None, 2) 18
=================================================================
Total params: 733,226
Trainable params: 733,226
Non-trainable params: 0
_________________________________________________________________
None
And here are the shapes of my data
train.shape
(1023, 5000, 4, 1)
trainT.shape
(1023, 2)
where 1023 is the number of samples, 5000 is the length of the sequence, so how many letter i have and 4 is the number of columns once the sequence is transformed by the one-Hot encoding. Now i create the checkpoint and I train my model
checkpoint_name = "/scratch/"+"test"+"/BW/"+'BW.hdf5'
try:
os.makedirs("/scratch/"+"test"+"/BW/")
except:
print("folder already exists")
checkpoint = keras.callbacks.ModelCheckpoint(checkpoint_name, monitor='val_accuracy', verbose = 1, save_best_only = True, mode ='max')
callbacks_list = [checkpoint]
NNResult=model.fit(train,trainT, epochs=1000, batch_size=64,validation_data=(test,testT),callbacks=callbacks_list)
Main problem val_loss does not go down but worst part is that even if I restart the training, setting up the same model over again, the val_loss value is exactly the same. If I change seed and restart also the python session the val_loss is always the same. Does anyone have any Idea of the reason? Also I'm wondering if the flatten layer that i have added is doing something bad to my training.
I also just tried to modify the network like this
model = Sequential()
model.add(Conv1D(32, (10), padding="same", activation="relu",input_shape= [test.shape[1],test.shape[2]]))
model.add(GlobalAveragePooling1D())
model.add(Dense(2, activation='relu'))
and is crazy i have the same effect as before. Do you think might be the data then?
Thank you