Getting differents results for the identical model FFNN in keras - keras

Question

I am building a model based on FFNN (Feed Forward Neural Network) using Keras.
I built a first version:
def mlp0(input_dim, loss):
model = Sequential()
model.add(Dropout(0.5, input_shape=(input_dim,)))
model.add(Dense(512, activation='sigmoid'))
model.add(Dense(1, activation='relu'))
model.compile(loss=loss, optimizer=Adagrad())
return model
This gives me very good results in k-fold cross-validation, but when I predict on validation set, the performance is bad.
So I tried another version.
def mlp1(input_dim, loss):
inputs = keras.Input(shape=(input_dim,))
x = keras.layers.Dropout(0.5)(inputs)
x = keras.layers.Dense(512, activation='sigmoid')(x)
outputs = keras.layers.Dense(1, activation='relu')(x)
model = keras.Model(inputs, outputs)
model.compile(loss=loss, optimizer=Adagrad())
return model
This second model gives worse results on cross-validation but the results are compatible with the results on the validation set.
To my eyes, they are identical models built in different ways, but for some reason they give me different answers. What am I doing wrong?
Edit:
These models behave the same way:
def mlp0(input_dim, loss):
model = Sequential()
model.add(Dense(512, activation='sigmoid', input_shape=(input_dim,), kernel_regularizer=regularizers.l2(0.01)))
model.add(Dense(1, activation='relu', kernel_regularizer=regularizers.l2(0.01)))
model.compile(loss=loss, optimizer=Adam())
return model
import keras
from keras import regularizers
def mlp1(input_dim, loss):
inputs = keras.Input(shape=(input_dim,))
x = keras.layers.Dense(512, activation='sigmoid', kernel_regularizer=regularizers.l2(0.01))(inputs)
outputs = keras.layers.Dense(1, activation='relu', kernel_regularizer=regularizers.l2(0.01))(x)
model = keras.Model(inputs, outputs)
model.compile(loss=loss, optimizer=Adam())
return model
These make me think there is a catch in prediction phase with the dropout