The following seems to be the new way of creating a FileWriter but I am not sure how to add_graph or do other things to get the model graph showing up in tensorboard.
train_writer = tf.summary.create_file_writer('logs/1/train')
You can do it by using tf.keras.callback.TensorBoard(/path/to/log/dir) in following way.
Create a build model function
def build_model():
model = keras.Sequential([
layers.Dense(64, activation='relu', input_shape=[len(train_dataset[col_to_norm].keys())]),
layers.Dense(64, activation="relu"),
layers.Dense(1)
])
optimizer = tf.keras.optimizers.RMSprop(0.001)
model.compile(loss='mse',
optimizer=optimizer,
metrics=['mae', 'mse'])
return model
2 . assign to a variable
model = build_model()
Fit the model using fit method
model.fit(
normalized_train_data, train_labels,
epochs=EPOCHS, validation_split = 0.2, verbose=0,
callbacks=[tf.keras.callbacks.TensorBoard('logs/1/train')])
Related
I am trying to tune hyper-parameters for my LSTM model using GridSearchCV but have used TimeseriesGenerator from keras.preprocessing.sequence. How do I modify the KerasRegressor wrapper to accommodate fit_generator() instead of the fit() method?
def create_model(layer1=50):
lstm_model = Sequential()
lstm_model.add(LSTM(layer1, input_shape=(10,11)))
lstm_model.add(Dense(11, activation='tanh'))
lstm_model.compile(loss='mean_squared_error', optimizer='rmsprop')
return lstm_model
model = KerasRegressor(build_fn=create_model, epochs=5, batch_size=10, verbose=0)
layer1 = [40,50]
param_grid = dict(layer1=layer1)
grid = GridSearchCV(estimator=model, param_grid=param_grid, n_jobs=-1, cv=3)
grid_result = grid.fit(X, y)
But how do I use KerasRegressor with a generator like
from keras.preprocessing.sequence import TimeseriesGenerator
train_data_gen = TimeseriesGenerator(X, y,length=10,batch_size=100)
I'm trying to follow a Tensorflow tutorial (i'm a beginner) for structured data models with some changes along the way.
My purpose is to create a model to which i provide data (in csv format) that looks something like this (the example has only 2 features but i want to extend it after i figure it out):
power_0,power_1,result
0.2,0.3,draw
0.8,0.1,win
0.3,0.1,draw
0.7,0.2,win
0.0,0.4,lose
I created the model using the following code:
def get_labels(df, label, mapping):
raw_y_true = df.pop(label)
y_true = np.zeros((len(raw_y_true)))
for i, raw_label in enumerate(raw_y_true):
y_true[i] = mapping[raw_label]
return y_true
tf.compat.v1.enable_eager_execution()
mapping_to_numbers = {'win': 0, 'draw': 1, 'lose': 2}
data_frame = pd.read_csv('data.csv')
data_frame.head()
train, test = train_test_split(data_frame, test_size=0.2)
train, val = train_test_split(train, test_size=0.2)
train_labels = np.array(get_labels(train, label='result', mapping=mapping_to_numbers))
val_labels = np.array(get_labels(val, label='result', mapping=mapping_to_numbers))
test_labels = np.array(get_labels(test, label='result', mapping=mapping_to_numbers))
train_features = np.array(train)
val_features = np.array(val)
test_features = np.array(test)
model = tf.keras.Sequential([
tf.keras.layers.Dense(128, activation='relu', input_shape=(train_features.shape[-1],)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Dense(3, activation='sigmoid'),
])
model.compile(
optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'],
run_eagerly=True)
epochs = 10
batch_size = 100
history = model.fit(
train_features,
train_labels,
epochs=epochs,
validation_data=(val_features, val_labels))
input_data_frame = pd.read_csv('input.csv')
input_data_frame.head()
input_data = np.array(input_data_frame)
print(model.predict(input_data))
input.csv looks as following:
power_0,power_1
0.8,0.1
0.7,0.2
And the actual result is:
[[0.00604381 0.00242573 0.00440606]
[0.01321151 0.00634229 0.01041476]]
I expected to get the probability for each label ('win', 'draw' and 'lose'), can anyone please help me with this?
Thanks in advance
Use softmax activation in this line tf.keras.layers.Dense(3, activation='sigmoid').
This works well for me with your example:
model = tf.keras.Sequential([
tf.keras.layers.Flatten(input_shape=(train_features.shape[-1],)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(16, activation='relu'),
tf.keras.layers.Dense(3, activation='softmax'),
])
model.compile(
optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'],
run_eagerly=True)
Using Flatten Layer
I have to write my suggestions here because i cant comment yet.
#zihaozhihao is right you have to use softmax instead of sigmoid because you dont work with a binary problem. Another problem might be your loss function which is:
model.compile(
optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'],
run_eagerly=True)
Try to use loss='categorical_crossentropy',because you are working with a multilabel classification. You could read more about multilable classification here and here.
As for your propability question. You get the propability of each class for your two test inputs.For example:
win draw loss
[[0.00604381 0.00242573 0.00440606]
[0.01321151 0.00634229 0.01041476]]
The Problem is your loss function and the activation function which leads to strange propability values. You might want to check this post here for more information.
Hope this helps a little and feel free to ask.
Since it is a multiclass classification problem, please use categorical_crossentropy instead of binary_crossentropy for loss function, also use softmax instead of sigmoid as activation function.
Also, you should increase your epochs for getting better convergence.
I have trained a model on colab using artifical neural network (keras) and in the end I want to save it but getting error. I have tried pydrive method also. I have all the required library previously when I run it on local computer it worked.
import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.models import model_from_json
model = Sequential() # create model
model.add(Dense(6, input_dim = 8, activation = 'relu'))
model.add(Dense(6, activation = 'relu')) # hidden layer
model.add(Dense(1, activation = 'sigmoid')) # output layer
model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
model.fit(X_train, y_train, epochs=100, batch_size=10)
def build_classifier():
model = Sequential() # create model
model.add(Dense(6, input_dim = 8, activation = 'relu'))
model.add(Dense(6, activation = 'relu')) # hidden layer
model.add(Dense(1, activation = 'sigmoid')) # output layer
model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
return model
model = KerasClassifier(build_fn = build_classifier, epochs=100, batch_size=32)
accuracies = cross_val_score(estimator = model,X = X_train, y = y_train, cv = 10, n_jobs = -1)
model.save("model.h5")
but getting error 'KerasClassifier' object has no attribute 'save'. does google colab required different method to save model?
'KerasClassifier' object has no attribute 'save'
Error occur because I tried to save KerasClassifier(Cross-validation function) Since the The purpose of cross-validation is model checking, not model building. So after changing parameter model will be saved.Thanks #MatiasValdenegro for insight.
Better explanation available at
https://stats.stackexchange.com/questions/52274/how-to-choose-a-predictive-model-after-k-fold-cross-validation
I tried to optimize the hyperparameters of my keras CNN made for image classification. I consider using grid search from sklearn and talos optimizer (https://github.com/autonomio/talos). I overcame the fundamental difficulty with making x and y out from flow_from_directory (code below), but... it still does not work! Any idea? Maybe someone faced the same problem.
def talos_model(train_flow, validation_flow, nb_train_samples, nb_validation_samples, params):
model = Sequential()
model.add(Conv2D(6,(5,5),activation="relu",padding="same",
input_shape=(img_width, img_height, 3)))
model.add(MaxPooling2D((2,2)))
model.add(Dropout(params['dropout']))
model.add(Conv2D(16,(5,5),activation="relu"))
model.add(MaxPooling2D((2,2)))
model.add(Dropout(params['dropout']))
model.add(Flatten())
model.add(Dense(120, activation='relu', kernel_initializer=params['kernel_initializer']))
model.add(Dropout(params['dropout']))
model.add(Dense(84, activation='relu', kernel_initializer=params['kernel_initializer']))
model.add(Dropout(params['dropout']))
model.add(Dense(10, activation='softmax'))
model.compile(loss=params['loss'],
optimizer=params['optimizer'],
metrics=['categorical_accuracy'])
checkpointer = ModelCheckpoint(filepath='talos_cnn.h5py',
monitor='val_categorical_accuracy', save_best_only=True)
history = model.fit_generator(generator=train_flow,
samples_per_epoch=nb_train_samples,
validation_data=validation_flow,
nb_val_samples=nb_validation_samples,
callbacks=[checkpointer],
verbose=1,
epochs=params['epochs'])
return history, model
train_generator = ImageDataGenerator(rescale=1/255)
validation_generator = ImageDataGenerator(rescale=1/255)
# Retrieve images and their classes for train and validation sets
train_flow = train_generator.flow_from_directory(directory=train_data_dir,
batch_size=batch_size,
target_size(img_height,img_width))
validation_flow = validation_generator.flow_from_directory(directory=validation_data_dir,
batch_size=batch_size,
target_size=(img_height,img_width),
shuffle = False)
#here I make x and y for talos
(X_train, Y_train) = train_flow.next()
#starting an experiment with talos
t = ta.Scan(x=X_train,
y=Y_train,
model=talos_model,
params=params,
dataset_name='landmarks',
experiment_no='1')
Error occurs in the last line:
The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
When I create a simple Keras Model
model = Sequential()
model.add(Dense(10, activation='tanh', input_dim=1))
model.add(Dense(1, activation='linear'))
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['mean_squared_error'])
and do a callback to Tensorboard
tensorboard = TensorBoard(log_dir='c:/temp/tensorboard/run1', histogram_freq=1, write_graph=True, write_images=False)
model.fit(x, y, epochs=1000, batch_size=1, callbacks=[tensorboard])
The output in Tensorboard looks like this:
In other words, it's a complete mess.
Is there anything I can do to make the graphs output look more structured?
How can I create histograms of weights with Keras and Tensorboard?
You can create a name scope to group layers in your model using with K.name_scope('name_scope').
Example:
with K.name_scope('CustomLayer'):
# add first layer in new scope
x = GlobalAveragePooling2D()(x)
# add a second fully connected layer
x = Dense(1024, activation='relu')(x)
Thanks to
https://github.com/fchollet/keras/pull/4233#issuecomment-316954784