Issues with keras Efficientnet import and Data Augmentation - keras

I am trying to train a model using transfer learning with data augmentation.
My image data is 32 x 32 x 3 and I want to import EfficientNet07, but every time I run
from tensorflow.keras.applications import EfficientNetB0
model = EfficientNetB0(weights='imagenet')
I get error message:
ImportError: cannot import name 'EfficientNetB7' from 'tensorflow.keras.applications' (C:\Users\…..
I unwillingly reverted to Resnet50, that works fine. But I really wish to work with efficientnets.
Secondly, I was hoping to use keras.expermental.preprocessing layers to add data augmentation while the model runs, but I get error
from tensorflow.keras.layers.experimental import preprocessing
from tensorflow.keras.models import Sequential
from tensorflow.keras import layers
img_augmentation = Sequential(
[
preprocessing.RandomRotation(factor=0.15),
preprocessing.RandomTranslation(height_factor=0.1, width_factor=0.1),
preprocessing.RandomFlip(),
preprocessing.RandomContrast(factor=0.1),
],
name="img_augmentation",
)
AttributeError: module 'tensorflow.keras.layers.experimental.preprocessing' has no attribute 'RandomRotation'
I took all this code from https://keras.io/examples/vision/image_classification_efficientnet_fine_tuning/
and have read it top to bottom thrice. I went on to use ImageDataGenerator and wrote this code
def choosen_args(args):
#this function is to randomly choose the augmentation techniques from the list available in keras
key_list = []
arg_list = list(args.keys())
for i in range(len(arg_list)):
if np.random.randint(2) == 0:
pass
else:
key_list.append(arg_list[i])
arg_dict = dict((k, args[k]) for k in key_list)
return arg_dict
#this is the list of all augmentation I am using - but only few will get selected for final transformation
args = dict(
rotation_range = 120,
width_shift_range=0.4,
height_shift_range=0.4,
shear_range=0.2,
zoom_range=0.2,
fill_mode ='nearest',
zca_whitening=True,
zca_epsilon=1e-06,
brightness_range=(9.0, 46.0),
channel_shift_range=60.0,
horizontal_flip=True,
vertical_flip=True,
rescale=random.randint(1,11))
data_augmentation_train = ImageDataGenerator( choosen_args(args) )
data_augmentation_validation = ImageDataGenerator( choosen_args(args) )
data_augmentation_train.fit(x_train, augment = True)
data_augmentation_validation.fit(x_test, augment = True)
Kindly help with:
How to get efficientnet07 and use its weights for transfer learninig
The data augmentation code I have used - how do I visualize what it does - that is, transforms my images
Is there an easier way to do data augmentation - how do I fix the Import and attribute errors
After I fit my data in generator - how do I use it with rest of my transfer learning code.
base_model = tf.keras.applications.ResNet50(weights='imagenet',
include_top = False, input_shape = input_size)
> x_train = tf.keras.applications.resnet50.preprocess_input(x_train)
> x_test = tf.keras.applications.resnet50.preprocess_input(x_test)
> base_model.trainable = False
inputs = keras.Input(shape=(32, 32, 3))
model = base_model_ResNet50(inputs, training=False)
model = keras.layers.GlobalAveragePooling2D()(model )
model= keras.layers.Flatten()(model)
model = keras.layers.Dropout(0.2)(model)
output = layers.Dense(100, activation="softmax")(model)
history = model.fit(data_augmentation_train.flow (x_train, labels_train, batch_size = 64), validation_data =
data_augmentation_validation.flow(x_test,labels_test))
Thank you

try importing as :
from keras_efficientnets import EfficientNetB0
model = EfficientNetB0(weights='imagenet')

First install efficientnet module:
!pip install -U efficientnet
Then import it as:
import efficientnet.keras as effnet
Create the model:
model = effnet.EfficientNetB0(weights = 'imagenet')

model = tf.keras.applications.EfficientNetB0()
Options to replace EfficientNetB0 are listed here: https://keras.io/api/applications/#usage-examples-for-image-classification-models
E.g. model = tf.keras.applications.EfficientNetB7() or model = tf.keras.applications.EfficientNetV2B3()
(FYI you may have the change size of the image input)
Hope this helps

Related

Questions about Multitask deep neural network modeling using Keras

I'm trying to develop a multitask deep neural network (MTDNN) to make prediction on small molecule bioactivity against kinase targets and something is definitely wrong with my model structure but I can't figure out what.
For my training data (highly imbalanced data with 0 as inactive and 1 as active), I have 423 unique kinase targets (tasks) and over 400k unique compounds. I first calculate the ECFP fingerprint using smiles, and then I randomly split the input data into train, test, and valid sets based on 8:1:1 ratio using RandomStratifiedSplitter from deepchem package. After training my model using the train set and I want to make prediction on the test set to check model performance.
Here's what my data looks like (screenshot example):
(https://i.stack.imgur.com/8Hp36.png)
Here's my code:
# Import Packages
import numpy as np
import pandas as pd
import deepchem as dc
from sklearn.metrics import roc_auc_score, roc_curve, auc, confusion_matrix
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import initializers, regularizers
from tensorflow.keras.models import Sequential, Model
from tensorflow.keras.layers import Dense, Input, Dropout, Reshape
from tensorflow.keras.optimizers import SGD
from rdkit import Chem
from rdkit.Chem import rdMolDescriptors
# Build Model
inputs = keras.Input(shape = (1024, ))
x = keras.layers.Dense(2000, activation='relu', name="dense2000",
kernel_initializer=initializers.RandomNormal(stddev=0.02),
bias_initializer=initializers.Ones(),
kernel_regularizer=regularizers.L2(l2=.0001))(inputs)
x = keras.layers.Dropout(rate=0.25)(x)
x = keras.layers.Dense(500, activation='relu', name='dense500')(x)
x = keras.layers.Dropout(rate=0.25)(x)
x = keras.layers.Dense(846, activation='relu', name='output1')(x)
logits = Reshape([423, 2])(x)
outputs = keras.layers.Softmax(axis=2)(logits)
Model1 = keras.Model(inputs=inputs, outputs=outputs, name='MTDNN')
Model1.summary()
opt = keras.optimizers.SGD(learning_rate=.0003, momentum=0.9)
def loss_function (output, labels):
loss = tf.nn.softmax_cross_entropy_with_logits(output,labels)
return loss
loss_fn = loss_function
Model1.compile(loss=loss_fn, optimizer=opt,
metrics=[keras.metrics.Accuracy(),
keras.metrics.AUC(),
keras.metrics.Precision(),
keras.metrics.Recall()])
for train, test, valid in split2:
trainX = pd.DataFrame(train.X)
trainy = pd.DataFrame(train.y)
trainy2 = tf.one_hot(trainy,2)
testX = pd.DataFrame(test.X)
testy = pd.DataFrame(test.y)
testy2 = tf.one_hot(testy,2)
validX = pd.DataFrame(valid.X)
validy = pd.DataFrame(valid.y)
validy2 = tf.one_hot(validy,2)
history = Model1.fit(x=trainX, y=trainy2,
shuffle=True,
epochs=10,
verbose=1,
batch_size=100,
validation_data=(validX, validy2))
y_pred = Model1.predict(testX)
y_pred2 = y_pred[:, :, 1]
y_pred3 = np.round(y_pred2)
# Check the # of nonzero in assay
(y_pred3!=0).sum () #all 0s
My questions are:
The roc and precision recall are all extremely high (>0.99), but the prediction result of test set contains all 0s, no actives at all. I also use the randomized dataset with same active:inactive ratio for each task to test if those values are too good to be true, and turns out all values are still above 0.99, including roc which is expected to be 0.5.
Can anyone help me to identify what is wrong with my model and how should I fix it please?
Can I use built-in functions in sklearn to calculate roc/accuracy/precision-recall? Or should I manually calculate the metrics based on confusion matrix on my own for multitasking purpose. Why and why not?

How to remove batch normalization layers from a Keras model?

I would like to remove all the batch normalization layers from a Keras model that includes short-skip connections. For example, let's consider EfficientNetB0 as follow:
import tensorflow as tf
model = tf.keras.applications.EfficientNetB0(weights=None, include_top=True)
I was using the 3D version of efficientnet, which I don't think is important for the question ,but I'm going to show it anyway:
import keras
from keras import layers
from keras.models import Model
input_shape = (32,32,32,3)
import efficientnet_3D.keras as efn
model = efn.EfficientNetB0(input_shape=input_shape, weights='imagenet')
X = model.layers[-1].output
X = layers.Flatten()(X)
X = layers.Dense(16)(X)
X = layers.Dense(16)(X)
X = layers.Dense(1)(X)
model = Model(inputs=model.inputs, outputs=X)
model.compile(loss='mse',
optimizer='adam',
metrics=['mean_absolute_error']
)
model.summary()
I tried to develop my own way of removing, and it seems to be totally wrong. Because the output model is pretty messy in terms of all the shortcut connections.
import keras
from keras import layers
from keras.models import Model
ind = [i for i, l in enumerate(model.layers) if 'bn' in l.name]
X = model.layers[0].output
for i in range(1, len(model.layers)+1):
# Skipping Batch Normalization layers
if i in ind:
# model.layers[i]._inbound_nodes = []
# model.layers[i]._outbound_nodes = []
continue
# If there is a short skip
if isinstance(model.layers[i].input, list):
input_names = [j.name for j in model.layers[i].input]
assert len(input_names) == 2
input_names.remove(X.name)
input_names = input_names[0].split('/')[0]
# X = [model.get_layer(input_names).output, X]
X = [model.layers[6].output, X]
if isinstance(X, list):
print(i)
X = model.layers[i](X)
new_model = Model(inputs=model.inputs, outputs=X)
I think there should be a better way that I'm not aware of. I tried a similar question for removing a layer, but I think because my model includes skip-connection, those methods don't work. Any help is appreciated.

How do I set up a custom input-pipeline for sequence classification for the huggingface transformer models?

I want to use one of the models for sequence classification provided by huggingface.It seems they are providing a function called
glue_convert_examples_to_features()
for preparing the data so that it can be input into the models.
However, it seems this conversion function only applies to the glue dataset. I can't find an easy solution to apply the conversion to my costum data. Am I overseen a prebuilt function like above ? What would be an easy way to convert my custom data with one sequence and two labels into the format the model expects ?
Huggingface added a fine-tuning with custom datasets guide that contains a lot of useful information. I was able to use the information in the IMDB sequence classification section to successfully adapt a notebook using a glue dataset with my own pandas dataframe.
from transformers import (
AutoConfig,
AutoTokenizer,
TFAutoModelForSequenceClassification,
AdamW
)
import tensorflow as tf
import pandas as pd
from sklearn.model_selection import train_test_split
model_name = 'bert-base-uncased'
tokenizer = AutoTokenizer.from_pretrained(model_name)
df = pd.read_pickle('data.pkl')
train_texts = df.text.values # an array of strings
train_labels = df.label.values # an array of integers
train_texts, val_texts, train_labels, val_labels = train_test_split(train_texts, train_labels, test_size=.2)
train_encodings = tokenizer(train_texts.tolist(), truncation=True, max_length=96, padding=True)
val_encodings = tokenizer(val_texts.tolist(), truncation=True, max_length=96, padding=True)
train_dataset = tf.data.Dataset.from_tensor_slices((
dict(train_encodings),
train_labels
))
val_dataset = tf.data.Dataset.from_tensor_slices((
dict(val_encodings),
val_labels
))
num_labels = 3
num_train_examples = len(train_dataset)
num_dev_examples = len(val_dataset)
train_dataset = train_dataset.shuffle(100).batch(train_batch_size)
val_dataset = val_dataset.shuffle(100).batch(eval_batch_size)
learning_rate = 2e-5
train_batch_size = 8
eval_batch_size = 8
num_epochs = 1
train_steps_per_epoch = int(num_train_examples / train_batch_size)
dev_steps_per_epoch = int(num_dev_examples / eval_batch_size)
config = AutoConfig.from_pretrained(model_name, num_labels=num_labels)
model = TFAutoModelForSequenceClassification.from_pretrained(model_name, config=config)
optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
metrics = [tf.keras.metrics.SparseCategoricalAccuracy('accuracy', dtype=tf.float32)]
model.compile(optimizer=optimizer, loss=loss, metrics=metrics)
history = model.fit(train_dataset,
epochs=num_epochs,
steps_per_epoch=train_steps_per_epoch,
validation_data=val_dataset,
validation_steps=dev_steps_per_epoch)
Notebook credits: digitalepidemiologylab covid-twitter-bert colab

How do I prepare the test set shape for the model.evaluate with keras and tensorflow?

I'm trying to run a simple example with a NN using the MNIST dataset provided by tensorflow itself, running on Google Colab. I want to get the raw data and mount by myself the structure that has the data. I'm able to train the NN, but when I try to predict one example from the test set, I get the error
ValueError: Error when checking input: expected dense_input to have shape (784,) but got array with shape (1,).
Could somebody help me with this issue? I'm pretty new to Python and Keras/TensorFlow.
When I run
print(inp.shape)
I get (784,) and not the (1,) as the error says.
I have also tried to evaluate the test set using
test_loss, test_accuracy = model.evaluate(test_input.T)
, but I also get the error
ValueError: Arguments and signature arguments do not match: 25 27.
The source code is the following:
# Importing stuff
import tensorflow as tf
import tensorflow_datasets as tfds
import matplotlib.pyplot as plt
import numpy as np
import math
import time
import keras
tf.enable_eager_execution()
# Functions
def normalize(images, labels):
images = tf.cast(images, tf.float32)
images /= 255
return images, labels
# Getting dataset
ds, meta = tfds.load('fashion_mnist', as_supervised=True, with_info=True)
test_ds, train_ds = ds['test'], ds['train']
# Preprocess the data
train_ds = train_ds.map(normalize)
test_ds = test_ds.map(normalize)
num_train_examples = meta.splits['train'].num_examples
num_test_examples = meta.splits['test'].num_examples
# Making the train set
train_input = np.empty(shape=(784, num_train_examples))
train_label = np.empty(shape=(1, num_train_examples))
i = 0
for image, label in train_ds:
image = image.numpy().reshape((784, 1))
train_input[:, i] = image.ravel()
label = label.numpy().reshape(1)
train_label[:, i] = label
i = i + 1;
# Making the test set
test_input = np.empty(shape=(784, num_test_examples))
test_label = np.empty(shape=(1, num_test_examples))
i = 0
for image, label in test_ds:
image = image.numpy().reshape((784, 1))
test_input[:, i] = image.ravel()
label = label.numpy().reshape(1)
test_label[:, i] = label
i = i + 1;
# Network
input_layer = tf.keras.layers.Dense(units=784, input_shape=[784])
h1 = tf.keras.layers.Dense(128, activation=tf.nn.relu)
output_layer = tf.keras.layers.Dense(10, activation=tf.nn.softmax)
model = tf.keras.Sequential([input_layer, h1, output_layer])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(train_input.T, train_label.T, epochs=3, steps_per_epoch=100, batch_size=1)
test_loss, test_accuracy = model.evaluate(test_input.T)
inp = test_input[:, 0].T
res = model.predict(inp)
All the API functions expect an input shape containing the batch size first. In your case, you're trying to feed only 1 example and thus no batch size is given. You just need to specify your batch size as 1 by reshaping the data.
Using numpy:
res = model.predict(np.reshape(inp, len(inp))
The argument to the predict method now receives the array with shape (1, 784) in your case, specifying the batch size as 1.
When you give the function more examples to evaluate by stacking them on top of one another, the batch size is implicitly given by the shape of the array, so no further transformation is necessary.

NameError: name 'classifier' is not defined

I am new to machine learning. I was trying to predict on a dataset but when I run the program it give me following error:
NameError: name 'classifier' is not defined
Here is my code:
import numpy as np
from keras.preprocessing import image
test_image = image.load_img('dataset/single_prediction/1.jpg', target_size = (64, 64))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = classifier.predict(test_image)
training_set.class_indices
if result[0][0] == 1:
prediction = 'nsfw'
else:
prediction = 'sfw'
You are using classifier to make predictions. But the classifier is not defined. That is what the error is.
To solve this, You must have the saved keras model that is trained for your specific problem with it. If you have that, you can load it and make predictions.
Below code shows how you can load the model.
from keras.models import load_model
classifier = load_model('path_to_your_model')
After the model is loaded you can use that to make predictions like you do.
import numpy as np
from keras.preprocessing import image
test_image = image.load_img('dataset/single_prediction/1.jpg', target_size = (64, 64))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = classifier.predict(test_image)
training_set.class_indices
if result[0][0] == 1:
prediction = 'nsfw'
else:
prediction = 'sfw'
You have to specify an 'empty' version, before you start adding the layers into the model.
You can simply fix this error by adding this line above your code:
import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.models import load_model
#empty model
classifier = Sequential()
Then continue with specifying like:
#add layers, start with hidden layer and first deep layer
classifier.add(Dense(output_dim=15, init="uniform", activation='relu',input_dim = 15))
classifier.add(Dropout(rate=0.1))

Resources