I want to use AdaBoostClassifier on CNN. I created a function called create_model that define my model and compile it. The function return the compiled CNN. Then I use KerasClassifier:
model = KerasClassifier(build_fn=create_model, epochs=2, batch_size=128, verbose=0, sample_weight=None)
bdt_discrete = AdaBoostClassifier( model, n_estimators=2, learning_rate=1.5, algorithm=”SAMME”)
bdt_discrete.fit(X_train, y_train, sample_weight=None)
It gives the following error:
ValueError: KerasClassifier doesn’t support sample_weight.
Is there any solutions for the error:
File “/usr/local/lib/python2.7/dist-packages/spyder/utils/site/sitecustomize.py”, line 688, in runfile
execfile(filename, namespace)
File “/usr/local/lib/python2.7/dist-packages/spyder/utils/site/sitecustomize.py”, line 93, in execfile
builtins.execfile(filename, *where)
File “~adaboost_CNN3.py”, line 251, in
bdt_discrete.fit(X_train, y_train, sample_weight=None)
File “/usr/local/lib/python2.7/dist-packages/sklearn/ensemble/weight_boosting.py”, line 413, in fit
return super(AdaBoostClassifier, self).fit(X, y, sample_weight)
File “/usr/local/lib/python2.7/dist-packages/sklearn/ensemble/weight_boosting.py”, line 130, in fit
self._validate_estimator()
File “/usr/local/lib/python2.7/dist-packages/sklearn/ensemble/weight_boosting.py”, line 431, in validate_estimator
% self.base_estimator.class.name)
ValueError: KerasClassifier doesn’t support sample_weight.
Please let me know if there are any advice.
For anyone dealing with this issue now, according to the source code the KerasClassifier, KerasRegressor classes packaged with tensorflow are now deprecated.
The official recommendation is to instead use https://github.com/adriangb/scikeras
Related
I just converted an existing project from TF 1.14 to TF 2.1 which uses the TPUEstimator API. After making the conversion, testing locally (i.e. use_tpu=False) runs successfully. However, I am getting errors when running on Google Cloud TPU (i.e. use_tpu=True).
Note: This is in the context of the AdaNet AutoML framework (v0.8.0), although I suspect this may be a general TPUEstimator-related error, as the errors appear to originate in the tpu_estimator.py and error_handling.py scripts seen in the Traceback below:
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3032, in train
rendezvous.record_error('training_loop', sys.exc_info())
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 81, in record_error
if value and value.op and value.op.type == _CHECK_NUMERIC_OP_NAME:
AttributeError: 'RuntimeError' object has no attribute 'op'
During handling of the above exception, another exception occurred:
File "workspace/trainer/train.py", line 331, in <module>
main(args=parsed_args)
File "workspace/trainer/train.py", line 177, in main
run_config=run_config)
File "workspace/trainer/train.py", line 68, in run_experiment
estimator.train(input_fn=train_input_fn, max_steps=total_train_steps)
File "/usr/local/lib/python3.6/site-packages/adanet/core/estimator.py", line 853, in train
saving_listeners=saving_listeners)
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3035, in train
rendezvous.raise_errors()
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/error_handling.py", line 143, in raise_errors
six.reraise(typ, value, traceback)
File "/usr/local/lib/python3.6/site-packages/six.py", line 703, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3030, in train
saving_listeners=saving_listeners)
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 374, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1164, in _train_model
return self._train_model_default(input_fn, hooks, saving_listeners)
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1194, in _train_model_default
features, labels, ModeKeys.TRAIN, self.config)
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2857, in _call_model_fn
config)
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1152, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 3186, in _model_fn
host_ops = host_call.create_tpu_hostcall()
File "/usr/local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/tpu/tpu_estimator.py", line 2226, in create_tpu_hostcall
'dimension, but got scalar {}'.format(dequeue_ops[i][0]))
RuntimeError: All tensors outfed from TPU should preserve batch size dimension, but got scalar Tensor("OutfeedDequeueTuple:1", shape=(), dtype=int64, device=/job:tpu_worker/task:0/device:CPU:0)'
The previous version of the project using TF 1.14 runs both locally and on TPU using TPUEstimator without issues. Is there something obvious I am potentially missing for the conversion over to TF 2.1 when using TPUEstimator API?
Have you applied the following:
dataset = ...
dataset = dataset.apply(tf.contrib.data.batch_and_drop_remainder(batch_size))
this potentially drops the last few samples from a file to ensure that every batch has a static shape of batch_size, which is required when training on TPUs.
I need your help please...
I am trying go get the following Text Classification Module working:
# Train and validate model.
history = model.fit(x_train,
train_labels,
epochs=epochs,
callbacks=callbacks,
validation_data=(x_val, val_labels),
verbose=2,
batch_size=batch_size) # Logs once per epoch.
Source File Can be Found Here: Google - Git Hub Text Classification Code
However I am getting the following error on execution:
Traceback (most recent call last):
File "train_ngram_model.py", line 113, in <module>
train_ngram_model(data)
File "train_ngram_model.py", line 93, in train_ngram_model
batch_size=batch_size) # Logs once per epoch.
File "C:\Users\joebloggs\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\python\keras\engine\training.py", line 819, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\joebloggs\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 235, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\joebloggs\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 593, in _process_training_inputs
use_multiprocessing=use_multiprocessing)
File "C:\Users\joebloggs\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 646, in _process_inputs
x, y, sample_weight=sample_weights)
File "C:\Users\joebloggs\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\python\keras\engine\training.py", line 2383, in _standardize_user_data
batch_size=batch_size)
File "C:\Users\joebloggs\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\python\keras\engine\training.py", line 2428, in _standardize_tensors
converted_x.append(_convert_scipy_sparse_tensor(a, b))
File "C:\Users\joebloggs\AppData\Roaming\Python\Python37\site-packages\tensorflow_core\python\keras\engine\training.py", line 3198, in _convert_scipy_sparse_tensor
raise ValueError('A SciPy sparse matrix was passed to a model '
ValueError: A SciPy sparse matrix was passed to a model that expects dense inputs. Please densify your inputs first, such as by calling `x.toarray()`.
I have spent several hours now to find a solution, and I haven't gotten anywhere.
Thank you in advance for your reply.
The relatively new keras-tuner module for tensorflow-2 is causing the error 'Failed to create a NewWriteableFile'. The tuner.search function is working, it is only after the trial completes that the error is thrown. This is a tutorial from the sentdex Youtube channel.
Here is the code:
from tensorflow import keras
from tensorflow.keras.datasets import fashion_mnist
from tensorflow.keras.layers import Dense, Conv2D, MaxPooling2D, Activation, Flatten
from kerastuner.tuners import RandomSearch
from kerastuner.engine.hyperparameters import HyperParameters
import matplotlib.pyplot as plt
import time
(x_train, y_train), (x_test, y_test) = fashion_mnist.load_data()
x_train = x_train[:1000].reshape(-1, 28, 28, 1)
x_test = x_test[:100].reshape(-1, 28, 28, 1)
y_train = y_train[:1000]
y_test = y_test[:100]
# x_train = x_train.reshape(-1, 28, 28, 1)
# x_test = x_test.reshape(-1, 28, 28, 1)
LOG_DIR = f"{int(time.time())}"
def build_model(hp):
model = keras.models.Sequential()
model.add(Conv2D(hp.Int("layer1_channels", min_value=32,
max_value=256, step=32), (3,3), input_shape=x_train.shape[1:]))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
for i in range(hp.Int("n_layers", 1, 4)):
model.add(Conv2D(hp.Int(f"conv_{i}_channels", min_value=32,
max_value=256, step=32), (3,3)))
model.add(Flatten())
model.add(Dense(10))
model.add(Activation('softmax'))
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
return model
tuner = RandomSearch(build_model,
objective = "val_accuracy",
max_trials = 1,
executions_per_trial = 1,
directory = LOG_DIR,
project_name = 'junk')
tuner.search(x_train,
y_train,
epochs=1,
batch_size=64,
validation_data=(x_test, y_test))
This is the traceback printout:
(tf_2.0) C:\Users\redex\OneDrive\Documents\Education\Sentdex Tutorials\Keras-Tuner>C:/Users/redex/Anaconda3/envs/tf_2.0/python.exe "c:/Users/redex/OneDrive/Documents/Education/Sentdex Tutorials/Keras-Tuner/keras-tuner.py"
2019-12-21 10:07:47.556531: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations: AVX AVX2
To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags.
2019-12-21 10:07:47.574699: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 8. Tune using inter_op_parallelism_threads for best performance.
Train on 1000 samples, validate on 100 samples
960/1000 [===========================>..] - ETA: 0s - loss: 64.0616 - accuracy: 0.2844
2019-12-21 10:07:55.080024: W tensorflow/core/framework/op_kernel.cc:1622] OP_REQUIRES failed at save_restore_v2_ops.cc:109 : Not found: Failed to create a NewWriteableFile: 1576951667\junk\trial_c5a5436b1d28a85446ce55c8d13f9657\checkpoints\epoch_0\checkpoint_temp_8a230a5ae2d046098456d1fdfc696690/part-00000-of-00001.data-00000-of-00001.tempstate15377864750281844169 : The system cannot find the path specified.
; No such process
Traceback (most recent call last):
File "c:/Users/redex/OneDrive/Documents/Education/Sentdex Tutorials/Keras-Tuner/keras-tuner.py", line 65, in <module>
validation_data=(x_test, y_test))
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\kerastuner\engine\base_tuner.py", line 122, in search
self.run_trial(trial, *fit_args, **fit_kwargs)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\kerastuner\engine\multi_execution_tuner.py", line 95, in run_trial
history = model.fit(*fit_args, **fit_kwargs, callbacks=callbacks)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 372, in fit
prefix='val_')
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\contextlib.py", line 119, in __exit__
next(self.gen)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 685, in on_epoch
self.callbacks.on_epoch_end(epoch, epoch_logs)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\keras\callbacks.py", line 298, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\keras\callbacks.py", line 965, in on_epoch_end
self._save_model(epoch=epoch, logs=logs)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\keras\callbacks.py", line 999, in _save_model
self.model.save_weights(filepath, overwrite=True)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\keras\engine\network.py", line 1090, in save_weights
self._trackable_saver.save(filepath, session=session)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\training\tracking\util.py", line 1155, in save
file_prefix=file_prefix_tensor, object_graph_tensor=object_graph_tensor)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\training\tracking\util.py", line 1103, in _save_cached_when_graph_building
save_op = saver.save(file_prefix)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\training\saving\functional_saver.py", line 230, in save
sharded_saves.append(saver.save(shard_prefix))
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\training\saving\functional_saver.py", line 72, in save
return io_ops.save_v2(file_prefix, tensor_names, tensor_slices, tensors)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\ops\gen_io_ops.py", line 1932, in save_v2
ctx=_ctx)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\ops\gen_io_ops.py", line 1969, in save_v2_eager_fallback
ctx=_ctx, name=name)
File "C:\Users\redex\Anaconda3\envs\tf_2.0\lib\site-packages\tensorflow_core\python\eager\execute.py", line 67, in quick_execute
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.NotFoundError: Failed to create a NewWriteableFile: 1576951667\junk\trial_c5a5436b1d28a85446ce55c8d13f9657\checkpoints\epoch_0\checkpoint_temp_8a230a5ae2d046098456d1fdfc696690/part-00000-of-00001.data-00000-of-00001.tempstate15377864750281844169 : The system cannot find the path specified.
; No such process [Op:SaveV2]
My machine is Windows 10
The keras-tuner documentation specifies Tensorflow 2.0 and Python 3.6 but I'm using 3.7.4. I presume more recent is OK. I'm no software expert so this is about all I know, any help is appreciated.
I had the similas problem while using kerastuner in Windows and I've solved it:
The first issue is that the path to the log directory may be too long. I had to reduced it.
The second problem is that python (or tf) doens't work in Windows with mixed slashes. But kerastuner forms the path with backslashes. So I should provide the path with backslashes. I've done this with os.path.normpath() method:
tuner=RandomSearch(build_model,objective='val_accuracy',max_trials=10,directory=os.path.normpath('C:/'))
tuner.search(x_train,y_train,batch_size=256,epochs=30,validation_split=0.2,verbose=1)
Now I don't receive this error.
In my case, the path exceeds the maximum length of path in windows because the length of generated path by Keras Turner is about 170. After I make my folder shorter, it works normally.
The problem it would appear is a Windows issue. Running the same code in a Linux environment had no issue in this regard.
System information
OS Platform and Distribution: Ubuntu 16.04 LTS
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 1.12.0
Python version: 3.5.2
CUDA/cuDNN version: release 9.0, V9.0.176
GPU model and memory: Tesla K80, 12 GB
Describe the current behavior
When I try to save my model using model.save() where model is a tf.keras.Model instance, it throws a TypeError: ('Not JSON Serializable:', <tf.Variable 'Variable:0' shape=() dtype=float32>) .
I am using a tf.keras.backend.variable() in loss_weights in model.compile.
Optimizer: tf.keras.optimizers.Adam
Interestingly, when I try to save my model weights only using model.save_weights where model is a tf.keras.Model instance, it works fine, NO ERROR.
Code to reproduce the issue
alpha = tf.keras.backend.variable(0.25)
Code any model using tf.keras
model= get_model()
model.compile(optimizer=optimizer,loss={"softmax1":generalized_dice_loss,"softmax2":generalized_dice_loss}, loss_weights=[1.0,alpha])
Do training using model.fit()-
model.save()
Other info / logs
Traceback (most recent call last):
File "main_latest.py", line 45, in
max_queue_size=10)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 2177, in fit_generator
initial_epoch=initial_epoch)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/training_generator.py", line 216, in fit_generator
callbacks.on_epoch_end(epoch, epoch_logs)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/callbacks.py", line 214, in on_epoch_end
callback.on_epoch_end(epoch, logs)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/callbacks.py", line 601, in on_epoch_end
self.model.save(filepath, overwrite=True)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/network.py", line 1363, in save
save_model(self, filepath, overwrite, include_optimizer)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/saving.py", line 134, in save_model
default=serialization.get_json_type).encode('utf8')
File "/usr/lib/python3.5/json/init.py", line 237, in dumps
**kw).encode(obj)
File "/usr/lib/python3.5/json/encoder.py", line 198, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.5/json/encoder.py", line 256, in iterencode
return _iterencode(o, 0)
File "/home/tejal/.local/lib/python3.5/site-packages/tensorflow/python/util/serialization.py", line 64, in get_json_type
raise TypeError('Not JSON Serializable:', obj)
TypeError: ('Not JSON Serializable:', <tf.Variable 'Variable:0' shape=() dtype=float32>)
Screenshot of Error:
model.save is trying to save a tf.Variable which is not JSON serializable.
model.fit saves everything, not just the model weights. I've seen this problem when my optimizer had a tf.Tensor which cannot be serialized.
Everything points to alpha being your problem in this case. Also the documentation for model.compile states that the loss_weights should be a Python float.
So using alpha.numpy() should solve your problem:
model.compile(optimizer=optimizer,loss={"softmax1":generalized_dice_loss,"softmax2":generalized_dice_loss}, loss_weights=[1.0,alpha.numpy()])
My try to obtain the batch size within a custom loss function using K.int_shape() demonstrated by the code below.
from keras import layers, Input, Model
import keras.backend as K
import numpy as np
train_X=np.random.random([100, 5])
train_Y=train_X.sum(axis=1)
inputs=Input(shape=(5,), dtype='float32', name='posts')
outputs=layers.Dense(1, activation='relu')(inputs)
model = Model(inputs, outputs)#, net_qc])
model.summary()
def myloss(y_true, y_pred):
n=K.int_shape(y_pred)[0]
return K.sum(y_pred)/n
model.compile(optimizer='adam', loss=myloss)
model.fit(train_X, train_Y, epochs=10, batch_size=10)
The error message below suggest K.int_shape returns None. I have tried several things without success, would really appreciate some helps.
Traceback (most recent call last):
File "./test_intshape.py", line 21, in <module>
model.compile(optimizer='adam', loss=myloss)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/keras/engine/training.py", line 830, in compile
sample_weight, mask)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/keras/engine/training.py", line 429, in weighted
score_array = fn(y_true, y_pred)
File "./test_intshape.py", line 19, in myloss
return K.sum(y_pred)/n
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 820, in binary_op_wrapper
y = ops.convert_to_tensor(y, dtype=x.dtype.base_dtype, name="y")
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 639, in convert_to_tensor
as_ref=False)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 704, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/home/ubuntu/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 360, in make_tensor_proto
raise ValueError("None values not supported.")
ValueError: None values not supported.
That is the expected behaviour because K.int_shape() doesn't return a symbolic tensor but the current known shape. Well you would only know the batch size at runtime and when constructing the graph it will be None. What you are looking for is K.shape() instead which will return the symbolic tensor that will have the batch size set at runtime, ie:
n = K.shape(y_pred)[0]