I get the following error when I train LightGBM model:
# Train the model
import lightgbm as lgb
lgb_train = lgb.Dataset(x_train, y_train)
lgb_val = lgb.Dataset(x_test, y_test)
parameters = {
'application': 'binary',
'objective': 'binary',
'metric': 'auc',
'is_unbalance': 'true',
'boosting': 'gbdt',
'num_leaves': 31,
'feature_fraction': 0.5,
'bagging_fraction': 0.5,
'bagging_freq': 20,
'learning_rate': 0.05,
'verbose': 0
}
model = lgb.train(parameters,
train_data,
valid_sets=test_data,
num_boost_round=5000,
early_stopping_rounds=100)
y_pred = model.predict(test_data)
If you used cut or qcut functions for binning and did not encode later (one-hot encoding, label encoding ..). this may be the cause of the error. Try to use an encoding.
I hope it works.
I had what might be the same problem.
Post the whole traceback to make sure.
For me it was a problem serializing to JSON, which LightGBM does under the hood to save the booster for later use.
Check your dataset for any date/datetime columns, or anything that remotely looks like a date, and either drop it or convert to something JSON can handle.
Mine had all been converted to categorical dtype by some Pandas code I had poorly written, and I usually do the initial GBM run fairly fast-n-dirty to see what variables show up as important. LightGBM let me make the data binaries for training (i.e. it would have thrown an error if they were datetime or timedelta dtypes before letting me run anything). It would run the training just fine, report an AUC, then fail after the last training step when it was dumping the categoricals to JSON. It was maddening, with a cryptic traceback.
Hope this helps.
If you have any time delta variable in the dataset convert it into an int using the dt.days attribute. I faced the same issue it is the issue reported in Github of light gbm
Related
I am trying to use GridSearchCV with xgbranker estimator from xgboost. I am trying to use GroupKFold and passing qid (group_ids) parameter to the grid fit method but it's not straightforward. After a bit of hit and trial with solutions already suggested on the web, I finally zeroed on a approach. I am still getting an error which seems to be in the scoring method passed. Any help or working example would be great?
Sample code:
from sklearn.model_selection import GroupKFold, GridSearchCV
from sklearn.metrics import make_scorer, ndcg_score
ndcg_scorer = make_scorer(ndcg_score)
param_grid = {
'learning_rate': [0.001, 0.01, 0.02],
'n_estimators': [10, 50]
}
splits = 3
gkf = GroupKFold(n_splits=splits)
cv_group = gkf.split(X_train, y_train, qids_train)
def group_gen():
for ids,_ in cv_group:
yield ids
grid = GridSearchCV(my_model, param_grid, cv=splits, scoring=ndcg_scorer, refit=False)
grid.fit(X_train, y_train, qid=next(group_gen()))
I get below error:
ValueError: Only ('multilabel-indicator', 'continuous-multioutput', 'multiclass-multioutput') formats are supported. Got multiclass instead
The error seems to be related to the scoring method you use, but you didn't share anything about your data. so it's hard to say what exactly is the problem.
It seems to me that you're using for the scoring a method that expects something else then you're providing as a label.
I'm trying to do a simple save of a resnet50 model and I'm getting an error. My code to reproduce the error:
from tensorflow import keras
import keras_resnet
inputs = keras.layers.Input(shape=(None, None, 3))
resnet = keras_resnet.models.ResNet50(inputs, include_top=False, freeze_bn=True)
resnet.save("my-model")
I get the error: "KeyError: 'inputs'". Is this a bug or is there something I'm missing with the keras save command? I tried the command on MacOS and in my ubuntu container. Same result.
EDIT: it's working with the official keras implementation of resnet. With this implementation though I have to change the code in resnet.py file of fizyr implementation of keras-retinanet. Specifically, having defined resnet with:
from keras.applications.resnet import ResNet50,ResNet101,ResNet152
resnet = ResNet50(input_tensor=inputs, include_top=False)
I have to change the code of the backbone layers from :
backbone_layers = {
'C2': resnet.outputs[0],
'C3': resnet.outputs[1],
'C4': resnet.outputs[2],
'C5': resnet.outputs[3]
}
to:
backbone_layers = {
'C2': resnet.layers[-137].output,
'C3': resnet.layers[-95].output,
'C4': resnet.layers[-33].output,
'C5': resnet.outputs[0]
}
I didn't test it yet but I think it should work.
The only caveat I see is that I don't have the freeze_bn parameter anymore. See https://github.com/fizyr/keras-retinanet/issues/974 for the reason of this parameter. I hope it will not adversely affect the training of my network.
You need to save the model with a format, e.g. h5.
I reproduced your error, fixed it with:
resnet.save("mymodel.h5")
I have saved more than 1000 models for each item. Now I need to load all these models into memory (a dataframe) to do predictions. If I just use "for" loop to load these models, each loading will be 3 seconds slower than the previous model loading. So I try to use multiprocessing.pool (ThreadPool).
But, strangely, using ThreadPool will cause the prediction "ValueError: Tensor Tensor". If using normal loading, the prediction is fine.
I tried thread also got error msg
#following code will lead to ValueError
from multiprocessing.pool import ThreadPool as Pool
def load_model(stock):
model_pred.at[0, stock] = keras.models.load_model (
'C:/Users/chenp/Documents/rqpro/models/{}_model.h5'.format (stock))
pool = Pool(processes=16)
for stock in trade_stocks['stock']:
pool.map (load_model, (stock,))
#Prediction
for stock in trade_stocks['stock']:
model = model_pred.loc[0, stock]
prediction = model.predict(pred_data)
#Get following msg:
ValueError: Tensor Tensor("dense_9/Softmax:0", shape=(?, 2), dtype=float32) is not an element of this graph.
#Normal code but too low efficient
for stock in trade_stocks['stock']:
model_pred.at[0, stock] = keras.models.load_model(
'C:/Users/chenp/Documents/rqpro/models/{}_model.h5'.format(stock))
#Get following msg:
ValueError: Tensor Tensor("dense_9/Softmax:0", shape=(?, 2), dtype=float32) is not an element of this graph.
This happens as Keras is not thread safe. For solving this problem, please use _make_predict_function() before predicting. For detailed answer, please check
I am working on OCR model. my final goal is to convert OCR code into coreML and deploy it into ios.
I have looked and run a couple of the github source codes namely:
here
here
as you have a look on them they all implemented loss as a custom layer with lambda layer.
the problem start when I want to convert this to coreML.
my piece of the code to convert to CoreMl:
import coremltools
def convert_lambda(layer):
# Only convert this Lambda layer if it is for our swish function.
if layer.function == ctc_lambda_func:
params = NeuralNetwork_pb2.CustomLayerParams()
# The name of the Swift or Obj-C class that implements this layer.
params.className = "x"
# The desciption is shown in Xcode's mlmodel viewer.
params.description = "A fancy new loss"
return params
else:
return None
print("\nConverting the model:")
# Convert the model to Core ML.
coreml_model = coremltools.converters.keras.convert(
model,
# 'weightswithoutstnlrchangedbackend.best.hdf5',
input_names="image",
image_input_names="image",
output_names="output",
add_custom_layers=True,
custom_conversion_functions={"Lambda": convert_lambda},
)
but it raises error
Converting the model:
Traceback (most recent call last):
File "/home/sgnbx/Downloads/projects/CRNN-with-STN-master/CRNN_with_STN.py", line 201, in <module>
custom_conversion_functions={"Lambda": convert_lambda},
File "/home/sgnbx/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/coremltools/converters/keras/_keras_converter.py", line 760, in convert
custom_conversion_functions=custom_conversion_functions)
File "/home/sgnbx/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/coremltools/converters/keras/_keras_converter.py", line 556, in convertToSpec
custom_objects=custom_objects)
File "/home/sgnbx/anaconda3/envs/tf_gpu/lib/python3.6/site-packages/coremltools/converters/keras/_keras2_converter.py", line 255, in _convert
if input_names[idx] in input_name_shape_dict:
IndexError: list index out of range
Input name length mismatch
I am kind of not sure I can resolve this as I did not find anything relevant to this error to resolve.
In other hand most codes for OCR have Custom Loss function which probably again I face with the same problem.
So in the end I have two question:
Do you know how to resolve this error
my main question do you know any source code for OCR which is in KERAS (As i have to convert it to coreMl) and do not have custom loss function so it will be ok converting to CoreMl without problem?
Thanks in advance:)
just to make my question thorough:
this is the custom loss function in the source I am working:
def ctc_lambda_func(args):
iy_pred, ilabels, iinput_length, ilabel_length = args
# the 2 is critical here since the first couple outputs of the RNN
# tend to be garbage:
iy_pred = iy_pred[:, 2:, :] # no such influence
return backend.ctc_batch_cost(ilabels, iy_pred, iinput_length, ilabel_length)
loss_out = Lambda(ctc_lambda_func, output_shape=(1,), name='ctc')
([fc_2, labels, input_length, label_length])
and then use it in compile:
model.compile(loss={'ctc': lambda y_true, y_pred: y_pred}, optimizer=sgd)
CoreML doesn't allow you train model, so it's not important to have a loss function or not. If you only want to use CRNN as predictor on iOS , you should just convert base_model in second link.
I’m new to tensorflow. I takes images and returns a preference score for each image, using a pre-trained siamese cnn model. I would like to figure out what parts of my code are expensive. Is there a way to do that with tensorflow or tensorboard? Or does anyone see what parts of my code could be reworked to increase efficiency? Any tips are greatly appreciated.
Code:
# function to get preference scores for unseen items
def pref_score(Item_x,user_x):
from collections import defaultdict
from io import BytesIO
res_x = defaultdict(lambda: defaultdict(float))
for jjj in list(Item_x.keys()):
tstImg3=np.round(np.array(Image.open(BytesIO(Item_x[jjj][b'imgs'])).convert('RGB').resize((224,224)),dtype=np.float32))
rep_gan=tf.reshape(tstImg3, shape=[-1, 224, 224, 3])
with tf.device('/gpu:0'):
gan_image=rep_gan
image=tf.image.resize_nearest_neighbor(images=gan_image, size=[224,224], align_corners=None, name=None)
with tf.variable_scope("DVBPR") as scope:
scope.reuse_variables()
result = CNN(image,1.0)
user=tf.placeholder(dtype=tf.int32,shape=[1])
idx=tf.reduce_sum(tf.matmul(result,tf.transpose(tf.gather(thetau,user))),1)
res_x[jjj] = sess.run([gan_image,idx],feed_dict={user:[user_x]})[1]
print(user_x,"===>",jjj)
return res_x
# running function to get preference scores on images
test_res=pref_score(Item_x=test_item,user_x=1)