I am a new user for yellowbrick. While implementing a sklearn LogisticRegression API in yellowbrick ClassificationReport, I found some unusual error. I have tried many syntaxes as suggested by yellowbrick official document as well as in most data science community user (medium etc.), but still I am getting the same error. Though I am getting the ClassificationReport but the error is quite annoying.
#Using yellowbrick library
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import confusion_matrix, roc_auc_score, recall_score, roc_curve, accuracy_score, auc, classification_report, plot_confusion_matrix, plot_roc_curve, precision_score, f1_score
from yellowbrick.classifier import ClassificationReport, discrimination_threshold, classification_report
classes = [0,1]
fig = plt.gcf()
ax = plt.subplot(111)
visualizer = ClassificationReport(log_model,classes=[0,1], size=(400,400),fontsize=15, cmap='GnBu', ax = ax)
ax.grid(False)
plt.title("Classification Report", fontsize=18)
visualizer.fit(X_train, y_train)
visualizer.score(X_test, y_test)
visualizer.poof()
#visualizer.show() #I even tried this (This also gives me an error like LogisticRegression object has no attribute 'show'
The output I am getting is:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-89-f01efe3e9a6d> in <module>()
16 visualizer.fit(X_train, y_train)
17 visualizer.score(X_test, y_test)
---> 18 visualizer.poof()
2 frames
/usr/local/lib/python3.7/dist-packages/yellowbrick/utils/wrapper.py in __getattr__(self, attr)
40 def __getattr__(self, attr):
41 # proxy to the wrapped object
---> 42 return getattr(self._wrapped, attr)
AttributeError: 'LogisticRegression' object has no attribute 'fig'
Appreciate any suggestion to get rid of this error.
To add, currently I am using following scikit-learn and yellowbrick versions:
print(sklearn.__version__)
print(yellowbrick.__version__)
0.24.2
0.9.1
sklearn version 0.22.2.post1 and yellowbrick version 0.9.1 fix the issue for me. Install these by running:
pip install scikit-learn==0.22.2.post1
and
pip install yellowbrick==0.9.1
Related
I am trying to use keras-rl2 DQNAgent to solve the taxi problem in open AI Gym.
For a quick refresh, please find it in Gym-Documentation, thank you!
https://www.gymlibrary.dev/environments/toy_text/taxi/
Here are my process:
0.Open the Taxi-v3 environment from gym
1.Build the deep learning model by keras Sequential API with Embedding and Dense layers
2.Import the Epsilon Greedy policy and Sequential Memory deque from keras-rl2's rl
3.input the model, policy, and the memory in to rl.agent.DQNAgent and compile the model
But when i fit the model(agent) the error pops up:
Training for 1000000 steps ...
Interval 1 (0 steps performed)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-180-908ee27d8389> in <module>
1 agent.compile(Adam(lr=0.001),metrics=['mae'])
----> 2 agent.fit(env, nb_steps=1000000, visualize=False, verbose=1, nb_max_episode_steps=99, log_interval=100000)
/usr/local/lib/python3.8/dist-packages/rl/core.py in fit(self, env, nb_steps, action_repetition, callbacks, verbose, visualize, nb_max_start_steps, start_step_policy, log_interval, nb_max_episode_steps)
179 observation, r, done, info = self.processor.process_step(observation, r, done, info)
180 for key, value in info.items():
--> 181 if not np.isreal(value):
182 continue
183 if key not in accumulated_info:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I tried to run the code to Cart Pole problem there's no error came out. I am wondering if the states in taxi problem is just a scalar (500), not like cart-pole has a state of an array with 4 elements? Please help or a little advise will help a lot, also if you can help me to extend the steps more than 200 is better!!(env._max_episode_steps=5000)
#import environment and visualization
import gym
from gym import wrappers
!pip install gym[classic_control]
#import Deep Learning api
import tensorflow as tf
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.layers import Dense, Flatten, Input, Embedding,Reshape
from tensorflow.keras.optimizers import Adam
#import rl agent library
!pip install gym
!pip install keras
!pip install keras-rl2
#data manipulation
import numpy as np
import pandas as pd
import random
#0
env = gym.make('Taxi-v3')
env.reset()
actions=env.action_space.n
states=env.observation_space.n
#1
def build_model(states,actions):
model=Sequential()
model.add(Embedding(states,10, input_length=1))
model.add(Reshape((10,)))
model.add(Dense(32,activation='relu'))
model.add(Dense(32,activation='relu'))
model.add(Dense(actions,activation='linear'))
return model
#2
import rl
from rl.agents import DQNAgent
from rl.policy import EpsGreedyQPolicy
from rl.memory import SequentialMemory
policy=EpsGreedyQPolicy()
memory=SequentialMemory(limit=100000,window_length=1)
#3
agent=DQNAgent(model=model1,memory=memory,policy=policy,nb_actions=actions,nb_steps_warmup=500, target_model_update=1e-2)
agent.compile(Adam(lr=0.001),metrics=['mae'])
agent.fit(env, nb_steps=1000000, visualize=False, verbose=1, nb_max_episode_steps=99,
This ValueError comes from the way Keras RL handles the info returned by the environment. As you can see on the line https://github.com/keras-rl/keras-rl/blob/v0.4.2/rl/core.py#L181, it loops on each item of the info map and runs np.isreal(value).
And quoting the Taxi documentation for gym:
In v0.25.0, info["action_mask"] contains a np.ndarray for each of the action specifying if the action will change the state.
You can run gym.__version__ to confirm that you have a version greater or equal to 0.25.0.
To leverage the current Keras RL library (up to 0.4.2), you should install a gym version less than 0.25.0. Additionally, you can submit a PR to keras-rl to handle np.ndarray values without error.
I'm trying to use the pre-trained model of detectron2. While running the following code, it shows NotImplementedError.
import torch
torch.__version__
import torchvision
#torchvision.__version__
!pip install detectron2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.7/index.html
import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()
import numpy as np
import os, json, cv2, random
import matplotlib.pyplot as plt
%matplotlib inline
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog, DatasetCatalog
from detectron2.structures import BoxMode
cfg = get_cfg()
cfg.merge_from_file(model_zoo.get_config_file("COCOInstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCOInstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
predictor = DefaultPredictor(cfg)
And it shows the following error:
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
<ipython-input-27-699e754fc9df> in <module>
----> 1 predictor = DefaultPredictor(cfg)
4 frames
/usr/local/lib/python3.8/dist-packages/iopath/common/file_io.py in _isfile(self, path, **kwargs)
438 bool: true if the path is a file
439 """
--> 440 raise NotImplementedError()
441
442 def _isdir(self, path: str, **kwargs: Any) -> bool:
NotImplementedError:
I had the same issue and solved it by manually downloading .pkl file and giving path to cfg.MODEL.WEIGHTS variable
You can try this:
model_weights_url = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")
print(f"Downloading model from {model_weights_url}...")
local_model_weights_path = Path("./temp/downloads/model.pkl")
os.makedirs(local_model_weights_path.parent, exist_ok=True)
urllib.request.urlretrieve(model_weights_url, local_model_weights_path)
cfg.MODEL.WEIGHTS = str(local_model_weights_path)
I had the same issue just today. I manually downloaded the R-50.pkl from https://dl.fbaipublicfiles.com/detectron2/ImageNetPretrained/MSRA/R-50.pkl and set cfg.MODEL.WEIGHTS = "R-50.pkl" #path to file
How to troubleshoot this? I've tried setting dtype=None in the image.img_to_array method.
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import matplotlib.pyplot as plt
from keras.preprocessing import image
image_size = (180, 180)
batch_size = 32
model = keras.models.load_model('best_model.h5')
img = keras.preprocessing.image.load_img(
"GarnetCreek_7-15-2019.jpeg", target_size=image_size
)
img_array = image.img_to_array(img)
img_array = tf.expand_dims(img_array, 0) # Create batch axis
predictions = model.predict(img_array)
score = predictions[0]
This raises the following error:
Traceback (most recent call last):
img_array = image.img_to_array(img, dtype=None)
return image.img_to_array(img, data_format=data_format, **kwargs)
x = np.asarray(img, dtype=dtype)
return array(a, dtype, copy=False, order=order)
TypeError: __array__() takes 1 positional argument but 2 were given
Has anyone seen this before? Many thanks!
This error sometimes is due to a bug in Pillow 8.3.0 as it is here. (You may not use import PIL directly in your code, however some libraries such as tf.keras.preprocessing.image.load_img use PIL internally)
So, downgrading from PIL 8.3.0 to 8.2.0 may work.
Check PIL version:
import PIL
print(PIL.__version__)
If it is 8.3.0, then you may downgrade to 8.2.0:
!pip install pillow==8.2.0
valueError image part
Here is the code for eliminating features where I am getting value errors. I want to use recursive feature elimination without specifying any features . I tried to use the RFE(Recursion feature elemination) model to automatically eliminate weak features with each iteration which I have unable to do.HERE is the link of the dataset. https://drive.google.com/file/d/1neYnunu6a_Mdn3NfRZsF8wE4gwMCpjAY/view?usp=sharing .I will be grateful if you suggest me how to do it.
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from pandas import DataFrame
%matplotlib inline
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_selection import SelectFromModel
from sklearn.metrics import accuracy_score
from sklearn import datasets
from sklearn.metrics import classification_report
from google.colab import drive
drive.mount('/content/drive', force_remount=True)
df.keys()
x=pd.DataFrame(df)
x.head()
X = df.drop(["Sub_Cat"],axis=1).values
y = df["Sub_Cat"].values
X_train,X_test,y_train,y_test = train_test_split(X,y,random_state=0)
X_train.shape,X_test.shape
sel=SelectFromModel(RandomForestClassifier(n_estimators=100,random_state=0,n_jobs=-1))
sel.fit(X_train,y_train)
sel.get_support()
[I am getting value error in this part][1]
Then i tried to do this also getting `X = df.drop(["Dst_IP","Timestamp","Flow_ID","Src_IP","Sub_Cat"],axis=1).values
y = df["Sub_Cat"].values
X_train,X_test,y_train,y_test = train_test_split(X,y,random_state=0)
X = df.drop(["Dst_IP","Timestamp","Flow_ID","Src_IP","Sub_Cat"],axis=1).values
y = df["Sub_Cat"].values
X_train,X_test,y_train,y_test = train_test_split(X,y,random_state=0)
sel=SelectFromModel(RandomForestClassifier(n_estimators=100,random_state=0,n_jobs=-1))
sel.fit(X_train,y_train)
sel.get_support()
I am still getting error:
ValueError Traceback (most recent call last)
in ()
1 sel=SelectFromModel(RandomForestClassifier(n_estimators=100,random_state=0,n_jobs=-1))
----> 2 sel.fit(X_train,y_train)
3 sel.get_support()
3 frames
/usr/local/lib/python3.7/dist-packages/numpy/core/_asarray.py in asarray(a, dtype, order)
81
82 """
---> 83 return array(a, dtype, copy=False, order=order)
84
85
ValueError: could not convert string to float: 'Anomaly'
Firstly I'm running on Ubuntu and using Anaconda for my kernels in Jupyter Notebook.
I keep getting the following error. I believe it suggests I'm running out of memory but I'm not sure how to solve this. Someone suggested uninstalling Python3 32 and installing Python3 64 but this broke my Ubuntu installation immediately after uninstalling python (whoops) and I had to reinstall.
---------------------------------------------------------------------------
MemoryError Traceback (most recent call last)
<ipython-input-25-8b765500013f> in <module>()
----> 1 _, S, _ = np.linalg.svd(features)
2 print("Condition number from the SVD: {0:.1f}".format(np.max(S)/np.min(S)))
3 print("Condition number from cond: {0:.1f}".format(np.linalg.cond(features)))
/home/n/anaconda/lib/python3.5/site-packages/numpy/linalg/linalg.py in svd(a, full_matrices, compute_uv)
1387
1388 signature = 'D->DdD' if isComplexType(t) else 'd->ddd'
-> 1389 u, s, vt = gufunc(a, signature=signature, extobj=extobj)
1390 u = u.astype(result_t, copy=False)
1391 s = s.astype(_realType(result_t), copy=False)
MemoryError:
I'm trying to process a dataset with a shape (91946, 171), all numerical, using the code below.
import pandas as pd
import numpy as np
import matplotlib.pylab as plt
pd.set_option('display.max_columns',180)
dfk = pd.read_csv('data/kick.csv')dfk = pd.read_csv('data/kick.csv')
response = dfk.state
features = dfk
features.drop('state', axis=1, inplace=True)
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(features, response,
random_state=321)
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
logreg = LogisticRegression()
logreg.fit(X_train,y_train)
ypred_lr = logreg.predict(X_test)
print("Accuracy on the test set: {0:.3f}".format(accuracy_score(ypred_lr, y_test)))
_, S, _ = np.linalg.svd(features) #fails here
print("Condition number from the SVD: {0:.1f}".format(np.max(S)/np.min(S)))
print("Condition number from cond: {0:.1f}".format(np.linalg.cond(features)))