I cannot succeed to use xgboost package in Azure Machine Learning Studio interpreter. I am trying to import a model using xgboost that I trained in order to deploy it here. But It seems that my package is not set correctly because I cannot access to certain functions, particularly "xgboost.sklearn"
My model is of course using the xgboost.sklearn.something to do the classification
I tried to implement the package following two different methods :
from the tar.gz principle like here :
How can certain python libraries be imported in azure ML?Like the line import humanfriendly gives error
and also by clean package with the sandbox like there :
Uploading xgboost to azure machine learning: %1 is not a valid Win32 application\r\nProcess returned with non-zero exit code 1
import sys
import sklearn
import pandas as pd
import pickle
import xgboost
def azureml_main(dataframe1 = None, dataframe2 = None):
sys.path.insert(0,".\Script Bundle")
model = pickle.load(open(".\\Script
Bundle\\xgboost\\XGBv1.pkl",'rb'))
dataframe1, dftrue = filterdata(dataframe1)
## one processing step
pred = predictorV1(dataframe1,dftrue)
dataframe1['Y'] = pred
return dataframe1
Here is the error I get
Error 0085: The following error occurred during script evaluation, please view the output log for more information:
---------- Start of error message from Python interpreter ----------
Caught exception while executing function: Traceback (most recent call last):
File "C:\server\invokepy.py", line 199, in batch
odfs = mod.azureml_main(*idfs)
File "C:\temp\1098d8754a52467181a9509ed16de8ac.py", line 89, in azureml_main
model = pickle.load(open(".\\Script Bundle\\xgboost\\XGBv1.pkl", 'rb'))
ImportError: No module named 'xgboost.sklearn'
Process returned with non-zero exit code 1
---------- End of error message from Python interpreter ----------
Start time: UTC 05/22/2019 13:11:08
End time: UTC 05/22/2019 13:11:49
Related
I am unable to train a self-supervised(ssl) model to create image embeddings using the lightly cli: Lightly Platform Link. I intend to select diverse example from my dataset to create an object detection model further downstream and the image embeddings created with the ssl model will help me to perform Active Learning.I have reproduced the error in the Notebook with public access -----> lightly_app_troubleshooting_stackoverflow.ipynb Link.
In the notebook shared above this cmd raises an exception:
!source /content/venv_1/bin/activate;lightly-magic \
input_dir="/content/Sunflowers" trainer.max_epochs=20 \
token='< your lightly token(free account) >' \
new_dataset_name="sunflowers_dataset" loader.batch_size=64
The exception stack trace produced is as below:
/content/venv_1/lib/python3.7/site-packages/hydra/_internal/hydra.py:127: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/next/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
configure_logging=with_log_configuration,
########## Starting to train an embedding model.
/content/venv_1/lib/python3.7/site-packages/pytorch_lightning/core/lightning.py:23: LightningDeprecationWarning: pytorch_lightning.core.lightning.LightningModule has been deprecated in v1.7 and will be removed in v1.9. Use the equivalent class from the pytorch_lightning.core.module.LightningModule class instead.
"pytorch_lightning.core.lightning.LightningModule has been deprecated in v1.7"
Error executing job with overrides: ['input_dir=/content/Sunflowers', 'trainer.max_epochs=20', 'token=5bbcf60e3a5c7c266dcd4e0e9056c8301684e0f2f8922bc5', 'new_dataset_name=sunflowers_dataset', 'loader.batch_size=64']
Traceback (most recent call last):
File "/content/venv_1/lib/python3.7/site-packages/lightly/cli/lightly_cli.py", line 115, in lightly_cli
return _lightly_cli(cfg)
File "/content/venv_1/lib/python3.7/site-packages/lightly/cli/lightly_cli.py", line 52, in _lightly_cli
checkpoint = _train_cli(cfg, is_cli_call)
File "/content/venv_1/lib/python3.7/site-packages/lightly/cli/train_cli.py", line 137, in _train_cli
encoder.train_embedding(**cfg['trainer'], strategy=distributed_strategy)
File "/content/venv_1/lib/python3.7/site-packages/lightly/embedding/_base.py", line 88, in train_embedding
trainer = pl.Trainer(**kwargs, callbacks=[self.checkpoint_callback])
File "/content/venv_1/lib/python3.7/site-packages/pytorch_lightning/utilities/argparse.py", line 345, in insert_env_defaults
return fn(self, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'weights_summary'
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
I could not create a new tag - "lightly" as I lack the stackoverflow reputation points to do so.
The error is from an incompatibility with the latest PyTorch Lightning version (version 1.7 at the time of this writing). A quick fix is to use a lower version (e.g. 1.6). We are working on a fix :)
Let me know in case that does not work for you!
I am trying to package the following Python code into an executable file using PyInstaller:
import pandas as pd
import teradatasql
with teradatasql.connect(host='abcdxxx', user='abcdxxx', password='abcdxxx') as connect:
query = "SHOW TABLE AdventureWorksDW.DimAccount"
df = pd.read_sql(query, connect)
print(df)
When I run the .exe file, it gives me the error:
PyInstallerImportError: Failed to load dynlib/dll
'C:\\Users\\GAX~1.P\\AppData\\Local\\Temp\\_MEI153202\\teradatasql\\teradatasql.dll'.
Most likely this dynlib/dll was not found when the application was frozen.
[9924] Failed to execute script 'hello' due to unhandled exception!
I tried to make the following changes to the .spec file:
b = [
('C:\Users\Path_to_Python\Python\Python310\Lib\site-
packages\teradatasql\teradatasql.dll', '.\\teradatasql')
]
a = Analysis(['hello.py'],
pathex=[],
binaries=b,
datas=[] # , .....
)
But it doesn't resolve the problem. How to fix this?
We provide an article explaining how to include the Teradata SQL Driver for Python into an application packaged by PyInstaller:
https://support.teradata.com/community?id=community_blog&sys_id=c327eac51b1e9c103b00bbb1cd4bcb37
When I convert a my trained pytorch model to coreml model, I got this error:
File "/Users/lion/Documents/MyLab/web_workspace/sky_replacement/venv/lib/python3.9/site-packages/torch/jit/_serialization.py", line 161, in load
cpp_module = torch._C.import_ir_module(cu, str(f), map_location, _extra_files)
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
This is my code:
from networks import *
import coremltools as ct
run_device = torch.device("cpu")
net_G = define_G(input_nc=3, output_nc=1, ngf=64,
netG='coord_resnet50').to(run_device)
checkpoint = torch.load('./model/best_ckpt.pt', map_location=run_device)
net_G.load_state_dict(checkpoint['model_G_state_dict'])
net_G.to(run_device)
net_G.eval()
model = ct.convert('./model/best_ckpt.pt', source='pytorch', inputs=[ct.ImageType()], skip_model_load=True)
model.save("result.mlmodel")
It could be a problem with the PyTorch version and saving mechanism. I had the same problem and solved it by passing the kwarg _use_new_zipfile_serialization=False when saving the model. More details here.
I had this issue because a failed git LFS smudge corrupted the checkpoint. Check the file size/checksum of your ckpt/pth file.
After upgrading my Fedora 24 to 25, I am having issue with running a python script which was running just fine under Fedora 24. No matter what I choose from that default list for backend in matplotlibrc file, I am not able to produce plots. In particular, when I choose Qt5Agg in that list for the backend, I am receiving this weird error message and it is really bothering that I cannot find any thing related to that on internet just by searching. But I am also aware that something in the upgrade could have gone wrong affecting my python and/or Qt packages. I just need to know why connectivity has to do with the choice of backend (if any at all) and why none of the default choices can get rid of any sort of error message? But to be specific, why choosing Qt5Agg as the default backend of matplotlibrc file is giving such an error message related to the function connect()? Please let me know if posting the script would help you with the answer. Here is the imports in the beginning of that script:
import numpy as np
from numpy import nan
import pandas as pd
import matplotlib as mpl
#import matplotlib
#matplotlib.use('Qt5Agg')
import matplotlib.pyplot as plt
import pylab as pl
from uncertainties import ufloat
from uncertainties.umath import *
from matplotlib.ticker import MaxNLocator
from collections import OrderedDict
import astropy.units as u
from astropy.cosmology import FlatLambdaCDM, z_at_value
from numpy import sqrt, mean, square, std, maximum, minimum
from sklearn.metrics import mean_squared_error
from scipy.stats import poisson, chi2
import math
import sys
And the error message:
QObject::connect: Cannot connect NavigationToolbar2QT::message(QString) to (null)::_show_message()
Traceback (most recent call last):
File "myscript.py", line 496, in <module>
f, ((ax1, ax6, ax11), (ax2, ax7, ax12), (ax3, ax8, ax13), (ax4, ax9, ax14), (ax5, ax10, ax15)) = plt.subplots(5, 3, sharex=True, sharey=False , figsize=(20,9))
File "/usr/lib/python3.5/site-packages/matplotlib/pyplot.py", line 1177, in subplots
fig = figure(**fig_kw)
File "/usr/lib/python3.5/site-packages/matplotlib/pyplot.py", line 527, in figure
**kwargs)
File "/usr/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 43, in new_figure_manager
return new_figure_manager_given_figure(num, thisFig)
File "/usr/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 51, in new_figure_manager_given_figure
return FigureManagerQT(canvas, num)
File "/usr/lib/python3.5/site-packages/matplotlib/backends/backend_qt5.py", line 465, in __init__
self.toolbar.message.connect(self._show_message)
TypeError: connect() failed between NavigationToolbar2QT.message[str] and _show_message()
It's a bug in that backend, exposed by more strict checking in PyQt 5.7.
It was fixed in July - I suggest you open a Fedora bug so they upgrade those packages or backport the fix.
As for why this happens: This is not related to connectivity as in network, but connecting of Qt's signals and slots.
It took me two days to install the requirements of deepQ(python version),then I tried to run it today but I faced this problem, and the code are as followed.
root#unicorn:/media/trump/Data1/wei/college/laboratory/deep_q_rl-master/deep_q_rl# python run_nips.py
A.L.E: Arcade Learning Environment (version 0.5.0)
[Powered by Stella]
Use -help for help screen.
Warning: couldn't load settings file: ./ale.cfg
Game console created:
ROM file: ../roms/breakout.bin
Cart Name: Breakout - Breakaway IV (1978) (Atari)
Cart MD5: f34f08e5eb96e500e851a80be3277a56
Display Format: AUTO-DETECT ==> NTSC
ROM Size: 2048
Bankswitch Type: AUTO-DETECT ==> 2K
Running ROM file...
Random seed is 65
Traceback (most recent call last):
File "run_nips.py", line 60, in <module>
launcher.launch(sys.argv[1:], Defaults, __doc__)
File "/media/trump/Data1/wei/college/laboratory/deep_q_rl-master/deep_q_rl/launcher.py", line 223, in launch
rng)
File "/media/trump/Data1/wei/college/laboratory/deep_q_rl-master/deep_q_rl/q_network.py", line 53, in __init__
num_actions, num_frames, batch_size)
File "/media/trump/Data1/wei/college/laboratory/deep_q_rl-master/deep_q_rl/q_network.py", line 168, in build_network
batch_size)
File "/media/trump/Data1/wei/college/laboratory/deep_q_rl-master/deep_q_rl/q_network.py", line 407, in build_nips_network_dnn
from lasagne.layers import dnn
File "/usr/local/lib/python2.7/dist-packages/Lasagne-0.2.dev1-py2.7.egg/lasagne/layers/dnn.py", line 13, in <module>
raise ImportError("dnn not available") # pragma: no cover
ImportError: dnn not available
I have already tested theano ,numpy, scipy and there was no errors coming out. But when I ran it, it said dnn not available. So I came to find dnn, and the code is like this
import theano
from theano.sandbox.cuda import dnn
from .. import init
from .. import nonlinearities
from .base import Layer
from .conv import conv_output_length
from .pool import pool_output_length
from ..utils import as_tuple
if not theano.config.device.startswith("gpu") or not dnn.dnn_available():
raise ImportError("dnn not available") # pragma: no cover
Just hope someone can help me.
Did you install CUDA and cuDNN?
Lasagne is build on top of Theano and is, in some cases, relying on cuda code (e.g. here), rather than abstracting it away.
This can be seen from the import:
from theano.sandbox.cuda import dnn
Also see: https://github.com/Lasagne/Lasagne/issues/242
To get cuDNN you need to register at NVidia as a developer, see:
https://developer.nvidia.com/accelerated-computing
Hope this helps.
Cheers,
Michael