Multiple bases have instance lay-out conflict in Robot Framework - python-3.x

I'm creating a new testing frameworks, I started to implement my own functions in *.py files, but when I try to run test, I've got following stack:
(venv) PLAMWS0024:OAT user$ robot -v CONFIG_FILE:"/variables-config.robot" ./catalog/tests/test1.robot
Traceback (most recent call last):
File "/Users/user/PycharmProjects/OAT/venv/bin/robot", line 5, in <module>
from robot.run import run_cli
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/__init__.py", line 44, in <module>
from robot.rebot import rebot, rebot_cli
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/rebot.py", line 45, in <module>
from robot.run import RobotFramework
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/run.py", line 44, in <module>
from robot.running.builder import TestSuiteBuilder
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/running/__init__.py", line 98, in <module>
from .builder import TestSuiteBuilder, ResourceFileBuilder
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/running/builder/__init__.py", line 16, in <module>
from .builders import TestSuiteBuilder, ResourceFileBuilder
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/running/builder/builders.py", line 20, in <module>
from robot.parsing import SuiteStructureBuilder, SuiteStructureVisitor
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/parsing/__init__.py", line 380, in <module>
from .model import ModelTransformer, ModelVisitor
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/parsing/model/__init__.py", line 18, in <module>
from .statements import Statement
File "/Users/user/PycharmProjects/OAT/venv/lib/python3.8/site-packages/robot/parsing/model/statements.py", line 453, in <module>
class Error(Statement, Exception):
TypeError: multiple bases have instance lay-out conflict
I suspect it's because in one of my files I'm trying to get variables from Robot Framework built in functionalities.
and I'm thinking it's because I'm trying to use protected methods, but I am not sure.
I found issue TypeError: multiple bases have instance lay-out conflict and it shows that there might be a mismatch in naming convention (or am I wrong?), but my project is a bit small, so the only option is that Robot can't see the function itself.
What can I miss?
Some code:
Test itself:
*** Settings ***
Documentation TO BE CHANGED
... SET IT TO CORRECT DESCRIPTION
Library ${EXECDIR}/file.py
Library String
*** Test Cases ***
User can do stuff
foo bar
from datetime import datetime
from robot.api import logger
from robot.libraries.BuiltIn import _Variables
from robot.parsing.model.statements import Error
import json
import datetime
from catalog.resources.utils.clipboardContext import get_value_from_clipboard
Vars = _Variables()
def foo_bar(params):
# Get all variables
country = get_value_from_clipboard('${COUNTRY}')
address = get_value_from_clipboard('${ADDRESS}')
city = get_value_from_clipboard('${CITY}')
postcode = get_value_from_clipboard('${POSTALCODE}')
And calling Vars:
from robot.libraries.BuiltIn import _Variables
from robot.parsing.model.statements import Error
Vars = _Variables()
def get_value_from_clipboard(name):
"""
Returns value saved inside variables passed in Robot Framework
:param name: name of the variable, needs to have ${} part
as example: ${var} passed in config file
:return: value itself, passed as string
"""
try:
return Vars.get_variable_value(name)
except Error as e:
raise Error('Missing parameter in the clipboard, stack:' + str(e))

What fixed issue:
uninstall all requirements from requirements.txt file and install all one-by-one.
Additional steps I tried:
comment out all files one-by-one and run only robot command - failed, got same errors
cleaned vnenv as described here: How to reset virtualenv and pip? (failed)
check out if any variable has same naming as described in python3.8/site-packages/robot/parsing/model/statements.py - none
So looks like there was some clash in installing requirements by PyCharm IDE

Related

Calling a Python File from another file and viceversa

I have few common functions written in file a.py which is called from file b.py. And, also I have few common functions written in file b.py which is called from file a.py. but when I try to run these files I get error as unable to import name. Below is the code for reference.
I have just provided an example below, these files have common piece of code in real scenario which is specific to a functionality and that is the reason its being called from one another
a.py code below
from b import get_partner_id
def get_customer_id(tenant_id):
customer_id = tenant_id + "tenant_name"
return customer_id
def get_partner_details():
partnerid = get_partner_id()
data = {"partnerId": partnerid}
print(data)
b.py code below
from a import get_customer_id
def get_partner_id():
customer_id = get_customer_id("7687")
env = 'sandbox-all' + customer_id
return env
get_partner_id()
Below is the error for reference,
Traceback (most recent call last):
File "C:/testCases/b.py", line 1, in <module>
from a import get_customer_id
File "C:\testCases\a.py", line 2, in <module>
from b import get_partner_id
File "C:\testCases\b.py", line 1, in <module>
from a import get_customer_id
ImportError: cannot import name 'get_customer_id'

Unable to access python packages installed in Azure ML

I am trying to deploy a pre-trained ML model (saved as .h5 file) to Azure ML. I have created an AKS cluster and trying to deploy the model as shown below:
from azureml.core import Workspace
from azureml.core.model import Model
from azureml.core.environment import Environment
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core.model import InferenceConfig
from azureml.core.webservice import AksWebservice, LocalWebservice
from azureml.core.compute import ComputeTarget
workspace = Workspace.from_config(path="config.json")
env = Environment.get(workspace, name='AzureML-TensorFlow-1.13-GPU')
# Installing packages present in my requirements file
with open('requirements.txt') as f:
dependencies = f.readlines()
dependencies = [x.strip() for x in dependencies if '# ' not in x]
dependencies.append("azureml-defaults>=1.0.45")
env.python.conda_dependencies = CondaDependencies.create(conda_packages=dependencies)
# Including the source folder so that all helper scripts are included in my deployment
inference_config = InferenceConfig(entry_script='app.py', environment=env, source_directory='./ProcessImage')
aks_target = ComputeTarget(workspace=workspace, name='sketch-ppt-vm')
# Deployment with suitable config
deployment_config = AksWebservice.deploy_configuration(cpu_cores=4, memory_gb=32)
model = Model(workspace, 'sketch-inference')
service = Model.deploy(workspace, "process-sketch-dev", [model], inference_config, deployment_config, deployment_target=aks_target, overwrite=True)
service.wait_for_deployment(show_output = True)
print(service.state)
My main entry script requires some additional helper scripts, which I include by mentioning the source folder in my inference config.
I was expecting that the helper scripts I add should be able to access the packages installed while setting up the environment during deployment, but I get ModuleNotFoundError.
Here is the error output, along with the a couple of environment variables I printed while executing entry script:
AZUREML_MODEL_DIR ---- azureml-models/sketch-inference/1
PYTHONPATH ---- /azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages:/var/azureml-server:
PATH ---- /azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/bin:/opt/miniconda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/intel/compilers_and_libraries/linux/mpi/bin64
Exception in worker process
Traceback (most recent call last):
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/azureml-envs/azureml_6dc005c11e151f8d9427c0c6091a1bb9/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
__import__(module)
File "/var/azureml-server/wsgi.py", line 1, in <module>
import create_app
File "/var/azureml-server/create_app.py", line 3, in <module>
from app import main
File "/var/azureml-server/app.py", line 32, in <module>
from aml_blueprint import AMLBlueprint
File "/var/azureml-server/aml_blueprint.py", line 25, in <module>
import main
File "/var/azureml-app/main.py", line 12, in <module>
driver_module_spec.loader.exec_module(driver_module)
File "/structure/azureml-app/ProcessImage/app.py", line 16, in <module>
from ProcessImage.samples.coco.inference import run as infer
File "/var/azureml-app/ProcessImage/samples/coco/inference.py", line 1, in <module>
import skimage.io
ModuleNotFoundError: No module named 'skimage'
The existing answers related to this aren't of much help. I believe there must be a simpler way to fix this, since AzureML specifically provides the feature to setup environment with pip/conda packages installed either by supplying requirements.txt file or individually.
What am I missing here? Kindly help.
So, after some trial and error, creating a fresh environment and then adding the packages solved the problem for me. I am still not clear on why this didn't work when I tried to use Environment.from_pip_requirements(). A detailed answer in this regard would be interesting to read.
My primary task was inference - object detection given an image, and we have our own model developed by our team. There are two types of imports I wanted to have:
1. Standard python packages (installed through pip)
This was solved by creating conda dependencies and add it to env object (Step 2)
2. Methods/vars from helper scripts (if you have pre/post processing to be done during model inference):
This was done by mentioning source_directory in InferenceConfig (step 3)
Here is my updated script which combines Environment creation, Inference and Deployment configs and using existing compute in the workspace (created through portal).
from azureml.core import Workspace
from azureml.core.model import Model
from azureml.core.environment import Environment, DEFAULT_GPU_IMAGE
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core.model import InferenceConfig
from azureml.core.webservice import AksWebservice, LocalWebservice
from azureml.core.compute import ComputeTarget
# 1. Instantiate the workspace
workspace = Workspace.from_config(path="config.json")
# 2. Setup the environment
env = Environment('sketchenv')
with open('requirements.txt') as f: # Fetch all dependencies as a list
dependencies = f.readlines()
dependencies = [x.strip() for x in dependencies if '# ' not in x]
env.docker.base_image = DEFAULT_GPU_IMAGE
env.python.conda_dependencies = CondaDependencies.create(conda_packages=['numpy==1.17.4', 'Cython'], pip_packages=dependencies)
# 3. Inference Config
inference_config = InferenceConfig(entry_script='app.py', environment=env, source_directory='./ProcessImage')
# 4. Compute target (using existing cluster from the workspacke)
aks_target = ComputeTarget(workspace=workspace, name='sketch-ppt-vm')
# 5. Deployment config
deployment_config = AksWebservice.deploy_configuration(cpu_cores=6, memory_gb=100)
# 6. Model deployment
model = Model(workspace, 'sketch-inference') # Registered model (which contains model files/folders)
service = Model.deploy(workspace, "process-sketch-dev", [model], inference_config, deployment_config, deployment_target=aks_target, overwrite=True)
service.wait_for_deployment(show_output = True)
print(service.state)

ImportError: cannot import name '_ni_support' -- Using cx-freeze with scipy

I have used cx-freeze to make an executable file from my python 3 script. The issue is that apparently cx-freeze is having a hard time importing scipy scripts. I had to resolve other issues previously, e.g. adding the tcl library files manually. Anyway, my setup files is below:
from cx_Freeze import setup, Executable
import os
os.environ['TCL_LIBRARY'] = "C:\\Users\\Gobryas\\AppData\\Local\\Continuum\\anaconda3\\tcl\\tcl8.6"
os.environ['TK_LIBRARY'] = "C:\\Users\\Gobryas\\AppData\\Local\\Continuum\\anaconda3\\tcl\\tk8.6"
additional_mods = ['numpy.core._methods', 'numpy.lib.format']
setup(name = "Curve Characterization" ,
options = {'build_exe': {'includes': additional_mods}},
version = "0.1" ,
description = "" ,
executables = [Executable("curvCharLite.py")])
This is the error that I get:
ImportError: cannot import name '_ni_support'
Full details
Traceback (most recent call last):
File "C:\Users\Gobryas\AppData\Local\Continuum\anaconda3\lib\site-packages\cx_Freeze\initscripts\__startup__.py", line 14, in run
module.run()
File "C:\Users\Gobryas\AppData\Local\Continuum\anaconda3\lib\site-packages\cx_Freeze\initscripts\Console.py", line 26, in run
exec(code, m.__dict__)
File "curvCharLite.py", line 21, in <module>
File "c:\users\Gobryas\documents\mo\project 1\ruptures\ruptures\__init__.py", line 8, in <module>
from .detection import (Binseg, BottomUp, Dynp, Omp, OmpK, Pelt, Window,
File "c:\users\Gobryas\documents\mo\project 1\ruptures\ruptures\detection\__init__.py", line 51, in <module>
from .window import Window
File "c:\users\Gobryas\documents\mo\project 1\ruptures\ruptures\detection\window.py", line 112, in <module>
from scipy.signal import argrelmax
File "C:\Users\Gobryas\AppData\Local\Continuum\anaconda3\lib\site-packages\scipy\signal\__init__.py", line 311, in <module>
from ._savitzky_golay import savgol_coeffs, savgol_filter
File "C:\Users\Gobryas\AppData\Local\Continuum\anaconda3\lib\site-packages\scipy\signal\_savitzky_golay.py", line 6, in <module>
from scipy.ndimage import convolve1d
File "C:\Users\Gobryas\AppData\Local\Continuum\anaconda3\lib\site-packages\scipy\ndimage\__init__.py", line 161, in <module>
from .filters import *
File "C:\Users\Gobryas\AppData\Local\Continuum\anaconda3\lib\site-packages\scipy\ndimage\filters.py", line 35, in <module>
from . import _ni_support
ImportError: cannot import name '_ni_support'
PS 1: There is a similar issue asked here, but no helpful answer really.
PS 2: I'm using scipy 1.1.0, cx-freeze 5.1.1, and python 3.6.5
So, I think I found a solution for this problem inspired by the post by #fepzzz.
Looks like there are bad conflicts between cx-freeze and scipy which can be avoided as follows. I needed to modify the include_files build option as below:
import os
import scipy
includefiles_list=[]
scipy_path = os.path.dirname(scipy.__file__)
includefiles_list.append(scipy_path)
build_options = dict(packages=['matplotlib'], #this line solves an issue w/ matplotlib
include_files=includefiles_list, #this line is for scipy issue
includes=['matplotlib.backends.backend_qt5agg']) #this line solves another issue w/ matplotlib
Hope this helps others.

surprise.dataset is not a package

When I try to execute this code
import surprise.dataset
file = 'ml-100k/u.data'
col = 'user item rating timestamp'
reader = surprise.dataset.Reader(line_format = col, sep = '\t')
data = surprise.dataset.Dataset.load_from_file(file, reader = reader)
data.split(n_folds = 5)
I get this error:
Traceback (most recent call last):
File "/home/user/PycharmProjects/prueba2/surprise.py", line 1, in <module>
import surprise.dataset
File "/home/user/PycharmProjects/prueba2/surprise.py", line 1, in <module>
import surprise.dataset
ModuleNotFoundError: No module named 'surprise.dataset'; 'surprise' is not a package
But surprise is installed. How can I use it?
If you want to import a submodule, instead of doing:
import surprise.dataset
do:
from surprise import dataset
then in the rest of the code, reference it as dataset instead of surprise.dataset (i.e. replace surprise.dataset.Reader with dataset.Reader). If you don’t want to update the rest of your code, you can also just import the whole surprise module instead with:
import surprise

Cython with Numpy / AttributeError: 'module' object has no attribute 'Handler'

I want to use Numpy in Cython, but encountered the following error. This error happens even if I run the simple code, so it should be an issue related to importing Numpy.
My environment:
OS X Yosemite
Python 3.4.3
setup.py:
import numpy as np
from distutils.core import setup
from Cython.Build import cythonize
setup(
name = 'my_code',
ext_modules = cythonize('my_code.pyx'),
include_path = [numpy.get_include()]
)
my_code.pyx:
cimport numpy as np
cdef int a
Execute in Terminal:
$ python3 setup.py build_ext --inplace
Traceback (most recent call last):
File "/Users/***/setup.py", line 1, in <module>
import numpy as np
File "/usr/local/lib/python3.4/site-packages/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/usr/local/lib/python3.4/site-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/usr/local/lib/python3.4/site-packages/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/usr/local/lib/python3.4/site-packages/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/usr/local/lib/python3.4/site-packages/numpy/core/__init__.py", line 58, in <module>
from numpy.testing import Tester
File "/usr/local/lib/python3.4/site-packages/numpy/testing/__init__.py", line 10, in <module>
from unittest import TestCase
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/__init__.py", line 59, in <module>
from .case import (TestCase, FunctionTestCase, SkipTest, skip, skipIf,
File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 253, in <module>
class _CapturingHandler(logging.Handler):
AttributeError: 'module' object has no attribute 'Handler'
Old question but I ran into this because I had a very similar issue with numpy.
It turns out I had a directory named logging as a sibling of the script I was trying to run. So the problem was a simple naming collision between my local project folder and the logging module numpy expected to find. Renaming the project logging folder solved the issue for me.
Looking at OP's error message, I suspect this was the case for them as well.

Resources