python tests using robot framework - python-3.x

Does anyone have a good example of testing a python program using Robot Framework.
I am trying to run my python program (chaptermarkers.py) with an argument of --test and checking the results.
Passing the argument has been an odyssey of google searches for 2 days.
[file: common_resourses.robot]
*** Settings ***
Library OperatingSystem
*** Variables ***
${CHAPTERMARKERS_EXEC} chaptermarkers
${LOG LEVEL} DEBUG
*** Keywords ***
# TODO
[Test Cases: file: default_suite.robot]
*** Settings ***
Documentation *Test Chapter Markers runs but has error in filename*
Metadata Github https://github.com/cbitterfield/chaptermarkers
Metadata Version 1.0.0
Metadata Executed At ${HOST}
# External libraries imports
Library Process
Library String
Resource common_resources.robot
*** Variables ***
${EXPECTED_MESSAGE} Movie Filename
${REPORT FILE} report.html
${LOG FILE} logfile.html
${LOG LEVEL} DEBUG
${OUTPUT DIR} /Users/colin/IdeaProjects/chaptermarkers
${test} --test
*** Test Cases ***
Scenerio test chaptermarkers run
[Tags] DEBUG
[Documentation] Verifies that chaptermarkers is executed well and without errors
${result}= Run process ${CHAPTERMARKERS_EXEC} ${test}
Should Contain ${result.stdout} ${EXPECTED_MESSAGE}
Should Be Empty ${result.stderr}
No matter how I try to put something after the program, I get an error of directory missing.
[Crazy unusable error messages]
$ robot --loglevel DEBUG --log log.html --report report.html stests/default_suite.robot
[ ERROR ] Error in file '/Users/colin/IdeaProjects/chaptermarkers/stests/default_suite.robot' on line 25: Invalid variable name '${test} --test'.
==============================================================================
Default Suite :: *Test Chapter Markers runs but has error in filename*
==============================================================================
Scenerio test chaptermarkers run :: Verifies that chaptermarkers i... | FAIL |
Variable '${test}' not found. Did you mean:
${TEST_TAGS}
${TEST_NAME}
------------------------------------------------------------------------------
Default Suite :: *Test Chapter Markers runs but has error in filen... | PASS |
0 critical tests, 0 passed, 0 failed
1 test total, 0 passed, 1 failed
==============================================================================
Output: /Users/colin/IdeaProjects/chaptermarkers/output.xml
[ ERROR ] Unexpected error: FileNotFoundError: [Errno 2] No such file or directory: '/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/htmldata/rebot/log.html'
Traceback (most recent call last):
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/utils/application.py", line 83, in _execute
rc = self.main(arguments, **options)
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/run.py", line 451, in main
writer.write_results(settings.get_rebot_settings())
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/reporting/resultwriter.py", line 65, in write_results
self._write_log(results.js_result, settings.log, config)
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/reporting/resultwriter.py", line 79, in _write_log
self._write('Log', LogWriter(js_result).write, path, config)
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/reporting/resultwriter.py", line 86, in _write
writer(path, *args)
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/reporting/logreportwriters.py", line 43, in write
self._write_file(path, config, LOG)
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/reporting/logreportwriters.py", line 36, in _write_file
writer.write(template)
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/htmldata/htmlfilewriter.py", line 33, in write
for line in HtmlTemplate(template):
File "/Users/colin/IdeaProjects/chaptermarkers/env/lib/python3.8/site-packages/robot/htmldata/normaltemplate.py", line 28, in __iter__
with codecs.open(self._path, encoding='UTF-8') as file:
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/codecs.py", line 905, in open
file = builtins.open(filename, mode, buffering)
(env) razzamataz:chaptermarkers colin$

A few things I noticed:
There needs to be at least two spaces between ${test} and --test in your variables section. You provided only one. The same in the Run Process line, there should be at least two spaces between exec and argument. The rule is that RF uses two spaces as delimiter.
Your test name is indented. RF inteprets it as a keyword calling which is unexpected. It should start from the begining of the line without indent
The file missing error looks weired and I don't know the reason. The file is part of RF library and if you installed RF correctly, it should be there. I suggest that you start a new virtual environment and try again if it's not too complex.
P.S. Just FYI, all RF buil-in libraries can be found here:
https://robotframework.org/#libraries.

Related

Trying to incorporate ML onnx model to Android App

I have a trained onnx model that I want to incorporate into an android app.
I'm actually working on a uni project, combining ML & Android development.
After a long research, since I don't want to use a python private REST API, I came to the conclusion that there are two ways I could continue from here: I can either try converting my onnx model into a TF model and then generate a TFLite model through TFLite Converter API, or give it a try with onnxruntime.
I tried the first way with TFLite, using the answer from this post, and hence this code:
import onnx
from onnx_tf.backend import prepare
onnx_model = onnx.load("input_path") # load onnx model
tf_rep = prepare(onnx_model) # prepare tf representation
tf_rep.export_graph("output_path") # export the model
but I’m stuck in that first conversion from .onnx to .pb, since I think onnx-tf doesn’t support dynamic dimensions (that my model has). I’m constantly getting an
"Input size (depth of inputs) must be accessible via shape inference," or
RuntimeError: Node name is not unique in your model. Please recreate your model with unique node name. and similar errors.
I also gave it a try with onnxruntime, but I can't seem to manage to "Create a minimal build for Android with NNAPI support". I'm getting this error while building:
[1/67] Building CXX object CMakeFiles/libprotobuf.dir/C_/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc.obj
FAILED: CMakeFiles/libprotobuf.dir/C_/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc.obj
C:\PROGRA~2\CODEBL~1\MinGW\bin\C__~1.EXE -DGOOGLE_PROTOBUF_CMAKE_BUILD -DHAVE_PTHREAD -I. -IC:/Users/chris/onnxruntime/cmake/external/protobuf/src -std=c++11 -MD -MT CMakeFiles/libprotobuf.dir/C_/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc.obj -MF CMakeFiles\libprotobuf.dir\C_\Users\chris\onnxruntime\cmake\external\protobuf\src\google\protobuf\io\io_win32.cc.obj.d -o CMakeFiles/libprotobuf.dir/C_/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc.obj -c C:/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc
C:/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc: In function 'int google::protobuf::io::win32::stat(const char*, google::protobuf::io::win32::_stat*)':
C:/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc:315:40: error: cannot convert 'google::protobuf::io::win32::_stat*' to '_stat*' for argument '2' to 'int _wstat(const wchar_t*, _stat*)'
return ::_wstat(wpath.c_str(), buffer);
^
In file included from C:/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc:52:0:
C:/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.h:75:51: note: class type 'google::protobuf::io::win32::_stat' is incomplete
PROTOBUF_EXPORT int stat(const char* path, struct _stat* buffer);
^
C:/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc: In function 'FILE* google::protobuf::io::win32::fopen(const char*, const char*)':
C:/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/io/io_win32.cc:337:10: error: '::_wfopen' has not been declared
return ::_wfopen(wpath.c_str(), wmode.c_str());
^
[6/67] Building CXX object CMakeFiles/libprotobuf.dir/C_/Users/chris/onnxruntime/cmake/external/protobuf/src/google/protobuf/message_lite.cc.obj
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
File "C:\Users\chris\onnxruntime\\tools\ci_build\build.py", line 2023, in <module>
sys.exit(main())
File "C:\Users\chris\onnxruntime\\tools\ci_build\build.py", line 1918, in main
cmake_path, source_dir, build_dir, args)
File "C:\Users\chris\onnxruntime\\tools\ci_build\build.py", line 1673, in build_protoc_for_host
run_subprocess(cmd_args)
File "C:\Users\chris\onnxruntime\\tools\ci_build\build.py", line 544, in run_subprocess
return run(*args, cwd=cwd, capture_stdout=capture_stdout, shell=shell, env=my_env)
File "C:\Users\chris\onnxruntime\tools\python\util\run.py", line 44, in run
env=env, shell=shell)
File "C:\Users\chris\AppData\Local\Programs\Python\Python37\lib\subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['C:\\Program Files\\CMake\\bin\\cmake.EXE', '--build', 'C:\\Users\\chris\\onnxruntime\\\\build\\Windows\\host_protoc', '--config', 'Release', '--target', 'protoc']' returned non-zero exit status 1.
Am I in a completely wrong path? It's my first time trying to combine ML with Android so I have no experience in this. Any advice would be very welcome.
For your issue with onnxruntime, try changing the CMakeCache.txt :
CMAKE_CXX_FLAGS:STRING=-U__STRICT_ANSI__
It should fix the error you mentioned. I think you're in the right path with onnxruntime.

For the IPython run magic the option to ignore SystemExist doesn't seem to work with modules

The docs for the %run magic -e option state:
ignore sys.exit() calls or SystemExit exceptions in the script being
run. This is particularly useful if IPython is being used to run
unittests, which always exit with a sys.exit() call. In such cases you
are interested in the output of the test results, not in seeing a
traceback of the unittest module.
This works when running scripts but doesn't seem to work when running modules.
So when I type %run -e -m pytest I still get a traceback when a test fails due the SystemExit thrown by pytest which is case mentioned in the docs above that -e is meant to address. I know I can type !pytest but I don't want to wait until pytest completed before I start to see results, and I also want to add the current directory to to module search path.
I am running IPython within Spyder but the behaviour is the same if I run IPython from the Windows command prompt. I there any way of doing what I want and avoiding the distracting traceback?
I ran the following test with %run -m pytest from the spyder ipython console
import pytest
def test_fail():
assert 0
The output was:
============================= test session starts =============================
platform win32 -- Python 3.7.7, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: D:\home\shane\temp\pytest
collected 1 item
test_dummy.py F [100%]
================================== FAILURES ===================================
__________________________________ test_fail __________________________________
def test_fail():
> assert 0
E AssertionError
test_dummy.py:21: AssertionError
=========================== short test summary info ===========================
FAILED test_dummy.py::test_fail - AssertionError
============================== 1 failed in 0.03s ==============================
Traceback (most recent call last):
File "c:\opt\python37\lib\runpy.py", line 205, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "c:\opt\python37\lib\runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "c:\opt\python37\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "c:\opt\python37\lib\site-packages\pytest\__main__.py", line 7, in <module>
raise SystemExit(pytest.main())
SystemExit: ExitCode.TESTS_FAILED

Opening Firefox via Selenium in Jenkins

When trying to run a Python Selenium script via Jenkins, it crashes, with a very unhelpful exception - "selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status: 0"
The same Python script runs perfectly fine from outside Python.
I've reduced the script to a very, very basic one, simply launching Firefox -
from selenium import webdriver
if __name__ == "__main__":
print("Hello World")
driver = webdriver.Firefox()
driver.get("http://www.google.com")
driver.maximize_window()
driver.quit()
print("Goodbye World")
Output from cmd prompt:
>C:\Users\kipod>C:\python_projects\Ranger\_just_open_browser.py
>Hello World
>Goodbye World
Works fine.
Running as a Jenkins job:
>##Started by user anonymous
>Building in workspace C:\Program Files (x86)\Jenkins\workspace\Second Test
>[Second Test] $ cmd /c call >C:\Users\kipod\AppData\Local\Temp\jenkins3557827225974274191.bat
>
>C:\Program Files (x86)\Jenkins\workspace\Second Test>python >C:\python_projects\Ranger\_just_open_browser.py
>Hello World
>Traceback (most recent call last):
> File "C:\python_projects\Ranger\_just_open_browser.py", line 8, in <module>
> driver = webdriver.Firefox()
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\firefox\webdriver.py", line 154, in __init__
> keep_alive=True)
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\remote\webdriver.py", line 151, in __init__
> self.start_session(desired_capabilities, browser_profile)
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\remote\webdriver.py", line 240, in start_session
> response = self.execute(Command.NEW_SESSION, parameters)
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\remote\webdriver.py", line 308, in execute
> self.error_handler.check_response(response)
> File "C:\Python\Python36-32\lib\site->packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
> **raise exception_class(message, screen, stacktrace)
>selenium.common.exceptions.WebDriverException: Message: Process unexpectedly >closed with status: 0**
>C:\Program Files (x86)\Jenkins\workspace\Second Test>exit 1
>Build step 'Execute Windows batch command' marked build as failure
>Finished: FAILURE
I suspect something is off with Jenkins's environment variables, but I have no idea what to check. And the exception is so unspecific ("I crashed. lols"), That I found nothing helpful on Google. And I've been searching for a while.
Versions:
Jenkins: 2.73.1
Python: 3.6.2
Geckodriver: 0.19.0
Selenium: 3.6.0
(Tried opening chrome instead of firefox - same behaviour.)
I think you need to run your python script using a jenkins slave configured to log on as a standard windows user account (for example "kipod") ? Make sure that auto login is set for the slave machine for the user account, in order to allow selenium to launch Firefox in a desktop.
Some documenation here: Step by step guide to set up master and slave machines on Windows

TensorFlow. When running fully_connected_feed.py, app.py occus a 'systemexit' error

I was just learning 'TensorFlow Mechanics 101'. Error occurred when I run fully_connected_feed.py for trainning the MNIST data:
Step 1800: loss = 0.30 (0.002 sec)
Step 1900: loss = 0.44 (0.003 sec)
Training Data Eval:
Num examples: 55000 Num correct: 49180 Precision # 1: 0.8942
Validation Data Eval:
Num examples: 5000 Num correct: 4509 Precision # 1: 0.9018
Test Data Eval:
Num examples: 10000 Num correct: 9023 Precision # 1: 0.9023
An exception has occurred, use %tb to see the full traceback.
SystemExit
D:\software\anaconda\envs\tensorflow\lib\site-packages\IPython\core\interactiveshell.py:2870:
UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)
After I typed '%tb', it shows:
%tb
Traceback (most recent call last):
File "<ipython-input-1-984b11309266>", line 1, in <module>
runfile('D:/wangjc/pythonTest/TensorFlow/testTensorFlow.py', wdir='D:/wangjc/pythonTest/TensorFlow')
File "D:\software\anaconda\envs\tensorflow\lib\site-packages\spyder\utils\site\sitecustomize.py", line 707, in runfile
execfile(filename, namespace)
File "D:\software\anaconda\envs\tensorflow\lib\site-packages\spyder\utils\site\sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "D:/wangjc/pythonTest/TensorFlow/testTensorFlow.py", line 277, in <module>
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "D:\software\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\platform\app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
SystemExit
I have found a question like me, but actually it is not the same problem as me. This error message shows different from me:
TypeError: run() got an unexpected keyword argument 'argv'
Also, I installed TensorFlow 1.1.0 by'pip install'.
And, I tried to use several kinds of 'fully_connected_feed.py' code in different TensorFlow version, but other err ( if using lower version ) or the same err occurs.
Some people says the 'SystemExit' error occurs because there is a CMD thread problem. But I don't know whether it is the root and where it is.
Please help me to solve this problem. Thanks!
My ide environment is:
windows 10
Anaconda Python 3.5
TensorFlow 1.1.0
TensorFlow was installed by this code:
(tensorflow) wangjc#wangjc-Inspiron-3668:~$ pip install --ignore-installed --upgrade https://xxxxxxxx.whl
The version shows below:
import tensorflow as tf
tf.VERSION
Out[4]: '1.1.0'
From your traceback:
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
the call to main is wrapped inside a _sys.exit() call that kills the quits the program once it has finished. The message you get comes from running the file inside an iPython iteractive shell. sys.exit() raises a SystemExit exception that normally is used to quit python. iPython's shell, however, captures that exception and shows it to you with a warning. This, however, does not affect your program. Just ignore the message or remove the _sys.exit() wrapping the call to main() in your script.

Invalid gopkg.in package and rev in

When attempting to build with Pants, I am seeing the following error:
File "build/bdist.macosx-10.10-intel/egg/pants/contrib/go/tasks/go_fetch.py", line 154, in _transitive_download_remote_libs
all_known_addresses)
File "build/bdist.macosx-10.10-intel/egg/pants/contrib/go/tasks/go_fetch.py", line 105, in _transitive_download_remote_libs
fetcher.fetch(go_remote_lib.import_path, dest=tmp_fetch_root, rev=go_remote_lib.rev)
File "build/bdist.macosx-10.10-intel/egg/pants/contrib/go/subsystems/fetchers.py", line 437, in fetch
github_root, github_rev = self._map_import_path(import_path, rev)
File "/Users/chad/.cache/pants/setup/bootstrap/pants.mbFDa8/install/lib/python2.7/site-packages/pants/util/memo.py", line 95, in memoize
result = func(*args, **kwargs)
File "build/bdist.macosx-10.10-intel/egg/pants/contrib/go/subsystems/fetchers.py", line 454, in _map_import_path
raise self.FetchError('Invalid gopkg.in package and rev in: {}'.format(import_path))
Exception message: Invalid gopkg.in package and rev in: gopkg.in/amz.v1/aws
Here is the contents of my BUILD file:
# Auto-generated by pants!
# To re-generate run: `pants buildgen.go --materialize --remote`
go_remote_library(rev='v1')
Looking into the code, I see that the error comes from a failure to match a regex in fetchers.py, on line 453.
I am running Pants version 0.0.59 on Mac OS X 10.10 (Yosemite)
Noting that #Huckphin stumbled on a bug here in pantsbuild.pants<=0.0.59. He filed an issue and now things are fixed up for handling gopkg.in remote import paths that point to sub-packages in the remote repo. The fix will be released with the regular Friday release on 11/20/2015 in 0.0.60.

Resources