I'm writing a ContentsManager for Jupyter-Notebook to store data on a local OpenStack SwiftStore. I'm taking S3Contents as my starting point.
I have a basic framework that will connect & create a container if it doesn't exist.... however before I go any further, I need to start creating unit tests.
My question is how do I see the DEBUG output?
If I have a method that starts:
def get(self, path, content=True, type=None, format=None):
"""Retrieve an object from the store, named in 'path'
"""
self.log.debug("SwiftContents[swiftmanager]: get '%s', path)
.... how do I see the output SwiftContents[swiftmanager]: get '\foo\bar'?
Also, what's the testing framework used by notebooks?
class SwiftContentsManager(ContentsManager):
# Initialise the instance
def __init__(self, *args, **kwargs):
super(SwiftContentsManager, self).__init__(*args, **kwargs)
self.swiftfs = SwiftFS(log=self.log)
It looks a bit like unittest, except it has setUp rather than startUp
To run tests: nosetests path/to/tests/test_foo.py
To run tests, and see what it's actually doing: nosetests -v path/to/tests/test_foo.py
To run tests, see what it's actually doing, and see the debug output: nosetests -v --debug path/to/tests/test_foo.py
For more information, see http://pythontesting.net/framework/nose/nose-introduction/
Related
I used customized pytest markers like below,
#pytest.mark.test_id("AB-1234")
def test_test1():
pass
#pytest.mark.test_id("AB-1234")
def test_test2():
pass
#pytest.mark.test_id("AB-5678")
def test_test3():
pass
Here test_id is the marker name and "AB-1234" is the value.
From command line how to run all the tests with marker test_id==AB-1234? I tried different variations with -m option but couldn't find a solution.
EDIT: I tried pytest -m "AB-1234" but it runs all the tests.
So I tried many things (from SO and more) getting my tests running but nothing worked this is my current code:
test.py which I call to run the tests: python3 ./src/preprocess/python/test.py
import unittest
if __name__ == '__main__':
testsuite = unittest.TestLoader().discover('.')
unittest.TextTestRunner(verbosity=2).run(testsuite)
the test file looks like this:
import unittest
from scrapes.pdf import full_path_to_destination_txt_file
print(full_path_to_destination_txt_file)
class PreprocessingTest(unittest.TestCase):
def path_txt_appending(self):
self.assertEqual(full_path_to_destination_txt_file(
"test", "/usr/test"), "/usr/test/test.txt")
if __name__ == '__main__':
unittest.main(verbosity=2)
But the output is always like this:
python3 ./src/preprocess/python/test.py
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Additional Information:
As you can see I call this not from my root directory. The test folder is in ./src/preprocess/python/test/ and has a __init__.pyfile included (there is also a init file on the level of test.py)
it would be okay for me if I have to code down all the calls for all the tests I just want to finish this
automatic search with -t does not work either so I thought the more robust method here with test.py would work...
using this framework is a requirement I have to follow
test_preprocessing.py is in the test folder and from scrapes.pdf import full_path_to_destination_txt_filescrapes is a module folder on the same level as test
When I call the single unit test directly in the command line it fails because of the relative import. But using the test.py (obviously) finds the modules.
What is wrong?
By default, unittest will only execute methods whose name starts with test:
testMethodPrefix
String giving the prefix of method names which will be interpreted as test methods. The default value is 'test'.
This affects getTestCaseNames() and all the loadTestsFrom*() methods.
from the docs.
Either change that attribute or (preferably) prefix your method name with test_.
I have the following problem.
I'd like to run tests on the local flask server before deploying to production. I use pytest for that. My conftest.py looks like that for the moment:
import pytest
from toolbox import Toolbox
import subprocess
def pytest_addoption(parser):
"""Add option to pass --testenv=local to pytest cli command"""
parser.addoption(
"--testenv", action="store", default="exodemo", help="my option: type1 or type2"
)
#pytest.fixture(scope="module")
def testenv(request):
return request.config.getoption("--testenv")
#pytest.fixture(scope="module")
def testurl(testenv):
if testenv == 'local':
return 'http://localhost:5000/'
else:
return 'https://api.domain.com/'
This allows me to test the production api by typing the command pytest and to test a local flask server by typing pytest --testenv=local
This code WORKS flawlessly.
My problem is that I have to manually instantiate the local flask server from the terminal each time I want to test locally like this:
source ../pypyenv/bin/activate
python ../app.py
Now I wanted to add a fixture that initiates a terminal in the background at the beginning of the tests and closes the server down after having finished testing. After a lot of research and testing, I still cannot get it to work. This is the line I added to the conftest.py:
#pytest.fixture(scope="module", autouse=True)
def spinup(testenv):
if testenv == 'local':
cmd = ['../pypyenv/bin/python', '../app.py']
p = subprocess.Popen(cmd, shell=True)
yield
p.terminate()
else:
pass
The errors I get are from the requests package that says that there is no connection/ refused.
E requests.exceptions.ConnectionError:
HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded
with url: /login (Caused by
NewConnectionError(': Failed to establish a new connection:
[Errno 111] Connection refused',))
/usr/lib/python3/dist-packages/requests/adapters.py:437:
ConnectionError
This means for me that the flask server under app.py is not online. Any suggestions? I am open to more elegant alternatives
For local testing the Flask test_client is a more elegant solution. See the docs on Testing. You can create a fixture for the test_client and create test requests with that:
#pytest.fixture
def app():
app = create_app()
yield app
# teardown here
#pytest.fixture
def client(app):
return app.test_client()
And use it like this:
def test_can_login(client):
response = client.post('/login', data={username='username', password='password'})
assert response.status_code == 200
If the only problem are the manual steps, maybe consider a bash script that does your manual setup for you and after that executes pytest.
I am using the following for this purpose so that testing configuration is also preserved in the test server
#pytest.fixture(scope="session")
def app():
db_fd, db_path = tempfile.mkstemp()
app = create_app({
'TESTING': True,
'DATABASE': db_path
})
yield app
os.close(db_fd)
os.unlink(db_path)
from flask import request
def shutdown_server():
func = request.environ.get('werkzeug.server.shutdown')
if func is None:
raise RuntimeError('Not running with the Werkzeug Server')
func()
#pytest.fixture
def server(app):
#app.route('/shutdown',methods=('POST',))
def shutdown():
shutdown_server()
return 'Shutting down server ...'
import threading
t = threading.Thread(target=app.run)
yield t.start()
import requests
requests.post('http://localhost:5000/shutdown')
References
https://flask.palletsprojects.com/en/1.1.x/tutorial/tests/
How do I terminate a flask app that's running as a service?
How to stop flask application without using ctrl-c
With a bash script (thanks #ekuusela) I now finally succeeded in what I wanted.
I added a fixture that calls the bashscript spinserver.sh in a new terminal window. This works in ubuntu, the command is different in different environments (see Execute terminal command from python in new terminal window? for other environments).
#pytest.fixture(scope="session", autouse=True)
def client(testenv):
if testenv != 'local':
pass
else:
p = subprocess.Popen(['gnome-terminal', '-x', './spinserver.sh'])
time.sleep(3)
yield
Here the very simple bashscript
#!/bin/bash
cd ..
source pypyenv/bin/activate
python app.py
The sleep command is necessary because the server takes some time to
initialize.
Don't forget to make your bash script executable (chmod
u+x spinserver.sh)
I tried to do a teardown after yield, but p.kill does not really
close the window. This is acceptable for me as it does not matter
if I have to manually close a terminal window & I can even see
flask debugging if necessary
I'm trying to benchmark the bundling process of our js bundles using pytest-benchmark. For accurate processing the target directory needs to empty. I've tried cleaning this on each run using the pedantic setup argument, but this only runs on initialization of the benchmark, and not in between runs. This is the code of my last try:
import shutil
import os
import pytest
def clean_bundles():
print("Cleaning bundles")
shutil.rmtree(os.path.abspath('precompiled'), True)
def bundle(gulpfile):
os.system("gulp --gulpfile %s createBundles" % gulpfile)
def test_bundle(benchmark):
benchmark.pedantic(lambda: bundle("gulpfile.js"), setup=clean_bundles(), rounds=5, iterations=1)
Is there anyway to run a clean between each iteration without making it part of the benchmark results?
Following these posts, I have managed to run my doctest within django with:
# myapp/tests.py
import doctest
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite())
return tests
Then running:
python manage.py tests
However, since I am used to test my (non-django) scripts with the simple command:
py.test --doctest-modules -x
I am now quite confused about:
testing procedure not stopping after first failure (my good'ol -x) (so I get flooded with results and I need to scroll back all the way up to the first problem each time)
option # doctest: +ELLIPSIS not being set by default.
How do I set this kind of options from this django load_tests() hook?
Okay, I've got it. Options flags like ELLIPSIS or FAIL_FAST can be
provided as an optionflags argument to DocTestSuite.
The right way to combine them, as reported here, is to bitwise OR them :)
So the following does work:
# myapp/tests.py
import doctest
def load_tests(loader, tests, ignore):
tests.addTests(doctest.DocTestSuite(
optionflags=doctest.ELLIPSIS | doctest.FAIL_FAST))
return tests