I am teaching an introductory python course (python 3+Jupyter), and have been formulating assignments using nbgrader. For those not familiar, this basically means marking student's code via a set of assert statements. If the assert doesn't pass, they don't get the mark.
One of the tests that I want to perform is to check that students are writing their own tests. As a very simple example, let's imagine that they're supposed to be defining my_function, and all the tests that they want to run on it are supposed to be a series of assert statements inside a function do_tests() which will return True if all the tests pass.
One thing that I can obviously require of their code is that do_tests() passes simply by calling assert do_tests(). I can also check that it fails if I del my_function. However, I also want to check a bit more detail about the content of do_tests(). As a first step, I simply wanted to count the number of assert statements that they have used within the definition, and was intending to use unittest.mock.patch, trying to adapt the code from here. However, I could not figure out how to mock assert. I tried something like
from unittest.mock import patch
with patch('__main__.assert') as mock_assert:
do_tests()
but I just get an error that main does not have a method assert, and couldn't work out what module assert should be a part of.
As a crude interim, I have ended up doing
import inspect
lines = inspect.getsource(do_tests)
assert lines.count("\n assert")>=3,"You don't appear to have enough assert statements"
but obviously that doesn't give me access to any other features that mocking might offer.
Related
Fixtures that parameterize tests can be created as such:
#pytest.fixture(scope="session", params=[“param1”, “param2"])
def fixture_that_parametrizes(request) -> str:
"""
This fixture parametrizes a test function,
as if #pytest.mark.parametrize(“param”, [“param1”, “param2”]) was used.
"""
return request.param
def parametrized_test(fixture_that_parametrizes):
“””
This test would run twice and print “param1” on the first
and “param2” on the second run.
print(fixture_that_parametrizes)
assert 0
What I would like to do, is to create a fixture, that parametrizes tests differently, based on what markers are set within the test.
I haven’t found anything in the documentation regarding this. An idea I had was to create a parameterizing fixture with specific parameters dynamically within a fixture, but this approach seems a bit clunky.
Are there any better solutions or alternatives?
From my understanding, toHaveBeenNthCalledWith within a jest test should test only the "Nth" call with "N" being a non-zero number. My test below returns the Nth call and the one after it. In the test, I give N a value of one and it is showing two. Now I can replaced the one with a two and then it will show 2 and 3. This is causing my test to fail. Are there any suggestions to why it is behaving like this??
Probably your mock is saving state between tests, this means that it "holds" previous calls, therefore, you see 4 calls.
Try to use new mock for each test, usually done by configuring the mock in a beforeEach hook.
Edit
You can use .toHaveBeenNthCalledWith which specifies which call is under test.
My overall problem was whitespace. Plus I had to change the way the underlining executeQuery function was being called in the actual controller. So the expect statements in the third image are actually correct. I copied the sql statement from the controller and pasted in the test. I did NOT add tabs and it worked.
I am working on Verigy 93K test program and I have a logic that I would like to know if there's an equivalent code in Origen.
I am working on Verigy 93K test program and I have this logic (IF condition) that I need to insert in my flow.
Basically, I have a variable called 'INSERTION' and this will have different values like 'GCORR', 'VCORR' and others.
I would like to know if there's an equivalent code like this in Origen.
I attached a snapshot, hope that it can help clarify my question more.
In this logic, I would like to check the INSERTION value and if the value is not equal to GCORR or VCORR, the logic should pass, else, fail.
Here is the screenshot:
This pull-request adds an official API for this.
This example would be implemented as:
whenever_any ne(:INSERTION, 'GCORR'), ne(:INSERTION, 'VCORR') do
# Your tests in here
end
That would produce something logically equivalent and which can be re-targeted to other platforms.
If you don't care about that and want to produce exactly as you have it in the above example, then this should work too (where the OR is hard-coded for V93K syntax):
whenever ne(:INSERTION, 'GCORR|VCORR') do
# Your tests in here
end
Here is the preliminary documentation of this feature from the above PR - https://github.com/Origen-SDK/origen_testers/blob/66345c9422d9fa6b2577af20110259e45c2bdd26/templates/origen_guides/program/flowapi.md.erb#L71
I couldn't find api support on flow control or variable values beyond "if/unless_enable" support which can help check for 1 or zero. One way is to use render.
render 'if #INSERTION != "GCORR|VCORR" then'
render '{'
# your code for non-GCORR_VCORR flow
render "} \n else \n { \n } "
I am running test cases using pytest; although I would like to collect some logs and zip everything, only if there is a failure
I did write the function that collect logs and zip it, although I can't find a way to actually trigger that for each failed case. I did implement a solution using the #classmethod decorators for setUpClass and tearDownClass; but this happen only after all the tests have been ran, and not when a specific test fail.
In each test I run a simple assertFalse(a < b, "the test did fail". I would like to trigger the function that does the log collection only when that condition is actually triggered.
Seems that there is no direct way to do it in the assert statement, or I was not able to find a way for it.
I just used try-catch and in the try I raise an exception,and in the catch code I call the function. Not sure if this is the best way, but it works in my case
try:
if (a < b):
raise ValueError ("wrong output, collecting logs")
except ValueError:
# call function to parse logs
self.parseoutputonerror(a, b, "/tmp/out.zip")
I have a python module which has several methods:
module.py
def a():
return "This is a method"
I want to add some additional functionality to method a by calling it from script to avoid modification in module itself.
How can I add additional operations or arguments to that module calling it from script?
For example I imported module.py in my script and in that script I add two arguments to method "a" and addition of those arguments in addition to:
return "This is a method"
Well, you cannot really change any imported methods as far as I am aware. That does not mean it is completely impossible, I just wouldn't know how.
Question is though if that is actually what you want (as I believe it is a very uncommon way of handling code and I am guessing not an advisable one)?
Could it be that what you actually want to achieve is something more like this:
module_script.py
def a(variable):
return_value = "This is a " + str(variable) + " method!"
return return_value
second_script.py
import module_script
print(module_script.a("nice"))
The output of the second script would be:
"This is a nice method!"