I'm trying to wrote a pex test, and I noticed that it always was feeding a false value as one of the params that I wanted. My test looked like this (simplified: there are/were more params, but otherwise no different):
[PexMethod]
public void TestCtor(bool value)
{
ArbitraryType myType = new ArbitraryType(value);
}
I wanted to test a scenario where I would have pex do the exploration, ensuring that value would be true. I made another test that looked like this:
[PexMethod]
public void TestCtor(bool value)
{
Contract.Requires(value == true);
ArbitraryType myType = new ArbitraryType(value);
}
But when I have Pex explore that, it still spits in false to value and the test it generates "passes". If I put a line after the requirement saying Contract.Assert(!value); It'll create another test and pass true for value to fail the assertion.
The question is, why isn't Pex satisfying the code contract?
I'm not sure what Pex is going to do with Contracts in the test methods, but I can't see it being a Good Thing :)
If you want Pex to do this, the correct thing to do is use PexAssume.IsTrue(value).
Related
I would like to know if it is possible to retrieve the name of a variable.
For example if I have a method:
def printSomething(def something){
//instead of having the literal String something, I want to be able to use the name of the variable that was passed
println('something is: ' + something)
}
If I call this method as follows:
def ordinary = 58
printSomething(ordinary)
I want to get:
ordinary is 58
On the other hand if I call this method like this:
def extraOrdinary = 67
printSomething(extraOrdinary)
I want to get:
extraOrdinary is 67
Edit
I need the variable name because I have this snippet of code which runs before each TestSuite in Katalon Studio, basically it gives you the flexibility of passing GlobalVariables using a katalon.features file. The idea is from: kazurayam/KatalonPropertiesDemo
#BeforeTestSuite
def sampleBeforeTestSuite(TestSuiteContext testSuiteContext) {
KatalonProperties props = new KatalonProperties()
// get appropriate value for GlobalVariable.hostname loaded from katalon.properties files
WebUI.comment(">>> GlobalVariable.G_Url default value: \'${GlobalVariable.G_Url}\'");
//gets the internal value of GlobalVariable.G_Url, if it's empty then use the one from katalon.features file
String preferedHostname = props.getProperty('GlobalVariable.G_Url')
if (preferedHostname != null) {
GlobalVariable.G_Url = preferedHostname;
WebUI.comment(">>> GlobalVariable.G_Url new value: \'${preferedHostname}\'");
} else {
WebUI.comment(">>> GlobalVariable.G_Url stays unchanged");
}
//doing the same for other variables is a lot of duplicate code
}
Now this only handles 1 variable value, if I do this for say 20 variables, that is a lot of duplicate code, so I wanted to create a helper function:
def setProperty(KatalonProperties props, GlobalVariable var){
WebUI.comment(">>> " + var.getName()" + default value: \'${var}\'");
//gets the internal value of var, if it's null then use the one from katalon.features file
GlobalVariable preferedVar = props.getProperty(var.getName())
if (preferedVar != null) {
var = preferedVar;
WebUI.comment(">>> " + var.getName() + " new value: \'${preferedVar}\'");
} else {
WebUI.comment(">>> " + var.getName() + " stays unchanged");
}
}
Here I just put var.getName() to explain what I am looking for, that is just a method I assume.
Yes, this is possible with ASTTransformations or with Macros (Groovy 2.5+).
I currently don't have a proper dev environment, but here are some pointers:
Not that both options are not trivial, are not what I would recommend a Groovy novice and you'll have to do some research. If I remember correctly either option requires a separate build/project from your calling code to work reliable. Also either of them might give you obscure and hard to debug compile time errors, for example when your code expects a variable as parameter but a literal or a method call is passed. So: there be dragons. That being said: I have worked a lot with these things and they can be really fun ;)
Groovy Documentation for Macros
If you are on Groovy 2.5+ you can use Macros. For your use-case take a look at the #Macro methods section. Your Method will have two parameters: MacroContext macroContext, MethodCallExpression callExpression the latter being the interesting one. The MethodCallExpression has the getArguments()-Methods, which allows you to access the Abstract Syntax Tree Nodes that where passed to the method as parameter. In your case that should be a VariableExpression which has the getName() method to give you the name that you're looking for.
Developing AST transformations
This is the more complicated version. You'll still get to the same VariableExpression as with the Macro-Method, but it'll be tedious to get there as you'll have to identify the correct MethodCallExpression yourself. You start from a ClassNode and work your way to the VariableExpression yourself. I would recommend to use a local transformation and create an Annotation. But identifying the correct MethodCallExpression is not trivial.
no. it's not possible.
however think about using map as a parameter and passing name and value of the property:
def printSomething(Map m){
println m
}
printSomething(ordinary:58)
printSomething(extraOrdinary:67)
printSomething(ordinary:11,extraOrdinary:22)
this will output
[ordinary:58]
[extraOrdinary:67]
[ordinary:11, extraOrdinary:22]
I am new to Groovy.
I have a function in which I am writing a value to map.
def addTraceEntry(key, value) {
def traceability = [:]
traceability[key] = value.trim()
println "This print happens in function addTraceEntry " + traceability
}
I have another function that needs to verify whether the above function works properly.
def testAddTraceEntry() {
def key = 'test_key'
def value = 'test_value'
addTraceEntry(key, value)
println "This print happens in function testAddTraceEntry " + traceability
assert value == traceability[key]
}
I am invoking the testAddTraceEntry() function using the function name:
testAddTraceEntry()
When I run this, I get the ERROR:
This print happens in function addTraceEntry [test_key:test_value]
Caught: groovy.lang.MissingPropertyException: No such property: traceability for class: HelloWorld
groovy.lang.MissingPropertyException: No such property: traceability for class: HelloWorld
at HelloWorld.testAddTraceEntry(HelloWorld.groovy:53)
at HelloWorld.run(HelloWorld.groovy:57)
In the function testAddTraceEntry it clearly does not know the value of traceability so seems like its giving an ERROR for that.
I tried to return the value of traceability.
def addTraceEntry(key, value) {
def traceability = [:]
traceability[key] = value.trim()
println "This print happens in function addTraceEntry " + traceability
return traceability
}
But this yields the same ERROR.
There are a bunch of things worth mentioning after seeing the code you have wrote.
First thing - the scope of variables and encapsulation. Let's throw away technicalities for a moment and focus on something even more important. In method addTraceEntry you persist some state, which is fine. However, the implementation of the method testAddTraceEntry reveals that this method tries to know way to much about the implementation details of addTraceEntry. It encapsulates (hides in other words) persistence logic (from the API point of view you, as a caller, don't know that it persists key and a value inside the map) and that is why testAddTraceEntry should never ever make assumptions that calling this method mutated some structure. If you do so, then:
your test method contracts side effects and not the expected business logic (storing data in some kind of global map - don't do it. Ever)
your test blocks any evolution of tested method implementation - imagine, that you decided to store key and value in a different structure. You may do it without breaking any API contract (your function produces the same results), but the test method will fail and you will have to modify it.
Second thing - your addTraceEntry method always produces a map with a single entry. It doesn't make much sense and if you call your function let's say 4 times you will end up with 4 maps where each one of them contain a single key mapped to a single value.
There are at least various ways to improve implementation of your methods. The simplest thing you can do is to implement a class that encapsulates logic for storing keys and values. Consider following example:
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.ConcurrentMap
class TraceEntriesStorage {
private final ConcurrentMap<String, Object> entries = [:] as ConcurrentHashMap
def addTraceEntry(String key, Object value) {
entries.put(key, value)
}
def containsTraceEntry(String key) {
return entries.containsKey(key)
}
def retrieveTraceEntryForKey(String key) {
return entries.get(key)
}
}
This is a simple class with 3 short methods. It stores trace entries inside the internal concurrent map (to solve problems with concurrent access). Now, your test method could look like this:
def storage = new TraceEntriesStorage()
storage.addTraceEntry("test_key", "test_value")
assert storage.containsTraceEntry("test_key")
assert storage.retrieveTraceEntryForKey("test_key") == "test_value"
You create an instance of this class, you add an entry and you check if methods containsTraceEntry and retrieveTraceEntryForKey return expected values. As you can see it doesn't matter where we stored this trace entry - it matters that the class we have implemented behaves as expected. To make this test method even better you could add an assertion that checks if there is no trace entry for test_key before we actually insert it - this way we know that adding trace entry change internal state of the class. But what is nice in this approach is that as long as we don't break the contract, we can experiment and modify implementation of TraceEntriesStorage. Because what is most important - adding trace entries have to allow to retrieve them back from the object. How it gets stored, where it gets stored - it doesn't matter.
I hope you find this answer useful and it will help you in learning Groovy and designing a better programs. Happy hacking!
You need to combine adding the return statement to addTraceEntry() with assigning the returned value to a variable in testAddTraceEntry():
def traceability = addTraceEntry(key, value)
I have written an AST transform that creates a setter for a JPA mapped property (it both sets the local field and calls setOwner on the other end of the relationship):
private static void createSetter(FieldNode field) {
Parameter parameter = GeneralUtils.param(field.getType(), field.getName());
BlockStatement body = new BlockStatement();
body.addStatement(assignS(fieldX(field), varX(parameter)));
MethodCallExpression setterCall = callX(varX(parameter), "setOwner", varX("this", field.getDeclaringClass()));
setterCall.setType(ClassHelper.VOID_TYPE);
body.addStatement(stmt(setterCall));
MethodNode method = new MethodNode(setterName(field.getName()), ACC_PUBLIC, ClassHelper.VOID_TYPE, new Parameter[] {parameter}, ClassNode.EMPTY_ARRAY, body);
field.getDeclaringClass().addMethod(method);
}
This works, but the generated method has a strange null statement at the end as disassembled by JD-GUI (in addition to an odd local variable):
public void setMore(Simple_MoreStuff more) {
Simple_MoreStuff localSimple_MoreStuff = more;
this.more = localSimple_MoreStuff;
more.setOwner(this);
null;
}
It doesn't seem to affect the actual correctness, but it's odd, and it seems like a bug. In MethodCallExpression, I found this comment but don't know if it relates, since my method is in fact void (I explicitly set it above, and it makes no difference):
//TODO: set correct type here
// if setting type and a methodcall is the last expression in a method,
// then the method will return null if the method itself is not void too!
// (in bytecode after call: aconst_null, areturn)
Is there a way to keep the generated method from having the spurious null?
I have not looked at JD-GUI, so I cannot tell how capable this tool is in understanding bytecode, that does not come from Java. But in general disassemblers can only somewhat show what Java code in that case might look like, by no means it is supposed to show correct code from a non-Java language. So better do not expect correct Java code if you disassemble Groovy.
In this case I suspect that JD-GUI stumbles over a workaround we have not gotten rid of yet. In several cases we add at the method end dead code, the const_null, areturn you have noticed. We do this because of problems with the verifier if a bytecode label is used at the end of a method. And since the dead code does not influence correctness we are currently using this solution.
I need to adapt my code to the stricter Java 7 verifier and have to add visitFrame calls in my MethodNode (I'm using the tree api). I could not find any information on how Type maps to the Object[]s used in visitFrame, so please help me out here...
This is what I have so far:
private Object getFrameType(Type type) {
switch (type.getSort()) {
case Type.BOOLEAN:
case Type.CHAR:
case Type.BYTE:
case Type.SHORT:
case Type.INT:
return Opcodes.INTEGER;
case Type.LONG:
return Opcodes.LONG;
case Type.FLOAT:
return Opcodes.FLOAT;
case Type.DOUBLE:
return Opcodes.DOUBLE;
case Type.OBJECT:
case Type.ARRAY:
return type.getInternalName();
}
throw new RuntimeException(type + " can not be converted to frame type");
}
What I'd like to know is: what are Type.VOID and Type.METHOD?
When do I need Opcodes.TOP, Opcodes.NULL and Opcodes.UNINITIALIZED_THIS?
I'm guessing UNINITIALIZED_THIS is only used in the constructor and I can probably ignore VOID and METHOD, but I'm not sure and I don't have the slightest idea what TOP is.
If I understood your need correctly, you could just let ASM calculate the frames for you. This will probably slow down the class generation a bit, but certainly worth a try.
When creating the ClassWriter, just add COMPUTE_FRAMES to the flags argument of the constructor, e.g.
new ClassWriter(ClassWriter.COMPUTE_FRAMES);
Similarly, if you are transforming a class, the ClassReader can be asked to expand the frames like:
ClassReader cr = ...;
ClassNode cn = new ClassNode(ASM4);
cr.accept(cn, ClassReader.EXPAND_FRAMES);
The former option has the benefit that you can forget about the frames (and "maxs") altogether while the latter option might require you to patch the frames yourself depending on what kind of transformation you do.
The examples are for ASM version 4, but these features have been supported at least since version 3 - the parameters are just passed a bit differently.
The business scenario I'm trying to test with cucumber/gherkin (specflow, actually) is that given a set of inputs on a web form, I make a request, and need to ensure that (under certain conditions), when the result is returned, a particular field hasn't changed (under other condition, it does). E.g.
Given I am on the data entry screen
When I select "do not update frobnicator"
And I submit the form
And the result is displayed
Then the frobnicator is not updated
How would I write the step "the frobnicator is not updated"?
One option is to have a step that runs before "I submit the form" that reads something like "I remember the value of the frobnicator", but that's a bit rubbish - it's a horrible leak of an implementation detail. It distracts from the test, and is not how the business would describe this. In fact, I have to explain such a line any time anyone sees it.
Does anyone have any ideas on how this could be implemented a bit nicer, ideally as written?
I disagree with the previous answer.
The gherkin text you felt like you wanted to write is probably right.
I'm going to modify it just a little to make it so that the When step is the specific action that is being tested.
Given I am on the data entry screen
And I have selected "do not update frobnicator"
When I submit the form
Then the frobnicator is not updated
How exactly you Assert the result will depend on how your program updates the frobnicator, and what options that gives you.. but to show it is possible, I'll assume you have decoupled your data access layer from your UI and are able to mock it - and therefore monitor updates.
The mock syntax I am using is from Moq.
...
private DataEntryScreen _testee;
[Given(#"I am on the data entry screen")]
public void SetUpDataEntryScreen()
{
var dataService = new Mock<IDataAccessLayer>();
var frobby = new Mock<IFrobnicator>();
dataService.Setup(x => x.SaveRecord(It.IsAny<IFrobnicator>())).Verifiable();
ScenarioContext.Current.Set(dataService, "mockDataService");
_testee = new DataEntryScreen(dataService.Object, frobby.Object);
}
The important thing to note here, is that the given step sets up the object we are testing with ALL the things it needs... We didn't need a separate clunky step to say "and i have a frobnicator that i'm going to memorise" - that would be bad for the stakeholders and bad for your code flexibility.
[Given(#"I have selected ""do not update frobnicator""")]
public void FrobnicatorUpdateIsSwitchedOff()
{
_testee.Settings.FrobnicatorUpdate = false;
}
[When(#"I submit the form")]
public void Submit()
{
_testee.Submit();
}
[Then(#"the frobnicator is not updated")]
public void CheckFrobnicatorUpdates()
{
var dataService = ScenarioContext.Current.Get<Mock<IDataAccessLayer>>("mockDataService");
dataService.Verify(x => x.SaveRecord(It.IsAny<IFrobnicator>()), Times.Never);
}
Adapt the principle of Arrange, Act, Assert depending on your circumstances.
Think about how you would test it manually:
Given I am on the data entry screen
And the blah is set to "foo"
When I set the blah to "bar"
And I select "do not update frobnicator"
And I submit the form
Then the blah should be "foo"