I'm trying to modularize my test cases, so I'm running a shared test case (as a procedure) that does something useful and returns a result value. As I need to pass-in non-string input properties, I have to run the test case from groovy:
def findLoopEndTC = testRunner.testCase.testSuite.testCases["TestCase - Find Loop End"]
assert findLoopEndTC != null, "Referred TC not found"
def runContext = new com.eviware.soapui.support.types.StringToObjectMap()
runContext.put("TestStepContext", context)
def runner = findLoopEndTC.run( runContext, false )
assert runner.status != com.eviware.soapui.model.testsuite.TestRunner.Status.FAILED : runner.reason
I've learned that the test case is run using the SINGLETON_AND_WAIT mode which ensures that the TestCase itself is run in a thread-safe way.
My question is how to return a value from the run test case in a thread-safe way?
I tried runner.getRunContext().getProperty("Result"), but it seems that the context properties are no longer there. So there seems to be only the "classical" way, findLoopEndTC.getPropertyValue("Result"), but this is aparently not thread-safe.
Are there other possibilities?
I use the free version of SoapUI.
I had the same problem. If I understand you correctly, this is what you want:
You’ve put the ‘calling’ context into a new context ‘runContext’:
context.get("TestStepContext").put("Results",resultList)
which has been passed in as the context for the test case to be run (synchronously). I’ll call the test case to be run ‘B’:
def runner = findLoopEndTC.run( runContext, false ) //in calling test case
To get something useful back from ‘B’, somewhere in it you need to put a value back into TestStepContext, e.g.:
context.get("TestStepContext").put("Results",resultList) //My results happened to be a list
In the calling test case, the line you need after the call to run the test case is:
def testResults = runContext.get("TestStepContext").get("Results")
Hope this makes sense.
I've been trying to work this out for the last few days too. I haven't been able to work out how to make it thread safe but I have an alternative approach which I think works pretty well.
I've based it on this http://forum.soapui.org/viewtopic.php?f=2&t=4681#p15731 suggestion from the SoapUI team. I found with the above solution it was still not thread safe, 99% of the time this worked but I found sometimes it's possible you can have two test cases both breaking out the loop at the same time.
To deal with this I set runningDeleteCar to the the hashcode for the current testRunner when it's broken out of the loop. I then double check this to make sure that some other test case hasn't gone in and changed it, if it doesn't match I just go back to the while loop. This stops the situation of two test cases breaking out at the same time.
This approach basically means only one test case can go through the shared test case at a time.
Related
I have some tests that depend on a certain thing being true (access to the internet, as it happens, but that isn't important and I don't want to discuss the details of the condition).
I can very easily write a static helper method which will test the (parameterless) condition and call Assert.Inconclusive("Explanatory Message") if it's true/false. And then call that at the start of each Test which has this requirement.
But I'd like to do this as an Attribute, if possible.
How, in detail, do I achieve that, in NUnit?
What I've tried so far:
There's an IApplyToTest interface, exposed by NUnit, which I can make my Attribute implement, and will allow me to hook into the TestRunner, but I can't get it to do what I want :(
That interface gives me access to an NUnit.Framework.Internal.Test object.
If I call:
test.RunState = RunState.NotRunnable;
then I get something equivalent to Assert.Fail("").
Similarly RunState.Skipped or RunState.Ignored give me the equivalent of Assert.Ignore("").
But none of these are setting a message on the Test, and there's no test.Message = "foo"; or equivalent (that I can see).
There's a test.MakeInvalid("Foo") which does set a message, but that's equivalent to Assert.Fail("Foo").
I found something that looked promising:
var result = test.MakeTestResult();
result.SetResult(ResultState.Inconclusive, "Custom Message text");
But that doesn't seem to do anything; the Test just Passes :( I looked for a test.SetAsCurrentResult(result) method in case I need to "attach" that result object back to the test? But nothing doing.
It feels like this is supposed to be possible, but I can't quite figure out how to make it all play together.
If anyone can even show me how to get to Skipped + Custom Message displayed, then I'd probably take that!
If you really want your test to be Inconclusive, then that's what Assume.That is there for. Use it just as you would use Assert.That and the specified constraint fails, your test result will be inconclusive.
That would be the simplest answer to your question.
However, reading the things you have tried, I don't think you actually want Inconclusive at least not as it is defined by NUnit.
In NUnit, Inconclusive means that the test doesn't count because it couldn't be run. The result basically disappears and the test run is successful.
You seem to be saying that you want to receive some notice that the condition failed. That makes sense in the situation where (for example) the internet was not available so your test run isn't definitive.
NUnit provides Assert.Ignore and Warn.If (also Warn.Unless) for those situations. Or you can set the corresponding result states in your custom attribute.
Regarding implementation... The RunState of a test applies to it's status before anyone has even tried to execute it. So, for example, the RunState may be Ignored if someone has used the IgnoreAttribute or it may be NotRunnable if it requires arguments and none are provided. There is no Inconclusive run sttate because that would mean the test is written to be inconclusive all the time, which makes no ssense. The IApplyToTest interface allows an attribute to change the status of a test at the point of discovery, before it is even run, so you would not want to use that.
After NUnit has attempted to run a test, it gets a ResultState, which might be Inconclusive. You can affect this in the code of the test but not currently through an attribute. What you want here is something that checks the conditions needed to run the test immediately before running it and skips execution if the conditions are not met. That attribute would need to be one that generates a command in the chain of commands that execute a test. It would probably need to implement ICommandWrapper to do that, which is a bit more complicated than IApplyToTest because the attribute code must generate a command instance that will work properly with NUnit itself and with other commands in the chain.
If I had this situation, I believe I would use a Run parameter to indicate whether the internet should be available. Then, the tests could
Assume.That(InternetIsNotNeeded());
silently ignoring those tests or fail as expected when the internet should be available.
#Stepwise
Class TestCaseOne extends Specification{
def test(){
expect:
assert something
}
def testValidation(){
expect:
assert something
}
def test(){}
expect:
assert something
}
def testValidation(){
expect:
assert something
}
}
I want testing should stop if test method fails but it should continue if testValidation method fails. Please let me know if it is possible.
I am using Groovy and spock.Thanks in advance.
According to this 'issue' which covers your question https://github.com/spockframework/spock/issues/456 recommended way if you want to achieve full test execution is to not use #Stepwise annotation.
robfletcher commented Aug 30, 2015
Just don't use #Stepwise then. Execution is sequential nevertheless. This might change
in case Spock itself ever gets some parallel execution support, but for now you'll be fine.
Reported by pniederw on 2013-10-24 08:47:44
What you are asking is not possible.
Either you use #Stepwise and it stops at the first failure. Or your don't use #Stepwise and it runs everything.
There is no way to mark specific methods on what should happen.
Split your test into two where one has the annotation and the other does not.
I am using spockframework and geb for test automation. I would like to execute after every feature a simple check to be sure that no error dialogs are shown, I have added the following cleanup() method:
def cleanup() {
expect:
$('.myErrrorDialogClass').isEmpty()
}
The code is executed after every feature but it does not throw any error when the dialog is shown.
Spock uses AST transforms to wire in the functionality for each test label (when, expect, etc); they may not run the transformations on the cleanup method. They are either not expecting or not encouraging assertions in cleanup, so that code may run but not actually assert anything.
You can get around this by using a standard Groovy assert call without the expect block.
Summarized from our comment discussion above - in case you want to accept it as an answer ;-)
I am a beginner in soapui testing. Hopefully you can help me solving this problem.
In my test project I have a test suites which contains several test cases. Multiple test case will start the same test case. To run this test case I need some property values to be transferred to this test case.
I tried to achieve this in two ways. But I failed in both.
I tried to call the test case and set the needed properties in the test case. I start the test case from a Groovy script. But I couldn't find a good example how to set the properties in the called test case.
I tried to get the property values of the calling parent test case inside the called test case. It looks like the parent test case that called the test case isn't available in the context of the running test case.
The test cases, that will call the same test case, will be run in parallel. So, I think it isn't a solution to first set the property values and then start the test case, because they will be overwritten by the other test cases that run at the same time. Also using test suite properties for these values won’t work because of running the test cases in parallel.
My test project looks like this.
MyProject
TestSuite_APLtests
TestCase_user_01
Properties test step
Run_test <groovy script>
Step_01
…..
TestCase_user_02
Properties test step
Run_test <groovy script>
Step_01
…..
TestCase_General
Properties test step
POST sessions
Step_01
…..
The ‘Properties test step’ of each ‘TestCase_user_’ contains a user and password needed in test case ‘TestCase_General’ and will be different for each test case.
In the ‘Run_test’ groovy script of each ‘TestCase_user_’ the test case ‘TestCase_General’ is started by using:
def myTestSuite = testRunner.testCase.testSuite.project.getTestSuiteByName("TestSuite_APLtests")
def myTestCase = myTestSuite.getTestCaseByName("TestCase_General")
myTestCase.run(null, false)
How can I add the properties user and password to the run comment that starts the test case?
If I try to get the property values with a groovy script in test case ‘TestCase_General’ I don’t know how to determine which test case has called ‘TestCase_General’. I found some posts on internet that suggests to use: context.getProperty("#CallingRunTestCaseStep#") to determine the calling test case. But this value is null. And when I try to check if the calling test case is available in the context by using: context.hasProperty("#CallingRunTestCaseStep#") this is false, so this doesn't work to find the calling test case.
Can someone tell me what the solution will be to get this working.
Thanks,
You can set Test Case properties from groovy script with setPropertyValue(name,value) method, however if you run the Test Cases in parallel, this properties as you said will be overwritten for each Test Case calling TestCase_General. So instead of use setPropertyValue you can pass the context properties through the run(StringToObjectMap properties, boolean async) method in the WsdlTestCase.java class. Your groovy code to call TestCase_General could be:
import com.eviware.soapui.support.types.StringToObjectMap
// get test suite
def myTestSuite = testRunner.testCase.testSuite.project.getTestSuiteByName("TestSuite_APLtests")
// get your test case
def myTestCase = myTestSuite.getTestCaseByName("TestCase_General")
// set the user and password properties in the context
context.setProperty("user","userTestCaseN")
context.setProperty("password","passwordTestCaseN")
// run the testCase passing the context
def contextMap = new StringToObjectMap( context )
myTestCase.run(contextMap,false);
To access the context properties in the groovy script of your TestCase_General use this code:
context.getProperty("userPassword")
Or if you prefer to use context.expand:
context.expand('${#user}')
Note that the use of # depends on how you are accessing the properties.
If you also need to use the context properties in the SOAP Test Request of your TestCase_General use this way ${#propetryName} i.e:
<Envelope xmlns="http://schemas.xmlsoap.org/soap/envelope/">
<Header/>
<Body>
<request>
<user>${#user}</user>
</request>
</Body>
</Envelope>
Hope this helps,
Wanted but not invoked: However, there were other interactions with this mock:
This is a mockito error you would catch when trying to verify the invocation on an object on specific method, but what happens is you have interacted with other method of that object but not the one mentioned.
If you have an object named CustomerService and say it has two methods named saveCustomer() and verifyExistingCustomer(),
and your mockito looks something like verify(customerService, atleast(1)).verifyExistingCustomer(customer), but in your actual service you called the saveCustomer() at least one.
Any idea how to resolve this ?
From what you are describing, it looks like you are telling your mocks that you are expecting verifyExistingCustomer() to be called but you are not actually calling it.
You should probably look at your test design, specifically ensuring that you can (via mocking) isolate your tests to test each method individually.
If there is something in your code that decides whether to call saveCustomer() or verifyExistingCustomer() then you should try to mock the data that the code inspects so that you can test each individually.
For example if your code looked like this:
if (customer.getId() == 0) {
saveCustomer(customer);
} else {
verifyExistingCustomer(customer);
}
Then you could have two separate tests that you could isolate by setting a zero value and non-zero value for the id in customer.
If you'd like to share your code I could probably give you a better example.