#Stepwise
Class TestCaseOne extends Specification{
def test(){
expect:
assert something
}
def testValidation(){
expect:
assert something
}
def test(){}
expect:
assert something
}
def testValidation(){
expect:
assert something
}
}
I want testing should stop if test method fails but it should continue if testValidation method fails. Please let me know if it is possible.
I am using Groovy and spock.Thanks in advance.
According to this 'issue' which covers your question https://github.com/spockframework/spock/issues/456 recommended way if you want to achieve full test execution is to not use #Stepwise annotation.
robfletcher commented Aug 30, 2015
Just don't use #Stepwise then. Execution is sequential nevertheless. This might change
in case Spock itself ever gets some parallel execution support, but for now you'll be fine.
Reported by pniederw on 2013-10-24 08:47:44
What you are asking is not possible.
Either you use #Stepwise and it stops at the first failure. Or your don't use #Stepwise and it runs everything.
There is no way to mark specific methods on what should happen.
Split your test into two where one has the annotation and the other does not.
Related
I'm beginning to write unit tests with spockframework, and I want to verify the arguments passed to some method of a class my test depends on. In Mockito we have ArgumentCaptor to do this, but I couldn't figure out a way to do that on spock.
Maybe some manipulation on the closure (I'm still trying to learn that)?
Or spock have some built in functionality?
I appreciate any guidance to learn that!
This is how you do it in Spock:
def "it checks the arguments passed to the helper"() {
given:
def cut =new Cut(helper:Mock(Helper))
when:
cut.doStuff()
then:
1 * cut.helper(expectedArg) >> returnedResult
where:
expectedArg="This is expected"
returnedResult="xyz"
}
Im using Spock for my tests and I have multiple classes to test. I want to tell Spock to test each class in specific order. Is there a way to do this? I noticed TestNG has got #AfterTest annotation, so does Spock have something similar?
You can't specify Spock test classes execution order. However Spock allows you to specify methods execution order when you add #Stepwise annotation to your test class - in this case Spock will execute all methods in order from top to bottom and if one method fails it will ignore remaining methods in this test class.
Indicates that a spec's feature methods should be run sequentially in their declared order (even in the presence of a parallel spec runner), always starting from the first method. If a method fails, the remaining methods will be skipped. Feature methods declared in super- and subspecs are not affected.
#Stepwise is useful for specs with (logical) dependencies between methods. In particular, it helps to avoid consecutive errors after a method has failed, which makes it easier to understand what really went wrong.
Reference: http://spockframework.org/spock/javadoc/1.1/spock/lang/Stepwise.html
#Stepwise
class StepwiseExample extends Specification {
def "first test method to run"() {
// ....
}
def "second test method to run"() {
// ....
}
def "if previous method failed this one will be ignored"() {
// ....
}
}
Using org.junit.runners.Suite
Jeff Scott Brown gave a good comment about JUnit's #Suite.SuiteClasses. In this case you create a class where you can aggregate your test classes (specifications) into a single test suite and those classes will be executed in the order they have been defined in the suite. Consider following example:
import org.junit.runner.RunWith
import org.junit.runners.Suite
#RunWith(Suite)
#Suite.SuiteClasses([
Test2Specification,
Test1Specification
])
class TestSuiteSpecification { }
This suite executes two specifications Test2Specification and Test1Specification in the defined order:
I have a test class:
class Homepage extends AbstractTest {
#TestCase('TC44424')
def "Rust Checkout - Homepage gallery blade"() {
given:
code....
when:
code..
then:
code..
when:
code..
then:
code..
}
}
So here if suppose my first When-Then block fails I don't want my script to fail I want it to go to another When-Then block and continue running.Is it possible?
Since these test cases aren't building up any state (and thus you think they could continue running after a previous one fails), you might consider putting each when/then in their own test method. You can move the given to a def setup() { ... } method as well.
Alternatively, you could construct your test with a where block. This might not be feasible depending on how different each test case is from each other.
I am using spockframework and geb for test automation. I would like to execute after every feature a simple check to be sure that no error dialogs are shown, I have added the following cleanup() method:
def cleanup() {
expect:
$('.myErrrorDialogClass').isEmpty()
}
The code is executed after every feature but it does not throw any error when the dialog is shown.
Spock uses AST transforms to wire in the functionality for each test label (when, expect, etc); they may not run the transformations on the cleanup method. They are either not expecting or not encouraging assertions in cleanup, so that code may run but not actually assert anything.
You can get around this by using a standard Groovy assert call without the expect block.
Summarized from our comment discussion above - in case you want to accept it as an answer ;-)
I'm trying to modularize my test cases, so I'm running a shared test case (as a procedure) that does something useful and returns a result value. As I need to pass-in non-string input properties, I have to run the test case from groovy:
def findLoopEndTC = testRunner.testCase.testSuite.testCases["TestCase - Find Loop End"]
assert findLoopEndTC != null, "Referred TC not found"
def runContext = new com.eviware.soapui.support.types.StringToObjectMap()
runContext.put("TestStepContext", context)
def runner = findLoopEndTC.run( runContext, false )
assert runner.status != com.eviware.soapui.model.testsuite.TestRunner.Status.FAILED : runner.reason
I've learned that the test case is run using the SINGLETON_AND_WAIT mode which ensures that the TestCase itself is run in a thread-safe way.
My question is how to return a value from the run test case in a thread-safe way?
I tried runner.getRunContext().getProperty("Result"), but it seems that the context properties are no longer there. So there seems to be only the "classical" way, findLoopEndTC.getPropertyValue("Result"), but this is aparently not thread-safe.
Are there other possibilities?
I use the free version of SoapUI.
I had the same problem. If I understand you correctly, this is what you want:
You’ve put the ‘calling’ context into a new context ‘runContext’:
context.get("TestStepContext").put("Results",resultList)
which has been passed in as the context for the test case to be run (synchronously). I’ll call the test case to be run ‘B’:
def runner = findLoopEndTC.run( runContext, false ) //in calling test case
To get something useful back from ‘B’, somewhere in it you need to put a value back into TestStepContext, e.g.:
context.get("TestStepContext").put("Results",resultList) //My results happened to be a list
In the calling test case, the line you need after the call to run the test case is:
def testResults = runContext.get("TestStepContext").get("Results")
Hope this makes sense.
I've been trying to work this out for the last few days too. I haven't been able to work out how to make it thread safe but I have an alternative approach which I think works pretty well.
I've based it on this http://forum.soapui.org/viewtopic.php?f=2&t=4681#p15731 suggestion from the SoapUI team. I found with the above solution it was still not thread safe, 99% of the time this worked but I found sometimes it's possible you can have two test cases both breaking out the loop at the same time.
To deal with this I set runningDeleteCar to the the hashcode for the current testRunner when it's broken out of the loop. I then double check this to make sure that some other test case hasn't gone in and changed it, if it doesn't match I just go back to the while loop. This stops the situation of two test cases breaking out at the same time.
This approach basically means only one test case can go through the shared test case at a time.