Im using Spock for my tests and I have multiple classes to test. I want to tell Spock to test each class in specific order. Is there a way to do this? I noticed TestNG has got #AfterTest annotation, so does Spock have something similar?
You can't specify Spock test classes execution order. However Spock allows you to specify methods execution order when you add #Stepwise annotation to your test class - in this case Spock will execute all methods in order from top to bottom and if one method fails it will ignore remaining methods in this test class.
Indicates that a spec's feature methods should be run sequentially in their declared order (even in the presence of a parallel spec runner), always starting from the first method. If a method fails, the remaining methods will be skipped. Feature methods declared in super- and subspecs are not affected.
#Stepwise is useful for specs with (logical) dependencies between methods. In particular, it helps to avoid consecutive errors after a method has failed, which makes it easier to understand what really went wrong.
Reference: http://spockframework.org/spock/javadoc/1.1/spock/lang/Stepwise.html
#Stepwise
class StepwiseExample extends Specification {
def "first test method to run"() {
// ....
}
def "second test method to run"() {
// ....
}
def "if previous method failed this one will be ignored"() {
// ....
}
}
Using org.junit.runners.Suite
Jeff Scott Brown gave a good comment about JUnit's #Suite.SuiteClasses. In this case you create a class where you can aggregate your test classes (specifications) into a single test suite and those classes will be executed in the order they have been defined in the suite. Consider following example:
import org.junit.runner.RunWith
import org.junit.runners.Suite
#RunWith(Suite)
#Suite.SuiteClasses([
Test2Specification,
Test1Specification
])
class TestSuiteSpecification { }
This suite executes two specifications Test2Specification and Test1Specification in the defined order:
Related
I am working on a shared library for Jenkins, and I want to access some utilities methods between some classes, but not all of them, thus I have established some statements:
I would like to avoid using static methods, since it does not access pipeline steps directly, and passing the pipeline instance every call would be a pain;
I would like to avoid a singleton as well, or prefixing every method call with the util class' instance;
Since it is not supposed to be shared between all classes I would like to avoid putting every method as a file on vars/ special directory, but I would like a similar behavior;
Despite extending the class would be a anti-pattern, it would be acceptable, though I would like to avoid the verbose Java syntax for declaring the class the same name as the file, once it is implicit in groovy;
This question does solve my problem partially, although there are issues with serialization, I noted that when I use checkpoint and some build is resumed from some stage, the instance loses all extra methods.
This other question would have helped me fix the serialization issue, however the author seems the have solved the root cause of his problem using a way that is not the original question titled for.
Is there a way to extends a implicit script class in groovy without using the class NameOfFile extends SomeOtherClass { put every thing inside this block } syntax? And without working with inner-class?
Or else, is there a way to declare a constructor using the script groovy syntax analogue as the previous question?
Or even, is there a way to change the serialization behavior to install the extra methods again after unserializing?
Appendix
The script syntax works more-or-less like this:
Consider the content of file src/cicd/pipeline/SomePipeline.groovy:
package cicd.pipeline
// there is no need to wrap everything inside class SomePipeline,
// since it is implicit
def method() {
// instance method, here I can access pipeline steps freely
}
def static otherMethod() {
// static method, here it is unable to access pipeline steps
// without a instance
}
#groovy.transform.Field
def field
def call() {
// if the class is used as method it will run
this.method()
SomePipeline.otherMethod() // or simply otherMethod() should work
this.field = 'foo'
println "this instance ${this.getClass().canonicalName} should be cicd.pipeline.SomePipeline"
}
// any code other than methods or variables with #Field
// annotation will be inside a implicit run method that is
// triggered likewise main method but isn't a static one
def localVar = 'foo'
println "It will not execute on constructor since it is on run: $localVar"
println "Method: ${org.codehaus.groovy.runtime.StackTraceUtils.sanitize(new Throwable()).stackTrace[0].methodName}"
println "this instance ${this.getClass().canonicalName} should be cicd.pipeline.SomePipeline"
If I was going to use the Java verbose syntax I would have to wrap almost everything inside a class SomePipeline which is implicit in groovy, this is the script syntax I want to keep.
I realised that this.getClass().superclass.canonicalName when outside Jenkins pipeline is groovy.lang.Script and when inside pipeline is org.jenkinsci.plugins.workflow.cps.CpsScript and based on this resource I was able to elaborate the following solution:
abstract class CustomScript extends org.jenkinsci.plugins.workflow.cps.CpsScript {
public CustomScript() {
// do something here, it will always execute regardless
// serialization, and before everything
}
}
#groovy.transform.BaseScript CustomScript baseScript
That is it, worked as expected! Of course you can elaborate this solution better in order to reduce repeating and avoid inner-classes, but I will leave it for your imagination.
I started to learn metaprogramming. So far I used it to read annotations in Java and create sql statements based on given class but I don't see any practical use in it. Can you give me an example where and how you used it in your programs?
Annotating a method to be recogonized as a unit test.
import static org.junit.jupiter.api.Assertions.assertEquals;
import example.util.Calculator;
import org.junit.jupiter.api.Test;
class MyFirstJUnitJupiterTests {
private final Calculator calculator = new Calculator();
#Test
void addition() {
assertEquals(2, calculator.add(1, 1));
}
}
Note that the annotation here #Test exists as a flag. When the test runner launches, it registers a listener for annotated methods, and if it encounters an annotated method that is annotated with the Test annotation interface, it will register that test in the "list of tests to run"
See https://junit.org/junit5/docs/current/user-guide/ for more details.
There are many examples of annotations in the wild. This is just but one of them.
I am mocking an interface array which throws java.lang.IllegalArgumentException: Cannot subclass final class class.
Following are the changes I did.
Added the following annotations at class level in this exact order:
#Runwith(PowerMockRunner.class)
#PrepareForTest({ Array1[].class, Array2[].class })
Inside the class I am doing like this:
Array1[] test1= PowerMockito.mock(Array1[].class);
Array2[] test2= PowerMockito.mock(Array2[].class);
and inside test method:
Mockito.when(staticclass.somemethod()).thenReturn(test1);
Mockito.when(staticclass.somediffmethod()).thenReturn(test2);
Basically I need to mock an array of interfaces.
Any help would be appreciated.
Opening up another perspective on your problem: I think you are getting unit tests wrong.
You only use mocking frameworks in order to control the behavior of individual objects that you provide to your code under test. But there is no sense in mocking an array of something.
When your "class under test" needs to deal with some array, list, map, whatever, then you provide an array, a list, or a map to it - you just make sure that the elements within that array/collection ... are as you need them. Maybe the array is empty for one test, maybe it contains a null for another test, and maybe it contains a mocked object for a third test.
Meaning - you don't do:
SomeInterface[] test1 = PowerMock.mock() ...
Instead you do:
SomeInterface[] test1 = new SomeInterface[] { PowerMock.mock(SomeInterface.class) };
And allow for some notes:
At least in your code, it looks like you called your interface "Array1" and "Array2". That is highly misleading. Give interfaces names that say what their behavior is about. The fact that you later create arrays containing objects of that interface ... doesn't matter at all!
Unless you have good reasons - consider not using PowerMock. PowerMock relies on byte-code manipulation; and can simply cause a lot of problems. In most situations, people wrote untestable code; and then they turn to PowerMock to somehow test that. But the correct answer is to rework that broken design, and to use a mocking framework that comes without "power" in its name. You can watch those videos giving you lengthy explanations how to write testable code!
I'm wondering,
If in #Before method I'm initializing a mock objects, shouldn't I nullify references to it in #After ? Or would that be redundant? And why?
Not necessary, JUnit creates a new instance of the test per test method.
However if it's static fields it's another story, and proper lifecycle should be implemented, but I strongly advise you to not use static fields in a JUnit test ! Instead think about implementing your own JUnit Runner.
And for TestNG, it's a different story, as TestNG creates a single instance of the test, so you have to be careful there on the lifecycle of the mocks.
"Nullifying" reference doesn't change anything here.
#Before annotated method is ran before each test method. If you are initializing mocks in such method they will be reinitialized before each test. There is a different annotation - #BeforeClass, this annotation causes a method to be executed only once before execution of any test method in that test class. In this case however "nullifying" a reference will not help you because you still need to create a new mock object and assign its reference to your field.
I have heard of this term many times (in the context of programming) but couldn't find any explanation of what it meant. Any good articles or explanations?
I think you're referring to test fixtures:
The purpose of a test fixture is to ensure that there is a well known
and fixed environment in which tests are run so that results are
repeatable. Some people call this the test context.
Examples of fixtures:
Loading a database with a specific, known set of data
Erasing a hard disk and installing a known clean operating system installation
Copying a specific known set of files
Preparation of input data and set-up/creation of fake or mock objects
(source: wikipedia, see link above)
Here are also some practical examples from the documentation of the 'Google Test' framework.
The term fixture varies based on context, programing language or framework.
1. A known state against which a test is running
One of the most time-consuming parts of writing tests is writing the
code to set the world up in a known state and then return it to its
original state when the test is complete. This known state is called
the fixture of the test.
PHP-Unit documentation
A test fixture (also known as a test context) is the set of
preconditions or state needed to run a test. The developer should set
up a known good state before the tests, and return to the original
state after the tests.
Wikipedia (xUnit)
2. A file containing sample data
Fixtures is a fancy word for sample data. Fixtures allow you to
populate your testing database with predefined data before your tests
run. Fixtures are database independent and written in YAML. There is
one file per model.
RubyOnRails.org
3. A process that sets up a required state.
A software test fixture sets up the system for the testing process by
providing it with all the necessary code to initialize it, thereby
satisfying whatever preconditions there may be. An example could be
loading up a database with known parameters from a customer site
before running your test.
Wikipedia
I think PHP-unit tests have very good explaining of this:
One of the most time-consuming parts of writing tests is writing the
code to set the world up in a known state and then return it to its
original state when the test is complete. This known state is called
the fixture of the test.
Also Yii documents described fixtures test in a good shape:
Automated tests need to be executed many times. To ensure the testing
process is repeatable, we would like to run the tests in some known
state called fixture. For example, to test the post creation feature
in a blog application, each time when we run the tests, the tables
storing relevant data about posts (e.g. the Post table, the Comment
table) should be restored to some fixed state.
Here's a simple example of fixtures test:
<?php
use PHPUnit\Framework\TestCase;
class StackTest extends TestCase
{
protected $stack;
protected function setUp()
{
$this->stack = [];
}
protected function tearDown()
{
$this->stack = [];
}
public function testEmpty()
{
$this->assertTrue(empty($this->stack));
}
public function testPush()
{
array_push($this->stack, 'foo');
$this->assertEquals('foo', $this->stack[count($this->stack)-1]);
$this->assertFalse(empty($this->stack));
}
public function testPop()
{
array_push($this->stack, 'foo');
$this->assertEquals('foo', array_pop($this->stack));
$this->assertTrue(empty($this->stack));
}
}
?>
This PHP unit test has functions with names setUp and tearDown so that before running your tests you setup your data and once finished you can restore them to the initial state.
Exactly to that topic, JUnit has a well explained doc. Here is the link!
The related portion of the article is:
Tests need to run against the background of a known set of objects. This set of objects is called a test fixture. When you are writing tests you will often find that you spend more time writing the code to set up the fixture than you do in actually testing values.
To some extent, you can make writing the fixture code easier by paying careful attention to the constructors you write. However, a much bigger savings comes from sharing fixture code. Often, you will be able to use the same fixture for several different tests. Each case will send slightly different messages or parameters to the fixture and will check for different results.
When you have a common fixture, here is what you do:
Add a field for each part of the fixture
Annotate a method with #org.junit.Before and initialize the variables in that method
Annotate a method with #org.junit.After to release any permanent resources you allocated in setUp
For example, to write several test cases that want to work with different combinations of 12 Swiss Francs, 14 Swiss Francs, and 28 US Dollars, first create a fixture:
public class MoneyTest {
private Money f12CHF;
private Money f14CHF;
private Money f28USD;
#Before public void setUp() {
f12CHF= new Money(12, "CHF");
f14CHF= new Money(14, "CHF");
f28USD= new Money(28, "USD");
}
}
In Xamarin.UITest it is explained as following:
Typically, each Xamarin.UITest is written as a method that is referred
to as a test. The class which contains the test is known as a test
fixture. The test fixture contains either a single test or a logical
grouping of tests and is responsible for any setup to make the test
run and any cleanup that needs to be performed when the test finishes.
Each test should follow the Arrange-Act-Assert pattern:
Arrange – The test will setup conditions and initialize things so that the test can be actioned.
Act – The test will interact with the application, enter text, pushing buttons, and so on.
Assert – The test examines the results of the actions performed in the Act step to determine correctness. For example, the
application may verify that a particular error message is
displayed.
Link for original article of the above Excerpt
And within Xamarin.UITest code it looks like following:
using System;
using System.IO;
using System.Linq;
using NUnit.Framework;
using Xamarin.UITest;
using Xamarin.UITest.Queries;
namespace xamarin_stembureau_poc_tests
{
[TestFixture(Platform.Android)]
[TestFixture(Platform.iOS)]
public class TestLaunchScreen
{
IApp app;
Platform platform;
public Tests(Platform platform)
{
this.platform = platform;
}
[SetUp]
public void BeforeEachTest()
{
app = AppInitializer.StartApp(platform);
}
[Test]
public void AppLaunches()
{
app.Screenshot("First screen.");
}
[Test]
public void LaunchScreenAnimationWorks()
{
app.Screenshot("Launch screen animation works.");
}
}
}
Hope this might be helpful to someone who is in search of better understanding about Fixtures in Programming.
I'm writing this answer as quick note for myself on what is "fixture".
same-data-multiple-tests
Test Fixtures: Using the Same Data Configuration for Multiple Tests
If you find yourself writing two or more tests that operate on similar data, you can use a test fixture. This allows you to reuse the same configuration of objects for several different tests.
you can read more at googletest
fixtures can be used for during integration test or during development (lets say ui development where data is comming from development database
fake users for database or testing
myproject/fixtures/my_fake_user.json
[
{
"model": "myapp.person",
"pk": 1,
"fields": {
"first_name": "John",
"last_name": "Lennon"
}
},
{
"model": "myapp.person",
"pk": 2,
"fields": {
"first_name": "Paul",
"last_name": "McCartney"
}
}
]
you can read more from django docs