Running tests in parallel using TestNG and gradle - groovy

I am trying to run my TestNG tests in parallel but they seem to just be running single-threaded. I am trying to run them using IntelliJ 14.1.4 Community Edition with the default built in gradle wrapper and Java 1.8.0_45.
I've also tried using standalone gradle-2.5.
The test section of my current build.gradle file looks like:
test {
systemProperties System.getProperties()
useTestNG() {
parallel 'tests'
threadCount 3
}
}
I've also tried:
test {
systemProperties System.getProperties()
useTestNG {
options {
parallel = 'tests'
threadCount = 3
}
}
}
and:
test {
systemProperties System.getProperties()
useTestNG {
options ->
options.parallel = 'tests'
options.threadCount = 3
}
}

I needed to use 'methods' instead of 'tests' because I was only running one test class, using -Dtest.single=TestClassName, expecting all the #Test methods inside that to be run in parallel.
The relevant documentation:
parallel="methods": TestNG will run all your test methods in separate threads. Dependent methods will also run in separate threads but they will respect the order that you specified.
parallel="tests": TestNG will run all the methods in the same <test> tag in the same thread, but each <test> tag will be in a separate thread. This allows you to group all your classes that are not thread safe in the same <test> and guarantee they will all run in the same thread while taking advantage of TestNG using as many threads as possible to run your tests.
From: http://testng.org/doc/documentation-main.html#parallel-tests

It depends on you testng.xml you have made.
Refer below link:-
http://howtodoinjava.com/2014/12/02/testng-executing-parallel-tests/
http://testng.org/doc/documentation-main.html#parallel-tests

Related

Using CLI to run feature files in parallel but scenarios within same thread?

I have read the documentation page on Parallel Execution and for the CLI the only option seems to be to use the --threads <COUNT> flag to increase parallelism. However, this will cause each scenario to be executed in parallel.
Is there a way I can indicate I want each feature to be executed in parallel but each scenario within that feature to be on the same thread (executed sequentially)?
I've seen it is possible to do that when using JUnit and Maven, but I am using JUnit and Gradle and that does not seem to be an option.
I think you may want to look at Courgette-JVM
It is an extension of cucumber-jvm and it can be run with gradle. It supports junit and testng and parallel testing using threads
You may have to alter your testrunner class and add #CourgetteOptions and include #CucumberOptions inside it like
import courgette.api.CourgetteRunLevel;
import courgette.api.testng.TestNGCourgette;
import cucumber.api.CucumberOptions;
import org.testng.annotations.Test;
#Test
#CourgetteOptions(
threads = 2,
runLevel = CourgetteRunLevel.FEATURE,
rerunFailedScenarios = true,
showTestOutput = true,
reportTargetDir = "build",
cucumberOptions = #CucumberOptions(
features = "src/test/resources/features",
glue = {"utils.hooks", "steps"},
tags = {"#Web"},
plugin = {
"pretty",
"json:build/cucumber-report/cucumber.json",
"html:build/cucumber-report/cucumber.html"},
strict = true
))
public class TestRunner extends TestNGCourgette {
}
If you would like to see Courgette-JVM in an example , have a look at this parallel test execution example using gradle

Running single cucumber test multiple times with different before steps

I have three different feature files which hold different test scenarios: configA.feature, configB.feature, common.feature.
In the steps file I have two tagged before hooks, one for each config (A/B):
#Before("#ConfigA")
public void configA() {
//some settings
}
#Before("#ConfigB")
public void configB() {
//some settings
}
These two configs are mutually exclusive, so only one should be executed for any particular scenario execution as the second would overwrite setting done by the first.
What I want to achieve is to be able to run scenarios as follows:
configA.feature with the ConfigA hook executed
configB.feature with the ConfigB hook executed
common.feature with the ConfigA hook executed
common.feature with the ConfigB hook executed
I've tried annotating the features in the feature files as:
configA with #ConfigA
configB with #ConfigB
common with both #ConfigA #ConfigB
but this results in common.feature being always executed with both of the before hooks at the same time.
As I'm using a JUnit wrapper with Cucumber runner I've also tried creating separate test classes with #CucumberOptions.tags specified, but this didn't work for me either.
Is, what I'm trying to do, even possible with cucumber? If so then how can I achieve this?

SparkSession doesn't shutdown properly between unit tests

I have a few unit tests that need to have their own sparkSession. I extended SQLTestUtils, and am overriding the beforeAll and afterAll functions that are used in many other Spark Unit tests (from the source). I have a few test suites that look something like this:
class MyTestSuite extends QueryTest with SQLTestUtils {
protected var spark: SparkSession = null
override def beforeAll(): Unit = {
super.beforeAll()
spark = // initialize sparkSession...
}
override def afterAll(): Unit = {
try {
spark.stop()
spark = null
} finally {
super.afterAll()
}
}
// ... my tests ...
}
If I run one of these, it's fine, but if I run two or more, I get this error:
Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/jenkins/workspace/Query/apache-spark/sql/hive-thriftserver-cat-server/metastore_db.
But I thought that the afterAll() was supposed to properly shut spark down so that I could create a new one. Is this not right? How do I accomplish this?
One way to do it this is to disable parallel test execution for your Spark app project to make sure only one instance of Spark Session object is active at the time. In sbt syntax it would like this:
project.in(file("your_spark_app"))
.settings(parallelExecution in Test := false)
The downside is that this is a per project setting and it would also affect the tests that would benefit from parallelization. A workaround would be to create a separate project for Spark tests.

Gradle task definition inheritance

Is it possible to inherit one task definition from another? What I want to do is create some test profiles, so I'd have default test -
test {
include 'com/something/something/**'
exclude 'com/something/else/**'
maxParallelForks 5
testLogging{
exceptionFormat "full"
showStackTraces = false
}
jvmArgs '-Xms128m', '-Xmx512m', '-XX:MaxPermSize=128m'
}
and some another test with overriden "include" or "maxParallelForks" part etc.
Is it possible without creating new Task class?
You could configure all those tasks in one go, provided they're of the same type using the following construct:
tasks.withType(Test) {
include 'com/something/something/**
...
}
This configures all the tasks of type "Test" in one go. After that you can override the configurations.
or if you don't want to setup all the tasks, or some of them have a different type, you can enumerate them as in the following snippet.
["test","anotherTestTask"].each { name ->
task "$name" {
include ...
}
}
Remember, you have the full scripting power of Groovy, so there are a lot of options here...

Using command line arguments or system properties from a Parameterized Junit test?

I use this method to setup my parameterized data:
#Parameterized.Parameters
public static Collection<Object[]> getStories() {
Collection<Object[]> allStories = new ArrayList<Object[]>()
new File(SPEC_DIRECTORY).eachFileRecurse(FileType.FILES) { file ->
if (file.getName().endsWith('.story')) {
Object[] args = [file.getName(), file]
allStories << args
}
}
return allStories
}
I invoke the main test with gradle via gradle test, but I do not seem to be able to see system properties.
If I use the invocation gradle -Dfile=test.story, then System.getProperty('file') is undefined. How can I pass arguments to this parameterized data builder?
Gradle runs all tests in a separate JVM. You have to configure the test task:
test {
systemProperty "file", "test.story"
}
For more information, see the Gradle DSL reference.
In the interest of any beginners out there(like myself) who stumble upon this question; I'd like to explicitly mention that the systemProperty in gradle test task simply passes on system properties to the test jvm. So if you simply use
test{
systemProperty 'Chrome', 'Browser' }
You won't be able to access the property in your junit tests via ' System.getProperty("Browser") ' . Instead it needs to be coupled with the command gradle test -DBrowser=Chrome
The above case is merely an illustration and it would probably make sense to use -
test{
systemProperty 'Chrome', System.getProperty('Browser') }
along with
gradle test -DBrowser=Chrome

Resources