I want to change the scenario status for known issues into the After hook.
Something like:
#After
public void afterScenario(Scenario scenario) {
if(scenario.isFailed() && scenario.getSourceTagNames().contains("knownIssue")){
//scenario.add(Result.SKIPPED)
}
}
The idea is tests, which fail because of known bug to be skipped into the test report.
Thanks,
Nayden
You can annotate the scenario with #KnownIssue and then run cucumber with --tags "not #KnownIssue" or its #CucumberOptions equivalent.
it is your test execution engine that is responsible for this aspect, aka TestNG, JUnit.
So the question is: How to change Test Execution Status Programmatically? Here is one explained. You can hook in to the Test Execution Engine either by the way explained in the artcicle or from your method - what is important - you have to work with the Test Execution Engine and not Cucumber.
Gherkin with qaf supports what you are looking for. With qaf
you can achieve it by test ng after method invocation listener.
In addition you also can add meta-data at step level and handle it in onFailure method of step listener to modify exception to Skip exception depending on step meta data. Which automatically skips tests where that step is called and failed.
Related
I want to have two different scenarios in the same feature.
The thing is that Scenario 1 needs to be executed before Scenario 2. I have seen that this can be achieved through cucumber Hooks but when digging in the explanations, there's no concrete cucumber implementation in the examples I have found.
How can I get Scenario 1 executed before Scenario 2?
The feature file is like this:
#Events #InsertExhPlan #DelExhPln
Feature: Insert an Exh Plan and then delete it
#InsertExhPlan
Scenario: Add a new ExhPlan
Given I login as admin
And I go to automated test
When I go to ExhPlan section
And Insert a new exh plan
Then The exh plan is listed
#DeleteExhPlan
Scenario: Delete an Exh Plan
Given I login as admin
And Open the automatized tests edition
When I go to the exh plan section
And The new exh plan is deleted
Then The new exhibitor plan is deleted
The Hooks file is:
package com.barrabes.utilities;
import cucumber.api.java.After;
import cucumber.api.java.Before;
import static com.aura.steps.rest.ParentRestStep.logger;
public class Hooks {
#Before(order=1)
public void beforeScenario(){
logger.info("================This will run before every Scenario================");
}
#Before(order=0)
public void beforeScenarioStart(){
logger.info("-----------------Start of Scenario-----------------");
}
#After(order=0)
public void afterScenarioFinish(){
logger.info("-----------------End of Scenario-----------------");
}
#After(order=1)
public void afterScenario(){
logger.info("================This will run after every Scenario================");
}
}
The order is now as it should be but I don't see how does the Hooks file control exection order.
You don't use Hooks for that purpose. Hooks are used for code that you need to run before and/or after tests, and/or before and/of after test suites; not to control the order of features and/or scenarios.
Cucumber scenarios are executed top to bottom. For the example you showed there, Scenario: Add a new ExhPlan will execute before Scenario: Delete an Exh Plan if you pass the tag #Events in the test runner. Also, you should not have the scenario tags at the feature level. So, you should remove #InsertExhPlan and #DelExhPln at the Feature level. Alternatively, you could pass a comma-separated list of scenario tags to the test runner in the order you want. For example, if you need to run scenario 2 before scenario 1, you would pass the tags for the corresponding scenarios in the order you wish them to be executed. Moreover, you can do this as well from your CI environment. For example, you can have Jenkins jobs that execute the tasks in a specific order by passing the scenario tags in that order. And, if you wish to be run in the default order, simply you can pass the feature tag.
About Hooks, this should be for code that needs to be run for all features and scenarios. For specific stuff you need to run for a particular feature, you need to use Background in the Cucumber file. Background block is run before each scenario in a given feature file.
I have three different feature files which hold different test scenarios: configA.feature, configB.feature, common.feature.
In the steps file I have two tagged before hooks, one for each config (A/B):
#Before("#ConfigA")
public void configA() {
//some settings
}
#Before("#ConfigB")
public void configB() {
//some settings
}
These two configs are mutually exclusive, so only one should be executed for any particular scenario execution as the second would overwrite setting done by the first.
What I want to achieve is to be able to run scenarios as follows:
configA.feature with the ConfigA hook executed
configB.feature with the ConfigB hook executed
common.feature with the ConfigA hook executed
common.feature with the ConfigB hook executed
I've tried annotating the features in the feature files as:
configA with #ConfigA
configB with #ConfigB
common with both #ConfigA #ConfigB
but this results in common.feature being always executed with both of the before hooks at the same time.
As I'm using a JUnit wrapper with Cucumber runner I've also tried creating separate test classes with #CucumberOptions.tags specified, but this didn't work for me either.
Is, what I'm trying to do, even possible with cucumber? If so then how can I achieve this?
I am using Serenity with BDD and need to perform a teardown step that must get executed after completion of each Scenario. Also, this teardown step should not be visible to report as it is technical thing and nothing to do with behavior to be exposed as part of cucumber such as releasing few expensive resource that got
I used cucumber's #After annotation which is working as expected, but the problem is now this step is also shown in my Report which I don't want to be visible.
Could someone please suggest me a solution that allows me to perform teardown step that gets executed per scenario but should not be added as step in my Serenity Report.
Current Solution I have is which does not satisfy my need:
Step Definition Class has following method:
#After
public void tearDown() {
systemAction.deleteCostlyResource(id);
}
but #After annotation makes it a candidate for Reporting Step.
If you are using Dependency Injection, you could have your DI framework teardown the resources at the end of the scenarios?
For instance, if you are using Spring:
If the "costly resource" is a class that you yourself have created, mark it with:
#Component
#Scope("cucumber-glue")
If the "costly resource" is not a class you created, but provided by a framework or whatever, you can register it as a bean in your spring (test)configuration and mark it with a "destroy method".
For example, to register Selenium WebDriver using annotation based configuration and making sure to quit after each Scenario, mark it with:
#Bean(destroyMethod = "quit")
In this example, quit() is WebDriver's method to quit(). In your situation, call "costly resource's" quit method, or equivalent thereof.
How can you get a jenkins groovy script to produce a junit xml results file? I'm doing this purely for the purpose of generating junit results with a specific number of passed/failed and skipped test cases. I need this so that I have a set of test data to test against for another application. This other app goes out to various jenkins jobs and analyzes the junit results from the job's json output. I want to point my functional tests at this jenkins job for testing. (I can't use my real continuous integration jobs because that wouldn't be deterministic).
I've got a basic groovy test case like what's below. It runs but doesn't produce junit output. I didn't expect it to, but I'm also not sure how to get it to generate one.
class BunchOfTests extends GroovyTestCase {
void testOne(){}
void testTwo(){fail()}
}
I also played around with writing code that prints the junit results xml but it's getting lengthy and quite ugly. I've seen the threads on here about what the junit results xsd looks like but I'm thinking there's got to be an easier route to generating some results without needing a pre-made results file. 10 results or so ought to be enough for what I need.
Generally unit testcases in groovy has to be written as below
import groovy.util.GroovyTestCase
class sampleTest extends GroovyTestCase {
assertEquals(true, val);
}
Incase you want only Junit reporting in groovy.
How would I produce JUnit test report for groovy tests, suitable for consumption by Jenkins/Hudson?
I am using spockframework and geb for test automation. I would like to execute after every feature a simple check to be sure that no error dialogs are shown, I have added the following cleanup() method:
def cleanup() {
expect:
$('.myErrrorDialogClass').isEmpty()
}
The code is executed after every feature but it does not throw any error when the dialog is shown.
Spock uses AST transforms to wire in the functionality for each test label (when, expect, etc); they may not run the transformations on the cleanup method. They are either not expecting or not encouraging assertions in cleanup, so that code may run but not actually assert anything.
You can get around this by using a standard Groovy assert call without the expect block.
Summarized from our comment discussion above - in case you want to accept it as an answer ;-)