I want to add pre-steps to Scenario Outline as Scenario, but I don't want to add pre-condition. How can I do that in Jira Xray?
Some illustrative example:
Scenario:
Given: Open website
Then: Check URL
Scenario Outline:
When I click this <button>
Then Something happens with <this> element
I want to have some "background" for Scenario Outline because this is something that repeats in several tests but it turns out that for every step in S.Outline Background is repeating so I want to create a normal Scenario as "artificial" background. How to do that in Xray? Can I add something like pre-step? Pre-condition always creates "Background" in Gherkin.
I don't want to have several Scenario Outline scenarios with a lot of repeated steps.
This is more a Gherkin related question than Xray specific. There are no dependencies between Scenarios in Gherkin.
The way to have some initial Gherkin statements that apply to all to several Scenarios, is by having a Background in the corresponding .feature file.
In Xray this means that you need to have a Precondition, of Cucumber type, with those Gherkin statements. Then you need to associate it all Tests (Scenario/Scenario Outline types) that you want. Whenever exporting those tests, a .feature file will be created with a Background corresponding to the previous Precondition.
The rules for generating .feature files based on Test and Preconditions issues in Xray is detailed here.
Note: in Cucumber libraries there also hooks, that contain code that can be executed before/after tests. However, this is used for something that isn't described as behaviour and more for easing implementation, and doing things like test setup/cleanup.
Related
I want to have an option on the cucumber report to mute/hide scenarios with a given tag from the results and numbers.
We have a bamboo build that runs our karate repository of features and scenarios. At the end it produces nice cucumber html reports. On the "overview-features.html" I would like to have an option added to the top right, which includes "Features", "Tags", "Steps" and "Failures", that says "Excluded Fails" or something like that. That when clicked provides the same exact information that the overview-features.html does, except that any scenario that's tagged with a special tag, for example #bug=abc-12345, is removed from the report and excluded from the numbers.
Why I need this. We have some existing scenarios that fail. They fail due to defects in our own software, that might not get fixed for 6 months to a year. We've tagged them with a specified tag, "#bug=abc-12345". I want them muted/excluded from the cucumber report that's produced at the end of the bamboo build for karate so I can quickly look at the number of passed features/scenarios and see if it's 100% or not. If it is, great that build is good. If not, I need to look into it further as we appear to have some regression. Without these scenarios that are expected to fail, and continue to fail until they're resolved, it is very tedious and time consuming to go through all the individual feature file reports and look at the failing scenarios and then look into why. I don't want them removed completely as when they start to pass I need to know so I can go back and remove the tag from the scenario.
Any ideas on how to accomplish this?
Karate 1.0 has overhauled the reporting system with the following key changes.
after the Runner completes you can massage the results and even re-try some tests
you can inject a custom HTML report renderer
This will require you to get into the details (some of this is not documented yet) and write some Java code. If that is not an option, you have to consider that what you are asking for is not supported by Karate.
If you are willing to go down that path, here are the links you need to get started.
a) Example of how to "post process" result-data before rendering a report: RetryTest.java and also see https://stackoverflow.com/a/67971681/143475
b) The code responsible for "pluggable" reports, where you can implement a new SuiteReports in theory. And in the Runner, there is a suiteReports() method you can call to provide your implementation.
Also note that there is an experimental "doc" keyword, by which you can inject custom HTML into a test-report: https://twitter.com/getkarate/status/1338892932691070976
Also see: https://twitter.com/KarateDSL/status/1427638609578967047
I want to have an option on the cucumber report to mute/hide scenarios with a given tag from the results and numbers.
We have a bamboo build that runs our karate repository of features and scenarios. At the end it produces nice cucumber html reports. On the "overview-features.html" I would like to have an option added to the top right, which includes "Features", "Tags", "Steps" and "Failures", that says "Excluded Fails" or something like that. That when clicked provides the same exact information that the overview-features.html does, except that any scenario that's tagged with a special tag, for example #bug=abc-12345, is removed from the report and excluded from the numbers.
Why I need this. We have some existing scenarios that fail. They fail due to defects in our own software, that might not get fixed for 6 months to a year. We've tagged them with a specified tag, "#bug=abc-12345". I want them muted/excluded from the cucumber report that's produced at the end of the bamboo build for karate so I can quickly look at the number of passed features/scenarios and see if it's 100% or not. If it is, great that build is good. If not, I need to look into it further as we appear to have some regression. Without these scenarios that are expected to fail, and continue to fail until they're resolved, it is very tedious and time consuming to go through all the individual feature file reports and look at the failing scenarios and then look into why. I don't want them removed completely as when they start to pass I need to know so I can go back and remove the tag from the scenario.
Any ideas on how to accomplish this?
Karate 1.0 has overhauled the reporting system with the following key changes.
after the Runner completes you can massage the results and even re-try some tests
you can inject a custom HTML report renderer
This will require you to get into the details (some of this is not documented yet) and write some Java code. If that is not an option, you have to consider that what you are asking for is not supported by Karate.
If you are willing to go down that path, here are the links you need to get started.
a) Example of how to "post process" result-data before rendering a report: RetryTest.java and also see https://stackoverflow.com/a/67971681/143475
b) The code responsible for "pluggable" reports, where you can implement a new SuiteReports in theory. And in the Runner, there is a suiteReports() method you can call to provide your implementation.
Also note that there is an experimental "doc" keyword, by which you can inject custom HTML into a test-report: https://twitter.com/getkarate/status/1338892932691070976
Also see: https://twitter.com/KarateDSL/status/1427638609578967047
I am doing BDD test on an app with cucumber, and I want to have clear instruction as it is recommanded in cucumber doc. The thing is that we have to do reusable step definitions so the maintenance cost is acceptable.
Example of scenario we have
Given I am on project page
When I click on 'buttonAddProject' //not easily readable
And I click on 'switchProjectPrivate'
And I click on 'buttonDeleteProject'
etc..
I don't want to have a function for each step like that: I change projet visibily or I delete project,
because this is basically just a click on a button, and we are going to have hundred of function like this. I also can't change the param in key to something more suitable, because every button key should be unique to avoid ambiguity.
So is there a way to do this with cucumber ?:
Given I am on project page
When I click on 'Add' //easily readable
And I click on 'Private'
And I click on 'Delete'
Bindings: //this keyword doesn't exist
'Add' : 'buttonAddProject'
'Private': 'switchProjectPrivate'
'Delete':'buttonDeleteProject'
I have tried that:
Scenario Outline:
Given I am on project page
When I click on <Add> //easily readable
And I click on <Private>
And I click on <Delete>
Examples:
|Add |Private |Delete |
|'buttonAddProject'|switchProjectPrivate'|'buttonDeleteProject'|
it works... but I need to do this for every scenario in the file, and if I really want to use scenario outline to iterate several times, I would have to copy paste this for every line, not really what I want.
How to organize this tests to make them more readable without making things to complex ?
First of all Cucumber scenarios that show HOW each thing is done are not maintainable or particularly useful.
What are cucumber scenario should describe and document is WHAT you are doing. To do this you need to determine WHY you are clicking on these buttons and what is achieved by these actions.
Now I have no idea from your scenarios about WHAT you are adding, WHY it is private or WHY you are then deleting it. But I can speculate from your post. The scenarios you should be writing should be something like.
Scenario: Delete a project
Given there is an existing project
And I am viewing the project
When I delete the project
Then ...
Scenario: Create a project
When I create a project
Then a project should be created
When you write your scenarios in this manner you push the details of how you interact with your UI down into your step definitions. So you might have something like
When 'I create a project' do
visit project_page
click "Create Project"
end
or better just
When 'I create a project' do
# must be on project page
click "Create Project"
When you work this way step definition re-use becomes less relevant and valuable. Each step does more and does something more specific.
You can continue this pattern of pushing the HOW down by having step definitions make calls to helper methods. This is particularly useful when dealing with Given's which get alot of re-use. Lets explore this with Given there is an existing project
Given 'there is an existing project' do
#project = create_project
end
Here we are pushing how we create an existing project down into the helper method create_project. The crude way to this would be to go through your UI visiting the project page and adding a new project. However this is really slow. You can optimise this process by bypassing your UI.
The most important point, whatever you decide to do, is that you are taking HOW you do something out of Cucumber and into some underlying code so now Cucumber is only interested in WHAT you are doing and WHY its important.
Making this change is probably the single most important thing you can do when Cuking. If you keep the HOW in your cucumber scenarios and step definitions you will end with a large number of brittle step definitions and very large scenarios that break all the time because everything is coupled together. You will get lots of bugs where making a change to get one step definition working causes lots of other scenarios to break. You will get lots of bugs where small changes to how you do a particular thing cause lots of unrelated scenarios to break.
Finally you are not doing BDD if you are writing the test after the code has been written. You can only do BDD if you write your scenarios collaboratively before the code is written.
Each step must be tied to a step definition. If you like to reuse an existing step def, you can just pass the command as argument (" Add", "Private","Delete"). You will have to use both the scenario name and the corresponding command to perform the required action.It will be something like this,
Scenario: scenario1_deleteproject
Given I am on project page
When I click on 'Add'
And I click on 'Private'
And I click on 'Delete'
Scenario: scenario2_createproject
Given I am on project page
When I click on 'Add'
And I click on 'Private'
And I click on 'Delete'
The step definition:
#When("When I click on {string}")
public void I_Click_On_Something(String command)
{
Switch(Command)
{
case Add:
//perform steps here
case delete:
//perform steps here
default:
}
If you want to differentiate the commands between the scenario, you will have to use scenario name ( need a class with definitions of scenario & command). You can grab the scenario name #Before hook.
Is it possible to add more than one tag to a cucumber scenario?
For example, I have a test which I need to include in smoke test as well as regression tests. When I add 2 tags to the scenario, it is getting skipped
Yes, you can add multiple tags. For more information, see the Cucumber documentation
I have to execute certain Examples from Scenario Outlines.
Let's see the following example:
Feature: Temp
Scenario Outline: Test.Something.On.<environment>
When action is performed on "<environment>"
Examples:
|environment|
|lab |
|prod |
I would like to execute only the example with lab from the upper presented Examples.
I tried the followings:
Filter by line number: mvn clean test -Dcucumber.options="src/test/resources/features/Temp.feature:8" - this way only the lab scenario was executed, however this is not a long term solution;
Filter by name: mvn clean test -Dcucumber.options="name lab" - it works for Scenario but not for Scenario Outline even if the name of the scenario will be Test.Something.On.lab;
decompose the Scenario Outline in Scenarios and tag the proper scenarios with #lab and #prod - I hate code duplication and the subsequent maintenance, therefore I hope there is an another solution for this.
Any suggestions?
Big thanks.
Another sub-keywords: maven, java, cucumber-java8, cucumber-junit, junit
Use two examples in yoour scenario outline and tag each example in a similar way as you have done in your third option. That should give you the behaviour you are asking for.