I have a large suite of feature files, and every single scenario is tagged #regression.
After running full regression I realized that some features do not need to be run for the current environment.
What is the best approach to ignore specific scenarios keeping in mind that each scenario is tagged with #regression?
You can use Tags to run certain features/scenarios, or not run them.
To specifically ignore them, see Ignoring a subset of scenarios:
"You can tell Cucumber to ignore scenarios with a particular tag:
Using JUnit runner class:
#CucumberOptions(tags = "not #smoke")
public class RunCucumberTest {}
"
Related
I have recently upgraded to version 1.0.0 from 0.9.6 and noticed that the generated karate-summary.html file, it doesn't display all the tested feature files in the JUnit 5 Runner unlike in 0.9.6.
What it displays instead was the last tested feature file only.
The below screenshots are from the provided SampleTest.java sample code (excluding other Tests for simplicity).
package karate;
import com.intuit.karate.junit5.Karate;
class SampleTest {
#Karate.Test
Karate testSample() {
return Karate.run("sample").relativeTo(getClass());
}
#Karate.Test
Karate testTags() {
return Karate.run("tags").relativeTo(getClass());
}
}
This is from Version 0.9.6.
And this one is from Version 1.0.0
However, when running the test below in 1.0.0, all the features are displayed in the summary correctly.
#Karate.Test
Karate testAll() {
return Karate.run().relativeTo(getClass());
}
Would anyone be kind to confirm if they are getting the similar result? It would be very much appreciated.
What it displays instead was the last tested feature file only.
This is because for each time you run a JUnit method, the reports directory is backed up by default. Look for other directories called target/karate-reports-<timestamp> and you may find your reports there. So maybe what is happening is that you have multiple JUnit tests that are all running, so you see this behavior. You may be able to over-ride this behavior by calling the method: .backupReportDir(false) on the builder. But I think it may not still work - because the JUnit runner has changed a little bit. It is designed to run one method at a time, when you are in local / dev-mode.
So the JUnit runner is just a convenience. You should use the Runner class / builder for CI execution, and when you want to run multiple tests and see them in one report: https://stackoverflow.com/a/65578167/143475
Here is an example: ExamplesTest.java
But in case there is a bug in the JUnit runner (which is quite possible) please follow the process and help the project developers replicate and then fix the issue to release as soon as possible.
I am studying Appium with cucumber
And I have different cucumber scenarios, for example:
login.feature with LoginSteps.java and Login.java to do actions with the app
I need to add more scenarios and I want to make different files like help.feature, payment.feature, etc.
All these scenarios need to do a login before, so, they need to get LoginSteps before to do HelpSteps
LoginSteps -> HelpSteps
LoginSteps -> PaymentSteps
...
What is the best way or the best architecture to build this kind of project?
Any reference is welcome.
I have a cucumber feature file 'A' that serves as setting up environment (data clean up and initialization). I want to have it executed before all other feature files can run.
It's it kind of like #before hook as in http://zsoltfabok.com/blog/2012/09/cucumber-jvm-hooks/. However, that does not work because my feature files 'A' contains hundreds of cucumber steps and it is not as simple as:
#Before
public void beforeScenario() {
tomcat.start();
tomcat.deploy("munger");
browser = new FirefoxDriver();
}
instead it's better to be able to run 'A' as a feature file as a whole.
I've searched around but did not find a answer. I am so surprised that no one has this type of requirement before.
The closest i found is 'background'. But that means i can have only one huge feature file with the content of 'A' as 'background' at the top, and rest of my test in the same file. I really do not want to do that.
Any suggestions?
By default, Cucumber features are run single thread in order by:
Alphabetically by feature file directory
Alphabetically by feature file name within directory
Scenario execution is then by order within the feature file.
So have your initialization feature in the first directory (alhpabetically) with a file name that sorts first (alphabetically) in that directory.
That being said it is generally a bad practice to require an execution order in your feature files. We run our feature files in parallel so order is meaningless. For Jenkins or TeamCity you could add a build step that executes the one feature file followed by a second build step that executes the rest of your feature files.
I have also a project, where we have a single feature file, that contains a very long scenario called Scenario: Test data with a lot of very long scenarios, like this:
Given the system knows about the following employees
|uuid|user-key|name|nickname|
|1|0101140000|Anna|annie|
... hundreds of lines like this follow ...
We see this long SystemKnows scenarios as quite valuable, so that our testers, Product Owner and developers have a baseline of what data are in the system. Our domain is quite complex, and we need this baseline of reference data for everyone to be able to understand the tests.
(These reference data become almost like well known personas, and are a shared team metaphore)
In the beginning, we were relying on the alphabetic naming convention, to have the AAA.feature to be run first.
Later, we discovered that this setup was brittle, and decided to use the following trick, inspired by the PageObject pattern:
Add a background with the single line Given(~'^I set test data for all feature files$')
In the step definition, have a factory to create the test data, and make sure inside the factore method, that it is only created once, like testFactory.createTestData()
In this way, you have both the convenience of expressing reference setup as a scenario, that enhances team communication, but you also have a stable test setup.
Hope this is helpful!
Agata
Is it possible to parameterise a feature file in the same way it is a scenario? So each scenario in the feature could refer to some variables which are later defined by a single table for the entire feature file?
All of the answers I've found so far (Feature and scenario outline name in cucumber before hook for example) use Ruby meta-programming, which doesn't inspire much hope for the jvm setup I'm using.
No its not, and for good reason. Feature files are meant to be simple and readable, they are not for programming. Even using scenario outlines and tables is generally not a good thing, so taking this further and having a feature that cannot be understood without reading some other thing that defines variables is counter productive.
You can however put all your variables and stuff in step definitions and write your feature at a higher level of abstraction. You'll find implementing this much easier, as you can use a programming language (which is good at this stuff).
One way of parameterising a feature file is to generate it from a template at compile-time. Then at runtime your cucumber runner executes the generated feature file.
This is fairly easy to do if you are using gradle. Here is an example:
In build.gradle, add groovy code like this:
import groovy.text.GStringTemplateEngine
task generateFeatureFiles {
doFirst {
File featuresDir = new File(sourceSets.main.output.resourcesDir, "features")
File templateFile = new File(featuresDir, "myFeature.template")
def(String bestDay, String currentDay) = ["Friday", "Sunday"]
File featureFile = new File(featuresDir, "${bestDay}-${currentDay}.feature")
Map bindings = [bestDay: bestDay, currentDay: currentDay]
String featureText = new GStringTemplateEngine().createTemplate(templateFile).make(bindings)
featureFile.text = featureText
}
}
processResources.finalizedBy(generateFeatureFiles)
myFeature.template is in the src/main/resources/features directory and might look like this:
Feature: Is it $bestDay yet?
Everybody wants to know when it's $bestDay
Scenario: $currentDay isn't $bestDay
Given today is $currentDay
When I ask whether it's $bestDay yet
Then I should be told "Nope"
Running the build task will create a Friday-Sunday.feature file in build/src/main/resources with the bestDay and currentDay parameters filled in.
The generateFeatureFiles custom task runs immediately after the processResources task. The generated feature file can then be executed by the cucumber runner.
You could generate any number of feature files from the feature template file. The code could read in parameters from a config file in your resources directory for example.
I'm attempting to run tagged features in the order that they are submitted.
example:
I have tests that i'd like to run in a specific order (#test1, #test2, #test3). After looking at the cucumber documentation is looks like i'm only able to run them in an and/or option like
cucumber features/.feature --t #test1; cucumber features/.feature --t #test2; cucumber features/*.feature --t #test3;
but this prevents me from having a single report which contains all of the results.
Is there anyway which I can run these tests in their respective order and have all of the results contained in the same report?
If you put the tests that have to run in a specific order in a feature file together cucumber will run them in the order they are given. As this will be in your normal test run it should all show up in the same report.
But it might be worth looking into why your tests are dependant on each other and if there is a way to remove this dependancy as it is generally bad practice to have it.