Is there a preferred approach when defining acceptance criteria in Gherkin format for BDD?
Should I split out the scenarios as follows...
Scenario: User saves contact details
Given I am on the contact details page
When I enter the following details
| email | phone |
| pete#gmail.com | 012345678 |
And I save the details
Then the details are correctly saved
Scenario: User saves contact details with a very long email address
Given I am on the contact details page
When I enter the following details
| email | phone |
| peterpeterperterlongemailaddress1234567890#gmailsomething.com | 012345678 |
And I save the details
Then the details are correctly saved
Scenario: User saves contact details with a very long phone number
etc
Or should I use a single scenario with multiple outlines?
Scenario: User saves contact details
Given I am on the contact details page
When I enter the following details
| email | phone |
| <email> | <phone> |
And I save the details
Then the details are correctly saved
Examples:
| email | phone |
| pete#gmail.com | 012345678 |
| peterpeterperterlongemailaddress1234567890#gmailsomething.com | 012345678 |
| pete#gmail.com | 012345678901234567890 |
The latter may be easier to adapt/add to but to me loses the basic concept of scenarios.
UPDATE
I have just come across the following article, which discusses this exact point. Suggesting the latter is a better approach.
https://www.hindsightsoftware.com/blog/scenario-outlines
It suggests something like the following:
Scenario: User saves contact details
Given I am on the contact details page
When I enter the following details
| email | phone |
| <email> | <phone> |
And I save the details
Then the details are correctly saved
Examples:
| scenario | email | phone |
| basic details saved | pete#gmail.com | 012345678 |
| very long email address | peterpeterperterlongemailaddress1234567890#gmailsomething.com | 012345678 |
| very long phone number | pete#gmail.com | 012345678901234567890 |
As a general matter of practice, if you vary the input, but expect the same output then you have a good case for a scenario outline.
You can parameterize the Then steps, but I advise against that, because changing the assertion of a test means you have fundamentally changed the test. It's a philosophical and code maintenance issue.
Consider the max length for a phone number, which by your example should be 11 characters. If 11 characters is the max, and you have a scenario outline that tests both sides of that boundary, then the scenario has multiple reasons to fail. First, when 11 is no longer the max length, and furthermore if 12 is now the max length, that test will fail too.
Tests with multiple reasons to fail are "flaky"
The scenario outline below varies the assertion as well as the input, which means this test has multiple reasons to fail.
Scenario Outline: User saves contact details
Given I am on the contact details page
When I enter the following details
| email | phone |
| pete#gmail.com | <Phone> |
And I save the details
Then the details <Assertion> correctly saved
Examples:
| Phone | Assertion |
| 012345678901234567890 | Are |
| 0123456789012345678901 | Are Not |
I would even advise separating the e-mail scenarios from the phone number scenarios.
Every test should only have one reason to fail.
The two scenarios below seem like duplicates of one another, but keeping the most of the input consistent, and only varying one of the inputs gives you more stable tests that fail for obvious reasons:
Scenario Outline: User saves contact phone number
Given I am on the contact details page
When I enter the following details
| email | phone |
| pete#gmail.com | <Phone> |
And I save the details
Then the details are correctly saved
Examples:
| Phone |
| 012345678 |
| 012345678901234567890 |
Scenario Outline: User saves contact e-mail address
Given I am on the contact details page
When I enter the following details
| email | phone |
| <Email> | 012345678 |
And I save the details
Then the details are correctly saved
Examples:
| Email |
| pete#gmail.com |
| peterpeterperterlongemailaddress1234567890#gmailsomething.com |
Now when a scenario fails, the scenario name gives you a hint about which part of the application may be to blame, which aids debugging.
Related
I need to verify bunch of sites that are using our templates. After login in I need to verify many info which I will put in the second table. How to write the gherkin code so that For each item in table "userinfo" it goes through All the items in table "siteinfo". I am seeing error calling them by userinfo.url or siteMenu.menuiitem. I am using python-bdd.
Scenario Outline: Login to SAP
Given User is on login page url <url>
When User enters username <username> and passwrod <password>
Then user should see <menuitem> <title> and <description>
Example: usefinfo
| url | username| password |
| siteA | Userx | pass1 |
| siteB | UserY | pass1 |
Example: siiteInfo
|menuitem | title | description |
|Community Managment | T1 | Text1 |
|User Managment | T2 | Text2 |
|Environment Management | T3 | Text3 |
|Loggin | T4 | Text4 |
|Miscellaneous | T5 | Text5 |
Use a different approach. Instead of trying to write one hugely complex low quality Cuke, write several simple high quality cukes.
Once you have a set of cukes that verify correctly that one particular site works correctly with your templates, then run that set of cukes against your other sites. You can do this by externalising site, perhaps as an environment variable, or by modifying cuke config, or by running the same set of cukes in different places.
This will be a much more sustainable solution in the medium term particularly when
you get asked to add new tests for each site
you get asked to add a site specific test.
I have a situation where I need to run the scenario outline along with all the datatable for different set of value. I am looking for an datatable inside another datatable. That I need to run my entire list of examples of a scenario outline repeatedly for the given list of products.
Note: I am trying to avoid write different scenario for each product.
I have given some example and my problem statement for better understanding as below
Scenario Outline : Check the behaviour of all the products
Given the POST retrieveProductdetails api url with valid authorization
When POST api is applied for the <"Products">
Then verify the behaviour of all the <"Properties"> and its <"result">
Examples:
|Properties |result|
|Appearance | Successful|
|reading | Successful|
|writing |Successful|
|memo |Successful|
|Singing |Successful|
|Help |Successful|
|Adancefeature |Successful|
|Antiquefeatuer |Succesful|
|AI nature |Successful|
|Interaction |Successful|
Note : I have around 20 Products to be validated and for each and every product i need validate all the 10 properties as mentioned .
If I start to write a an third Variable like as below , I will end up in writing 200 lines/examples (20 *10 = 200 ). And similar to the above scenario i have around 25 to 30 details which needed to be validate for all 20 products . The maintenance will be very difficult. Is there any better option for this ?
Examples:
|Properties |result |Products|
|Appearance | Successful |Alexa|
List of Products
|Products|
|Alexa|
|firetv|
|GoogleHome|
|Chromecast|
|SmartHub|
|SmartTV|
|AmazonVideo|
|AmazonPhoto|
|Echo|
|Echo Dot|
|Echo Show|
|Ring|
.
.
.
.
|SmartHome|
You are making this very difficult on yourself for a couple of reasons.
You are not describing the behavior of the system. I'm not sure what your application is supposed to do but it seems that you have a particular type of product and all instances of that type of product should have certain flags set.
However you didn't write this down, rather you appear to be retrieving all products of that type from a database and checking if these have the right flags set. So I have to infer the behavior of the system from your scenario. This should be the other way around.
You are trying to programming in Gherkin. Steps in Gherkin are not steps in a test script. They do not have to describe the exact operation needed to get some result. When you use Gherkin to describe the behavior of a system it shouldn't matter if you talk to the system in a unit tests, via http or a browser.
However by describing the exact operations you are painting yourself in a corner. It means that you can't effectively generalize without using programming language constructs like loops. If you step away from describing exact operations and rather try to describe what the system does you can use a much bigger vocabulary.
You appear to be testing against fixed data. Your data appears to have been put into the system already. You are merely checking if it comes out alright. This is not a good test because it assumes the system is in a particular state rather then creating the system in that state or verifying it is.
So to fix your feature file you might want to something like this:
Scenario: All smart home products are in the category of AI powered spy-devices
Given the smart home product "<Product>"
When I inspect this smart home product
Then it has all the properties of an AI powered spy-device:
| Appearance |
| reading |
| writing |
| memo |
| Singing |
| Help |
| Adancefeature |
| Antiquefeatuer |
| AI nature |
| Interaction |
Examples:
| Product |
| Alexa |
| firetv |
| GoogleHome |
| Chromecast |
| SmartHub |
| SmartTV |
| AmazonVideo |
| AmazonPhoto |
| Echo |
| Echo Dot |
| Echo Show |
| Ring |
.
.
.
.
| SmartHome |
While in the Given step you'd normally create the product, in your case you'll have to fetch the catalogue of products and verify that the catalogue contains the product. In the When step you'd probably fetch the details for the product. Finally in the Then step you'd verify if all properties have been set when looking at the details.
edit:
If you actually want to check if all the data has been entered into the system correctly you could also do something like this:
Scenario: All smart home products are in the category of AI powered spy-devices
Given the smart home product "<Product>"
When I inspect this smart home product
Then it has all the properties:
| Appearance | <Apperance> |
| reading | <Reading> |
| writing | <....> |
| memo | |
| Singing |
| Help |
| Adancefeature |
| Antiquefeatuer |
| AI nature |
| Interaction |
Examples:
| Product | Apperance | Reading | ....
| Alexa | Yes | No
| firetv | No | Yes
| GoogleHome | Yes | No
.
.
.
.
| SmartHome | No | Yes | ....
But I would suggest not using Cucumber for this. In that case you'd be better of putting your data into an excel file and using JUnit5s parameterized test.
I'm trying to write some Gherkin feature files in order to do BDD acceptance testing using SpecFlow. The system I'm trying to test consists of multiple RESTful APIs - system has a microservice architecture. In a scenario, I need to be certain that some records already exist in the database prior to going with the actual scenario, so I've included a Background section with a given part. The problem I'm having is that each of those records that need to exist are created through APIs that require lots of data in their schema contact and the team requires that I specify each and every fields and their respective values in a record in a gherkin table. The result is something like this:
| PassportExpireDate|PassportNumber|PassportCountry |Firstname|Lastname|LocalFirstname|LocalLastname | Birthday | NationalNumber | NationalityCountryId | PassengerType | Gender |PartyId | SourceTravelerId | CellNumber | Price|
This is the header of one of my tables which is going to be used to create a Traveler record in the database before starting the actual test by specification. However, as you can see this table has too much fields and therefore is too long too fit on the screen and thus very hard to read and maintain. secondly it's tightly coupled to the DTO schema. I argued that we shouldn't put this much detail on our specificatons, trying to include only vital high-level data (e.g. given we have an existing traveler named "James Peterson") but the team and the CTO insisted that these details should be present on the feature file. In my next attempt, I broke the tables into multiple tables (e.g. personal data, order data, passport data, etc.).
But I'm still confused and I think I'm still not doing the wrie thing. What's your recommendation? Do we have any rule of thumb or best practices for this?
Can you transpose the filed and values in the data table as below.
|field |values |
| PassportExpireDate |[] |
| PassportNumber |[] |
| PassportCountry |[] |
| Firstname |[] |
| Lastname |[] |
| LocalFirstname |[] |
| LocalLastname |[] |
| Birthday |[] |
| NationalNumber |[] |
| NationalityCountryId |[] |
| PassengerType |[] |
| Gender |[] |
| PartyId |[] |
| SourceTravelerId |[] |
| CellNumber |[] |
| Price |[] |
And in the step def implement the logic to get the values from the values array.
Specflow supports external data binding for such cases. You can use Excel binding to keep your feature file fit.
Scenario Outline: Add Traveler
Given ...
When ....
Then ....
#source:TravelerRecordsExamples.xlsx
Examples:
| PassportExpireDate|PassportNumber|PassportCountry |Firstname|Lastname|LocalFirstname|LocalLastname | Birthday | NationalNumber | NationalityCountryId | PassengerType | Gender |PartyId | SourceTravelerId | CellNumber | Price|
TLDR None
Don't put definitions and data in a Gherkin tables, its incredibly counter productive and error prone. Instead use something else to specify the fields (ideally the source code of the api) and name each thing.
Then use simple Givens to create you things.
Now in your case you seem to be creating travelers. The behavior your Gherkin is documenting is the creation of travelers. HOW travelers are created and what their characteristics are have no place in this description of the behavior.
So your background steps become something like
Given there are foo travelers
an the implementation is something like
Given 'there are foo travelers' do
create_foo_travelers
end
and now you have translated you problem from
from
How do I do something incredibly stupid and difficult in Gherkin
to
How do I write some code to create travelers
This is the approach you should take to writing all scenarios when Cuking. The scenario should only document the WHAT and WHY of the behavior. Any details about HOW the behavior is implemented have no place in the scenario.
The true power of cuking is using natural language, naming and abstraction to make you cukes simple. Use these skills to delegate the complexity of HOW to more appropriate tools.
I have some documents to upload it. For that, I wrote a scenario. I need to run the same feature file multiple times. How can I run the feature file Multiple times?
Instead of using multiple feature file use scenario outline which will execute the same scenario multiple time as per the example table you will provide to it
Example:
Scenario Outline: Create ABC
Given I open the application
When I enter username as <username>
And I enter password as <password>
Then I enter title as <title>
And press submit
Examples:
| username | password | title |
| Rob | xyz1 | title1 |
| Bob | xyz1 | title2 |
So here the same scenario will run 2 times as example table have 2 data, you can add as many data table and cucumber will execute that number of times
i am new to cucumber frame work just i want know about how to use
As i defining feature as below it was not working.i just want to know where is the error and can i use two scenario outline in the same feature file
Scenario Outline: Checkout descriptions
Given I am logged in as "<user>"
And I have gone to the checkout after selecting various products
When I purchase the items
Then I should be able to checkout <with?> a description
Examples:
| user | with? |
| Arun | with |
| Ajay | with |
| Ashok | with |
| Arun | without |
| Ajay | without |
| Ashok | without |
Scenario Outline: Checkout with description and accept the terms of service*
Given I am logged in as "<user>"
And I have gone to the checkout after selecting various products
When I purchase the items
Then I should be able to checkout with a description
And I should be able to accept the terms of service*
Examples:
| user |
| Arun |
| Ajay |
| Ashok |
If that's your entire feature file, then you're missing one line from the top:
Feature: Checkout Descriptions
Scenario Outline: Checkout Descriptions
...
Cucumber cannot parse a feature file with no Feature / Business Need / Ability in the top line.