I've got to test a message-routing application, whose functionality is broadly as follows:
- message gets sent to app
- app examines message
- message gets forwarded somewhere, based on the content of the message
The vast majority of test cases are near-identical; generate a particular type of message, load it into the system, wait a few seconds, then check the required destination to ensure that the message was forwarded correctly.
Rather than generate 100s of near-identical test cases in Cucumber, is there a recommended way to generate the one test case, and have it repeatedly cycle through all the (message, required_destination) tuples? I'd prefer to have these tuples maintained into a YAML file rather than a database, for ease of maintenance, but either solution would be fine.
You should try Scenario Outline using Examples
Normal Scenario
Scenario: Login
Given I am on login page
When I enter username "Jonas" and password "secrect" and press login
Then I get redirected to "Jonas Home Page"
Scenario Outline
Scenario Outline: Login
Given I am on login page
When I enter username <username> and password <password> and press login
Then I get redirected to <redirect_page>
Examples:
| username | password | redirect_page |
| "Jonas" | "secret" | "Jonas Home Page" |
| "Anna" | "Data" | "Annas Home Page" |
Read More:
https://github.com/cucumber/cucumber/wiki/Scenario-Outlines
I dont have this kind of situation in cucumber, but I do use pipe separated files in specs for massive data testing, hope it will help:
You can find examples here in description of DamerauLevenshteinMod and here in read_test_file
I dont see why the same approach cannot be used for cucumber as well.
Related
I have a scenario with following structure:
Scenario Outline: Sections displayed on login
Given Robert is at the home page
When Robert logs in as "<role>"
Then the "<section1>" should be displayed
And the "<section2>" should be displayed
Examples:
| email | section1 | section2 |
| Administrator | Administration | Profile |
| External | IGNORED | Profile |
As you can see, there is a parameter "IGNORED" in the second example.
In case I detect this keyword during runtime, I want the step, the keyword appeared in, to be skipped.
At the moment I use this:
#Then("the {string} should be displayed")
public void the_section_should_be_displayed(String sectionName) {
if(sectionName.contentEquals("IGNORED"))
return;
// rest of code
}
Here the step will be set as PASSED if the IGNORED keyword is there.
What I want instead, is that this single step is marked as SKIPPED and that the following steps are still executed and the full test scenario is afterwards marked as PASSED or FAILED depending on the status of the other steps.
So my question is:
Is there a way to manually set a Cucumber step to status SKIPPED without skipping the rest of the scenario?
I am aware that I could simply create two scenarios, one for each role. But I currently have no choice. I can not change anything within the feature file and need to solve this with code. So please try to answer the question if you can.
I appreciate any help!
Instead of adding logic to your step definition, consider making a separate scenario that skips this step. I'm assuming there is a reason that "section1" is not relevant to the role of "External"? It would be considered good practice to capture that in a different scenario with a different name. Remember that a goal of the feature files is to describe the intended behaviour of your system; using 2 different scenarios here allows you to specify what the difference in behaviour is for different roles.
For example, as follows:
Scenario Outline: Admin section is visible to Admin
Given Robert is at the home page
When Robert logs in as Administrator
Then the Administration section should be displayed
And the Profile section should be displayed
Scenario Outline: Relevant sections are visible to External
Given Robert is at the home page
When Robert logs in as External
Then the Profile section should be displayed
Alternatively, if you want to check that specific section are / are not displayed, you could do something like this, and use the step definition to check a sectio is / is not displayed based on the provided value:
Scenario Outline: Sections displayed on login
Given Robert is at the home page
When Robert logs in as "<role>"
Then the admin section "<admin>" be displayed
And the "<section2>" should be displayed
Examples:
| email | admin | section2 |
| Administrator | should | Profile |
| External | should not | Profile |
(My example might not be correct Gherkin, but should give you an idea)
If you have a simple form, you enter your name, Gender and save it. When you select the Gender drop-down it will list [Male | Female].
So when writing test scenarios for the above, the typical way is to Write it in steps.
Precondition : User Loged in and Form Opened
Steps | Expected Result
------------------------------------------------------------------------
1 : User Enters the name | Entered name should be there
2 : User Clicks on Gender Drop-down | Male and Female should be listed
3 : Users Selects a Gender | Selected Gender should be there
4 : Clicks on Save | Information get saved.
Following is one way to represent this is Cucumber.
Given User Loged in and Form Opened
When User Enters the name
Then Entered name should be there
When User Clicks on Gender Drop-down
Then Male and Female should be listed
When Users Selects a Gender
Then Selected Gender should be there
When Clicks on Save
Then Information get saved.
Now How can we represent this in Cucumber? As I read you are not suppose to have multiple actions within the same scenario. If you add multiple testcases for each step above, the Testcases will grow exponentially. What is the best way to handle this?
My approach would be to think about documenting in Gherkin (using principles of BDD) - the application behavior and not test cases.
In fact, this article provides some good context around misconceptions of BDD.
With that said, even though you are trying to validate the above criteria in a single scenario, I would recommend at least splitting into 2, to keep your acceptance tests succinct, readable and maintainable.
Here's my attempt, at reformulating:
Scenario: Gender Dropdown values
Given a user is logged in
When the user clicks the Gender dropdown
Then Male and Female options are displayed
#EndtoEnd
Scenario Outline: Saving User information
Given a user is logged in
When the user enters a <Name> and selects a <Gender>
And Clicks on Save
Then the information gets saved
Examples:
|Name|Gender|
|Keith|Male|
|Jenn|Female|
||Female| # Negative
|Rob|| # Negative
Additionally, you can also think about throwing in some negative test scenarios as well.
If you need to test on cucumber with multiple data, you can go ahead and use the scenario outline feature, where you load the data in the examples after the scenarios.
Scenario Outline: User enters details and stores
Given User Loged in and Form Opened
When User Enters the <Name>
And Users Enters the <Gender>
And Clicks on Save
Then Information get saved.
Then I Go back (In case view info)
Then I assert for <Name>
Then I assert for <Gender>
Examples:
|Name| |Gender|
|x| |Male|
|y| |female|
Cucumber is not about testing how you do things, its about specifying why you do things. This scenario is all about how the user interacts with the form, its all about clicking on this and entering that, none of this has any place in scenarios.
Your scenario and question doesn't even tell us what you are trying to do, so I'll just have to make something up. Lets say you are trying to add a friend
Feature: Adding a friend
Scenario: Add Alison
Given I am logged in
When I add Alison to my friends
Then I should see Alison is my friend
and thats pretty much that. Note how none of the details about how you add a friend are captured in the scenario, thats not what Cucumber is for.
Now on the technical side lets explore how we get this scenario to actually add a friend. First of all the step definition
When 'I add Alison to my friends' do
add_a_friend name: 'Alison' # perhaps add a gender param here
end
Notice how even the step definition doesn't know how to fill in the form.
Finally a helper method to actually do the work (done in Ruby cos its clearer)
module FriendStepHelper
def add_a_friend(name:)
# here is where we fill in the form and choose the gender
end
end
World FriendStepHelper # makes add_a_friend visible to step def
So technically this is how you represent your problem in Cucumber you write WHY you doing things in using your scenarios and you push down HOW you do things to helper methods used by your step definitions
Could any please suggest me some way we could use cucumber to repeat a test (e.g Login/Logout) multiple time, but changing only username/password for each time?
I could do for a single pair of username/password, but I'm looking for a way to re-use the test for multiple data set (The data set may be an array which store username/password, and the expected result which is true or false).
Thanks.
Scenario Outline allows you to run your Features based on the provided Examples in a tabular format, so to test login multiple time with different values you instruct to your step definition to run for every provided examples, In this we need to mention table column field in angle brackets in feature file.
Scenario Outline: Creating a new account
Given I am not authenticated
When I go to Login page
And I fill in "user_email" with "<email>"
And I fill in "user_password" with "<password>"
And I press "Sign up"
Then I should see "logged in as <email>"
Examples:
| email | password |
| testing#xyz.com | secretpass |
| testing2#xyz.com | fr33z3 |
For more references, Cucumber tutorials
I have started writing the following feature within an app designed to manage a cleaning business:
Feature: Creating a new cleaner
In order to allow Franchisees to allocate cleaners to jobs they need to be uploaded to the system
Background:
Given I am currently logged in to my account
And I have navigated to the "Cleaners" page
And I want to add a new cleaner to the database
Scenario: Add a new cleaner to the system
Given I have brought up the "Add Cleaner" form
Then I will need to complete the fields within the following form:
| first_name |
| last_name |
| email |
| date_of_birth |
| postcode |
| mobile |
| other_phone |
| address_1 |
| address_2 |
| work_radius |
| **days_available** |
| notes |
When I have entered valid data
Then I can save to the database
And I will have added a new cleaner to the system
In addition to welcoming comments on the way I have written the scenarios etc, my main problem is that I can't work out how to simulate selecting from a pre-populated field:
Populating the days_available should allow the franchisee to choose which days of the week, and which hours within those days, that a cleaner will be available for work. This obviously makes it possible to return queries which only show available cleaners for any given day/time of day.
Really hope someone can explain how this is done?
Just a quick comment on the structure of your feature file ... the 'Then' step in your feature should be asserting that something has or has not been done successfully.
Given I have logged into the site
When I add a new Cleaner to the site
Then I should see that the Cleaner has been added successfully
I would recommend using language that can be easily understood. Your scenario doesn't need to be instructions on how to use the site. Excessive navigational steps can make you lose track of the purpose of the scenario.
To answer your question regarding days_available accurately, would require some knowledge of how the site is structured and how the days_available are entered. Are you choosing from select lists, filling in form fields, etc? Also, since you are testing, you could consider setting the data from within your step (ie. hash, array) instead of passing all of the info in via a table.
Just some food for thought. Cheers.
Based on your updated post, I would suggest the following:
The step And I want to add a new cleaner to the database doesn't seem like an actionable step and could be removed. Same for the step When I have entered valid data. If you handle filling out the form in the previous step, you have already entered valid data.
If you need to multiple available days, I would consider making it its own step
And(/^the cleaner is available from (.*?) to (.*?) on (.*?)$/) do |start_time, end_time, day|
#fill in start time
#fill in end time
#select day
end
Background:
Given I am currently logged in to my account
And I have navigated to the "Cleaners" page
Scenario:
And I bring up the "Add Cleaners" form
And I complete the form with
| first name | Bob |
| last name | Smith |
...
And the cleaner is available from 0600 to 1800 on W
When I submit the Add Cleaners form
Then I should see the new cleaner has been successfully added
Here are three example BDD statements that should help explain my question:
Scenario: User logs in
Given I am on the login screen
When I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
Then I should see the login successful page
Scenario: User buys a product
Given I am logged into the system using username "myUsername" and "myPassword"
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
vs
Scenario: User buys a product
Given I am on the login screen
And I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
So scenario 1 above is fine, but which is best out of statement 2 and 3. Statement 2 reads nicely and more concisely. But my step definition for
"Given I am logged into the system using username "myUsername" and "myPassword""
will need to repeat calls to the Page Objects (or equivalent) that scenario 1 called... seems like more dev effort.
So really just wondering if anyone knows which is best practise. I have searched online and found the following document:
http://docs.behat.org/guides/1.gherkin.html
This suggestions scenario 2 is best, but then writes: "Authenticate a user (An exception to the no-interaction recommendation. Things that “happened earlier” are ok)" which kinda lends itself to scenario 3.
Cheers,
Charlie
Here is my review of the scenarios you've written.
Scenario 1
Scenario: User logs in
Given I am on the login screen
When I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
Then I should see the login successful page
Pros : You are correctly using the Given, When and Then statements. In this scenario the Given sets the initial state of the system, the When indicates actions which a user will take and the Then details the assertions made to verify the behaviour of the system.
Cons : Whilst what you have written will work, the problem you have is that this test is brittle. If your company was to mandate that a time-dependent security token also had to be specified during log-in (for example), you'd have to add another step to input this additional field. However if you rewrote this step to be declarative e.g.
Given I am on the login screen
When I submit valid log-in criteria
Then I should see the login successful page
Then if the log-in process was changed, you would only need to alter the code, the scenario would remain the same.
Scenario 2
Scenario: User buys a product
Given I am logged into the system using username "myUsername" and "myPassword"
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
Pros : Same as above.
Cons : Again the test is brittle as it's imperative i.e. you are specifying the exact log-in credentials and a specific product. I'd re-write this as:
Given I am logged into the system
When I purchase a product
Then I should have that product in the product inventory
You can save the product specified in the "When" step in ScenarioContext.Current. You would then be able to re-use this value in your "Then" step to assert that it is present in the product inventory.
Scenario 3
Scenario: User buys a product
Given I am on the login screen
And I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
Cons : This is the worst of your scenarios as you are incorrectly using the Given statement. A Given statement should be used to define an initial system state for the test, so in this case "Given I am on the login screen" is a correct use, but "Given I enter the valid username "myUsername"" is an incorrect use. It is incorrect as it is indicating a user action, hence it should be covered by a When. Yes, you can use a Given to perform the same programmatic steps as a When, but it doesn't make it right!
I'd change this scenario to the version I suggested in Scenario 2.
Firstly, there is absolutely nothing in Gherkin that prevents you from writing specifications in any order, you can even produce compound specifications such as
Given ...
When...
Then ...
When ...
Then ...
Given ...
When ...
Then ...
So why would you want to do this?
Well first lets consider a fourth variant of your scenario
Scenario: User logs in and buys a product
Given I am on the login screen
When I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
Then I should see the login successful page
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
This is of course just a compound of 1 and 2. You might have written this because you had finished testing login and wanted to quickly write and test buying a product. You already have the code for the Bindings in scenario one and now simply need to write the bindings for scenario two. You might consider this your simplest pragmatic change, and you will refactor it later. This nothing wrong with it, running the tests could be quicker, but also its not exactly ideal.
Now imagine that due to the nature of your shop you have written many tests that test different buying processes. We could be testing what happens when you add the same item again to your basket, or if you try to buy something out of stock, or different checkout experiences if you want special delivery. In fact your shop is so successful that you need to be really secure on the web and change your login process. Unfortunately you now have those three lines
Given I am on the login screen
When I enter the valid username "myUsername"
And I enter the valid password "myPassword"
repeated throughout your scenarios. If we accidentally break our login process with some bad code, suddenly all of our tests fail. You are presented with a wall of red and can't really narrow down where to start looking at the problems. Because we are dependant on logging in before we run our scenario we cannot isolate login from purchase.
This really is the difference between a Given and a When. A When is there to indicate that we are performing a process, a Given is a means of directly affecting the run time environment so that we simply have the correct state. It's basically the difference between
//Given
isLoggedIn = true
//When
if CheckValidPasswordForUser(user, password)
isLoggedIn = true
The Given has no way to fail.
So coming all the way back to your original question,
Can you re-use Givens as Whens? Yes, but it in the long term it will confuse you.
But if you ask Can you re-use Whens as Givens? then I would definitely advise you not to. This will hide the fact that something that could break is being tested.
Finally there is one other thing to consider and that is the domain of your specification. Dan North has a really good article on this Whose Domain is it anyway?, but the general gist as applied to your example here is that when you are looking at product buying you can simply write
Given I am logged in as a customer
or
Given I am logged in as an administrator
because the username and password parts are to with login and not products, and that way you are protected from re-writing all your scenarios when you change your login process to use something else.