If you have a simple form, you enter your name, Gender and save it. When you select the Gender drop-down it will list [Male | Female].
So when writing test scenarios for the above, the typical way is to Write it in steps.
Precondition : User Loged in and Form Opened
Steps | Expected Result
------------------------------------------------------------------------
1 : User Enters the name | Entered name should be there
2 : User Clicks on Gender Drop-down | Male and Female should be listed
3 : Users Selects a Gender | Selected Gender should be there
4 : Clicks on Save | Information get saved.
Following is one way to represent this is Cucumber.
Given User Loged in and Form Opened
When User Enters the name
Then Entered name should be there
When User Clicks on Gender Drop-down
Then Male and Female should be listed
When Users Selects a Gender
Then Selected Gender should be there
When Clicks on Save
Then Information get saved.
Now How can we represent this in Cucumber? As I read you are not suppose to have multiple actions within the same scenario. If you add multiple testcases for each step above, the Testcases will grow exponentially. What is the best way to handle this?
My approach would be to think about documenting in Gherkin (using principles of BDD) - the application behavior and not test cases.
In fact, this article provides some good context around misconceptions of BDD.
With that said, even though you are trying to validate the above criteria in a single scenario, I would recommend at least splitting into 2, to keep your acceptance tests succinct, readable and maintainable.
Here's my attempt, at reformulating:
Scenario: Gender Dropdown values
Given a user is logged in
When the user clicks the Gender dropdown
Then Male and Female options are displayed
#EndtoEnd
Scenario Outline: Saving User information
Given a user is logged in
When the user enters a <Name> and selects a <Gender>
And Clicks on Save
Then the information gets saved
Examples:
|Name|Gender|
|Keith|Male|
|Jenn|Female|
||Female| # Negative
|Rob|| # Negative
Additionally, you can also think about throwing in some negative test scenarios as well.
If you need to test on cucumber with multiple data, you can go ahead and use the scenario outline feature, where you load the data in the examples after the scenarios.
Scenario Outline: User enters details and stores
Given User Loged in and Form Opened
When User Enters the <Name>
And Users Enters the <Gender>
And Clicks on Save
Then Information get saved.
Then I Go back (In case view info)
Then I assert for <Name>
Then I assert for <Gender>
Examples:
|Name| |Gender|
|x| |Male|
|y| |female|
Cucumber is not about testing how you do things, its about specifying why you do things. This scenario is all about how the user interacts with the form, its all about clicking on this and entering that, none of this has any place in scenarios.
Your scenario and question doesn't even tell us what you are trying to do, so I'll just have to make something up. Lets say you are trying to add a friend
Feature: Adding a friend
Scenario: Add Alison
Given I am logged in
When I add Alison to my friends
Then I should see Alison is my friend
and thats pretty much that. Note how none of the details about how you add a friend are captured in the scenario, thats not what Cucumber is for.
Now on the technical side lets explore how we get this scenario to actually add a friend. First of all the step definition
When 'I add Alison to my friends' do
add_a_friend name: 'Alison' # perhaps add a gender param here
end
Notice how even the step definition doesn't know how to fill in the form.
Finally a helper method to actually do the work (done in Ruby cos its clearer)
module FriendStepHelper
def add_a_friend(name:)
# here is where we fill in the form and choose the gender
end
end
World FriendStepHelper # makes add_a_friend visible to step def
So technically this is how you represent your problem in Cucumber you write WHY you doing things in using your scenarios and you push down HOW you do things to helper methods used by your step definitions
Related
I am trying to build a bot(custom UI in my website) where a user will enter a product name to view the details of it and I will provide a link to the product full details page. I have a situation where if the user enters a name and there are multiple results from my database, I want to show him those products as quick replies so that he can select one from them.
How do I recognize that the user has entered the product name and anything else? I can use #sys.any, but all small conversation will also go there, which will be of no use.
The same problem occurs when I display him a list of products with matching name. But now when the user clicks on any of the button I am taking him to a custom follow-up intent where I have entered the template for a product entity. But, dialogflow only recognizes the products that have been defined in the entity(listed few products and checked auto expand).
I have tried using #sys.any instead, but the intent is called for any string the user types in. Lets say, the user does not respond and after a while he types in "hi", my intent with any is being called. How do I overcome this situation?
So far as I understand, I can see two ways to solve this query. First, using an entity & defining your product list over there for bot to understand user responses (which you have done) but this will become an overhead when you have a list of say 1000/more products. Second way, you can continue using #sys.any & define a parameter, write a webhook where you validate user entered response to product list in database & check if it is present over there, if yes, show product details or say, entered response is incorrect.
I am trying to convert Selenium test to Gherkin. Is there way to implement if statements in Gherkin?
Example : assume the code is written in the below format. I am just writing description as below. Please understand the part after double slash is the actual Selenium code:
// launch the application
// login to application
// navigate to page
String str;
if(str== "XYZ")
{
// verify title
}
//verify text field 1
//verify test field 2
//verify select box
For this I am trying to write code in Gherkin as follows
Given user launches the application
When user login with valid credentials
and navigate to required page
When String str is "XYZ"
Then verify title
And verify text field 1
And verify test field 2
And verify select box
but this code is incorrect because if the str is not equal to "XYZ" we want that title should not be verified but other verification like text field1,2 and select box should be verified.
You don't implement if in Gherkin.
Gherkin is about communication and those you want to communicate with, non coders, don't know what an if statement is. They don't care either.
The solution? Two scenarios to cover both cases.
Ideally, this level of detail would not be in your Gherkin scenario. The best approach is describe business use cases, not low level details. This is what Gherkin is designed for: communicating with non-technical stakeholders so that you can work out if you are building the right thing in the first place. Here is what I would write:
Given the user is logged in
And the user is on the required page
When they enter data that requires the optional fields to be validated
And they enter invalid data in the optional fields
Then the form shows an error on the optional fields
The low level details don't matter (that the string is specifically "XYZ" or that it is the title field is not important), so these should be hidden in the step definition and/or unit tests.
In order to continue to check the other fields, you can just add another step after this:
When they enter invalid data in all of the other fields
Then each other field has an error message attached to it.
Again, there is no need to specify the actual fields, or separate them into their own steps. The idea is to express the high level business value of the scenario, i.e. that the form is validated when it should be.
The advantage to keeping things high level is that when the form changes (as it eventually probably will), then this scenario can remain untouched. Which is correct as the business case is the same: it should validate when it's supposed to. All the changes will be in the step definitions. This means that there is no reason to have another discussion with your stakeholders about whether your scenarios are still testing the right thing.
You can write the scenario, somewhat like this:
Given the user launches the application
When user login with valid credentials
And navigates to required page
Then he should see the page datails
Inside the Then step you manage all the logic.
Then(/^he should see the page details$/) do
if condition
...
else
...
end
end
Gherkin is not a programming language to use if or else conditions. It is a part of BDD framework, that is implemented, to make the stakeholders and other non technical resources understand what the test process is about. Hence, it is always recommended, you keep the gherkin as simple and as generic as possible.
Strictly speaking you should create an alternative statement along the lines of:
Given user launches the application
When user login with valid credentials
and navigate to required page
When String str is NOT "XYZ"
Here are three example BDD statements that should help explain my question:
Scenario: User logs in
Given I am on the login screen
When I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
Then I should see the login successful page
Scenario: User buys a product
Given I am logged into the system using username "myUsername" and "myPassword"
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
vs
Scenario: User buys a product
Given I am on the login screen
And I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
So scenario 1 above is fine, but which is best out of statement 2 and 3. Statement 2 reads nicely and more concisely. But my step definition for
"Given I am logged into the system using username "myUsername" and "myPassword""
will need to repeat calls to the Page Objects (or equivalent) that scenario 1 called... seems like more dev effort.
So really just wondering if anyone knows which is best practise. I have searched online and found the following document:
http://docs.behat.org/guides/1.gherkin.html
This suggestions scenario 2 is best, but then writes: "Authenticate a user (An exception to the no-interaction recommendation. Things that “happened earlier” are ok)" which kinda lends itself to scenario 3.
Cheers,
Charlie
Here is my review of the scenarios you've written.
Scenario 1
Scenario: User logs in
Given I am on the login screen
When I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
Then I should see the login successful page
Pros : You are correctly using the Given, When and Then statements. In this scenario the Given sets the initial state of the system, the When indicates actions which a user will take and the Then details the assertions made to verify the behaviour of the system.
Cons : Whilst what you have written will work, the problem you have is that this test is brittle. If your company was to mandate that a time-dependent security token also had to be specified during log-in (for example), you'd have to add another step to input this additional field. However if you rewrote this step to be declarative e.g.
Given I am on the login screen
When I submit valid log-in criteria
Then I should see the login successful page
Then if the log-in process was changed, you would only need to alter the code, the scenario would remain the same.
Scenario 2
Scenario: User buys a product
Given I am logged into the system using username "myUsername" and "myPassword"
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
Pros : Same as above.
Cons : Again the test is brittle as it's imperative i.e. you are specifying the exact log-in credentials and a specific product. I'd re-write this as:
Given I am logged into the system
When I purchase a product
Then I should have that product in the product inventory
You can save the product specified in the "When" step in ScenarioContext.Current. You would then be able to re-use this value in your "Then" step to assert that it is present in the product inventory.
Scenario 3
Scenario: User buys a product
Given I am on the login screen
And I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
Cons : This is the worst of your scenarios as you are incorrectly using the Given statement. A Given statement should be used to define an initial system state for the test, so in this case "Given I am on the login screen" is a correct use, but "Given I enter the valid username "myUsername"" is an incorrect use. It is incorrect as it is indicating a user action, hence it should be covered by a When. Yes, you can use a Given to perform the same programmatic steps as a When, but it doesn't make it right!
I'd change this scenario to the version I suggested in Scenario 2.
Firstly, there is absolutely nothing in Gherkin that prevents you from writing specifications in any order, you can even produce compound specifications such as
Given ...
When...
Then ...
When ...
Then ...
Given ...
When ...
Then ...
So why would you want to do this?
Well first lets consider a fourth variant of your scenario
Scenario: User logs in and buys a product
Given I am on the login screen
When I enter the valid username "myUsername"
And I enter the valid password "myPassword"
And I press the login button
Then I should see the login successful page
When I purchase the product "myProduct"
Then I should have "myProduct" in the product inventory
This is of course just a compound of 1 and 2. You might have written this because you had finished testing login and wanted to quickly write and test buying a product. You already have the code for the Bindings in scenario one and now simply need to write the bindings for scenario two. You might consider this your simplest pragmatic change, and you will refactor it later. This nothing wrong with it, running the tests could be quicker, but also its not exactly ideal.
Now imagine that due to the nature of your shop you have written many tests that test different buying processes. We could be testing what happens when you add the same item again to your basket, or if you try to buy something out of stock, or different checkout experiences if you want special delivery. In fact your shop is so successful that you need to be really secure on the web and change your login process. Unfortunately you now have those three lines
Given I am on the login screen
When I enter the valid username "myUsername"
And I enter the valid password "myPassword"
repeated throughout your scenarios. If we accidentally break our login process with some bad code, suddenly all of our tests fail. You are presented with a wall of red and can't really narrow down where to start looking at the problems. Because we are dependant on logging in before we run our scenario we cannot isolate login from purchase.
This really is the difference between a Given and a When. A When is there to indicate that we are performing a process, a Given is a means of directly affecting the run time environment so that we simply have the correct state. It's basically the difference between
//Given
isLoggedIn = true
//When
if CheckValidPasswordForUser(user, password)
isLoggedIn = true
The Given has no way to fail.
So coming all the way back to your original question,
Can you re-use Givens as Whens? Yes, but it in the long term it will confuse you.
But if you ask Can you re-use Whens as Givens? then I would definitely advise you not to. This will hide the fact that something that could break is being tested.
Finally there is one other thing to consider and that is the domain of your specification. Dan North has a really good article on this Whose Domain is it anyway?, but the general gist as applied to your example here is that when you are looking at product buying you can simply write
Given I am logged in as a customer
or
Given I am logged in as an administrator
because the username and password parts are to with login and not products, and that way you are protected from re-writing all your scenarios when you change your login process to use something else.
I have the following Gherkins:
Given i go to the URL
And i enter the <NRIC or FIN> as NRIC / FIN
And i select the option to proceed Next
And i select the first available available Handset Color
And i select the option to proceed Next
And i enter <Full Name> as Name
When i select the option to proceed Next
The "I select the option to proceed Next" appears three times. How should this be written as in the Step Definitions Java class file?
As well as the Declarative/Imperative question have a think about what requirement you're describing. I find it's helpful to include the Scenario: in the examples for this reason. It's very unusual to have that much detail in a Given step
Are you testing the creation of an order (bit of a guess)? If so then your enter/select steps should be the Whens in the scenario, so something like:
Scenario: Create a new order
Given I have gone to the URL
When I enter the NRIC/FIN: <NRIC/FIN>
And I choose the first available Handset Colour
And I enter <Full Name> as Name
Then my new order should be confirmed (or whatever)
However, if you're creating the order simply to set up the scenario to test something else then you can cheat and just state that the order should exist:
Scenario: Check the status of an order
Given that I have created an order:
| NRIC/FIN | Colour | Full Name |
| xxxxx | Red | John Doe |
When I check the status of my order
Then I should see a new order...
It's up to the automation whether it clicks through the screens to create that order or just inserts it into the database, all that matters is that by the time you get to the Whens the order should exist.
In an ideal BDD world the Gherkin would be written before the implementation, but it often doesn't work that way. I still find it useful to try and pretend I'm writing the Features in that ideal world. Asking "How would I have written this before we started development?" can help to separate the actual requirements (I can enter an order) from implementation details (I click Next after entering each item of data).
When writing cucumber scenarios you can adopt either a imperative or declarative style. I would prefer to write the same thing as below.
Given i go to the URL
And i enter NRIC / FIN
And i select the first available Handset Color
And i enter <Full Name> as Name
Then I should see that
So it depends on who is going to read your scenarios. A link worth reading is imperative - declarative
This requires a detailed explanation.
Imagine that I have an Excel spreadsheet with test cases in one worksheet
and I may have expected (validation) messages in another (in addition to expected messages in the first worksheet).
There is also some linking between the values of fields in one to the second worksheets.
See: Welcome, <First Name> <Last Name> as an example.
You can see in the "Expected Results" field in "Test cases" worksheet the value of the field is:
"The user is taken to My Account page and following welcome message is displayed:
"&Messages!$B$1244&", where First name is Dave and Last Name is Brown."
so "&Messages!$B$1244&", denotes field B1244 in worksheet "Messages"
Now the question.
If I am given all test cases like the example below for an ecommerce web site, how can I use Coded UI Testing based on this input? Can I automate Excel, use the steps in test case worksheet and combine that with Coded UI recording of data input and verification.
I believe I would need to do manual coding, partially using recorded input steps and verifications from Coded UI recorder and possibly using manual programming for verification.
I would like to hear if others have done something similar.
I would like to incorporate this into Specflow BDD, by writing feature/user story and these test cases will be scenarios.
Any success, thoughts on using Excel test automation as data driven testing.
Thanks
Rad
Test cases worksheet named "Test cases":
=====================
Test Case Name Test Case Objective
frontstore.01-3 Register a shopper from order
confirmation page with valid inputs
# Step Data Expected Results
------------------------------------------------------------------------------------------------
1 Launch the test storefront http://testserver.com/index Welcome page is loaded.
2 Click Sign In link Sign In page is loaded.
3 Click Register under New Customer Register page is loaded.
4 Enter valid inputs and click Submit "Logon ID = TestUser
Firstname = John
Lastname = Clark
... (other fields) Registration Successful.
The user is taken to My Account page
and following welcome message is displayed:
Welcome, <First Name> <Last Name>, where First name is
David and Last Name is Brown."
Validation Messages worksheet named "Messages":
=====================
#Text used in MyAccountPages
---------------------------------------------------------------------------
MA_WELCOME Welcome, <First Name> <Last Name>
After reading a bit about Coded UI testing:
It can certainly be done, but data/sentences like:
“Launch the test storefront”
“Click Sign In link”
“Registration Successful.
The user is taken to My Account page
and following welcome message is displayed:
Welcome, , where First name is
David and Last Name is Brown."
contain both actions and data so I need to drill down into parts of the sentence to
translate it to actions and binding to parameters.
If I understand well data binding can only be used to bind column values to some parameters.
So I need some way to automatically recognize the meaning of these sentences and use some binding
from parts of it.
So if I have a sentence:
“Launch the test storefront” that would be translated to:
CurrentBrowser.Navigate(Helper.TranslateTargetUrlFrom(“test storefront”))
where “test storefront” might resolve to http://testserver.com/index storefront home
page and I can ignore Data column for URL
or I can capture Launch keyword to mean CurrentBrowser.Navigate(ColumnValue(Data)) and ignore “test storefront” part of the sentence.
“Click Sign In link” could be translated to CurrentBrowser.FindLink(“Sign In”).Click(),
so it this case I will need to know that Sign In is
the text of the link, again I need to extract “Sign In” to mean the text of a link.
I see this as pretty manual style of CodedUI where I could do small recording for some actions and rely on manual extractions of terms from
given sentences.
I would like to know how can I semantically write better test cases to allow automation. I would probably need some kind of free form
test case parser that would recognize the semantic meaning of some words like: click, navigate, launch, enter, click under etc and translate this into
code by re-using existing helper methods and recorded actions and do some manual binding, but not with the whole data value in the column, but
an extracted value.
Any idea of this kind of automation?
I think yould could do this by data binding the input parameters and just reading Excel as a datasource, you are going to need to use CodedUI for that not MTM + Fast Forward