I've got a handful of specflow tests that look something like this:
Scenario: Person is new and needs an email
Given a person
And the person does not exist in the repository
When I run the new user batch job
Then the person should be sent an email
Scenario: Person is not new and needs an email
Given a person
And the person does exist in the repository
When I run the new user batch job
Then the person should not be sent an email
Except instead of just 2 scenarios, I've got 10 very similar scenarios, all with the type of steps so I want to use a "Scenario Outline". Unfortunately, I'm having a really hard time coming up with a readable way to re-write my steps.
Currently, I've come up with this but looks clunky:
Scenario: Email batch job is run
Given a person
And the person '<personDoes/NotExist>' exist in the repository
When I run the new user batch job
Then the person '<personShould/NotGetEmail>' be sent an email
Examples:
| !notes | personDoes/NotExist | personShould/NotGetEmail |
| Exists | does not | should |
| No Exist | does | should not |
I also considered this, and while it is cleaner it doesn't convey meaning nearly as well
Scenario: Email batch job is run
Given a person
And the person does exist in the repository (is '<personExist>')
When I run the new user batch job
Then the person should be sent an email (is '<sendEmail>')
Examples:
| !notes | personExist | sendEmail |
| Exists | false | true |
| No Exist | does | false |
Does anybody have a better way of parameterizing concepts like "does", "does not", "should", "should not", "has", "has not"? At this point, I'm thinking about leaving the everything as a different scenario because it is more readable.
Here is what I've done in the past:
Given these people exist in the external system
| Id | First Name | Last Name | Email |
| 1 | John | Galt | x |
| 2 | Howard | Roark | y |
And the following people exist in the account repository
| Id | External Id | First Name | Last Name |
| 45 | 1 | John | Galt |
When I run the new user batch job
Then the following people should exist in the account repository
| External Id | First Name | Last Name | Email |
| 1 | John | Galt | x |
| 2 | Howard | Roark | y |
And the following accounts should have been sent an email
| External Id | Email |
| 2 | y |
You can use the table.CreateSet() and table.CreateSet() helper methods in SpecFlow to quickly turn the tables into data for your fake external system repository and your account table in the database.
Then you can use table.CompareToSet(accountRepository.GetAccounts() to compare the table in your "Then" clause to the records in your database.
The neat thing is, all of the steps you wrote are reusable for multiple situations. All you do is change the data in the tables, and SpecFlow writes the tests for you.
Hope that helps!
Maybe you should split them into two scenarios
Scenario Outline: User exists in the repository
Given a person
| Field | Value |
| First | <first> |
| Last | <last> |
And the person exists in the repository
When the user attempts to register
Then the person should be sent an email
Examples:
| first | last |
| Bob | Smith |
| Sarah | Jane |
And then another scenario for the opposite. This keeps the scenario meaning very clear. If your common steps are worded genericly you can reuse them. I also try to come from the approach of the user
Related
Scenario to automate:
Given <precondition> was fulfilled
And <user> is authorized
When user requests <endpoint>
Then user should receive <code> response
Test data matrix:
| precondition | endpoint | user1 | user 2 | ....
| | /users | OK | Not Found |
| | /roles | OK | OK |
| | /create_user | OK | OK |
| object user exists | /update_user | OK | OK |
| object user exists | /delete_user | OK | OK |
| | /create_data_role | OK | Not Found |
| data role exists | /update_data_role | OK | Not Found |
....
There's around 20 users with different role combination and around 20 endpoints.
Need to verify each endpoint for each user - so it should be a nested cycle.
How do I do it?
Don't do this in Cucumber - reasons
1) You get no benefit from putting all these routes and conditions in Gherkin. Nobody can read them and make sense of them especially if you trying something combinatorial
2) Cuke scenarios run slowly, and you want to run lots of them, you could dramatically reduce your run time by writing a fast unit test instead.
3) If you write this test in code you can write it much more elegantly than you can in Gherkin.
4) Dealing with errors is painful (as you've already pointed out)
You are using the wrong tool for this particular job, use something else.
Yet I come up with this option, but it doesn't follow gherkin convention because When and Then steps a jammed in one
1. Preconditions moved to #Before hook
2. Scenario
Given <user> is authorized
Then <user> requests functionality appropriate response code should be received
| ENDPOINT | USER1 | USER2|
| /users | 200 | 404 |
| /create_user | 200 | 404 |
| /update_user | 200 | 404 |
Examples:
| username |
| USER1 |
| USER2 |
It's also inconvenient because when tests failed it takes time to identify faulty endpoint(S)
Let's say there are Alice, Bob, Eve and Arbitrator.
And let's say
Alice has a table of records
| id | pet type | birth date |
|----------------------------------
| 1 | cat | 2010-03-03 |
| 2 | dog | 2011-06-12 |
Bob has a table of records
| id | pet type | color |
|-------------------------------|
| 2 | dog | white |
| 3 | bird | green |
Eve has a table of records
| id | pet type | size |
|--------------------------------
| 1 | cat | small |
| 3 | bird | small |
Now everyone wants to enrich his own data by the neighbor's data with the corresponding id, but without disclosuring this id, for example,
Alice wants her data to be like the following
| id | pet type | birth date | color | size |
|------------------------------------------------------
| 1 | cat | 2010-03-03 | | small |
| 2 | dog | 2011-06-12 | white | |
Bob wants his data to be like the following
| id | pet type | birth date | color | size |
|------------------------------------------------------
| 2 | dog | 2011-06-12 | white | |
| 3 | bird | | green | small |
and so on.
Arbitrator coordinates all the exchange operations between the parties and also matches the data using corresponding encrypted id fields from the dataset of each party, so parties must communicate through the arbitrator, but not directly to each other.
Also arbitrator must be able to ensure that
hash(Alice's id = 2) = hash(Bob's id = 2), hash(Bob's id = 3) = hash(Eve's id = 3)
and so on, but must not be able to recover original identifiers, and also arbitrator must not be able to brute-force the encrypted identifiers (so if talking about some kind of hashes - they must be salted)
To simplify things for Alice, Bob and Eve - they would like to have only a single key to encrypt own identifiers, but this key should be different for each party, i.e.
F1(alive_key(alice_id)) = F2(bob_key(bob_id)) = F3(eve_key(eve_id))
where, F1, F2, F3 - are some functions the arbitrator applies to encrypted identifiers of Alice, Bob and Eve, and these functions does not decrypt the original identifiers, but lead the encrypted identifiers to be the same.
So the question - is there any algorithm that can help to solve such an issue?
We are creating Gherkin feature files for our application to create executable specifications. Currently we have files that look like this:
Given product <type> is found
When the product is clicked
Then detailed information on the product appears
And the field text has a value
And the field price has a value
And the field buy is available
We are wondering if this whole list of and keywords that validate if fields are visible on the screen is the way to go, or if we should shorten that to something like 'validate input'.
We have a similar case in that our service can return a lot of 10's of elements for each case that we could validate. We do not validate every element for each interaction, we only test the elements that are relevant to the test case.
To make it easier to maintain and switch which elements we are using, we use scenario outlines and tables of examples.
Scenario Outline: PO Boxes correctly located
When we search in the USA for "<Input>"
Then the address contains
| Label | Text |
| PO Box | <PoBox> |
| City name | <CityName> |
| State code | <StateCode> |
| ZIP Code | <ZipCode> |
| +4 code | <ZipPlus4> |
Examples:
| ID | Input | PoBox | CityName | StateCode | ZipCode |
| 01 | PO Box 123, 12345 | PO Box 123 | Boston | MA | 12345 |
| 02 | PO Box 321, Whitefish | PO Box 123 | Whitefish | MN | 54321 |
By doing it this way, we have a generic step "the address contains" that uses the 'Label' and 'Text' to test the individual elements. It is a neat and tidy way to test a lot of potential combinations - but it probably depends on your individual use case - how important all of the fields are.
You only need to validate the ones that provide business value, which is probably all of them. I would avoid using tech terms like "field" because it isn't related to a behavior. Al Mills is right on for using the tables.
I'd word it like this:
Scenario Outline: Review product details
Given I find the product <Type>
When I select the product
Then detailed information on the product appears including
| Description | <Description> |
| Price | <Price> |
And I can buy the product
Examples:
| Type | Description | Price |
| Hose | Rubber Hose | 31.99 |
| Sprinkler | Rotating Sprinker | 12.99 |
The words I chose are behaviors or whats, not technical implementations or hows.
I was trying to run a simple Feature file but i was getting Exception like :
Exception in thread "main" cucumber.runtime.CucumberException: Error parsing feature file.
which is Caused by: gherkin.lexer.LexingError: Lexing error
i am trying to parametrized a When statement and got this Exception:
Scenario: Login to Gmail
Given User is on Gmail login page
When User enters <userName> and <pwd>
And Clicks on login button
Then User should redirect to home page
scenario outline(tried Examples as well but didn't worked):
|userName | pwd |
|ravivani10 | abc |
The correct syntax for a scenario outline is to start with the keyword Scenario Outline: and list the examples with the Examples: keyword.
Scenario Outline: Login to Gmail
Given User is on Gmail login page
When User enters <userName> and <pwd>
And Clicks on login button
Then User should redirect to home page
Examples:
| userName | pwd |
| ravivani10 | abc |
I had this same problem but I was using correct syntax. Turns out my formatting was wrong, yes you read correctly: formatting. My scenario looked like this:
Scenario Outline: Confirm that hitting the endpoint returns the expected data
Given uri url/to/a/service/to/test/param/{interval} and method GET
And system user
When I call the web service
Then I expect that 'http status is' '200'
And the following rules must apply to the response
| element | expectation | value |
| $ | is not null | |
| objectType | value = | Volume |
| objectData | is not null | |
| objectData | count = | 1 |
| objectData[0].value | is not null | |
| objectData[0].value | data type is | float |
| objectData[0].value | value = | <value> |
Examples:
| interval | value |
| int1 | 355.77 |
| int2 | 332.995 |
| int3 | 353.71125 |
Above test will fail with Lexing Error. Now take a look at the indentation of the Example piece of my test (it is indented one level down the Scenario Ouline).
If I indent my test as follows (same level as Scenario Outline):
Scenario Outline: Confirm that hitting the endpoint returns the expected data
Given uri url/to/a/service/to/test/param/{interval} and method GET
And system user
When I call the web service
Then I expect that 'http status is' '200'
And the following rules must apply to the response
| element | expectation | value |
| $ | is not null | |
| objectType | value = | Volume |
| objectData | is not null | |
| objectData | count = | 1 |
| objectData[0].value | is not null | |
| objectData[0].value | data type is | float |
| objectData[0].value | value = | <value> |
Examples:
| interval | value |
| int1 | 355.77 |
| int2 | 332.995 |
| int3 | 353.71125 |
Above test will pass. Totally dumb to me but that's how it works.
This can be caused by not having the final | at the end of each line of data. That's not the reason for the OP but might help someone else.
A lexing error from cucumber just means that the feature file wasn't in the format that cucumber is expecting. This could be things like having a scenario title with no content or having the title "Feature: blah" twice. This will happen even if the error isn't in the scenario that you are running.
The lexing error will usually give you a line number. Can you post the line it complains about please?
I got the same error and it was caused by having a space between the word 'Outline' and the colon symbol
Scenario Outline : Convert currencies
When I removed the space, I had this:
Scenario Outline: Convert currencies
..and the issue got resolved.
To find out the offender, check your error logs and you will find in the output the line number where the error lies. I hope this helps someone
You need to do a couple of things: A) remove the spaces in Feature : and Scenario Outline : keywords; and B) change the Scenario Outline to Scenario (or add the missing examples for the outline).
If you run this feature:
Feature: Proof of concept that my framework works
Scenario: My first Test
Given this is my first test
When This is my second step
Then This is the final step
Then cucumber will output the to-be-completed step definitions:
You can implement step definitions for undefined steps with these snippets:
Given(/^this is my first test$/) do
pending # Write code here that turns the phrase above into concrete actions
end
When(/^This is my second step$/) do
pending # Write code here that turns the phrase above into concrete actions
end
Then(/^This is the final step$/) do
pending # Write code here that turns the phrase above into concrete actions
end
Is there any way to reuse data in SpecFlow feature files?
E.g. I have two scenarios, which both uses the same data table:
Scenario: Some scenario 1
Given I have a data table
| Field Name | Value |
| Name | "Tom" |
| Age | 16 |
When ...
Scenario: Some scenario 2
Given I have a data table
| Field Name | Value |
| Name | "Tom" |
| Age | 16 |
And I have another data table
| Field Name | Value |
| Brand | "Volvo" |
| City | "London" |
When ...
In these simple examples the tables are small and there not a big problem, however in my case, the tables have 20+ rows and will be used in at least 5 tests each.
I'd imagine something like this:
Having data table "Employee"
| Field Name | Value |
| Name | "Tom" |
| Age | 16 |
Scenario: Some scenario 1
Given I have a data table "Employee"
When ...
Scenario: Some scenario 2
Given I have a data table "Employee"
And I have another data table
| Field Name | Value |
| Brand | "Volvo" |
| City | "London" |
When ...
I couldn't find anything like this in SpecFlow documentation. The only suggestion for sharing data was to put it into *.cs files. However, I can't do that because the Feature Files will be used by non-technical people.
The Background is the place for common data like this until the data gets too large and your Background section ends up spanning several pages. It sounds like that might be the case for you.
You mention the tables having 20+ rows each and having several data tables like this. That would be a lot of Background for readers to wade through before the get to the Scenarios. Is there another way you could describe the data? When I had tables of data like this in the past I put the details into a fixtures class in the automation code and then described just the important aspects in the Feature file.
Assuming for the sake of an example that "Tom" is a potential car buyer and you're running some sort of car showroom then his data table might include:
| Field | Value |
| Name | Tom |
| Age | 16 |
| Address | .... |
| Phone Number | .... |
| Fav Colour | Red |
| Country | UK |
Your Scenario 2 might be "Under 18s shouldn't be able to buy a car" (in the UK at least). Given that scenario we don't care about Tom's address phone number, only his age. We could write that scenario as:
Scenario: Under 18s shouldnt be able to buy a car
Given there is a customer "Tom" who is under 16
When he tries to buy a car
Then I should politely refuse
Instead of keeping that table of Tom's details in the Feature file we just reference the significant parts. When the Given step runs the automation can lookup "Tom" from our fixtures. The step references his age so that a) it's clear to the reader of the Feature file who Tom is and b) to make sure the fixture data is still valid.
A reader of that scenario will immediately understand what's important about Tom (he's 16), and they don't have to continuously reference between the Scenario and Background. Other Scenarios can also use Tom and if they are interested in other aspects of his information (e.g. Address) then they can specify the relevant information Given there is a customer "Tom" who lives at 10 Downing Street.
Which approach is best depends how much of this data you've got. If it's a small number of fields across a couple of tables then put it in the Background, but once it gets to be 10+ fields or large numbers of tables (presumably we have many potential customers) then I'd suggest moving it outside the Feature file and just describing the relevant information in each Scenario.
Yes, you use a background, i.e. from https://github.com/cucumber/cucumber/wiki/Background
Background:
Given I have a data table "Employee"
| Field Name | Value |
| Name | "Tom" |
| Age | 16 |
Scenario: Some scenario 1
When ...
Scenario: Some scenario 2
Given I have another data table
| Field Name | Value |
| Brand | "Volvo" |
| City | "London" |
If ever you aren't sure I find http://www.specflow.org/documentation/Using-Gherkin-Language-in-SpecFlow/ a great resource