I'm trying to write a scenario :
As a user
......
Scenario: Breaking a rule
Given:I'm an active user
When: I break the rule 'X'
Then: I should be added to rule breakers' list
Butrule breakers' list is defined in another story.
So this story is no more independent.
We can change the Then phrase to this:
Then: The rule 'X' should be breaked by me
But the behavior of my scenario doesn't seem clear enough.
More examples:
Scenario: Registration
.....
Then: I should be loged in
(While login belongs to another scenario)vs
Then: I should be registered
And the real problem comes to the action when navigation is something your stakeholders care about.
My question:
How can I have an independent scenarios which are clearer in situations like that?
If the rule breaker's list is defined elsewhere, it's part of the state in which the scenario starts, so it's a "given".
Your scenario should resemble something like:
As a user
......
Scenario: Breaking a rule
Given:I'm an active user
And: I'm not in the rule breakers' list <- Add another given
When: I break the rule 'X'
Then: I should be added to rule breakers' list
The extra "given" can reuse the same code as the other scenario, if that's useful. Alternatively, you can use a shortcut, like putting data directly into the database rather than going through the UI. It's fine for this scenario to have no knowledge of how that list comes into being, as long as there's an example of that somewhere else.
What is a "user story scenario"? User stories come and go they are not much more than a piece of paper and some discussion. After the user story is finished, it has unsignificant relevance. However, your gherkins are there to stay as automated tests and living documentation. After the user story is finished, you can deliver a new function. What is that function? Your gherkin should describe exactly that and nothing more.
In your example: ok, there might be some plans that we are also going to keep a list of rule-breakers, but let's concentrate first on the immediate user story and forget about such a list. That list might be a separate user story later on or it may never even happen. So the question is: what immediate value has my current user story, what is the functionality that the users will see. If it's just some feedback to the user along some rules, than that is what my gherkins should reflect: Given some preconditions When I'm breaking rule 'X' Than I should receive feedback this or that way.
If it is difficult to come up with sensible gherkin scenarios, chances are high that you have split up your user stories inappropriately (which BTW is not a problem, but something best found out as early as possible - which is the very reason you are using bdd.)
Related
In Behavior Driven Development style of writing automated functional tests, it is generally understood that Givens should be the pre-condition that the system must be in, in order to begin the test, When should be the user action performed and Then should be asserting whether the observed matches the expected and fail or pass the test accordingly.
Off late my team has started performing assertions in Givens and Whens too which lead me to wonder if this is a correct practice.
For e.g -
Given a user with xyz privileges is logged in
When I click on the abc tab
Then records should be displayed
Should the Given in this test actually assert that the logged in user indeed has xyz privileges or assume the user has required privileges and just perform login
Also should the When assert that tab is visible before clicking?
If "logging in" is a behaviour that's interesting* and you need examples to illustrate it** then it should be a "When", with the context in which it happens being the "Given", and the outcome that results being the "Then".
This is the case for any behaviour you need to illustrate with an example.
However, sometimes it can be useful to make assertions in a Given, just to check that the context really is set up properly. Sometimes when people start adopting BDD the environments can be a bit flaky, and it's handy to know if it's your scenario finding a bug that made it fail, or something earlier in the process. So for that reason, you might find assertions there.
The Given doesn't concern itself with how the system got into that state, though. If it has assertions, it should merely be to check that it is.
Another form I've seen is a check that the system is in the correct state for the context, with corrective action if it isn't.
Note that these are largely interim patterns. They can be helpful while teams are adopting BDD and getting their pipelines and automated deployments into shape.
Assertions which check the results of the "When" are part of the outcome, so part of the "Then". I can't imagine a case where you'd need to check the results of a "When" without it being an outcome. If you've got one, please give me an example.
We discourage using clicking and UI details in scenarios. Work out what you're trying to achieve, and do that. Hide the clicking under the covers.
Most of the time automated scenarios aren't actually there to catch bugs; they're living documentation that helps people think about what they're trying to achieve and what the system already does, thus encouraging good design and preventing bugs in the first place.
I'd say something like "navigate to the ABC tab" and just do it; you'll get a relevant exception if it isn't there, and that won't happen as often as people reading the scenario will.
* It's logging in. It probably isn't.
** It's logging in. You probably don't.
Assertions in Givens and Whens are generally an indicator of immaturity of step definitions. So I might put one in a step that I am working on, but I wouldn't keep it there for very long.
I'd implement your step
Given a user with xyz privileges is logged in
as something like
'Given a user with xyz privileges is logged in' do
user = create_user(privileges: xyz)
login_as user: user
end
I would expect that the create_user method would be trusted pretty quickly and would not need an assertion. Same for the login_as method. (if methods like this aren't working properly you'd expect many scenarios to be broken)
Notice how this code makes it clear that there are two things going on and provides/uses an api that you would expect many other stepdefs to use. And how any assertion you might want to keep really belongs in the helper methods not in the step def.
There are no strict rules however in my experience I find it handy to define that there are no assertions there so it is clear what is actually tested and the test itself will run faster.
In the case if your abc tab is not displayed it shouldn't be clickable and then the test will fail either when identifying the object to click or when performing the next step(depends on the tools and method you use).
However you should make sure that the actual implementation of the test is not cheating and working with internals that are able to trigger a click even if the actual component isn't.
There is another point about the Given, there normally it is recommended to set up the environment for the test. This means you make sure that in your system there is a user and that user has been logged in. It makes no meaning to very that as you set it up, however if any failure occur you should fail fast to know what is wrong.
Generally I would be careful about using assertions in the Given or When steps in the automation code.
However, I use login steps like this all over the place in my code as - when used correctly - can turn the step into a piece of context rather than an assertion itself.
For instance:
Is this user logged in?
Yes => Do Nothing.
No => Log out, then Log in as this user
In the example you gave, we know that we need the specific user to be logged in for this scenario to work, however we do not know if there is a different user logged in (other scenarios may have been run before it). If you use this step as a check to make sure that the correct user is logged in (as in my example), it is part of the context and it will speed up run time of the automation as you won't have to log in before and log out after every scenario.
As my ubiquitous language I have some phrases like :
Feature : Display A Post
In order to be able to check mistakes in a post
As an admin or customer
I want to be able to view the post
Scenario : Display Post
When : I select a post
Then : the post should be viewed
Is that a right user story? Such scenarios may have some minimal differences at UI level. Should I violate the DRY principle and repeat the feature for another role?
Different users may need different requirements over time, and I think this is the reason we usually write user stories per the user role.So should I be worry about how the requirements may change over time for different roles or I can leave a single user story (and the same test code,production code, databse ...) with multiple roles and refactor when their requirements forced me to separate them ?
I am not sure what your problem here and will try to guess. So first, your first three lines is just a description and not real steps. This enables adding custom text that will not run.
As to your other 2 steps, it is very hard to say whether they are good or not. As you might have already noticed, you are not bound by Cucumber to have a specific scenario flow. Cucumber gives you the freedom to design and write your code the way it makes more sense to YOU and YOUR business logic.
Saying that, I see no issue in repeating similar steps to test another role. In order to make the feature file a bit more DRY you can use the Scenario Outline option. It might look something like this:
Scenario Outline: Display Post as <role>
When I select a post as <role>
Then the post should be viewed
Examples:
|role |
|role1|
|role2|
In this case, two scenarios run one after another while rolevalue changes according to the Examples list.
Now, in regards to your possible changes in future. You can't always predict what will happen in future and unless continuously changing current requirements is a normal practice for you or your team, I wouldn't worry too much about this. If sometime in future current scenarios will become obsolete, you will review them and rewrite them or add new ones accordingly.
If multiple roles are required in a feature, then that means it is an epic, not a feature. It is a must to break down each feature so it only has one role, and it can deliver a single value to a single group of users.
I think the problem here is your language which needs refinement to clarify what you want to do here and why its important.
It seems to me that as an admin looking to fix mistakes in a post that what I need to is to be able to change a post.
A similar thing applies for the customer (should that be author?). If you explore what they will do when a post has been authored with a mistake then you will probably find that different roles interact in different ways. You'll start to ask questions about what happens if the customer and the admin make fixes, and how the customer responds when the admin makes a fix that the customer doesn't like and all sorts of other scenarios.
If you do this you'll probably find that most of your duplication goes away, and you'll learn lots about the differences between customer and admin behaviour in this particular context.
When defining scenarios in Gherkin sometimes there is no clear distinction between Given and When steps, i.e. there is no active interaction with the system from the user and the purpose of the validation is to verify how the system should look under certain circumstances.
Consider the following:
Scenario: Show current balance
Given user is on account page
Then user should see his balance
vs
Scenario: Show current balance
When user goes to account page
Then user should see his balance
I am not sure I would always use the second variant. If I have multiple scenarios sharing the context "user is on account page" and some of them have additional user actions while others don't, then it seems to me it should be valid to keep "user in account page" as a Given step even though it may lack "When" for some scenarios. Is this a valid approach?
Formally and technically Cucumber/SpecFlow doesn't require you to write a When-step or rather Given/When/Then's are just executed in the order they are written in the scenario. In that regard you don't need a When-step.
But, as Andy Waite wrote about, the When-step shows on the action or event that your system takes from the "Setup" to get to the new state that you verifies in the Then-step. In that regard a When-step should be present in every test (as you wrote: what are we testing otherwise).
That leaves your final comment; what about verifying just the setup (Given the system is started, Then the database is clean as a naïve example). In such scenarios the When-step could be skipped.
So, as always, it comes down to readability and understanding. Scenarios are written to make our thoughts about the systems behavior concrete and clear. Use the form that optimize for understanding and learning about the behavior in question.
Without thinking too hard on this I would probably guess that the general advice is to always use a When-step that makes the event or behavior very apparent and clear. I would shy away from implicit and hidden behavior when possible.
I hope this helps.
Generally a scenario consists of 3 parts:
Setup (the Given)
Action (the When)
Verification (the Then)
Sometimes the setup isn't required (or it's implicit). But I can't think of any situations in which you wouldn't need an action and verification.
Agree with Andy + Marcus here but I've a few comments that may be of use.
Gherkin feature files should act as living documentation for the behaviour of your system.
For this reason scenarios should provide enough detail to convey to a developer and other project stakeholders (product owner, testers etc) the business rules that embody that feature.
I think your question may have arisen from not considering this business rule end to end when articulating the scenario. I'd have to ask someone the question , what is a balance? Therefore I feel you may need a step to at least convey the concept - that before a user can look at their balance, they have to have one.
Scenario: Show current balance
Given I have a balance
When I go to my account page
Then I should see my balance
It's important to set system state (i.e. any 'Given' step) to allow you to clearly test that the system is working correctly - otherwise how are you going to determine that the balance is actually correct? You may wish to make this more explicit by specifying some arguments:
Scenario: Show current balance
Given my balance is £10
When I go to my account page
Then I should see my balance as £10
I'm not sure which BDD framework you are using but I use Behat which allows you to map more than one Gherkin step to a step definition. I.e
user is on account page
user goes to account page
can both map to a step definition which navigates a user to a page. The system behaviour is the same, the only reason to distinguish between the two, as you have, would be to make your scenarios more readable.
To my understanding when you write a scenario the are 3 steps that are needed.
A state that your application should be in at the begining.
What the user has to do to reach a certain state.
The outcome/input of the user's action i.e the end point of your scenario.
So the scenario will be something like :
Given the user is on the profile page
When the user goes to the balance page
Then the user should see their balance
The profile page will be where the user can click a button or link to acess their balance.
Then have a background :
Given the user is logged in
And the user has a balance
How do you handle user stories/acceptance tests that have long chains like this one, where the Then/When mingle together? Is it best to split this into a separate acceptance test where one tests that the dialog appears and then the second one tests the behavior after the dialog has been shown?
Feature: Confirmation before removing products from cart
In order to avoid accidentally removing an item from my cart
As a Customer
I want a confirmation dialog to ask me if I'm sure I want to remove an item
Scenario: I want to remove an item from my cart
Given I have added item "xyz" to my cart
When I click "Remove"
Then a confirmation dialog pops up
And it asks "Are you sure you want to remove this from your cart"
When I click "Yes"
Then item "xyz" should be removed from my cart
Your scenario seems a little long, and it's quite heavily tied to the gui. What would happen if you tied it to the capabilities of the system instead?
Scenario: I want to remove an item from my cart
Given I have a cart containing "xyz"
When I remove "xyz" from my cart
Then my cart should be empty.
The scenario now describes stuff that's useful to the user, and it's easier to refactor.
I love BDD as much as I do because I had a situation much like this. We had 120 acceptance tests and they were mostly failing. Someone had put a confirmation dialogue box in much like the one you describe, and instantly broke over 80 acceptance tests. By turning them into scenarios with high-level, reusable steps instead, we can easily refactor and keep the tests working even if the mechanisms we use to implement the capabilities of the system change. The actual clicking of buttons happens within those reusable steps, and it's OK to have more than one UI action per step.
I wrote a scenario here which does this if it's useful (it's a DSL rather than English but you should get the idea):
http://code.google.com/p/wipflash/source/browse/Example.PetShop.Scenarios/PetRegistrationAndPurchase.cs
The question is really one of what the "branches" are.
If there are multiple steps there must be user choices at each step. There should be multiple "When"'s. This should form a rich tree with lots of user-selected alternatives at each branch. Each possible outcome should have it's own test to make the various choices and arrive at that outcome.
A three step sequence with two user choices is 8 possible paths. Different paths may arrive at the same outcome (or may not). But you should have multiple paths through this.
If it's just sequential (because someone felt like writing sequential steps) and the user has no choices, then it's not really driven by consideration of the user's behavior, is it?
I don't see the choices. No choices == bad smell. But easy to test since there's only one outcome with a sequence of captive steps where the user has few or no choices.
If you work out the choices properly, then each step has multiple outcomes and each step should be tested independently.
I have a set of User Stories and I have a set of business rules (primarily laws binding my requirements to be compliant). In Agile SDLC I'm not sure where these "rules" are attached to my user stories.
For example, a user story like:
As a doctor I want to add patient information in order to create a new patient file.
And a rule like:
The following information must be entered in the record of each patient:
(a) patient:
(i) name and given name;
(ii) address;
(iii) date of birth; and
(iv) sex;
These two clearly come together, but how can I link them? As test acceptance definitions in my user story? Another user story?
There are a few different ways I've seen this handled:
An artifact is created to hold the business rule and is stored in some central repository of all the rules so this is known throughout the development team and a storehouse of knowledge is maintained. This can get ugly as there can be hundreds of rules within just a few years of building out an application.
The rules may be put onto separate cards within the user story. Thus, while the user story is that one line, there may be 6-8 cards that make up all the tasks for that story to be completed. For example, there has to be a new patient form created, validation on the form, etc. Thus, it isn't hard to see this crop up down the line on a card as a way to track the requirement that way. This is the most natural to my mind though this isn't where the specific list is going to be 100% written down either as the card could be "ensure some fields on the form are mandatory."
There isn't an explicit link but rather the rule is something for QA or a BA to note for the user to verify that the form does enforce this rule. This is similar to one but the question is what is the responsibility of the developer in this. In this case, it is something for QA to track rather than developers possibly.
The user story is intended to start a discussion, not be a comprehensive list of the requirements. The rule is something that should come up when the developer discusses with the user what does it take to create a new patient file to my mind.
I like the idea of hanging on to cards for a few sprints after the story was done, but I do see the point that the cards will ultimately be destroyed. At the same time, there should be code somewhere that implements the rules in its proper area. To use the example you posted, it may be that in a few places the list of required fields would be noticed as there is the UI layer that has to show the fields and probably an error message but also there should be some Business Logic Layer that has this logic to see that some fields were specifically completed before trying to create a new patient file. The system being built will also house the rules in some form or other, too.
As acceptance criteria. After all these are rules that can be executed as tests. Definitely not new stories, that would just be wrong as there is no deliverable goal.