I have a list of tests which i need to run on several sites
I can do scenario outline in each test with examples of sites, but if list of sites is changed i need to change a list in each scenario. Background does not have outline function.
#test
Scenario Outline: Test for several sites
When I navigate to <site>
Then something
Examples:
|site|
|https://www.test.com|
|https://www.test2.com|
|…|
Maybe somebody have ideas how more correctly do it?
Alot depends on how many tests are common and how many tests are specific to each site.
If all the tests are common the best way is to extract the site out of the test and use an environment variable to set the site. Then you would run cucumber with the environment variable e.g. something like
cucumber site_tests SITE=http/your.site.com
cucumber site_tests SITE=http/my.site.com
For things like user accounts etc. You can be clever and have site specific things in support code in your features, or use more environment variables.
This will keep you features much simpler and avoid the need to use scenario outlines.
If you only have a couple of tests that have to be run on many sites then I would write a feature or perhaps just scenarios for each site e.g.
mysite.feature
Background:
Given site is my site
Scenario: Login
Given I am registered
When I login
Then I should see mystic
and
yoursite.feature
Background:
Given site is your site
Scenario: Login
Given I am registered
When I login
Then I should see your site
You can still share most of your step def and code if you are clever and write scenarios with a high level of abstraction.
There are of course alot of other solutions in between, but in general I would strongly advise against a solution based on scenario outlines.
Hope thats useful :)
One way is that you can assign variables to all these sites and use these variables in the feature files. you can read the value of these variables in step definition during execution from a constants file. that way you can simply update them in a single place.
Related
I have developed a program using curses, everything is cool so far but I was wondering myself if there is a good pattern to split different views/panels of my program into smaller chunks callable by my main loop?
Further informations:
This program is a rather small automation tool/wizard aiming to ease our application for customers requiring the on-premise installation.
This wizard is a 3 steps one and it’s used to grab informations from our customer installation needs depending of it’s chosen architecture.
The first step is requesting the customer to give us its identification informations such has contract number, company name, licence number and preferred contact.
The second step is requesting the customer to give us informations about either he want a standalone installation (All-In-One install) or a N-Tiers installation plus the required informations like the requested custom SSL VHostName or Tiers IP/Credentials.
The third and final step is showing the customer a progress bar and informations of the required services (MySQL/HTTPd/HAProxy/PHP-FPM) and our application.
I know that I do not especially need to use curses library for such a program but our UX Team requested it as it is part of our customer experience with the solution.
You can look at the Forms library. It's a nice extension to ncurses that allows you to better manage input forms like yours. It offers a simple function interface to read the fields, change their properties, etc., as well as many different field types (including regexp-validated fields). In your case, you can simply create three forms, and post/unpost them in succession.
as such way to do things is not really usual, do not expect any framework to be available (like those available for WebUI for instance).
I so decided to create my own "Framework/factory" and so to be able to split every aspect of my app in a logic that would be similar of those used by web applications.
The source-code is dirty and really not pythonic, but it is well working so far and quite easy to maintain.
Thanks everyone for your answers and ideas.
Given a java web application
And that it has a restful back-end
And serves a single page html/js front-end
When I use cucumber to test my application
Then which layer should I drive my tests through?
Possible Options:
1) Domain layer: StepsDefs delegate directly to services and repositories
2) REST layer: StepsDefs delegate to REST client which fires HTTP requests at container deployed app
3) User interface: StepsDefs delegate to web driver such as selenium and manipulate the user interface.
PS) Feel free to write your answer in given-when-then notation :)
I think you are asking 2 separate questions, one in the title and another in the body.
1) What is the 'correct' level of abstraction?
Executable specifications should be written in the domain/ubiquitous language that makes sense to all the relevant stakeholders (especially non-technical). Each scenario should usually validate a single behaviour and the text should include only relevant information - redundant or incidental detail should be omitted.
The test for correctness is "are the people reading this scenario interested in it?" If the answer is "yes", you will probably be getting valuable feedback from them. If the answer is "no" then you need to collaboratively refine your domain language and focus on behaviours that do interest them.
You may find that you have various stakeholders that have different interests. That's fine. Separate the scenarios into different feature files, each targeted on a segment of your stakeholders. Think of these as different levels of details within a large printed manual.
Any tests that the technical team want to write that no non-technical stakeholder seems interested in can be written using your favourite "unit" testing framework. You could use Cucumber/Gherkin, but is the cost of maintaining domain language for these test worthwhile? You need to decide.
2) How should the StepDefs interact with the application?
This question is orthogonal to 1). And the answer is, as always, it depends. I apply the testing pyramid approach and favour tests that exercise as little of the application as makes sense. If I'm testing the behaviour of a component I'd like to interact just with that component through the simplest interface it presents. As I move up the pyramid, I start testing protocols between components, and finally I'm ensuring that the whole application has deployed correctly and 'hangs together'.
Sometimes the only interface available is the UI. That sucks, but we have to live with it if that's the way the application has already been built. This often leads to slow and brittle executable specifications that need significant maintenance. Next time, drive the development from the outside and ensure you have ways of exercising the application underneath the UI.
A technique that #everzet and I arrived at from different directions, is to use tags to alter how StepDefs interact with the application. The domain language remains unchanged, but the tag signals to the test code whether it should interact through the UI, the REST API or direct calls to the code.
He has documented his approach in "modeling by example". I used the same technique in the opposite direction to rebuild trust between dev & test and described in The Cucumber for Java Book
I am getting ready to release a new web site in the coming weeks, and would like the ability to run multivariate or a/b tests between two version of the site.
The site is hosted on azure, and I am using the Service Gateway to split traffic between the instances of the site, both of which are deployed from Visual Studio Online. One from the main branch and the other from an "experimental" branch.
Can I configure Google analytics to assist me in tracking the success of my tests. From what I have read Google analytics seems to focus on multiple versions of a page within the same site for running its experiments.
I have though of perhaps using 2 separate tracking codes, but my customers are not overly technically savvy, so I would like to keep things as simple as possible. I have also considered collecting my own metrics inside the application, but I would prefer to use an existing tool as I don't really have the time to implement something like that.
can this be done? are there better options? is there a good nugget package that might fulfil my needs? any advice welcome.
I'd suggest setting a custom dimension that tells you which version of the site the user is on. Then in the reports you can segment and compare the data.
I'm possibly developing a web-based application that allows users to create individual pages. I would like users to be able to use their own domains/sub-domains to access the pages.
So far I've considered:
A) Getting users to forward with masking to their pages. Probably the most in-efficient option, as having used this before myself I'm pretty sure it iFrames the page (not entirely sure though).
B) Having the users download certain files, which then make calls to the server for information for their specific account settings via a user key of some sort. The most efficient in my mind at the moment, however, this requires letting users see a fair degree of source code, something I'd rather not do if possible
C) Getting the users to add a C-NAME record to their DNS settings, which is semi in-efficient (most of these users will be used to uploading files via FTP hence why B is the most efficient option), but at the same time means no source code will be seen by them.
The downside is, I have no idea how to implement C or what would be needed.
I got the idea from: http://unbounce.com/features/custom-urls/.
I'm wondering what method of the three I should use to allow custom urls for users, I would prefer to do C, but I have no idea how to implement it (I'm kind of asking how), and whether or not the time spent learning how-to/getting that kind of functionality set-up would even be worth it.
Any answers/opinions/comments would be very much appreciated :)!
Option C is called wildcard DNS: I've linked to a writeup that gives an example of how to do it using Apache. Other web server setups should be able to do this as well: for what you want it is well worth it.
How can I communicate in a very simple and effective way the path the user takes through my application?
I'm currently working in a Ruby and Rails environment, so I tend to visualize my application in terms of RESTful URIs. So for example, if I want my users to sign up, I could match a new route called /users/new. The thing is, I would like to see beyond that specific action, and visualize how many pages or forms does it take to create an account and some business logic associated with the process in general. In other words, I'd like to see a mix of a workflow diagram and some implementation details (at an interface level).
I was thinking in showing mockup pictures wrapped in boxes, and relate them through arrows with their corresponding GET, POST, PUT, DELETE methods and URIs attached to them. I think it is a good idea, but I haven't seen examples yet that inspire me.
In your experience, what helps you see the big picture? Balsamiq mockups allow to define links and navigate through the app, but it doesn't help to conceptualize.
Have you thought of using a mind-map? You could try the free FreeMind
If you stick with UML, you could consider an Activity diagram.
I think you're on the right path. Showing different screens with possible combinations of users' transactions between them is a good technique. Basically you would be able to show user's flow through your application and stress out decisions a user will make on the way.
The good example for it was presented here http://vimeo.com/43869717
This technique called Storyboarding. You should be able to find some examples. But the one I mentioned above is one of the best Storyboarding techniques. I use it all the time to show the big picture and present application workflow from user perspective to my team.