I'm looking to set up a bunch of integration tests for an Rails 3 app that is already built.
The app is built with Rails 3 and Ruby 1.9.2. I've seen recommendations for Capybara, Cucumber and RSpec 2 but I'm not sure what the advantages of each are.
I've also noticed that they seem to be closely tied together. The post I've seen always seem to talk about using Capybara with Cucumber, or using Rspec with Cucumber.
What are the advantages/disadvantages for each of them? Are there certain combinations that work best together?
All these test tools fall in different parts of the testing environment.
If you want to set up integration tests, then you should use Cucumber because it has no real alternative. Cucumber is designed to easy Behaviour Driven Development but even if you don't BDD it is perfect for integration testing.
Capybara mission statement is "webrat alternative which aims to support all browser simulators". So to simulate the browser part (http request, DOM manipulation, etc) you have two alternatives Webrat or Capybara. Cucumber integrates fine with both of them. In fact it detects which one you have installed in your system and by default uses it.
On the other side is Rspec. Rspec is not a tool for Integration Testing but for Unit Testing (with a BDD approach). In http://www.pragprog.com/titles/achbd/the-rspec-book it is explained very clearly. Cucumber is in an outer circle of application behaviour and rspec is in an inner circle of class behaviour. The alternative to rspec is classic Test::Unit classes.
For more information see:
Cucumber
Capybara
Webrat
RSpec
Test::Unit
In summary, use all three.
Use RSpec...
...for testing methods in your models, controllers, and helpers in isolation.
Also known as Unit testing.
Use Cucumber...
...for testing high level features of your application.
Also known as integration testing. Verifies that all the pieces work together.
Good for testing from a user's perspective.
Use Capybara with Cucumber...
...for navigating your app like a user would through the browser.
...for testing your views contain the content that a user would expect to see.
Related
I'm new in the area of software developing, and want to extend my knowledge over automation testing. I read a lot for Geb and Spock testing frameworks, but did not come to the conclusion for what purpose are they.
Am I correct:
Geb is for Page Objects (static covering the UI pages selectors, elements, etc.)
Spock is for writing the test order (Cases/Steps for test) (also is able to run API tests as well, mock data, parameterised endpoints, values and etc.)
--> What other differences they have? I already checked both documentations, but cannot understand.
I will appreciate every practical advice or example, differences between both as well, because I covered a lot of things, but there are only theoretical examples, but the main catch I did not get it.
Thank you in advance!
Geb is a browser/web automation solution. You can use it to test the functionality of your web pages. Geb can automatically launch a web page, fill out form fields and click buttons on a web page. From the official website:
"It can be used for scripting, scraping and general automation — or equally as a functional/web/acceptance testing solution via integration with testing frameworks such as Spock, JUnit & TestNG."
The catch for Geb is web automation.
The catch for Spock is that it is testing and specification framework.
Examples
You can Geb to check the following:
When one opens mywebsite.com/login and enters a wrong username or password an error message should be displayed, say in a div.
When one opens mywebsite.com/submitData, fills in item name and price and clicks on the submit button, expect a message to show like "Thank you, the total number of items is now 5"
GEB is build on top of WebDriver library and is compatible all the Browsers and Drivers which work with WebDriver.
Most commonly used way of achieving UI automation is though implementation of Page Object Model design pattern, Geb supports Page Object Model by implementing all the boiler plate code in its 'Page' class. The custom pages in an Automation Framework are required to extend this 'Page' class in order to access ready made functions and closures.
Some Additional points:
It uses jQuery-ish navigator API to identify elements on the page.
There is a ready made 'js' object which lets you execute JavaScripts on your page.
There are closures which let you switch to another Window/iFrame/Alert of your webpage
Simplified handling of Dropdowns,Radio buttons,File Uploads,Checkboxes etc.
Introduces 'Interact' blocks which builds and perform the user actions (by utilizing Action class of Selenium WebDriver API)
geb.config file allows the developer to add support for multiple environments , drivers, Reporting , waiting etc.
Read more about Geb in the The Book of Geb
SPOCK on the other hand is a BDD Testing and Specification Framework inspired from frameworks like JUnit, jMock, RSpec, Groovy, Scala, Vulcans etc.
SPOCK is highly compatible with Geb and provides a 'GebReportingSpec' class which is required to be extended by the test classes in order to establish compatibility with Geb
We are looking to replace our Selenium approach to automated web data collection and have been recommended Puppeteer or Chromeless.
One of the things I like is the ability to go headless with chrome running on AWS lambda. That reason was sold as the main reason for going with Chromeless or Puppeteer. However, I see posts online indicating that the same can be done with Selenium. If that is true, what over advantages do Chromeless and Puppeteer offer over Selenium ?
We are going to be using NodeJS
Having used both Selenium and Puppeteer, these would be my observations as to why it's currently being recommended so highly:
Puppeteer is really easy to configure and execute. No setting specific drivers required. Just write your test scripts, point node towards your scripts and watch it go. Everything even runs in parallel!
It's a zero setup framework in that it comes bundled with the version of Chromium which it runs best with.
Another benefit is speed. Puppeteer is really fast since it uses headless Chrome.
It integrates very nicely with other popular test frameworks such as jest and mocha.
Using Puppeteers API is really straightforward. Everything is simple to write, easy to understand and basically allows for simple user interactions to be automated using a single line of code.
It's really easy to debug your automation scripts. Simply set headless to false and turn slowMo up from 0 to, say, 250 and you can easily see what's going on and fix any problems you may have.
It's easy to pick up and use no matter what your previous experience levels: on the team I'm working on, everyone (even those with no real automation test script writing experience) has found working with Puppeteer a really nice and relaxed experience. Everyone is getting the grasp of it within a few minutes of basic research and getting scripts running quickly and with no hassle or stress.
It should be noted that Selenium does do everything that Puppeteer does (and vice versa) but that's not the point of Puppeteer. Puppeteer allows for a team to build a large library of automation scripts very quickly using an easy to use API and get tests running now rather than having to deal with building ultra-robust test frameworks which work cross browser and / or cross device.
If you really must have cross browser testing then Selenium or perhaps InternJS (a personal favourite of mine) are still the choices to make.
Puppeteer only supports executing tests on Chrome but, at the end of the day, it's much better to have a lot of tests running (even if it's just on Chrome) as opposed to having none.
What is the appropriate place to provide RestAPI details while using cucumber? i.e. In Feature file or Step Definition file?
The RestAPI is not really part of the business problem you are solving. Rest is not your core functionality. It is just one, technical, way of using the real functionality you are building.
I would therefore not describe a RestAPI in the feature files using Gherkin. I would hide these implementation details in the glue code connecting the examples written in Gherkin with the system under test.
Cucumber is a tool all about communication between groups of people throughout the business you are working with.
If you're working on a project alone, there is less of the need to describe features in business language (although, it is useful for drafting up ideas using a BDD mindset).
My advice, keep the feature files without code, CSS or any other things that could throw off a layperson. Move these to the step definition file instead, and describe what you're doing in terms that a layperson can understand.
You should store it in the step_definitions file. Keep all the coding tagged to the gherkin steps in the feature file to the step_definitons folder inside which you create a file.
Its always better to keep feature files without the code, they are mainly for better understanding of the testing process and business scenarios and best practise when you view it from BDD perspective.
In case your API values are frequently changing, you can also, mention the values in the cucumber gherkin steps, and pass it as an argument to the step_definitions file. So that any change in the values can be handled in the feature file itself, within the "double quotes" instead of going to the code every-time
On a side note, what gem are you using to test on the REST API? Airborne?
I was learning about Behavior driven development (BDD) recently, i see that its good for CRUD web application.
Is BDD tools such as Cucumber suitable for games, specifically are they good for HTML5 Canvas games?
Is there any other BDD tools for HTML5 canvas games? Or Is BDD only for CRUD applications?
I've just recently been experimenting with BDD and Games Development. But I'm building on the .NET & XNA Frameworks with C#. I'm using Visual Studio as my IDE and testing with SpecFlow and NUnit.
SpecFlow is the BDD/Gherkin tool - and as a group we've been talking about the game and fleshing out some requirements and documenting them in Gherkin... and going from there. The key is designing the game to be testable.
We've abstracted the game engine away from main "game loop" implementation that XNA requires so we are able to load it up and inject all of the dependencies it requires. We then run the tests on the engine and make asserts on the code afterwards to make sure that it is behaving correctly.
We decided early on that Graphics/Drawing/Sound/etc were out of scope of the testing... if they were broken in the end product, it would be very obvious. The test suite we've managed to build up just gives us the confidence that the mechanics of the game itself are working as intended (i.e. is game object A in the right location? does game object B die when hit with projectile C? etc. etc.).
So far it has been quite a good experience.
The question is: can you think of ways to interact with the canvas-based game in JavaScript?
Practically, could you open the JS console and interact with the game from there? Can you fire clicks at the right coordinates, can you press keys and most importantly, can you make assertions on what's being output by the game?
If the answer is yes, then you are able to automate the game in JavaScript and either Cucumber or Cucumber.js can definitely be used on top of that.
If the answer is no and you can't figure out a way to automate the canvas content in a similar fashion than what described above, then I'm afraid no automation tool could do the job.
To answer your second question: BDD has absolutely nothing to do with "CRUD" applications. It's about describing behaviours and automating examples that illustrate those behaviours. You can therefore take a BDD approach on virtually any types of application.
Shouldn't you find a way to automate your application, you could still consider writing scenarios to document the (expected) behaviour of your app. The automation phase is not mandatory, contrary to common beliefs :)
I am planning of using Specflow for .NET. I come from a RoR background, and I have used Cucumber and Capybara. From what I have heard, Specflow is similar to Cucumber; however, does it have features that Capybara provides?
Thanks!
Coypu is the answer, I think: https://github.com/featurist/coypu
I ended up using Watin engine to simulate browser. Watin runs mainly on IE.
In some occasions, I do not need the overhead of launching an entire browser to run my specflow tests. In those cases, I use SimpleBrowser (headless browser).
As I understand it Capybara is using Selenium as the web driver. You may want to check out an article on developers who are using Selenium with SpecFlow.
http://blog.jsolutions.co.uk/?p=112