So we write functional test automation for our AUT in Coded UI. The development team uses dotCover to do unit test coverage analysis. I wonder if anyone knows if it is possible for Coded UI test cases to use dotCover to analyze the UI code in the AUT that is tested?
I readily confess that I know next to nothing about dotCover or coverage analysis. But it I were to learn that this was possible, I would learn more.
Thanks
S
Currently it is not possible to cover coded UI tests via dotCover.
Related
I am working on creating an application which is based on SharePoint Foundation 2010. I'm looking for a way to take advantage of the TDD methodology. While reseacrhing for TDD I read about Pex and Moles. Can anyone advice on what is the best approach to implement TDD for a SharePoint based custom solution. Please provide links to relevant articles or videos which can help.
It is not possible to use PEX for TDD. TDD requires you to write a failing test first, then write code to make the test pass. PEX requires you to write code first, then it generates tests for you. Something like NUnit is more suitable for TDD on .NET. Sorry, I don't know enough about SharePoint to comment on anything specific that it requires.
We are looking into coded UI, and since we have C# coding experience we'll be coding up the automated tests ourselves.
I already know that it is possible to have a test case with defined test steps. Each test step is actually a function inside the test method and the description comes from the summary.
This is a great way for the c# coders to expose what they're doing to the non coding testers who will interact with these automated tests using MTM (Microsoft test manager).
However, it would be awesome if the C# coders were able to create generic functions as "Shared Steps" perhaps, and in this way non technical testers were able to assemble new tests in MTM using these building blocks.
Functions such as : Open the settings dialog, Maximize Window, Add an item (with parameters), Close all open tabs, etc.
However these would be coded functions, not from a recording.
The idea would be that once these functions are created and stored in TFS (Again as shared steps), the tester would then be able to use these functions as building blocks for more automated tests.
I also have to stress that these functions will be fully automated, and so new tests assembled in MTM too need to be fully automated without needing to open up Visual Studio.
Is this at all possible?
An approach used by some people is to create a test framework containing many useful functions. The test itself is then written as a data driven Coded UI test, each step being one entry from the data source. (Typically each step might be one line in CSV file.) Some columns of the data source would be commands and others data for those commands.
The commands could be relatively low level such as: find a window, navigate to a page, run an application, find a control in a window, enter text into a field, click a button or a link, verify that a field contains a given string.
Commands could be for higher level activities in your applications, such as: login, fill in a form.
Frameworks can let non coders write tests using terminology they understand. Frameworks are challenging to write.
This isn't really possible in the way you're asking - you will not be able to have your non-technical testers creating automated tests from test manager. A coded UI test will always have to exist as a test in a code file.
All of the details for a test manager test are available through the TFS API, so I suppose it should be possible to create your own layer that could generate coded UI code based on a shared step mapping system, but it would be a fairly major undertaking - its probably easier to have automated tests created by technical people.
I was learning about Behavior driven development (BDD) recently, i see that its good for CRUD web application.
Is BDD tools such as Cucumber suitable for games, specifically are they good for HTML5 Canvas games?
Is there any other BDD tools for HTML5 canvas games? Or Is BDD only for CRUD applications?
I've just recently been experimenting with BDD and Games Development. But I'm building on the .NET & XNA Frameworks with C#. I'm using Visual Studio as my IDE and testing with SpecFlow and NUnit.
SpecFlow is the BDD/Gherkin tool - and as a group we've been talking about the game and fleshing out some requirements and documenting them in Gherkin... and going from there. The key is designing the game to be testable.
We've abstracted the game engine away from main "game loop" implementation that XNA requires so we are able to load it up and inject all of the dependencies it requires. We then run the tests on the engine and make asserts on the code afterwards to make sure that it is behaving correctly.
We decided early on that Graphics/Drawing/Sound/etc were out of scope of the testing... if they were broken in the end product, it would be very obvious. The test suite we've managed to build up just gives us the confidence that the mechanics of the game itself are working as intended (i.e. is game object A in the right location? does game object B die when hit with projectile C? etc. etc.).
So far it has been quite a good experience.
The question is: can you think of ways to interact with the canvas-based game in JavaScript?
Practically, could you open the JS console and interact with the game from there? Can you fire clicks at the right coordinates, can you press keys and most importantly, can you make assertions on what's being output by the game?
If the answer is yes, then you are able to automate the game in JavaScript and either Cucumber or Cucumber.js can definitely be used on top of that.
If the answer is no and you can't figure out a way to automate the canvas content in a similar fashion than what described above, then I'm afraid no automation tool could do the job.
To answer your second question: BDD has absolutely nothing to do with "CRUD" applications. It's about describing behaviours and automating examples that illustrate those behaviours. You can therefore take a BDD approach on virtually any types of application.
Shouldn't you find a way to automate your application, you could still consider writing scenarios to document the (expected) behaviour of your app. The automation phase is not mandatory, contrary to common beliefs :)
I am looking to run a load of automated functionality tests on a user interface application of mine and was wondering what is the best software out there to carry out these tests. Preferably the software will be able to intergrate with Visuall C++ 2005. I have googled various software however there is so much out there I'm not sure what is best for what I need. Any help would be awesome, thanks.
for automated software unit tests I would recommend google test. There is a very good q&a on this platform, which you can find here.
Additionally, there is CPPUnitLite, which is developed by the author of "Working Effectively with Legacy Code", Michael Feathers.
I used AutoIt Scripts for testing a MFC application just a little bit, but it was not that easy to maintain them properly and build an effective logging system for failed tests.
However, the unit tests depend heavily on the architecture of your program and the structure of your class - especially the dependencies to other components / classes. So if you already have an existing MFC application, which was not built with unit tests in mind, you probably have to refactor a lot of things. Therefore, I would recommend the mentioned book. You can also use the classic "Refactoring" by Martin Fowler.
Did you take a look at AutoIt? It's a generic GUI scripting language which can also be used from C++/C#. I'm using it to automate parts of testing a Visual Studio addin.
I'm looking to set up a bunch of integration tests for an Rails 3 app that is already built.
The app is built with Rails 3 and Ruby 1.9.2. I've seen recommendations for Capybara, Cucumber and RSpec 2 but I'm not sure what the advantages of each are.
I've also noticed that they seem to be closely tied together. The post I've seen always seem to talk about using Capybara with Cucumber, or using Rspec with Cucumber.
What are the advantages/disadvantages for each of them? Are there certain combinations that work best together?
All these test tools fall in different parts of the testing environment.
If you want to set up integration tests, then you should use Cucumber because it has no real alternative. Cucumber is designed to easy Behaviour Driven Development but even if you don't BDD it is perfect for integration testing.
Capybara mission statement is "webrat alternative which aims to support all browser simulators". So to simulate the browser part (http request, DOM manipulation, etc) you have two alternatives Webrat or Capybara. Cucumber integrates fine with both of them. In fact it detects which one you have installed in your system and by default uses it.
On the other side is Rspec. Rspec is not a tool for Integration Testing but for Unit Testing (with a BDD approach). In http://www.pragprog.com/titles/achbd/the-rspec-book it is explained very clearly. Cucumber is in an outer circle of application behaviour and rspec is in an inner circle of class behaviour. The alternative to rspec is classic Test::Unit classes.
For more information see:
Cucumber
Capybara
Webrat
RSpec
Test::Unit
In summary, use all three.
Use RSpec...
...for testing methods in your models, controllers, and helpers in isolation.
Also known as Unit testing.
Use Cucumber...
...for testing high level features of your application.
Also known as integration testing. Verifies that all the pieces work together.
Good for testing from a user's perspective.
Use Capybara with Cucumber...
...for navigating your app like a user would through the browser.
...for testing your views contain the content that a user would expect to see.