Has anyone done test automation for Revit? What tools did you use?
WinappDriver is able to open revit window but is not able to recognise the elements inside it
Your Revit add-in can only run inside a running Revit.exe session, and the Revit API is completely event-driven, and only Revit.exe can raise the corresponding events, so it is a bit tricky to drive it by a unit testing framework. There are several ways to work around this, though. A few of them are discussed by The Building Coder in the topic group on Unit Testing.
Related
What is the appropriate place to provide RestAPI details while using cucumber? i.e. In Feature file or Step Definition file?
The RestAPI is not really part of the business problem you are solving. Rest is not your core functionality. It is just one, technical, way of using the real functionality you are building.
I would therefore not describe a RestAPI in the feature files using Gherkin. I would hide these implementation details in the glue code connecting the examples written in Gherkin with the system under test.
Cucumber is a tool all about communication between groups of people throughout the business you are working with.
If you're working on a project alone, there is less of the need to describe features in business language (although, it is useful for drafting up ideas using a BDD mindset).
My advice, keep the feature files without code, CSS or any other things that could throw off a layperson. Move these to the step definition file instead, and describe what you're doing in terms that a layperson can understand.
You should store it in the step_definitions file. Keep all the coding tagged to the gherkin steps in the feature file to the step_definitons folder inside which you create a file.
Its always better to keep feature files without the code, they are mainly for better understanding of the testing process and business scenarios and best practise when you view it from BDD perspective.
In case your API values are frequently changing, you can also, mention the values in the cucumber gherkin steps, and pass it as an argument to the step_definitions file. So that any change in the values can be handled in the feature file itself, within the "double quotes" instead of going to the code every-time
On a side note, what gem are you using to test on the REST API? Airborne?
We are looking into coded UI, and since we have C# coding experience we'll be coding up the automated tests ourselves.
I already know that it is possible to have a test case with defined test steps. Each test step is actually a function inside the test method and the description comes from the summary.
This is a great way for the c# coders to expose what they're doing to the non coding testers who will interact with these automated tests using MTM (Microsoft test manager).
However, it would be awesome if the C# coders were able to create generic functions as "Shared Steps" perhaps, and in this way non technical testers were able to assemble new tests in MTM using these building blocks.
Functions such as : Open the settings dialog, Maximize Window, Add an item (with parameters), Close all open tabs, etc.
However these would be coded functions, not from a recording.
The idea would be that once these functions are created and stored in TFS (Again as shared steps), the tester would then be able to use these functions as building blocks for more automated tests.
I also have to stress that these functions will be fully automated, and so new tests assembled in MTM too need to be fully automated without needing to open up Visual Studio.
Is this at all possible?
An approach used by some people is to create a test framework containing many useful functions. The test itself is then written as a data driven Coded UI test, each step being one entry from the data source. (Typically each step might be one line in CSV file.) Some columns of the data source would be commands and others data for those commands.
The commands could be relatively low level such as: find a window, navigate to a page, run an application, find a control in a window, enter text into a field, click a button or a link, verify that a field contains a given string.
Commands could be for higher level activities in your applications, such as: login, fill in a form.
Frameworks can let non coders write tests using terminology they understand. Frameworks are challenging to write.
This isn't really possible in the way you're asking - you will not be able to have your non-technical testers creating automated tests from test manager. A coded UI test will always have to exist as a test in a code file.
All of the details for a test manager test are available through the TFS API, so I suppose it should be possible to create your own layer that could generate coded UI code based on a shared step mapping system, but it would be a fairly major undertaking - its probably easier to have automated tests created by technical people.
Microsoft Coded UI Test (CUIT). Can I create an independent (external) Visual Studio 2012 Premium or Ultimate solution in C# to record on my application UI without the application's source code? This solution can reference the application's assemblies. The CUIT API does not seem to help. It is not desirable to use Microsoft guidelines in doing CUIT, as it creates and inserts a test project into the application's solution. The development team does not want this test project, created by the QA team, to be part of their source code to check into their source control.
Coded UI testing with Visual Studio is about testing the user interface of an application. Coded UI does not care how the application was constructed (but see Note below). Coded UI test projects do not need to be in the same solution as the application. Coded UI can be used to test websites and web applications (via a web browser) where there is no Visual Studio solution available for the application. Coded UI can also be used as an automation tool, to automate routine operations that use several programs and websites to perform some action. For such automation there would be no single solution file that should have the Coded UI project.
My understanding of Microsoft's Coded UI guidelines is that sometimes it is good to have application and test in the same solution, just to keep them together so as to encourage testers and developers to use the test and keep them up to date. But there is nothing to force such togetherness.
I do not understand the impact of this sentence in the original question: "This solution can reference the application's assemblies."
Note: Coded UI needs the application under test to expose some of its user interface properties via the MSAA or UIA mechanisms. This is done for you with many types of software. See the "Platform Support" section of the "Content Index for Coded UI Test" at http://blogs.msdn.com/b/mathew_aniyan/archive/2010/02/11/content-index-for-coded-ui-test.aspx .
Added 2013-05-02:
The clarification, within the comment, about wanting to link a Coded UI test to the DLLs of an application suggests there may be some confusion between what Coded UI tests are for and what unit tests are for.
Coded UI accesses the user interface of an application much like a screen reader for the visually impaired would. It also moves the mouse pointer, clicks buttons or links and enters text much like the software for people who cannot use a normal keyboard or mouse. Coded UI does not access the code directly so it does not need to reference any DLLs of the application. Before testing your application, I suggest you experiment with writing Coded UI tests to drive the Windows Calculator or to fill in forms on web pages. There are many videos and tutorials on the internet that demonstrate how to use Coded UI. Once you have some understanding of what Coded UI does and how to use it, you should be in a much better position to test your application.
What is the difference between CodedUI and UISpy? Does both use same mechanism?
I need to implement support for UISPY in my own custom grid control[WinForms] to recognize each cell as like in the MS DataGridView. currently UISPY recognizes my entire control but not individual cell.
Any help would be highly appreciated
I'm not sure how UISpy works but CodedUI uses the UI Automation framework to access the UI components in an app.
Regardless, you might want to look at using something like HawkEye or WinForms Spy as a starting point for doing what you need.
The UI Spy tool is obsolete and no longer available. Developers should use other tools such as Inspect.exe that are available in the Windows Software Development Kit (SDK).
I was learning about Behavior driven development (BDD) recently, i see that its good for CRUD web application.
Is BDD tools such as Cucumber suitable for games, specifically are they good for HTML5 Canvas games?
Is there any other BDD tools for HTML5 canvas games? Or Is BDD only for CRUD applications?
I've just recently been experimenting with BDD and Games Development. But I'm building on the .NET & XNA Frameworks with C#. I'm using Visual Studio as my IDE and testing with SpecFlow and NUnit.
SpecFlow is the BDD/Gherkin tool - and as a group we've been talking about the game and fleshing out some requirements and documenting them in Gherkin... and going from there. The key is designing the game to be testable.
We've abstracted the game engine away from main "game loop" implementation that XNA requires so we are able to load it up and inject all of the dependencies it requires. We then run the tests on the engine and make asserts on the code afterwards to make sure that it is behaving correctly.
We decided early on that Graphics/Drawing/Sound/etc were out of scope of the testing... if they were broken in the end product, it would be very obvious. The test suite we've managed to build up just gives us the confidence that the mechanics of the game itself are working as intended (i.e. is game object A in the right location? does game object B die when hit with projectile C? etc. etc.).
So far it has been quite a good experience.
The question is: can you think of ways to interact with the canvas-based game in JavaScript?
Practically, could you open the JS console and interact with the game from there? Can you fire clicks at the right coordinates, can you press keys and most importantly, can you make assertions on what's being output by the game?
If the answer is yes, then you are able to automate the game in JavaScript and either Cucumber or Cucumber.js can definitely be used on top of that.
If the answer is no and you can't figure out a way to automate the canvas content in a similar fashion than what described above, then I'm afraid no automation tool could do the job.
To answer your second question: BDD has absolutely nothing to do with "CRUD" applications. It's about describing behaviours and automating examples that illustrate those behaviours. You can therefore take a BDD approach on virtually any types of application.
Shouldn't you find a way to automate your application, you could still consider writing scenarios to document the (expected) behaviour of your app. The automation phase is not mandatory, contrary to common beliefs :)