I am using Windows 8 Pro and Visual Studio 2012 Ultimate (both RTM).
I create a Metro-style class library, and then a corresponding unit test library. Both target WinRT (not the .NET framework).
I can run unit tests fine, but when I attempt to "Analyze code coverage for all tests", the output window comes up with the results of the rebuild, and that's it. The Code Coverage Results window is all grayed out.
From reading on the web, some have alluded to the idea that code coverage is not enabled for WinRT assemblies, but I cannot confirm. I have also tried creating a .testsettings file, like was needed in VS2010 and explicitly turn on code coverage, and I got the same results.
Can anyone confirm or deny that code coverage is not available for WinRT-targeted assemblies, in VS2012 RTM?? I'm just trying to figure out if I'm missing some setting - or if it's not possible.
It's not possible at this stage due to the sandboxed nature of WinRT apps.
P.S. It's on the list of things the team is looking to resolve in the near future (no, I don't have a timeframe for it)
Related
We are using VS2008 prof. in our application. In 2008 prof. code coverage feature isn’t available. So we are achieving this by creating unit test project and added its entry in .vsmdi file and running test cases we can get code coverage feature.
Now we are planning to migrate our application from VS2008 prof. to VS2012 prof. In VS2012 prof. also code coverage feature isn’t available. So we also trying the same VS2008 prof. approach to achieve it but still I am not getting code coverage option.
Please let me know if there is any way to achieve code coverage feature in VS2012.
Code Coverage is native only available in VS12 Premium and Ultimate not in Profesional. See here at "Determine how much code is being tested with code coverage analysis".
There are other alternatives that you can use to get code coverage (assuming you are using .NET) see What can I use for good quality Code Coverage for C#/.NET?.
I am partial to OpenCover for obvious reasons :)
I am re-writing a console application and trying to use Unit Tests. I'm using NUnit, C#, .NET Framework 4 in Visual Studio 2010 on a Windows 7 PC.
When I change code within the application my unit tests don't pick up the latest changes. I can see that NUnit is reloading but for some reason it's not getting the latest version. The only way to get NUnit to work with the latest version is to remove the reference to the Console app, add it back and compile.
Is there a way to avoid doing this? I think the problem is that the the console app is compiled as an EXE, whereas with most applications I've unit-tested they have tended to be DLLs.
EDIT
I had hoped that this question: Unit Testing a Console Application inside Visual Studio contained the answer. If you look at the accepted answer this was the way I'd been proceeding but my test project wasn't picking up the changes. I've now created a class library and I'm applying my tests to that. My console app will simply load, then call, this class library.
Second Edit
I've now got my code to a stage where I can test functionality and I've noticed the same problem with my console app. Although it 'says' it is referencing the compiled DLL, it only picks up the latest change when I delete and re-attach the reference. So, I'm thinking that the problem isn't necessarily connected to NUnit, rather an issue/quirk with console apps.
We have to decide which technology to use for our unit testing. Currently we use Visual Studio 2010 and not happy with MSTest that came with that. It is buggy, poor in deployment (E.g the test setting output directory is not recognized correctly), and have several issues when trying to test assemblies in 32bit and 64bit versions. To make matters worst MSTest does not have a good impedance match with our Jenkins build system. We therefore thought of moving into NUnit. However, no one in our team has a good exposure to NUnit. Also,we will be soon moving into Visual Studio 2012.
I need to know the pros and cons of Visual Studio 2012 MSTest vs Nunit latest version. Since most of the articles on stack overflow are related to older versions of VS they are not related to us. I guess Microsoft has improved MSTest a lot since 2010. Please provide an unbiased comparison with detail technical issues you have faced in both technologies (newer versions only)
I'm using both MSTest and NUnit at the moment. IMHO NUnit is still a better solution. If you have Visual Studio 2012 Premium edition then it's actually quite nice, except for the fact that you can't seem to group together tests. I like the fact it's integrated into Visual Studio, but the lack of grouping and the ability to run a subset of tests without manually selecting them is a huge problem.
The coverage analysis is also pretty neat in Premium. It's fast and gives you what you need quickly. It is a Premium feature though.
Since there are still lacking features in MSTest (even removed features since vs2010), I would still recommend using NUnit for unit tests. The benefits include test grouping by namespace, the ability to add test case annotations (running the same test multiple times with different parameters) and it works well with Opencover and Report Generator for coverage analysis. The main cited con is that it's not integrated like MSTest, so it really depends how much that matters to you as to whether or not that is a con.
#Biranchi: It doesn't matter anymore which unit test framework you use in Visual Studio 2012 (and upwards). See my blogpost here, the sequel to the one you refer to. http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/20/part-2-using-traits-with-different-test-frameworks-in-the-unit-test-explorer.aspx
You can even mix and match tests from different frameworks, you can even do that down to the method level !!
This means you can even move legacy code from one to another with no bad sideeffects.
Also see this for how to use Nuget to install the NUnit adapter into the solution, freeing the developer for installing it herself.
http://blogs.msdn.com/b/visualstudioalm/archive/2013/06/11/part-3-unit-testing-with-traits-and-code-coverage-in-visual-studio-2012-using-the-tfs-build-and-the-new-nuget-adapter-approach.aspx
#Sriwantha: MSTest is a simpler framework than NUnit. NUnit (and also XUnit) give you more flexibility, which also leads to less code to write. One example: If you are using categories (and you should), MSTest require a category to decorate every method. NUnit allows you to decorate the class - that will take effect for all the methods in that class. NUnit also allow to you use strongly typed categories
public class Integration : Category {}
This is enough to declare a category that you can use instead of
Category("Integration");
where you risk spelling errors.
NUnit has much better support for data driven tests.
NUnit has also support for theories
to name a few.
VS2012 does allow categorization of tests into groups if you have "Update 1" or later:
http://msdn.microsoft.com/en-us/library/hh270865.aspx#BKMK_Grouping_and_filtering_the_test_list
Have you looked into Traits functionality of VS ?
http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/09/how-to-manage-unit-tests-in-visual-studio-2012-update-1-part-1-using-traits-in-the-unit-test-explorer.aspx
Grouping is much better in mstest (2012 update 1) compared to nunit.
One more thing to add here. It looks like MSTEST engine doesn't work well with TFS Build in certain scenarios. If you are using TFS build, it won't report the skipped tests (marked with Ignore attribute). It will only include Assert.Inconclusive. If that's an issue, you should use NUnit instead of MSTest.
So I've downloaded VS2012 RTM and upgraded my Coded UI test projects to the VS2012 versions.
The reason I have done this is because I was having issues with VS2010 Coded UI tools due to partial support of IE9.
So I ran the tests from my developer machine both locally and remotely (making use of the new Visual Studio agent tools), and the tests worked successfully. Presumably, the reason my tests now work is that VS2012 upgraded them so that they now work with IE9, right?
So - this is my problem. I want to automate these tests, by launching them from our build machine - but our build machine still runs on VS2010, and for the time-being this is not going to change.
So I checked in my new VS2012 Coded UI test solution into TFS and queued a new build - so the build machine built my solution. And the build was successful. All good there.
So, next I created a new test case in Microsoft Test Manager and associated with the ordered test list in my new solution. Then I launched the tests (using the existing VS2010 test controller) on my remote test environment (which has the existing VS2010 test agent tools).
But the tests failed - the same issues that affected my tests with the VS2010 Coded UI tests (due to lack of full support for IE9)
Why did they fail?
Do I need the new VS 2012 agent tools for my test controller and agents? Must I build my solution in VS2012?
Ideally, I don't want to have to install VS2012 RTM on my build machine - I want to do the minimum possible to get my tests working and automating.
Is there a way around this?
Your coded ui tests reference assemblies that come with VS installation or your agent's installation (the assemblies defininf the WpfControl, Mouse, Keyboard, Playback and other classes).
So if you run it on a build/test machine with older versions of the dll's the same issues are going to stay, the tests will not start using the newer fixed assemblies thatt were shipped with VS2012.
As a temporary workaround you can check what dll's you reference and make sure that the build process puts them in the same directory with your tests assembly. That way the lookup for them will find the in the current directory without using the /path and finding them in the VS installation directory.
If I debug some unit tests with Resharper, and those unit tests use the SMO SDK, I get this error:
Any idea how to avoid it? I'm running ReSharper 6.1.37.86 (Full Edition)
UPDATE: After upgrading to Resharper 6.1.1000.82, I get a slightly different error:
Filed in JetBrain's issue tracker: http://youtrack.jetbrains.com/issue/RSRP-326369
I managed to find a two-step workaround:
Use .NET 3.5, not .NET 4.0.
After this change, debugger stopped crashing. However, it still was very slow, and sometimes my breakpoints were not hit. The second step that needs to be done for this workaround to make your debugger fully work is:
Go to Properties of your .NET project, "Debug" tab, disable the option "Enable the Visual Studio hosting process"
An example of the effect these two changes have in your project files is this diff:
https://github.com/7digital/DatabaseMigraine/commit/badfebce83a1c4e39ddfce8a707b0e0ca62ad5a0