Consider this controller method from an ASP.Net MVC3 project:
Having run dotCover code coverage, the Details method has clearly been covered by a test, as indicated by the green highlighting (arrowed).
However, when I try to use ReSharper to navigate to "covering tests", it reports no coverage:
dotCover clearly knows which tests have covered the code, as shown in the ReSharper test runner:
So did I miss something? Shouldn't ReSharper be able to tesll me which tests covered a particular statement? Did I find a bug? Should I report it against dotCover, ReSharper or MSpec?
This is unfortunately a known bug with dotCover and it's not clear from comments to this bug report whether the problem is going to be fixed anytime soon.
BTW "Navigate to covering tests" is dotCover's functionality, not ReSharper's.
Related
I'm in the process of writing some new C++/WinRT based components in order to replace some much older C++/CX code. The goal is to be able to use third-party C++ tools that don't understand CX (static code analyzers, etc).
However the first step in the journey is to ensure I can properly unit test my own code. Unit testing C++/CX code typically used the "C++ Unit Test App" project type, which is C++/CX based and has its own issues (lack of code coverage support, run all required before tests show up in the explorer, stability, etc)
Browsing through the available project types in Visual Studio 2017, I did not see a unit test project template for C++/WinRT based projects. Is my only option to use the "C++ Unit Test App" template with all its failings, or is there another way to build tests for a C++/WinRT library?
Perhaps there is a way to configure either the "Native Unit Test Project" or "Google Test" project templates to support what I'm looking for?
Ideally what I'm looking for is something that doesn't require launching a UI, is pure C++(/WinRT), and supports Visual Studio's Code Coverage Analysis.
There is no unit test project that is specific to C++/WinRT, much like there isn't one for other libraries like STL. I would recommend Catch2 as it supports C++17 (a requirement for C++/WinRT) and works well on Windows. It is also what we use for testing C++/WinRT itself. Catch2 is nice because it helps you create a simple console app that acts as the test driver that includes all of the tests.
For code coverage I don't have a strong recommendation, but if you are using Visual Studio then you might want to try VSInstr. It can be used for code coverage and produces a report that can be viewed with Visual Studio.
Make sure your code is built using the /profile linker option. This will ensure that profile hooks are included in a dedicated section of the PE file. Next, run vsinstr to instrument any of the binaries you're interested in (that were previously built with /profile):
vsinstr /coverage tests.exe
Now run vsperfcmd to begin collecting coverage data:
vsperfcmd /start:coverage /output:report
Run the code as normal. For Catch2, you can simply run the executable at the command line. Then you need to stop the collection as follows:
vsperfcmd /shutdown
And you're done. You can now view the report in Visual Studio:
devenv report.coverage
Hope that helps. Again, this is not specific to C++/WinRT and since C++/WinRT is a header-only library you are liable to get a lot of noise that is unrelated to your specific project. I haven't found a good way to deal with that yet.
Expanding on my comment to #KennyKerr's answer for those that are interested...
If you are planning on using Catch2 as recommended, then the C++/WinRT Windows Console Application template is a great starting point. Pretty much all you have to do is tweak the main() to setup Catch2 and start writing your test cases. My only complaint is that the C++/WinRT templates don't allow you to add Windows Runtime Component project references via the UI (must be done by editing the vcxproj). There is probably a similar problem adding NuGet package references.
As noted in my comment above, there is a Catch2 test adapter for Visual Studio 2017/2019 in the marketplace. Be aware that it requires a .runsettings file to enable the adapter and to tell it which projects are Catch2 test applications (via a regex). Without a properly configured runsettings, it will not find your tests. I also had to increase the discovery timeout, otherwise it "forgot" my tests occasionally.
With regards the code coverage, when using Visual Studio you can configure the code coverage to include/exclude functions in the .runsettings file. See Microsoft's Site for details. For myself I added the following in the CodeCoverage section and it works pretty well so far:
<Functions>
<Include>
<Function>.*YourNamespaceHere.*</Function>
</Include>
<Exclude>
<Function>winrt.*GetRuntimeClassName</Function>
<Function>winrt::impl.*</Function>
<Function>winrt::(?!YourNamespaceHere).*</Function>
</Exclude>
</Functions>
For those that are trying to test a C++/WinRT Windows Runtime Component like me, and have code that is not exposed as part of the WRC interface, here is what I did to make that testable...
Create a C++ Shared Items Project
Move all of the code for your Windows Runtime Component (WRC) project into the shared items project, and out of the WRC project. Going forward, only add/remove files from the shared project. That way you don't have to touch the WRC or Test projects when files are added/removed.
Add a reference to this shared items project in both your original WRC project, and your test project
Make sure your test project and WRC project are configured similarly with respect compile settings and project/NuGet references
Edit the test project and ensure the RootNamespace is configured the same as the WRC project (probably has to be done via your favorite editor). This is required otherwise the generated headers will be prefixed with the namespace, and thus won't be found by the shared code.
(Optional for Code Coverage) In the test project, enable profiling (Linker > Advanced > Profile > Yes)
You should now be able to write tests that exercise the private code. As to whether or not this is the best approach, I leave to the reader. It works for me, and the code I'm testing is simple enough that I'm not overly concerned with the project definitions not aligning perfectly. Your mileage may vary.
I will note that the above can also be used to make the "Native Unit Test Project" work with C++/WinRT, you just have the extra steps of integrating the C++/WinRT bits into the test project first.
Guys can anyone will explain me the use of the parameter's like automated test, run settings , analyze test impact, (each of them) etc present under Test in process section of Build Definition in TFS for Visual studio 2013. I am a fresher and new to this technology so can anyone explain me this in detail.
This is a very general and basic question. And most is related to the concept.
Such as run settings.
Unit tests in Visual Studio can be configured by using a
*.runsettings file.
analyze test impact
Used to determine which tests should be run since a previous build, only for manually test on TFS2013, more details please refer this tutorial.
Because of the limitation of length, no more tautology here. If you are a totally fresher to test in TFS, you could take some time go through all the topics in this link Testing the application. Which include all the answers you want and can help you quickly,comprehensively understand knowledge in this area.
I have a node.js project running mocha tests, and I'm generating a coverage report using blanket. I've managed to get the coverage report generated, but I'm not sure how to generate a report that can be consumed by and viewed in Jenkins. Any suggestions? I'm looking for a result similar to the Cobertura plugin (https://wiki.jenkins-ci.org/display/JENKINS/Cobertura+Plugin).
Edit: Sorry i misread your question, if the coverage report gets published with the xunit report i dont know. So the following might not help you.
The XUnit reporter should create a report that can be parsed by jenkins.
Check out this blog post.
Also, have a look into the XUnit Plugin, it allows to specify the parser for various kinds of report formats.
Quote for persistence:
Source https://blog.dylants.com/2013/06/21/jenkins-and-node/
Let me start by saying I'm new to both ReSharper and dotCover and that I'm using v10.0.2 of both.
The attached screenshot shows solution explorer in VS and the coverage tree for a set of tests.
Whenever I run coverage, it always shows the same subset of assemblies in the coverage tree. Importantly, all of the tests shown are for code in either the Services or Infrastructure assemblies, neither of which show in the coverage tree.
Clearly, the product is not doing something right or I'm not.
Why are only some of the assemblies shown in the coverage tree?
Why aren't any of the assemblies covered by the tests I'm running
shown in the coverage tree?
How do I make it work properly?
EDIT
If it makes any difference, I'm using xUnit and have the xUnit running extension installed in ReSharper and the tests themselves run just fine.
This is due to shadow copying - when enabled, dotCover expects .pdb files to be copied too, and the standard shadow copy that xunit performs doesn't do this. If you disable shadow copy in the Unit Testing options page, it'll work fine. I think the xunit runner can be updated to fix this.
The YouTrack issue that describes what's going on is here: DCVR-7976
In my case the *.pdb files where deleted by a post-build event. After changing that, coverage-analysis worked again.
This post from the support forum of jetbrains helped me
In the options menu of ReSharper, under Tools > Unit Testing > MSTest, you can enable MSTest, and then there is a checkbox "Run tests even if Code Coverage is enabled".
Wat is the impact of enabling this option?
It was initally off, but as Code Coverage is enabled on our project, I had to check this option for ReSharper to run the unit tests. But why would ReSharper, by default, not want to run MSTest unit tests when Code Coverage is enabled?
Apparently, ReSharper test runner must have had problems with MSTest when Code Coverage is enabled. I haven't explored this in full detail, because, as gorhoroh mentioned in the comment on my question, the option is no longer there.
If anyone cares to elaborate on this further, I'll be happy to mark that as an answer, provided it really explains what the implications were of course.