I have:
1. Gradle 1.6
2. SonarQube instance 5.0 (also have 4.1+ instances)
3. sonar-runner (aka SonarQube runner) version: 2.4
4. Groovy Version: 2.3.3 JVM: 1.7.0_40 Vendor: Oracle Corporation OS: Linux
I have project's with Java, Java + Groovy or totally Groovy based project.
All of these projects have valid Unit and non-Unit tests (Integration, Selenium etc). I'm also running various static analysic tools for both Java/Groovy (pmd, findbugs, checkstyle, codenarc etc).
For a Java based project, when I run sonarRunner (Gradle task) OR sonar-runner, I see valid data in project's Sonar dashboard for everything (including combined code coverage and other project health info). So, everything works for a JAVA based project.
The same though, is not working for a Groovy based project. i.e. when I pass "sonar.language=grvy" for a Groovy based project and run Sonar analysis, I do NOT get Unit and IT test coverage as it says 0% coverage OR it doesn't even LIST those in Sonar project's dashboard but I successfully get project health info (issues, lines of code, etc etc).
When I pass the above sonar language variable as "sonar.language=java" for this Groovy project, I do NOT see Groovy project's data info (issues, lines of code etc etc) and any code coverage info for Unit / Integration tests and their combined tests. I have valid jacoco exec files for both Unit tests and Integration tests (jacocoIT.exec is created after jacocoagent is attached to the target Tomcat JVM and after Tomcat instance is stopped).
Any ideas! why sonar is not working for a Groovy based project OR for a project which has both Java+Groovy (source Java, tests in Groovy or vice versa or mixed).
PS: I tried sonar.language=grvy,java OR sonar.language=java,grvy BUT, it didn't make any improvement on the project's sonar dashboard.
The latest SonarQube version that I have (5.0) does support multiple language support.
Sonar's Project Motion Chart plugin also doesn't show any info as well even though Lines of Code is showing up at the right side (when sonar.languages is passed as grvy).
When I pass sonar.language=java I get, here Project Motion Chart is showing nothing (not even a blank graph) says "No Data". All other project health related info is gone too (Technical debt etc fields says 0):
With the latest SonarQube version, I see there are new sonar.*.* fields and I'm trying to find if I'm missing to provide a field or it's correct value.
OK.. what happens??? when I use SonarQube instance version 4.1.2 (instead of 5.0)
When I use sonar.language=java
I see some info and Unit test coverage says 0% but Unit test success says 100% successful. Even though I have valid jacoco IT exec file for Integratiojn tests, it still not showing that widget info.
When SonarQube instance is 4.1.2 and if I set sonar.language=grvy, I do NOT see any Unit tests coverage or IT coverage of state of tests being successful % at all:
This is what I have in my project's sonar-project.properties file (when I use sonar-runner utility).
sonar.projectName=GradlePlugin
sonar.projectKey=com:company:projectABC:GradlePlugin
sonar.projectVersion=1.0
# optional description
sonar.projectDescription=ContainerSvc Business Service
#sonar.language=grvy,java
sonar.language=java
#Tells SonarQube that the code coverage tool by unit tests is JaCoCo
sonar.java.coveragePlugin=jacoco
#Tells SonarQube to reuse existing reports for unit tests execution and coverage reports
sonar.dynamicAnalysis=reuseReports
#Tells SonarQube where the unit tests execution reports are
sonar.surefire.reportsPath=build/test-results/UT
# Some properties that will be inherited by the modules
#sonar.sources=src/main/groovy,src/main/java,src/test/java,src/test/groovy,src/java-test,src/groovy-test
# Sonar Unit Test Report path
sonar.jacoco.reportPath=build/jacoco/UT/jacocoUT.exec
# Sonar Integration Test Report Path
#sonar.jacoco.itReportPath=build/jacoco/IT/jacocoIT.exec
sonar.jacoco.itReportPath=build/jacoco/UT/jacocoUT.exec
# Sonar Binaries
sonar.binaries=build/classes/main
sonar.java.binaries=build/classes/main
sonar.groovy.binaries=build/classes/main
sonar.sources=src/main/groovy
sonar.tests=src/test/groovy
# Encoding of the source code
sonar.sourceEncoding=UTF-8
sonar.scm.url=scm:svn:http://kobaloki/Core/GradlePlugin/trunk
sonar.junit.reportsPath=build/test-results/UT
sonar.java.coveragePlugin=jacoco
sonar.groovy.coveragePlugin=jacoco
sonar.grvy.coveragePlugin=jacoco
I used/tweaked the above values many time with different values while running sonar-runner.
I used the same values with -D / -P parameters when I run sonarRunner task in Gradle.
Note: SonarQube 5.0 won't work with Gradle (1.6) version (that I'm using) as I get an error about "Failed to download Libraries" etc. (this error won't come if I use Gradle > 2.2 version).
One other point I noticed is that, even though I'm using Groovy compile option debug=true for compiling both Groovy project's source and tests code, while doing sonar-runner (with both SonarQube instance 4.1.2 or 5.0 isntances) --or running sonarRunner Gradle task (with 4.1.2), I see the following lines in the analysis output.
15:32:20.647 INFO - Analysing /production/jenkins/AKS/ma/GradlePlugin/build/jacoco/IT/jacocoIT.exec
15:32:21.025 WARN - Coverage information was not collected. Perhaps you forget to include debug information into compiled classes?
and
15:32:24.681 INFO - Analysing /production/jenkins/AKS/ma/GradlePlugin/build/jacoco/UT/jacocoUT.exec
15:32:24.791 WARN - Coverage information was not collected. Perhaps you forget to include debug information into compiled classes?
15:32:24.792 INFO - Sensor JaCoCoSensor done: 112 ms
Related
I recently tried to include cobertura plugin for our project. But I ran to a strange problem. Now, if I try to do this:
mvn clean package cobertura:check-integration-test
My project assembles just fine. But after deploy here is what I get as a message:
java.lang.NoClassDefFoundError:
net/sourceforge/cobertura/coveragedata/LightClassmapListener
This happens, as I understand, because cobertura alters the bytecode of my classes. So, how should I proceed with making checks and building the code? Right now I come up with the following solution for teamcity:
First step runs cobertura:
mvn clean package cobertura:check-integration-test
After that second step runs the normal assembly:
mvn clean package
This seems to work, but it seems kind of strange, that I have to run the process of packaging twice (this takes double time), just in order to make a clean build with bytecode unaffected by cobertura plugin.
If you use qualinsight-mojo-cobertura, your classes will backuped at instrumentation time, then restored at reporting time (which will in turn result in having non-instrumented classes in the generated package).
Between intrumentation and reporting you run your tests (be it UTs, ITs) as you want (unlike maven-cobertura-plugin, qualinsight-mojo-cobertura does not run tests). For more information, check the project page (especially the Maven phases the plugin is bound to by default.)
We have a windows client that our QA team wrote coded UI tests for. I'm trying to get OpenCover to work so we can see how much of the app their tests are really hitting. They wrote their tests using a custom framework on top of the MSTest framework.
Their framework uses ApplicationUnderTest.Launch to start the application as different users to test security settings. I can capture coverage of the test dlls but the application it self.
I've forced the app to build in 32bit, made sure all pdbs are present in the folders and included the pdb directory in the targetdir as well.
Has anyone else seen this issue? I also tried replacing my batch file with a wrapper exe and got coverage for that but not the main application. Everything is running as the user they are starting the application as who is also an admin on the machine.
The command I'm running is:
C:\Users\kkindt.CORP\AppData\Local\Apps\OpenCover\OpenCover.Console.exe -register "-target:C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" -output:C:\CodeCoverage\CollectionResults\CodedUICover.xml "-targetargs:C:\CodeCoverage\Tests\EllisWinAppTest.dll /Platform:x86 /Framework:framework40 /Tests:LaunchEllisTest"
I strongly suspect it is due to how you are launching the application under test
OpenCover is a .NET profiler and for .NET application to be launched with a profiler attached requires some environment variables to be available to the new process. I suspect that ApplicationUnderTest.Launch does not propagate all the current environment variables and so the profiler does not start and therefore does not report coverage.
To get this to work you should look into using the ApplicationUnderTest.Launch overload that uses a ProcessStartInfo and then you need to propagate the following environment variables
Cor_Profiler
Cor_Enable_Profiling
OpenCover_Profiler_Key
OpenCover_Profiler_Namespace
OpenCover_Profiler_Threshold
OpenCover_Profiler_TraceByTest (if available)
The Cor_* are required by the runtime to launch a profiler and the OpenCover_* entries are to allow the profiler and the host to find each other - a list of these environment variables are available on github
I have solution with a WPF application, its class libraries, WiX installer and numerous MSTest test projects.
When I build the solution the test runner (we are using Visual Studio 2012.3) correctly discovers all the tests and allows us to run them.
Now I have created a Coded UI test project with just 1 CUIT test in it. I have added the project to the solution; because it is a complete app test of the thing the solution builds this seems logically correct to me.
However the default behaviour is the new test projects CUIT test is being discovered by the test runner and so gets run when I run all tests. I do not want this I only want the other (unit) tests to be found and run this way.
Is there a way (test class attributes perhaps) to supress the DISCOVERY of the test by the test runner?
(Note this is a similar question to Testing an WPF app with CodedUI tests, should the coded ui test project share a solution or not? but asking the specific question that is;
If a solution contains some test how can I prevent those tests being discovered by the test runner?
Any solution should still allow the "undiscovered" test to be selected in a lab build for automation of MTM tests.
I simply unload the project until I want to run the tests.
Which projects are unloaded are saved in the .suo file, which can be excluded from source control if your lab build is being done automatically, or you can reload in manually if your lab building is created manually.
I have the following setup
Visual studio 2012 Update 2
NUnit Test Adapter Beta 4
NUnit 2.62
I mark a test with either of the following attributes Category from NUnit or TestCategory from MSTest:
[Category("WebServer")]
public void FooTest() {
//test
}
[TestCategory("WebServer")]
public void FooTest2() {
//test
}
In the TFS Build Template, I Set the property
Basic -> 1. Test Source -> Test Case Filter to the value: TestCategory!=WebServer
When the build executes NO TESTS execute. Removing the Filter and all tests run again.
The output from the build log is
No test is available in C:\Builds\2\Proj\Build\bin\Debug\Tests.Integration.dll C:\Builds\2\Proj\Build\bin\Debug\Tests.Unit.dll C:\Builds\2\Proj\Build\bin\Debug\Tests.Web.dll C:\Builds\2\Proj\Build\bin\Debug\TestStack.BDDfy.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
Any clues how I can get a test to be excluded base on Category name?
I can easily verify the attribute works if I use the NUnit console runner.
When using NUnit Test Adapter v1.1+, test Categories can be used on TFSBuild. You just need to install the package with your Test Project, and configure test case filter on your build definition.
From here it would appear that you cannot use the TestCategory filters on NUnit tests, only on MSTest tests.
As a note, it would also appear that you are changing the property on the TFS Build Definition, not the Build Template. This is what I would expect, as changing the build definition would be the wrong place to change this.
Answering an old question as this was one of the first results Google returned.
From here it would seem that this was a bug that has been fixed in version 1.1, which unfortunately (as of 22 Jan 2014) has not yet been released to nuget
In our build process we're currently using MBUnit 2.x tests, called from an NAnt task, called from CruiseControl.NET. I've uprgaded to Gallio and MBUnit 3.x locally and am able to run tests from VS2008. I'm having trouble getting our build process upgraded. It looks like we have two options, either run the tests from CruiseControl.net or from NAnt. Based on this, I have two questions:
With all other things being equal, where should these tests be run, NAnt or CruiseControl.net, or does it really matter at all?
If you like doing this sort of thing in NAnt, do you have any documentation or examples of how to implement it? I've found documentation for the CC.NET soution in my Gallio\extra\CCNet directory in the Gallio installation.
Thanks
Your options for running tests as part of your build with Gallio are similar to what they were with MbUnit v2.
I recommend running the tests from your NAnt or MSBuild scripts using the provided tasks in Gallio.NAntTasks.dll and Gallio.MSBuildTasks.dll.
However, if you want to run the tests directly from the CCNet configuration, then you can add a CCNet task to execute the Gallio.Echo.exe program.
Here's a bit more info on the NAnt task that someone else blogged about. Some of the details have changed in recent releases but it should get you on the right track.
http://testdrivendevelopment.wordpress.com/2008/12/01/use-nant-to-run-mbunit-tests-using-gallio/