Code coverage tab not showing in Azure DevOps - azure

I have a relative simple test project under Azure DevOps and I want to generate code coverage.
This works... kinda. I get this:
I get the files I needed ( I think at least) But the tab is missing.
I have those three steps:
Do .NET test task
Install report generator
Run report generator to convert ( -reporttypes:HtmlInline_AzurePipelines;Cobertura")
publish result (s)
But the tab is not showing up? Any ideas?
- stage: Run_Unit_tests
jobs:
- job: 'Tests'
pool:
vmImage: 'windows-latest'
variables:
buildConfiguration: 'Release'
continueOnError: true
steps:
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- task: DotNetCoreCLI#2
displayName: Test .NET
inputs:
command: test
projects: '**/*Test/*.csproj'
arguments: '--configuration $(buildConfiguration) --logger trx --collect:"XPlat Code Coverage"'
condition: succeededOrFailed()
- task: reportgenerator#4
inputs:
reports: '$(Agent.TempDirectory)\**\coverage.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)\coverlet\reports'
verbosity: 'Verbose'
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(Build.SourcesDirectory)\coverlet\reports\Cobertura.xml
failIfCoverageEmpty: false
reportDirectory: $(Build.SourcesDirectory)\coverlet\reports\
I tried with code generator, without, enable code coverage variable or disable, tried with report generator and without...

I had the same problem, and just pressed F5 and it appeared!
It's mad, but it actually does it consistently.
It seems there's occasionally a bug in the devops front-end code?

You can try below yaml to publish code coverage.
First you need to make sure your project reference to nuget package coverlet.msbuild
<PackageReference Include="coverlet.msbuild" Version="2.5.1">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
Then in your dotnet test task to enable CollectCoverage
arguments: '/p:CollectCoverage=true /p:CoverletOutput=$(Build.SourcesDirectory)\TestResult\ /p:CoverletOutputFormat=cobertura'
Then in reportgenerator task specify the reports folder to the CoverletOutput folder reports: '$(Build.SourcesDirectory)\TestResult\**\coverage.cobertura.xml'
Please check below yaml for reference:
steps:
- task: UseDotNet#2
inputs:
version: 2.2.x
- task: DotNetCoreCLI#2
inputs:
command: restore
projects: '**\*.csproj'
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- task: DotNetCoreCLI#2
displayName: Test .NET
inputs:
command: test
projects: '**\*Test*.csproj'
publishTestResults: false
arguments: '--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutput=$(Build.SourcesDirectory)\TestResult\ /p:CoverletOutputFormat=cobertura'
condition: succeededOrFailed()
- task: reportgenerator#4
inputs:
reports: '$(Build.SourcesDirectory)\TestResult\**\coverage.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)\coverlet\reports'
verbosity: 'Verbose'
condition: succeededOrFailed()
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(Build.SourcesDirectory)\coverlet\reports\Cobertura.xml
failIfCoverageEmpty: false
reportDirectory: $(Build.SourcesDirectory)\coverlet\reports\
condition: succeededOrFailed()
You can also refer to this blog.

I had this issue too and tracked it down to having a mixture of VS Test and dotnet test tasks in my pipeline. Once I removed the VS Test task it all worked.
It seems the VS Test task uploads its own code coverage results and the combination of publishing cobertura results + this VS Test task confused Azure Devops.

Related

dotnet test case does not obey solution build order

I have a YAML pipeline which doesnt always obey the project build order
- job: Job_1
displayName: Agent job 1
pool:
vmImage: windows-2019
steps:
- task: UseDotNet#2
displayName: '.NET Core 3.1.404'
inputs:
version: '3.1.404'
packageType: sdk
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: falsefo
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration)
publishTestResults: true
This means that for some executions of the pipeline I get the error
Unable to find package NETStandard.Library with version (>= 2.0.3)
This occurs on a project that is not first to build in Visual Studio
Is there anyway of enforcing the build order? Or getting this package?
Paul
Unable to find package NETStandard.Library with version (>= 2.0.3). Is
there anyway of enforcing the build order? Or getting this package?
The problem is that the NETStandard.Library package was not found. I think it has nothing to do with the build order of the projects. NETStandard.Library is a nuget package, you can try to add a nuget restore task to the pipeline.
- task: NuGetCommand#2
inputs:
command: 'restore'
feedsToUse: 'config'
nugetConfigPath: 'nuget.config'
Is there anyway of enforcing the build order? Or getting this package?
You could build/test the specify project at first:
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: falsefo
inputs:
command: test
projects: '**/*test/TheSpecifyProject.csproj'
arguments: --configuration $(buildConfiguration)
publishTestResults: true
Then build/test other projects:
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: falsefo
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration)
publishTestResults: true
Or
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: falsefo
inputs:
command: test
projects: |
**/*test/*.csproj
!**/*test/TheSpecifyProject.csproj
arguments: --configuration $(buildConfiguration)
publishTestResults: true

Failed Test does not fail task in YAML

I have a build pipeline as shown below
variables:
- name: BuildParameters.solution
value: '**/*.sln'
- name: buildConfiguration
value: Release
name: $(date:yyyyMMdd)$(rev:.r)
trigger: none
resources:
repositories:
- repository: self
type: git
stages:
- stage: __default
jobs:
- job: Job_1
displayName: Agent job 1
pool:
vmImage: windows-2019
steps:
- task: UseDotNet#2
displayName: '.NET Core 3.1.404'
inputs:
version: '3.1.404'
packageType: sdk
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: true
timeoutInMinutes: 0
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration) --verbosity n --collect "Code coverage"
publishTestResults: true
- task: PublishTestResults#2
inputs:
testResultsFormat: XUnit
testResultsFiles: '**/TEST.xml'
mergeTestResults: true
failTaskOnFailedTests: true
I have broken a test on purpose
This test fails, but the publish test results task doesnt fail outright
I get the warning
##[warning]No test result files matching **/TEST.xml were found. xunit
Has anyone ever had this?
Annoyingly if a test fails the Azure pipeline classes this as a partial success which is not ideal. I am trying to get into a situation where if a test fails, my pipeline fails
This pipeline is used to verify pull requests - i.e. do a build of the code in the PR and ensure that all tests pass
When I searched for this there were references to installing .NET 4.6.2 or later, but this is being used inside a build pipeline, so not sure how to do that
Have I missed something in the dot net test task?
Paul
When you set publishTestResults field to true for dotnet test task. See below extract from here.
This option appends --logger trx --results-directory $(Agent.TempDirectory) to the command line arguments.
Above commands will generate the test result in VSTest(trx) format and output the result to folder $(Agent.TempDirectory)
So you need to change the PublishTestResults#2 task like below:
- task: PublishTestResults#2
inputs:
testResultsFormat: VSTest #change the format to VSTest
testResultsFiles: '**/*.trx' #change to find .trx file
mergeTestResults: true
failTaskOnFailedTests: true
searchFolder: '$(Agent.TempDirectory)' #search the .trx file in $(Agent.TempDirectory)
Usually the continueOnError field is set to false by default. And the dotnet test task will fail if there is any test fails, See below:
If the continueOnError field is set to true. The dotnet test task and the pipeline will show Warnig state.
So you can have a try setting the continueOnError field to false, which will fail the task and pipeline if there is any test fails. And there will not be no need to use PublishTestResults task. For publishTestResults is set to true for dotnet test task, the test result will be automatically published by the dotnet test task. See below:
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: false
timeoutInMinutes: 0
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration) --verbosity n --collect "Code coverage"
publishTestResults: true

Azure Devops Build Pipeline for solution with multiple project types

I have a solution that contains the following
Several Asp.net Projects (Microservices and Gateway)
.Net Core + Angular 8 (Front End)
When I hit the build button in Visual Studio every project is built. I have created a repo and uploaded the solution. Now I'm trying to figure out how to setup the pipeline to build each microservice and deploy them to individual Azure Web Apps.
I have the following Pipeline for the Angular Project. Should I define separate tasks like this? Is there a way to replicate the Visual Studio build here?
# ASP.NET
# Build and test ASP.NET projects.
# Add steps that publish symbols, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/apps/aspnet/build-aspnet-4
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
solution: '**/*.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: Npm#1
inputs:
command: install
workingDir: 'd:\a\1\s\Ok.Web\ClientApp'
- task: Npm#1
inputs:
command: custom
workingDir: 'd:\a\1\s\Ok.Web\ClientApp'
customCommand: run build
- task: CopyFiles#2
inputs:
targetFolder: '$(Build.ArtifactStagingDirectory)'
- task: PublishBuildArtifacts#1
You may apply one of two approaches here:
One pipeline for whole repo
One pipeline for project
In both cases you may use templates to avoid repeating yourself; so you will define common tasks for building a .NET project and then use them in pipelines. I recently made a blog post about this, but please take a look at the documentation too.
To achieve this you need to first define a YAML file with common steps. For instance:
parameters:
- name: buildConfiguration # name of the parameter; required
default: 'Release'
- name: projectFolder
default: ' '
steps:
- task: DotNetCoreCLI#2
displayName: Restore nuget packages
inputs:
command: restore
projects: '*.csproj'
workingDirectory: '${{ parameters.projectFolder}}'
- task: DotNetCoreCLI#2
displayName: Build
inputs:
command: build
projects: '${{ parameters.projectFolder}}'
arguments: '--configuration ${{ parameters.buildConfiguration }}'
# You just added coverlet.collector to use 'XPlat Code Coverage'
- task: DotNetCoreCLI#2
displayName: Test
inputs:
command: test
projects: '*Tests/*.csproj'
arguments: '--configuration ${{ parameters.buildConfiguration }} --collect:"XPlat Code Coverage" -- RunConfiguration.DisableAppDomain=true'
workingDirectory: '${{ parameters.projectFolder}}'
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- script: ./reportgenerator -reports:$(Agent.TempDirectory)/**/coverage.cobertura.xml -targetdir:${{ parameters.projectFolder}}/coverlet/reports -reporttypes:"Cobertura"
displayName: Create reports
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: ${{ parameters.projectFolder}}/coverlet/reports/Cobertura.xml
And then invoke this file from you main build file:
variables:
buildConfiguration: 'Release'
projectFolder: 'path to your project'
steps:
- template: build-and-test.yaml
parameters:
buildConfiguration: $(buildConfiguration)
- script: echo Some steps to create artifacts!
displayName: 'Run a one-line script'
In approach number 1 you will build all projects even if you change just one project, so I would recommend you use approach number 2. For this you may define multiple pipelines (one per project) and limit triggers to changes in specific folder. Please take a look here.
Here you have an example of how you can limit triggers to specific folder for master branch:
trigger:
branches:
include:
- master
paths:
include:
- gated-checkin-with-template/*
exclude:
- gated-checkin-with-template/azure-pipelines-gc.yml

How to get nUnit unit tests to work with Xamarin.Forms in Azure pipelines?

I'm trying to get nUnit unit tests working with Xamarin.Forms. Everything works fine on my machine, but when I push to Azure and it goes into the build pipeline, I get the following error :
The current .NET SDK does not support targeting .NET Core 3.1. Either target .NET Core 2.2 or lower, or use a version of the .NET SDK that supports .NET Core 3.1 [/Users/runner/runners/2.164.6/work/1/s/TestProject1/TestProject1.csproj
When I remove the unit test project, the build is successful. I think the problem has something to do with the fact that Xamarin.Forms uses .Net Standard, while my unit test project is using .NET Core. How do I reconcile this? Specific advice or a pointer to a tutorial that goes over unit testing with Xamarin.Forms on Azure would be greatly appreciated. I'm really new to playing around with devops, so some kind of training track would be nice.
Also, here is my Azure Pipeline YAML file :
pool:
vmImage: 'macOS 10.13'
steps:
# To manually select a Xamarin SDK version on the Hosted macOS agent, enable this script with the SDK version you want to target
# https://go.microsoft.com/fwlink/?linkid=871629
- script: sudo $AGENT_HOMEDIRECTORY/scripts/select-xamarin-sdk.sh 5_4_1
displayName: 'Select Xamarin SDK version'
enabled: false
- task: NuGetToolInstaller#0
- task: NuGetCommand#2
inputs:
restoreSolution: '**/*.sln'
- task: XamariniOS#2
inputs:
solutionFile: '**/*.sln'
configuration: 'Release'
buildForSimulator: true
packageApp: false
Thanks!
As you already mentioned the unit test project is a .NET Core project and thus has to be executed with .NET Core. In a YAML build definition you can do this as follows:
- task: UseDotNet#2
displayName: 'Install .NET Core SDK'
inputs:
version: 3.1.x
performMultiLevelLookup: true
- task: DotNetCoreCLI#2
displayName: Build Tests
inputs:
command: 'build'
projects: '**\*Test.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: Run Tests
inputs:
command: 'test'
projects: '**\*Test.csproj'
arguments: '--configuration Release'
Here an example of a complete build pipeline definition to build a Xamarin Forms application with unit tests:
trigger:
branches:
include:
- '*'
pool:
vmImage: 'windows-2019'
steps:
- task: NuGetToolInstaller#0
displayName: Instal Nuget
inputs:
checkLatest: true
- task: NuGetCommand#2
displayName: Restore Nuget Packages
inputs:
restoreSolution: '**/*.sln'
- task: UseDotNet#2
displayName: 'Install .NET Core SDK'
inputs:
version: 3.1.x
performMultiLevelLookup: true
- task: DotNetCoreCLI#2
displayName: Build Tests
inputs:
command: 'build'
projects: '**\*Test.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: Run Tests
inputs:
command: 'test'
projects: '**\*Test.csproj'
arguments: '--configuration Release'
- task: XamarinAndroid#1
displayName: Build Android App
inputs:
projectFile: '**/*Android*.csproj'
outputDirectory: '$(build.binariesDirectory)/Release'
configuration: 'Release'
Guess the answer is a little too late for combodev1, but maybe it helps someone else who ends up here.

Trying to automate deployment with Azure Pipelines but warning due to not using Windows?

I'm trying to use Azure Pipelines for the first time and have finally managed to configure my azure-pipelines.yml file so it runs tests and should automatically deploy if the tests pass. However, I'm receiving this error:
# ASP.NET Core
# Build and test ASP.NET Core projects targeting .NET Core.
# Add steps that run tests, create a NuGet package, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/dotnet-core
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
buildConfiguration: 'Release'
steps:
- script: dotnet build --configuration $(buildConfiguration)
displayName: 'dotnet build $(buildConfiguration)'
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
inputs:
command: publish
publishWebProjects: True
arguments: '--configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: True
- task: AzureRmWebAppDeployment#4
displayName: 'Azure App Service Deploy: ThermostatTestProject'
inputs:
azureSubscription: <mySubscription>
WebAppName: <myWebAppName>
packageForLinux: '$(build.artifactstagingdirectory)/**/*.zip'
- task: PublishSymbols#2
displayName: 'Publish symbols path'
inputs:
SearchPattern: '**\bin\**\*.pdb'
PublishSymbols: false
continueOnError: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()
I'm using a Mac - does anyone know how I can edit the yml file to make sure that it's Mac compatible and to get rid of this warning? I've been searching all day and cannot find a solution :(
Thank you
Trying to automate deployment with Azure Pipelines but warning due to not using Windows?
That because the task Index Sources & Publish Symbols task was written for Windows only not for Linux. When you execute this build pipeline on the agent ubuntu-latest, you will get that error.
So, this issue is not about Mac compatible but that task not compatible with Linux.
For this issue, MS replied:
Currently we don't support publishing symbols from a Linux machine.
What you could do is use SourceLink to index as usual as part of your
build and then have a job that runs on Windows to publish the symbols.
When we look at support *.snupkg packaging we will look to address the
Linux symbol publishing scenario.
Ticket here: https://github.com/MicrosoftDocs/vsts-docs/issues/3041
Besides, if you do not need to publish symbols from a Linux machine, you can disable this task.
Hope this helps.

Resources