How do we include unit test in .Net azure build pipeline. Unit testing should happen before before the build process and when the unit test is Succeeded only then the build should happen.
You need to build your app first to be able to run unit tests. Or you can just run dotnet test command which makes sure that first your nuget packages are restored, app is build and then unit test run.
Please check this documentation
Here is an example pipeline:
- task: UseDotNet#2
inputs:
version: '5.0.x'
includePreviewVersions: true # Required for preview versions
- task: DotNetCoreCLI#2
displayName: 'dotnet build'
inputs:
command: 'build'
configuration: $(buildConfiguration)
- task: DotNetCoreCLI#2
displayName: 'dotnet test'
inputs:
command: 'test'
arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura'
publishTestResults: true
projects: 'MyTestLibrary' # update with your test project directory
Related
We have a durable function that executes multiple processing steps in a specific order, using the Function Chaining pattern.
Since the Function App has 2 clients, 2 orchestrators and multiple activities, the folder structure of our solution is as shown in the screenshot below
Since all of our code is in a Git Repo in Azure DevOps, we use a CI/CD process with Azure Pipelines, to build the solution for every new PR created, and after a new version of the development branch is created, we have a release pipeline that should continuously deploy the new version to our staging environment in Azure.
The build pipeline looks like this:
variables:
- name: BuildConfiguration
value: Release
trigger:
branches:
include:
- main
- develop
pool:
vmImage: windows-latest
jobs:
- job: 'CI_Job'
displayName: 'Build and Publish Function App'
continueOnError: false
steps:
- checkout: self
clean: true
- task: UseDotNet#2
displayName: 'Install .NET SDK'
inputs:
packageType: 'sdk'
version: '6.x'
- task: DotNetCoreCLI#2
displayName: 'Restore NuGet Packages'
inputs:
command: 'restore'
projects: '**/*.csproj'
- task: DotNetCoreCLI#2
displayName: "Build Solution"
inputs:
command: 'build'
projects: '**/*.csproj'
arguments: '--configuration $(BuildConfiguration) --no-restore'
- task: DotNetCoreCLI#2
displayName: "Run Unit Tests"
inputs:
command: 'test'
projects: 'test/**/*Tests.csproj'
arguments: '--no-restore'
testRunTitle: 'Test Functions'
- task: DotNetCoreCLI#2
displayName: "Publish Solution"
inputs:
command: 'publish'
projects: '**/*.csproj'
publishWebProjects: false
modifyOutputPath: false
zipAfterPublish: true
arguments: --configuration $(BuildConfiguration) --no-restore --no-build --output $(Build.ArtifactStagingDirectory)
- task: PublishBuildArtifacts#1
displayName: "Publish Build Artifacts"
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: '$(Build.Repository.Name)'
publishLocation: 'Container'
The release pipeline looks like this:
The problem is, the deployment seems to run successfully (the logs dont show any erros and even in the Deployment Center in Azure everthings looks good), but there are no functions listed in the Function App in Azure after the deployment is done.
What we have tried so far:
The release pipeline automatically sets the value WEBSITE_RUN_FROM_PACKAGE=1, we also checked that in the Azure portal to verify the value for this setting is as expected.
We also tried to execute a remote build by setting the value SCM_DO_BUILD_DURING_DEPLOYMENT=true in our appsettings in Azure.
When we deploy the Function App with a ZIP deployment from Visual Studio, everything works as expected.
We have another Function App that gets deployed to a deployment slot where the release pipeline does basically the same (except that it does deploy to a slot, not the Function App itself) where everything works fine. But we can't find the issue here.
So my question is what are we doing wrong or what are we not aware of?
Any advice would be highly appreciated!
After taking a deeper look into the artifact created by the build pipeline I could resolve the issue.
Since the production code is in the src folder and the dotnet publish task was not restricted to only publish content inside that folder, I ended up getting a ZIP folder containing all the DLLs of the solution, and inside that another ZIP folder containing the function.json files as well as the bin folder.
After adjusting the dotnet publish task as follows
- task: DotNetCoreCLI#2
displayName: "Publish Solution"
inputs:
command: 'publish'
projects: 'src/**/*.csproj'
publishWebProjects: false
modifyOutputPath: false
zipAfterPublish: true
arguments: --configuration $(BuildConfiguration) --no-restore --no-build --output $(Build.ArtifactStagingDirectory)
I had a much smaller artifact, containing only the function.json files for my functions (inside their corresponding folders) as well as the bin folder containing the other DLLs.
I have a YAML pipeline which doesnt always obey the project build order
- job: Job_1
displayName: Agent job 1
pool:
vmImage: windows-2019
steps:
- task: UseDotNet#2
displayName: '.NET Core 3.1.404'
inputs:
version: '3.1.404'
packageType: sdk
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: falsefo
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration)
publishTestResults: true
This means that for some executions of the pipeline I get the error
Unable to find package NETStandard.Library with version (>= 2.0.3)
This occurs on a project that is not first to build in Visual Studio
Is there anyway of enforcing the build order? Or getting this package?
Paul
Unable to find package NETStandard.Library with version (>= 2.0.3). Is
there anyway of enforcing the build order? Or getting this package?
The problem is that the NETStandard.Library package was not found. I think it has nothing to do with the build order of the projects. NETStandard.Library is a nuget package, you can try to add a nuget restore task to the pipeline.
- task: NuGetCommand#2
inputs:
command: 'restore'
feedsToUse: 'config'
nugetConfigPath: 'nuget.config'
Is there anyway of enforcing the build order? Or getting this package?
You could build/test the specify project at first:
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: falsefo
inputs:
command: test
projects: '**/*test/TheSpecifyProject.csproj'
arguments: --configuration $(buildConfiguration)
publishTestResults: true
Then build/test other projects:
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: falsefo
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration)
publishTestResults: true
Or
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: falsefo
inputs:
command: test
projects: |
**/*test/*.csproj
!**/*test/TheSpecifyProject.csproj
arguments: --configuration $(buildConfiguration)
publishTestResults: true
I have a build pipeline as shown below
variables:
- name: BuildParameters.solution
value: '**/*.sln'
- name: buildConfiguration
value: Release
name: $(date:yyyyMMdd)$(rev:.r)
trigger: none
resources:
repositories:
- repository: self
type: git
stages:
- stage: __default
jobs:
- job: Job_1
displayName: Agent job 1
pool:
vmImage: windows-2019
steps:
- task: UseDotNet#2
displayName: '.NET Core 3.1.404'
inputs:
version: '3.1.404'
packageType: sdk
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: true
timeoutInMinutes: 0
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration) --verbosity n --collect "Code coverage"
publishTestResults: true
- task: PublishTestResults#2
inputs:
testResultsFormat: XUnit
testResultsFiles: '**/TEST.xml'
mergeTestResults: true
failTaskOnFailedTests: true
I have broken a test on purpose
This test fails, but the publish test results task doesnt fail outright
I get the warning
##[warning]No test result files matching **/TEST.xml were found. xunit
Has anyone ever had this?
Annoyingly if a test fails the Azure pipeline classes this as a partial success which is not ideal. I am trying to get into a situation where if a test fails, my pipeline fails
This pipeline is used to verify pull requests - i.e. do a build of the code in the PR and ensure that all tests pass
When I searched for this there were references to installing .NET 4.6.2 or later, but this is being used inside a build pipeline, so not sure how to do that
Have I missed something in the dot net test task?
Paul
When you set publishTestResults field to true for dotnet test task. See below extract from here.
This option appends --logger trx --results-directory $(Agent.TempDirectory) to the command line arguments.
Above commands will generate the test result in VSTest(trx) format and output the result to folder $(Agent.TempDirectory)
So you need to change the PublishTestResults#2 task like below:
- task: PublishTestResults#2
inputs:
testResultsFormat: VSTest #change the format to VSTest
testResultsFiles: '**/*.trx' #change to find .trx file
mergeTestResults: true
failTaskOnFailedTests: true
searchFolder: '$(Agent.TempDirectory)' #search the .trx file in $(Agent.TempDirectory)
Usually the continueOnError field is set to false by default. And the dotnet test task will fail if there is any test fails, See below:
If the continueOnError field is set to true. The dotnet test task and the pipeline will show Warnig state.
So you can have a try setting the continueOnError field to false, which will fail the task and pipeline if there is any test fails. And there will not be no need to use PublishTestResults task. For publishTestResults is set to true for dotnet test task, the test result will be automatically published by the dotnet test task. See below:
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: false
timeoutInMinutes: 0
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration) --verbosity n --collect "Code coverage"
publishTestResults: true
I have a relative simple test project under Azure DevOps and I want to generate code coverage.
This works... kinda. I get this:
I get the files I needed ( I think at least) But the tab is missing.
I have those three steps:
Do .NET test task
Install report generator
Run report generator to convert ( -reporttypes:HtmlInline_AzurePipelines;Cobertura")
publish result (s)
But the tab is not showing up? Any ideas?
- stage: Run_Unit_tests
jobs:
- job: 'Tests'
pool:
vmImage: 'windows-latest'
variables:
buildConfiguration: 'Release'
continueOnError: true
steps:
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- task: DotNetCoreCLI#2
displayName: Test .NET
inputs:
command: test
projects: '**/*Test/*.csproj'
arguments: '--configuration $(buildConfiguration) --logger trx --collect:"XPlat Code Coverage"'
condition: succeededOrFailed()
- task: reportgenerator#4
inputs:
reports: '$(Agent.TempDirectory)\**\coverage.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)\coverlet\reports'
verbosity: 'Verbose'
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(Build.SourcesDirectory)\coverlet\reports\Cobertura.xml
failIfCoverageEmpty: false
reportDirectory: $(Build.SourcesDirectory)\coverlet\reports\
I tried with code generator, without, enable code coverage variable or disable, tried with report generator and without...
I had the same problem, and just pressed F5 and it appeared!
It's mad, but it actually does it consistently.
It seems there's occasionally a bug in the devops front-end code?
You can try below yaml to publish code coverage.
First you need to make sure your project reference to nuget package coverlet.msbuild
<PackageReference Include="coverlet.msbuild" Version="2.5.1">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
Then in your dotnet test task to enable CollectCoverage
arguments: '/p:CollectCoverage=true /p:CoverletOutput=$(Build.SourcesDirectory)\TestResult\ /p:CoverletOutputFormat=cobertura'
Then in reportgenerator task specify the reports folder to the CoverletOutput folder reports: '$(Build.SourcesDirectory)\TestResult\**\coverage.cobertura.xml'
Please check below yaml for reference:
steps:
- task: UseDotNet#2
inputs:
version: 2.2.x
- task: DotNetCoreCLI#2
inputs:
command: restore
projects: '**\*.csproj'
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- task: DotNetCoreCLI#2
displayName: Test .NET
inputs:
command: test
projects: '**\*Test*.csproj'
publishTestResults: false
arguments: '--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutput=$(Build.SourcesDirectory)\TestResult\ /p:CoverletOutputFormat=cobertura'
condition: succeededOrFailed()
- task: reportgenerator#4
inputs:
reports: '$(Build.SourcesDirectory)\TestResult\**\coverage.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)\coverlet\reports'
verbosity: 'Verbose'
condition: succeededOrFailed()
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(Build.SourcesDirectory)\coverlet\reports\Cobertura.xml
failIfCoverageEmpty: false
reportDirectory: $(Build.SourcesDirectory)\coverlet\reports\
condition: succeededOrFailed()
You can also refer to this blog.
I had this issue too and tracked it down to having a mixture of VS Test and dotnet test tasks in my pipeline. Once I removed the VS Test task it all worked.
It seems the VS Test task uploads its own code coverage results and the combination of publishing cobertura results + this VS Test task confused Azure Devops.
I'm trying to use Azure Pipelines for the first time and have finally managed to configure my azure-pipelines.yml file so it runs tests and should automatically deploy if the tests pass. However, I'm receiving this error:
# ASP.NET Core
# Build and test ASP.NET Core projects targeting .NET Core.
# Add steps that run tests, create a NuGet package, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/dotnet-core
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
variables:
buildConfiguration: 'Release'
steps:
- script: dotnet build --configuration $(buildConfiguration)
displayName: 'dotnet build $(buildConfiguration)'
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
inputs:
command: publish
publishWebProjects: True
arguments: '--configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: True
- task: AzureRmWebAppDeployment#4
displayName: 'Azure App Service Deploy: ThermostatTestProject'
inputs:
azureSubscription: <mySubscription>
WebAppName: <myWebAppName>
packageForLinux: '$(build.artifactstagingdirectory)/**/*.zip'
- task: PublishSymbols#2
displayName: 'Publish symbols path'
inputs:
SearchPattern: '**\bin\**\*.pdb'
PublishSymbols: false
continueOnError: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()
I'm using a Mac - does anyone know how I can edit the yml file to make sure that it's Mac compatible and to get rid of this warning? I've been searching all day and cannot find a solution :(
Thank you
Trying to automate deployment with Azure Pipelines but warning due to not using Windows?
That because the task Index Sources & Publish Symbols task was written for Windows only not for Linux. When you execute this build pipeline on the agent ubuntu-latest, you will get that error.
So, this issue is not about Mac compatible but that task not compatible with Linux.
For this issue, MS replied:
Currently we don't support publishing symbols from a Linux machine.
What you could do is use SourceLink to index as usual as part of your
build and then have a job that runs on Windows to publish the symbols.
When we look at support *.snupkg packaging we will look to address the
Linux symbol publishing scenario.
Ticket here: https://github.com/MicrosoftDocs/vsts-docs/issues/3041
Besides, if you do not need to publish symbols from a Linux machine, you can disable this task.
Hope this helps.