I apply the new man tests to the code I wrote in the pipeline and I get the code covarage as in the picture. My goal is to show this code covarage percentage in the team dashboard. I couldn't find an extension for it.
ScreenShot
I looked under the Azure devops extension but couldn't find any ready extensions.
Hi I suppose that you could refer to this extension Code Coverage Widgets
With this widget, I could display the code coverage of my specific pipeline in the dashboards.
Updated on 12/21
You need to configure this widget with a test run pipeline
=========================================================
update 2
I suppose that you need to add your code coverage report to the build summary with publish code coverage
My yaml pipeline as below. You will need additional argument to generate code coverage file as format Cobertura.
steps:
- task: DotNetCoreCLI#2
displayName: 'dotnet test'
inputs:
command: test
projects: ''
arguments: '--collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura'
publishTestResults: false
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage from **\*\coverage.cobertura.xml'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '**\*\coverage.cobertura.xml'
reportDirectory: '$(System.DefaultWorkingDirectory)\reports'
Related
I want the pipeline to fail if the threshold of failing test is more than 70%, how should I implement it in Azure Devops Pipeline. The test are running with mvn install command and test script.
Update:
Based on your reply, you need to fail the pipeline based on the number of failed tests.
There is no out-of-box method can achieve it in Azure Pipeline.
I suggest that you can check the result based on the Test Result file(XML file) created by Maven test.
You can use the Maven task to run the test and generate the test xml file.
Then you can use PowerShell task to determine if the pipeline can pass.
Here is an example:
steps:
- task: Maven#3
displayName: 'Maven pom.xml'
inputs:
goals: test command
testResultsFiles: '$(build.artifactstagingdirectory/surefire-reports/TEST.xml'
continueOnError: true
- powershell: |
[xml]$xml= Get-Content $(build.artifactstagingdirectory)\surefire-reports\Test.xml
$failures= $xml.testsuite.failures
If($failures -gt xx)
{
Write-host "##vso[task.complete result=failed;]DONE"
}
displayName: 'PowerShell Script'
I just wanted to test how this is supposed to work. So the function in question is just the basic HTTP trigger template from visual studio and pushed to devops repo, the function app is manually made via portal, and I am using classic editor to make my pipelines
So I first build it
pool:
name: Azure Pipelines
steps:
- task: DotNetCoreCLI#2
displayName: 'Build project'
inputs:
projects: '**/*.csproj'
arguments: '--output publish_output --configuration Release'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)/publish_output'
includeRootFolder: false
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
and then use the artifact for the release pipeline
#Your build pipeline references an undefined variable named ‘Parameters.AzureSubscription’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972
#Your build pipeline references an undefined variable named ‘Parameters.AppType’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972
#Your build pipeline references an undefined variable named ‘Parameters.AppName’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972
steps:
- task: AzureFunctionApp#1
displayName: 'Deploy Azure Function App'
inputs:
azureSubscription: '$(Parameters.AzureSubscription)'
appType: '$(Parameters.AppType)'
appName: '$(Parameters.AppName)'
package: '$(System.DefaultWorkingDirectory)/_testcase1-Azure Functions for .NET-CI/drop'
These are the generated YAML so yes I have the subscription, app type, and app name filled in. But anyway here's the classic editor if that say anything more.
To build artifact:
To release:
The thing is, both pipeline runs perfectly fine, they give green light claiming to be successful. It's just that I don't end up with a function in my function app. I have manually uploaded my function from visual studio, using a publishing profile, and then I run this pipeline, it effectively removes the function that I have manually published. I'm guessing there's something wrong with my artifact? That's why it's deploying absolutely nothing, and that's why it removes any existing functions on my app? What else might be the problem here?
Currently, I have a pipeline set up for this .NET Core solution, composed of several project apps, but I'm building each "deliverable project" independently (REST APIs, gRPC APIs, ASP.NET Core MVC apps and so on - one dotnet restore, dotnet build, dotnet test, dotnet publish, docker build and docker push tasks for each executable in this solution). It works.
Each "project sub-pipeline" is an "Agent Job" inside of the main pipeline. It's required by the client that I build the projects with the win-x64 RID and pack each App/API in Windows Containers (dotnet CLI doesn't allow passing a RID to build a whole solution - error NETSDK1134).
The question is: How could I do a single restore/build/test cycle for the whole solution (to improve build time) and only have the publish steps separated for each project?
I'm aware of this question but I think it is a different thing. Could you guys point me to some docs where I could find more info related to my case?
I researched a bit about MSBuild Publish Profiles and custom MSBuild projects but didn't manage to find something similar.
Thank you in advance.
If you want to do a single restore/build/test you need to publish your whole source folder as an artifact then in your artifact you will get your code and build assemblies. Then reuse this artifact for later steps. It could look like:
trigger:
- master
pool:
vmImage: 'windows-latest'
variables:
buildConfiguration: 'Release'
steps:
- task: DotNetCoreCLI#2
inputs:
command: 'restore'
feedsToUse: 'select'
vstsFeed: 'my-vsts-feed' # A series of numbers and letters
- task: DotNetCoreCLI#2
inputs:
command: 'build'
arguments: '--configuration $(buildConfiguration)'
displayName: 'dotnet build $(buildConfiguration)'
- task: DotNetCoreCLI#2
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(System.DefaultWorkingDirectory)'
artifactName: 'solution'
then on deployment job you will get whole solution with restored packages and build code ready for further steps.
Regarding .NET RID as far as I see here you have two options:
A single RID can be set in the element of your project file.
pass it with --runtime to dotnet build
I am using Cypress.io (Version 5.1.0) for testing my project.
My project is in azure DevOps. Now i want to include my cypress tests in Azure DevOps so my tests will run automatically.
I set up the JUnit reporter on my Cypress project:
into my “package.json” file i added
"cypress-multi-reporters": "^1.2.4",
"mocha-junit-reporter": "^1.23.3"
then run
npm install
than added
"scripts": {
"scripts": "cypress run",
"test": "npm run scripts"
}
Into my “cypress.json” file i added
"reporter": "mocha-junit-reporter",
"reporterOptions": {
"mochaFile": "cypress/reports/junit/test-results.[hash].xml",
"testsuitesTitle": false
}
After this I created a new Pipeline using Azure Repos in Azure DevOps.
For Pipeline Configuration i selected Node.js.
Now I have a YAML file. Here i removed npm build from the first script.
Now I picked npm from the assisstant. On the npm configurations, I selected custom and write the command run test . Now I Select the result format as “JUnit” and set Test results files to “*.xml”
At last I selected the option "Merge test results".
Now I saved and run the pipeline.
This is what my Job does:
Pool: Azure Pipelines
Image: ubuntu-latest
Agent: Hosted Agent
Started: Yesterday at 17:31
Expanded: Object
Result: True
Evaluating: not(containsValue(job['steps']['*']['task']['id'], '6d15af64-176c-496d-b583-fd2ae21d4df4'))
Expanded: not(containsValue(Object, '6d15af64-176c-496d-b583-fd2ae21d4df4'))
Result: True
Evaluating: resources['repositories']['self']['checkoutOptions']
Result: Object
Finished evaluating template 'system-pre-steps.yml'
********************************************************************************
Template and static variable resolution complete. Final runtime YAML document:
steps:
- task: 6d15af64-176c-496d-b583-fd2ae21d4df4#1
inputs:
repository: self
MaxConcurrency: 0
What is wrong with my automation? How can I fix this?
Update:
Thats my yml file:
# Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
displayName: 'npm install'
- task: Npm#1
inputs:
command: 'custom'
customCommand: 'run test'
continueOnError: true
- task: PublishTestResults#2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '*.xml'
searchFolder: '$(System.DefaultWorkingDirectory)/cypress/reports/junit'
mergeTestResults: true
testRunTitle: 'Publish Test Results'
I got an email with this Details
Job 1 error(s), 1 warning(s) Error: Npm failed with return code: 254
The issue may be due to the agent rather than your code and scripts.
You can try the following solutions:
Change your agent image. As you are currently using the ubuntu-latest, it is recommanded to try the ubuntu-20.04 or ubuntu-16.04.
Use a self-hosted agent. If you don't have a self-hosted agent, click Self-hosted Linux agent for detailed steps.
Change the orgnization. Choose another organization that can run the build correctly, and just in case, it is better to create a new organization. Then create a new project and try your tests.
As stated already, the problem most likely lies with the Azure environment. Cypress has a dependency on a browser (electron, chrome) in order to execute. For example, if you are using docker, they provide an official image called cypress/browsers:node14.7.0-chrome84 that has everything you need out of the box. The Dockerfile also has useful info on the environment needed. Make sure to provide a headless configuration as well, something like:
cypress run --headless --browser chrome --spec yourSpecHere.js
I've an yaml based azure CI/CT pipeline. I've a task defined as below:
- task: PublishTestResults#2
displayName: 'Publish PCOM Test Results'
condition: always()
inputs:
testResultsFormat: NUnit
testResultsFiles: '*.xml'
searchFolder: '$(Common.TestResultsDirectory)/$(Build.SourceBranchName)/$(Build.BuildNumber)/test-results/PCOM'
mergeTestResults: true
testRunTitle: 'PCOM Tests'
It looks for all '*.xml' files under a particular s3 location. But the xml files will be created only if the below task is successful:
- bash: 'perl run_aws_cli.pl $(Build.BuildNumber) $(Build.SourceBranchName) $(s3Bucket) $(Build.SourcesDirectory)'
workingDirectory: 'azure-pipeline/scripts'
displayName: 'Run ECS Tasks'
So I want a custom condition to check if the later task is successful then only search xml files in the directory else don't search for it.
Currently it's searching always and the earlier task(Publish PCOM Test Results) is failing.
just do condition: succeeded() on your second task. this will only run if all the tasks before that run successfully. if you want a more sophisticated option, check this thread.
Alternatively, you can probably get more information on the build status from the API.
https://learn.microsoft.com/en-us/rest/api/azure/devops/build/builds/get?view=azure-devops-rest-5.1