How to Publish Test Results in Azure DevOps - node.js

I have created the below YAML file with the PublishTestResults task to display the mocha test results in Azure DevOps portal
# Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.14.1'
displayName: 'Install Node.js'
- script: |
npm install
mocha test --reporter mocha-junit-reporter
npm test
displayName: 'npm install and test'
- task: PublishTestResults#2
condition: succeededOrFailed()
inputs:
testRunner: JUnit
testResultsFiles: '**/TEST-RESULTS.xml'
- task: ArchiveFiles#2
displayName: 'Archive $(System.DefaultWorkingDirectory)'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/node.zip'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
but whenever I run the build I am getting the following warning
"No test result files matching **/TEST-RESULTS.xml were found."
The main motive is to display the mocha test results separately in a dashboard or a test tab. So that we don't have to check the test task in build process to see the test results.

I encountered same issue,
After trying bunch of things could solve it as below.
Step 1. Update package.json
Include in scripts section:
"test": "cross-env CI=true react-scripts test --env=jsdom
--testPathIgnorePatterns=assets --reporters=default --reporters=jest-junit"
Include below in jest config or jest section in package.json
"jest-junit": {
"suiteNameTemplate": "{filepath}",
"outputDirectory": ".",
"outputName": "junit.xml"
}
Run npm run test locally and see if junit.xml is created.
Step 2. In Azure Pipeline YAML file
Include an Npm task with custom command as below
- task: Npm#1
displayName: 'Run Tests'
inputs:
command: 'custom'
workingDir: "folder where your package.json exists"
customCommand: 'test'
Task to publish Results
- task: PublishTestResults#2
displayName: 'Publish Unit Test Results'
condition: succeededOrFailed()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '**/junit.xml'
mergeTestResults: true
failTaskOnFailedTests: true
testRunTitle: 'React App Test'
Publish Test Results

Related

Playwright pipeline endless loading

Hiall,
I developed a ui test with playwright.
I run this in azure pipeline, when a test is pass, everything is OK, the
steps:
run test - pass
publish artifact - pass
post job - pass
but when i add an error to the test -
await expect(activitiesHeader).toHaveClass("goodclass badclass")
I get an error in pipeline log, it is ok, and reach the timeout too
but the pipeline is stuck in run test step (so the next steps never execute)
my yaml:
task: NodeTool#0
inputs:
versionSpec: '14.x'
displayName: 'Install Node.js'
script: npm install
script: npm ci
script: npm install #playwright/test
script: npx playwright install --with-deps
displayName: "Install browser dependencies"
script: npm i playwright-chromium
displayName: "run the test"
condition: succeeded()
script: npm run azurelogin
task: CopyFiles#1
condition: succeededOrFailed() # Run task even if previous ones fail
inputs:
#leave it empty, the copying is done from the root folder
#sourceFolder: '$(Build.SourcesDirectory)/_daniavander_hydro-test/playwright-report'
contents: |
playwright-report/**
screenshot/**
test-results/**
targetFolder: '$(Build.ArtifactStagingDirectory)'
task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: drop'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'

Cobertura format code coverage report for React App on Azure Devops

I have a react app, created with npx create-react-app command, and configured a yaml based azure devops pipeline for CI/CD. Then I added some simple unit tests.
I added a script task in my yaml pipeline to generate coverage results as follows.
- script: |
cd reactonazure
npm run test -- --coverage --ci --reporters=default --reporters=jest-junit --coverageReporters=cobertura
displayName: 'test code coverage'
But this tasks displays the following output and then task continues to run and never ends. I have to cancel the pipeline run to ultimately stop it. Not sure whats happening. Any ideas?
No tests found related to files changed since last commit.
But when I prefix with CI=true to make the command as follows, then the command works fine and I get test results, but no where I find coverage report(cobertura-coverage.xml).
CI=true npm test -- --reporters=jest-junit --reporters=default --coverageReporters=cobertura
I added a simple script task shown below to list out the files and folders to enable me search for cobertura-coverage.xml. But no where I found the file. Note that jest.xml file is being created. And that is the reason why I am able to see the test results shown above. The problem is with test coverage report.
- script: |
pwd
cd ..
pwd
ls -Rla
displayName: 'list out the files and folders for inspection'
Also when I run the following command on vs code command prompt on my machine(Windows 10), I do see a file named cobertura-coverage.xml in coverage folder.
npm run test -- --coverage --ci --reporters=default --reporters=jest-junit --coverageReporters=cobertura
The full pipeline yaml is pasted below.
trigger:
- master
variables:
azureSubscription: 'AzureServiceConnection'
# Web app name
webAppName: BasicReactApp
# Environment name
environmentName: Dev
# Agent VM image name
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '16.x'
displayName: 'Install Node.js'
- script: |
cd reactonazure
npm install
npm run build --if-present
displayName: 'npm install, build '
- script: |
cd reactonazure
CI=true npm test -- --reporters=jest-junit --reporters=default --coverageReporters=cobertura
displayName: 'test code coverage'
- task: PublishTestResults#2
displayName: "Publish Test Results"
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'reactonazure/junit.xml'
failTaskOnFailedTests: true
mergeTestResults: true
condition: succeededOrFailed()
# The following, two are not functioning as no coverage.cobertura.xml file is being created.
- task: PublishCodeCoverageResults#1
displayName: "Publish code coverage"
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: "reactonazure/$(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml"
reportDirectory: "reactonazure/$(System.DefaultWorkingDirectory)/coverage"
failIfCoverageEmpty: false
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage report'
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Agent.TempDirectory)/**/coverage.cobertura.xml'
failIfCoverageEmpty: true
Are there any other ways to generate report on the pipeline?
The path you provided to the coverage file is incorrect. It should be $(System.DefaultWorkingDirectory)/reactonazure/coverage/cobertura-coverage.xml.
The task would look like this:
- task: PublishCodeCoverageResults#1
displayName: "Publish code coverage"
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: "$(System.DefaultWorkingDirectory)/reactonazure/coverage/cobertura-coverage.xml"
reportDirectory: "$(System.DefaultWorkingDirectory)/reactonazure/coverage"
failIfCoverageEmpty: false
The command to generate the report that I presented earlier is not working. The following is finally working.
CI=true npm run test -- --coverage --watchAll=false --ci --reporters=default --reporters=jest-junit --coverageReporters=cobertura
The full and final yaml after taking into account the suggested answer is below.
trigger:
- master
variables:
azureSubscription: 'AzureServiceConnection'
webAppName: BasicReactApp
environmentName: Dev
vmImageName: 'ubuntu-latest'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '16.x'
displayName: 'Install Node.js'
- script: |
cd reactonazure
npm install
npm run build --if-present
displayName: 'npm install, build '
- script: |
cd reactonazure
CI=true npm run test -- --coverage --watchAll=false --ci --reporters=default --reporters=jest-junit --coverageReporters=cobertura
displayName: 'Test with code coverage'
- task: PublishTestResults#2
displayName: "Publish Test Results"
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'reactonazure/junit.xml'
failTaskOnFailedTests: true
mergeTestResults: true
condition: succeededOrFailed()
- task: PublishCodeCoverageResults#1
displayName: "Publish code coverage"
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: "$(System.DefaultWorkingDirectory)/reactonazure/coverage/cobertura-coverage.xml"
reportDirectory: "$(System.DefaultWorkingDirectory)/reactonazure/coverage"
failIfCoverageEmpty: false
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)/reactonazure/build'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: $(environmentName)
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy'
inputs:
azureSubscription: $(azureSubscription)
#appType: webAppLinux
appType: webApp
appName: $(webAppName)
runtimeStack: 'NODE|16.x'
package: $(Pipeline.Workspace)/drop/$(Build.BuildId).zip
startUpCommand: 'npm run start'

PublishCucumberReport - Can't find output path of cucumber report

The plugin can't find files and generate HTML report on Azure DevOps
Azure Devops plugin - https://marketplace.visualstudio.com/items?itemName=MaciejMaciejewski.azure-pipelines-cucumber
azure-pipelines.yml
jobs:
# Build Electron
- job: UserAcceptanceTest
displayName: E2E-Tests
pool:
name: ado-win-pool
timeoutInMinutes: 120
steps:
- task: CopyFiles#2
inputs:
sourceFolder: $(Build.SourcesDirectory)
targetFolder: $(Build.ArtifactStagingDirectory)
- task: PublishPipelineArtifact#1
displayName: 'Publishing build artifacts'
inputs:
targetPath: $(Build.ArtifactStagingDirectory)
- task: NodeTool#0
displayName: 'Install Node 12.x'
inputs:
versionSpec: 12.x
- task: PublishCucumberReport#1
displayName: 'Publish Cucumber Report'
inputs:
jsonDir: target/results/cucumber/
outputPath: target/results/cucumber/
Actual:
Found 0 matching C:/agent/_work/13/s/target/results/cucumber pattern
##[warning]Error: Not found outputPath: C:\agent\_work\13\s\target\results\cucumber
Finishing: Publish Cucumber Report
Expected:
Found cucumber JSON file.
In your case, the problem is in the path provided for the cucumber report.
It should be a path to the folder with cucumber reports in json format, but not to some particular json.
So, the correct snippet in yaml would be:
- task: PublishCucumberReport#1
displayName: 'Publish Cucumber Report'
inputs:
jsonDir: target/results/cucumber/
outputPath: target/results/cucumber/
In above yaml pipeline, you didnot have the step to run your cucumber test to generate the cucumber JSON file.
If the cucumber JSON file is already existing in your repo. Then the error from PublishCucumberReport task is because the directory target/results/cucumber doesnot exist in your repo.
Then You need to check where the cucumber JSON file is located in your repo and specify the correct path for PublishCucumberReport task.
If there is no cucumber JSON fileexisting in your repo. You should add steps in the yaml pipeline to run your tests.
If you have your test scripts configured in the package.json file, like below(report folder must exist in the repo):
You can just run the npm test to execute your tests and generate the json report in the report folder. See below;
steps:
- task: NodeTool#0
displayName: 'Install Node 12.x'
inputs:
versionSpec: 12.x
- script: |
npm install
npm test
displayName: 'Run tests'
- task: PublishCucumberReport#1
inputs:
jsonDir: report
outputPath: report
If there is no test script defined in your package.json file. You can run the cucumber-js command in the yaml pipeline to generate the json file. See below:
- script: |
#npm install cucumber
npm install
./node_modules/.bin/cucumber-js features -f json:report/cucumber_report.json
displayName: 'Run tests'

Code coverage tab not showing in Azure DevOps

I have a relative simple test project under Azure DevOps and I want to generate code coverage.
This works... kinda. I get this:
I get the files I needed ( I think at least) But the tab is missing.
I have those three steps:
Do .NET test task
Install report generator
Run report generator to convert ( -reporttypes:HtmlInline_AzurePipelines;Cobertura")
publish result (s)
But the tab is not showing up? Any ideas?
- stage: Run_Unit_tests
jobs:
- job: 'Tests'
pool:
vmImage: 'windows-latest'
variables:
buildConfiguration: 'Release'
continueOnError: true
steps:
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- task: DotNetCoreCLI#2
displayName: Test .NET
inputs:
command: test
projects: '**/*Test/*.csproj'
arguments: '--configuration $(buildConfiguration) --logger trx --collect:"XPlat Code Coverage"'
condition: succeededOrFailed()
- task: reportgenerator#4
inputs:
reports: '$(Agent.TempDirectory)\**\coverage.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)\coverlet\reports'
verbosity: 'Verbose'
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(Build.SourcesDirectory)\coverlet\reports\Cobertura.xml
failIfCoverageEmpty: false
reportDirectory: $(Build.SourcesDirectory)\coverlet\reports\
I tried with code generator, without, enable code coverage variable or disable, tried with report generator and without...
I had the same problem, and just pressed F5 and it appeared!
It's mad, but it actually does it consistently.
It seems there's occasionally a bug in the devops front-end code?
You can try below yaml to publish code coverage.
First you need to make sure your project reference to nuget package coverlet.msbuild
<PackageReference Include="coverlet.msbuild" Version="2.5.1">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers</IncludeAssets>
</PackageReference>
Then in your dotnet test task to enable CollectCoverage
arguments: '/p:CollectCoverage=true /p:CoverletOutput=$(Build.SourcesDirectory)\TestResult\ /p:CoverletOutputFormat=cobertura'
Then in reportgenerator task specify the reports folder to the CoverletOutput folder reports: '$(Build.SourcesDirectory)\TestResult\**\coverage.cobertura.xml'
Please check below yaml for reference:
steps:
- task: UseDotNet#2
inputs:
version: 2.2.x
- task: DotNetCoreCLI#2
inputs:
command: restore
projects: '**\*.csproj'
- task: DotNetCoreCLI#2
inputs:
command: custom
custom: tool
arguments: install --tool-path . dotnet-reportgenerator-globaltool
displayName: Install ReportGenerator tool
- task: DotNetCoreCLI#2
displayName: Test .NET
inputs:
command: test
projects: '**\*Test*.csproj'
publishTestResults: false
arguments: '--configuration $(buildConfiguration) /p:CollectCoverage=true /p:CoverletOutput=$(Build.SourcesDirectory)\TestResult\ /p:CoverletOutputFormat=cobertura'
condition: succeededOrFailed()
- task: reportgenerator#4
inputs:
reports: '$(Build.SourcesDirectory)\TestResult\**\coverage.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)\coverlet\reports'
verbosity: 'Verbose'
condition: succeededOrFailed()
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(Build.SourcesDirectory)\coverlet\reports\Cobertura.xml
failIfCoverageEmpty: false
reportDirectory: $(Build.SourcesDirectory)\coverlet\reports\
condition: succeededOrFailed()
You can also refer to this blog.
I had this issue too and tracked it down to having a mixture of VS Test and dotnet test tasks in my pipeline. Once I removed the VS Test task it all worked.
It seems the VS Test task uploads its own code coverage results and the combination of publishing cobertura results + this VS Test task confused Azure Devops.

Waiting for console output in azure pipeline

I am trying to run some unit tests in my azure pipeline but it keeps saying 'Waiting for console output'. This goes on for about an hour until it eventually fails. I do not understand why it can't access the output from console. New to working with Azure/pipelines in general and any help would be appreciated. Thank you.
Here is my yaml file:
pr:
- $(branch)
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.14'
displayName: 'Install Node.js'
- script: |
npm install -g #angular/cli
npm install
displayName: 'Install Angular Dependencies'
- task: Npm#1
displayName: 'Lint Angular'
inputs:
command: custom
customCommand: run lint -- --format=stylish
- script: |
npm run test
displayName: 'Run Unit Tests'
- task: PublishTestResults#2
displayName: 'Publish unit tests results'
condition: succeededOrFailed()
inputs:
searchFolder: $(System.DefaultWorkingDirectory)/src/tests/junit
testRunTitle: Angular
testRunner: JUnit
testResultsFiles: "**/TESTS-*.xml"
- task: PublishCodeCoverageResults#1
displayName: 'Publish unit test code coverage results'
condition: succeededOrFailed()
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(System.DefaultWorkingDirectory)/src/tests/coverage/cobertura-coverage.xml
reportDirectory: $(System.DefaultWorkingDirectory)/src/tests/coverage
failIfCoverageEmpty: true
- script: |
$(npm-script)
displayName: 'Build App'
- script: |
cd $(System.DefaultWorkingDirectory)/
ls -a
cp -r dist $(Build.ArtifactStagingDirectory)/
cp ecosystem.config.js $(Build.ArtifactStagingDirectory)/
cd $(Build.ArtifactStagingDirectory)/
ls -a
# rm -r node_modules
# ls -a
displayName: 'Copy Files to Archive'
- task: ArchiveFiles#2
inputs:
rootFolderOrFile: '$(Build.ArtifactStagingDirectory)/'
includeRootFolder: false
archiveType: 'zip'
archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip' # '$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip'
replaceExistingArchive: true
- script: |
cd $(System.DefaultWorkingDirectory)/
ls -a
cd $(Build.ArtifactStagingDirectory)/
ls -a
displayName: 'Check Files 2'
- task: CopyFiles#2
displayName: 'Copy File to: $(TargetFolder)'
inputs:
SourceFolder: '$(Build.ArtifactStagingDirectory)/'
Contents: '$(Build.ArtifactStagingDirectory)/**/*.zip'
TargetFolder: '$(Build.ArtifactStagingDirectory)/ArtifactsToBePublished'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact: App'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)/ArtifactsToBePublished'
# - script: |
# npm run pre-e2e
# npm run e2e
# displayName: 'Run E2E Tests'
- script: |
pwd
cd $(Build.ArtifactStagingDirectory)/ArtifactsToBePublished
ls -a
displayName: 'See Content'
I can reproduce the same issue with npm run test script. You can have a try running unit tests with Npm task instead of using script task.
- task: Npm#1
displayName: 'Unit Test'
inputs:
command: custom
customCommand: run test -- --watch=false --code-coverage
Or
- task: Npm#1
displayName: 'Test e2e Angular'
inputs:
command: custom
customCommand: run e2e
I came across with this issue today and, based on this hint, I figured out that the browser was preventing some events from happening. So I disabled my ad blocker—in my case, Brave Shield, because I use Brave browser—and the problem is solved.

Resources