Failed Test does not fail task in YAML - azure

Question

I have a build pipeline as shown below
variables:
- name: BuildParameters.solution
value: '**/*.sln'
- name: buildConfiguration
value: Release
name: $(date:yyyyMMdd)$(rev:.r)
trigger: none
resources:
repositories:
- repository: self
type: git
stages:
- stage: __default
jobs:
- job: Job_1
displayName: Agent job 1
pool:
vmImage: windows-2019
steps:
- task: UseDotNet#2
displayName: '.NET Core 3.1.404'
inputs:
version: '3.1.404'
packageType: sdk
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: true
timeoutInMinutes: 0
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration) --verbosity n --collect "Code coverage"
publishTestResults: true
- task: PublishTestResults#2
inputs:
testResultsFormat: XUnit
testResultsFiles: '**/TEST.xml'
mergeTestResults: true
failTaskOnFailedTests: true
I have broken a test on purpose
This test fails, but the publish test results task doesnt fail outright
I get the warning
##[warning]No test result files matching **/TEST.xml were found. xunit
Has anyone ever had this?
Annoyingly if a test fails the Azure pipeline classes this as a partial success which is not ideal. I am trying to get into a situation where if a test fails, my pipeline fails
This pipeline is used to verify pull requests - i.e. do a build of the code in the PR and ensure that all tests pass
When I searched for this there were references to installing .NET 4.6.2 or later, but this is being used inside a build pipeline, so not sure how to do that
Have I missed something in the dot net test task?
Paul

Answer 1

When you set publishTestResults field to true for dotnet test task. See below extract from here.
This option appends --logger trx --results-directory $(Agent.TempDirectory) to the command line arguments.
Above commands will generate the test result in VSTest(trx) format and output the result to folder $(Agent.TempDirectory)
So you need to change the PublishTestResults#2 task like below:
- task: PublishTestResults#2
inputs:
testResultsFormat: VSTest #change the format to VSTest
testResultsFiles: '**/*.trx' #change to find .trx file
mergeTestResults: true
failTaskOnFailedTests: true
searchFolder: '$(Agent.TempDirectory)' #search the .trx file in $(Agent.TempDirectory)
Usually the continueOnError field is set to false by default. And the dotnet test task will fail if there is any test fails, See below:
If the continueOnError field is set to true. The dotnet test task and the pipeline will show Warnig state.
So you can have a try setting the continueOnError field to false, which will fail the task and pipeline if there is any test fails. And there will not be no need to use PublishTestResults task. For publishTestResults is set to true for dotnet test task, the test result will be automatically published by the dotnet test task. See below:
- task: DotNetCoreCLI#2
enabled: true
displayName: dotnet test
continueOnError: false
timeoutInMinutes: 0
inputs:
command: test
projects: '**/*test/*.csproj'
arguments: --configuration $(buildConfiguration) --verbosity n --collect "Code coverage"
publishTestResults: true