Why is test automation only partially successful with Cypress and Azure DevOps? - azure

Question

I am using Cypress.io (Version 5.1.0) for testing my project.
My project is in azure DevOps. Now i want to include my cypress tests in Azure DevOps so my tests will run automatically.
I set up the JUnit reporter on my Cypress project:
into my “package.json” file i added
"cypress-multi-reporters": "^1.2.4",
"mocha-junit-reporter": "^1.23.3"
then run
npm install
than added
"scripts": {
"scripts": "cypress run",
"test": "npm run scripts"
}
Into my “cypress.json” file i added
"reporter": "mocha-junit-reporter",
"reporterOptions": {
"mochaFile": "cypress/reports/junit/test-results.[hash].xml",
"testsuitesTitle": false
}
After this I created a new Pipeline using Azure Repos in Azure DevOps.
For Pipeline Configuration i selected Node.js.
Now I have a YAML file. Here i removed npm build from the first script.
Now I picked npm from the assisstant. On the npm configurations, I selected custom and write the command run test . Now I Select the result format as “JUnit” and set Test results files to “*.xml”
At last I selected the option "Merge test results".
Now I saved and run the pipeline.
This is what my Job does:
Pool: Azure Pipelines
Image: ubuntu-latest
Agent: Hosted Agent
Started: Yesterday at 17:31
Expanded: Object
Result: True
Evaluating: not(containsValue(job['steps']['*']['task']['id'], '6d15af64-176c-496d-b583-fd2ae21d4df4'))
Expanded: not(containsValue(Object, '6d15af64-176c-496d-b583-fd2ae21d4df4'))
Result: True
Evaluating: resources['repositories']['self']['checkoutOptions']
Result: Object
Finished evaluating template 'system-pre-steps.yml'
********************************************************************************
Template and static variable resolution complete. Final runtime YAML document:
steps:
- task: 6d15af64-176c-496d-b583-fd2ae21d4df4#1
inputs:
repository: self
MaxConcurrency: 0
What is wrong with my automation? How can I fix this?
Update:
Thats my yml file:
# Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/javascript
trigger:
- master
pool:
vmImage: 'ubuntu-latest'
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
displayName: 'npm install'
- task: Npm#1
inputs:
command: 'custom'
customCommand: 'run test'
continueOnError: true
- task: PublishTestResults#2
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: '*.xml'
searchFolder: '$(System.DefaultWorkingDirectory)/cypress/reports/junit'
mergeTestResults: true
testRunTitle: 'Publish Test Results'
I got an email with this Details
Job 1 error(s), 1 warning(s) Error: Npm failed with return code: 254

Answer 1

The issue may be due to the agent rather than your code and scripts.
You can try the following solutions:
Change your agent image. As you are currently using the ubuntu-latest, it is recommanded to try the ubuntu-20.04 or ubuntu-16.04.
Use a self-hosted agent. If you don't have a self-hosted agent, click Self-hosted Linux agent for detailed steps.
Change the orgnization. Choose another organization that can run the build correctly, and just in case, it is better to create a new organization. Then create a new project and try your tests.

Answer 2

As stated already, the problem most likely lies with the Azure environment. Cypress has a dependency on a browser (electron, chrome) in order to execute. For example, if you are using docker, they provide an official image called cypress/browsers:node14.7.0-chrome84 that has everything you need out of the box. The Dockerfile also has useful info on the environment needed. Make sure to provide a headless configuration as well, something like:
cypress run --headless --browser chrome --spec yourSpecHere.js