I have set up a cron job for my cypress tests however I want to run it on a specific test alone.
This is my cron job:
name: Cypress Tests
on: [push]
jobs:
cypress-run:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
# Install NPM dependencies, cache them correctly
# and run all Cypress tests
- name: Cypress run
uses: cypress-io/github-action#v4.x.x # use the explicit version number
with:
build: npm run build
start: npm start
I just want to run this one spec file:
Is there a way to run this spec file alone?
Try this out!
Add new command in script section of package.json. Use file or folder here.
"scripts": {
"cypress:spec-run": "cypress run --browser chrome --spec test.spec.js"
},
Run the commond in respective crone job
npm run cypress:spec-run
Related
I've been running my end to end script for 3+ hours and i want to shortened it by running it on parallel. I've researched a ton of hours but i can't do it properly cypress cloud is supporting parallelization but this is not my option since it's paid service
I also tried this npm cypress-gitlab-parallel-runner but no luck it runs in series
I also tried it to do manually with different job in the same stage but it runs sequentially
Heres the snippet:
stages:
- build
- test
build:
stage: build
image: cypress/browsers:node18.12.0-chrome107
script:
- npm ci && npm install
test1:
image: cypress/browsers:node18.12.0-chrome107
stage: test
script:
- npm ci
- npm run test-run1
test2:
image: cypress/browsers:node18.12.0-chrome107
stage: test
script:
- npm ci
- npm run test-run2
How can I run an e2e script on parallel?
I am using yarn workspaces (yarn version 1.22.19) and I would like to run tests for all workspaces, without stopping even if tests fail for one of the workspaces.
This is so I can collect all failing tests across all workspaces in one run. I'm running the tests on a github action.
I am running the following command:
yarn workspaces run test --passWithNoTests
All workspaces have a test script in the package.json that runs the tests with Jest.
Jest returns an exit code of 1 when tests fail. This causes the yarn workspaces run command to fail and stop. I would like it to continue and fail only after running tests for all workspaces.
How can I make the yarn workspaces run continue even if tests fail for one of the workspaces, yet still have it fail at the end?
Edit:
I am running bash.
Using workarounds like set -e or || true might help swallow the error, but I do want the command to fail ultimately, I just want it to fail after running all tests.
For example:
Say I have 3 workspaces - workspace a, workspace b and workspace c. All of them have the following script in their package.json:
test: "jest"
Say tests pass for workspace a and workspace c, but fail for workspace b. My desired result is that running yarn workspaces run test will run tests for all workspaces (and not stop after tests fail for workspace b) but for it to fail after running all tests.
Here is my github workflow. It just installs dependencies and runs the test script which runs the command yarn workspaces run test --passWithNoTests.
name: Run All Tests
on:
pull_request:
branches: ['develop']
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
steps:
- uses: actions/checkout#v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Run tests
run: yarn test
For future reference, this is what I ended up doing:
Add a test:ci script to all package.json file with the following defintion:
"test:ci": "jest --ci --reporters=jest-junit --reporters=default --passWithNoTests || true"
This makes it so the command passes even if tests fails.
Use a jest-junit test reporter to output an xml with test results.
In the action, run yarn workspaces run test which runs tests for all workspaces (packages).
Use the dorny/test-reporter#v1 to collect all test result xml files into a nice view.
Set fail-on-error: 'true' for the action dorny/test-reporter#v1 which will make the step fail if any test failed.
Here's the full github workflow:
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs
name: Build & Test
on:
pull_request:
branches: ['develop']
# cancel any previous runs that are still in progress if a new commit is pushed
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [16.x]
steps:
- uses: actions/checkout#v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: yarn install --frozen-lockfile
- name: Run tests
run: yarn test:ci
- name: Unlink all symbolic links # so we don't go over the same file twice
if: success() || failure() # run this step even if previous step failed
run: find node_modules -type l -exec unlink {} \;
- name: Test Report
uses: dorny/test-reporter#v1
if: success() || failure() # run this step even if previous step failed
with:
name: Jest Test Results # Name of the check run which will be created
path: '**/jest-junit.xml'
reporter: jest-junit # Format of test results
list-suites: 'failed'
list-tests: 'failed'
fail-on-error: 'true'
This achieves everything I was looking for - running all tests for all workspaces, even if some fail, while still failing the workflow.
I've created around 180 end-to-end tests around a web application. Now I can't afford to run those sequentially. I've tried running it in parallel via the cypress dashboard. But they provide only 500 test runs per month and then it doesn't work in parallel. In my git lab runner, I am seeing this error:
Can anyone suggest how can I run tests in parallel with cypress and GitLab only?
You will need to do some hackings/workarounds hahaha but will work.
First you need to have in your root path the file .gitlab-ci.yml. In your .gitlab-ci.yml define how many parallel jobs do you want, in my case I will use as example two parallel jobs to run the same tests in different browsers(chrome and firefox):
stages:
- triggers
smoke-test-chrome:
stage: triggers
trigger:
include: gitlab-ci/smoke-test-chrome/.gitlab-ci.yml
smoke-test-firefox:
stage: triggers
trigger:
include: gitlab-ci/smoke-test-firefox/.gitlab-ci.yml
Now you need to create the .gitlab-ci.yml for each parallel job in different directories but all included in a main directory called gitlab-ci/
In my example I will create two files with the following path:
gitlab-ci/smoke-test-chrome/.gitlab-ci.yml
gitlab-ci/smoke-test-firefox/.gitlab-ci.yml
In the gitlab-ci/smoke-test-chrome/.gitlab-ci.yml file I will have:
stages:
- triggers
smoke-test-chrome:
image: cypress/browsers:node16.14.0-slim-chrome99-ff97
stage: triggers
script:
- npm ci
- npm run smoke:test-chrome
And in the gitlab-ci/smoke-test-firefox/.gitlab-ci.yml file I will have:
stages:
- triggers
smoke-test-firefox:
image: cypress/browsers:node16.14.0-slim-chrome99-ff97
stage: triggers
script:
- npm ci
- npm run smoke:test-firefox
Last create your specific scripts in your package.json, in my example I created:
"smoke:test-chrome": "cypress run --browser chrome --spec 'cypress/integration/Signup/SmokeTests.test.ts'",
"smoke:test-firefox": "cypress run --browser firefox --spec 'cypress/integration/Signup/SmokeTests.test.ts'",
In your case, you can just change the spec files you want to called in each specific job by changing those specs in package.json script and calling those scripts in the .gitlab-ci.yml files you have created before
I have a very simple config in order to run e2e tests with Cypress using Github Actions in a Nextjs app. When it reaches the npm start command, although it seems to work since it gives the correct output: > Ready on http://localhost:3000, the step stays in pending state without ever advancing to the next step.
Any suggestions on how to fix this?
Following github actions config (.github/workflows/nodejs.yml):
name: Node CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [8.x, 10.x, 12.x]
steps:
- uses: actions/checkout#v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: npm install, build, and test
run: |
npm ci
npm run build --if-present
npm start
npx wait-on http://localhost:3000
env:
CI: true
- name: Run Cypress
run: |
npx cypress run
env:
CI: true
Using the control operator & works for me. Try the following.
- name: npm install, build, and test
run: |
npm ci
npm run build --if-present
npm start & npx wait-on http://localhost:3000
env:
CI: true
From man bash
If a command is terminated by the control operator &, the shell executes the command in
the background in a subshell. The shell does not wait for the command to finish, and the
return status is 0. These are referred to as asynchronous commands. Commands separated
by a ; are executed sequentially; the shell waits for each command to terminate in turn.
The return status is the exit status of the last command executed.
I started working on my first CodePipeline with node.js app which is hosted on github. I would like to create simple pipe as follow:
Github repo triggers pipe
Test env (Elastic Beanstalk app) is built from S3 .zip file
Test env runs npm test and npm lint
If everything is OK then QA env (another EB app) is built
For above pipe I've created .config files under .ebextensions directory:
I would like to use npm install --production for QA and PROD env, but it seems that EC2 can't find node nor npm. I checked logs and EC2 triggered npm install by default in temporary folder, then it fails on my first script and app catalogue is always empty.
container_commands:
install-dev:
command: "npm install"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
install-prod:
command: "npm install --production"
test: "[ \"$NODE_ENV\" != \"TEST\" ]"
ignoreErrors: false
Is it posible to run unit tests and linting without jenkins?
container_commands:
lint:
command: "npm run lint"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
test:
command: "npm run test"
test: "[ \"$NODE_ENV\" = \"TEST\" ]"
ignoreErrors: false
I set NODE_ENV for each Elastic Beanstalk instance. No matter what I will do every time my pipe fails because of npm is not recognized, but how is it possible if I'm running 64bit Amazon Linux with node.js ? What's more I cannot find any examples about CodePipeline with node.js in AWS Docs. Thanks in advance!
If you're using AWS for CI/CD, you can use CodeBuild. However, Github provides a great feature called Actions for running Unit Tests, which I find much simpler than AWS. Anyway, I will walk you through both examples:
Using AWS for running Unit Tests
Essentially, you could create a new stage into your CodePipeline, and configure the CodeBuild for running Unit Tests, e.g.
First, add a buildspec.yml file in the root folder of your app so you can use the following example:
version: 0.2
phases:
install:
runtime-versions:
nodejs: 10
commands:
- echo Installing Mocha globally...
- npm install -g mocha
pre_build:
commands:
- echo Installing dependencies...
- npm install
- npm install unit.js
build:
commands:
- echo Build started on `date`
- echo Run Unit Tests and so on
- npm run test
- npm run lint
post_build:
commands:
- echo Build completed on `date`
# THIS IS OPTIONAL
artifacts:
files:
- app.js
- package.json
- src/app/*
- node_modules/**/*
You can find everything you need in the BackSpace Academy, this course is for free:
AWS DevOps CI/CD - CodePipeline, Elastic Beanstalk and Mocha
Using Github for running Unit Tests
You could create your custom actions using Github, it will automatically set up everything you need in your root folder, e.g.
After choosing the appropriate workflow, it will automatically generate a folder/file ".github > workflow > nodejs.yml".
So it will look like this:
name: Node CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [8.x, 10.x, 12.x]
steps:
- uses: actions/checkout#v1
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v1
with:
node-version: ${{ matrix.node-version }}
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
npm test
env:
CI: true
I hope you could find everything you need in this answer. Cheers
Have you incorporated CodeBuild into your pipeline?
You should
1) Create a pipeline whose source is your github account. Go through the setup procedure so that commits on a particular branch trigger the Codepipeline
2) Create a test stage in your Codepipeline which leverages the CodeBuild service. In order to run your Node tests, you might need to provide a configured build environment. And you probably also need to provide a build spec file that specifies the tests to run etc.
3) Assuming that the test stage passes, you can determine if the pipeline continues to another stage which is linked to an elasticbeanstalk app environment which supports the Node platform. These environments are purely for artifacts that have passed testing, so I see no need to have the .ebextensions commands written above.
Have a read of what CodeBuild can do to help you run tests for Node,
https://aws.amazon.com/codebuild/
Good luck!