I'm using mocha to run nodeJS tests. All tests seems to pass, but gitlab runner keeps waiting for something.
Anyone have had similar problem?
My test stage implementation looks like this.
test:
stage: test
script:
### Run with debugging
- mocha -R mocha-pretty-bunyan-nyan build/test/v1/
- mocha -R mocha-pretty-bunyan-nyan build/test/v2/
#- mocha build/test/v1/
I've also tried without the pretty reporter. Result is same. Runner gets stuck and keeps waiting for something.
Found out it is needed to run mocha tests with --exit flag.
Related
I've created around 180 end-to-end tests around a web application. Now I can't afford to run those sequentially. I've tried running it in parallel via the cypress dashboard. But they provide only 500 test runs per month and then it doesn't work in parallel. In my git lab runner, I am seeing this error:
Can anyone suggest how can I run tests in parallel with cypress and GitLab only?
You will need to do some hackings/workarounds hahaha but will work.
First you need to have in your root path the file .gitlab-ci.yml. In your .gitlab-ci.yml define how many parallel jobs do you want, in my case I will use as example two parallel jobs to run the same tests in different browsers(chrome and firefox):
stages:
- triggers
smoke-test-chrome:
stage: triggers
trigger:
include: gitlab-ci/smoke-test-chrome/.gitlab-ci.yml
smoke-test-firefox:
stage: triggers
trigger:
include: gitlab-ci/smoke-test-firefox/.gitlab-ci.yml
Now you need to create the .gitlab-ci.yml for each parallel job in different directories but all included in a main directory called gitlab-ci/
In my example I will create two files with the following path:
gitlab-ci/smoke-test-chrome/.gitlab-ci.yml
gitlab-ci/smoke-test-firefox/.gitlab-ci.yml
In the gitlab-ci/smoke-test-chrome/.gitlab-ci.yml file I will have:
stages:
- triggers
smoke-test-chrome:
image: cypress/browsers:node16.14.0-slim-chrome99-ff97
stage: triggers
script:
- npm ci
- npm run smoke:test-chrome
And in the gitlab-ci/smoke-test-firefox/.gitlab-ci.yml file I will have:
stages:
- triggers
smoke-test-firefox:
image: cypress/browsers:node16.14.0-slim-chrome99-ff97
stage: triggers
script:
- npm ci
- npm run smoke:test-firefox
Last create your specific scripts in your package.json, in my example I created:
"smoke:test-chrome": "cypress run --browser chrome --spec 'cypress/integration/Signup/SmokeTests.test.ts'",
"smoke:test-firefox": "cypress run --browser firefox --spec 'cypress/integration/Signup/SmokeTests.test.ts'",
In your case, you can just change the spec files you want to called in each specific job by changing those specs in package.json script and calling those scripts in the .gitlab-ci.yml files you have created before
We have jest unit testing for our react app and need to set a threshold value of 80% test case coverage. I know that we can get the coverage report in npm test -- --coverage --watchAll=false but I am now tasked with failing the pipeline if the coverage goes below 80%. I saw that there is a test pipeline stage which is commented right now.
I have the following script, I need to somehow get the coverage, and compare if it is 80 or more else fail the pipleline, what should I do
test:
stage: test
image: node:16.13.1
before_script:
- npm i
- npx node -v
- npx npm -v
script:
- echo "running test coverage"
- npm test -- --coverage --watchAll=false
coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
Within your script section you should be able to use a Regex expression to retrieve the coverage value from the report.
Then you can use some bash scripting to remove the % sign from the output and compare the value with your required minimum. If the value is below, simply return false which I think gitlab should interpret as a failure.
If it’s above your minimum value return true and it should pass the job.
I have a test command in my repo that should work when my server is up because the tests interact with the server once it is running.
On my local i use two commands on first terminal npm run dev - this gets the server running and on second terminal i run the command npm run test that runs test which only pass when the first command is running. How do i achieve this in my gitlab CICD test stage job?
currently i am doing this
test_job:
stage: test
script:
- npm run dev
- npm run test
so the pipeline executes npm run dev which doesnt self terminate and my pipeline gets stuck cant seem to find the solution. Help and suggestions are appreciated. Stack is typescript express graphql
You have two options that you can use for this:
If you're building your server into a container prior to this test job, you can run the container as a service, which will allow you to access it via it's service alias. That would look something like this:
test_job:
stage: test
services:
- name: my_test_container:latest
alias: test_container
script:
- npm run test # should hit http://test_container:<port> to access service
You can use the nohup linux command to run your service in the background. This will run the command in the background once it's started up, and it will die when the CI job ends (as part of shutting down the job). That would look like this:
test_job:
stage: test
script:
- nohup npm run dev &
- npm run test
I am trying to show test coverage visualization in Gitlab for our monorepo as described here Test Coverage Visualization
We are using a self-managed gitlab runner with the docker+machine executor hosted on AWS EC2 instances. We are using Gitlab SaaS. The job from the gitlab-ci.yml is below
sdk:
stage: test
needs: []
artifacts:
when: always
reports:
cobertura: /builds/path/to/cobertura/coverage/cobertura-coverage.xml
<<: *main
<<: *tests
<<: *rules
<<: *tags
The line in the script that runs the tests and outputs code coverage...
- npm run test -- --watchAll=false --coverage --coverageReporters=cobertura
The artifact gets saved just fine and looks normal when I download it, but I don't get any visualization as described in the documentation linked above. I just updated the gitlab runner to V14.0.0 thinking that might be the problem, it's not.
I don't have any sort of regex pattern setup, as from my understanding that is only for printing the coverage to stdout.
I'm sure I am overlooking something trivial and I really need a sanity check here as I have already spent way more time on this than I can afford.
The issue was that the regex pattern needed to be set in the repository settings. I had experimented with adding a regex pattern, but it hadn't worked by the time I posted this question because the regex pattern I was using was not correct.
I am running jest test with code coverage in GitLab CI and GitLab captures the percentage from stdout of a runner in gitlab.
jest --coverage produces the coverage in stdout and gitlab captures it using /All files[^|]*\|[^|]*\s+([\d\.]+)/ regexp but when I run
jest --coverage --json --outputFile=xyz.json sadly jest doesn't print the code coverage to stdout.
What can I do to get code coverage in stdout from jest when --json arguments is given to jest?
jest version : v22.4.3 same for jest-cli
The following configuration will let GitLab interpret the coverage report generated by Jest:
stages:
- test
Unit tests:
image: node:12.17.0
stage: test
script:
- jest --coverage
coverage: /All\sfiles.*?\s+(\d+.\d+)/
There's an open issue on GitLab which contains the correct regex for coverage reports generated using Jest (which is used by Create React App).
I'm using the following regex to parse the text-summary coverage reports from Jest for Gitlab: ^(?:Statements|Branches|Functions|Lines)\s*:\s*([^%]+)
Note that Gitlab will only consider the last match though. So above could be written as ^Lines\s*:\s*([^%]+). I included the full example so that you can choose the one that makes the most sense for your project.
The "text-summary" report looks like this in StdOut:
=============================== Coverage summary ===============================
Statements : 80.49% ( 2611/3244 )
Branches : 65.37% ( 923/1412 )
Functions : 76.48% ( 582/761 )
Lines : 80.44% ( 2583/3211 )
================================================================================
Make sure you have included text-summary as a coverage reporter in your jest.config.js:
coverageReporters: ['text-summary', 'lcov', 'cobertura'],
I'm not familiar with Jest, but if you are creating a JSON the simplest way would be to simply cat the JSON then change the regex accordingly