Jest test coverage does not fail when threshold is not met - jestjs

Utilizing create-react-app, when running tests in my CI pipeline, if the code coverage thresholds are not met, I expect the console to return a non-zero response.
package.json
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"test:coverage": "npm run test -- --coverage --watchAll=false",
},
"jest": {
"collectCoverageFrom": [
"src/components/**/*.js",
"src/state/**/*.js",
"src/templates/**/*.js",
"src/routes/**/*.js"
],
"coverageThreshold": {
"global": {
"branches": 80,
"functions": 80,
"lines": 80,
"statements": 80
}
}
}
When running test:coverage the console reports that thresholds were not met, but still returns 0. My understanding from the Jest documentation is that an error should be returned when coverage thresholds are not met.
https://jestjs.io/docs/en/configuration#coveragethreshold-object
Specifically...
If thresholds aren't met, jest will fail.
Is anyone familiar with this issue? I have been through Jest and CRA github issues with mixed results and most findings are related to outdated versions.

To stop further execution when command fails:
command || exit 0
{
"test:coverage": "npm run test -- --coverage --watchAll=false || exit 0"
}
ref: don't fail jenkins build if execute shell fails

Add "collectCoverage": true, in your jest configuration. After that jest run operation will fail, if the coverage is not met.

Defining the config in the packjage.json file did not fail the jest command when coverage threshold was not met. But it worked when I defined it through the jest.config.js file!

This will make the test fail if the conditions are not met, but please note that this will slow down your tests:
"scripts": {
"test": "jest --collectCoverage=true",
"coverage" : "jest --coverage"
},

Related

Unit test count is not shown on the sonarqube dashboard

I am using nodejs Project repository and the test cases are written in the Jasmine framework. On sonarqube dashboard, I cant see the unit test count block.
Sonar-template-configurations was written as below:
sonar.projectKey={{sonar_projectKey}}
sonar.projectName={{sonar_projectName}}
sonar.host.url={{{sonar_host_url}}}
sonar.javascript.environments = node
# Source code for analysis
sonar.sources=.
sonar.tests = .
sonar.test.inclusions = **/*.spec.js
# Exclusions
sonar.exclusions=test/**
# Reports
sonar.ts.tslint.configPath=tslint.json
sonar.javascript.lcov.reportPaths=coverage/lcov.info
sonar.javascript.coverage.lcovReportPath=coverage/lcov.info
sonar.junit.reportsPath= unit_test_results.xml
sonar.testExecutionReportPaths=unit_test_results.xml
sonar.log.level=DEBUG
sonar.verbose=true
Script used in package.json as below for generating report and running test cases:
"scripts": {
"start": "nodemon --verbose --ignore '/meta.conf/**' --ignore '/data/**' app.js",
"test": "nyc ./node_modules/jasmine-xml-reporter/bin/jasmine.js --junitreport --filePrefix=unit_test_results",
"coverage": "nyc --reporter=lcov --reporter=text-lcov npm test"
},
Does anyone know the solution for it, to show the Unit tests block on the sonarqube dashboard?
Thanks.

npm test -- --coverage never exits

I am using create-react-app to create a react application. When I executes npm test -- --coverage the test never exists. npm test actually runs react-scripts test. Any Idea?
-- --coverage part won't work, and should use one of the commands below to set CI to true.
By default npm test runs the watcher with interactive CLI. However, you can force it to run tests once and finish the process by setting an environment variable called CI.
source: React docs
Windows (cmd.exe)
set CI=true && npm test
set CI=true && npm run build
Windows (Powershell)
($env:CI = "true") -and (npm test)
($env:CI = "true") -and (npm run build)
Linux, macOS (Bash)
CI=true npm test
CI=true npm run build
NOT included in the docs
For Docker (node and react):
docker run -e CI=true [myImage] npm run test
Coverage won't work with Jest in watch mode.
Because "react-scripts test --env=jsdom" works in watch mode by default, the watch mode has to be switched off while generating the coverage output.
The following excerpt from the package.json contains a line "coverage" for illustration, how code coverage can be achieved within an app which was bootet by create-react-app.
It's just the modified "test" script, where the options --watchAll=false and --coverage are added in combination:
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test --env=jsdom",
"coverage": "react-scripts test --env=jsdom --watchAll=false --coverage",
"eject": "react-scripts eject"
}
Please note that it is obsolete to use standalone double-dash -- .
Most of the time this issue can be occur because of following reasons.
Not mentioning the required npm-script arguments in the
package.json file. If you use create-react-app to create your
react application, then it will not accept any command line
arguments. To resolve this problem, add following line under the
script tag in your package.json.
"test": "react-scripts test --coverage --watchAll", //mark --watchAll=false if you want.
Not mentioning the required jest configuration arguments in
the package.json or jest.config.js files. You should mention the files
which needed to include in your test coverage under the jest
configurations. Add following configurations in your
package.json.
package.json
"jest": {
"collectCoverageFrom": [
"src/**/*.js",
"!src/index.js", // files you need to avoid in test coverage
"!src/hooks/*.js",
"!src/context/*.js"
],
"coverageThreshold": {
"global": {
"branches": 90,
"functions": 90,
"lines": 90,
"statements": 90
}
},
"coverageReporters": [
"html",
"text"
]
},
Specifying a directory worked in my case
"test:cover": "react-scripts test --coverage src"
I tried all the solutions above, and for me it was still hanging with the message: Ran all test suites..
But this little hack helped:
"test:ci": "cross-env CI=true react-scripts test --forceExit --detectOpenHandles",
Explanation: The problem was coming from Jest not being able to close all processes. The above is a quick workaround. Ideally you should track the process that's stopping Jest from exiting.
In my case just added a new script "test:coverage": "react-scripts test --coverage"
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"test:coverage": "react-scripts test --coverage",
"eject": "react-scripts eject"
},

How to collect code coverage for spawned sub-processes using Jest?

I am writing integration tests for a CLI. All the unit tests are written using Jest as out-of-the-box it produces code coverage without any configuration, but unfortunately it does not instrument sub-processes, for example executed via Node's spawn and fork commands.
I have tried to introduce nyc into the mix as suggested in this comment on a GitHub issue however it has not worked for me.
I have played with various configurations (based on the initial aforementioned suggestion and also ideas from these issues: 1, 2), however either I get no coverage statistics at all or only coverage for my unit tests, not the integration tests that spawn sub-processes.
The relevant parts of my package.json which configures nyc and Jest:
"scripts": {
"test": "cross-env NODE_ENV=test nyc --clean jest --coverage",
},
"jest": {
"verbose": true,
"testURL": "http://localhost/",
"globalSetup": "./jest.setup.js",
"transform": {
"^.+\\.js$": "babel-jest"
},
"collectCoverage": false
},
"nyc": {
"include": [
"packages/*/src/**/*.js"
],
"reporter": [
"html"
]
},
I am using execa to run the sub-processes and do so as follows:
await execa("nyc --reporter none node", args);

NPM project.json script section tasks are not executing correctly

This is part of my project.json
"scripts": {
"prestart": "babel-node buildScripts/startMsg.js",
"start": "npm-run-all --parallel security-check open:src lint:watch",
"open:src": "babel-node buildScripts/srcServer.js",
"lint": "esw webpack.config.* src buildScripts --color",
"lint:watch": "npm run lint -- --watch",
"security-check": "nsp check",
"localtunnel": "lt --port 3000",
"share": "npm-run-all --parallel open:src localtunnel"
}
When I run my tasks manually like:
"nsp check" or "npm run lint -- --watch"
it works fine I see output in the console.
Problem is when I'm running my whole app "npm start" or "npm run lint:watch" these tasks are seems not running although I'm not getting any errors in the console and app seems to run fine I'm getting expected content.
What could be the problem here? and what is the difference if run nsp check or npm run security-check shouldn't the same thing happened?
Example when what shows my console when I execute these two above:
console output

How to stop npm start , when lint or Unit testcase fails?

I am using npm start with lint and unit test case, if lint or test case fails, npm should not up the api, but I am not able to do.
Here is code I am using:
"scripts": {
"start": "gulp lint & npm test & node src/server/index.js",
"test": "mocha --timeout 10000"
},
Please help me what I have to do for it, I searched on internet also, but not getting the proper solution.
You can simply rewrite the code like this:
"scripts": {
"start": "gulp lint && npm test && node src/server/index.js",
"test": "mocha --timeout 10000"
},
Double && will never start execution of node server if previous steps contains errors.
Use && instead of &:
"scripts": {
"start": "gulp lint && npm test && node src/server/index.js",
"test": "mocha --timeout 10000"
},

Resources