Problem
I have a Node project with 2 different jest runs that I perform through different npm tasks. They both use the same jest.config.js, which has a declaration for coverageThreshold. I want to apply that threshold to the combined output of the jest runs.
How can I apply a jest coverage threshold to the combined output of multiple test runs?
Background
For reference, see the code for the sample project described below here
Sample project
To illustrate the problem, assume the project has a single file that looks like this:
Code under test
# src/index.js
function run(cond) {
if (cond) {
return "foo";
}
return "bar";
}
module.exports = run;
There are also 2 tests in the project:
Test 1
# src/__tests__/index.group1.test.js
var run = require("../index");
describe("when cond is true", () => {
it("should return 'foo'", () => {
expect(run(true)).toEqual("foo");
});
});
Test 2
# src/__tests__/index.group1.test.js
var run = require("../index");
describe("when cond is false", () => {
it("should return 'bar'", () => {
expect(run(false)).toEqual("bar");
})
});
The test coverage npm scripts in package.json look like this:
"test:group1:coverage": "jest --testPathPattern=group1 --coverage",
"test:group2:coverage": "jest --testPathPattern=group2 --coverage"
Running npm run test:group1:coverage generates:
----------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files | 75 | 50 | 100 | 75 | |
index.js | 75 | 50 | 100 | 75 | 5 |
----------|----------|----------|----------|----------|-------------------|
Running npm run test:group2:coverage generates:
----------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files | 75 | 50 | 100 | 75 | |
index.js | 75 | 50 | 100 | 75 | 3 |
----------|----------|----------|----------|----------|-------------------|
Combining coverage reports
Based on this post, I can have a separate npm task like this ...
"test:coverage": "npm run test:group1:coverage && mv ./coverage/coverage-final.json ./coverage/coverage-group1-final.json && npm run test:group2:coverage && mv ./coverage/coverage-final.json ./coverage/coverage-group2-final.json && node ./scripts/map-coverage.js"
... to generate a combined coverage report that looks like:
----------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files | 100 | 100 | 100 | 100 | |
index.js | 100 | 100 | 100 | 100 | |
----------|----------|----------|----------|----------|-------------------|
It's all good until this point. The problem arises when trying to apply a coverage threshold to the combined run.
Issue
Now, if I add coverageThreshold to the jest.config.js like this ...
module.exports = {
coverageThreshold: {
global: {
branches: 90,
functions: 90,
lines: 90,
statements: 90,
},
},
};
... and run npm run test:coverage, the tests fails ...
----------|----------|----------|----------|----------|-------------------|
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files | 75 | 50 | 100 | 75 | |
index.js | 75 | 50 | 100 | 75 | 5 |
----------|----------|----------|----------|----------|-------------------|
Jest: "global" coverage threshold for statements (90%) not met: 75%
Jest: "global" coverage threshold for branches (90%) not met: 50%
Jest: "global" coverage threshold for lines (90%) not met: 75%
... because it first runs group1 tests, which on its own fails to get over the coverage threshold set in jest.config.js. I want to ignore this threshold for individual runs, and only apply them to the combined report.
Note
I know I could split the jest.config.js file into separate ones and apply individual coverage thresholds per test group, but I don't want to manage thresholds individually per test group. I want to measure/apply it to the project as a whole, after all individual test runs have been completed and combined.
Instead of combining the tasks, combine only the pathPattern.
You can try to create a single task for group1 and group2 something like below
"test:commongroup:coverage": "jest --testPathPattern=commonGroup --coverage",
where commonGroup is pattern that includes both group1 and group2.
Related
I'm trying to run cargo fix on a project that uses slqx and am getting the following error:
error: proc macro panicked
--> src/twitter/domain/user.rs:54:5
|
54 | / sqlx::query!(
55 | | r#"
56 | | INSERT INTO users
57 | | (id, created_at,
... |
84 | | user["public_metrics"]["tweet_count"].as_i64(),
85 | | )
| |_____^
|
= help: message: Lazy instance has previously been poisoned
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
...for every instance of sqlx macro I have in my code. The weird thing is it used to work just fine, but for some reason doesn't anymore.
The only mention of the error on Google that I found is here, but I don't think it's relevant.
What might be wrong?
Started trying to integrate nyc/istambul into my ava test suites:
./node_modules/.bin/nyc --include modules/todo.js --es-modules ./node_modules/.bin/ava
1 tests passed
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 0 | 0 | 0 | 0 |
----------|---------|----------|---------|---------|-------------------
The output fails to list any of the files!
If I add the --all flag I get the following even though the tests cover both functions:
ERROR: Coverage for lines (0%) does not meet global threshold (90%)
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 0 | 0 | 0 | 0 |
todo.js | 0 | 0 | 0 | 0 | 6-30
----------|---------|----------|---------|---------|-------------------
So it looks as if the output from the ava testing tool is not being picked up by the nyc coverage tool.
The simple setup is as follows:
/* todo.js (uses ES6 modules) */
'use strict'
let data = []
function clear() {
data = []
}
function add(item, qty = 1) {
qty = Number(qty)
if(isNaN(qty)) throw new Error('qty must be a number')
data.push({item: item, qty: qty})
return true
}
export { add, clear }
/* todo.spec.js */
import { add, clear } from './modules/todo.js'
import test from 'ava'
test('add a single item', test => {
clear()
const ok = add('bread')
test.truthy(ok)
})
I tried creating a .nycrc.json file but this didn't help:
{
"check-coverage": true,
"include": [
"modules/todo.js"
],
"reporter": ["text", "html"]
}
Hi all I am trying to get the mocha chai Unit test coverage report.
I am getting the test result passed.
Also generated the coverage as html but, it showing Unknown. Please see below image .
Package.json added the below config in script.
"scripts": {
"test": "mocha",
"test-with-coverage": "nyc --reporter=html mocha"
},
And using the below command for run the test.
npm run test-with-coverage
Edit:
When change report as text , getting the below report in terminal.
----------|---------|----------|---------|---------|-------------------
File | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files | 0 | 0 | 0 | 0 |
----------|---------|----------|---------|---------|-------------------
Please try with the below command,
nyc -x \"**/tests/**\" --reporter=cobertura --reporter=html mocha 'your test folder path'
I am currently trying to learn the syntax of Rust by solving little tasks. I compare the execution time as sanity-checks if I am using the language the right way.
One task is:
Create an array of 10000000 random integers in the range 0 - 1000000000
Sort it and measure the time
Print the time for sorting it
I got the following results:
| # | Language | Speed | LOCs |
| --- | -------------------- | ------ | ---- |
| 1 | C++ (with -O3) | 1.36s | 1 |
| 2 | Python (with PyPy) | 3.14s | 1 |
| 3 | Ruby | 5.04s | 1 |
| 4 | Go | 6.17s | 1 |
| 5 | C++ | 7.95s | 1 |
| 6 | Python (with Cython) | 11.51s | 1 |
| 7 | PHP | 36.28s | 1 |
Now I wrote the following Rust code:
rust.rs
extern crate rand;
extern crate time;
use rand::Rng;
use time::PreciseTime;
fn main() {
let n = 10000000;
let mut array = Vec::new();
let mut rng = rand::thread_rng();
for _ in 0..n {
//array[i] = rng.gen::<i32>();
array.push(rng.gen::<i32>());
}
// Sort
let start = PreciseTime::now();
array.sort();
let end = PreciseTime::now();
println!("{} seconds for sorting {} integers.", start.to(end), n);
}
with the following Cargo.toml:
[package]
name = "hello_world" # the name of the package
version = "0.0.1" # the current version, obeying semver
authors = [ "you#example.com" ]
[[bin]]
name = "rust"
path = "rust.rs"
[dependencies]
rand = "*" # Or a specific version
time = "*"
I compiled it with cargo run rust.rs and ran the binary. It outputs
PT18.207168155S seconds for sorting 10000000 integers.
Note that this is much slower than Python. I guess I am doing something wrong. (The complete code of rust and of the other languages is here if you are interested.)
Why does it take so long to sort with Rust? How can I make it faster?
I Tried your code on my computer, running it with cargo run gives:
PT11.634640178S seconds for sorting 10000000 integers.
And with cargo run --release (turning on optimizations) gives:
PT1.004434739S seconds for sorting 10000000 integers.
My assumption is that any module tested using Intern will automatically be covered by Istanbul's code coverage. For reasons unknown to me, my module is not being included.
I am:
running Intern 1.6.2 (installed with npm locally)
testing NodeJS code
using callbacks, not promises
using CommonJS modules, not AMD modules
Directory Structure (only showing relevant files):
plister
|
|--libraries
| |--file-type-support.js
|
|--tests
| |--intern.js
| |--unit
| |--file-type-support.js
|
|--node_modules
|--intern
plister/tests/intern.js
define({
useLoader: {
'host-node': 'dojo/dojo'
},
loader: {
packages: [
{name: 'libraries', location: 'libraries'}
]
},
reporters: ['console'],
suites: ['tests/unit/file-type-support'],
functionalSuites: [],
excludeInstrumentation: /^(tests|node_modules)\//
});
plister/tests/unit/file-type-support.js
define([
'intern!bdd',
'intern/chai!expect',
'intern/dojo/node!fs',
'intern/dojo/node!path',
'intern/dojo/node!stream-equal',
'intern/dojo/node!../../libraries/file-type-support'
], function (bdd, expect, fs, path, streamEqual, fileTypeSupport) {
'use strict';
bdd.describe('file-type-support', function doTest() {
bdd.it('should show that the example output.plist matches the ' +
'temp.plist generated by the module', function () {
var deferred = this.async(),
input = path.normalize('tests/resources/input.plist'),
output = path.normalize('tests/resources/output.plist'),
temporary = path.normalize('tests/resources/temp.plist');
// Test deactivate function by checking output produced by
// function against test output.
fileTypeSupport.deactivate(fs.createReadStream(input),
fs.createWriteStream(temporary),
deferred.rejectOnError(function onFinish() {
streamEqual(fs.createReadStream(output),
fs.createReadStream(temporary),
deferred.callback(function checkEqual(error, equal) {
expect(equal).to.be.true;
}));
}));
});
});
});
Output:
PASS: main - file-type-support - should show that the example output.plist matches the temp.plist generated by the module (29ms)
1/1 tests passed
1/1 tests passed
Output (on failure):
FAIL: main - file-type-support - should show that the example output.plist matches the temp.plist generated by the module (30ms)
AssertionError: expected true to be false
AssertionError: expected true to be false
0/1 tests passed
0/1 tests passed
npm ERR! Test failed. See above for more details.
npm ERR! not ok code 0
Output (after removing excludeInstrumentation):
PASS: main - file-type-support - should show that the example output.plist matches the temp.plist generated by the module (25ms)
1/1 tests passed
1/1 tests passed
------------------------------------------+-----------+-----------+-----------+-----------+
File | % Stmts |% Branches | % Funcs | % Lines |
------------------------------------------+-----------+-----------+-----------+-----------+
node_modules/intern/ | 70 | 50 | 100 | 70 |
chai.js | 70 | 50 | 100 | 70 |
node_modules/intern/lib/ | 79.71 | 42.86 | 72.22 | 79.71 |
Test.js | 79.71 | 42.86 | 72.22 | 79.71 |
node_modules/intern/lib/interfaces/ | 80 | 50 | 63.64 | 80 |
bdd.js | 100 | 100 | 100 | 100 |
tdd.js | 76.19 | 50 | 55.56 | 76.19 |
node_modules/intern/lib/reporters/ | 56.52 | 35 | 57.14 | 56.52 |
console.js | 56.52 | 35 | 57.14 | 56.52 |
node_modules/intern/node_modules/chai/ | 37.9 | 8.73 | 26.38 | 39.34 |
chai.js | 37.9 | 8.73 | 26.38 | 39.34 |
tests/unit/ | 100 | 100 | 100 | 100 |
file-type-support.js | 100 | 100 | 100 | 100 |
------------------------------------------+-----------+-----------+-----------+-----------+
All files | 42.14 | 11.35 | 33.45 | 43.63 |
------------------------------------------+-----------+-----------+-----------+-----------+
My module passes the test and I can make it fail too. It just will not show up in the code coverage. I have done the tutorial hosted on GitHub without any problems.
I tried dissecting the Istanbul and Intern dependencies. I place a console.log where it seems files to be covered go through, but my module doesn't get passed. I have tried every variation of deferred.callback and deferred.rejectOnError with no difference to the code coverage.
Also, any feedback on my use of deferred.callback and deferred.rejectOnError will be greatly appreciated. I am still a little uncertain on their usage.
Thanks!
As of Intern 1.6, only require('vm').runInThisContext is hooked to add code coverage data, not require. Instrumentation of require was added in Intern 2.0.
The use of callback/rejectOnError in the above code is correct.