I have project Excellent.js setup for automatic testing with jest and puppeteer, which successfully runs all the tests, which can be seen on Travis CI.
But after a lot of configuration tweaks I have been unable to make it report correct coverage. No matter what tests are executed, the coverage does not reflect it at all.
The library contains only a single JavaScript file excellent.js, and my jest.config.js was set up as instructed for coverage:
module.exports = {
collectCoverage: true,
collectCoverageFrom: [
'src/excellent.js'
],
testURL: 'http://localhost/',
setupFiles: [
'./src/excellent.js'
]
};
Here're all the tests, which all pass if you do first npm install, and then npm test.
So what am I missing? Why can't I get the coverage reported correctly?
ISSUE
Most of the tests are using Puppeteer and when the code is executed in the browser provided by Puppeteer, that code execution is not reflected in the Jest code coverage reports.
SOLUTION
None of the tests require Puppeteer so I refactored them as Jest tests. The code coverage is now accurate and is currently the following:
excellent.js | 63.47 | 48.7 | 57.78 | 62.96
I created a pull request with these changes.
Additional Info
It is now possible to generate code coverage reports for Puppeteer pages and there is a library to help view them in Instanbul but those code coverage reports are generated independently from Jest.
To do testing in Puppeteer pages and have the coverage from those tests reflected in the reports generated by Jest would require merging the Puppeteer page coverage reports with the Jest coverage report.
Related
In my application, while developing, I run:
npm run test src/components/component.test.tsx
This runs the specific test suite for the component I'm working on.
On top of that, I can then change it to:
npm run test src/components/component.test.tsx -- --coverage --coverageReporters=text-summary --collectCoverageFrom=src/components/component.tsx
Which will print a coverage report for that specific file once the tests have been run.
As you can see this is extremely wordy and only gets worse if I want to test two or three files at the same time.
Is there any way to automate collectCoverageFrom to collect coverage from the files that have been tested (not from all files in the project) so that I don't have to type it out manually every single time?
Just omit the "collectCoverageFrom" (or explicitly set it to an empty glob if you're overriding the config file).
Jest will then only collect coverage from files that are used during the test run.
Set it up in your jest configuration file.
your npm script will look like jest -c path/to/jest.config.js
jest.config.js will look like
module.exports = {
collectCoverage: true,
// The directory where Jest should output its coverage files
coverageDirectory: "./coverage",
// Indicates which provider should be used to instrument code for coverage
coverageProvider: "v8",
// A list of reporter names that Jest uses when writing coverage reports
coverageReporters: ["html", "text", "cobertura"],
}
If you do jest --init it will help you build a new config file
Side note: You may want to set up a jest wildcard so you don't need to individually write down every file you want to test.
I am using jest to test my reactjs application. It works fine but jest doesn't give me a good report about test case results. The output of jest shows each test suite followed by its output. I have more than 50 test suites and it is hard for me to scroll up to check each failed test cases. Is there a brief jest report which printing a brief summary about the whole test cases?
below is my jest.conf file:
{
"testRegex": "/tests/.*\\.test\\.jsx?$",
"testEnvironment": "node",
"roots": ["./src"],
"coverageReporters": ["text-summary", "html"]
}
There is also a JEST HTML Reporter module which you might find useful.
It's configurable to specify what level of detail you want to show in the test report for failures
https://www.npmjs.com/package/jest-html-reporter
You can run jest with the --coverage flag.
If you want something different than the default reporters, you have to set them in your jest config file.
jest.json
{
"coverageReporters": ["text-summary", "html"]
}
text-summary gives you a short summary beneath all tests that tells you how many suites/tests are successful/failed.
html gives you a some html pages that you can browse through to see exactly what got tested.
CLI
$ ./node_modules/.bin/jest --config ./path/to/jest.json --coverage
You might want to adjust which files are covered etc.
See all coverage options in the jest docs.
https://github.com/jest-community/awesome-jest#reporters contains a list of other reporters. jest-stare will let you filter off passing tests so you can just see the ones that failed.
Use Istanbul to generate Reports
In the Script section of the Package.json just add :
"scripts": {
"test": "jest",
"test-coverage": "jest --coverage",
}
Save the project and run below command:
Yarn test : For Terminal Coverage
Yarn test-coverage for Report Generation.Find HTML report in Coverage > Icov-report >index.html
If you can use a web-based reporter try Tesults - a reporter for Jest is available to help with integration: https://www.npmjs.com/package/jest-tesults-reporter.
You main issue, with respect to navigating test suites is handled well. I should add a disclaimer that I work at Tesults, but on the other hand if this is for a very small team or it's open source it is free to use, just contact support.
I'm trying to get coverage report for the API code. I have all the test cases running perfectly in mocha. My problem is that my API server and the test cases are written in separate repositories.
I start my node API server on localhost, on a particular port, and then using supertest in mocha, hit the localhost url to test server's response.
Can you suggest me the best way to generate a coverage report for those APIs?
Testing env
If you want to get coverage, supertest should be able to bootstrap the app server, like in the express example.
The drawback is that you must not run your tests against a running server, like
var api = request('http://127.0.0.1:8080');
but you must include your app entrypoint to allow supertest to start it like
var app = require('../yourapp');
var api = request(app);
Of course, this may (or may not) result in a bit of refactoring on your app bootstrap process.
As other options, you can use node CLI debug capabilities or use node-inspector.
Coverage setup
Supposing you are willing to install istanbul in association with mocha to get coverage.
npm install -g istanbul
then
istanbul cover mocha --root <path> -- --recursive <test-path>
cover is the command use to generate code coverage
mocha is the executable js file used to run tests
--root <path> the root path to look for files to instrument (aka the "source files")
-- is used to pass arguments to your test runner
--recursive <test-path> the root path to look for test files
You can then add --include-all-sources to get cover info on all your source files.
In addition you can get more help running
istanbul help cover
I have a project that contains tests that I am running with Mocha from the command line. I have set up a test script in my packages.json, which looks as follows:
"test": "mocha ./**/*.spec.js --reporter dot --require jsdom-global/register"
I have currently got a simple task set up in Visual Studio Team Services, which just runs the npm test command, this runs Mocha within a console and continues/fails the build depending on whether the tests pass.
What I'd like to be able to do is have the results of my tests populate the "tests" tab in the build definition after it's run. In the same way that I can get this tab populated if I'm running tests on C# code.
I've tried using Chutzpah for this, but it's overly complicated and seems to require that I jump through all sorts of hoops that mean changing my tests and writing long config files. I already have loads of tests written, so really don't want to have to do that. When it did finally discover any of my tests, it complained about require and other things related to Node modules.
Is what I'm asking for actually possible? Is there a simple way of achieving this that's compatible with running my tests in Node?
I've found a good way of doing it that requires no third-party adapter (eg. Chutzpah). It involves getting Mocha to output its report in an XML format, and setting up Visual Studio Team Services to publish the results in an extra step of the build definition.
I installed mocha-junit-reporter (https://www.npmjs.com/package/mocha-junit-reporter) and altered my test script to the following:
"test": "mocha ./temp/**/*.spec.js --reporter mocha-junit-reporter --require jsdom-global/register"
I then created a new step in my build definition using the "Publish Test Results" task. I set the result format to "JUnit" and added the correct path for the outputted test-results.xml file created by the reporter.
It is worth noting that although Mocha comes with an "XUnit" reporter, this format appears to not work correctly with VSTS even though it's listed as an option.
The results of npm test now show up in the "tests" tab alongside any other tests from MSTest etc.
I'm using karma and got this to work in the same way as #dylan-parry suggested. Some excepts below in case it helps others:
package.json
"scripts": {
"test": "cross-env NODE_ENV=test karma start"
}
karma.conf.js
const webpackCfg = require('./webpack.config')('test');
module.exports = function karmaConfig(config) {
config.set({
reporters: ['mocha', 'coverage', 'junit'],
junitReporter: {
outputDir: 'coverage',
outputFile: 'junit-result.xml',
useBrowserName: false
}
})
...
TFS
It may also be worth adding I'm using branch policies on my git branch to prevent PR's being merged if the tests fail, info via this link:
https://www.visualstudio.com/en-us/docs/git/branch-policies
Here's the output in TFS:
Next step is to get the coverage working too!
I'm trying to run mocha tests directly on the WebStorm IDE but without any success.
This is my jsTestDriver.conf file:
server: http://localhost:4224
load:
- node_modules/mocha/mocha.js
- node_modules/chai/chai.js
- node_modules/requirejs/require.js
- mocha-jstd-adapter/src/MochaAdapter.js
- routes/*.js
test:
- test/*.js
timeout: 90
Now I'm taking a look at the buster-format module that works correctly. Any help is appreciated (I'll post the solution if I did it).
Mocha support will be included in WebStorm 7.0.2. You can try the RC build available at http://confluence.jetbrains.com/display/WI/WebStorm+7+EAP
The result is that it's not possible currently.
As stated on the documentation, you can only:
do tests with mocha and other frameworks for front-end javascript with coverage and everything using JSTestDriver or Karma;
do tests with ONLY nodeunit for node.js, so NOT mocha or something else, but you cannot have the coverage with nodeunit either.
Hopefully this will option will be implemented soon.
It's possible to do test with mocha but without code-coverage.