I am developing an extension for vscode using typescript. I set up a GitHub Action which is executing my test cases. To gather the test coverage I am using nyc.
These are my node scripts:
"compile": "tsc -p ./",
"test": "npm run compile && node ./out/test/runTest.js",
"coverage": "nyc npm run test"
When I run npm run coverage I see all my test cases are getting executed (they are not stored in the file runTest.js), but only the coverage for the test file runTest.js is gathered. The relevant classes lib.ts and extension.ts are not shown in the coverage.
Why is this so? What am I doing wrong?
See my package.json and .nycrc.json for configuration.
I could fix this problem by sticking to the great blogpost which I found posted on a similar Question.
The problem is, that NYC is a command line tool. When executing the test I focused on my configuration in the .nycrc file. Fact is, that the visual studio code test runner is not using this configuration and has to be configured inside the test runner.
I fixed the broken test coverage generation inside this commit.
Related
I have been trying to set up server-side testing for my application but have run into an issue with bamboo recognizing more than one test type. In the past, it worked just fine using only karma as the unit test, but I need to add server-side testing which works much better with jest.
The main problem I am running into is that when both are run as shown below the coverage report is only created for the karma(unit) tests.
test-unit=npm run test:unit && npm run test:server
I have also tried running jest under bamboos test-e2e and test-contract-test but when I do this nothing is reported. So is there a way to set up server-side testing separately in the .bamboorc file
So, I found an answer!
In the .bamboorc file move the test-unit = npm run test:unit to the end of the file and for the first test use
test-contract-consumer=npm run test:server -- --collect-coverage
This should then collect the coverage for the server tests in bamboo under the contract tests consumer artifacts, and the karma unit test should still show up under the unit test coverage artifacts.
I have an app that was generated with Nx CLI. I can run tests for this app with Jest using the the command nx test myApp. This works fine. However, I would like to use Jest's "watch mode". when running my tests, and can't figure out how to achieve this. Nx's documentation says simply:
To run unit tests for your application:
nx test myapp
I have been unable to find any more detailed documentation for this command, or how to add any other options/flags to it. Is there anyway to use watch mode with nx test?
TL;DR: yarn affected:test --all --parallel --maxParallel 10 --watch should work.
nx test actually runs ng test which in its turn is using the #nrwl/jest:jest builder that has a --watch option (in addition to some other jest specific options). You can see all the options by running nx test --help
Warning: If you have multiple apps/libs in your repository, this will only watch your default project's tests.
If you really want to watch all your projects tests, you will have to run:
yarn affected:test --all --parallel --maxParallel 10 --watch
--all we need this otherwise, this will not run tests on projects that you "affect" after running the command
--parallel because you want to run all the tests in parallel, otherwise this will only watch the first project in the list
--maxParallel because the default limit is 3 so if you have more than 3 projects, it will only watch tests for the 3 first projects
A simple command to run test for your applications with watch is
nx test <packageName> --watch
If you want to run test for a specific file in watch mode.
nx test <packageName> --testFile=<fileName> --watch
If you want to clear cache
nx test <packageName> --clearCache
If all your projects are using jest, there should be no issues with simply running:
jest --watch
Running adding a script for that and running it in the root directory works pretty well for me!
Has anyone attempted to integrate jest unit tests with TFS 2015? I tried to use Chutzpah Test Adapter (https://visualstudiogallery.msdn.microsoft.com/f8741f04-bae4-4900-81c7-7c9bfb9ed1fe?SRC=VSIDE) however it's not able to recognize jest. I receive below error:
Can't find variable Jest
When I run the unit tests through "npm test" I get the results. However to integrate with TFS 2015 I need a test runner which can run Jest unit test so that I can run the unit tests in conjunction with vstest.console.exe which the TFS 2015 provides so it can manage build results and publish results in the build summary report.
Any help would be appreciated!!
Any test runner which can run tests using below command should work (considering VS 2015 installed on the system):
"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "\test.js" /UseVsixExtensions:true
Extending on Merlin's answer, here is how I've implemented publishing jest test results AND code coverage to TFS2015 vNext builds (I am using create-react-app boilerplate):
First install required packages on the Server you are running your Agent on:
npm install -g jest-json-to-tap
npm install -g tap-xunit
configure jest to output json, by changing in package.json the "test" task to:
"test": "react-scripts test --env=jsdom --json",
configure jest options in package.json:
"jest": { "coverageReporters": ["cobertura"] }
created a vNext build (TFS2015v4) with the following tasks:
a. "npm" task, command=run, arguments=test -- --coverage | jest-json-to-tap | tap-xunit > TEST-result.xml
b. "publish test results" task, format=JUnit
c. "public code coverage results" task, code coverage tool=Cobertura, Summary file=$(Build.Repository.LocalPath)\coverage\cobertura-coverage.xml
make sure your build's "Variables" include setting the environment variable "CI"="true"
NOTES:
- test results will not include times nor assemblies - something to extend for the future...
Voila'! Running this build will correctly publish the test results and code coverage stats, as well as report artifacts.
I'm not sure about jest, but there's a neat npm package that can convert TAP based results to xUnit XMLformat, and then you can publish that to TFS.
Take a look at tap-xunit.
I had a build environment where javascript testing was done by various tools and frameworks (AVA, Mocha, Jasmine etc). We decided to export them all to TAP format, run them throw tap-xunit and then publish to TFS.
Basically, you need something like this:
npm test | tap-xunit > results.xml
You pipe the results to tap-xunit and save them to an XML. This gives you an XML formatted as xUnit that you can publish to TFS. If you're running TFS 2015, I strongly recommend going with vNext builds, a lot easier to get these running. Check the "Publish Test Results" build step.
If you are running with XAML build, this link will help you: Javascript Unit Tests on Team Foundation Service with Chutzpah
If you are running with vNext build, please try the detail steps mentioned with Jasmine.JS test(also a kind of JavaScript test ) in this blog.
I can't for the life of me find documentation or a tutorial for kicking off mocha unit tests in Visual Studio Online builds.
I have node.js app that is building in VSO and being deployed to Azure. That all works wonderfully. I can't seem to figure out how to kick off the spec files through the build process.
How is this done? Is there documentation available somewhere that I'm missing.
Assume you have setup Mocha tests with your package.json, i.e. you run tests with npm test. For more information, refer to https://docs.npmjs.com/cli/test.
In your Visual Studio Online build/release:
Add a "npm" task to install JUnit reporter
Run custom command install mocha-junit-reporter
Add a "npm" task
Run custom command test -- --reporter mocha-junit-reporter
Tips: You may want to increase timeout by adding --timeout 30000 because the build agent maybe running slower than your dev box
Then, add a "Publish Test Results" task
Set "Test result format" to "JUnit"
Check the box on "Continue on error"
Under "Control Options" > "Run this task", set it to "Even if a previous task has failed, unless the build was canceled"
Queue a build, you should see Mocha test results in your VSO build.
BONUS! You can also add code coverage to your Mocha run with nyc (formerly known as Istanbul)
On top of the steps above:
Install Istanbul locally to your package.json
Run npm install nyc--save-dev
Modify your scripts in package.json
Update { "scripts": { "test": "nyc --repoter=cobertura mocha" } }
Modify the "npm test" task
Run custom command test -- --reporter mocha-junit-reporter
Add a "Publish Code Coverage Results" task
Set "Code Coverage Tool" to "Cobertura"
Set "Summary File" to $(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml
Set "Report Directory" to $(System.DefaultWorkingDirectory)/coverage/
Check the box on "Continue on error"
Under "Control Options" > "Run this task", set it to "Even if a previous task has failed, unless the build was canceled"
Add a new build variable NPM_CONFIG_COVERAGE and set it to true
Now you got both unit tests and code coverage results in your build report.
If you've configured you package.json to be able to run tests, adding a npm step that executes npm run test should do it. If you want to publish the test results you need to make sure that Mocha is writing its results to a format understood by Visual Studio Team Services. JUnit format would be a safe bet. Then follow up with a Publish test Results step that uploads the test results.
You can also use the Visual Studio Test Runner, combined with Chutzpah to run your tests, but I suppose that's going to be a lot of additional work to setup and isn't going to add much.
After quite a bit of fiddling around i got it to work by adding a "Command line task" to my build definition, i used the following parameters:
Set Tool to node
Set Arguments to $(Build.SourcesDirectory)\node_modules\jasmine-node\bin\jasmine-node --verbose test\
My tests are under a "test" folder, also make sure you have jasmine-node as a dev dependency
I am running some node js unit tests using "Istanbul cover test.js", where test.js is the master test file which will call the actual scripts in our codebase. The problem is that the coverage report it generates is only on the test.js file and not on the actual lines of code in the codebase. Pardon me if this is a dumb question, but how do I get it to show coverage for the actual files that the tests refer to?
You have to run istanbul cover against the tests that are run.
The example in the docs is a bit unclear about this: istanbul cover test.js assumes that test.js is the executable that is running all your tests, not the test itself.
For example, if you're using mocha as your test runner, it should look like istanbul cover node_modules/.bin/_mocha (assuming mocha is installed as local devDependency) or istanbul cover mocha, if it's installed as global module.
on Windows:
If you installed jasmine-node globally:
istanbul cover /d/Users/rxxx/AppData/Roaming/npm/node_modules/jasmine-node/bin/jasmine-node ./
If you installed jasmine-node locally:
istanbul cover ../node_modules/jasmine-node/bin/jasmine-node ./
In my case the following command worked when run as a script defined in package.json:
istanbul cover ../jasmine/bin/jasmine.js