Has anyone attempted to integrate jest unit tests with TFS 2015? I tried to use Chutzpah Test Adapter (https://visualstudiogallery.msdn.microsoft.com/f8741f04-bae4-4900-81c7-7c9bfb9ed1fe?SRC=VSIDE) however it's not able to recognize jest. I receive below error:
Can't find variable Jest
When I run the unit tests through "npm test" I get the results. However to integrate with TFS 2015 I need a test runner which can run Jest unit test so that I can run the unit tests in conjunction with vstest.console.exe which the TFS 2015 provides so it can manage build results and publish results in the build summary report.
Any help would be appreciated!!
Any test runner which can run tests using below command should work (considering VS 2015 installed on the system):
"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "\test.js" /UseVsixExtensions:true
Extending on Merlin's answer, here is how I've implemented publishing jest test results AND code coverage to TFS2015 vNext builds (I am using create-react-app boilerplate):
First install required packages on the Server you are running your Agent on:
npm install -g jest-json-to-tap
npm install -g tap-xunit
configure jest to output json, by changing in package.json the "test" task to:
"test": "react-scripts test --env=jsdom --json",
configure jest options in package.json:
"jest": { "coverageReporters": ["cobertura"] }
created a vNext build (TFS2015v4) with the following tasks:
a. "npm" task, command=run, arguments=test -- --coverage | jest-json-to-tap | tap-xunit > TEST-result.xml
b. "publish test results" task, format=JUnit
c. "public code coverage results" task, code coverage tool=Cobertura, Summary file=$(Build.Repository.LocalPath)\coverage\cobertura-coverage.xml
make sure your build's "Variables" include setting the environment variable "CI"="true"
NOTES:
- test results will not include times nor assemblies - something to extend for the future...
Voila'! Running this build will correctly publish the test results and code coverage stats, as well as report artifacts.
I'm not sure about jest, but there's a neat npm package that can convert TAP based results to xUnit XMLformat, and then you can publish that to TFS.
Take a look at tap-xunit.
I had a build environment where javascript testing was done by various tools and frameworks (AVA, Mocha, Jasmine etc). We decided to export them all to TAP format, run them throw tap-xunit and then publish to TFS.
Basically, you need something like this:
npm test | tap-xunit > results.xml
You pipe the results to tap-xunit and save them to an XML. This gives you an XML formatted as xUnit that you can publish to TFS. If you're running TFS 2015, I strongly recommend going with vNext builds, a lot easier to get these running. Check the "Publish Test Results" build step.
If you are running with XAML build, this link will help you: Javascript Unit Tests on Team Foundation Service with Chutzpah
If you are running with vNext build, please try the detail steps mentioned with Jasmine.JS test(also a kind of JavaScript test ) in this blog.
Related
I am developing an extension for vscode using typescript. I set up a GitHub Action which is executing my test cases. To gather the test coverage I am using nyc.
These are my node scripts:
"compile": "tsc -p ./",
"test": "npm run compile && node ./out/test/runTest.js",
"coverage": "nyc npm run test"
When I run npm run coverage I see all my test cases are getting executed (they are not stored in the file runTest.js), but only the coverage for the test file runTest.js is gathered. The relevant classes lib.ts and extension.ts are not shown in the coverage.
Why is this so? What am I doing wrong?
See my package.json and .nycrc.json for configuration.
I could fix this problem by sticking to the great blogpost which I found posted on a similar Question.
The problem is, that NYC is a command line tool. When executing the test I focused on my configuration in the .nycrc file. Fact is, that the visual studio code test runner is not using this configuration and has to be configured inside the test runner.
I fixed the broken test coverage generation inside this commit.
I have been trying to set up server-side testing for my application but have run into an issue with bamboo recognizing more than one test type. In the past, it worked just fine using only karma as the unit test, but I need to add server-side testing which works much better with jest.
The main problem I am running into is that when both are run as shown below the coverage report is only created for the karma(unit) tests.
test-unit=npm run test:unit && npm run test:server
I have also tried running jest under bamboos test-e2e and test-contract-test but when I do this nothing is reported. So is there a way to set up server-side testing separately in the .bamboorc file
So, I found an answer!
In the .bamboorc file move the test-unit = npm run test:unit to the end of the file and for the first test use
test-contract-consumer=npm run test:server -- --collect-coverage
This should then collect the coverage for the server tests in bamboo under the contract tests consumer artifacts, and the karma unit test should still show up under the unit test coverage artifacts.
I following a step by step guide to configure Visual Studio 2019 in order develop a Node.js-React app.
The guide is here:
https://learn.microsoft.com/en-us/visualstudio/javascript/tutorial-nodejs-with-react-and-jsx?view=vs-2019
Briefly, the guide tells to configure an npm script (called "build") that should be fired when visual studio compiles, in order to generate app-bundle.js through typescript, using the app.tsx which contains a react component.
This must be done by adding the following code snipped to package.json
"scripts": {
"build": "webpack-cli app.tsx --config webpack-config.js"
}
however it doesn't work, because when i change something in app.tsx and run visual studio debbugger, the web page don't change at all.
If I run the following command using the nuget package console
npm run-script build
then it works fine!
So, is this a VS2019 bug, or is there some trick i miss?
Many Thanks for the help
I had similar problems like yours.
I solved this problem by this undocumented feature of visual studio 2017,2019:
BY manual editing,
You can add the following post-build event to package.json
"-vs-binding": { "AfterBuild": [ "build" ] }
by GUI,
in task runner explorer window,
select "build" task and let popup context menu open using right-mouse-button,
select bindings -> After build (make its preceding check box on)
this will add "-vs-binding" property to your package.json
Either one is okay. This works on vs 2017 and may works on vs 2019 either.
VS2017,2019 nodejs project build process seems
not to actually call my "build" script at all.
we should specify the "build" script as post-build event
for vs default nodejs build action.
It seems weired. I hope Microsoft solve this fault using patches.
task runner explorer capture image
We have upgraded to Angular 6 so we had to change the ng build env=prod to
ng build configuration=production because angular-cli.json is not being used anymore.
Locally, ng build configuration=production works properly and the settings I need in prod were there. But when we trigger the build step/task in azure VSTS, ng build configuration=production does not replace the dev settings to production.
Did anyone encounter this? what did you folks do to fix this?
First, try changing your syntax to
ng run [project]:build:production
where [project] is replaced with the name of your project as specified in the projects object in angular.json.
For example, if your project is called "my-app", then try
ng run my-app:build:production
Second, in angular.json, make sure that my-app > architect > build > configurations > production is defined the way that you expect.
Third, notice that the output of the build will be dist\[project]\production (assuming your "outputPath" is "dist"). You need to ensure that your VSTS build's artifact include these files (for example, by archiving them into a ZIP and then copying them to $(Build.ArtifactStagingDirectory)).
I can't for the life of me find documentation or a tutorial for kicking off mocha unit tests in Visual Studio Online builds.
I have node.js app that is building in VSO and being deployed to Azure. That all works wonderfully. I can't seem to figure out how to kick off the spec files through the build process.
How is this done? Is there documentation available somewhere that I'm missing.
Assume you have setup Mocha tests with your package.json, i.e. you run tests with npm test. For more information, refer to https://docs.npmjs.com/cli/test.
In your Visual Studio Online build/release:
Add a "npm" task to install JUnit reporter
Run custom command install mocha-junit-reporter
Add a "npm" task
Run custom command test -- --reporter mocha-junit-reporter
Tips: You may want to increase timeout by adding --timeout 30000 because the build agent maybe running slower than your dev box
Then, add a "Publish Test Results" task
Set "Test result format" to "JUnit"
Check the box on "Continue on error"
Under "Control Options" > "Run this task", set it to "Even if a previous task has failed, unless the build was canceled"
Queue a build, you should see Mocha test results in your VSO build.
BONUS! You can also add code coverage to your Mocha run with nyc (formerly known as Istanbul)
On top of the steps above:
Install Istanbul locally to your package.json
Run npm install nyc--save-dev
Modify your scripts in package.json
Update { "scripts": { "test": "nyc --repoter=cobertura mocha" } }
Modify the "npm test" task
Run custom command test -- --reporter mocha-junit-reporter
Add a "Publish Code Coverage Results" task
Set "Code Coverage Tool" to "Cobertura"
Set "Summary File" to $(System.DefaultWorkingDirectory)/coverage/cobertura-coverage.xml
Set "Report Directory" to $(System.DefaultWorkingDirectory)/coverage/
Check the box on "Continue on error"
Under "Control Options" > "Run this task", set it to "Even if a previous task has failed, unless the build was canceled"
Add a new build variable NPM_CONFIG_COVERAGE and set it to true
Now you got both unit tests and code coverage results in your build report.
If you've configured you package.json to be able to run tests, adding a npm step that executes npm run test should do it. If you want to publish the test results you need to make sure that Mocha is writing its results to a format understood by Visual Studio Team Services. JUnit format would be a safe bet. Then follow up with a Publish test Results step that uploads the test results.
You can also use the Visual Studio Test Runner, combined with Chutzpah to run your tests, but I suppose that's going to be a lot of additional work to setup and isn't going to add much.
After quite a bit of fiddling around i got it to work by adding a "Command line task" to my build definition, i used the following parameters:
Set Tool to node
Set Arguments to $(Build.SourcesDirectory)\node_modules\jasmine-node\bin\jasmine-node --verbose test\
My tests are under a "test" folder, also make sure you have jasmine-node as a dev dependency