I have a MEAN project. Using Jenkins on an EC2 machine I build this using the following shell script:
npm install && PORT=8888 npm test
mocha returns 2 (number of failing tests) but still jenkins says:
Finished: SUCCESS.
If tests are failing I expect to see
Finished: FAILURE
Do you know why its not working fine?
You can:
Use a test runner like Karma, or
Tell Mocha to report in, for example, XUnit format, by passing Mocha the --reporter xunit flag. XUnit closely aligns with JUnit which Jenkins understands, or
Add in a custom reporter — mocha-jenkins-reporter is a decent option.
In the end I used a different solution: installed Jenkins Text Finder and if "expected - actual" is found in log (test failed), I let this plugin to mark the build as "Unstable".
Related
I'm using Stencil.js to create a web component library and I'm heavily relying on E2E tests. As they're rather slow it becomes more and more cumbersome to run the entire test suite (using the Stencil.js CLI) while developing new components.
However, I'm not able to run single tests in my IDE (IntelliJ IDEA) or via command line. It works perfectly fine for unit tests though.
My Jest config looks like this:
module.exports = {
"roots": [
"<rootDir>/src"
],
"preset": "#stencil/core/testing"
}
When I try to run tests in a single file (jest --config jest.config.js --testPathPattern src/components/button/button.e2e.ts$)
it fails, because
newE2EPage() is only available from E2E tests, and ran with the --e2e cmd line flag.
newE2EPage() comes with Stencil.js and I don't know what Stencil.js's CLI does in the background. Furthermore, I cloned the Stencil.js repository, just to see if it is working with their E2E tests (https://github.com/ionic-team/stencil/tree/master/test/end-to-end) but it doesn't work either.
Any idea how I can configure Jest so that it's able to run Stencil.js-E2E tests from the command line?
The --e2e flag is used for the npm script in the package.json. To start e2e tests, you can add this in your package.json:
"scripts": {
"test:e2e": "stencil test --e2e"
}
And run npm run test:e2e. For a specific file, you add it at the end like this:
npm run test:e2e src/components/button/button.e2e.ts
For more info, see the StencilJS doc: https://stenciljs.com/docs/end-to-end-testing
i have the same problem. IntelliJ and 'Run' single 'it' didnt work.
newE2EPage() is only available from E2E tests, and ran with the --e2e cmd line flag.
when i run 'npm run test' everything will work fine. the difference is that npm run stencil before and only jest dont work.
here is the stencil jest dir https://github.com/ionic-team/stencil/tree/master/src/testing/jest aswell a config.
i found in here https://stenciljs.com/docs/testing-overview a VS-CODE run jest code but no Intellij setup.
im on the run to get the path of the current file to run stencil via npm and the path the e2e file. but i cant find the correct variable for the run config.
i hope we got this solved soon.
cheers
I am not a VS Code user, but in contrast to IntelliJ there is a launch.json for VSC to run single tests: https://github.com/ionic-team/stencil-site/pull/480
While creating package.json from command line using npm init for creating a module in Node.js, there is a test command field that I don't know about. There's no mention of it in the docs too on executing npm help json also in the CLI.
Please explain what it is about.
The test command is the command that is run whenever you call npm test.
This is important when integrating with continuous integration/continuous deployment tools (such as jenkins, codeship, teamcity).
Example:
- say you deploy a project to AWS or some other cloud hosting provider,
- you can set up your infrastructure to automatically run npm test.
- If there are problems within those tests, your ci/cd will automatically rollback before deploying.
To execute tests
You can use karma, jest, or selenium/nightmare/phantomjs or about any other test scripting library/framework that allows you to write and execute tests and then set the required command in scripts.test and finally run it from npm test.
Assuming you mean scripts.test:
"scripts" : {
"test" : "echo \"Error: no test specified\" && exit 1"
}
This field contains the program(/command line) that should run when you call npm test. Typically, that program is a test-runner like mocha, ava, jest, ...
The default value is a placeholder that prints an error message (try running npm test in the same directory as your package.json).
I using jasmine-node for testing node app, and have integration with bamboo, but the problem is when some of the test failed bamboo reports that command jasmine-node test-name.js exits with 1 and that on bamboo job dashboard I have label testless build without reports about tests.
Also I made wrapper in node to run that command and still getting that output of executing is 1.
Can I use this wrapper for bamboo, because I'm using --junitreport flag and artifact for junit tests?
Or I can setup jasmine-node to prevent throwing error if some of the test fails?
Just put the exit 0 at end of script in bamboo job.
I have several open-source projects that use Travis-CI to check the build status on my commits. I also report my test coverage to Coveralls.io. In the past few weeks, the reliability of coveralls has been spotty at best, and I had my tests running the coverage stuff, but they would fail the build because the coveralls api was returning a 503 error.
In response to this, I moved my coverage stuff to an after_success script in my .travis.yml, but now it seems that it's not sending the info to Coveralls.io. Am I approaching this the wrong way? Has anyone else run into this issue?
Here are some more details on the projects I'm running into issues with:
all of them are node modules
all of them are tested using mocha
all of them have test coverage using istanbul
I am using the node-coveralls module to report coverage reports to coveralls.io
If more information is needed, please let me know.
UPDATE:
Now it seems to be reporting correctly to coveralls, but it's only reporting 50%. Any additional help would be much appreciated
I use the following in my yml file. It concats the lcovs from karma and mocha, then sends them along.
after_script:
- ./node_modules/karma/bin/karma start
- istanbul cover ./node_modules/mocha/bin/_mocha server/api/**/*.spec.js
- cat ./coverage/lcov.info ./coverage/karma/**/*.info > lcov.info
- node node_modules/lcov-filter/index.js lcov.info config | ./node_modules/coveralls/bin/coveralls.js && rm -rf ./coverage
I run jasmine tests in my node.js application via grunt. I've added grunt test to Webstorm's run configurations.
In "Run/Debug Configurations" the "Grunt Test" has following properties specified: node interpreter (node.exe in Program Files), working directory (project location), JavaScript file (grunt binary), and application parameters (test - grunt task name).
The configuration works correctly but stops without printing full jasmine output. Sometimes before printing any output, most often after some part of jasmine log, but before test summary or error details. Always the last line of output is
Process finished with exit code 0
When I take the Webstorm command and run it manually in console it works fine and always prints full output:
"C:\Program Files\nodejs\node.exe" C:\...\node_modules\grunt-cli\bin\grunt test
My tests includes asynchronous cases, so it takes about 20 seconds to run them all.
I noticed that each attempt to run tests via Webstorm prints a bit longer output. First attempt ends without any but after ten further I got full output with test summary.
Known issue, please see http://youtrack.jetbrains.com/issue/WEB-1926 and discussion on stackoverflow (mocha + webstorm - error message broken)
To fix the case described in question we can run jasmine tests directly by jasmine-node command (skipping grunt).
We have to set up new "Run/Debug Configuration" based on "Node.js" pre-conf.
Node interpreter (as usual):
C:\Program Files\nodejs\node.exe
Working directory: project root,
JavaScript file:
node_modules\grunt-jasmine-node\node_modules\jasmine-node\lib\jasmine-node\cli.js
This is for my case (using grunt-jasmine-node). We can also provide path to cli.js of jasmine-node installed globally.
Application variables:
--verbose src/test/js
Of course the last one is a path to directory of tests in my project. Here we can specify list of files or use other jasmine-node commands and parameters.
The problem that process.exit() does not wait the stdout. You should defer the exit call with one tick, that's all.