I've been trying to run some e2e tests with Karma.
It's not working for me at all.
Right now I'm getting the following error:
Firefox 28.0.0 (Windows 7) ERROR
ReferenceError: module is not defined
at C:/MYPATH/Test/node_modules/karma-ng-scenario/lib/ind
ex.js:12
Firefox 28.0.0 (Windows 7) ERROR
ReferenceError: browser is not defined
at C:/MYPATH/Test/e2e/scenarios.js:12
My config file looks like this:
module.exports = function(config){
config.set({
basePath : './',
frameworks: ['ng-scenario'],
files : [
'./node_modules/karma-ng-scenario/lib/*.js',
'./e2e/*.js'
],
autoWatch : true,
singleRun : true,
browsers : ['Firefox'],
plugins : [
'karma-ng-scenario',
'karma-chrome-launcher',
'karma-firefox-launcher'
],
junitReporter : {
outputFile: 'test_out/unit.xml',
suite: 'unit'
},
urlRoot : '/__karma/',
proxies : {
'/public/' : 'http://localhost:8080'
}
});
};
My scenarios file just tests to see if the base path redirects.
I've already done a lot of messing with npm to get to this point, most recently "npm install karma-ng-scenario --save-dev" but no luck unfortunately.
In files[] specify path to angular.js as a very first, then any other angular modules you use (angular-mocks.js, angular-resource.js, angular-cookies.js), then any library you use, then your own code.
In files:[...] array you have to specify all the files that contain the actual code to which you are writing tests, if your module is using other modules, then you should add files of all the modules on which your module depends.
suppose you are testing 'someModule' module you have to include the 'someModule' controllers, views, services, directives and other modules if your module depends on them
Note: Assuming your files are in their specific directories
files: [
... //angularjs, angular-mocks, karma, protractor and other files you need
...
'scripts/someModule/controllers/*.js',
'scripts/someModule/services/*.js',
'scripts/someModule/directives/*.js',
'views/someModule/*.html'
]
or simply
files: [
... //angularjs, angular-mocks, karma, protractor and other files you need
...
'scripts/someModule/**/*.js',
'views/someModule/*.html'
]
And make sure to install and include all the testing libraries you depend on (i.e, angularjs, angular-mocks, protrator, karma etc) in the files array
So the configuration I finally got my E2E tests to work with was this:
// Karma configuration
// Generated on Fri Apr 11 2014 02:35:20 GMT+0800 (China Standard Time)
module.exports = function(config) {
config.set({
// base path, that will be used to resolve files and exclude
basePath: '..',
// frameworks to use
frameworks: ['jasmine'],
// list of files / patterns to load in the browser
files: [
ANGULAR_SCENARIO,
ANGULAR_SCENARIO_ADAPTER,
'test/e2e/*.js',
],
// list of files to exclude
exclude: [
],
// test results reporter to use
// possible values: 'dots', 'progress', 'junit', 'growl', 'coverage'
reporters: ['progress'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// Start these browsers, currently available:
// - Chrome
// - ChromeCanary
// - Firefox
// - Opera (has to be installed with `npm install karma-opera-launcher`)
// - Safari (only Mac; has to be installed with `npm install karma-safari-launcher`)
// - PhantomJS
// - IE (only Windows; has to be installed with `npm install karma-ie-launcher`)
//browsers: ['PhantomJS', 'Firefox', 'Chrome'],
browsers: ['Firefox', 'Chrome'],
// If browser does not capture in given timeout [ms], kill it
captureTimeout: 60000,
// Continuous Integration mode
// if true, it capture browsers, run tests and exit
singleRun: false,
proxies : {
"/": "http://localhost:8080"
},
urlRoot : "/__karma/"
});
};
It seems that a better solution for running end to end tests is to use protractor. NG-SCENARIO is depreciated and will cause warnings but this should still run. https://github.com/angular/protractor
Related
I created a sample react starter kit project in webstorm using webstorm's pre-defined project template and am trying to set breakpoints in debug mode.
I first built the project using npm run build then set the debug configuration to run build/server.js.
However it won't recognize any of the breakpoints in the original source files and seems to be ignoring the sourcemaps. How can I get it to recognize the sourcemaps and allow me to both set breakpoints in the source files as well as step into the source files.
There is this issue in the react starter kit repo: https://github.com/kriasoft/react-starter-kit/issues/121 but I couldn't see what the resolution was, and unlike the commenter, I couldn't even get it to step into the source files... it just stayed on the generated js files instead.
Well...
WebStorm 10 has no support for sourcemaps generated by Webpack. They are partially supported in WebStorm 11 for client-side applications (see http://blog.jetbrains.com/webstorm/2015/09/debugging-webpack-applications-in-webstorm/), but not supported for Node.js.
so, you can't debug server.js in WebStorm 11, but you can debug client side. To do this, try the following:
change appConfig in src/config.js as follows:
const appConfig = merge({}, config, {
entry: [
...(WATCH ? ['webpack-hot-middleware/client'] : []),
'./src/app.js',
],
output: {
path: path.join(__dirname, '../build/public'),
filename: 'app.js',
},
devtool: 'source-map',
module: {
loaders: [
WATCH ? {
...JS_LOADER,
query: {
// Wraps all React components into arbitrary transforms
// https://github.com/gaearon/babel-plugin-react-transform
plugins: ['react-transform'],
extra: {
'react-transform': {
transforms: [
{
transform: 'react-transform-hmr',
imports: ['react'],
locals: ['module'],
}, {
transform: 'react-transform-catch-errors',
imports: ['react', 'redbox-react'],
},
],
},
},
},
} : JS_LOADER,
...config.module.loaders,
{
test: /\.css$/,
loader: 'style-loader/useable!css-loader!postcss-loader',
},
],
},
});
set up the javascript debug run configuration:
URL: http://localhost:5000
Remote URLs: map project root folder to 'webpack:///path/to/react-starter-kit', like 'webpack:///C:/WebstormProjects/react-starter-kit'
map build/public to http://localhost:5000
This doesn't work perfectly, but works in general - breakpoints in src/routes.js, src/app.js are hit
I am using Karma and Jasmine to run tests on my project built with React. When I try to run my Karma tests I get this error in the console:
Running "karma:test" (karma) task
WARN `start` method is deprecated since 0.13. It will be removed in 0.14. Please use
server = new Server(config, [done])
server.start()
instead.
07 09 2015 14:46:56.552:WARN [plugin]: Error during loading "karma-opera-launcher" plugin:
ENOENT, no such file or directory './config/prefs.ini'
Hash: 8344a6c0a9b3c44a5636
Version: webpack 1.12.1
Time: 6ms
webpack: bundle is now VALID.
07 09 2015 14:46:56.685:INFO [karma]: Karma v0.13.9 server started at http://localhost:9876/
07 09 2015 14:46:56.689:INFO [launcher]: Starting browser Chrome
07 09 2015 14:46:56.700:INFO [launcher]: Starting browser Firefox
07 09 2015 14:46:56.713:INFO [launcher]: Starting browser PhantomJS
07 09 2015 14:46:57.063:INFO [PhantomJS 1.9.8 (Linux 0.0.0)]: Connected on socket X4i_xm1JTKdTSJO2AAAA with id 74978391
PhantomJS 1.9.8 (Linux 0.0.0) ERROR
Error: Module name "simple_test.js" has not been loaded yet for context: _. Use require([])
http://requirejs.org/docs/errors.html#notloaded
at /home/michael/repository/short-stories/node_modules/requirejs/require.js:140
07 09 2015 14:46:58.890:INFO [Chromium 44.0.2403 (Ubuntu 0.0.0)]: Connected on socket K4FoGDjsszglqxvVAAAB with id 26278080
Chromium 44.0.2403 (Ubuntu 0.0.0) ERROR
Uncaught Error: Module name "simple_test.js" has not been loaded yet for context: _. Use require([])
http://requirejs.org/docs/errors.html#notloaded
at /home/michael/repository/short-stories/node_modules/requirejs/require.js:140
07 09 2015 14:47:02.441:INFO [Firefox 40.0.0 (Ubuntu 0.0.0)]: Connected on socket EJQMu5bHS1DnJLRXAAAC with id 52731426
Firefox 40.0.0 (Ubuntu 0.0.0) ERROR
Error: Module name "simple_test.js" has not been loaded yet for context: _. Use require([])
http://requirejs.org/docs/errors.html#notloaded
at /home/michael/repository/short-stories/node_modules/requirejs/require.js:165
Warning: Task "karma:test" failed. Use --force to continue.
I have tried the recommendations at http://requirejs.org/docs/errors.html#notloaded, but it didn't change the error. I also tried making my require statements into one big callback hell, but it didn't fix the issue. I'm sure the problem is that it's trying to run the tests before loading the modules, but if I can't make it wait by using callbacks, I'm not sure how to deal with the issue. Any help would be greatly appreciated, Thanks in advance.
I'll paste my karma.conf.js, story_test.js, and test_entry.js files below. Let me know if there is more info that will help. You can view the full project on my GitHub Repo.
Here's my karam.conf.js:
// Karma configuration
// Generated on Thu Jul 02 2015 15:56:34 GMT-0700 (PDT)
module.exports = function(config) {
config.set({
// base path that will be used to resolve all patterns (eg. files, exclude)
basePath: '/',
// frameworks to use
// available frameworks: https://npmjs.org/browse/keyword/karma-adapter
frameworks: ['jasmine', 'requirejs'],
// list of files / patterns to load in the browser
files: [
__dirname + '/node_modules/phantomjs-polyfill/bind-polyfill.js',
__dirname + '/node_modules/requirejs/require.js',
__dirname + '/node_modules/karma-requirejs/lib/adapter.js',
__dirname + '/test/karma_tests/*entry.js'
],
//plugins to start browsers
plugins : [
'karma-junit-reporter',
'karma-phantomjs-launcher',
'karma-chrome-launcher',
'karma-firefox-launcher',
'karma-opera-launcher',
'karma-ie-launcher',
'karma-jasmine',
'karma-chai',
'karma-webpack',
'karma-requirejs',
'karma-script-launcher'
],
// list of files to exclude
exclude: [
],
// preprocess matching files before serving them to the browser
// available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
preprocessors: {
'test/karma_tests/*test.js': ['webpack'],
// 'test/**/*_test.js': ['webpack']
},
webpack: {
// karma watches the test entry points
// (you don't need to specify the entry option)
// webpack watches dependencies
// webpack configuration
module: {
loaders: [{
test: /\.jsx$/,
loader:'jsx-loader'
}]
}
},
// test results reporter to use
// possible values: 'dots', 'progress'
// available reporters: https://npmjs.org/browse/keyword/karma-reporter
reporters: ['progress'],
// web server port
port: 9876,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
logLevel: config.LOG_INFO,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: false,
// start these browsers
// available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
browsers: ['Chrome', 'Firefox', 'PhantomJS'],
// Continuous Integration mode
// if true, Karma captures browsers, runs the tests and exits
singleRun: true,
// Set timeout to 100 seconds
// browserNoActivityTimeout: 100000
});
};
Here's my story_test.js:
'use strict';
var React = require('react/addons');
var Story = require('../../app/js/components/story.jsx');
var TestUtils = React.addons.TestUtils;
var testUtilsAdditions = require('react-testutils-additions');
describe('Story component', function () {
var component;
beforeEach(function () {
var element = React.createElement(Story);
element.props.data = {
storyTitle: 'front end test title',
author : 'front end author',
storyText : 'front end story text'
};
component = TestUtils.renderIntoDocument(element);
});
it('should display a story', function () {
expect(component.props.data).toBeDefined();
expect(component.props.data.storyTitle).toBeDefined();
expect(component.props.data.storyTitle).toBe('front end test title');
expect(component.props.data.author).toBe('front end author');
expect(component.props.data.storyText).toBe('front end story text');
});
});
And finally here's the test_entry.js:
'use strict';
require('./simple_test.js');
require('./story_test.js');
The error seems to come from the http://requirejs.org project and I think is trying to get you to change from using synchronous CommonJS require calls such as;
require('./simple_test.js');
require('./story_test.js');
to asynchronous AMD calls such as;
require(['./simple_test.js', './story_test.js'], function() {
// simple_test and story_test are now loaded
});
It's not clear to me from what you've posted which modules are requirejs/AMD and which aren't, it looks like you may possibly have a confusing mixture of the two? Sorry I can't help much more than this.
I have a project where Intern unit tests are supposed to be in a different directory tree than the source code under test. Somewhat like this:
projectRoot
projectRoot/src
projectRoot/tests
projectRoot/tests/intern.js
projectRoot/tests/node_modules/intern
projectRoot/tests/MyTestSuite.js
In the Intern configuration file, I define an AMD package that uses relative paths with ../ to reach src from the unit test suites. Here's an example configuration:
define({
environments: [ { browserName: 'chrome', platform: 'WINDOWS' }],
webdriver: { host: 'localhost', port: 4444 },
useSauceConnect: false,
loader: {
packages: [
{ name: 'testSuites', location: '.' },
{ name: 'externalDep', location: '../src' }
]
},
suites: [ 'testSuites/MyTestSuite' ]
});
And a matching unit test suite
define([ "intern!tdd", "intern/chai!assert","externalDep/ExternalDep"],
function(tdd, assert, ExternalDep) {
tdd.suite("Suite that has external dependency", function() {
tdd.test("Test if external dependency is loaded correctly", function() {
assert(ExternalDep === "hello");
});
});
}
);
This works fine when tested directly in the browser (client.html) or node (client.js). When fired off through a Selenium Server (with runner.js), however, the client.html running in the browser started by Selenium can't find the external dependencies. In the above example, it tries to request ExternalDep at http://localhost:9000/__intern/src/ExternalDep.js, which is a 404 because the src directory is not within intern.
I suppose that if I put intern.js at the highest common super-directory of both the tests and the source code, it would work. But our project is currently set up in a way which makes that impractical. Is there a way for configuring sources that live beyond the location of the Intern config file, or did I just make a silly mistake?
Thanks!
There is no problem putting the tests in a different directory from the rest of the code, but projectRoot needs to be the working directory from which you start the runner, and you need to change your loader configuration to match.
So, instead of right now where you are starting Intern from projectRoot/tests like this:
…/projectRoot/tests$ ./.bin/intern-runner config=intern
you need to start it from projectRoot:
…/projectRoot$ ./tests/.bin/intern-runner config=tests/intern
…and change your loader configuration:
loader: {
packages: [
{ name: 'testSuites', location: 'tests' },
{ name: 'externalDep', location: 'src' }
]
},
The title says it all really. Despite trawling the internet I haven't found a single example of a solution to this problem.
Here are some near misses
https://github.com/amitayd/grunt-browserify-jasmine-node-example - grunt, browserify and jasmine
https://github.com/gotwarlost/istanbul/issues/59#issuecomment-18799734 - browserify and istanbul
Here is my in-progress code https://github.com/wheresrhys/on-guard/tree/browserify (note it's the 'browserify' branch - Gruntfile.js is a bit of a mess but will tidy it up shortly). My initial investigations using console.log indicate that somehow bundle.src.js is being loaded in the page but when the tests are run (and passed!) the code in bundle.src.js isn't being run, so I have a feeling it might be an aliasing problem... though one that's limited to phantomjs as when I open the specrunner in chrome the code is getting run.
I'm using grunt-browserify + browserify-istanbul + grunt-contrib-jasmine + grunt-template-jasmine-istanbul as solution. This solution has also excluded third party libraries when building source files using browserify.
Show the code first, I'll explain later,
grunt.initConfig({
browserify: {
// build specs using browserify
specs: {
src: ["spec/**/*Spec.js"],
dest: "spec/build/specs.js",
options: {
debug: true
}
},
// build source files using browserify and browserify-istanbul
dev: {
options: {
debug: true,
browserifyOptions: {
standalone: 'abc'
},
transform: [['browserify-istanbul', {
ignore: ['**/node_modules/**'], // ignore third party libs
defaultIgnore: true
}]]
},
src: ['abc.js'],
dest: 'dist/abc.js'
}
},
connect: {
server: {
options: {
port: 7000
}
}
},
// test using jasmine, generate coverage report using istanbul
jasmine: {
coverage: {
src: ['dist/abc.js'],
options: {
junit: {
path: 'bin/junit'
},
host: 'http://localhost:7000/',
specs: 'spec/build/specs.js',
keepRunner: true,
summary: true,
template: require('grunt-template-jasmine-istanbul'),
templateOptions: {
replace: false, // *** this option is very important
coverage: 'bin/coverage/coverage.json',
report: [
{
type: 'html',
options: {
dir: 'spec/coverage/html'
}
}]
}
}
}
}
grunt.registerTask('specs', ['browserify:specs', 'browserify:dev', 'connect', 'jasmine']);
The steps of generating istanbul coverage report can be concluded into three:
Instrument code
Run test
Generate coverage report
In our solution, we use browerify-istanbul in step 1, grunt-contrib-jasmine and runt-template-jasmine-istanbul in step 2 and 3.
browserify-istanbul will let you instrument code in browserify building step, in this way, we can easily ignore third party libs. But the grunt-template-jasmine-istanbul will instrument code again. To avoid this, you can set replace to false in the options.
Refs:
Istanbul steps
broswerify-istanbul
grunt-contrib-jasmine
grunt-template-jasmine-istanbul -- replace option
Is this possible with CommonJS?
Basically I'm trying to take the API testing from http://thewayofcode.wordpress.com/2013/04/21/how-to-build-and-test-rest-api-with-nodejs-express-mocha/
and use Karma to run the tests.
I'm attempted to use RequireJS with karma, based off http://karma-runner.github.io/0.10/plus/requirejs.html
My package.json is correctly setup and 'npm install' gets everything I need,
but when I do 'karma start test/karma.conf.js' , no tests run
DEBUG [karma]: All browsers are ready, executing
DEBUG [web-server]: serving: /home/npoklitar/project/node_modules/karma/static/context.html
DEBUG [web-server]: serving: /home/npoklitar/project/node_modules/karma-requirejs/lib/require.js
DEBUG [web-server]: serving: /home/npoklitar/project/node_modules/karma-requirejs/lib/adapter.js
DEBUG [web-server]: serving: /home/npoklitar/project/node_modules/mocha/mocha.js
DEBUG [web-server]: serving: /home/npoklitar/project/node_modules/karma-mocha/lib/adapter.js
DEBUG [web-server]: serving: /home/npoklitar/project/test/routerSpec.js
DEBUG [web-server]: serving: /home/npoklitar/project/test/test-main.js
ERROR: 'There is no timestamp for /base/supertest.js!'
Chrome 30.0.1599 (Linux): Executed 0 of 0 SUCCESS (0 secs / 0 secs)
ERROR: 'There is no timestamp for /base/should.js!'
Chrome 30.0.1599 (Linux): Executed 0 of 0 SUCCESS (0 secs / 0 secs)
ERROR: 'There is no timestamp for /base/assert.js!'
Chrome 30.0.1599 (Linux): Executed 0 of 0 SUCCESS (0 secs / 0 secs)
Chrome 30.0.1599 (Linux): Executed 0 of 0 ERROR (0.355 secs / 0 secs)
DEBUG [launcher]: Disconnecting all browsers
DEBUG [launcher]: Killing Chrome
test/rounterSpec.js
require(['supertest','should','assert'], function(supertest,should,assert){
describe('Routing:', function() {
var url = 'http://localhost:16000';
describe('API', function() {
it('should return the success string and request headers', function(done){
supertest(url)
.get('/api')
.expect(200)
.end(function(err, res) {
if (err) {
throw err;
}
var text = res.text;
var splitted = text.split('!');
splitted[0].should.include('request successfully proxied to API');
done();
});
});
});
});
});
test/karma.conf.js
module.exports = function (karma) {
karma.set({
// base path, that will be used to resolve files and exclude
basePath: '../',
frameworks: ['mocha','requirejs'],
// list of files / patterns to load in the browser
files: [
// {pattern: 'node_modules/chai/chai.js', include: true},
// {pattern: '*.js', include: false},
'test/*.js',
'test/test-main.js'
],
// list of files to exclude
exclude: [
'karma.conf.js'
],
// use dots reporter, as travis terminal does not support escaping sequences
// possible values: 'dots', 'progress', 'junit', 'teamcity'
// CLI --reporters progress
reporters: ['progress', 'junit', 'coverage'],
junitReporter: {
// will be resolved to basePath (in the same way as files/exclude patterns)
outputFile: 'junit-report/test-results.xml'
},
preprocessors: {
'src/*.js': 'coverage'
},
//Code Coverage options. report type available:
//- html (default)
//- lcov (lcov and html)
//- lcovonly
//- text (standard output)
//- text-summary (standard output)
//- cobertura (xml format supported by Jenkins)
coverageReporter: {
// cf. http://gotwarlost.github.com/istanbul/public/apidocs/
type: 'lcov',
dir: 'coverage/'
},
// web server port
port: 9876,
// cli runner port
runnerPort: 9100,
// enable / disable colors in the output (reporters and logs)
colors: true,
// level of logging
// possible values: LOG_DISABLE || LOG_ERROR || LOG_WARN || LOG_INFO || LOG_DEBUG
logLevel: LOG_DEBUG,
// enable / disable watching file and executing tests whenever any file changes
autoWatch: true,
// Start these browsers, currently available:
// - Chrome
// - ChromeCanary
// - Firefox
// - Opera
// - Safari (only Mac)
// - PhantomJS
// - IE (only Windows)
// CLI --browsers Chrome,Firefox,Safari
browsers: ['Chrome'],
// If browser does not capture in given timeout [ms], kill it
captureTimeout: 6000,
// Continuous Integration mode
// if true, it capture browsers, run tests and exit
singleRun: true,
plugins: [
'karma-mocha',
'karma-chrome-launcher',
'karma-firefox-launcher',
'karma-junit-reporter',
'karma-coverage',
'karma-requirejs'
]
});
}
test/test-main.js
var tests = [];
for (var file in window.__karma__.files) {
if (/Spec\.js$/.test(file)) {
tests.push(file);
}
}
requirejs.config({
// Karma serves files from '/base'
baseUrl: '/base',
/*
paths: {
'jquery': '../lib/jquery',
'underscore': '../lib/underscore',
},
shim: {
'underscore': {
exports: '_'
}
},
*/
// nodeRequire: require, //doesnt work with or without this commented
// ask Require.js to load these files (all our tests)
deps: tests,
// start test run, once Require.js is done
callback: window.__karma__.start
});
I've created a plugin for Karma here: https://www.npmjs.com/package/karma-common-js
It let's you write tests as if you're using Browserify, but the plugin doesn't use Browserify. Not using Browserify has a few advantages:
There's no bundle created, so it's very fast for watching file changes
Line numbers and file names are preserved in stack traces without needing source maps
Works with karma-coverage
Lets you pass a second argument to require to pass in mocks
All (I hope) of Browserifys core features work. Such as transforms, respecting the browser field in package.json, requiring builtin modules uses the same shims as Browserify, etc.
There is now a CommonJS plugin for Karma: https://github.com/karma-runner/karma-commonjs
After trying a bunch of different plugins, I ended up using the karma-browserifast plugin that actually works quite well - especially if you run it in debug mode.