I am writing mocha tests with selenium-webdriver.js, and trying to take screenshot only if the current test failed.
In Mocha, I can get the current test info like title and state as follows:
afterEach(function(){
console.log('afterEach', this.currentTest.title, this.currentTest.state);
});
But selenium-webdriver.js wraps around Mocha's interface with selenium-webdriver/testing, and the original this.currentTest is not exposed anymore:
var test = require('selenium-webdriver/testing');
test.afterEach(function(){
//console.log('afterEach', this.currentTest.title, this.currentTest.state);
});
I am wondering if such information is still exposed somehow or is there any workaround for this.
this.title - returns suite name
this.ctx.currentTest.title - returns current test name
this.ctx.currentTest.state - returns current test state
Doesn't work if used arrow function for "describe".
Related
I have an application that runs a Jest test suite from the command line, then takes the JSON output, parses it and then fills table in a database as per the output file. The application runs shell command:
npm run all
and in the package.json file the all script looks like this:
"scripts": {
"all": "../node_modules/.bin/jest --json --outputFile=testResults.json",`
......
}
So I get the testResults.json file and I am able to parse it - so far so good.
But during the test case run I would like to add some extra data to the output. Something like details - where the problems is, how to fix it, some troubleshooting information etc. For example to put one more field in :
require('testResults.json').testResults[x].assertionResults[y].details
You see, the detail property is not part of the json output file format. But can I create it from within the test case (pseudo example):
test('Industry code should match ind_full_code', async () => {
result = await stageDb.query(QUERY);
// And here I want to add this custom information to some global property available?
reporter.thisTestCase.assertionResults.details = "Here is what you should do to fix this ...." // <- Ideally this is how easy I imagine it to be.
expect(result.results).toEqual([]);
}, 2 * 100 * 1000)
I just want to give a little bit more information to the QA or whomever on test failure.
In other words I need the option to change the output from within the test case.
I've been looking into custom reporters, but their listeners are passed the same information as to the json reported.
I've found a need for a similar feature in Jest. The ability to add documentation to the test is rarely supported by test frameworks.
However I found a way to do this with the soon to be default runner: Jest Circus. I then made my own Jest Circus environment. A custom Jest Circus environment provides more test events/lifecycles and access to the actual test code that is being ran.
// Example of a custom Jest Circus environment
export default class MyCustomNodeEnvironment extends NodeEnvironment {
handleTestEvent(event: Circus.Event, state: Circus.State) {
if(event.name === 'test_fn_start') {
console.log(event.test.toString())
// will log the actual test code.
}
}
}
// jest.config.js
{
"testEnvironment": "<rootDir>/my-custom-environment.js",
"testRunner": "jest-circus/runner"
}
I then used regex patterns to find comments in the test functions and add them to the Allure report (Allure report demo).
If you'd like to create your own Jest environment and implement this yourself I've made a template repo or if you prefer a gist of a basic Jest Circus environment.
If you like how Allure reports look you should checkout my open source project jest-circus-allure-environment.
well before specifying my problem, i want to tell that i'm new to the field of testing, so here is my problem:
i developed a rest api using express + sequelize(mysql), and i want to write some test for my api. i choosed to use jasmine library for testing.
so right know i want to test the create and update rest endpoint, i will need access to a database, but the problem is that the test cases are run in parallel, and there is only one database, so if i want to delete all item from a table in a test case, and another test case have create a row in that table, there will be a problem.
const request = require('superagent');
const models = require('../../src/models');
const Station = models.Station;
describe("station testing", function () {
before(() => {
// delete and recreate all database table
// before running any test
});
describe("crud station", function () {
it('should create model', () => {
Station.create({
'name': 'test',
lat: 12,
long: 123,
}).then( model => {
expect(model).toBeTruthy();
});
});
it('should delete evrything', () => {
Station.deleteAll().then( () => {
// problem here if after the first model is created and before create model except is executed
expect(Station.Count()).toEqual(0);
}
});
});
});
Your problem is that you are not writing unit tests here.
You need to understand the most important rule of unit testing - only test one unit at a time. A unit can be thought of as an area of your code. In a traditional desktop project (Java, C#, etc), a unit would be one class. In the case of Javascript, a unit is harder to define, but it certainly will only include the Javacript. If you are including any server code (for example, the database) in your tests, then you are not unit testing, you are doing integration testing (which is also very important, but much harder).
Your Javascript will have dependencies (ie other code that it calls, say via Ajax calls), which in your case will include the server code that is called. In order to unit test, you need to make sure that you are only testing the Javascript, which means that when running the tests, you don't want the server code to be called at all. That way, you isolate any errors in that unit of code, and can be confident that any problems found are indeed in that unit. If you include other units, then it could be the other units that have the problem.
In a strongly-typed language (like Java, C#, etc), there are frameworks that allow you to set up a mock for each dependency. Whilst I haven't tried any myself (that's this week's job), there are mocking frameworks for Javascript, and you would probably need to use one of them to do real unit testing. You mock out the server code, so when you run the test, it doesn't actually hit the database at all. Apart from solving your problem, it avoids a whole load of other issues that you will likely hit at some point with your current approach.
If you don't want to use a mocking framework, one other way to do it is to change your Javascript so that the function you are testing takes an extra parameter, which is a function that does the actual server call. So, instead of...
deleteCustomer(42);
deleteCustomer(id) {
validate(id);
$.ajax(...);
}
...your code would look like this...
deleteCustomer(42, callServer);
deleteCustomer(id, serverCall) {
validate(id);
serverCall(id);
}
...where serverCall() contains the Ajax call.
Then, to unit test, you would test something like this...
deleteCustomer(42, function(){});
...so that instead of calling the server, nothing is actually done.
This is obviously going to require some rewriting of your code, which could be avoided by mocking, but would work. My advice would be to spend some time learning how to use a mocking framework. It will pay off in the long run.
Sorry this has been a bit long. Unfortunately, you're getting into a complex area of unit testing, and it's important to understand what you're doing. I strongly recommend you read up about unit testing before you go any further, as a good understanding of the basics will save you a lot of trouble later on. Anything by Robert Martin (aka Uncle Bob) on the subject will be good, but there are plenty of resources around the web.
Hope this helps. If you want any more info, or clarification, ask away.
Jasmine supports a function for beforeEach which run before each spec in a describe block.
You can use that.
describe("A spec using beforeEach and afterEach", function() {
var foo = 0;
beforeEach(function() { foo += 1; });
afterEach(function() { foo = 0; });
it("is just a function, so it can contain any code", function() {
expect(foo).toEqual(1);
});
it("can have more than one expectation", function() {
expect(foo).toEqual(1)
expect(true).toEqual(true);
});
});
So you could let the beforeEach take care of the delete operation.
I'm trying to understand how Sinon is used properly in a node project. I have gone through examples, and the docs, but I'm still not getting it. I have setup a directory with the following structure to try and work through the various Sinon features and understand where they fit in
|--lib
|--index.js
|--test
|--test.js
index.js is
var myFuncs = {};
myFuncs.func1 = function () {
myFuncs.func2();
return 200;
};
myFuncs.func2 = function(data) {
};
module.exports = myFuncs;
test.js begins with the following
var assert = require('assert');
var sinon = require('sinon');
var myFuncs = require('../lib/index.js');
var spyFunc1 = sinon.spy(myFuncs.func1);
var spyFunc2 = sinon.spy(myFuncs.func2);
Admittedly this is very contrived, but as it stands I would want to test that any call to func1 causes func2 to be called, so I'd use
describe('Function 2', function(){
it('should be called by Function 1', function(){
myFuncs.func1();
assert(spyFunc2.calledOnce);
});
});
I would also want to test that func1 will return 200 so I could use
describe('Function 1', function(){
it('should return 200', function(){
assert.equal(myFuncs.func1(), 200);
});
});
but I have also seen examples where stubs are used in this sort of instance, such as
describe('Function 1', function(){
it('should return 200', function(){
var test = sinon.stub().returns(200);
assert.equal(myFuncs.func1(test), 200);
});
});
How are these different? What does the stub give that a simple assertion test doesn't?
What I am having the most trouble getting my head around is how these simple testing approaches would evolve once my program gets more complex. Say I start using mysql and add a new function
myFuncs.func3 = function(data, callback) {
connection.query('SELECT name FROM users WHERE name IN (?)', [data], function(err, rows) {
if (err) throw err;
names = _.pluck(rows, 'name');
return callback(null, names);
});
};
I know when it comes to databases some advise having a test db for this purpose, but my end-goal might be a db with many tables, and it could be messy to duplicate this for testing. I have seen references to mocking a db with sinon, and tried following this answer but I can't figure out what's the best approach.
You have asked so many different questions on one post... I will try to sort out.
Testing myFuncs with two functions.
Sinon is a mocking library with wide features. "Mocking" means you are supposed to replace some part of what is going to be tested with mocks or stubs. There is a good article among Sinon documentation which describes the difference well.
When you created a spy in this case...
var spyFunc1 = sinon.spy(myFuncs.func1);
var spyFunc2 = sinon.spy(myFuncs.func2);
...you've just created a watcher. myFuncs.func1 and myFuncs.func2 will be substituted with a spy-function, but it will be used to record the calling arguments and call real function after that. This is a possible scenario, but mind that all possibly complicated logic of myFuncs.func1/func2 will run after being called in the test (ex.: database query).
2.1. The describe('Function 1', ...) test suite looks too contrived to me.
It is not obvious which question you actually mean.
A function that returns a constant value is not a real life example.. In most cases there will be some parameters and the function under test will implement some algorithm of transmuting the input arguments. So in your test you're going to implement the same algorithm partly to check that the function works correctly. That is where TDD comes in place, which actually supposes you start implementation from a test and take parts of unit test code to implement the method being tested.
2.2. Stub. The second version of unit test looks useless in the given example. func1 is not accepting any parameter.
var test = sinon.stub().returns(200);
assert.equal(myFuncs.func1(test), 200);
Even if you replace the return part with 100 the test will run successfully.
The one that makes sense is, for example, replacing func2 with a stub to avoid heavy calculation/remote request (DB query, http or other API request) being launched in a test.
myFuncs.func2 = sinon.spy();
assert.equal(myFuncs.func1(test), 200);
assert(myFuncs.func2.calledOnce);
The basic rule for unit testing is that unit test should be kept as simple as possible, providing check for as smallest possible fragment of code. In this test func1 is being tested so we can neglect the logic of func2. Which should be tested in another unit test.
Mind that doing the following attempt is useless:
myFuncs.func1 = sinon.stub().returns(200);
assert.equal(myFuncs.func1(test), 200);
Because in this case you masked the real func1 logic with a stub and you're actually testing sinon.stub().return(). Believe me it works well! :D
Mocking database queries.
Mocking a database has always been a hurdle. I could provide some advises.
3.1. Have well fragmented environment. Even for a small project there better exist a development, stage and production completely independent environments. Including databases. Which means you have an automated way of DB creation: scripts or ORM.
In this scenario you will be easily maintaining test DB in your test engine using before()/beforeEach() to have a clean structure for your tests.
3.2. Have well fragmented code. There'd better exist several layers. The lowest (DAL) should be separated from business logic. In this case you will write code for business class, simply mocking DAL. To test DAL you can have the approach you mentioned (sinon.mock the whole module) or some specific libraries (ex.: replace db engine with SQLite for tests as described here)
Conclusion. "how these simple testing approaches would evolve once my program gets more complex".
It is hard to maintain unit tests unless you develop your application with tests in mind and, thus, well fragmented. Stick to the main rule - keep each unit test as small as possible. Otherwise you're right, it will eventually get messy. Because the constantly evolving logic of your application will be involved in your test code.
Is it possible for a single functional test to handle 2 different pages? For example, when I execute the following test:
return this.remote
.get(testPage)
.waitForElementByCssSelector('.alfresco-core-Page.allWidgetsProcessed', 5000)
.elementByCss('#UNIT_TEST_MODEL_FIELD>DIV.control>TEXTAREA')
.type(testData)
.end()
.elementByCss("#LOAD_TEST_BUTTON")
.click()
.sleep(2000)
.waitForElementByCssSelector('.alfresco-core-Page.allWidgetsProcessed', 5000)
.elementByCss("#DD1")
.click()
.end();
I get the following error:
Test main - firstTest - Test1 FAILED on chrome 31.0.1650.57 on LINUX:
Error: Cannot call method 'apply' of undefined
TypeError: Cannot call method 'apply' of undefined at null.<anonymous> (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/lib/util.js:108:10)
at /home/dave/ScratchPad/ShareInternTests/node_modules/intern/lib/wd.js:769:29
at signalListener (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/node_modules/dojo/Deferred.js:37 :21)
at signalWaiting (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/node_modules/dojo/Deferred.js:28 :4)
at resolve (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/node_modules/dojo/Deferred.js:19 2:5)
at signalDeferred (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/node_modules/dojo/Deferred.js:81 :15)
at signalListener (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/node_modules/dojo/Deferred.js:52 :6)
at signalWaiting (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/node_modules/dojo/Deferred.js:28 :4)
at resolve (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/node_modules/dojo/Deferred.js:19 2:5)
at signalDeferred (/home/dave/ScratchPad/ShareInternTests/node_modules/intern/node_modules/dojo/Deferred.js:81 :15)
Because of the framework that I'm testing I need to load a "bootstrap" test page which I write the test data into and then POST it to persist a test model into the HTTP session which is then rendered in a resultant page.
However, whilst the first part of the test works fine (I see the test data entered and the next page is submitted, I seem to get the error on the 2nd .waitForElementByCssSelector call. I've tried various permutations but can't get this to work.
If I run a completely second test on the second page then this works fine, but ideally I'd like it all captured within a single test.
Is what I'd like to do possible or do I have to break it into separate tests?
Please try using setImplicitWaitTimeout and use the elementBy* methods instead of using waitForElementByCssSelector. It is more efficient, and should work properly.
A second option is to make sure you call end before the second waitForElementByCssSelector call; it looks like there is a defect in the way the waitForElement methods are called, where the #LOAD_TEST_BUTTON element is being used as the context for the call but waitForElement accepts no context.
In Node.js unit tests, what is the way to create data driven unit tests?
For Example, I've a common function / method, which I want to re-use in multiple unit tests with different sets of data. I tried looking into nodeunit, vows, whiskey, qunit, expresso; But I wasn't able to figure out a way to achieve this functionality.
I was not looking at calling the function literally in multiple tests, but rather use the common method in a loop to pick up the data in each iteration and execute it, as a unittest
The reason for this is, I've atleast 1000 rows of parameterized data, for which I want to write unittest. Obviously I cannot go on writing 1000 unittests physically.
Anyone could you please point me a way to achieve the above.
There is qunit addon which allows to run parameterized quint tests
https://github.com/AStepaniuk/qunit-parameterize
So you can separate test data and test method and run the same test method against different data sets.
This is a pretty old post, but I just hit this problem myself and wasn't able to find a clean solution for QUnit without using the plugin referenced by the other comment (qunit-parameterize). Honestly, I couldn't figure out how to integrate the plugin with my company's project and gave up after about an hour.
This is how I ended up solving it:
Just define an array with your inputs (and expected outputs, if needed), iterate over your array, and define the QUnit test in the callback! Super simple, really, but worked quite well.
const testCases = [
{ input: "01/01/2015", expected: "2015-01-01" },
{ input: "09/25/2015", expected: "2015-09-01" },
{ input: "12/31/2015", expected: "2015-12-01" }
];
testCases.forEach(testCase => {
QUnit.test("gets first of month",
() => {
const actual = new classUnderTest().getFirstOfMonth(testCase.input);
strictEqual(actual, testCase.expected);
});
});
I wasn't sure that QUnit would discover the test if it were nested as such, but it does just fine.
Enjoy!