TheIntern JS testing framework: can I customize the output of a test in the browser client? - intern

Question

In my equality tests I need to compare two data structures that have a really intuitive graphical representation, easily displayable on an HTML5 canvas.
Can I write some sort of plugin to alter how a mismatch is reported, i.e. to display two canvases side-by-side and additional UI elements to aid the user in conducting a visual test?
I started by looking into custom reporters and this is what I started to come up with (for simplicity, no canvases).
define([], function () {
var MyReporter = function (config) {
this.document = config.document || window.document;
this.localStorage = config.localStorage || window.localStorage;
};
MyReporter.prototype.testStart = function (test) {
test.correctResult = this.localStorage.getItem("uts." + test.parent.name + "." + test.name);
};
MyReporter.prototype.testEnd = function (test) {
var me = this;
var div = this.document.createElement("DIV");
if (test.hasPassed) {
div.innerHTML = test.name + ": OK";
}
else {
if ("actual" in test.error && "expected" in test.error) {
div.innerHTML = test.name + ": Fail. Do you want to set '" + test.error.actual + "' as the new expected result?";
var btn = document.createElement("BUTTON");
btn.innerHTML = "Mark as correct";
btn.addEventListener("click", function () {
me.localStorage.setItem("uts." + test.parent.name + "." + test.name, test.error.actual);
}, false);
div.appendChild(btn);
}
else {
div.innerHTML = test.name + ": Fail.";
}
}
this.document.body.appendChild(div);
};
return MyReporter;
});
Basically, if there is an equality assertion error the output is customized and the user is given the option to mark the result as correct (I need this because there is no initial ground truth as to what is correct and what is not; the user is supposed to inspect the actual and expected graphical representations and accept them once).
My actual questions, in detail:
Is this a proper use of the custom reporters feature?
In particular, note that I set test.correctResult in the testStart event, to use it in my suite. Is this ok?
Currently the reporter I wrote outputs a dull black & white page. How can I retain the stylish HtmlReporter visuals and customize the inside of the test result box? I was thinking of subclassing HtmlReporter, somehow, but before venturing into it I would like some feedback.
Is it possible to apply the reporter to a single test suite? I have several tests, most of them traditional; I don't want my custom reporter to kick in for those. But the reporters setting is a global of the config object. Is there a way around this?

Answer 1

Is this a proper use of the custom reporters feature?
Yes, reporters are the intended way to provide custom output.
In particular, note that I set test.correctResult in the testStart event, to use it in my suite. Is this ok?
Data isn't expected to flow from the reporter back into the testing process. There's no guarantee that the "test" passed to a reporter will be the actual test object that is run (it may not even be an instance of Test). Expected data should be loaded in the test itself. You could call a utility function or use a test function wrapper to avoid having to repeat as much code.
function addStorageTest(suite, testName, testFunc) {
suite[testName] = function () {
var correctResult = this.localStorage.getItem("uts." +
suite.name + "." + testName);
var actualResult = testFunc();
assert.strictEqual(actualResult, correctResult);
}
}
var suite = { name: 'a suite' };
addStorageTest(suite, 'a test', function () {
// do test stuff
return result;
});
addStorageTest(suite, 'another test', function () {
// do different test stuff
return result;
});
registerSuite(suite);
Currently the reporter I wrote outputs a dull black & white page. How can I retain the stylish HtmlReporter visuals and customize the inside of the test result box? I was thinking of subclassing HtmlReporter, somehow, but before venturing into it I would like some feedback.
Subclassing the Html reporter would probably be the easiest way to keep the same look. Another option would be for the custom reporter to store whatever data it needed to generate the report, then update the existing Html report all at once in the DOM when the testing process finished.
Is it possible to apply the reporter to a single test suite? I have several tests, most of them traditional; I don't want my custom reporter to kick in for those. But the reporters setting is a global of the config object. Is there a way around this?
There's currently no way to filter what tests and suites are sent to a reporter, but the reporter can filter the tests and suites that it receives. It can inspect the test and suite IDs and only process those that match some filter. The filter itself could be configured using a reporter config property.