Is there a conventional way to attempt a group of asserts to always be evaluated before failing the test?
Let's say my test assesses the presence of some names on a page:
var pageContent = 'dummy page content';
//.include(haystack, needle, [message])
//Asserts that haystack includes needle.
assert.include(pageContent, 'Alice');
assert.include(pageContent, 'Bob');
assert.include(pageContent, 'John');
Now, if Alice is missing, the test would fail with a single error:
>AssertionError: expected 'dummy page content' to contain 'Alice'
However I want to be notified that all three names are missing since in this case failing one condition does not prevent from evaluating others.
Rather than writing a wrapper method that aggregates the would-be output of these checks and throws a single error, I was hoping there would third-party library that "specializes" in this sort of thing or perhaps in-built functionality I'm overlooking.
I can offer two approaches.
The first one, mentioned by #Peter Lyons, relies on converting multiple assertions into a single assertion on multiple values. To keep the assertion error message useful, it's best to assert on the list of names:
var expected = ['Alice', 'Bob', 'John'];
var found = expected.filter(function(name) {
return pageContent.indexOf(name) >= 0;
}
// assuming Node's require('assert')
assert.deepEqual(found, expected);
// Example error message:
// AssertionError: ["Alice","Bob","John"] deepEqual ["Alice"]
The second approach uses "parameterised test". I'll assume you are using BDD-style for specifying test cases in my code.
describe('some page', function() {
for (var name in ['Alice', 'Bob', 'John'])
itContainsString(name);
function itContainsString(name) {
it('contains "' + name + '"', function() {
var pageContent = 'dummy page content';
assert.include(pageContent, 'Alice');
});
}
}
var found = ['Alice', 'Bob', 'John'].map(function (name) {
return pageContent.indexOf(name) >= 0;
});
assert.include(found, true);
If I may opine that your desire for a wrapper library for fuzzy asserting sounds misguided. Your fuzzy rules and heuristics about what is a "soft" vs "hard" assertion failure seem like a much less sensible alternative than good old programming using the existing assertion paradigm. It's testing. It's supposed to be straightforward and easy to reason about.
Keep in mind you can always take logic such as the above and wrap it in a function called includesOne(pageContent, needles) so it is conveniently reusable across tests.
Another approach to validating multiple assertions and getting feedback from all of them, regardless of which one fails first, is to use a Node.js module I wrote and published called multi-assert. Now, before we continue, I want to point out that, in your specific use case, if you're really only asserting 3 names, then I think Miroslav's answer is quite sufficient. As long as it serves your needs for quickly determining what broke and what to do to fix it, then you should continue on with that solution, and there's no need for additional dependencies in your project.
But the areas where I've run into major difficulties, or needed to spend more time debugging test failures or help others do the same, is when we've asserted properties of a really large object or array. For instance, when using deepEqual, I've personally run into cases where the error message can be quite complex, confusing, and unreadable in both HTML reports as well as the logs.
With the multi-assert module, the error messages are transparently displayed and specific to what we want to measure. For example, we can do something like this:
// import { multiAssert } from 'multi-assert';
const { assert } = require('chai');
const { multiAssert } = require('multi-assert');
describe('Page Content Tests', () => {
it('should contain Alice, Bob, and John somewhere in the content', () => {
var pageContent = 'dummy page content';
//.include(haystack, needle, [message])
//Asserts that haystack includes needle.
multiAssert([
() => assert.include(pageContent, 'Alice'),
() => assert.include(pageContent, 'Bob'),
() => assert.include(pageContent, 'John')
]);
});
});
And by running these tests, with "dummy page content", we're going to see the following error messages reported back to us with full transparency:
Page Content Tests
1) should contain Alice, Bob, and John somewhere in the content
0 passing (7ms)
1 failing
1) Page Content Tests
should contain Alice, Bob, and John somewhere in the content:
AssertionError:
MultipleAssertionError: expected 'dummy page content' to include 'Alice'
at /Users/user123/Dev/page-content-example/test/page-content.spec.js:13:20
at /Users/user123/Dev/page-content-example/node_modules/multi-assert/src/multi-assert.js:10:13
at Array.forEach (<anonymous>)
MultipleAssertionError: expected 'dummy page content' to include 'Bob'
at /Users/user123/Dev/page-content-example/test/page-content.spec.js:14:20
at /Users/user123/Dev/page-content-example/node_modules/multi-assert/src/multi-assert.js:10:13
at Array.forEach (<anonymous>)
MultipleAssertionError: expected 'dummy page content' to include 'John'
at /Users/user123/Dev/page-content-example/test/page-content.spec.js:15:20
at /Users/user123/Dev/page-content-example/node_modules/multi-assert/src/multi-assert.js:10:13
at Array.forEach (<anonymous>)
at multiAssert (node_modules/multi-assert/src/multi-assert.js:19:15)
at Context.<anonymous> (test/page-content.spec.js:12:5)
at processImmediate (node:internal/timers:466:21)
I want to also note that the multi-assert module also works with other testing frameworks; it's not just limited to Mocha and Chai; however, it's worth noting that Jasmine has been evaluating soft assertions by default for quite some time due to the nature of how their test runner and assertion library are more tightly integrated. If switching to Jasmine is not an easy or desired solution, and if existing assertion methods alone don't provide the desired level of feedback, then you can see the simplicity in wrapping up existing assertions in a multiAssert function call in order to achieve this transparency in your test cases. Hoping this helps!
Related
I'm a long time tape.js user and I'm working on learning how to work jest. I'm interested in providing descriptions for each my test cases as part of the assertion, ala this tape test
function myCoolTest(t) {
t.equal('batman'.length, 6, 'batman should have the right number of characters in it');
t.ok(1 === 1, 'basic truths should stay true');
t.deepEqual({test: 1}, {test: 1}, 'deep equality of objects works sensibly');
t.end();
}
I like being able to annotate my tests (eg 'batman should have the right number of characters in it'), that way as I'm reading the output it's clear whats passed and what's failed. As far as I can tell the jest equivalent is
test('example test', () => {
expect('batman'.length).toBe(6);
expect(1 === 1).toBeTruthy();
expect({test: 1}).toBe({test: 1});
});
Which totally lacks description found in the first? While that's okay for simple examples like ^. The examples i've seen other places seem to suggest that if I want description I should add comments next to the relevant test, but this seems to prevent creating utility tests, eg
const expectEqual = (a: string, b: string): void =>
expect(JSON.parse(a)).toEqual(JSON.parse(b));
Am i just out of luck or are there methods that I am missing?
When I run this code I get push is not a function. I have gone over the code so many times and can't figure out where I went wrong. i have also read many of post and I still can't figure it out. I am new to programming and could use the help.
const fs= require('fs')
const getNotes = function() {
return 'This just returns get notes'
enter code here
};
const addNote = function (title, body) {
const notes = loadNotes()
notes.push({
title: title,
boby: body
})
saveNotes(notes)
};
const saveNotes = function (notes) {
const dataJSON = JSON.stringify(notes)
fs.writeFileSync('notes.json',dataJSON)
}
// Code below loads the notes. Above, addNote adds the note.
const loadNotes = function () {
try {
const dataBuffer = fs.readFileSync('notes.json')
const dataJSON= dataBuffer.toString()
return JSON.parse(dataJSON)
} catch (error) {
return('Note such file')
}
}
module.exports ={
getNotes: getNotes,
addNote: addNote
}
So, you have this:
const notes = loadNotes()
notes.push({
title: title,
boby: body
});
If you're getting an error that notes.push is not a function, then that is because loadNotes() is not return an array. That could be for a couple reasons:
JSON.parse(dataJson) successfully parses your json, but its top level object is not an array.
JSON.parse(dataJson) throws and you end up returning a string instead of an array.
You can fairly easily diagnose this by adding a console.log() statement like this:
const notes = loadNotes();
console.log(notes); // see what this shows
notes.push({
title: title,
boby: body
});
FYI, returning a string fromloadNotes()as an error really doesn't make much sense unless you're going to check for a string after calling that function. IMO, it would make more sense to either return null for an error or just let it throw. Both would be simpler and easier to check after calling loadNotes().
And, in either case, you must check for an error return value after calling loadNotes() unless you want loadNotes() to throw upon error like it is.
I am trying to check for the existence of an optional field in an API request, and if that field exists, perform a nested validation to check if two other fields (one or the other, or implicitly both) exist inside of it. I am using Express Validator to try and accomplish this task.
// Sample request body
{
<...>
thresholds: {
min: 3,
max: 5
}
}
// (Attempted) validation chain
check('thresholds').optional()
.custom( innerBody => {
console.log('THRESHOLDS', innerBody);
oneOf([
check('innerBody.min').optional(),
check('innerBody.max').optional()
]);
})
The above snippet is part of a larger validation chain I'm validating the full request body on. I also tried removing the innerBody. string from the inner checks but still no luck. I am console.loging the threshold body, and it prints out correctly, however I still get a validation error, when I'm trying to get my integration test to pass:
{"name":"ValidationError","message":"ValidationError: Validation failed","errors":[{"location":"body","param":"thresholds","value":{"min":3,"max":5},"msg":"Invalid value"}]}
I am relatively new to Express Validator so if I'm chaining the validation wrong/not using oneOf correctly or something would love some pointers!
Thanks
Looks like the .custom function needs to return a Promise. Answer below:
.custom(innerBody => {
if (!(innerBody.min) || !(innerBody.max)) return Promise.reject('Missing min or max');
return Promise.resolve();
})
Remember: Always return a boolean value from the callback of .custom()
function. Otherwise your validation might not work as desired.
Source: Custom validation guide
In general, you might have needs in use of Promises if you deal with asynchronous .custom() function. Then you'll be obligated to return Promise.resolve() / Promise.reject() for correct validator behaviour.
Source: SO answer
I have written a piece of software that parses and formats the fourth parameter of a graphql resolver function (the info object) to be used elsewhere. I would like to write unit tests for this software. Specifically, I do not want to build the GraphQLResolveInfo object myself, because doing that would be very cumbersome, error-prone and hard to maintain. Instead, I want to write human-readable query strings and convert them to GraphQLResolveInfo objects so I can pass those to my software.
After extensive googling and reading of the graphql-js source code, I have not found a simple way to do what they are doing internally. I'm really hoping that I am missing something.
What I am not trying to do is use the graphql-tag library, because that just generates an AST which has a very different format from the GraphQLResolveInfo type.
Has anyone done this before? Help would be much appreciated!
I will keep monitoring this question to see if a better answer comes along, but I've finally managed to solve my particular issue by creating as close an approximation of the GraphQLResolveInfo object as I need for my particular use case.
The GraphQLResolveInfo object is composed of several attributes, two of which are called fieldNodes and fragments. Both are in fact parts of the same AST that graphql-tag generates from a query string. These are the only parts of the GraphQLResolveInfo object that concern the software I wrote, the rest of it is ignored.
So here is what I did:
import gql from 'graphql-tag';
// The converter function
const convertQueryToResolveInfo = (query) => {
const operation = query.definitions
.find(({ kind }) => kind === 'OperationDefinition');
const fragments = query.definitions
.filter(({ kind }) => kind === 'FragmentDefinition')
.reduce((result, current) => ({
...result,
[current.name.value]: current,
}), {});
return {
fieldNodes: operation.selectionSet.selections,
fragments,
};
};
// An example call
const query = gql`
query {
foo {
bar
}
}
`;
const info = convertQueryToResolveInfo(query);
From the AST generated by graphql-tag, I extract and modify the operation and fragment definitions so that they look the way they do within the GraphQLResolveInfo object. This is by no means perfect and may be subject to change in the future depending on how my software evolves, but it is a relatively brief solution for my particular problem.
I am doing unit testing with xunit (ASP.NET MVC).
I have written Action in controller which returns JsonResult.
For returning JsonResult, in Action I have written:
return Json(new { ok = true, newurl = Url.Action("Login") });
For testing the action, I have written in unit test case as:
JsonResult jsonResult = _accountController.ForgotPassword(ValidUserName) as JsonResult;
Assert.Equal("{ ok = true, newurl = Url.Action('Login') }", jsonResult.Data.ToString());
But, it's not working. Please guide me to correct it.
I would suggest debugging your unit test and looking at the value returned by jsonResult.Data.ToString(). My guess is newurl doesn't contain what you think it does. Assuming you're using VS2013 you can easily debug a unit test by setting a breakpoint, right-click inside the test method body, and select Debug Tests.
An alternative approach to verifying the information is to cast the JsonResult.Data to a dynamic. This allows you to compare each property individually instead of stringifying the JSON and comparing the results. This approach is, in my opinion, cleaner when the JSON becomes larger than a few properties.
var dynData = (dynamic)jsonResult.Data;
Assert.IsTrue( (bool)dynData.ok );
Assert.AreEqual( Url.Action("Login"), (string)dynData.newurl );