I have got myself to the stage of unit testing, and to be honest, with all the different examples online I have got myself confused. I have a good understanding of Mocha & Chai, but Sinon is a different story.
So I have what I think is a pretty straight forward setup. I have a POST route that calls a controller. This controller is like so (removed some basic validation code)
const { createUser } = require('../services/user.service');
const apiResponse = require('../helpers/apiResponse');
const postUser = async (req, res) => {
const user = {
account_id: req.body.id,
status: req.body.status,
created_at: new Date(),
updated_at: new Date(),
};
const result = await createUser(user);
return apiResponse.successResponseWithData(res, 'User added.', result.affectedRows);
} catch (err) {
return apiResponse.errorResponse(res, err);
}
};
module.exports = {
postUser,
};
So all it really does is validate, and then creates a user object with the req and pass that to a service class. This services class does nothing more than pass the data to a database class.
const { addUserToDb } = require('../database/user.db');
const createUser = async (user) => {
try {
const createdUser = await addUserToDb(user);
return createdUser;
} catch (err) {
throw new Error(err);
}
};
module.exports = {
createUser,
};
I wont show the database class because what I want to focus on first is the controller, and then I can hopefully do the rest myself.
So from what I understand, I should be testing functions. If a function makes an external call, I should spy, mock, stub that call? I should only spy, mock or stub this functions dependencies, if one of the dependencies
has its own dependency (like the service module above having a database call dependency), this should be performed in another test? Sorry, just a few questions to help me understand.
Anyways, so I have created a user.controller.test.js file. I have not got far with it, but this is what I have so far
const chai = require('chai');
const sinon = require('sinon');
const { expect } = chai;
const faker = require('faker');
const controller = require('../controllers/user.controller');
const service = require('../services/user.service');
const flushPromises = () => new Promise(setImmediate);
describe('user.controller', () => {
describe('postUser', () => {
beforeEach(() => {
//I see a lot of code use a beforeEach, what should I be doing here?
});
it('should create a user when account_id and status params are provided', async () => {
const req = {
body: { account_id: faker.datatype.uuid(), status: 'true' },
};
const stubValue = {
id: faker.datatype.id(),
account_id: faker.datatype.uuid(),
status: 'true',
created_at: faker.date.past(),
updated_at: faker.date.past(),
};
});
});
});
If I am being totally honest I am pretty lost as to what I should be testing here. From my understanding, I need to mock the service module I think.
Could someone kindly provide some insight as to what I should be doing in this test?
Many thanks
Update
Thank you for your detailed response, I have managed to get a spy working which is a step forward. So I want to do a test on my service module, createUser method.
You can see that my createUser method takes a user Object as a parameter and passes this to a database module where it is inserted into the database and then the user object returned.
So when testing my service class, I need to mock this call to my database module.
const chai = require('chai');
const sinon = require('sinon');
const { expect } = chai;
const faker = require('faker');
const service = require('../services/user.service');
const database = require('../database/user.db');
describe('user.service', () => {
describe('createUser', () => {
it('should create a user when user object is provided', async () => {
const user = {
id: faker.datatype.string(),
status: 'true',
created_at: faker.date.past(),
updated_at: faker.date.past(),
};
const expectedUser = {
id: user.id,
status: user.status,
created_at: user.created_at,
updated_at: user.updated_at,
};
const mockedDatabase = sinon.mock(database);
mockedDatabase.expects('addUserToDb').once().withArgs(expectedUser);
await service.createUser(user);
mockedDatabase.verify();
mockedDatabase.restore();
});
});
});
When I test this, I seem to be getting this response, and it still seems to be inserting the record into my database.
ExpectationError: Expected addUserToDb({
id: 'yX7AX\\J&gf',
status: 'true',
created_at: 2020-06-03T03:10:23.472Z,
updated_at: 2020-05-24T14:44:14.749Z
}, '[...]') once (never called)
at Object.fail (node_modules\sinon\lib\sinon\mock-expectation.js:314:25)
Do you have any idea what I am doing wrong?
Thanks
before I try, I would like to suggest to drop the try/catch blocks everywhere, I will assume you're using expressJs in your Node application, and for such, take a look at express-promise-router as using that Router (instead the default one) will automatically catch anything it was thrown and you just need to focus on the code...
taking your example, you would write:
const { addUserToDb } = require('../database/user.db');
const createUser = async (user) => addUserToDb(user);
module.exports = {
createUser,
};
and
const { createUser } = require('../services/user.service');
const apiResponse = require('../helpers/apiResponse');
const postUser = async (req, res) => {
const { id: account_id, status } = res.body;
const result = await createUser({ account_id, status }); // set the date in the fn
return apiResponse.successResponseWithData(res, 'User added.', result.affectedRows);
};
module.exports = {
postUser,
};
if there's an error and in some place on the route an error is thrown, you will get a nice message back in the response with the error
regarding the code it self, seems a lot cleaner to read - keep in mind that code is for humans, the machine does not even care how you name your variables 😊
Now, regarding the tests ... I do tend to split things into 3 parts
unit tests: the functions itself, single one, like validation, helpers, etc
integration tests: when you call your API endpoint what should be returned
GUI tests (or end-to-end/e2e): applied when a GUI exists, will skip this for now
so in your case, the first thing to make sure of is what are you testing... and taking that, start from the small blocks (unit tests) and move up to the blocks that make sure all is glued together (e2e)
So all it really does is validate, and then creates a user object with the req and pass that to a service class. This services class does nothing more than pass the data to a database class.
Seems a great way to start, so it "validates" ... let's test our validation, let's pass null, undefined, string when all you want is int and so on, until we get a pretty good idea that whatever it passes, we will reply correctly with and without an error
Note I tend to use OpenAPI specs, which makes things easier for me as it provides 2 things
documentation of the endpoints
validation of the endpoints with a nice error message to the consumer
and yes, I always test some validation just to make sure it's working as expected, even though I trust the tool 100% 😜
So from what I understand, I should be testing functions.
well, an application is a group of functions, so all good there 💪
If a function makes an external call, I should spy, mock, stub that call?
I'll try to explain as best as I can what spies, stubs and mocks in Sinon are, please be gentle 🙏
Spies
they tell us information about functions calls, like, number of times called, arguments, return value, and more - they have two types, anonymous spies or spies that wrap methods in our code
function testMyCallback(callback) { callback(); }
describe('testMyCallback fn', function() {
it('should call the callback', function() {
const callbackSpy = sinon.spy(); // anonymous spy - no arguments
testMyCallback(callbackSpy);
expect(callbackSpy).to.have.been.calledOnce;
});
});
const user = {
setNname: function(name) {
this.name = name;
}
}
describe('setname fn', function() {
it('should be called with name', function() {
const setNameSpy = sinon.spy(user, 'setName'); // wrap method spy
user.setName('Katie');
expect(setNameSpy).to.have.been.calledOnce;
expect(setNameSpy).to.have.been.valledWith('Katie');
setNameSpy.restore(); // to remove the Spy and prevent future errors
});
});
Stubs
are power-spies, as they have all the functionality of Spies, but they replace the target function, they have methods that can return a specific value or throw a specific exception and a bit more
they are great to be used with your question regarding external calls, as they replace calls (so you can mock the call behavior and never use the original call)
the simplest of the examples is:
function isAdult(age) {
return age > 21;
}
describe('Sinon Stub Example', () => {
it('should pass', (done) => {
const isAdult = sinon.stub().returns('something');
isAdult(0).should.eql('something');
isAdult(0).should.not.eql(false);
done();
});
});
we've STUB'ed our function, and explicitly said it's a "function" that returns a string something... and for now on, we will never need to go to the function itself, as we have STUB it, we've replaced the real behavior with our own
another example of using STUBs when calling our API application in our integration tests
describe('when we stub our API call', () => {
beforeEach(() => {
this.get = sinon.stub(request, 'get'); // stub "request.get" function
});
afterEach(() => {
request.get.restore(); // remove our power-spy
});
describe('GET /api/v1/accounts', () => {
const responseObject = {
status: 200,
headers: {
'content-type': 'application/json'
}
};
const responseBody = {
status: 'success',
data: [
{
accountId: 1,
status: 'active'
},
{
accountId: 2,
status: 'disabled'
}
]
};
it('should return all accounts', (done) => {
// the 3 objects of our callback (err, res, body)
this.get.yields(null, responseObject, JSON.stringify(responseBody));
request.get(`${base}/api/v1/movies`, (err, res, body) => {
expect(res.statusCode).to.be.eql(200);
expect(res.headers['content-type']).to.contain('application/json');
body = JSON.parse(body);
expect(body).to.be.an('array').that.includes(2);
done();
});
});
});
});
you can also stub axios, but you will need a new library, either moxios, or proxyquire or more...
Mocks
are a bit similar to Stubs (our Power-Spies) but they can be used to replace whole objects and alter their behavior, they are mostly used when you need to stub more than one function from a single object - if all you need is to replace a single function, a stub is easier to use
Mocks can make things oversimplify and you could break your application without even knowing, so be aware...
a normally use is, for example
function setupNewAccount(info, callback) {
const account = {
account_id: info.id,
status: info.status,
created_at: new Date(),
updated_at: new Date()
};
try { Database.save(account, callback); }
catch (err) { callback(err); }
}
describe('setupNewAccount', function() {
it('', function() {
const account = { account_id: 1, status: 'active' };
const expectedAccount = {
account_id: account.id, status: account.status
};
const database = sinon.mock(Database);
database.expectes('save').once().withArgs(expectedAccount);
setupNewAccount(account, function() {});
database.verify();
database.restore();
});
});
something that we will keep forgetting is the .restore() part, and for that, there's a package (one more...) called sinon-test that will auto cleanup at the end of a test
I just hope it helped you with some of your questions and it's a bit clearer now 😏
BTW, for stubbing HTTP requests, I use nock as I think it's much easier to read and use than Sinon, especially for anyone that is reading code for the first time and has no experience in either Sinon or Nock...
I have a GET endpoint, which basically makes some API calls to the Spoonacular API. Essentially, I make two API calls within the endpoint.
The first API call gets the list of recipe ID's for the specific ingredients
The second API calls gets the metadata for each of the recipe ID's.
After the first API call I store all the Id's in an array (recipeArray), and I want to make the second api call for each ID in my array (function recipeTest does this).
When I try to do this and then return my response to the front end, it always returns a response before completing all the API calls in the second step.
Here, is my code. The first API calls works just fine, but the second API call (recipeTest function), is where it messed up. Before that function finishes making all the API calls to the Spoonacular API, my endpoint returns an empty Array (res.send(toSend)). So, I was just wondering if there is any way around this?
Thank you so much in advance, I really appreciate it!
module.exports = (app) => {
app.get('/api/search', async (req, res) => {
console.log("endpoint working");
let ingredientList = "apples,+eggs,+bacon"; // needs to be given from the front end
let ingredientSearchUrl = `https://api.spoonacular.com/recipes/findByIngredients?ingredients=${ingredientList}&number=1&ignorePantry=true&apiKey=${keys.spoonacularKey}`;
try {
const ingredientSearchResult = await axios({
method: 'get',
url: ingredientSearchUrl
});
var recipeArray = ingredientSearchResult.data.map(info => {
return info.id;
});
} catch (err) {
console.log("error in finding recipe ID ", err);
}
let toSend = [];
try {
const check = await recipeTest(recipeArray, toSend);
} catch (err) {
console.log("error in finding recipe information ", err);
}
res.send(toSend);
});
}
const recipeTest = async (recipeArray, toSend) => {
return Promise.all(
_.forEach(recipeArray, async (recipeId) => {
let recipeInfoUrl = `https://api.spoonacular.com/recipes/${recipeId}/information?includeNutrition=false&apiKey=${keys.spoonacularKey}`;
let recipeInfo = {};
const recipeData = await axios({
method: 'get',
url: recipeInfoUrl
});
// console.log("recipeInfo search working", recipeData.data);
recipeInfo['id'] = recipeData.data.id;
recipeInfo['title'] = recipeData.data.title;
recipeInfo['time'] = recipeData.data.readyInMinutes;
recipeInfo['recipeUrl'] = recipeData.data.sourceUrl;
recipeInfo['imageUrl'] = recipeData.data.image;
// console.log('recipe info dict', recipeInfo);
toSend.push(recipeInfo);
console.log('toSend inside', toSend);
})
);
}
_.forEach return collection itself and not all your async handlers.
Use recipeArray.map to get an array of async functions to let Promise.all do its work:
Promise.all(
recipeArray.map(x => async (recipeId) => {
I'm building a Slackbot that makes a call to an Express app, which then needs to 1) fetch some other data from the Slack API, and 2) insert resulting data in my database. I think I have the flow right finally using async await, but the operation is timing out because the original call from the Slackbot needs to receive a response within some fixed time I can't control. It would be fine for my purposes to ping the bot with a response immediately, and then execute the rest of the logic asynchronously. But I'm wondering the best way to set this up.
My Express route looks like:
const express = require('express');
const router = express.Router();
const knex = require('../../db/knex.js');
const slack = require('../../services/slack_helpers');
// POST api/slack/foo
router.post('/foo', async (req, res) => {
let [body, images] = await slack.grab_context(req);
knex('texts')
.insert({ body: body,
image_ids: images })
.then(text => { res.send('worked!'); }) // This sends a response back to the original Slackbot call
.catch(err => { res.send(err); })
});
module.exports = router;
And then the slack_helpers module looks like:
const { WebClient } = require('#slack/web-api');
const Slack = new WebClient(process.env.SLACKBOT_TOKEN);
async function grab_context(req) {
try {
const context = await Slack.conversations.history({ // This is the part that takes too long
channel: req.body.channel_id,
latest: req.headers['X-Slack-Request-Timestamp'],
inclusive: true,
limit: 5
});
} catch (error) {
return [error.toString(), 'error'];
}
return await parse_context(context);
};
function parse_context(context) {
var body = [];
context.messages.forEach(message => {
body.push(message.text);
});
body = body.join(' \n');
return [body, ''];
}
module.exports = {
grab_context
};
I'm still getting my head around asynchronous programming, so I may be missing something obvious. I think basically something like res.send perhaps needs to come before the grab_context call? But again, not sure the best flow here.
Update
I've also tried this pattern in the API route, but still getting a timeout:
slack.grab_context(req).then((body, images) => {
knex ...
})
Your timeout may not be coming from where you think. From what I see, it is coming from grab_context. Consider the following simplified version of grab_context
async function grab_context_simple() {
try {
const context = { hello: 'world' }
} catch (error) {
return [error.toString(), 'error']
}
return context
}
grab_context_simple() /* => Promise {
<rejected> ReferenceError: context is not defined
...
} */
You are trying to return context outside of the try block where it was defined, so grab_context will reject with a ReferenceError. It's very likely that this error is being swallowed at the moment, so it would seem like it is timing out.
The fix is to move a single line in grab_context
async function grab_context(req) {
try {
const context = await Slack.conversations.history({
channel: req.body.channel_id,
latest: req.headers['X-Slack-Request-Timestamp'],
inclusive: true,
limit: 5
});
return await parse_context(context); // <- moved this
} catch (error) {
return [error.toString(), 'error'];
}
};
I'm wondering the best way to set this up.
You could add a higher level try/catch block to handle errors that arise from the /foo route. You could also improve readability by staying consistent between async/await and promise chains. Below is how you could use async/await with knex, as well as the aforementioned try/catch block
const express = require('express');
const router = express.Router();
const knex = require('../../db/knex.js');
const slack = require('../../services/slack_helpers');
const insertInto = table => payload => knex(table).insert(payload)
const onFooRequest = async (req, res) => {
try {
let [body, images] = await slack.grab_context(req);
const text = await insertInto('texts')({
body: body,
image_ids: images,
});
res.send('worked!');
} catch (err) {
res.send(err);
}
}
router.post('/foo', onFooRequest);
module.exports = router;
I am having trouble with my lambda function. This is the current setup:
Lambda makes post request to API. API fetches data from postgres database and returns this data. When I use Postman or a local version of my lambda function, this works. When I use the actual lambda function, the API returns data with null.
Below are some code snippets:
lambda:
const axios = require('axios')
exports.handler = async (event) => {
let rows = await axios.post('http:/server/getData', {
params: {
var: event.var
}
})
if(rows.data.statusCode == 204){
//no data available
}
else{
data = rows.data.info
}
};
API Router Component
var router = express.Router()
const Pool = require('pg').Pool
const mgmtdb = new Pool({ ... })
router.post('/getData', function(req, res){
database.query("SELECT info FROM table WHERE var=$1", [req.body.var], (error, results) => {
const rows = results.rows
if (error) {
throw error
}
let status = rows.length == 0 ? 204 : 200
var responseJSON ={};
responseJSON.statusCode = status;
responseJSON.info= rows[0] ? rows[0].info : null;
res.json(responseJSON);
})
})
module.exports = router
When I call the API from Postman I get statusCode: 200 (data available).
If I call the API with the exact same data from lambda I get statusCode: 204 (no data available).
I believe that this is some async timing problem. I don't know how the responses from the API can differ ..
Is it possible that the API streams the response back to the originator for some time not just an impulse? And starts by streaming "no data available" and then, after a few milliseconds "oh, i found some data, here it is"? And that Postman waits for the stream to finish and the lambda function doesn't?
Thanks in advance!
Looks like your query is returning null because the req.body.var doesn't exist. Can you try changing your lambda function to this:
const axios = require('axios')
exports.handler = async (event) => {
let rows = await axios.post('http:/server/getData', {
var: event.var
})
if(rows.data.statusCode == 204){
//no data available
}
else{
data = rows.data.info
}
};
This basically removes the extra level params and makes the req.body.var work.
Sometimes lambda have issue with async await so i suggest you to write code like this
exports.handler = async (event, callback) => {
axios.post('http:/server/getData', {
var: event.var
})
.then((response) => {
callback(null, {
statusCode: 200,
body: JSON.stringify(response)
})
})
};
I've been using firebase functions test to do some testing on my functions. I have some code that is supposed to post a thing to firestore, basically in the same way that the examples show to do in the realtime database examples:
exports.addMessage = functions.https.onRequest((req, res) => {
const original = req.query.text;
admin.firestore()
.collection('messages')
.add({ original })
.then(documentReference => res.send(documentReference))
.catch(error => res.send(error));
});
For my test, I've spoofed some basic functionality using sinon, mocha and chai. Here is my current test, which is failing with the error message: TypeError: firestoreService.snapshot_ is not a function
describe('addMessage', () => {
// add message should add a message to the database
let oldDatabase;
before(() => {
// Save the old database method so it can be restored after the test.
oldDatabase = admin.firestore;
});
after(() => {
// Restoring admin.database() to the original method.
admin.firestore = oldDatabase;
});
it('should return the correct data', (done) => {
// create stubs
const refStub = sinon.stub();
// create a fake request object
const req = {
query : {
text: 'fly you fools!'
}
};
const snap = test.firestore.makeDocumentSnapshot({ original: req.query.text }, 'messages/1234');
// create a fake document reference
const fakeDocRef = snap._ref;
// create a fake response object
const res = {
send: returnedDocRef => {
// test the result
assert.equal(returnedDocRef, fakeDocRef);
done();
}
};
// spoof firestore
const adminStub = sinon.stub(admin, 'firestore').get(() => () => {
return {
collection: () => {
return {
add: (data) => {
const secondSnap = test.firestore.makeDocumentSnapshot(data, 'messages/1234');
const anotherFakeDocRef = secondSnap._ref;
return Promise.resolve(anotherFakeDocRef);
}
}
}
}
});
// call the function to execute the test above
myFunctions.addMessage(req, res);
});
});
My question is how the heck do I fix this?
I previously had a test that was just passing the first snap and fakeDocRef, and my test was passing fine, but as soon as I resolve the promise with the new fake document reference, it fails...
Any help would be appreciated! Thanks!
There are three different types of the calls, that are different:
Operating on the Collections.
Operating on the Documents.
Operating on the results of the query.
They have to be used consistently.
Please refer a documentation to see the difference operation on the collection and the document.