Validate Mongoose Schema against external API - node.js

I have a mongoose schema like the following:
const User: Schema = new Schema({
// some other fields
email: {type: String, unique: true, require: true, validate: [myValidator, 'invalid email provided'],
// some more fields
)};
My myValidator uses validatorJS isEmail function to check if a valid email address has been entered and looks like this:
function myValidator(email: String) {
return isEmail(email: String, {require_tld: true, domain_specific_validation: true});
So far all of this works as expected. But now I have the following problem, which might be linked to my limited understanding of Typescript/JaveScript.
I would like to extend myValidator to check the entered email address against an external API.
So far I've tried something like this:
function myValidator(email: String) {
let isValid = false;
request('https://disposable.debounce.io/?email=' + email, { json: true }, async (err, res, body) => {
if (err) { return console.log(err); }
isValid = body.disposable;
});
if (isValid && !isEmail(email, {require_tld: true, domain_specific_validation: true})) {
return false;
}
But this obviously fails as request is an async operation so isValid will always be false
So I 'attempted' to make this function an async/await function.
function myValidator(email: String) {
const remoteCheck = function() {
return new Promise(function() {
request('https://disposable.debounce.io/?email=' + email, { json: true }, async (err, res, body) => {
if (err) { return console.log(err); }
return body.disposable;
});
});
};
async function f() {
const isValid = await remoteCheck();
if (isValid === true) {
return false;
} else {
isEmail(email, {require_tld: true, domain_specific_validation: true});
}
}
if(f()) {
return true;
} else {
return false;
}
But this gives me an error, because this function is not assignable to validate.
Please note that this is my first attempt on async/await functions. Could somebody point out what I'm missing here?
UPDATE:
So after I've spend a whole day now to familiarize myself with the concepts of callbacks and promises I came up with a working solution. I'll provide it as an answer below

I checked SO and I couldn't find a solution to this. So if anybody comes across this issue again, I'll provide a working solution for this.
If you want to validate a field against an external data-source, try the following. This validator will check the email-address provided against an API (you can of course use any other API as well) that checks if the address is a DEA and also use ValidatorJS isEmail to check for syntax:
const YourSchema = new Schema({
// your fields
})
SchemaName.path('fieldNameToCheck').validate({
validator: function(value) {
return new Promise(function (resolve, reject) {
request.get('https://disposable.debounce.io/?email=' + value, function (error, response, data) {
if (error) reject(error);
const content = JSON.parse(data);
resolve((content.disposable === 'false' && isEmail(value)))
}
});
}
});
The mistake I made was that I tried to resolve my Promise outside of the callback_function of my request. But the data I want to use is obviously only available inside of it on time.
EDIT:
for the purpose of understanding the API call. The API returns a JSON in the form of {disposable: boolean} whether the entered email-address belongs to a DEA (disposable Email Address) service

Related

Mocking with Sinon against a service module

I have got myself to the stage of unit testing, and to be honest, with all the different examples online I have got myself confused. I have a good understanding of Mocha & Chai, but Sinon is a different story.
So I have what I think is a pretty straight forward setup. I have a POST route that calls a controller. This controller is like so (removed some basic validation code)
const { createUser } = require('../services/user.service');
const apiResponse = require('../helpers/apiResponse');
const postUser = async (req, res) => {
const user = {
account_id: req.body.id,
status: req.body.status,
created_at: new Date(),
updated_at: new Date(),
};
const result = await createUser(user);
return apiResponse.successResponseWithData(res, 'User added.', result.affectedRows);
} catch (err) {
return apiResponse.errorResponse(res, err);
}
};
module.exports = {
postUser,
};
So all it really does is validate, and then creates a user object with the req and pass that to a service class. This services class does nothing more than pass the data to a database class.
const { addUserToDb } = require('../database/user.db');
const createUser = async (user) => {
try {
const createdUser = await addUserToDb(user);
return createdUser;
} catch (err) {
throw new Error(err);
}
};
module.exports = {
createUser,
};
I wont show the database class because what I want to focus on first is the controller, and then I can hopefully do the rest myself.
So from what I understand, I should be testing functions. If a function makes an external call, I should spy, mock, stub that call? I should only spy, mock or stub this functions dependencies, if one of the dependencies
has its own dependency (like the service module above having a database call dependency), this should be performed in another test? Sorry, just a few questions to help me understand.
Anyways, so I have created a user.controller.test.js file. I have not got far with it, but this is what I have so far
const chai = require('chai');
const sinon = require('sinon');
const { expect } = chai;
const faker = require('faker');
const controller = require('../controllers/user.controller');
const service = require('../services/user.service');
const flushPromises = () => new Promise(setImmediate);
describe('user.controller', () => {
describe('postUser', () => {
beforeEach(() => {
//I see a lot of code use a beforeEach, what should I be doing here?
});
it('should create a user when account_id and status params are provided', async () => {
const req = {
body: { account_id: faker.datatype.uuid(), status: 'true' },
};
const stubValue = {
id: faker.datatype.id(),
account_id: faker.datatype.uuid(),
status: 'true',
created_at: faker.date.past(),
updated_at: faker.date.past(),
};
});
});
});
If I am being totally honest I am pretty lost as to what I should be testing here. From my understanding, I need to mock the service module I think.
Could someone kindly provide some insight as to what I should be doing in this test?
Many thanks
Update
Thank you for your detailed response, I have managed to get a spy working which is a step forward. So I want to do a test on my service module, createUser method.
You can see that my createUser method takes a user Object as a parameter and passes this to a database module where it is inserted into the database and then the user object returned.
So when testing my service class, I need to mock this call to my database module.
const chai = require('chai');
const sinon = require('sinon');
const { expect } = chai;
const faker = require('faker');
const service = require('../services/user.service');
const database = require('../database/user.db');
describe('user.service', () => {
describe('createUser', () => {
it('should create a user when user object is provided', async () => {
const user = {
id: faker.datatype.string(),
status: 'true',
created_at: faker.date.past(),
updated_at: faker.date.past(),
};
const expectedUser = {
id: user.id,
status: user.status,
created_at: user.created_at,
updated_at: user.updated_at,
};
const mockedDatabase = sinon.mock(database);
mockedDatabase.expects('addUserToDb').once().withArgs(expectedUser);
await service.createUser(user);
mockedDatabase.verify();
mockedDatabase.restore();
});
});
});
When I test this, I seem to be getting this response, and it still seems to be inserting the record into my database.
ExpectationError: Expected addUserToDb({
id: 'yX7AX\\J&gf',
status: 'true',
created_at: 2020-06-03T03:10:23.472Z,
updated_at: 2020-05-24T14:44:14.749Z
}, '[...]') once (never called)
at Object.fail (node_modules\sinon\lib\sinon\mock-expectation.js:314:25)
Do you have any idea what I am doing wrong?
Thanks
before I try, I would like to suggest to drop the try/catch blocks everywhere, I will assume you're using expressJs in your Node application, and for such, take a look at express-promise-router as using that Router (instead the default one) will automatically catch anything it was thrown and you just need to focus on the code...
taking your example, you would write:
const { addUserToDb } = require('../database/user.db');
const createUser = async (user) => addUserToDb(user);
module.exports = {
createUser,
};
and
const { createUser } = require('../services/user.service');
const apiResponse = require('../helpers/apiResponse');
const postUser = async (req, res) => {
const { id: account_id, status } = res.body;
const result = await createUser({ account_id, status }); // set the date in the fn
return apiResponse.successResponseWithData(res, 'User added.', result.affectedRows);
};
module.exports = {
postUser,
};
if there's an error and in some place on the route an error is thrown, you will get a nice message back in the response with the error
regarding the code it self, seems a lot cleaner to read - keep in mind that code is for humans, the machine does not even care how you name your variables 😊
Now, regarding the tests ... I do tend to split things into 3 parts
unit tests: the functions itself, single one, like validation, helpers, etc
integration tests: when you call your API endpoint what should be returned
GUI tests (or end-to-end/e2e): applied when a GUI exists, will skip this for now
so in your case, the first thing to make sure of is what are you testing... and taking that, start from the small blocks (unit tests) and move up to the blocks that make sure all is glued together (e2e)
So all it really does is validate, and then creates a user object with the req and pass that to a service class. This services class does nothing more than pass the data to a database class.
Seems a great way to start, so it "validates" ... let's test our validation, let's pass null, undefined, string when all you want is int and so on, until we get a pretty good idea that whatever it passes, we will reply correctly with and without an error
Note I tend to use OpenAPI specs, which makes things easier for me as it provides 2 things
documentation of the endpoints
validation of the endpoints with a nice error message to the consumer
and yes, I always test some validation just to make sure it's working as expected, even though I trust the tool 100% 😜
So from what I understand, I should be testing functions.
well, an application is a group of functions, so all good there 💪
If a function makes an external call, I should spy, mock, stub that call?
I'll try to explain as best as I can what spies, stubs and mocks in Sinon are, please be gentle 🙏
Spies
they tell us information about functions calls, like, number of times called, arguments, return value, and more - they have two types, anonymous spies or spies that wrap methods in our code
function testMyCallback(callback) { callback(); }
describe('testMyCallback fn', function() {
it('should call the callback', function() {
const callbackSpy = sinon.spy(); // anonymous spy - no arguments
testMyCallback(callbackSpy);
expect(callbackSpy).to.have.been.calledOnce;
});
});
const user = {
setNname: function(name) {
this.name = name;
}
}
describe('setname fn', function() {
it('should be called with name', function() {
const setNameSpy = sinon.spy(user, 'setName'); // wrap method spy
user.setName('Katie');
expect(setNameSpy).to.have.been.calledOnce;
expect(setNameSpy).to.have.been.valledWith('Katie');
setNameSpy.restore(); // to remove the Spy and prevent future errors
});
});
Stubs
are power-spies, as they have all the functionality of Spies, but they replace the target function, they have methods that can return a specific value or throw a specific exception and a bit more
they are great to be used with your question regarding external calls, as they replace calls (so you can mock the call behavior and never use the original call)
the simplest of the examples is:
function isAdult(age) {
return age > 21;
}
describe('Sinon Stub Example', () => {
it('should pass', (done) => {
const isAdult = sinon.stub().returns('something');
isAdult(0).should.eql('something');
isAdult(0).should.not.eql(false);
done();
});
});
we've STUB'ed our function, and explicitly said it's a "function" that returns a string something... and for now on, we will never need to go to the function itself, as we have STUB it, we've replaced the real behavior with our own
another example of using STUBs when calling our API application in our integration tests
describe('when we stub our API call', () => {
beforeEach(() => {
this.get = sinon.stub(request, 'get'); // stub "request.get" function
});
afterEach(() => {
request.get.restore(); // remove our power-spy
});
describe('GET /api/v1/accounts', () => {
const responseObject = {
status: 200,
headers: {
'content-type': 'application/json'
}
};
const responseBody = {
status: 'success',
data: [
{
accountId: 1,
status: 'active'
},
{
accountId: 2,
status: 'disabled'
}
]
};
it('should return all accounts', (done) => {
// the 3 objects of our callback (err, res, body)
this.get.yields(null, responseObject, JSON.stringify(responseBody));
request.get(`${base}/api/v1/movies`, (err, res, body) => {
expect(res.statusCode).to.be.eql(200);
expect(res.headers['content-type']).to.contain('application/json');
body = JSON.parse(body);
expect(body).to.be.an('array').that.includes(2);
done();
});
});
});
});
you can also stub axios, but you will need a new library, either moxios, or proxyquire or more...
Mocks
are a bit similar to Stubs (our Power-Spies) but they can be used to replace whole objects and alter their behavior, they are mostly used when you need to stub more than one function from a single object - if all you need is to replace a single function, a stub is easier to use
Mocks can make things oversimplify and you could break your application without even knowing, so be aware...
a normally use is, for example
function setupNewAccount(info, callback) {
const account = {
account_id: info.id,
status: info.status,
created_at: new Date(),
updated_at: new Date()
};
try { Database.save(account, callback); }
catch (err) { callback(err); }
}
describe('setupNewAccount', function() {
it('', function() {
const account = { account_id: 1, status: 'active' };
const expectedAccount = {
account_id: account.id, status: account.status
};
const database = sinon.mock(Database);
database.expectes('save').once().withArgs(expectedAccount);
setupNewAccount(account, function() {});
database.verify();
database.restore();
});
});
something that we will keep forgetting is the .restore() part, and for that, there's a package (one more...) called sinon-test that will auto cleanup at the end of a test
I just hope it helped you with some of your questions and it's a bit clearer now 😏
BTW, for stubbing HTTP requests, I use nock as I think it's much easier to read and use than Sinon, especially for anyone that is reading code for the first time and has no experience in either Sinon or Nock...

Middleware Email duplicate controls bad code modeling

I need to add a node.js express middleware for Emails duplicates controls, and a short and easy looking code.
This is my middleware.js file :
// ----------------------------------- MIDDLEWARE FUNCTIONS -------------------------------------------
module.exports = {
/*
* CHECKING FOR DUPLICATE EMAIL
* #params
* #return NEXT()
* #error Status 403s
*/
async duplicate_email(db, req, res, next) {
let availableEmail = await db.collection("users").findOne({ 'email': req.body.email })
if (!availableEmail) {
console.log(" FORBIDDEN ")
res.status(403).send({ errorCode: "403" })
return
} else {
next() // continue the process
}
}
}
And this is my Register web service, in another file called usersCrud.js , I'm struggling since 3 days to make it work but no way :
/*
* Register anonymous user
* #params JSON OBJECT : {}
* #return 200
* #error 400
*/
app.post("/registerUser", middleware.duplicate_email(), function(req, res) {
try {
var user = new User({
_id: null,
prenom: req.body.prenom,
nom: req.body.nom,
email: req.body.email,
password: bcrypt.hashSync(req.body.password, 10),
role: "user",
permissions: middleware.create_permissions("user"),
filenames: [],
groups: [],
last_update: new Date(),
img: "",
birthday: "",
age: "",
job: "",
mentra: "",
});
db.collection("users").insertOne(user);
console.log("Added one user");
res.sendStatus(200);
} catch (e) {
console.log(e);
res.sendStatus(400);
}
});
This is the error :
Error: Route.post() requires a callback function but got a [object Promise]
I have tried tons of things, like adding the db parameter, use a return function, removing all parameters, but I can' t make it work. The code would be really simplier if that thing could work like this, but I don't know if it is possible.
Maybe it is easy to solve for you .
The error message tells you what's wrong, since you invoke middleware.duplicate_email() in your app.post and thus returns a promise. You need to pass the function without invoking it or define it as a factory function that returns the middleware and takes the db object as a parameter. I'd go with the second approach, which would look something like:
module.exports = {
duplicate_email(db) {
return async (req, res, next) => {
let availableEmail = await db.collection("users").findOne({'email': req.body.email})
if (!availableEmail) {
console.log(" FORBIDDEN ")
res.status(403).send({errorCode: "403"})
return
} else {
next() // continue the process
}
}
}
}
You can then use it like this:
app.post("/registerUser", middleware.duplicate_email(db), function(req, res) { ... }
Have you tried to do something like this?
app.post("/registerUser", await middleware.duplicate_email(), function(req, res)
It has to be included in the async function of course to be able to use the await keyword. As the error says, the callback is expected, but you provided a not resolved Promise.
Another thing that may work is:
app.post("/registerUser", async () => {
const result = await middleware.duplicate_email();
return result;
}, function(req, res) {
// stuff
}

Handling Callbacks in Classes

I'm building classes to find and quickly operate actions on mongodb documents. This is the UserCursor class. (Not talking about MongoDB's cursor)
exports { UserCursor };
class UserCursor {
private __id: object;
constructor(query: { _id?: object, otherId?: number }) {
let { _id, otherId } = query; // Shortens the vars' name
if (!_id && !otherId) return; // Checks if 1 identifier is provided
if (_id) { // If a _id is provided
Users.findOne({ _id }, (err, user) => {
this.__id = user._id;
});
} else if (otherId) { // If a otherId is provided
Users.findOne({ otherId }, (err, user) => {
console.log(1); // Debug, you'll see later
this.__id = user._id;
});
}
}
// Returns this.__id (which should have been initialized in the constructor)
get _id() {
console.log(2)
return this.__id;
}
}
When run, the console returns
2
1
I think you got the problem: the mongo callback in the constructor gets on after _id operates. How could I manage that, since the constructor gets activated each time the class is used?
It's not entirely clear to me, what exactly you want to happen and how you use this class but I assume you want to instantiate it and then be able to get _id instantaneously. If it's not the case, you may still get some useful info from my answer. Feel free to provide more details, I will update it.
So mongodb operations are asynchronous, if you do
const cursor = new UserCursor(...)
console.log(cursor._id)
(I assume you want this), first all operations in this thread will run, including the call to get _id(), then the callback code will. The problem with such asynchronous things is that now to use this _id you will have to make all of your code asynchronous as well.
So you will need to store a Promise that resolves with _id from mongodb and make a method getId that returns this promise like this:
private __id: Promise<object>
constructor(...) {
// ...
if(_id) {
this.__id = new Promise((resolve, reject) => {
Users.findOne({ _id }, (err, user) => {
if(err) return reject(err)
resolve(user._id)
});
})
} else {
// Same here
}
}
public getId() {
return this.__id;
}
Then use it:
const cursor = new UserCursor(...)
cursor.getId().then(_id => {
// Do smth with _id
})
Or
async function doStuff() {
const cursor = new UserCursor()
const _id = await cursor.getId()
}
doStuff()
If you now do it inside some function, you'll also have to make that function async
Also you could leave a getter like you have now, that will return a promise, but I find it less readable than getId():
cursor._id.then(_id => { ... })
const _id = await cursor._id

Chaining together promises with a db insert

I'm struggling trying to chain together three requests that require synchrony in node.js. Here is my attempt at using promises, but i am getting an error saying that db.run isn't a function. The first action should insert into my sqlite db. The most important thing i need is the
this.lastID variable, which lists the id of the last enetered item. Before attempting to use promises, I was having trouble with scoping. This is important because i need to take this value and use use it in my JSON object under the callback key. Lastly, Im using the requests npm package to send the request.
I am using the bluebird promises library, sqlite3 npm package, nodejs, express.
Any help with this would be awesome because I'm lost.
function db() {
return new Promise(function(resolve, reject) {
db.run(`INSERT INTO scan_requests(name, date) VALUES(?,?);`, [name,date], function(err) {
if (err) {
console.log(err)
}
let q = this.lastID
resolve(q)
})
})
}
db()
.then(function(q) {
let options = {
url: 'API',
body: {
name: req.name,
scan_callback: `http://localhost:80/${q}`
},
json: true
}
resolve(options)
}).then(function(options) {
console.log(options)
})
1st rule of "promises" ... always return your "promises". Except when you create a new one.
Try this ...
app.post('/route', function (req,res) {
new Promise(function(resolve, reject) {
db.run(`INSERT INTO scan_requests(req.name,req.date) VALUES(?,?);`, [name,date]).then(function(result) {
let options = {
url: 'http://API',
body: {
name: req.name,
date: req.date
callback: `http://localhost:80/${this.lastID}`,
},
json: true
}
// this resolves this promise ... it is now passed on
resolve(options);
}).then(function(options) {
// options is now the result from the promise
console.log(options)
request
.post(options)
.on('error', function(err) {
console.log(err)
})
.pipe(res)
});
});
});
UPDATE (question modified)
You're using resolve(options) but resolve is not in scope there (it doesn't exist). Remember the first rule of promises ...
db()
.then(function(q) {
let options = {
url: 'API',
body: {
name: req.name,
scan_callback: `http://localhost:80/${q}`
},
json: true
}
// *** change the following line ***
// --- you must return your data ---
return options;
}).then(function(options) {
console.log(options)
// -------------------------
// --- contrived example ---
// -------------------------
return { success: true };
}).then(status => {
console.log(`Success ${status.success}`);
});
The example includes a rather useless but illustrative example of how to continue passing data down the "promise chain".

NodeJs async sql to variable

I want a result like this
var rolecheck = ['289773584216358912','281531832938266625'];
Only fetched from a database, so I can compare it to another array with Id's (and yes it's supposed to be a string)
The purpoose of this is to check, before executing a command, if the user has a specific role with permission for that role. So it needs to be a function able to be called.
I've never worked with NodeJs async functions, so i have no clue how to convert this sql to an array:
The content of the .then is just some code of me trying to find out how it works, so ignore the consolelogs etc. Note: the logs do return the correct roles, but i just need them to return them to use them in my compare function.
sql.all("SELECT roleId FROM roles WHERE punish = 'true' and guildId = '"+guildids+"'").then(row => {
if (row) {
var rolecheck = [];
row.forEach(function(row){
rolecheck.push(row.roleId);
});
console.log(rolecheck);
}
});
returning does not work, so I need a workaround.
Here's where i compare it: (this works fine as long as rolecheck and role.id are defined correctly, which they aren't. It does work when i hardcode the rolecheck array.
member.forEach(function(role){
if(HasRole(rolecheck, role.id)){
console.log('user has role: '+role.name);
return true;
}
});
In case you can use a node version with async/await support, like version 7, here is a way to write promise-based code in a synchronous manner. This makes it simpler to pass around the row value.
async function myFunction () {
try {
let member = ''; // whatever member should be
let row = await sql.all("SELECT roleId FROM roles WHERE punish = 'true' and guildId = '"+guildids+"'");
// now you have the row available, outside of 'then' blocks
var rolecheck = [];
if (row) {
row.forEach(function(row){
rolecheck.push(row.roleId);
});
console.log(rolecheck);
}
member.forEach(function(role){
if (HasRole(rolecheck, role.id)) {
console.log('user has role: '+role.name);
return true;
}
});
} catch (error) {
consol.log(error.stack);
}
}
If sql.all did not return a promise you could instead do something like this which would also work with callback based functions
async function myFunction () {
try {
let row = await runSql();
// everything else same as first example above
}
async function runSql () {
try {
return new Promise(function (resolve, reject) {
sql.all("SELECT roleId FROM roles WHERE punish = 'true' and guildId = '"+guildids+"'")
.then(row => {
if (row) {
var rolecheck = [];
row.forEach(function(row){
rolecheck.push(row.roleId);
});
console.log(rolecheck);
resolve(row);
} else {
reject('Row not found');
}
});
});
} catch (error) {
console.log('erro')
}
};

Resources