Basically to run the tests, I need the database connection to MongoDb to be ready. The database manager has a connect method to get a connection and a get method to get a reference to that connection. The connect method is supposed to be called only once, at startup. This design works fine when running the app because the app starts only if the connect method has been called and returned a connection.
The database manager using mongo native (no mongoskin, no mongoose) (database.js):
var config = require('../config')
var MongoClient = require('mongodb').MongoClient
var connection = null
module.exports.connect = function () {
return MongoClient.connect(config.mongodb.uri + config.mongodb.db)
.then(function (db) {
connection = db
})
}
module.exports.get = function () {
if (!connection) {
throw new Error('Call connect first!')
}
return connection
}
Start the app:
var db = require('./services/database');
db.connect()
.then(function() {
logger.info("mongodb is running");
require('./main')
});
In the application, there are service modules and repository modules. The service modules require repository modules when they have to use the database. I have a BaseRepository module where I define the common queries findOne, findAll, etc. For all the MongoDb collections I have a specific repository that inherits from the BaseRepository. In the constructor, I set the collection by calling the database connection.
It means that when a service module which has a dependency on a repository module, the database connect method has to be called. This is the issue because I don't know how to start all the tests after the call to that connect method. I am open to suggestions if you think it's a design flow.
A test:
var usersService = require('../../services/rest/UsersService');
describe('IT Test', function () {
})
The service:
var usersRepository = require('../dao/UsersRepository');
The repository:
var db = require('../database');
var BaseRepository = require('./BaseRepository');
var util = require('util');
util.inherits(UsersRepository, BaseRepository);
function UsersRepository() {
this.collection = db.get().collection('users'); // <====== The connect method has not been called yet
}
The base repository:
function BaseRepository() {
this.collection = undefined;
}
BaseRepository.prototype.findOne = function (filters) {
return this.collection.findOne(filters)
}
I ended up writing a wrapper for all the tests
'use strict'
var logger = require('../services/logger')()
var db = require('../services/database')
describe('All tests wrapper', function () {
it('should pass', function () {
return db.connect()
.then(function () {
logger.info('MongoDB is running')
require('./agenda/AgendaJobsIT')
require('./dashboard/DashboardsServiceIT')
require('./twitter/TwitterServiceTest')
require('./authentication/TokenServiceTest')
})
})
})
The best way is mocha --delayed switch. mocha doc says
If you need to perform asynchronous operations before any of your suites are run (e.g., for dynamically generating tests), you may delay the root suite. Run mocha with the --delay flag.
For example, use mocha in this way mocha --recursive --exit --delay --ui tdd tests.js and --delayed enable you to trigger running the root suite via calling run() explicitly.
const fn = async x => {
return new Promise(resolve => {
setTimeout(resolve, 1500, 2 * x);
});
};
(async function() {
const z = await fn(3);
suite("won't run until run() executes", () => {})
run();
})();
For more information, please read https://mochajs.org/#delayed-root-suite.
Related
Hihi.
I'm having a problem where jest it's sometimes failing a couple of tests randomly, most of the time because of this error "mongodb memory server cannot perform operation: a background operation is currently running for collection".
In another post I read something about building differents mongo instance for each block of tests.
What I have so far is a globalsetup file where I start the mongo replica set like this:
// global.ts
import { MongoMemoryReplSet } from "mongodb-memory-server";
const replSet = new MongoMemoryReplSet({
replSet: { storageEngine: "wiredTiger" },
});
module.exports = async () => {
await replSet.waitUntilRunning();
const uri = await replSet.getUri();
process.env.MONGODB_URI = uri;
};
and my db.ts is like this
// db.ts
export const connect = async () => {
mongoose.set("useFindAndModify", false);
const conn = mongoose.connect(
process.env.MONGODB_URI || config.connectionString, connectionSettings
);
When trying to call it from a test file I do something like this
// test.spec.ts
import db from "./db";
beforeAll(async () => {
await db.connect();
});
afterAll(async (done) => {
await db.dropCollections();
await db.disconnect(done);
});
beforeEach(async () => {
await seed();
});
describe('Some test', () => {
it('Should not fail and get the seeders', () => {
// some random tests using the seeds values
})
})
What I read in that post is instead of using globalSetup use a setupFile that will run for every test instead of one globally and then I MIGHT be able to solve the concurrency issue I have with my tests.
So, to conclude, does anyone knows if there is a proper way to configure the mongodb in memory or if I am doing something THAT BAD that is allowing this to happend or if is there any kind of improvement I can do that will prevent "mongodb memory server cannot perform operation: a background operation is currently running for collection" this to happen?
I have to add some features, and their corresponding tests, to an API backend I developed some months ago using NodeJS. Checking the test cases I wrote back then, I've come across with this code:
index-route.spec.js
const app = require('../../app');
const request = require('supertest');
function delay() {
return new Promise((resolve, reject) => {
setTimeout(() => {
resolve();
}, 3000);
});
}
beforeAll(async () => {
await delay();
});
// START TESTS
The reason of this delay is to give time to the application to initialize the database connection (I use the Sequelize ORM). This is done in a file (db.js) required from the application main file (app.js):
db.js
async function run() {
try {
// Creates database if it doesn't exist
await DB_init();
// Test database connection
await DB_test();
// Connection was OK, sync tables and relationships
await DB_sync();
// Check if the DB is empty and, in this case, fill the basic data
await DB_init_values();
} catch(error => {
log.error("Couldn't connect to the database '"+dbName+"':", error);
process.exit(1);
}
}
// Init the DB
run();
This works, but now that I have to add more tests, I'd like to refactor this code to avoid the added artificial delay. How could I wait for the database initialization code to be finished before starting the tests? It seems that it's not possible to require a module synchronally, is it?
Cheers,
Suppose I have a the following module, as database.js
const initOptions = {}
const pgp = require('pg-promise')(initOptions)
const config = require('../../config')
const db = pgp({
host: config.database.host,
port: config.database.port,
database: config.database.database,
user: config.database.user,
password: config.database.password
})
module.exports = db
And the following module as create.js
const db = require('./database')
function create (name) {
return new Promise((resolve, reject) => {
db.func('create', name)
.then(data => {
return resolve(data)
})
.catch(err => {
return reject(err)
})
})
}
module.exports = create
I'm trying to run a unit test on create.js that will test that db.func is called with 'create' as first argument and 'name' as the second, but without actually needing to set up a database connection (So tests can run offline).
From what I can gather, this is what libraries like sinon.JS can be used for, so I tried creating a test and stubbed the db object.
const chai = require('chai')
const chaiAsPromised = require('chai-as-promised')
chai.use(chaiAsPromised)
const sinon = require('sinon')
const expect = chai.expect
const create = require('./create.js')
describe('Test Module', () => {
it('should test stuff', () => {
const db = require('./database')
const dbStub = sinon.stub(db, 'func').callsFake(Promise.resolve(null))
expect(create('test').then).to.be.a('Function')
})
})
However, it fails with
TypeError: Cannot redefine property: func
Most likely due to my limited exposure to sinon...
How do I go about stubbing (or maybe I need to mock?) the db function so that I can test it an ensure db.func was called?
You can make the properties configurable by disabling locks with the no noLocking option in Initialization Options. This allows sinon to replace the properties:
const initOptions = { noLocking : true };
On a related note:
Your create function is creating an unnecessary promise wrapper, which is a promise anti-pattern. You should just return the result from db.func, which is a promise already:
function create(name) {
return db.func('create', name);
}
Also callsFake takes a function and you are giving it a promise. Use returns instead:
const dbStub = sinon.stub(db, 'func').returns(Promise.resolve(null))
I'm having trouble setting the noLocking option. The docs state it can be set after initialization, however if I set it with db.$config.options.noLocking = true, the same error occurs. However, if I set it in the database.js init options it works fine.
From the author of pg-promise...
It is because at that point the noLocking can only affect tasks and transactions. And since the db level of the protocol is initiated only once, setting noLocking after the library's initialization doesn't effect it.
I have just updated the documentation to clarify it:
This option is dynamic (can be set before or after initialization). However, changing it after the library's initialization will not affect Database objects that have already been created.
I have written a web service that indeed consumes an external web service, after performing some validations and data manipulations (plain business logic). I need to write unit tests for the same and need to have code coverage report as well. Many suggestions are available on the net, Mocha being the prominent one. But in all available examples, actual DB call or external service call happens. My requirement is not to have actual external service call, but just mock the call (Just how, we do using EasyMock in Java). Any help would be highly appreciated. Is it possible to create a mock of DB call or external service call in node js ?
You can use Sinon with Mocha to provide stubbing, or spying of calls. It is important to also write you code to aid mocking. For instance, don't assume the database handler exists in the web component, pass it in through the constructor. That way its easy for testing to replace the database with your own "mocked" version.
As an example - here is the start of one of my bunch of tests where I instanciate a mocked connection pool to my database handler and pass it into the test. I am actually also mocking the web server, and just testing the API.
(function(){
'use strict';
const DBHOST = 'dummy';
const DBNAME = 'dummy';
const expect = require('chai').expect;
const sinon = require('sinon');
const mock = require('../mock/server');
const DB = require('../database');
const Logging = require('../log');
Logging.debug(true); //set to log debug message
const logger = Logging.logger;
const poolPromise = mock.getPool();
const db = new DB(mock.Pool,DBNAME,logger);
const API = require('../api');
const api = new API(mock.server,db,logger);
describe('API tests',function() {
let sandbox, testConnection;
beforeEach(function(){
sandbox = sinon.sandbox.create();
testConnection = new mock.Connection();
return poolPromise.then(pool => {
pool.getConnection().then(cb => {
cb(null,testConnection);
});
});
});
afterEach(function(){
sandbox.restore();
});
describe('/logon',function() {
let params;
describe('Admin', function() {
beforeEach(function() {
params = {name: 'Admin', password: 'abcde'};
})
it('Performs logon if username is "Admin" and password is good', function(done) {
mock.server.emit('/logon', params, (status,user) => {
expect(status).to.be.true;
expect(user).to.be.an('object');
expect(user.uid).to.be.equal(0);
expect(user.name).to.be.equal('Super User');
expect(user.keys).to.be.equal('A');
done();
});
testConnection.next.then(value => {
let request = value.request;
expect(value.name).to.equal('execSql');
expect(request.sqlTextOrProcedure).to.equal('SELECT passwordsalt FROM Admin WHERE AdminID = 0');
request.emit('row', mock.makeRow('passwordsalt',['salt']));
request.callback(null,1);
return testConnection.next;
}).then(value => {
let request = value.request
expect(value.name).to.equal('callProcedure');
expect(request.sqlTextOrProcedure).to.equal('CheckPassword');
request.emit('row',mock.makeRow('a',[1]));
request.callback(null,0);
});
});
I am writing integration tests for application written in NodeJS with MongoDB.
On CI server I would like to have some sort of embedded MongoDB for faster performance and easier control.
Currently I have MongoDB on other server, but tests are slow. Before each test I need to drop all collections. I am using mongoose as ORM.
So far I have only found embedded MongoDB for Java.
As of this writing, I'd recommend using mongodb-memory-server. The package downloads a mongod binary to your home directory and instantiates a new memory-backed MondoDB instance as needed. This should work well for your CI setup because you can spin up a new server for each set of tests, which means you can run them in parallel.
See the documentation for details on how to use it with mongoose.
For readers using jest and the native mongodb driver, you may find this class useful:
const { MongoClient } = require('mongodb');
const { MongoMemoryServer } = require('mongodb-memory-server');
// Extend the default timeout so MongoDB binaries can download
jest.setTimeout(60000);
// List your collection names here
const COLLECTIONS = [];
class DBManager {
constructor() {
this.db = null;
this.server = new MongoMemoryServer();
this.connection = null;
}
async start() {
const url = await this.server.getUri();
this.connection = await MongoClient.connect(url, { useNewUrlParser: true });
this.db = this.connection.db(await this.server.getDbName());
}
stop() {
this.connection.close();
return this.server.stop();
}
cleanup() {
return Promise.all(COLLECTIONS.map(c => this.db.collection(c).remove({})));
}
}
module.exports = DBManager;
Then in each test file you can do the following:
const dbman = new DBManager();
afterAll(() => dbman.stop());
beforeAll(() => dbman.start());
afterEach(() => dbman.cleanup());
Following the "don’t use test doubles for types you don’t own" principle, consider continue using a real MongoDB instance for your integration test. Look at this nice article for details.
Our team has been stubbing out the mongo skin calls. Depending on your testing packages you can do the same thing. It takes a little bit of work but it is worth it. Create a stub function and then just declare what you need in your test.
// Object based stubbing
function createObjStub(obj) {
return {
getDb: function() {
return {
collection: function() {
var coll = {};
for (var name in obj) {
var func = obj[name];
if (typeof func === 'object') {
coll = func;
} else {
coll[name] = func;
}
}
return coll;
}
};
}
}
};
// Stubbed mongodb call
var moduleSvc = new ModulesService(createObjStub({
findById: function(query, options, cb) {
return cb({
'name': 'test'
}, null);
//return cb(null, null);
}
}),{getProperties: function(){return{get: function(){} }; } });