Mocha testing PostgreSQL with Knex is giving me a MigrationLocked error - node.js

Question

I have got a local development environment working for Node/PostgreSQL/Knex in the sense that I can post to a development database on my machine using an API. I am now trying to create tests for this functionality but am getting an error.
Here's my config:
//knexfile.js
module.exports = {
development: {
client: 'pg',
connection: {
host: '127.0.0.1',
user: 'dbUser',
password: 'dbpword',
port: 5432,
database: 'example-name'
},
migrations: {
directory: __dirname + '/db/migrations'
},
seeds: {
directory: __dirname + '/db/seeds/development'
}
},
}
//db.js
const config = require('../knexfile.js');
const env = process.env.NODE_ENV || 'development';
const knex = require("knex")(config[env]);
module.exports = knex;
knex.migrate.latest([config]);
And then my tests:
import chai from 'chai';
import { expect } from 'chai';
import chaiHttp from 'chai-http';
import knex from '../../db/db';
import app from '../../server';
chai.use(chaiHttp);
describe('Tokens API', () => {
beforeEach((done) => {
knex.migrate.rollback()
.then(() => {
knex.migrate.latest()
.then(() => {
return knex.seed.run()
.then(() => {
done();
});
});
});
});
afterEach((done) => {
knex.migrate.rollback()
.then(() => {
done();
});
});
describe('POST /users', () => {
it('posts a list of users to the database with all mandatory fields', (done) => {
chai.request(app)
.post('/users')
.send({
"users": [
"steve",
"whoever",
"matt",
"another"]})
.end((err, res) => {
expect(err).to.be.null;
expect(res).to.have.status(200);
expect(res).to.be.json;
done();
});
});
});
});
When I run this I get the following error twice - I think for the knex calls in the beforeEach block:
Knex:warning - Can't take lock to run migrations: Migration table is already locked
Knex:warning - If you are sure migrations are not running you can release the lock manually by deleting all the rows from migrations lock table: knex_migrations_lock
Unhandled rejection MigrationLocked: Migration table is already locked
I have tried numerous things - including clearing out the knex_migrations_lock table. The only support I can find online is this thread, which suggests clearing out the lock table using DELETE FROM Migrations_lock where id <> 0;, however my lock table only has a is_locked column with zero values.
Any idea what's going on?
EDIT: I've just realised if you edit out all the knex calls, the test actually passes. Could this be because I am effectively calling knex twice - once from db.js and once indirectly through server.js? If that's the case, how do I avoid doing this - because surely I need to call the knex setup for Node to run it?

Answer 1

For anyone stumbling across this, the problem was actually coming from db.js, specifically the last line:
const config = require('../knexfile.js');
const env = process.env.NODE_ENV || 'development';
const knex = require("knex")(config[env]);
module.exports = knex;
knex.migrate.latest([config]);
Of course this is asynchronous, and the tests were importing this file before trying to run their own knex functions, causing the lock. I got round this by adding a clause to block this running while testing:
if(process.env.NODE_ENV != 'test') {
knex.migrate.latest([config])
}
You can then create a test environment by adding process.env.NODE_ENV='test' to each spec file, or by installing the npm env test module.

Answer 2

Had the exact same issue, ended up being due to my API calling the database when being initialized by the supertest library.
For example, my test file:
var db = require('../db');
var api = require('../api');
var supertest = require('supertest')(api);
describe('Session routes', () => {
beforeEach((done) => {
db.migrate.rollback()
.then(() => {
db.migrate.latest()
.then(() => {
return db.seed.run()
.then(() => {
done();
});
});
});
});
afterEach((done) => {
db.migrate.rollback()
.then(() => {
done();
});
});
it('GET /session should error with no token', (done) => {
supertest
.get('/session')
.set('Accept', 'application/json')
.expect('Content-Type', /json/)
.expect(401, {
error: 'Unauthorized'
}, done);
});
});
On line 2, it requires my api - when my api is required the following code gets run straight away to initialize my api's external services API:
var db = require('./other-postgres-library');
var servicesApi = require('./services/api')(db);
This would connect to a bunch of external services and write the results to the database.
So when tests were running my APP was throwing errors because it was trying to write to a database which was being rolled back/migrated/seeded etc.
I changed my inner services API to initialize lazily and all my problems have disappeared.
In your case, I would hazard to guess when your tests runs this line
import app from '../../server'; your app/server code is trying to running some queries against the database.