I'm fairly new to automated testing and was wondering how I should go about writing tests for the database. The project I'm working on right now is running PostgreSQL with Sequelize as the ORM on a Node.JS environment. If it matters, I'm also using Jest as the testing library right now.
In my app I use a config module to control configuration settings for different environments. When running tests the process.env.APP_ENV is set to test, and it will set the dialect to sqlite. Note that you will not have any data or data persistence, so you will need to populate it with all the data needed for your tests.
Include sqlite3
yarn add -D sqlite3
or
npm i -D sqlite3
Config
module.exports = {
database: {
name: 'dbname',
user: 'user',
password: 'password',
host: 'host',
// Use "sqlite" for "test", the connection settings above are ignored
dialect: process.env.APP_ENV === 'test' ? 'sqlite' : 'mysql',
},
};
Database/Sequelize
// get our config
const config = require('../config');
// ... code
const instance = new Sequelize(
config.database.name,
config.database.user,
config.database.password,
{
host: config.database.host,
// set the dialect, will be "sqlite" for "test"
dialect: config.database.dialect,
}
);
Test Class (Mocha)
const TestUtils = require('./lib/test-utils');
describe('Some Tests', () => {
let app = null;
// run before the tests start
before((done) => {
// Mock up our services
TestUtils.mock();
// these are instantiated after the mocking
app = require('../server');
// Populate redis data
TestUtils.populateRedis(() => {
// Populate db data
TestUtils.syncAndPopulateDatabase('test-data', () => {
done();
});
});
});
// run code after tests have completed
after(() => {
TestUtils.unMock();
});
describe('/my/route', () => {
it('should do something', (done) => {
return done();
});
});
});
Run Tests
APP_ENV=test ./node_modules/.bin/mocha
You could use ENV variables in other ways to set the dialect and connection parameters as well - the above is just an example based on what we have done with a lot of supporting code.
If you're not doing anything particularly complicated on the DB side, take a look at pg-mem:
https://swizec.com/blog/pg-mem-and-jest-for-smooth-integration-testing/
https://github.com/oguimbal/pg-mem
It's really cool in that it tests actual PG syntax and can pick up a bunch of errors that using a different DB or mock DB won't pick up. However, it's not a perfect implementation and missing a bunch of features (e.g. triggers, decent "not exists" handling, lots of functions) some of which are easy to work around with the hooks provided and some aren't.
For me, having the test DB initialized with the same schema initialization scripts as the real DB is a big win.
Related
When I'm trying to create a TypeORM connection to a local postgres database in a beforeAll Jest hook, TypeORM's createConnection keeps hanging for indefinite amount of time.
I don't want to have it globally because the majority of the tests don't need this database connection.
jest.config.ts
/** #type {import('ts-jest/dist/types').InitialOptionsTsJest} */
module.exports = {
preset: 'ts-jest',
testEnvironment: 'node',
};
example.spec.ts
let conn;
describe('GET /healthz', () => {
beforeAll(async () => {
conn = await createConnection({
name: 'default',
type: 'postgres',
host: 'localhost',
port: 5433, // <- not a typo, I tested on both 5433 and 5432
database: 'test-local',
username: 'user',
password: 'pwd',
synchronize: true,
logging: true,
});
});
afterEach(async () => {
// omitted, but truncates all tables after every test
});
afterAll(async () => {
await conn.close();
});
it('should be true', () => {
expect(true).toBe(true);
});
});
Output of running jest with --detectOpenHandles:
However when I copy exactly these connection options in my normal application, it works correctly without any errors. And also in my jest it doesn't throw any errors so I'm pretty lost on what's going on here. I tried it in globalSetup before, but even there it just hangs. It just doesn't get past the createConnection. Any ideas or suggestions is much appreciated!
Altough the --detectOpenHandle pointed at the createDbConnection of TypeORM, it was actually a totally different thing that hanged.
It was very misleading, but I started cronjobs somewhere in the express app, which were hanging instead of the TypeORM createConnection.
I have a node application running in docker with mongodb and it works fine on development environment. However, I'm creating some tests with mocha and chai and I can't connect to mongo when I run these tests.
The function I want to test is:
const Interactor = require("interactor");
const Donation = require("../models/donations");
module.exports = class CreateDonation extends Interactor {
async run(context) {
this.context = context;
this.donation = new Donation.Model({
donationId: context.id,
status: context.status,
amount: context.chargeInfo.donatedValue,
donatorEmail: context.donatorInfo.email,
source: context.source,
});
await this.donation.save();
}
rollback() {
Donation.Model.findOneAndRemove({ donationId: this.context.id });
}
};
My test:
/* eslint-disable no-unused-vars */
/* eslint-disable no-undef */
const chai = require("chai");
const chaiHttp = require("chai-http");
const CreateDonation = require("../../interactors/create-donation");
require("../../config/db");
const should = chai.should();
const { expect } = chai;
chai.use(chaiHttp);
describe("CreateDonation", () => {
it("Creates a donation when context passed is correct", async (done) => {
const context = {
id: "123123",
status: "AUTHORIZED",
chargeInfo: {
donatedValue: 25.0,
},
donatorInfo: {
email: "test#example.com",
},
source: "CREDIT_CARD",
};
const result = await CreateDonation.run(context);
console.log(result);
done();
});
});
My db config file:
const mongoose = require("mongoose");
require("dotenv/config");
mongoose
.connect("mongodb://db:27017/donations", {
useNewUrlParser: true,
useUnifiedTopology: true,
reconnectInterval: 5000,
reconnectTries: 50,
})
.then(() => {
console.log("good");
})
.catch((err) => {
console.log(err);
});
mongoose.Promise = global.Promise;
module.exports = mongoose;
The error I get from the test above is:
MongooseServerSelectionError: getaddrinfo ENOTFOUND db
What am I doing wrong? Am I missing to import something?
When you run your services inside docker with a docker compose file, they'll get an hostname based on the name you wrote for the service inside the docker-compose file.
Example:
version: "3.9"
services:
web:
build: .
ports:
- "5000:5000"
redis:
image: "redis:alpine"
In this example, the web service can reach the redis db at the redis hostname.
If you change the service name in this way:
db:
image: "redis:alpine"
The web service must connect to the db host.
So, when you run the compose file, your the db service is reached with the db hostname from you app service. But when you run your tests outside a docker compose, the db hostname isn't available and you need to use localhost because your db is running on your OS directly (or it is running inside a container with the 27017 port mapped on the main host).
If you're using a unix OS, you can solve your problem adding an alias in your /etc/hosts file:
127.0.0.1 localhost db
In this way you can run your tests keeping the db connection string.
Otherwise, and this is the suggested solution, you can use an environment variable to change the connection string at the application startup:
mongoose.connect(process.env.MONGO_URI)
And run it using
MONGO_URI=mongodb://db:27017/donations npm start
Then in the docker compose you can add a fixed environment variable using this code:
environment:
- MONGO_URI=mongodb://db:27017/donations
Just found out that when testing, I need to use "localhost" on my connection string for mongo (I was using the name from docker-compose). So with the URI as "mongodb://localhost:27017/donations" it worked. I don't know why.
I use this script to connect node.js with Azure Postgresql.
But the ssl verification of our firewall blocks the connection, so in the past I need to use a proxy. Where in the code can I add the proxy settings as like host and port?
Means when I start the code, vscode should connect through the proxy to postgresql.
const pg = require('pg');
const config = {
host: '<your-db-server-name>.postgres.database.azure.com',
// Do not hard code your username and password.
// Consider using Node environment variables.
user: '<your-db-username>',
password: '<your-password>',
database: '<name-of-database>',
port: 5432,
ssl: true
};
const client = new pg.Client(config);
client.connect(err => {
if (err) throw err;
else { queryDatabase(); }
});
function queryDatabase() {
console.log(`Running query to PostgreSQL server: ${config.host}`);
const query = 'SELECT * FROM inventory;';
client.query(query)
.then(res => {
const rows = res.rows;
rows.map(row => {
console.log(`Read: ${JSON.stringify(row)}`);
});
process.exit();
})
.catch(err => {
console.log(err);
});
}
To configure proxy for Visual Studio Code
Edit the settings.json file
Depending on your platform, the user settings file is located here:
Windows: %APPDATA%\Code\User\settings.json
macOS: $HOME/Library/Application Support/Code/User/settings.json
Linux: $HOME/.config/Code/User/settings.json
Modify and Add the below lines to configure your proxy
"http.proxy": "http://user:pass#proxy.com:portnumber",
"https.proxy": "http://user:pass#proxy.com:portnumber",
"http.proxyStrictSSL": false
If your proxy doesn't require authentication, you could simply use
"http.proxy": "http://proxy.com:portnumber",
"https.proxy": "http://proxy.com:portnumber"
"http.proxyStrictSSL": false
Restart VS Code
The documentation related to settings and schema of the settings.json file is here for reference
I am using sequelize with MySQL on the Serverless offline app.
I am not sure how to sync all models on serverless start?
I am tried to do this
import Sequelize from "sequelize";
import mysql2 from 'mysql2';
const sequelize = new Sequelize('database', 'root', '', {
dialect: 'mysql',
dialectModule: mysql2,
host: 'localhost',
});
const getSequelize = () => {
sequelize.sync({ force: false })
.then(() => {
console.log(`Database & tables synchronised!`)
});
return sequelize;
}
export default getSequelize();
This approach syncs models that are imported into controller, so the only way in this case to sync all models is to include all models in every controller.
This does not look like a good example.
Do you have any idea?
Use sequelize.sync only for development purposes. Use sequelize migrations for production.
There must be some way to run console command db:migrate in your environment. When you push all the migration code use that command to update your database to the last version.
In designing a simple node.js project, I tried using Sequelize to connect to a mysql server using the following configuration parameters:
{ dbname: users
username: jimmy
password: users
params:
host: localhost
port: 3306
dialect: mysql }
though the 'users' database didn't exist yet.
I got a server error:
InternalServerError: {"name":"SequelizeConnectionError","parent":{"code":"ER_BAD_DB_ERROR","errno":1049,"sqlState":"42000","sqlMessage":"Unknown database 'users'"},"original":{"code":"ER_BAD_DB_ERROR","errno":1049,"sqlState":"42000","sqlMessage":"Unknown database 'users'"}}
But the Sequelize docs: Manual | Sequelize indicate that Sequelize can connect to a new database. Please, how can Sequelize be used to connect to a
new(non-existing) database?
You should read the documentation again carefully (because the wording is indeed confusing!).
New databases versus existing databases
If you are starting a project from scratch, and your database does not exist yet, Sequelize
can be used since the beginning in order to automate the creation of
every table in your database.
Also, if you want to use Sequelize to connect to a database that is
already filled with tables and data, that works as well! Sequelize has
got you covered in both cases.
It will connect only to an existing DB, it can help you by creating the tables if they do not exist yet but the DB has to be there in order for sequelize to connect with it.
Following this question, I wrote this PR to update the documentation to be less confusing. Hope it help!
Found a similar problem on SO, and an answer by osifo:
//create the sequelize instance omitting the database-name arg
const sequelize = new Sequelize("", "<db_user>", "<db_password>", {
dialect: "<dialect>"
});
return sequelize.query("CREATE DATABASE `<database_name>`;").then(data
=> {
// code to run after successful creation.
});
So, I was able to implement it in my own code:
var SQUser;
var sequlz;
async function connectDB() {
sequlz =
new Sequelize("", "jimmy", "users", {host: "localhost", port: 3306, dialect: "mysql"});
await sequlz.query("CREATE DATABASE users;");
await sequlz.query("Use users;");
SQUser = sequlz.define('Table',
{
// define table schema
});
return SQUser.sync();
};
//use sequelize to populate the table
async function create() {
const SQUser = await connectDB();
return SQUser.create(
//values
);
}