I just pulled a project to a new machine, I use PostgreSQL for my DB, sequelize as an ORM, and feathersjs and express or middleware. I try to start the project as I always do but it doesn't work.
I have an .env file with all my environment configuration info, what I usually do is first run sequelize db:create, to create the DB, after that, I use npm start to create the tables. However when I do it it doesn't create any table or show any error, it just shows the info: Feathers application started on http://localhost:3030 but when I open pgAdmin the Database is empty. I tried logging in in sequelize but it's not showing anything at all. I also deleted the ENV and it doesn't show any error at npm start (it does show an error at sequelize db:create. This is driving me nuts.
Heres my sequelize.js:
require('dotenv').config();
const Sequelize = require('sequelize');
const { Op } = Sequelize;
const operatorsAliases = {
// ...
};
module.exports = function (app) {
const connectionString = `postgres://${process.env.DB_USERNAME}:${process.env.DB_PASSWORD}#` +
`${process.env.DB_HOST}:${process.env.DB_PORT}/${process.env.DB_NAME}`;
const sequelize = new Sequelize(connectionString, {
dialect: 'postgres',
dialectOptions: {
ssl: false
},
logging: console.log,
operatorsAliases,
define: {
freezeTableName: true
}
});
const oldSetup = app.setup;
app.set('sequelizeClient', sequelize);
app.setup = function (...args) {
const result = oldSetup.apply(this, args);
// Set up data relationships
const models = sequelize.models;
Object.keys(models).forEach(name => {
if ('associate' in models[name]) {
models[name].associate(models);
}
});
// Sync to the database
sequelize.sync({ logging: console.log });
return result;
};
};
My npm start script just points to an app.js file where I just import sequelize with app.configure(sequelize);
You may need to run sequelize db:migrate after creating the database
So it turns out is actually an issue with node 14. I had to install a newer version of node-Postgres (pg#8.0.3) as pointed out in here
Related
Good evening.
I'm building an Express REST API using SQLite, with Sequelize being my ORM of choice.
I was working with an existing database, so I used a package called sequelize-auto in order
to generate my models.
Problem is, Sequelize for some reason won't see my database, a .db file, or won't detect the table, despite being clearly defined. The problem occurs in this block of code:
const Sequelize = require('sequelize');
const sequelize = new Sequelize('expenserdb', 'user', 'pass', {
dialect: 'sqlite',
host: '../database/expenserdb.db'
});
const initModels = require('../models/init-models');
const models = initModels(sequelize);
module.exports = class dbService {
editUserName(newName) {
models.User.update({Name: newName}, {
where: {
UserId: 1
}
})
.catch((err) => {console.log(err)});
}
}
This is my dbService.js file, and as you can see, the database is clearly defined, with the name, location and everything being correct. Despite all of this, I'm still getting:
Error: SQLITE_ERROR: no such table: User {errno: 1, code: 'SQLITE_ERROR', sql: 'UPDATE `User` SET `Name`=$1 WHERE `UserId` = $2'
This service is being injected into my controller via awilix, and the method is called inside the controller:
const container = require("../containerConfig");
const dbService = container.resolve("dbService");
exports.runNameChange = async (req) => {
const newName = JSON.stringify(req.body.name);
const result = await dbService.editUserName(newName);
return result;
};
And in turn, the controller method is ran when this endpoint is hit:
app.post('/updateuser', async function(req, res) {
const result = await userController.runNameChange(req);
res.status(200).send(String(result));
});
Also, the dboptions.json file:
{
"storage":"./database/expenserdb.db"
}
I'm new to Node, Express, and Sequelize in general. I might have missed something crucial, but I can't seem to realize what. Help is greatly appreciated.
Update: just figured it out, you need the FULL path and not the relative path in the
verbose constructor, as such:
const sequelize = new Sequelize('expenserdb', 'user', 'pass', {
dialect: 'sqlite',
storage: 'E:/desktopshortcuts/yahalom/expenser/expenser-server/database/expenserdb.db'
});
I'm doing a Node.js project that contains sub projects. One sub project will have one Mongodb database and Mongoose will be use for wrapping and querying db. But the problem is
Mongoose doesn't allow to use multiple databases in single mongoose instance as the models are build on one connection.
To use multiple mongoose instances, Node.js doesn't allow multiple module instances as it has caching system in require(). I know disable module caching in Node.js but I think it is not the good solution as it is only need for mongoose.
I've tried to use createConnection() and openSet() in mongoose, but it was not the solution.
I've tried to deep copy the mongoose instance (http://blog.imaginea.com/deep-copy-in-javascript/) to pass new mongoose instances to the sub project, but it throwing RangeError: Maximum call stack size exceeded.
I want to know is there anyways to use multiple database with mongoose or any workaround for this problem? Because I think mongoose is quite easy and fast. Or any other modules as recommendations?
According to the fine manual, createConnection() can be used to connect to multiple databases.
However, you need to create separate models for each connection/database:
var conn = mongoose.createConnection('mongodb://localhost/testA');
var conn2 = mongoose.createConnection('mongodb://localhost/testB');
// stored in 'testA' database
var ModelA = conn.model('Model', new mongoose.Schema({
title : { type : String, default : 'model in testA database' }
}));
// stored in 'testB' database
var ModelB = conn2.model('Model', new mongoose.Schema({
title : { type : String, default : 'model in testB database' }
}));
I'm pretty sure that you can share the schema between them, but you have to check to make sure.
Pretty late but this might help someone. The current answers assumes you are using the same file for your connections and models.
In real life, there is a high chance that you are splitting your models into different files. You can use something like this in your main file:
mongoose.connect('mongodb://localhost/default');
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', () => {
console.log('connected');
});
which is just how it is described in the docs. And then in your model files, do something like the following:
import mongoose, { Schema } from 'mongoose';
const userInfoSchema = new Schema({
createdAt: {
type: Date,
required: true,
default: new Date(),
},
// ...other fields
});
const myDB = mongoose.connection.useDb('myDB');
const UserInfo = myDB.model('userInfo', userInfoSchema);
export default UserInfo;
Where myDB is your database name.
One thing you can do is, you might have subfolders for each projects. So, install mongoose in that subfolders and require() mongoose from own folders in each sub applications. Not from the project root or from global. So one sub project, one mongoose installation and one mongoose instance.
-app_root/
--foo_app/
---db_access.js
---foo_db_connect.js
---node_modules/
----mongoose/
--bar_app/
---db_access.js
---bar_db_connect.js
---node_modules/
----mongoose/
In foo_db_connect.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/foo_db');
module.exports = exports = mongoose;
In bar_db_connect.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/bar_db');
module.exports = exports = mongoose;
In db_access.js files
var mongoose = require("./foo_db_connect.js"); // bar_db_connect.js for bar app
Now, you can access multiple databases with mongoose.
As an alternative approach, Mongoose does export a constructor for a new instance on the default instance. So something like this is possible.
var Mongoose = require('mongoose').Mongoose;
var instance1 = new Mongoose();
instance1.connect('foo');
var instance2 = new Mongoose();
instance2.connect('bar');
This is very useful when working with separate data sources, and also when you want to have a separate database context for each user or request. You will need to be careful, as it is possible to create a LOT of connections when doing this. Make sure to call disconnect() when instances are not needed, and also to limit the pool size created by each instance.
Mongoose and multiple database in single node.js project
use useDb to solve this issue
example
//product databse
const myDB = mongoose.connection.useDb('product');
module.exports = myDB.model("Snack", snackSchema);
//user databse
const myDB = mongoose.connection.useDb('user');
module.exports = myDB.model("User", userSchema);
A bit optimized(for me atleast) solution. write this to a file db.js and require this to wherever required and call it with a function call and you are good to go.
const MongoClient = require('mongodb').MongoClient;
async function getConnections(url,db){
return new Promise((resolve,reject)=>{
MongoClient.connect(url, { useUnifiedTopology: true },function(err, client) {
if(err) { console.error(err)
resolve(false);
}
else{
resolve(client.db(db));
}
})
});
}
module.exports = async function(){
let dbs = [];
dbs['db1'] = await getConnections('mongodb://localhost:27017/','db1');
dbs['db2'] = await getConnections('mongodb://localhost:27017/','db2');
return dbs;
};
I have been using this method and it works great for me until now.
const mongoose = require('mongoose');
function makeNewConnection(uri) {
const db = mongoose.createConnection(uri, {
useNewUrlParser: true,
useUnifiedTopology: true
});
db.on('error', function (error) {
console.log(`MongoDB :: connection ${this.name} ${JSON.stringify(error)}`);
db.close().catch(() => console.log(`MongoDB :: failed to close connection ${this.name}`));
});
db.on('connected', function () {
mongoose.set('debug', function (col, method, query, doc) {
console.log(`MongoDB :: ${this.conn.name} ${col}.${method}(${JSON.stringify(query)},${JSON.stringify(doc)})`);
});
console.log(`MongoDB :: connected ${this.name}`);
});
db.on('disconnected', function () {
console.log(`MongoDB :: disconnected ${this.name}`);
});
return db;
}
// Use
const db1 = makeNewConnection(MONGO_URI_DB1);
const db2 = makeNewConnection(MONGO_URI_DB2);
module.exports = {
db1,
db2
}
I'm doing a Node.js project that contains sub projects. One sub project will have one Mongodb database and Mongoose will be use for wrapping and querying db. But the problem is
Mongoose doesn't allow to use multiple databases in single mongoose instance as the models are build on one connection.
To use multiple mongoose instances, Node.js doesn't allow multiple module instances as it has caching system in require(). I know disable module caching in Node.js but I think it is not the good solution as it is only need for mongoose.
I've tried to use createConnection() and openSet() in mongoose, but it was not the solution.
I've tried to deep copy the mongoose instance (http://blog.imaginea.com/deep-copy-in-javascript/) to pass new mongoose instances to the sub project, but it throwing RangeError: Maximum call stack size exceeded.
I want to know is there anyways to use multiple database with mongoose or any workaround for this problem? Because I think mongoose is quite easy and fast. Or any other modules as recommendations?
According to the fine manual, createConnection() can be used to connect to multiple databases.
However, you need to create separate models for each connection/database:
var conn = mongoose.createConnection('mongodb://localhost/testA');
var conn2 = mongoose.createConnection('mongodb://localhost/testB');
// stored in 'testA' database
var ModelA = conn.model('Model', new mongoose.Schema({
title : { type : String, default : 'model in testA database' }
}));
// stored in 'testB' database
var ModelB = conn2.model('Model', new mongoose.Schema({
title : { type : String, default : 'model in testB database' }
}));
I'm pretty sure that you can share the schema between them, but you have to check to make sure.
Pretty late but this might help someone. The current answers assumes you are using the same file for your connections and models.
In real life, there is a high chance that you are splitting your models into different files. You can use something like this in your main file:
mongoose.connect('mongodb://localhost/default');
const db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', () => {
console.log('connected');
});
which is just how it is described in the docs. And then in your model files, do something like the following:
import mongoose, { Schema } from 'mongoose';
const userInfoSchema = new Schema({
createdAt: {
type: Date,
required: true,
default: new Date(),
},
// ...other fields
});
const myDB = mongoose.connection.useDb('myDB');
const UserInfo = myDB.model('userInfo', userInfoSchema);
export default UserInfo;
Where myDB is your database name.
One thing you can do is, you might have subfolders for each projects. So, install mongoose in that subfolders and require() mongoose from own folders in each sub applications. Not from the project root or from global. So one sub project, one mongoose installation and one mongoose instance.
-app_root/
--foo_app/
---db_access.js
---foo_db_connect.js
---node_modules/
----mongoose/
--bar_app/
---db_access.js
---bar_db_connect.js
---node_modules/
----mongoose/
In foo_db_connect.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/foo_db');
module.exports = exports = mongoose;
In bar_db_connect.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/bar_db');
module.exports = exports = mongoose;
In db_access.js files
var mongoose = require("./foo_db_connect.js"); // bar_db_connect.js for bar app
Now, you can access multiple databases with mongoose.
As an alternative approach, Mongoose does export a constructor for a new instance on the default instance. So something like this is possible.
var Mongoose = require('mongoose').Mongoose;
var instance1 = new Mongoose();
instance1.connect('foo');
var instance2 = new Mongoose();
instance2.connect('bar');
This is very useful when working with separate data sources, and also when you want to have a separate database context for each user or request. You will need to be careful, as it is possible to create a LOT of connections when doing this. Make sure to call disconnect() when instances are not needed, and also to limit the pool size created by each instance.
Mongoose and multiple database in single node.js project
use useDb to solve this issue
example
//product databse
const myDB = mongoose.connection.useDb('product');
module.exports = myDB.model("Snack", snackSchema);
//user databse
const myDB = mongoose.connection.useDb('user');
module.exports = myDB.model("User", userSchema);
A bit optimized(for me atleast) solution. write this to a file db.js and require this to wherever required and call it with a function call and you are good to go.
const MongoClient = require('mongodb').MongoClient;
async function getConnections(url,db){
return new Promise((resolve,reject)=>{
MongoClient.connect(url, { useUnifiedTopology: true },function(err, client) {
if(err) { console.error(err)
resolve(false);
}
else{
resolve(client.db(db));
}
})
});
}
module.exports = async function(){
let dbs = [];
dbs['db1'] = await getConnections('mongodb://localhost:27017/','db1');
dbs['db2'] = await getConnections('mongodb://localhost:27017/','db2');
return dbs;
};
I have been using this method and it works great for me until now.
const mongoose = require('mongoose');
function makeNewConnection(uri) {
const db = mongoose.createConnection(uri, {
useNewUrlParser: true,
useUnifiedTopology: true
});
db.on('error', function (error) {
console.log(`MongoDB :: connection ${this.name} ${JSON.stringify(error)}`);
db.close().catch(() => console.log(`MongoDB :: failed to close connection ${this.name}`));
});
db.on('connected', function () {
mongoose.set('debug', function (col, method, query, doc) {
console.log(`MongoDB :: ${this.conn.name} ${col}.${method}(${JSON.stringify(query)},${JSON.stringify(doc)})`);
});
console.log(`MongoDB :: connected ${this.name}`);
});
db.on('disconnected', function () {
console.log(`MongoDB :: disconnected ${this.name}`);
});
return db;
}
// Use
const db1 = makeNewConnection(MONGO_URI_DB1);
const db2 = makeNewConnection(MONGO_URI_DB2);
module.exports = {
db1,
db2
}
I am using sequelize js and developed a node js application, which is deployed to production and have live DB.
while in development mode, if I need to alter the DB, I used to do it using Sequelize.sync({ force: true }) and it worked well.
But now, after development is done I want to alter a table and add a column to it.
I searched many posts but, didn't get an exact example on how to run these migrations.
I tried to use Umzug and run a migration, but it is throwing me errors.
Here is the code I tried,
migrations/barmigration.js:
var Sequelize = require('sequelize');
"use strict";
module.exports = {
up: function(migration, DataTypes) {
return [
migration.addColumn(
'Bars',
'PrinterId',
Sequelize.STRING
),
migration.addColumn(
'Bars',
'PrinterStatus',
Sequelize.STRING
)]
},
down: function(migration, DataTypes) {
return
[
migration.removeColumn('Bars', 'PrinterStatus'),
migration.removeColumn('Bars', 'PrinterId')
]
}
};
Here is the umzug configuration:
var Umzug = require('umzug');
var sequelize = require('sequelize');
var umzug = new Umzug({
// storage: 'sequelize',
model: 'Bar',
storageOptions: {
sequelize: sequelize,
},
migrations: {
path: './migrations',
pattern: /\.js$/
}
});
// umzug.up().then(function(migrations) {
// console.log('Migration complete!');
// });
umzug.down().then(function(migrations) {
console.log('Migration complete!');
});
When I run that file, I am getting an error in up function, at this position return migration.addColumn
Error:
Unhandled rejection TypeError: Cannot read property 'addColumn' of undefined
So, the parameter migration seems to be undefined. Pls help me out.
You need to define the params property inside migrations attribute of the object passed to the constructor of Umzug - it defines the parameters that are passed to the up and down functions. I suppose that the configuration object should be as follows
{
storage: 'sequelize',
storageOptions: {
sequelize: sequelize // here should be a sequelize instance, not the Sequelize module
},
migrations: {
params: [
sequelize.getQueryInterface(),
Sequelize // Sequelize constructor - the required module
],
path: './migrations',
pattern: /\.js$/
}
}
With above definition the migration parameter would now become the sequelize.getQueryInterface(), so simply queryInterface, and the DataTypes parameter is the Sequelize itself.
The model attribute is used to define a Sequelize migrations model, you do not need to define it if you are satisfied with the default SequelizeMeta table creation. Here is how the Umzug constructor object should look like, and here is how the storageOptions can look like.
EDIT
In separate file you need to create a Sequelize instance (further used to define models etc.) and export it. Let's assume files tree as below
- db
- database.js
- umzug.js
So in the database.js we create a Sequelize instance and export it.
// database.js
const Sequelize = require('sequelize');
const db = {
sequelize: new Sequelize(connectionString, options),
Sequelize: Sequelize
};
module.exports = db;
Then, in the Umzug configuration file
// umzug.js
const db = require('./database');
// here comes the configuration and initialization of Umzug instance with use of db object
// db.sequelize -> sequelize instance
// db.Sequelize -> sequelize constructor (class)
I can't seem to find how to log into a database in my FeathersJS app.
I would prefer to specify database info and login info in the service, but I just need it to work.
In myApp/config/default.json there is the following line:
"postgres": "postgres://postgress:#localhost:5432/feathers_db",
at:
http://docs.sequelizejs.com/en/latest/docs/getting-started/
It says that a string like the above should be:
"postgres": "postgres://username:user_password#localhost:5432/feathers_db",
But this does not work. It is also not very Feathers-like as now I am locked into one postgress db for all my postgress transactions.
In services/index.js there is the following line:
const sequelize = new Sequelize(app.get('postgres'), {
dialect: 'postgres',
logging: false
});
I could customize the above line to be what Sequelize says to do in their guide and have username and password as an argument, but then why is the template not already laid out like this?
There is also this line:
app.set('sequelize', sequelize);
If I have several postgress databases what do I do? DO I make new Sequelize objects and do something like:
app.set('sequelize_db1', sequelize_db1);
app.set('sequelize_db2', sequelize_db2);
Or do I specify db info, including user info in the service's model?
What does the logging in process for Postgress look like if one is using the generic db language rather than sequelize?
So in a word "yes". Everything I asked in my question was yes.
I can connect to the db like shown in the sequelize documentation. I can also configure the config.json file to have a "postgres" configuration with my user and db name. I could also place the full path in the services/index.js file when creating the new sequelize object. The best way to check that there is a connection is to have the following code after creating the new sequelize object:
new_sequelize_object
.authenticate()
.then(function(err) {
console.log('Connection to the DB has been established successfully.');
})
.catch(function (err) {
console.log('Unable to connect to the database:', err);
});
(taken from: http://docs.sequelizejs.com/en/latest/docs/getting-started/)
one can also define several sequelize objects and set them in the app. Then when defining the model in the specific service's index.js file, place the new bound name in the app.get('new_sequelize_object').
Here is the services/index.js file with two databases defined:
'use strict';
const service1 = require('./service1');
const authentication = require('./authentication');
const user = require('./user');
const Sequelize = require('sequelize');
module.exports = function() {
const app = this;
const sequelize = new Sequelize('feathers_db1', 'u1', 'upw', {
host: 'localhost',
port: 5432,
dialect: 'postgres',
logging: false
});
const sequelize2 = new Sequelize('postgres://u1:upw#localhost:5432/feathers_db2', {
dialect: 'postgres',
logging: false
});
app.set('sequelize', sequelize);
app.set('sequelize2', sequelize2);
sequelize
.authenticate()
.then(function(err) {
console.log('Connection to sequelize has been established successfully.');
})
.catch(function (err) {
console.log('Unable to connect to the database:', err);
});
sequelize2
.authenticate()
.then(function(err) {
console.log('Connection has been established to sequelize2 successfully.');
})
.catch(function (err) {
console.log('Unable to connect to the database:', err);
});
app.configure(authentication);
app.configure(user);
app.configure(service1);
};
And here is the service1/index.js file that uses service sequelize2:
'use strict';
const service = require('feathers-sequelize');
const service1 = require('./service1-model');
const hooks = require('./hooks');
module.exports = function(){
const app = this;
const options = {
//Here is where one sets the name of the differeng sequelize objects
Model: service1(app.get('sequelize2')),
paginate: {
default: 5,
max: 25
}
};
// Initialize our service with any options it requires
app.use('/service1', service(options));
// Get our initialize service to that we can bind hooks
const service1Service = app.service('/service1');
// Set up our before hooks
service1Service.before(hooks.before);
// Set up our after hooks
service1Service.after(hooks.after);
};