I make connection to my database like the following:
var mongoClient = new MongoClient(new Server('localhost', 27017, {auto_reconnect: true}));
mongoClient.open(function (err, mongoClient) {
var db = mongoClient.db('db_name');
db.authenticate('user', 'password', function () {err, result} {
if (err || !result) return console.log('error');
app.use(express.session({
store: new MongoStore({
db: db
})
});
});
});
And I want to share db connection with MongoStore but it's seem not work. How should I do that?
EDIT: I'm using authentication on my database but after new MongoStore() get executes I'm getting the following error:
not authorized for query on site.system.indexes
This is how it works for me,
var connectionString = "mongodb://username:password#localhost:27017/db_name";
var dbOptions = {
server:{
'auto_reconnect': true,
'poolSize': 20,
socketOptions: {keepAlive: 1}
}
}
// For long running applictions it is often prudent to enable keepAlive. Without it,
// after some period of time you may start to see "connection closed" errors for what
// seems like no reason.
MongoClient.connect(connectionString, dbOptions, function(err, db) {
if(err){
console.log(err);
}
app.use(express.session({
store:new mongoStore({db: db}),
secret: 'secret'
}));
})
This works perfectly for me and it will not give you not authorized issues as well. Previously we don't need to give keepAlive option and it works perfectly witout it but with a release of mongodb 2.4 for long running applications we need to give keepAlive option otherwise we keep getting connection closed or not authorized sort of errors.
Related
I am working on to integrate redis, #socket.io/redis-adapter, socket.io and express-session. The following is the code :
// Promisify the methods so we can use async / await with redis
bluebird.promisifyAll(redis);
const redisClient = redis.createClient({ legacyMode: true });
const subClient = redisClient.duplicate({ legacyMode: true });
bluebird.promisifyAll(redisClient);
bluebird.promisifyAll(subClient);
const redisStore = new RedisStore({ client: redisClient });
// Applying Redis middleware
app.use(
session({
secret: process.env.SESSION_SECRET,
// create new redis store.
store: redisStore,
cookie: {
secure: !isDevEnv, // if true: only transmit cookie over https
httpOnly: true, // if true: prevents client side JS from reading the cookie
maxAge: 30 * 24 * 60 * 60 * 1000, // session max age in milliseconds
},
resave: false,
saveUninitialized: true,
rolling: true,
}),
);
In the below code redisClient and subClient are used to pass into createAdapter function . I am using redisClient in another function called IdentityCheck where we use that function to maintain single session for a user and we set the visitorId ( user id) using redis set and get function inside the function IdentityCheck
// Start the server
const server = app.listen(PORT, console.log(`App listening on port: ${PORT}`));
server.setTimeout(6 * 60 * 1000);
// Start socket io with cors enabled
const io = new Server();
io.adapter(adapter.createAdapter(redisClient, subClient));
(async () => {
Promise.allSettled([redisClient.connect(), subClient.connect()]).then(() => {
io.listen(server, {
cors: {
origin: [
'http://localhost:3000'
],
},
transports: ['websocket'],
});
identityCheck.init(io, redisClient); // using redisClient in other file where we use redis get, set methods
redisClient.on('error', (error) => console.error('error from redisClient', error));
subClient.on('error', (error) => console.error('error from subClient', error));
});
})();
Even though I set the error handler as following:
redisClient.on('error', (error) => console.error('error from redisClient', error));
subClient.on('error', (error) => console.error('error from subClient', error));
I still get the missing 'error' handler on this Redis client. Am I doing anything wrong here. I am using the following versions :
{
"express": "^4.17.1",
"express-session": "^1.17.1",
"redis": "4.5.1",
"socket.io": "4.5.4",
"socket.io-client": "4.5.4",
"#socket.io/redis-adapter": "8.0.1",
}
redis version 4.5.1, socket.io, socket.io-client-4.5.4 and #socket.io/redis-adapter": "8.0.1",
I downgraded the redis version to 3.1.2 and the error has gone! I think there's something to do with redis version 4 . If anyone has correct reason please feel free to answer. thanks
Here is my complete code for sql connection, all code I have got from stackoverflow issues.
Everywhere, I found the same code is being suggested, hence I also tried with the same.
I have some other application which uses same connection with NextJs and it works fine, however, If I try only with NodeJS code, it gives some socket hang up error (code:'ESOCKET' name:'ConnectionError').
Please make a note that TCP is already configured on remote server and its working fine with other applications.
Any help is appreciated, thank you.
const express = require('express');
const fs = require('fs');
const path = require('path');
const cheerio = require("cheerio");
const sql = require('mssql');
require('dotenv').config(); //to use the env variables
// config for your database
var config = {
user: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
server: process.env.DATABASE_HOST,
database: process.env.SOMEDB,
port: 14345, // process.env.DATABASE_PORT,
options: {
encrypt: true, // for azure
trustServerCertificate: false // change to true for local dev / self-signed certs
}
};
// make sure that any items are correctly URL encoded in the connection string
let appPool = new sql.ConnectionPool(config);
//I got error on below connect
sql.connect(config).then(function(pool) {
//It never reaches here, it directly goes to the catch block
app.locals.db = pool;
const server = app.listen(3000, function () {
const host = server.address().address
const port = server.address().port
console.log('Example app listening at http://%s:%s', host, port)
})
}).catch(function(err) {
console.error('Error creating connection pool', err)
});
I have the same issue.
Try to use mssql version 6.0.1, it works on my code, but for sure we need to figure out the problem, since we can't think to mantain forever an old version of a package.
I kept trying to find the solution with different different configuration changes.
Finally, I have made a proper config, which worked and now its connecting properly as well as returning the data from the table.
require('dotenv').config(); //to access the process.env params
const sql = require("mssql"); //mssql object
var dbConfig = {
user: "ajay",
password: "abcd123",
server: "your_remote_sql_server_path",
port: 1433,
database: "your_database_name",
options: {
database: 'your_database_name',
trustServerCertificate: true
}
};
try {
//connection config will be used here to connect to the local/remote db
sql.connect(dbConfig)
.then(async function () {
// Function to retrieve the data from table
const result = await sql.query`select top 1 * from table_name`
console.dir(result)
}).catch(function (error) {
console.dir(error);
});
} catch (error) {
console.dir(error);
}
I am not sure what was the exact issue, but as per the previous config and this one, it seems like adding database name to the options has solved the issue.
Please make sure to save all the sensitive data to the .env file. (which you can access as PROCESS.env.parametername)
For me in driver mssql#9.1.1 making encrypt=false worked
const config = {
user: process.env.DATABASE_USER,
password: process.env.DATABASE_PASSWORD,
server: process.env.DATABASE_HOST,
database: process.env.SOMEDB,
port: 14345, // process.env.DATABASE_PORT,
options: {
encrypt: false
}
};
My credentials work perfectly with Robomongo but I can't make the connection with node.js
I have tried to make the connection using ssh2 and tunnel-ssh npm module and failed both times.
-The mongo connection does not require a password
-The ssh connection is made with a pem key
This is the code I've used with ssh2 module, I can establish the tunneling correctly but the mongo connection fails
var Client = require('ssh2').Client;
var conn = new Client();
conn.on('ready', function() {
console.log('Client :: ready');
//mongo connection
mongoose.connect('mongodb://localhost:27000/');
var db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function() {
console.log("database connection established");
var users = db.collection('user');
var getallUsers = function (date, callback){
users.find({}).toArray(function(err,data){
callback(data);
})
};
getallUsers(null, function (data){
console.log('data :'+ data);
});
});
//end of mongo connection
}).connect({
host: '**.**.**.**.**',
port: 22,
username: 'ec2-user',
privateKey: key
});
And the code the tunnel-ssh
var config = {
dstPort: 27000,
user: 'ec2-user',
host: '**.**.**.**.**',
privateKey: key
};
var server = tunnel(config, function (error, server) {
if(error){
console.log("SSH connection error: " + error);
}
console.log('database connection initalizing');
mongoose.connect('mongodb://localhost:27000/');
var db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function() {
console.log("database connection established");
var users = db.collection('user');
var getallUsers = function (date, callback){
users.find({}).toArray(function(err,data){
callback(data);
})
};
getallUsers(null, function (data){
console.log(data);
});
});
});
I'm not sure whether to use the regular MongoDB connection string after establishing the tunnel or referring to the database as localhost such as
mongodb://localhost:portnumber.
or
mongodb://databasepath.subpath.mongodbdns.com:27000
Localhost gives me a permission denied error, the latter gives me a timeout
As mscdex mentioned ssh2 isn't a good module to use to make an ssh tunnel connection to a database. tunnel-ssh is more appropriate.
Here are the configuration options I've used :
dstPort: remote database connection port
localPort: same as dstPort, It'll be the port you'll use for your local machine
username: SSH username,
host: SSH address
dstHost: database connection url (...mongodbns.com) ,
privateKey: SSH key
Then once your tunnel is connected connect via mongoose to your localhost such as mondodb://localhost:27000 (use the localport you defined in localPort)
var server = tunnel(config, function (error, server) {
if(error){
console.log("SSH connection error: " + error);
}
mongoose.connect('mongodb://localhost:27000/');
//...rest of mongoose connection
}
Since mongoose does not support passing in a stream to use as the underlying connection, you will have to listen on a local port (e.g. 27000) and forward incoming connections to that port over the ssh connection.
Fortunately there exists third party modules that build on ssh2 that provide this kind of functionality for you, such as tunnel-ssh. Try using one of those.
You can do it with official mongodb client for node
const sshTunnelConfig = {
agent: process.env.SSH_AUTH_SOCK,
username: 'ec2-user',
privateKey: require('fs').readFileSync('./path-to-ec2-key.pem'),
host: '3.98.174.12', //IP adress of VPS which is the SSH server
port: 22,
dstHost: 'docdb-cluster-vmabwxueb51y.eu-central-1.docdb.amazonaws.com',
dstPort: 27017,
localHost: '127.0.0.1',
localPort: 27018 //or anything else unused you want
};
const connectionProperties = {
sslValidate: true,
ssl: true,
sslCA: [fs.readFileSync('rds-combined-ca-bundle.pem')],
useNewUrlParser: true,
useUnifiedTopology: true,
authMechanism: 'SCRAM-SHA-1',
auth: {
user: 'docdbuser',
password: '<PASSWORD>'
},
tlsAllowInvalidHostnames: true,
tlsAllowInvalidCertificates: true,
};
tunnel(sshTunnelConfig, async (error, server) => {
if (error) {
console.log('SSH connection error: ', error);
}
const MongoClient = require('mongodb').MongoClient;
const client = MongoClient.connect('mongodb://localhost:27018/', propertiesConnection,
function(err, client) {
if(err)
throw err;
//Specify the database to be used
db = client.db('database-name');
//Specify the collection to be used
col = db.collection('collection-name');
//Insert a single document
col.insertOne({'hello':'Amazon DocumentDB'}, function(err, result){
//Find the document that was previously written
col.findOne({'hello':'Amazon DocumentDB'}, function(err, result){
//Print the result to the screen
console.log(result);
//Close the connection
client.close()
});
});
});
});
Because all the answers above didn't work for me for some reason, I am posting the code that worked for me. I'm tunneling from my Nodejs webserver to an PostgreSQL Database on an online ubuntu vm:
const SSH2Promise = require('ssh2-promise');
const {Client } = require('pg');
let config = {
host:process.env.SSH_HOST, //your machine IP-address like [193.xxx.xx.xxx]
port:process.env.SSH_PORT, //port you ssh to, probably 22
username: process.env.SSH_USERNAME, //username of your machine
privateKey: fs.readFileSync(path.join(__dirname, "../" + process.env.PRIVATE_KEY)) //your ssh private key to log in
};
function getDBConfig(port) {
return new Client({
user: process.env.DB_USER,
host: process.env.DB_HOST,
database: process.env.DB_NAME,
password: process.env.DB_PASS,
port: port,
});
}
async function makeDb(port) {
let dbClient = getDBConfig(port);
await dbClient.connect();
return {
async query(sql) {
return (await dbClient.query(sql)).rows;
}
};
}
const sshConn = new SSH2Promise(config);
let con;
(async function(){
await sshConn.connect();
console.log("Connection established");
let tunnel = await sshConn.addTunnel({remoteAddr: process.env.REMOTE_HOST, remotePort: process.env.REMOTE_PORT});
//Remote host: just use 127.0.0.1
//Remote port: port where your db is connected to ex: 5432
con = await makeDb(tunnel.localPort);
})();
//use connection like this:
await con.query("SELECT ... sql statement here);
I had an Error setting TTL error when starting my application in express. Maybe the problem is because I use for sessions and for db operations the same database through different connections.
So it there a specific sequence of requiring connect-mongo and mongoose that needs to be respected if I want to store my sessions in mongodb via the connect-mongo middleware and use mongo as my database for my app specific data?
Currently my app looks like this:
App.app.use(express.session({
store: new MongoStore({
db: settings.cookie.db,
host: settings.cookie.host,
port: settings.cookie.port
}),
secret: settings.cookie.secret
}))
and later I set start the connection for mongo:
function connect(connectionString) {
mongoose.connect(connectionString)
var db = mongoose.connection
db.on('error', console.error.bind(console, 'connection error'))
db.once('open', function callbck() {
console.log('Mongoose connected at: ', connectionString)
})
}
There are no error logs apart
Also how do I tear down properly mongo connections when I close my app (from command line let's say)? For this question I found the answer here I think.
First of all i've created a sessionStore module
var mongoose = require('mongoose'),
express = require('express'),
MongoStore = require('connect-mongo')(express),
sessionStore = new MongoStore({mongoose_connection: mongoose.connection});
module.exports = sessionStore;
Then i've included it into app
sessionStore = require('libs/sessionStore');
And finaly
app.use(express.session({
secret: config.get('session:secret'),
key: config.get('session:key'),
cookie: config.get('session:cookie'),
store: sessionStore
}));
That's config
"session": {
"secret": "secret",
"key": "connect.sid",
"cookie": {
"path": "/",
"httpOnly": true,
"maxAge": null
}
},
I am getting errors left and right while I try to configure the express.js session storage with mongodb. I am using locomotive for my framework and have configured mongoose.
In my initializers directory for 02_mongoose.js I have this.
module.exports = function() {
mongoose.connect('mongodb://localhost:27018/basbac');
mongoose.model('User', UserSchema);
}
I know I have a connection to the database because I can pull my users in my controller.
DeveloperController.show = function() {
var self = this;
var user = mongoose.model('User');
user.find().exec(function(error, users) {
if(error) {
console.log(error);
} else {
self.res.json({response: { id: self.param('id'), api: self.param('api'), users: users } });
}
});
}
http://localhost:3000/developer/test/?api=hhfkgjhukdsfkjhvsduhvudhcsiudvlskejfbk
{
response: {
id: "test",
api: "hhfkgjhukdsfkjhvsduhvudhcsiudvlskejfbk",
users: [
{
_id: "52706695a43c83a739358de5",
firstname: "cad",
lastname: "bane",
address: "duro",
email: "cad#bane.com"
},
{
_id: "52706695a43c83a739358de6",
firstname: "jar jar",
lastname: "binks",
address: "naboo",
email: "jarjar#binks.com"
}
]
}
}
Inside my config/all.js I have this as my configuration for sessions
var MongoStore = require('connect-mongo')(express);
var mongoose = require('mongoose');
this.use(express.cookieParser());
this.use(express.session({
secret: 'keyboard cat',
store: new MongoStore({
mongoose_connection: mongoose.connection
})
}));
But this throws an error.
this.db = new mongo.Db(options.mongoose_connection.db.databaseName,
TypeError: Cannot read property 'databaseName' of undefined
I also tried to do it like the connect-mongo docs where saying but I get an error with that as well. (https://github.com/kcbanner/connect-mongo) mongoose_connection in the form: someMongooseDb.connections[0] to use an existing mongoose connection. (optional)
this.use(express.session({
secret: 'keyboard cat',
store: new MongoStore({
mongoose_connection: mongoose.connections[0]
})
}));
But I get the same error as before.
this.db = new mongo.Db(options.mongoose_connection.db.databaseName,
TypeError: Cannot read property 'databaseName' of undefined
I also tried to do as many articles are saying to do. Here is one for example of someones working configuration (Logout in ExpressJS, PassportJS and MongoStore)
this.use(express.session({
secret: 'keyboard cat',
store: new MongoStore({
db: mongoose.connection.db
})
}));
But that also produces an error, and I know that the db key is actually undefined
throw new Error('Required MongoStore option `db` missing');
What am I doing wrong to pass this connection into the new MongoStore? When I start console.log() the mongoose object I am not able to find any information about the connection it is using. I do see a base object but it does not have a db key inside it. Do I need to pass some more options into the mongoose configuration?
The problem is the order in which Locomotive starts up. According to the docs:
When a Locomotive application is started, it proceeds through a
sequence of steps:
Configure the Environment
In this step, config/environments/all.js is executed followed by the
configuration file for the current environment. For instance, when
running in development, config/environments/development.js is
executed.
Invoke Initializers
After the environment has been configured, initializers are invoked.
Initializers are used to configure sub-systems and connect to
databases, message queues, and other services utilized by the
application.
So when your environment code is being called, the initializer hasn't yet run and Mongoose isn't configured. Try moving the Mongoose setup to a separate file (I use app/db/mongoose myself, but that's a matter of personal preference) and require that in your environment file.