In my application, I have different projects as db in MongoDB. So whenever an API call is happening I connect the db by its projectId and once the query fetching has happened I will disconnect the current db. It takes 5 to 10 seconds to reconnect the db with respective project ids. Because of that API calls are not happening as expected.
Here, I'm trying to make dynamic db connections, which will always connected to the respective project db and it disconnects once the db state is inactive.
I didn't get any reference to connect dynamic DB's, Can anyone pls suggest me some solution to this?
Connection Method
const connectionString = env.MONGO_DB_URL + projectId + env.MONGO_DB_OPTIONS
mongoose.connect(connectionString,{
useNewUrlParser: true,
useUnifiedTopology: true,
});
This projectId in the connection string will be a dynamic one.
Related
I created a MEAN project in heroku.
In MongoDB I have a DB / Collection like this:
db == content / collection == android_main
I have verified that the collection is in the proper database (and not in admin) via the following MongoSH exchange:
Atlas atlas-xxxxxx-shard-0 [primary] content> use admin
switched to db admin
Atlas atlas-xxxxxx-shard-0 [primary] admin> show collections
Atlas atlas-xxxxxx-shard-0 [primary] admin> use content
switched to db content
Atlas atlas-xxxxxx-shard-0 [primary] content> show collections
android_main
In MongoDB I also have a user w/ specific privs to read this db/collection.
In the code (Node.js) I use the following connection string:
mongodb+srv://<USER>:<PASS>#<MONGO URL>/content?retryWrites=true&w=majority
The connection completes successfully. I pass in the user I mentioned above, e.g. the user with just the specific read privs on the database (content) and collection (android_main).
In the code, the mechanism I use to get the database variable via the connection and connection string specifically is:
mongodb.MongoClient.connect(process.env.MONGODB_URI, function (err, database) {
Now, in the code, in response to an `HTTP GET` I issue the following:
db.collection("android_main").find({}).toArray(function(err, docs) {
...
}
I get back this error:
ERROR: user is not allowed to do action [find] on [admin.android_main]
Question: how can I modify my code a/o setup to ensure the db.collection.find() call references the proper database? I would have thought this was taken care of in the connection, where the database is explicitly called out.
I'm going to post the following answer which solves the issue above.
I found it by just playing around with random ideas.
I'd like to thank Heroku and their pathetic documentation for turning a 30 minute task into a 3 day ordeal.
client.connect(
process.env.MONGODB_URI,
function (err, database) {
console.log("CONNECT...");
if (err) {
console.log(err);
process.exit(1);
}
db = database.db(DATABASE);
var server = app.listen(process.env.PORT || 8080,
function () {
var port = server.address().port;
console.log("CONNECT App now running on port", port);
}
);
console.log("CONNECT Done.");
}
);
Apparently, to properly set the variable db (the one which you will use in the queries later) you should note that the database you receive in the connection callback is the admin database (that which was used for auth), and from that you make the db() request passing in the database you intend to use. All of this is utterly redundant b/c we pass the database we intend to use in the connection string. What a complete CF; but it's certainly not the 1st one. Enjoy.
I am trying to implement a feature which a user can decide on login to which DB to connect. As it is a web-app, running on a server which all the clients approach, how can I implement this feature without changing every client DB?
At our company we are using mongoose as the MongoDB API.
I read all the docs, and didn't notice any functionality for using multiple connections to different DB's on different hosts within the same App at once - without damaging other's client work.
The most valuable thing I have accomplished is to open few connections based on multiple mongoose instances, based on this post:
Mongoose and multiple database in single node.js project
I have created few files for example:
var mongoose = require('mongoose');
mongoose.createConnection('mongodb://10.20.100.71:27017/DB_NAME');
module.exports = exports = mongoose;
And then I required them:
let stageAccess = require('./databsesConnections/stageAccess');
let prodAccess = require('./databsesConnections/prodAccess');
I debugged the files and checked the connections are establishing.
Further more I checked in the mongoose docs and concluded that I can choose which connection is the default connection, as the docs state:
"Mongoose creates a default connection when you call mongoose.connect(). You can access the default connection using mongoose.connection."
So I tried:
mongoose.connection = mongoose.connections[1];
And it works fine.
So the actual question is, what will happen if client 1 approach the app, select to connect dbNum1 and starts to work,
then client 2 approach the app and select to connect to dbNum2?
I am working on a project that requires a dedicated database per registered user. I prefer working with MongoDB so I'm using that for the same (Am I Right?). The app uses a REST API as the backend (written in Node Express) and an AngularJS App. So, what I think of doing is whenever a user makes a request to some API endpoint say, a GET request to api/user/mydata, I would create a connection to his particular database, fetch the required data, close the connection and return the fetched data as the response. Is this approach correct? Also, I'm using Mongoose as the ODM and PassportJS for user Authentication. Moreover, users of my app are mutually exclusive. There is no data connection between a user with any other registered user.
There's a way to do that but only without using Mongoose. You would have to create a root connection to your MongoDB server (mind it, not to a particular database on that server) using the mongodb node module and then you can switch between the database as per your query requirement without creating a new connection per database as shown below:
const MongoClient = require('mongodb').MongoClient;
const assert = require('assert');
// URL to the root of MongoDB Server and not a particular db
const url = 'mongodb://localhost:27017';
// Database Names
const dbName1 = 'myproject1';
const dbName2 = 'myproject2';
// Use connect method to connect to the server
MongoClient.connect(url, function(err, client) {
assert.equal(null, err);
console.log("Connected successfully to server");
const db1 = client.db(dbName1);
const db2 = client.db(dbName2);
client.close();
});
You can't do this through mongoose, as mongoose and its models require connection to be made to a particular database and not to just the root db server. Anyways, I didn't want to give up mongoose for my own project so I just had to resort to initializing the db connection and its models per HTTP request by the user and closing the connection upon response.
The problem: (node.js application + mongodb Native driver)
I have a JSON file with more than 60000 Json Documents.the documents always a creation date and unique id called vid. and I need to insert in a MongoDB collection.
I need to insert the new vid or update the ones already existing with another document more recent.
What I already did:
https://github.com/TelmoIvo/PFC/blob/master/cfginit.js
What is happening:
After inserting/updating like 500 times and getting 287 documents in collection I get this error:
AssertionError: null == { [MongoError: connect EADDRINUSE] name : 'MongoError', message: 'connect EADDRINUSE' } at the line assert.equal (null, err);
from what I read, it's saying I have the connection to DB already in use. but I close after I insert/update everytime.
Any advice?
I wouldn't be calling MongoClient.connect every time. That's causing a ton of connections to open and close all the time which is overloading mongo. You should let the MongoClient manage the connection pool. Change it so that you store the db object from MongoClient.connect maybe in your init file add something like
//store this outside your init so its accessible to other functions
//this is what you will use to access the database
var db;
//add this to your init function
MongoClient.connect(url, function(err, database){
db = database;
}
Then in your functions to add and update use the db object you stored to update your collections and you won't need to keep opening connections. You can drop all the MongoClient.connect code and don't call db.close() since your connections are being shared to the object so let MongoClient manage them.
I have an Express App which connects to a MongoDB server at startup and serves requests on-demand (I don't disconnect - it's a single threaded server so no pooling - fairly simple stuff)
Problem is that it's possible the MongoDB server will be unavailable for periods of time (it's not on-site) and whilst the Express App doesn't crash, it seems that any requests made to the server will run indefinately until the connection is restored!
I'd like to limit that (e.g. throw an error back after a period of time) but I can't seem to make that happen...
I'm using connect options "{server: {auto_reconnect: true}}" which seems to ensure that once the MongoDB server reappears, requests complete (without it, requests made during downtime seem to run forever...) - and I don't have access to the client code so I can't fix it there...
I'd assumed a combination of 'connectTimeoutMS' or 'socketTimeoutMS' would allow me to terminate requests when MongoDB is unavailable for longer periods, but I just can't get those to work (I've tried them as connect options, passing them in the URI etc. etc.)
Any attempt to open a Collection and Find/Insert/Update just 'hangs' until the MongoDB reappears - I've left it over 30 mins and everything was just sitting these (and completed AOK when the network was restored!)
What's the best way around this? Should I open a connection specifically for each request (not really a performance issue - it's not a high volume app) or is there something else I'm missing?
Updated to add the connect code
var myDB
var mongodb = require('mongodb')
var uri = // some env vars and stuff
mongodb.MongoClient.connect(uri, {server: {auto_reconnect: true}}, function (err, db) {
myDB = db
})
myDB is then used elsewhere to open collections - and the handle from that is used to find/insert etc.
If the connection to the DB is interrupted, myDB.collection() calls (or calls to find/insert on their handles) will simply hang until the connection is restored - nothing I've tried will cause them to 'time out' sooner!?
I assume that you are using mongoose as a driver.
You'd catch the error by this.
var db = require('domain').create();
db.on('error', function(err) {
console.log('DB got a problem');
});
db.run(function() {
mongoose.connect(config, options);
});
or you can directly access
mongoose.connection.readyState
to check the statement of your DB.
Connection ready state
0 = disconnected
1 = connected
2 = connecting
3 = disconnecting
Each state change emits its associated event name.
http://mongoosejs.com/docs/api.html