Creating a Database programmatically with node-postgres - node.js

I want to create a Database when a user launches an application if it does not already exist. The way I tried doing this is, because there's no way that I know of to create a Database without having an existing connection, is to connect to the standard postgres Database, which is a default pre-existing one (I think).
Once I'm connected, I execute this from the default connection:
`SELECT 'CREATE DATABASE dbName WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'dbName')\\gexec`
This supposingly creates the Database if it doesn't exist. Once this runs, and I guaranteed to have a dbName database that exists. I thus shut off the other connection for it is no longer needed with
postgresCon.end();
And I connect to the recently created Database, in hopes to then use it to do whatever queries I would like and have the user be able to add workouts/exercises/whatever else.
The Problem
Whenever I run the Select create database workout... statement, I catch the error, and what it gives me is "Error: Connection Terminated", safe to say that this error isn't really... descriptive. In fact, this happens when workout database already exists. If it doesn't, it simply complains (later - when I try to execute a query on this database) that it doesn't exist.
This is the code, my theory is that it's Asynchronous and so though the connection "started connecting", the rest of the Queries are already being run and returning with "doesn't exist", because it hasn't been fully created yet.
Here is the code on pastebin for some highlighting (though apparently it's not happy with '' so the highlighting may be a bit scuffed): https://pastebin.com/fiiK35j7
If you need more detail/code/more clarity, do ask!
EDIT I've been told that my Theory is correct and that I should use async/await, but I am unsure how I could stop queries from being ran before the connection has been successfully completed, I assume it'd have to be something along the lines of:
async connection() => {await db.connect()}
connection()
.then(can do queries here)
.catch(panic)
but I am unsure what will actually work

I don't believe you can call \gexec from application code. That's a command which can only be used from the psql cli tool.
You'll have to issue and query and then do the conditional in your application code to subsequently create (or not) the database.
As was commented above, connecting with the DB superuser from an application is a bad idea. But there are some scenarios where it might be warranted, i.e. from a CI/CD pipeline where you want to provision databases and users if they don't exist for new environments. I did the following (using ts-postgres instead of node-postgres):
const [host, username, password] = ['localhost', 'user', 'pass']
const client = new Client({host, user: username, password})
await client.connect()
/////
// There is no CREATE DATABASE IF NOT EXISTS in postgres, which is decidedly annoying
// instead, we have to query and see if the database and user exists, and if not, create them
const dbQuery = await client.query(`SELECT FROM pg_database WHERE datname = $1`, [applicationDatabaseName])
if (dbQuery.rows.length === 0) {
// database does not exist, make it:
await client.query(`CREATE DATABASE ${applicationDatabaseName}`)
}
const userQuery = await client.query(`SELECT FROM pg_roles where rolname = $1`, [applicationDatabaseUser])
if(userQuery.rows.length === 0) {
//user doesn't exist. make it
await client.query(`CREATE USER ${applicationDatabaseUser} with ENCRYPTED PASSWORD '${applicationDatabasePassword}'`)
await client.query(`GRANT ALL PRIVILEGES ON DATABASE ${applicationDatabaseName} TO ${applicationDatabaseUser}`)
}
await client.end()

Related

Mongodb database.collection confusion

I created a MEAN project in heroku.
In MongoDB I have a DB / Collection like this:
db == content / collection == android_main
I have verified that the collection is in the proper database (and not in admin) via the following MongoSH exchange:
Atlas atlas-xxxxxx-shard-0 [primary] content> use admin
switched to db admin
Atlas atlas-xxxxxx-shard-0 [primary] admin> show collections
Atlas atlas-xxxxxx-shard-0 [primary] admin> use content
switched to db content
Atlas atlas-xxxxxx-shard-0 [primary] content> show collections
android_main
In MongoDB I also have a user w/ specific privs to read this db/collection.
In the code (Node.js) I use the following connection string:
mongodb+srv://<USER>:<PASS>#<MONGO URL>/content?retryWrites=true&w=majority
The connection completes successfully. I pass in the user I mentioned above, e.g. the user with just the specific read privs on the database (content) and collection (android_main).
In the code, the mechanism I use to get the database variable via the connection and connection string specifically is:
mongodb.MongoClient.connect(process.env.MONGODB_URI, function (err, database) {
Now, in the code, in response to an `HTTP GET` I issue the following:
db.collection("android_main").find({}).toArray(function(err, docs) {
...
}
I get back this error:
ERROR: user is not allowed to do action [find] on [admin.android_main]
Question: how can I modify my code a/o setup to ensure the db.collection.find() call references the proper database? I would have thought this was taken care of in the connection, where the database is explicitly called out.
I'm going to post the following answer which solves the issue above.
I found it by just playing around with random ideas.
I'd like to thank Heroku and their pathetic documentation for turning a 30 minute task into a 3 day ordeal.
client.connect(
process.env.MONGODB_URI,
function (err, database) {
console.log("CONNECT...");
if (err) {
console.log(err);
process.exit(1);
}
db = database.db(DATABASE);
var server = app.listen(process.env.PORT || 8080,
function () {
var port = server.address().port;
console.log("CONNECT App now running on port", port);
}
);
console.log("CONNECT Done.");
}
);
Apparently, to properly set the variable db (the one which you will use in the queries later) you should note that the database you receive in the connection callback is the admin database (that which was used for auth), and from that you make the db() request passing in the database you intend to use. All of this is utterly redundant b/c we pass the database we intend to use in the connection string. What a complete CF; but it's certainly not the 1st one. Enjoy.

Force pool requests to resolve in the order that they're made

I have a Node.js server running with a PostgreSQL database. I have the client set up to make a request to edit data, wait for said request to resolve, and then send another request to receive back all the data with the new changes applied.
This works well, however for some reason it fails every so often and I don't know why. The data in the database DOES get changed, and the new data IS requested, but for some reason the new changes aren't showing up in the new data (however refreshing the page causes it to reflect the new changes). I assumed it was because my requests were sometimes sending out of order, but that doesn't appear to be the case. My best guess at what's happening is that the database is falling behind, and so my Node.js code executes extremely fast and tells the client that it's ready to send the new data, but by the time it asks the database to give it the new data the database has sometimes not had enough time to finish updating its contents, and so it is giving old data...
The code I'm using to make calls to the database is such:
const {Pool} = require("pg");
const pool = new Pool({
user: "postgres",
database: "databasename",
password: "mypassword",
port: 5432
});
pool.connect((err,client,done) => {
client.query("SELECT id, title, description, people, due_date, real_due_date FROM table ORDER BY due_date;",(error,result) => {
if(error){
console.log(error);
resolve("error");
}else{
resolve(JSON.stringify(result.rows));
}
});
done();
});
(this is just a snippet to demonstrate the structure I use, the actual code's content isn't specific to the question.)
I'm using the same pool to make all the requests to the database. I'm wondering, is there a way that I can force this pool to complete the requests in the order they're received such that it will only execute the SELECT query after the UPDATE has fully completed? If so, do you think that will fix my problem?

Dynamically passing dbName to mongoose.connect

For a MEAN app, I need to pass the dbName depending on the logged-in user. The flow of events is as follows.
User logs-in and gets authenticated using an auth specific REST-API.
Auth-API returns to Angular the user account data which includes the userSpecificDbName.
Thereafter, Angular makes all calls to the app's main REST-API. With all calls to the API, account data is passed and the API shd use the dbName from the account data in the following statement of app.js
mongoose.connect(uri, { dbName: <userSpecificDbName> })
.then( () => {
console.log('Connection to the Atlas Cluster is successful!')
})
.catch( (err) => console.error(err));
Is this possible? And how do I pass data in app.js for this purpose?
It is certainly possible to do that, Although keep in mind you would need to open and close connections per request to do that, which is not a regular pattern in Node.js
Normally you would connect to the database once and then fire queries. But in your case, you would disconnect every time after you've sent a response and reconnect and the beginning to the DB.
Alternatively, you could most probably do a useDb call as suggested in the following answer:
switching database with mongoose
That would help you switch the database.

Cannot read/write on a MongoDB Atlas database using Mongoose

I have a new sandbox cluster on MongoDB Atlas that I am connecting to with mongoose on a Node.js server. This is the code I'm using to connect:
const mongoDbUrl =
`mongodb+srv://${username}:${password}#mydb-sandbox-12345.mongodb.net/testdb`
mongoose.connect(mongoDbUrl)
const connection = mongoose.connection
connection.on('connected', () => {
console.log('Connected to mongodb')
})
In the Atlas dashboard I have a readWriteAnyDatabase user that I am authenticating with. Authentication works as expected. I am able to connect to the database and no error is thrown on the connection. I can confirm this by removing a character in the password - authentication fails and I'm unable to connect.
The problem is when I try to insert documents.
const UserModel = require('./models/User')
UserModel.create({
name: 'Hello Atlas'
})
I get the following error:
MongoError: not authorized on admin to execute command {
insert: "users",
documents: [
[{
name Hello Atlas
} {
_id ObjectIdHex("5aa17933d72d25730a340611")
} {
__v 0
}]
],
ordered: false
}
As far as I know the user I'm authenticating with should have permission to read and write on the database I'm connecting to. The other part I don't understand is that the error shows that it's trying to write to admin even though my url is connecting to testdb.
Not sure if you have seen this post, but it could be because you are on a free cluster? Hope this helps.
UPDATE
I looked into the problem further and reproduced it on my own. I got the same error. However, I noticed that at one point Atlas provided me with a choice of connection strings. I went back to that page and chose I am using driver 3.4 or earlier.
The connection string looks like this:
const mongoDbUrl = `mongodb://${username}:${password}#cluster0-shard-00-00-1wntz.mongodb.net:27017,cluster0-shard-00-01-1wntz.mongodb.net:27017,cluster0-shard-00-02-1wntz.mongodb.net:27017/testdb?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin`;
It worked with that connection string.
It looks like the free version of MongoDB Atlas launches with v3.4
If you are using free cluster. change 'admin' to 'test' in the path:
mongodb+srv://username:password#cluster0-yauj8.mongodb.net/test?retryWrites=true&w=majority
This worked for me
Going off #rithesh-mys 's answer. Replacing it with "test" is specifying the db. So you have to change it to the db name that you would use.
I had the same problem. I was trying a lot of connection strings, but the one for olders mongo shell (3.4 or earlier) worked for me.
mongodb://my_username:my_password#cluster0-shard-00-00.osobw.mongodb.net:27017,cluster0-shard-00-01.osobw.mongodb.net:27017,cluster0-shard-00-02.osobw.mongodb.net:27017/my_database?ssl=true&replicaSet=atlas-xw3rfy-shard-0&authSource=admin
I think that newer versions of connection strings don't work with mongoose, at least with free clusters.
Make sure the user you have created has the write and read privileges.

How do I correctly authenticate in mongo for a user that has access to multiple databases?

I have a user with the role: {role: 'root', db:'admin'} which should have access to all the databases in my mongo instance. I'm using mongoskin in node.js to interact with mongo. My problem is that it isn't correctly accessing my databases. If I authenticate with
mongodb://owner:mylocalpassword#localhost:27017/mydatabase
It simply gives me MongoError: Authentication failed. If I instead auth with:
mongodb://owner:mylocalpassword#localhost:27017/admin
it authenticates, but I can't access mydatabase.
Here's my connection code:
var connection = mongoskin.db("mongodb://owner:mylocalpassword#localhost:27017/admin", {journal:true, auto_reconnect:true})
I assume that since I'm accessing the admin database there, that's the only one it interacts with. So I tried do then do this:
var mydatabaseConnection = connection.db('mydatabase')
But when I use that, my app is returning no results for queries on collections that I know have data. What am I doing wrong here? How do I get this user to access a database other than admin?
Ok, so I found out that mongoskin's db method simply doesn't work. Finally I'm forced to completely remove mongoskin from my codebase. The real answer here is don't use mongoskin.
This code worked with mongo native:
MongoClient.connect("mongodb://owner:mylocalpassword#localhost:27017/admin", {journal: true, auto_reconnect:true}).then(function(db) {
console.log("Connected!")
var mydb = db.db('mydatabase')
var User = mydb.collection('User')
return User.find({}).toArray().then(function(users) {
console.log(users)
db.close()
})
}).catch(function(e) {
console.log(e.stack)
})

Resources