Relation does not exist error with pg-promise and postgresql? - node.js

I am using pg-promise on Node 7.2.1 with PostgreSQL 9.4. I am using the code below to connect to the PostgreSQL server:
// pg-promise connection elements.
// Database initialization options.
const pg_options = {
// Initialization options.
};
// Create an instance of the pg-promise library.
const g_pg_promise = require('pg-promise')(pg_options);
// Connection string for a local connection to the 'summarize' database.
const g_pg_connection =
{
host: 'localhost',
port: 5432,
database: 'db_stats',
user: 'db_user',
password: '{password here}'
}
// Connect to the database.
const g_pg_database = g_pg_promise(g_pg_connection);
// Expose the database object globally. The database object should only
// be instantiated once and then shared.
module.exports = {
g_pg_promise: g_pg_promise,
g_pg_database: g_pg_database
};
I know the connection parameters are valid because I use the exact same values in other non-Node.JS apps to connect to the same PostgreSQL server. I also know that db_stats is a valid database name. I have worked with that database for quite a while with other non Node.JS apps and via pgAdmin 3.x.
However, when I attempt to connect using pg-promise I get the following error:
error: relation "db_stats" does not exist
I did see the SO post below:
Unable to query PostgreSQL database in NodeJS using pg-promise - "relation does not exist"
But that post did not make sense to me because I believe Vitaly, the author of pg-promise, was telling the poster that he did not have a table called users, when it looks to me that the poster was trying to access the database called users and he definitely had a database with that name.
In any case, I definitely have a database named db_stats so I'm not sure why I am getting this error. How can I solve this?

error: relation "db_stats" does not exist
The error you get from PostgreSQL is not referring to a database named "db_stats", but to a relation (either a table or a view) with that name.
That is, db_stats most probably appears in the FROM clause of a SELECT query (although it may be also a INSERT, UPDATE, DELETE, ...).
There is neither a db_stats table nor a view in your database. Or, maybe, it exists, but it exists in a schema that is not part of your current search_path.
To find out, check two things:
SELECT
*
FROM
information_schema.tables
WHERE
table_name = 'db_stats' ;
If there is a table there... you already know which schema(s) contains it/them.
Then, make sure that this schema is part of your search_path by means of:
SHOW search_path ;
You might have to execute this SQL statements by adding the adequate code to your application, and use a debugger to check what's returned in that environment.

Related

How to access one single collection in a database in MongoDB

When connecting to MongoDB with a Node.JS application using express, how do you connect to just one collection if the database has multiple collections?
I have the following MONGODB_URI:
MONGODB_URI=mongodb+srv://username:******#cluster0.dayw5.mongodb.net/phonebook?retryWrites=true&w=majority
The database "phonebook" has three collections, "numbers", "users" and "people". How do you change the MONGODB_URI to just connect to "numbers" ?
At the moment, the connection is successful, but nothing is being fetched.
You don't; the connection URI is granular to the database only, in your case phonebook. From the resources returned, you must specify a collection to access e.g. coll = db['users'] and then perform operations against that collection object e.g.
c = coll.find({name:"foo"});
c.forEach(function(doc) {
printjson(doc);
});
Both answers given above are basically correct.
I tried to connect to an existing collection but was unable to do so. Instead, MongoDB automatically creates a new collection in the database and gives it a new, custom name: "people" in my case which I did not specify anywhere.

Mongoose - Keep local database in sync with remote database

I have access to two separate databases that I'd like to keep in sync, a new one and an existing one, which will be in separate physical locations. The new one is going to be used to service an external API, so to cut down on request time, I think it makes sense to only query the local database for API requests.
My initial approach was to use mongoose.createConnection and limit the local collection to minor metadata and directly access the remote collection, but that's what I'm now looking to avoid.
Another approach might be to use mongoose.createConnection to periodically query the remote db and update the local one, but it could be costly if I want to do make frequent updates.
There are ways to cut down the cost - for example, there is a lastUpdated property in the relevant collection on the existing database, which could be used to limit the remote query to recently updated records such as:
RemoteCollection.find({
lastUpdated: {$gte: Date.now() - lookbackPeriod}
})
But I'm wondering if there's any native functionality of mongoose/mongoDB that can be used more efficiently make the updates. I also thought about mongodump and mongorestore to keep a full local copy of the records I needed, but that also seems costly.
Any help is appreciated.
After a bit of reading and thanks to Jake's comment, it looks like it's working. I need to do some more setup, but the code below should work and is based on this section from the docs:
https://mongoosejs.com/docs/models.html#change-streams
The first step would be start mongod with the --replSet flag:
mongod --replSet "rs0" --bind_ip localhost,<hostname(s)|ip address(es)>
Then close and restart mongo and run rs.initiate() on the database. You can then check the status of the replica set with rs.status(). If that command works and returns a result, the replica set functionality should be there.
Then within Node, you can do something like this:
// The docs reference creating a new model but you can just import an existing one
const RemotePerson = require('./models/RemotePerson');
const LocalPerson = require('./models/LocalPerson');
RemotePerson.watch().on('change', data => {
if (data.operationType === "insert") {
LocalPerson.create(data.fullDocument);
} else if (data.operationType === "update") {
LocalPerson.findByIdAndUpdate(data.documentKey, {
$set: data.updateDescription.updatedFields
});
}
});

How to specify a schema name in the Postgres URL for connecting to a PostgreSQL database on Heroku

I am connecting my parse-server application to a PostgreSQL database hosted on the Heroku-PostgreSQL service.
My database is with a schema called gc which is different to the default public schema on Postgresql.
I used the following to connect to the database from my parse-server application.
"postgres://{USERNAME}:{PASSWORD}#{HOSTNAME_ON_AWS}:5432/{DATABASE_NAME}?ssl=true"
But the issue was it was connected to the public schema but not the gc schema I wanted.
Is there a way to specify the schema name in Postgres URL?
I don't think you can,
what you can do however is associate a schema search path with a database user, so if you want a different schema you'd need to use a different username to connect as.
SQL:
alter user fred set search_path to 'gc';
I attach the parameter options=-csearch_path=XXX to the URI. It works.
psql "postgresql://user:pwd#127.0.0.1:6789/postgres?options=-csearch_path%3Dabc"
It shows the current schema below.
postgres=> show search_path;
search_path
-------------
abc
(1 row)
And take notice that I use the URLencode for the character =. Its URL encoding is %3D. Otherwise, you can't get the correct result in the shell.
Here are some references:
https://www.postgresql.org/docs/current/libpq-connect.html
https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS

Authentication error when connecting to specific mongodb database

I am currently using nodejs with mongodb native driver. So my mongodb has been set with admin auth with root role, so I can log in using robomongo or command line just fine. So back to my project, I m able to connect to mongodb just fine if i set my connection string:
mongodb://admin:password#localhost:27017/
However, if I use this connection string:
mongodb://admin:password#localhost:27017/specificdb
It return as:
MongoError: Authentication failed
I am able to access the db using command and robomongo, is there anything I can do? I have added the admin user under the db, but still got the same problem.
The database you specify is the one you authenticate on. If the user is not known/has no roles for that database it cannot authenticate to it.
https://docs.mongodb.com/manual/reference/connection-string/
What you can do is either create the (or a new) user for that database or use an authentication database parameter.
this worked for me :
first solution on connection :
mongoose.connect('mongodb://....', {
// add this line
authSource:"admin",
....
})
or second solution add to the uri "?authSource=admin" :
mongodb://user:password#host/yourDB?authSource=admin
** to note that I am using mongoose and this is the documentation specefic statement and the url :
authSource - The database to use when authenticating with user and pass. In MongoDB, users are scoped to a database. If you are getting an unexpected login failure, you may need to set this option.
https://mongoosejs.com/docs/connections.html#options

User specific database in MongoDB

I am currently working on an inventory management software in Node js and MongoDB. I am pretty new to MongoDB, having worked in Oracle and MySQL for most of my projects.
Is it possible to create a separate database schema for every client who uses my software, with each client having access only to his copy of the database schema and collections?
The equivalent of selecting data in Oracle database would be
Select * from User1.table,
Select * from User2.table etc
Also, if it were possible, how would it be implemented using a node js mongo db client like mongoose?
I looked at MongoDB documentation, but it talks mainly about adding users to a database for authorization.
I apologize if it seems like a silly question, but id appreciate it if someone could point me in the right direction for this.
Before starting to invest a lot of time in the development of your project, check out other possible approaches to the scenario that you are trying to build.
I did a quick search on SO and found some additional threads with similar scenarios:
MongoDB Database vs. Collection
MongoDB Web App - Database per User
Additional info about mongoose database creation
Whenever you call the connect method on the mongoose object, you are either connecting to an existing database or you are creating it in case it doesn't already exist.
You could have a function that allows you to pass in a name argument with the name and create databases programmatically:
function createDatabase(name) {
var conn_string = 'mongodb://localhost/';
if (typeof name == 'string') {
conn_string += name;
}else{
return false;
}
mongoose.connect(conn_string);
}
Also, be aware that a database will be created when you first insert a record in a collection of that particular database.
It is not sufficient to only connect to the database, you also have to insert a record.
As per my previous example, you could also pass a schema parameter to the function, tailored to each user's profile and fire an insert statement after you connect to that database.

Resources