Firstly, I'm new to everything here... and new to StackOverflow, so apologies in advance for being a newbie and I'm ready for my thrashing... LOL.
We use a Heroku.addon for Postgres and utilize/reference environment variables globally to access the right database.
We have a config.js file in our application's root directory like:
db: process.env.DB_URL || {
client: 'pg',
connection: {
database: 'db_name',
user: 'user_name'
}
},
Would someone here be able to guide me on how to integrate code that initializes 'custom pool' information into this setup like the example found on
[http://knexjs.org/#Installation-pooling][http://knexjs.org/#Installation-pooling]
var knex = require('knex')({
client: 'mysql',
connection: {
host : '127.0.0.1',
user : 'your_database_user',
password : 'your_database_password',
database : 'myapp_test'
},
pool: { min: 0, max: 7 }
});
On Heroku, process.env.DB_URL is a complex URL that is similar to:
postgres://(redacted)#ec5-87-1-47-54.compute-1.amazonaws.com:5432/d8n2e9ebd0q9it
So, I am hoping there is a clean way to also pass 'custom pool' information as well either here or in another file/location.
The database is referenced throughout the backend of our application via Bookshelf/Knex. The reference to bookshelf looks similar to:
var knex = require('knex')(config.db);
var bookshelf = require('bookshelf')(knex);
You might be able to do it by reading just connection details from env variable, like this:
db: {
client: 'pg',
connection: process.env.DB_URL || {
database: 'db_name',
user: 'user_name'
},
pool: { min:5, max:20 }
},
Related
Three months ago, I created an Strapi App that has deployed on Heroku, and everything works fine. I used macOS 10.13.6 and node 14.15.4 for the local environment
The configuration of database was created inside a file named database.js which located in rootApp/config/env/production/database.js
The following are everything config inside these file (database.js):
const parse = require('pg-connection-string').parse;
const config = parse(process.env.HEROKU_POSTGRESQL_MAROON_URL);
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: "ec2-35-169-184-61.compute-1.amazonaws.com",
port: 5432,
database: "d3d9tcukxxx",
username: "mwtwuvkwxxxx",
password: "42f0337xxxxx",
},
options: {
ssl:true,
},
},
},
});
But after 3 months (right now), I checked from heroku logs --tail then these app getting an error and the message was:
error error: no pg_hba.conf entry for host "3.86.36.125", user "mwtwuvkwtrqpir", database "d3d9tcukrk5fgh", SSL off
I used Strapi 3.2.5 , and I was deployed on Heroku Postgres with Plan free (Hobby).
I hope everyone helping me for this questions, and hope helping others for same case.
Thank you
We had the same issue on our Heroku instances and just recently found a fix.
Adding rejectUnauthorized to the database config appears to work.
config/database.js
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: env('DATABASE_HOST', 'localhost'),
port: env.int('DATABASE_PORT', 5432),
database: env('DATABASE_NAME', 'strapi'),
username: env('DATABASE_USERNAME', 'strapi'),
password: env('DATABASE_PASSWORD', 'strapi'),
schema: env('DATABASE_SCHEMA', 'public'), // Not Required
ssl: {
rejectUnauthorized: env.bool('DATABASE_SSL_SELF', false), // For self-signed certificates
},
},
options: {
ssl: env.bool('DATABASE_SSL', false),
},
},
},
});
I cannot take full credit however, it was this post on the Strapi forum that led me to the answer:
https://forum.strapi.io/t/error-no-pg-hba-conf-entry-for-host-ssl-off/3409
subsequently this link:
https://strapi.io/documentation/developer-docs/latest/setup-deployment-guides/configurations.html#database
I am currently using node-postgres to create my pool. This is my current code:
const { Pool } = require('pg')
const pgPool = new Pool({
user: process.env.PGUSER,
password: process.env.PGPASSWORD,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
port: process.env.PGPORT,
ssl: {
rejectUnauthorized: true,
// Would like to add line below
// ca: process.env.CACERT,
},
})
I found another post where they read in the cert using 'fs' which can be seen below.
const config = {
database: 'database-name',
host: 'host-or-ip',
user: 'username',
password: 'password',
port: 1234,
// this object will be passed to the TLSSocket constructor
ssl: {
ca: fs.readFileSync('/path/to/digitalOcean/certificate.crt').toString()
}
}
I am unable to do that as I am using git to deploy my application. Specifically Digital Oceans new App Platform. I have attempted reaching out to them with no success. I would prefer not to commit my certificate in my source control. I see a lot of posts of people suggesting to set
ssl : { rejectUnauthorized: false}
That is not the approach I want to take. My code does work with that but I want it to be secure.
Any help is appreciated thanks.
Alright I finally was able to figure it out. I think the issue was multiline and just unfamiliarity with dotenv for my local developing environment.
I was able to get it all working with my code like this. It also worked with the fs.readFileSync() but I didn't want to commit that to my source control.
const { Pool } = require('pg')
const fs = require('fs')
const pgPool = new Pool({
user: process.env.PGUSER,
password: process.env.PGPASSWORD,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
port: process.env.PGPORT,
ssl: {
rejectUnauthorized: true,
// ca: fs.readFileSync(
// `${process.cwd()}/cert/ca-certificate.crt`.toString()
// ),
ca: process.env.CA_CERT,
},
})
.on('connect', () => {
console.log('connected to the database!')
})
.on('error', (err) => {
console.log('error connecting to database ', err)
})
Now in my config.env I had to make it look like this:
CA_CERT="-----BEGIN CERTIFICATE-----\nVALUES HERE WITH NO SPACES AND A \n
AFTER EACH LINE\n-----END CERTIFICATE-----"
I had to keep it as a single line string to have it work. But I was finally to connect with
{rejectUnauthorized:true}
For the digital ocean app platform environment variable, I copied everything including the double quotes and pasted it in there. Seems to work great. I do not think you will be able to have this setting set to true with their $7 development database though. I had to upgrade to the managed one in order to find any CA cert to download.
I am trying to use Sequelize (v 5.21.13) to connect to my SQL Server database in my Expressjs app.
dbconfig.js
var dbConfig = {
server: process.env.DB_HOST,
authentication: {
type: 'default',
options: {
userName: process.env.DB_USERNAME,
password: process.env.DB_PASSWORD
}
},
options: {
database: process.env.DB_NAME
}
};
module.exports = dbConfig;
index.js:
const dbConfig = require('./dbConfig');
const Sequelize = require('sequelize');
const connection = new Sequelize(
dbConfig.options.database,
dbConfig.authentication.options.userName,
dbConfig.authentication.options.password,
{
host: dbConfig.server,
dialect: 'mssql',
}
);
connection.sync().then(() => {
console.log('Connected!');
}).catch((e) => {
console.log('Error:\n', e);
});
Now the thing is that each time I run the server, I get this error
AccessDeniedError [SequelizeAccessDeniedError]: Login failed for user 'master'.
I have also tried adding additional properties to the new Sequelize() like the following with no luck.
dialectOptions: {
instanceName: 'instance',
options: {
encrypt: true,
trustServerCertificate: true,
requestTimeout: 30000
}
}
I even tried changing the password to a very simple one with no special characters, connection with Datagrip works after changing but not using Sequelize.
Everything on the dbconfig object is correct so I don't see what the issue might be.
Solved it. I was putting the the db instance id as the database name, I realized that the database name was different. Changed it and I'm now connected through Sequelize.
This is my first time try to host nodeJS application - built with hapi.js, typeorm and postgresql - on heroku. I've create two apps on heroku - for "staging" (server-staging) and "production" (server-prod) - that using same code but will use different configuration. Why different configuration? because each application on heroku will use different postgres credential, as it's attached as an add-ons.
objective
My objective/main question is How and where I have to set the database config for my application?
I use .env file (which I ignore in .gitignore - I don't want to put the credential in my repo) to connect the application to my local database. Here is how the .env looks like:
HOST=localhost
PORT=3001
TYPEORM_CONNECTION=postgres
TYPEORM_HOST=localhost
TYPEORM_USERNAME=postgres
TYPEORM_PASSWORD=password
TYPEORM_DATABASE=database
TYPEORM_PORT=5432
TYPEORM_SYNCHRONIZE=true
TYPEORM_LOGGING=false
In the application, I never do/write code such process.env.TYPEORM_USERNAME since its done by the typeorm node_modules. What I do to start the connection is by doing this:
const server = new Hapi.Server({
port: process.env.PORT,
host: process.env.HOST,
routes: {
cors: Cors,
},
});
await server.register(Plugins);
server.route(Router);
await createConnection();
await server.start();
And my application automatically connected to the specified database as defined in the .env. Now, in heroku, the credential is lies here:
All information lies there, but, [Q1] I don't know how to tell my application (of course, without store the credential in my code/repo) that I have to use the config as defined in above picture? Also, as stated in above image, "Heroku rotates credentials periodically and updates applications where this database is attached.". Does it means the credentials will changed periodically? [Q2] If yes, is there any way to make my application auto recognise the new credential?
Sorry if my explanation make confused. If you did not understand what I am trying to achieve, please ask things that you don't understand, so I can fix/update my question to make it understandable.
Anyway, I found this example first-example and second-example. But, they are using process.env.DATABASE_URL, which contain credential. I think, it means that they not ignore their .env file in their repo?
*) Note: Q1 means Question 1, and so for the rest
Write a ormconfig.js file in the root of your repo. This way you can access the environment variables like the url provided from heroku and you don't have credentials in your repo.
require('dotenv').config();
module.exports = [
{
name: 'default',
type: 'postgres',
url: process.env.DATABASE_URL,
synchronize: false,
logging: false,
entities: ['dist/entities/*.*'],
migrations: ['dist/database/migrations/**/*.js'],
subscribers: ['dist/database/subscribers/**/*.js'],
cli: {
entitiesDir: 'dist/entities',
migrationsDir: 'dist/database/migrations',
subscribersDir: 'dist/database/subscribers',
},
},
{
name: 'development',
type: 'postgres',
host: process.env.POSTGRES_HOST,
port: process.env.POSTGRES_PORT,
username: process.env.POSTGRES_USER,
password: process.env.POSTGRES_PASSWORD,
database: process.env.POSTGRES_DB,
synchronize: true,
logging: true,
entities: ['src/entities/*.*'],
migrations: ['src/database/migrations/**/*.ts'],
subscribers: ['src/database/subscribers/**/*.ts'],
cli: {
entitiesDir: 'src/entities',
migrationsDir: 'src/database/migrations',
subscribersDir: 'src/database/subscribers',
},
},
];
With this configuration you can then get a specific configuration in javascript/typescript:
let connectionOptions: ConnectionOptions;
if(process.env.NODE_ENV ==='development') {
connectionOptions = await getConnectionOptions("development");
} else {
connectionOptions = await getConnectionOptions("default");
}
await createConnection(connectionOptions);
I've come across several posts for this question however, none of them seem to have an actual answer. Several ideas, yet none of them work.
After digging around both the Sequelize and Tedious packages and watching my config get passed down correctly, I'm at a loss.
I am trying to run migrations against a new database in MSSQL. I have no problem connecting to it with the same creds I'm using here so I know that's not the issue.
I have my config.js that is pulling env vars. With the exception of my custom console statements, this file was auto generated from sequelize and is correctly referenced in my sequelizerc
require('dotenv').config()
console.log('[+] Loading database config...')
if (process.env.NODE_ENV === 'production') {
console.log(`[+] Using database: ${process.env.PROD_DB_DATABASE}`)
} else if (process.env.NODE_ENV === 'development') {
console.log(`[+] Using database: ${process.env.DEV_DB_DATABASE}`)
} else if (process.env.NODE_ENV === 'test') {
console.log(`[+] Using database: ${process.env.TEST_DB_DATABASE}`)
} else if (process.env.NODE_ENV === 'local') {
console.log(`[+] Using database: ${process.env.LOCAL_DB_DATABASE}`)
} else {
console.log(`[-] CANNOT LOAD DATABASE FROM ENV: ${process.env.NODE_ENV}`)
process.exit()
}
module.exports = {
production: {
database: process.env.PROD_DB_DATABASE,
username: process.env.PROD_DB_USERNAME,
password: process.env.PROD_DB_PASSWORD,
host: process.env.PROD_DB_HOST,
port: process.env.PROD_DB_PORT,
dialect: process.env.PROD_DB_DIALECT,
storage: process.env.PROD_DB_STORAGE,
logging: false,
dialectOptions: {
instanceName: process.env.PROD_INSTANCE_NAME
},
pool: {
min: 5,
max: 1,
acquire: 6000,
idle: 6000
}
},
development: {
database: process.env.DEV_DB_DATABASE,
username: process.env.DEV_DB_USERNAME,
password: process.env.DEV_DB_PASSWORD,
host: process.env.DEV_DB_HOST,
port: process.env.DEV_DB_PORT,
dialect: process.env.DEV_DB_DIALECT,
storage: process.env.DEV_DB_STORAGE,
logging: console.log,
dialectOptions: {
instanceName: process.env.DEV_INSTANCE_NAME,
debug: true
},
pool: {
min: 5,
max: 1,
acquire: 6000,
idle: 6000
}
},
test: {
database: process.env.TEST_DB_DATABASE,
username: process.env.TEST_DB_USERNAME,
password: process.env.TEST_DB_PASSWORD,
host: process.env.TEST_DB_HOST,
port: process.env.TEST_DB_PORT,
dialect: process.env.TEST_DB_DIALECT,
storage: process.env.TEST_DB_STORAGE,
logging: false
},
local: {
database: process.env.LOCAL_DB_DATABASE,
username: process.env.LOCAL_DB_USERNAME,
password: process.env.LOCAL_DB_PASSWORD,
host: process.env.LOCAL_DB_HOST,
port: process.env.LOCAL_DB_PORT,
dialect: process.env.LOCAL_DB_DIALECT,
storage: process.env.LOCAL_DB_STORAGE,
logging: false
}
}
When i run my migration i get the error:
> node_modules/.bin/sequelize db:migrate
// ERROR: Login failed for user ''.
As mentioned above I dug through sequelize and tedious and my config is getting passed properly through both so i know it's not an env var issue or a NODE_ENV issue.
Anyone have any ideas here? I'm about to smash my face into my keyboard.
More for older versions:
If you are using sequelize#4, then it seems there is a hidden requirement that you must use tedious#<=5.
Which version of Sequelize are you using? If it's v5,
According to Sequelize v5's document:
Sequelize now works with tedious >= 6.0.0
However, in its package.json, it does not depend on tedious at all.
Since your program still runs, I guess you manually installed an older version of tedious before, which caused this strange problem.
Manually installing tedious of version>=6 should solve this problem, just like stated in its Getting started document page:
You'll also have to manually install the driver for your database of choice:
# One of the following:
$ npm install --save pg pg-hstore # Postgres
$ npm install --save mysql2
$ npm install --save mariadb
$ npm install --save sqlite3
$ npm install --save tedious # Microsoft SQL Server
const Sequelize = require('sequelize');
const sequelize = new Sequelize(
process.env.DB_NAME, process.env.DB_USER, process.env.DB_PASS, {
dialect: 'mssql',
host: process.env.DB_HOST, //This is an IP
dialectOptions: {
options: {
instanceName: process.env.DB_INSTANCE_NAME,
trustServerCertificate: true
},
}
}
);
module.exports = {
sequelize,
Sequelize
};
Here is another solution, it's working for me.
I was getting the same error. The reason was due to explicitly mentioning the name of the DB in the sequelize config file and it did not exist. The reason could be different in your case but a quick look at SQL Server error logs will give you the reason for the failure.
Login failed for user 'user'. Reason: Failed to open the explicitly specified database 'dbo'. [CLIENT: XX.XX.XX.XX]