Heroku Postgres Maintenance change env var and break the app - node.js

I deployed a NodeJS application to Heroku and attach to it a Postgres database with Heroku Postgres addons, all was working perfectly.
After a few hours, I was unable to authenticate my application. After checking the logs, I understand that the application is unable to connect to the database because of a wrong password.
After that, I also got 2 emails from Heroku that alert me of the maintenance.
So I checked the credentials database and the credentials are all the same except the URI.
I also checked on my env vars and I saw that Heroku added 2 more variables HEROKU_DATABASE_URL and HEROKU_POSTGRESQL_CRIMSON_URL, I tried to edit them with the new URI of the updated Database but do have permission ...
I also tried to delete and recreate a new database but every time the Heroku Maintenance come back and break my app
Here is my connection code to Postgres (using nest js)
TypeOrmModule.forRootAsync({
imports: [ConfigModule],
inject: [ConfigService],
useFactory: async (configService: ConfigService) => {
const isProduction = configService.get('STAGE') === 'prod';
return {
ssl: isProduction,
extra: {
ssl: isProduction ? { rejectUnauthorized: false } : null,
},
type: 'postgres',
autoLoadEntities: true,
synchronize: true,
host: configService.get('DB_HOST'),
port: configService.get('DB_PORT'),
username: configService.get('DB_USERNAME'),
password: configService.get('DB_PASSWORD'),
database: configService.get('DB_DATABASE'),
};
},
}),
Here is the error i get
ERROR [ExceptionHandler] password authentication failed for user
"ryooypezxslwmn"

Heroku Postgres provides and maintains one environment variable per provisioned database:
As part of the provisioning process, a DATABASE_URL config var is added to your app’s configuration. DATABASE_URL contains the URL your app uses to access the database. If your app already has a Heroku Postgres database and you’ve provisioned another one, this config var’s name instead has the format HEROKU_POSTGRESQL_<COLOR>_URL (for example, HEROKU_POSTGRESQL_YELLOW_URL).
When your database connection string needs to change, Heroku automatically updates the DATABASE_URL and HEROKU_POSTGRESQL_<COLOR>_URL variables.
But you aren't using these variables. The DB_HOST, DB_PORT, etc., environment variables aren't set by Heroku, and Heroku won't update them for you. Rather than manually setting those variables, I strongly urge you to use the variables Heroku provides:
The value of your app’s DATABASE_URL config var can change at any time. Do not rely on this value either inside or outside your Heroku app.
I'm not familiar with TypeORM, but it looks like you can pass it a url directly:
TypeOrmModule.forRootAsync({
imports: [ConfigModule],
inject: [ConfigService],
useFactory: async (configService: ConfigService) => {
const isProduction = configService.get('STAGE') === 'prod';
return {
ssl: isProduction,
extra: {
ssl: isProduction ? { rejectUnauthorized: false } : null,
},
type: 'postgres',
autoLoadEntities: true,
synchronize: true,
url: configService.get('DATABASE_URL'), // <-- here
};
},
}),

Related

Strapi CMS, Heroku error: no pg_hba.conf entry for host

Three months ago, I created an Strapi App that has deployed on Heroku, and everything works fine. I used macOS 10.13.6 and node 14.15.4 for the local environment
The configuration of database was created inside a file named database.js which located in rootApp/config/env/production/database.js
The following are everything config inside these file (database.js):
const parse = require('pg-connection-string').parse;
const config = parse(process.env.HEROKU_POSTGRESQL_MAROON_URL);
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: "ec2-35-169-184-61.compute-1.amazonaws.com",
port: 5432,
database: "d3d9tcukxxx",
username: "mwtwuvkwxxxx",
password: "42f0337xxxxx",
},
options: {
ssl:true,
},
},
},
});
But after 3 months (right now), I checked from heroku logs --tail then these app getting an error and the message was:
error error: no pg_hba.conf entry for host "3.86.36.125", user "mwtwuvkwtrqpir", database "d3d9tcukrk5fgh", SSL off
I used Strapi 3.2.5 , and I was deployed on Heroku Postgres with Plan free (Hobby).
I hope everyone helping me for this questions, and hope helping others for same case.
Thank you
We had the same issue on our Heroku instances and just recently found a fix.
Adding rejectUnauthorized to the database config appears to work.
config/database.js
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: env('DATABASE_HOST', 'localhost'),
port: env.int('DATABASE_PORT', 5432),
database: env('DATABASE_NAME', 'strapi'),
username: env('DATABASE_USERNAME', 'strapi'),
password: env('DATABASE_PASSWORD', 'strapi'),
schema: env('DATABASE_SCHEMA', 'public'), // Not Required
ssl: {
rejectUnauthorized: env.bool('DATABASE_SSL_SELF', false), // For self-signed certificates
},
},
options: {
ssl: env.bool('DATABASE_SSL', false),
},
},
},
});
I cannot take full credit however, it was this post on the Strapi forum that led me to the answer:
https://forum.strapi.io/t/error-no-pg-hba-conf-entry-for-host-ssl-off/3409
subsequently this link:
https://strapi.io/documentation/developer-docs/latest/setup-deployment-guides/configurations.html#database

How to add an SSL certificate (ca-cert) to node.js environment variables in order to connect to Digital Ocean Postgres Managed Database?

I am currently using node-postgres to create my pool. This is my current code:
const { Pool } = require('pg')
const pgPool = new Pool({
user: process.env.PGUSER,
password: process.env.PGPASSWORD,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
port: process.env.PGPORT,
ssl: {
rejectUnauthorized: true,
// Would like to add line below
// ca: process.env.CACERT,
},
})
I found another post where they read in the cert using 'fs' which can be seen below.
const config = {
database: 'database-name',
host: 'host-or-ip',
user: 'username',
password: 'password',
port: 1234,
// this object will be passed to the TLSSocket constructor
ssl: {
ca: fs.readFileSync('/path/to/digitalOcean/certificate.crt').toString()
}
}
I am unable to do that as I am using git to deploy my application. Specifically Digital Oceans new App Platform. I have attempted reaching out to them with no success. I would prefer not to commit my certificate in my source control. I see a lot of posts of people suggesting to set
ssl : { rejectUnauthorized: false}
That is not the approach I want to take. My code does work with that but I want it to be secure.
Any help is appreciated thanks.
Alright I finally was able to figure it out. I think the issue was multiline and just unfamiliarity with dotenv for my local developing environment.
I was able to get it all working with my code like this. It also worked with the fs.readFileSync() but I didn't want to commit that to my source control.
const { Pool } = require('pg')
const fs = require('fs')
const pgPool = new Pool({
user: process.env.PGUSER,
password: process.env.PGPASSWORD,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
port: process.env.PGPORT,
ssl: {
rejectUnauthorized: true,
// ca: fs.readFileSync(
// `${process.cwd()}/cert/ca-certificate.crt`.toString()
// ),
ca: process.env.CA_CERT,
},
})
.on('connect', () => {
console.log('connected to the database!')
})
.on('error', (err) => {
console.log('error connecting to database ', err)
})
Now in my config.env I had to make it look like this:
CA_CERT="-----BEGIN CERTIFICATE-----\nVALUES HERE WITH NO SPACES AND A \n
AFTER EACH LINE\n-----END CERTIFICATE-----"
I had to keep it as a single line string to have it work. But I was finally to connect with
{rejectUnauthorized:true}
For the digital ocean app platform environment variable, I copied everything including the double quotes and pasted it in there. Seems to work great. I do not think you will be able to have this setting set to true with their $7 development database though. I had to upgrade to the managed one in order to find any CA cert to download.

Strapi giving me DB errors in production, even though I'm using correct credentials

EDIT: I found a file at /config/database.js which is used to connect to sqlite in development. When I change the client name from sqlite to postgres, that's when the trouble starts.
Isn't strapi supposed to ignore files like this in production? How can I get strapi to ignore this file, and just use my postgres db?
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'sqlite',
filename: env('DATABASE_FILENAME', '.tmp/data.db'),
},
options: {
useNullAsDefault: true,
},
},
},
});
End edit.
I'm trying to get my strapi app to start up in production, but it keeps erroring out saying
[2020-07-22T01:15:40.246Z] debug ⛔️ Server wasn't able to start properly.
[2020-07-22T01:15:40.247Z] error error: password authentication failed for user "<redacted>"
The rest of the output is related to pg, which leads me to think that this is a DB connection error.
I can log into my db from the command line using psql -U postgres -W, which confirms that I know my password.
In addition, I'm using pm2 to run things, and instead of using process.env in that file, I just added the db variables directly, but that made no difference.
The application has been built in production mode. I have 3 dbs in pg, one called postgres, one with my apps name, and another called strapi.
Thanks
my /config/enviroronments/production.database.json looks like this
{
"defaultConnection": "default",
"connections": {
"default": {
"connector": "bookshelf",
"settings": {
"client": "postgres",
"host": "${process.env.DATABASE_HOST || '127.0.0.1'}",
"port": "${process.env.DATABASE_PORT || 27017}",
"database": "${process.env.DATABASE_NAME || 'strapi'}",
"username": "${process.env.DATABASE_USERNAME || ''}",
"password": "${process.env.DATABASE_PASSWORD || ''}"
},
"options": {
"ssl": false
}
}
}
}
and I have a .env file at the root of the backend app that looks like this
DATABASE_HOST=localhost
DATABASE_PORT=5432
DATABASE_NAME="<redacted - all letters>"
DATABASE_USERNAME="<redacted - all letters>"
DATABASE_PASSWORD="<redacted - all letters>"
Found the issue. When I created the app, I used sqlite as my db. As a result, the default database.js file wasn't set up in a way that could be overwritten with env variables.
I created a new local Strapi app with pgsql as my db, and copied the contents of the database.js file to my server. All working now.
New file for reference
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: env('DATABASE_HOST', '127.0.0.1'),
port: env.int('DATABASE_PORT', 5432),
database: env('DATABASE_NAME', 'my-strapi-project'),
username: env('DATABASE_USERNAME', 'testing'),
password: env('DATABASE_PASSWORD', 'testing'),
ssl: env.bool('DATABASE_SSL', false),
},
options: {}
},
},
});
I had the same situation in development. I created a strapi app with SQLite and decided to use PostgreSQL. That's where the trouble came in. So the fix was as follows:
app_name/config/database.js
module.exports = ({ env }) => ({
connection: {
client: 'postgres',
connection: {
host: env('DATABASE_HOST', '127.0.0.1'),
port: env.int('DATABASE_PORT', 5432),
database: env('DATABASE_NAME', 'db_name'),
user: env('DATABASE_USERNAME', 'postgres'),
password: env('DATABASE_PASSWORD', 'postgres'),
ssl: env.bool('DATABASE_SSL', false),
},
},
});
Your dependencies under app_name/package.json should be like
"dependencies": {
"#strapi/strapi": "4.1.8",
"#strapi/plugin-users-permissions": "4.1.8",
"#strapi/plugin-i18n": "4.1.8",
"pg": "8.6.0"
}
[2023-02-19 11:27:27.197] debug: ⛔️ Server wasn't able to start properly.
[2023-02-19 11:27:27.199] error: password authentication failed for user "root"
FIX==>
su - postgres
psql postgres
CREATE ROLE root SUPERUSER LOGIN PASSWORD 'password';
The point of interest here is the module used with strapi .
configuration file database.js
module.exports = ({ env }) => ({
defaultConnection: "default",
connection: {
client: "postgres",
connection: {
host: "127.0.0.1",
port: 5432,
database: "dbname",
username: "postgres",
password: "password",
ssl: false
},
debug: true,
useNullAsDefault: true
}
});
version package.json
"#_sh/strapi-plugin-ckeditor": "^2.0.3",
"#strapi/plugin-i18n": "4.6.1,",
"#strapi/plugin-users-permissions": "4.6.1,",
"#strapi/strapi": "4.6.1,",
"better-sqlite3": "8.0.1",
"pg": "8.6.0"
check version
/etc/postgresql/{{version-postsql}}/main/pg_hba.conf
local replication all peer
host replication all 127.0.0.1/32 md5
host replication all ::1/128 md5
host all postgres 127.0.0.1/32 trust
host all all ::1/128 trust
restart postgresql
sudo systemctl restart postgresql.service
su - postgres
psql
DROP root;
CREATE ROLE root WITH SUPERUSER CREATEDB CREATEROLE LOGIN ENCRYPTED PASSWOR 'password......';
CREATEDB dbname;
I don't know why it took the initial role of root but the above simple solution worked for me

I need to pass username and password to typeORM on runtime

On NodeJS (NestJS) I used TypeORM to connect to Database (Oracle). I would like to pass the username and password during runtime of the application.
Due to security reasons, in my company, the security is configured in away that for each user, we create a database schema. He log into The application using his DB credential. I know I t is not very popular practice in the industry.
const connection = await createConnection({
type: "oracle",
host: "localhost",
port: 3306,
username: "test", // this need to pass during run time
password: "test", // same thing for the password
database: "test"
});
Could you please share with me any reference/ hints on how to achieve this on typeORM and nodeJs/NestJs?
Thanks
You can use register TypeORM in the root level, and use a configuration service to provide the connection details:
TypeOrmModule.forRootAsync({
imports: [ConfigModule],
useFactory: (config: ConfigService) => ({
host: config.get('DB_HOST'),
username: config.get('DB_USER'),
password: config.get('DB_PASSWORD'),
database: config.get('DB_NAME'),
entities: [__dirname + '/**/*.entity{.ts,.js}'],
port: 3306,
type: 'oracle',
}),
inject: [ConfigService],
}),
ConfigService is just a simple service that implements a get function that provides the specific requested configuration.
More on that here: https://docs.nestjs.com/techniques/database#async-configuration

How to set up postgresql config with heroku and nodejs?

This is my first time try to host nodeJS application - built with hapi.js, typeorm and postgresql - on heroku. I've create two apps on heroku - for "staging" (server-staging) and "production" (server-prod) - that using same code but will use different configuration. Why different configuration? because each application on heroku will use different postgres credential, as it's attached as an add-ons.
objective
My objective/main question is How and where I have to set the database config for my application?
I use .env file (which I ignore in .gitignore - I don't want to put the credential in my repo) to connect the application to my local database. Here is how the .env looks like:
HOST=localhost
PORT=3001
TYPEORM_CONNECTION=postgres
TYPEORM_HOST=localhost
TYPEORM_USERNAME=postgres
TYPEORM_PASSWORD=password
TYPEORM_DATABASE=database
TYPEORM_PORT=5432
TYPEORM_SYNCHRONIZE=true
TYPEORM_LOGGING=false
In the application, I never do/write code such process.env.TYPEORM_USERNAME since its done by the typeorm node_modules. What I do to start the connection is by doing this:
const server = new Hapi.Server({
port: process.env.PORT,
host: process.env.HOST,
routes: {
cors: Cors,
},
});
await server.register(Plugins);
server.route(Router);
await createConnection();
await server.start();
And my application automatically connected to the specified database as defined in the .env. Now, in heroku, the credential is lies here:
All information lies there, but, [Q1] I don't know how to tell my application (of course, without store the credential in my code/repo) that I have to use the config as defined in above picture? Also, as stated in above image, "Heroku rotates credentials periodically and updates applications where this database is attached.". Does it means the credentials will changed periodically? [Q2] If yes, is there any way to make my application auto recognise the new credential?
Sorry if my explanation make confused. If you did not understand what I am trying to achieve, please ask things that you don't understand, so I can fix/update my question to make it understandable.
Anyway, I found this example first-example and second-example. But, they are using process.env.DATABASE_URL, which contain credential. I think, it means that they not ignore their .env file in their repo?
*) Note: Q1 means Question 1, and so for the rest
Write a ormconfig.js file in the root of your repo. This way you can access the environment variables like the url provided from heroku and you don't have credentials in your repo.
require('dotenv').config();
module.exports = [
{
name: 'default',
type: 'postgres',
url: process.env.DATABASE_URL,
synchronize: false,
logging: false,
entities: ['dist/entities/*.*'],
migrations: ['dist/database/migrations/**/*.js'],
subscribers: ['dist/database/subscribers/**/*.js'],
cli: {
entitiesDir: 'dist/entities',
migrationsDir: 'dist/database/migrations',
subscribersDir: 'dist/database/subscribers',
},
},
{
name: 'development',
type: 'postgres',
host: process.env.POSTGRES_HOST,
port: process.env.POSTGRES_PORT,
username: process.env.POSTGRES_USER,
password: process.env.POSTGRES_PASSWORD,
database: process.env.POSTGRES_DB,
synchronize: true,
logging: true,
entities: ['src/entities/*.*'],
migrations: ['src/database/migrations/**/*.ts'],
subscribers: ['src/database/subscribers/**/*.ts'],
cli: {
entitiesDir: 'src/entities',
migrationsDir: 'src/database/migrations',
subscribersDir: 'src/database/subscribers',
},
},
];
With this configuration you can then get a specific configuration in javascript/typescript:
let connectionOptions: ConnectionOptions;
if(process.env.NODE_ENV ==='development') {
connectionOptions = await getConnectionOptions("development");
} else {
connectionOptions = await getConnectionOptions("default");
}
await createConnection(connectionOptions);

Resources