Defining Sequelize on google cloud sql nodejs - node.js

Hi i having an issue connecting to Google Cloud SQL from GAE.
My app is running inside docker container here is the docker file
FROM node:8.10.0-alpine
ARG NODE_ENV=production
ENV NODE_ENV=$NODE_ENV
# env like sql user db instance connection name
# Set a working directory
WORKDIR /usr/src/app
COPY ./build/package.json .
COPY ./build/yarn.lock .
# Install Node.js dependencies
RUN yarn install --production --no-progress
# Copy application files
COPY ./build .
COPY ./src/db/seeders ./seeds
COPY ./src/db/migrations ./migrations
COPY ./scripts ./scripts
RUN yarn run db:seed # -> failed to run this line
RUN yarn run db:migrate
# Run the container under "node" user by default
USER node
CMD [ "node", "server.js" ]
to connect to the db i'm using Sequealize this is my connection config
module.exports = {
production: {
dialect: 'postgres',
seederStorage: 'sequelize',
seederStorageTableName: 'sequelize_seeder',
username: process.env.SQL_USER,
password: process.env.SQL_PASSWORD,
database: process.env.SQL_DATABASE,
host: `/cloudsql/${process.env.INSTANCE_CONNECTION_NAME}`,
logging: true,
dialectOptions: {
socketPath: `/cloudsql/${process.env.INSTANCE_CONNECTION_NAME}`,
supportBigNumbers: true,
bigNumberStrings: true,
ssl: false,
},
pool: {
max: 5,
idle: 30000,
acquire: 60000,
},
operatorsAliases: false,
define: {
freezeTableName: true,
},
},
};
I tried almost everything from setting the host to localhost/127.0.0.1
While doing so i'm getting SequelizeConnectionRefusedError: connect ECONNREFUSED 127.0.0.1:5432
If i'm setting host: host:/cloudsql/${process.env.INSTANCE_CONNECTION_NAME}``
I'm getting a different error
SequelizeConnectionError: connect ENOENT {MY_INSTANCE_CONNECTION_NAME}.s.PGSQL.5432
my app.yaml file
env: flex
runtime: custom
env_variables:
#env db user etc..
beta_settings:
cloud_sql_instances: MY_INSTANCE_CONNECTION_NAME
I tried to log in with knex and I managed to connect so i assuming something wrong with my configuration

Spent whole day today trying to connect from Google App Engine app to Google Cloud SQL (PostreSQL) when deploying through Bitbucket pipelines.
Here is the configs that worked for me (may be they will save someone few hours of life):
const sequelize = new Sequelize(DB_NAME, USERNAME, PASSWORD, {
dialect: 'postgres',
// e.g. host: '/cloudsql/my-awesome-project:us-central1:my-cloud-sql-instance'
host: '/cloudsql/${INSTANCE_CONNECTION_NAME}',
pool: {
max: 5,
min: 0,
acquire: 30000,
idle: 10000
},
dialectOptions: {
// e.g. socketPath: '/cloudsql/my-awesome-project:us-central1:my-cloud-sql-instance'
// same as host string above
socketPath: '/cloudsql/${INSTANCE_CONNECTION_NAME}'
},
logging: false,
operatorsAliases: false
});
app.yaml file:
runtime: nodejs
env: flex
# make sure to include code below
beta_settings:
cloud_sql_instances: my-awesome-project:us-central1:my-cloud-sql-instance
In my case when I did not provide host it failed as well all other variants with connections string when connecting to the Google Cloud SQL.
Cheers!

It looks like something is wrong with the params passed to sequlize.
Try using a simple connection string
var conString = "postgres://UserName:Password#Host:5432/YourDatabase";

Related

Nodejs mssql server connection throws an error

I want to connect my mssql database from nodejs.
I am using sequelize and tedious module for this.
My connection configs like this
const sequelize = new Sequelize(
"mydb",
"username", "password", {
host:config_data.host,
"dialect":"mssql",
"port":1433
dialectOptions: {
instanceName: "MSSQLSERVER"
},
},
);
When I tried to run script, it throws an error.
(node:14584) UnhandledPromiseRejectionWarning: SequelizeConnectionError: Failed to connect to 192.168.10.220:1433 - Could not connect (sequence)
I apply all steps in configuration manager.
Enabled TCP/IP
Started Server SQL browser
Added 1433 port to Firewall
There is also additional error when host is "localhost".(Currently writed IP address)
(node:13552) UnhandledPromiseRejectionWarning: SequelizeConnectionError: Failed to connect to localhost:1433
- 4292:error:1425F102:SSL routines:ssl_choose_client_version:unsupported protocol:c:\ws\deps\openssl\openssl\ssl\statem\statem_lib.c:1947:
I need to help, is there someone have any idea ?
You need to set encrypt to false in your Sequelize inside options.
const sequelize = new Sequelize(
"mydb",
"username", "password", {
host:config_data.host,
"dialect":"mssql",
"port":1433,
"options": {
encrypt: false,
enableArithAbort: false
},
dialectOptions: {
instanceName: "MSSQLSERVER"
},
},
);
The issue is because of TLS protocol mismatch between Source & Destination server. In my case my App server was Ubuntu 20.04 & my SQL Server(Express 2012) was on Windows server. I also tried to downgrade TLS protocols on Ubuntu but nothing worked. So finally I disabled tls encryption in Sequelize.
Make sure options are given as expected by Sequelize
const sequelize = new Sequelize(dbConfig.DB, dbConfig.USER, dbConfig.PASSWORD, {
host: dbConfig.HOST,
port: dbConfig.PORT,
dialect: dbConfig.dialect,
pool: {
max: dbConfig.pool.max,
min: dbConfig.pool.min,
acquire: dbConfig.pool.acquire,
idle: dbConfig.pool.idle,
},
// The below options are important to supress ssl issue on
// AWS EC2 ubuntu when db server is on windows. There is TLS protocol issue
// Which by using these options we disable tls encryption
dialectOptions: {
// Observe the need for this nested `options` field for MSSQL
options: {
encrypt: false,
enableArithAbort: false
}
}
});

Strapi giving me DB errors in production, even though I'm using correct credentials

EDIT: I found a file at /config/database.js which is used to connect to sqlite in development. When I change the client name from sqlite to postgres, that's when the trouble starts.
Isn't strapi supposed to ignore files like this in production? How can I get strapi to ignore this file, and just use my postgres db?
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'sqlite',
filename: env('DATABASE_FILENAME', '.tmp/data.db'),
},
options: {
useNullAsDefault: true,
},
},
},
});
End edit.
I'm trying to get my strapi app to start up in production, but it keeps erroring out saying
[2020-07-22T01:15:40.246Z] debug ⛔️ Server wasn't able to start properly.
[2020-07-22T01:15:40.247Z] error error: password authentication failed for user "<redacted>"
The rest of the output is related to pg, which leads me to think that this is a DB connection error.
I can log into my db from the command line using psql -U postgres -W, which confirms that I know my password.
In addition, I'm using pm2 to run things, and instead of using process.env in that file, I just added the db variables directly, but that made no difference.
The application has been built in production mode. I have 3 dbs in pg, one called postgres, one with my apps name, and another called strapi.
Thanks
my /config/enviroronments/production.database.json looks like this
{
"defaultConnection": "default",
"connections": {
"default": {
"connector": "bookshelf",
"settings": {
"client": "postgres",
"host": "${process.env.DATABASE_HOST || '127.0.0.1'}",
"port": "${process.env.DATABASE_PORT || 27017}",
"database": "${process.env.DATABASE_NAME || 'strapi'}",
"username": "${process.env.DATABASE_USERNAME || ''}",
"password": "${process.env.DATABASE_PASSWORD || ''}"
},
"options": {
"ssl": false
}
}
}
}
and I have a .env file at the root of the backend app that looks like this
DATABASE_HOST=localhost
DATABASE_PORT=5432
DATABASE_NAME="<redacted - all letters>"
DATABASE_USERNAME="<redacted - all letters>"
DATABASE_PASSWORD="<redacted - all letters>"
Found the issue. When I created the app, I used sqlite as my db. As a result, the default database.js file wasn't set up in a way that could be overwritten with env variables.
I created a new local Strapi app with pgsql as my db, and copied the contents of the database.js file to my server. All working now.
New file for reference
module.exports = ({ env }) => ({
defaultConnection: 'default',
connections: {
default: {
connector: 'bookshelf',
settings: {
client: 'postgres',
host: env('DATABASE_HOST', '127.0.0.1'),
port: env.int('DATABASE_PORT', 5432),
database: env('DATABASE_NAME', 'my-strapi-project'),
username: env('DATABASE_USERNAME', 'testing'),
password: env('DATABASE_PASSWORD', 'testing'),
ssl: env.bool('DATABASE_SSL', false),
},
options: {}
},
},
});
I had the same situation in development. I created a strapi app with SQLite and decided to use PostgreSQL. That's where the trouble came in. So the fix was as follows:
app_name/config/database.js
module.exports = ({ env }) => ({
connection: {
client: 'postgres',
connection: {
host: env('DATABASE_HOST', '127.0.0.1'),
port: env.int('DATABASE_PORT', 5432),
database: env('DATABASE_NAME', 'db_name'),
user: env('DATABASE_USERNAME', 'postgres'),
password: env('DATABASE_PASSWORD', 'postgres'),
ssl: env.bool('DATABASE_SSL', false),
},
},
});
Your dependencies under app_name/package.json should be like
"dependencies": {
"#strapi/strapi": "4.1.8",
"#strapi/plugin-users-permissions": "4.1.8",
"#strapi/plugin-i18n": "4.1.8",
"pg": "8.6.0"
}
[2023-02-19 11:27:27.197] debug: ⛔️ Server wasn't able to start properly.
[2023-02-19 11:27:27.199] error: password authentication failed for user "root"
FIX==>
su - postgres
psql postgres
CREATE ROLE root SUPERUSER LOGIN PASSWORD 'password';
The point of interest here is the module used with strapi .
configuration file database.js
module.exports = ({ env }) => ({
defaultConnection: "default",
connection: {
client: "postgres",
connection: {
host: "127.0.0.1",
port: 5432,
database: "dbname",
username: "postgres",
password: "password",
ssl: false
},
debug: true,
useNullAsDefault: true
}
});
version package.json
"#_sh/strapi-plugin-ckeditor": "^2.0.3",
"#strapi/plugin-i18n": "4.6.1,",
"#strapi/plugin-users-permissions": "4.6.1,",
"#strapi/strapi": "4.6.1,",
"better-sqlite3": "8.0.1",
"pg": "8.6.0"
check version
/etc/postgresql/{{version-postsql}}/main/pg_hba.conf
local replication all peer
host replication all 127.0.0.1/32 md5
host replication all ::1/128 md5
host all postgres 127.0.0.1/32 trust
host all all ::1/128 trust
restart postgresql
sudo systemctl restart postgresql.service
su - postgres
psql
DROP root;
CREATE ROLE root WITH SUPERUSER CREATEDB CREATEROLE LOGIN ENCRYPTED PASSWOR 'password......';
CREATEDB dbname;
I don't know why it took the initial role of root but the above simple solution worked for me

Knex Heroku server connection issues to postgres db - Node.js

I am trying to run my node-express server with GraphQL and Knex and connect it up to the PostgresQL database in heroku.
When I run the heroku bash CLI and attempt to migrate I get this error
~ $ npm run migrate
> syncify-server#0.0.0 migrate /app
> knex migrate:latest
Using environment: staging
Error: connect ECONNREFUSED 127.0.0.1:5432
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! syncify-server#0.0.0 migrate: `knex migrate:latest`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the syncify-server#0.0.0 migrate script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /app/.npm/_logs/2020-06-21T00_20_17_455Z-debug.log
It is working fine locally in development. My knex.js file
import dotenv from 'dotenv'
import knex from 'knex'
import mockKnex from 'mock-knex'
dotenv.config()
let knexConnection
if (process.env.NODE_ENV === 'test') {
knexConnection = knex({
client: 'pg',
debug: false,
})
mockKnex.mock(knexConnection)
} else {
knexConnection = knex({
client: 'pg',
connection: {
url: process.env.DATABASE_URL,
type: 'postgres',
charset: 'utf8',
ssl: false
},
})
}
export default knexConnection
and knexfile.js
require('dotenv').config()
module.exports = {
development: {
client: 'pg',
connection: {
url: process.env.DATABASE_URL,
charset: 'utf8',
},
},
staging: {
client: 'pg',
connection: {
url: process.env.DATABASE_URL,
charset: 'utf8',
},
pool: {
min: 2,
max: 10,
},
migrations: {
tableName: 'knex_migrations',
},
},
production: {
client: 'pg',
connection: {
url: process.env.DATABASE_URL,
charset: 'utf8',
},
pool: {
min: 2,
max: 10,
},
migrations: {
tableName: 'knex_migrations',
},
},
}
I've been trying to isolate the problem but feel like I might have more than one.
In config vars I have theDATABASE_URL as the heroku psql db URL and NODE_ENV as "staging" as well as all the auth0 settings.
I am able to access the online database using psql in the command line. I have the correct tables and can create and retrieve data using SQL statements.
When I configure my local server to use the heroku psql db I get the error message
relation "users" does not exist
The same if I try other tables in the database.
I've tried changing my SSL to true which threw errors and to false which didn't seem to harm anything. (I've tried many other things)
If I hit the online heroku server it just throw a generic error without any details.
Source code here
After losing days to this I've fixed it.
Classic env variable errors.
I hadn't included AUTH0_ISSUER in my config which was implemented in an update a while ago but after the first deployment to heroku.
Also despite configuring my knex files to use DATABASE_URL it was not picking it up and only connects to the database if I used all five individiual settings.
DB_NAME=
DB_USER=
DB_PASSWORD=
DB_HOST=
DB_PORT=
I couldn't work out why exactly but I have server.js files in './dist' which have these referenced in minified code so I guess it must but somehow transpiled already to use that and I a bit stuck.
Would prefer to use DATABASE_URL as I think heroku dynamically updates this value from time to time and not sure how to protect my server from that. 😬

How to set up postgresql config with heroku and nodejs?

This is my first time try to host nodeJS application - built with hapi.js, typeorm and postgresql - on heroku. I've create two apps on heroku - for "staging" (server-staging) and "production" (server-prod) - that using same code but will use different configuration. Why different configuration? because each application on heroku will use different postgres credential, as it's attached as an add-ons.
objective
My objective/main question is How and where I have to set the database config for my application?
I use .env file (which I ignore in .gitignore - I don't want to put the credential in my repo) to connect the application to my local database. Here is how the .env looks like:
HOST=localhost
PORT=3001
TYPEORM_CONNECTION=postgres
TYPEORM_HOST=localhost
TYPEORM_USERNAME=postgres
TYPEORM_PASSWORD=password
TYPEORM_DATABASE=database
TYPEORM_PORT=5432
TYPEORM_SYNCHRONIZE=true
TYPEORM_LOGGING=false
In the application, I never do/write code such process.env.TYPEORM_USERNAME since its done by the typeorm node_modules. What I do to start the connection is by doing this:
const server = new Hapi.Server({
port: process.env.PORT,
host: process.env.HOST,
routes: {
cors: Cors,
},
});
await server.register(Plugins);
server.route(Router);
await createConnection();
await server.start();
And my application automatically connected to the specified database as defined in the .env. Now, in heroku, the credential is lies here:
All information lies there, but, [Q1] I don't know how to tell my application (of course, without store the credential in my code/repo) that I have to use the config as defined in above picture? Also, as stated in above image, "Heroku rotates credentials periodically and updates applications where this database is attached.". Does it means the credentials will changed periodically? [Q2] If yes, is there any way to make my application auto recognise the new credential?
Sorry if my explanation make confused. If you did not understand what I am trying to achieve, please ask things that you don't understand, so I can fix/update my question to make it understandable.
Anyway, I found this example first-example and second-example. But, they are using process.env.DATABASE_URL, which contain credential. I think, it means that they not ignore their .env file in their repo?
*) Note: Q1 means Question 1, and so for the rest
Write a ormconfig.js file in the root of your repo. This way you can access the environment variables like the url provided from heroku and you don't have credentials in your repo.
require('dotenv').config();
module.exports = [
{
name: 'default',
type: 'postgres',
url: process.env.DATABASE_URL,
synchronize: false,
logging: false,
entities: ['dist/entities/*.*'],
migrations: ['dist/database/migrations/**/*.js'],
subscribers: ['dist/database/subscribers/**/*.js'],
cli: {
entitiesDir: 'dist/entities',
migrationsDir: 'dist/database/migrations',
subscribersDir: 'dist/database/subscribers',
},
},
{
name: 'development',
type: 'postgres',
host: process.env.POSTGRES_HOST,
port: process.env.POSTGRES_PORT,
username: process.env.POSTGRES_USER,
password: process.env.POSTGRES_PASSWORD,
database: process.env.POSTGRES_DB,
synchronize: true,
logging: true,
entities: ['src/entities/*.*'],
migrations: ['src/database/migrations/**/*.ts'],
subscribers: ['src/database/subscribers/**/*.ts'],
cli: {
entitiesDir: 'src/entities',
migrationsDir: 'src/database/migrations',
subscribersDir: 'src/database/subscribers',
},
},
];
With this configuration you can then get a specific configuration in javascript/typescript:
let connectionOptions: ConnectionOptions;
if(process.env.NODE_ENV ==='development') {
connectionOptions = await getConnectionOptions("development");
} else {
connectionOptions = await getConnectionOptions("default");
}
await createConnection(connectionOptions);

Node Sequelize (MSSQL) - Login failed for user ''

I've come across several posts for this question however, none of them seem to have an actual answer. Several ideas, yet none of them work.
After digging around both the Sequelize and Tedious packages and watching my config get passed down correctly, I'm at a loss.
I am trying to run migrations against a new database in MSSQL. I have no problem connecting to it with the same creds I'm using here so I know that's not the issue.
I have my config.js that is pulling env vars. With the exception of my custom console statements, this file was auto generated from sequelize and is correctly referenced in my sequelizerc
require('dotenv').config()
console.log('[+] Loading database config...')
if (process.env.NODE_ENV === 'production') {
console.log(`[+] Using database: ${process.env.PROD_DB_DATABASE}`)
} else if (process.env.NODE_ENV === 'development') {
console.log(`[+] Using database: ${process.env.DEV_DB_DATABASE}`)
} else if (process.env.NODE_ENV === 'test') {
console.log(`[+] Using database: ${process.env.TEST_DB_DATABASE}`)
} else if (process.env.NODE_ENV === 'local') {
console.log(`[+] Using database: ${process.env.LOCAL_DB_DATABASE}`)
} else {
console.log(`[-] CANNOT LOAD DATABASE FROM ENV: ${process.env.NODE_ENV}`)
process.exit()
}
module.exports = {
production: {
database: process.env.PROD_DB_DATABASE,
username: process.env.PROD_DB_USERNAME,
password: process.env.PROD_DB_PASSWORD,
host: process.env.PROD_DB_HOST,
port: process.env.PROD_DB_PORT,
dialect: process.env.PROD_DB_DIALECT,
storage: process.env.PROD_DB_STORAGE,
logging: false,
dialectOptions: {
instanceName: process.env.PROD_INSTANCE_NAME
},
pool: {
min: 5,
max: 1,
acquire: 6000,
idle: 6000
}
},
development: {
database: process.env.DEV_DB_DATABASE,
username: process.env.DEV_DB_USERNAME,
password: process.env.DEV_DB_PASSWORD,
host: process.env.DEV_DB_HOST,
port: process.env.DEV_DB_PORT,
dialect: process.env.DEV_DB_DIALECT,
storage: process.env.DEV_DB_STORAGE,
logging: console.log,
dialectOptions: {
instanceName: process.env.DEV_INSTANCE_NAME,
debug: true
},
pool: {
min: 5,
max: 1,
acquire: 6000,
idle: 6000
}
},
test: {
database: process.env.TEST_DB_DATABASE,
username: process.env.TEST_DB_USERNAME,
password: process.env.TEST_DB_PASSWORD,
host: process.env.TEST_DB_HOST,
port: process.env.TEST_DB_PORT,
dialect: process.env.TEST_DB_DIALECT,
storage: process.env.TEST_DB_STORAGE,
logging: false
},
local: {
database: process.env.LOCAL_DB_DATABASE,
username: process.env.LOCAL_DB_USERNAME,
password: process.env.LOCAL_DB_PASSWORD,
host: process.env.LOCAL_DB_HOST,
port: process.env.LOCAL_DB_PORT,
dialect: process.env.LOCAL_DB_DIALECT,
storage: process.env.LOCAL_DB_STORAGE,
logging: false
}
}
When i run my migration i get the error:
> node_modules/.bin/sequelize db:migrate
// ERROR: Login failed for user ''.
As mentioned above I dug through sequelize and tedious and my config is getting passed properly through both so i know it's not an env var issue or a NODE_ENV issue.
Anyone have any ideas here? I'm about to smash my face into my keyboard.
More for older versions:
If you are using sequelize#4, then it seems there is a hidden requirement that you must use tedious#<=5.
Which version of Sequelize are you using? If it's v5,
According to Sequelize v5's document:
Sequelize now works with tedious >= 6.0.0
However, in its package.json, it does not depend on tedious at all.
Since your program still runs, I guess you manually installed an older version of tedious before, which caused this strange problem.
Manually installing tedious of version>=6 should solve this problem, just like stated in its Getting started document page:
You'll also have to manually install the driver for your database of choice:
# One of the following:
$ npm install --save pg pg-hstore # Postgres
$ npm install --save mysql2
$ npm install --save mariadb
$ npm install --save sqlite3
$ npm install --save tedious # Microsoft SQL Server
const Sequelize = require('sequelize');
const sequelize = new Sequelize(
process.env.DB_NAME, process.env.DB_USER, process.env.DB_PASS, {
dialect: 'mssql',
host: process.env.DB_HOST, //This is an IP
dialectOptions: {
options: {
instanceName: process.env.DB_INSTANCE_NAME,
trustServerCertificate: true
},
}
}
);
module.exports = {
sequelize,
Sequelize
};
Here is another solution, it's working for me.
I was getting the same error. The reason was due to explicitly mentioning the name of the DB in the sequelize config file and it did not exist. The reason could be different in your case but a quick look at SQL Server error logs will give you the reason for the failure.
Login failed for user 'user'. Reason: Failed to open the explicitly specified database 'dbo'. [CLIENT: XX.XX.XX.XX]

Resources