I am trying to connect to an amazon postgreSQL RDS using a NodeJS lambda.
The lambda is in the same VPC as the RDS instance and as far as I can tell the security groups are set up to give the lambda access to the RDS. The lambda is called through API gateway and I'm using knex js as a query builder. When the lambda attempts to connect to the database it throws an "unable to get local issuer certificate" error, but the connection parameters are what I expect them to be.
I know this connection is possible as I've already implemented it in a different environment, without receiving the certificate issue. I've compared the two environments but cannot find any immediate differences.
The connection code looks like this:
import AWS from 'aws-sdk';
import { types } from 'pg';
import { Moment } from 'moment';
import knex from 'knex';
const TIMESTAMP_OID = 1114;
// Example value string: "2018-10-04 12:30:21.199"
types.setTypeParser(TIMESTAMP_OID, (value) => value && new Date(`${value}+00`));
export default class Database {
/**
* Gets the connection information through AWS Secrets Manager
*/
static getConnection = async () => {
const client = new AWS.SecretsManager({
region: '<region>',
});
if (process.env.databaseSecret == null) {
throw 'Database secret not defined';
}
const response = await client
.getSecretValue({ SecretId: process.env.databaseSecret })
.promise();
if (response.SecretString == undefined) {
throw 'Cannot find secret string';
}
return JSON.parse(response.SecretString);
};
static knexConnection = knex({
client: 'postgres',
connection: async () => {
const secret = await Database.getConnection();
return {
host: secret.host,
port: secret.port,
user: secret.username,
password: secret.password,
database: secret.dbname,
ssl: true,
};
},
});
}
Any guidance on how to solve this issue or even where to start looking would be greatly appreciated.
First of all, it is not a good idea to bypass ssl verification, and doing so can make you vulnerable to various exploits and skips a critical step in the TLS handshake.
What you can do is programmatically download the ca certificate chain bundle from Amazon and place it in the root directory of the lambda along side the handler.
wget https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem -P path/to/handler
Note: you can do this in your buildspec.yaml or in your script that packages the zip file that gets uploaded to aws
Then set the ssl configuration option to the contents of the pem file in your code postgres client configuration, like this:
let pgClient = new postgres.Client({
user: 'postgres',
host: 'rds-cluster.cluster-abc.us-west-2.rds.amazonaws.com',
database: 'mydatabase',
password: 'postgres',
port: 5432,
ssl: {
ca: fs.readFileSync(path.resolve('rds-combined-ca-bundle.pem'), "utf-8")
}
})
I know this is old, but just ran into this today. Running with node 10 and an older version of the pg library worked just fine. Updating to node 16 with pg version 8.x caused this error (simplified):
UNABLE_TO_GET_ISSUER_CERT_LOCALLY
In the past, you could indeed just set the ssl parameter to true or 'true' and it would work with the default AWS RDS certificate. Now, it seems we need to at least tell node/pg to ignore the cert verification (since it's self generated).
Using ssl: 'no-verify' works, enabling ssl and telling pg to ignore the verification of the cert chain.
source
UPDATE
For clarity, here's what the connection string would look like. With Knex, the same client info is passed to pg, so it should look similar to a pg client connection.
static knexConnection = knex({
client: 'postgres',
connection: async () => {
const secret = await Database.getConnection();
return {
host: secret.host,
port: secret.port,
user: secret.username,
password: secret.password,
database: secret.dbname,
ssl: 'no-verify',
};
}
Related
When I run a firebase functions locally on emulator, it works expectedly on Windows, but the same code doesn't work on Mac environment(I tried on two Mac: M1 and Intel, and neither doesn't work).
What I want to do is axios.post() to a specific API endpoint with:
query strings as authorization keys
accessing through squid proxy(SSL enabled) to make source IP address fixed
My firebase functions code here:
import functions from 'firebase-functions'
import https from 'https'
import axios from 'axios'
import {createHash} from 'crypto'
export default functions.region('asia-northeast1').https.onRequest(async (req, res) => {
const sharedKey = "abcdefghijklmn"
const randomKey = "opqrstuvwxyz"
const sha256Hex = createHash("sha256").update(sharedKey+randomKey).digest("hex")
const base64Encoded = Buffer.from(sha256Hex).toString('base64')
await axios.post(
`https://my.target.api.com`,
{},
{
params: {
auth_cd: base64Encoded,
auth_key: randomKey,
companyId: "myCompany"
},
// Below is just to avoid axios's error "Hostname/IP does not match certificate's
httpsAgent: new https.Agent({
rejectUnauthorized: false,
}),
proxy: {
protocol: "https",
host: "my.squid.proxy.com",
port: 8080,
auth: {
username: "username",
password: "password"
}
}
}
)
.then(result => {
console.log(result)
}).catch(error => {
console.log(error);
})
res.status(200).end()
})
And this gives me an error "ERROR_LOGIN_AUTH_FAILED" on Mac, which is reproducible on Windows too in condition that one or more of the three params (auth_cd/auth_key/companyId) are wrong.
Also, I made development environment same both on Mac and Windows:
Node.js: 14.19.3 (actually, here is another problem that if it's v16, the code above doesn't work on Windows too...)
Java(firebase-functions uses): 11.0.16
all the npm packages versions
axios/firebase-functions/firebase-tools are the latest
The other codes are the same too (cloned freshly from remote git repo)
I suspect this must be triggered some sort of difference on internal handling (like char code?) in axios, but no clues in my hand...
Any advice would be appreciated.
i'm having some trouble connecting node to the database, it keeps throwing me an error of ssl and i tried a lot of different videos and stuff to see if it works but nothing does, here is what i'm currently doing
import sql from 'mssql'
const dbSettings = {
user: 'admin',
password: 'system',
server: 'localhost',
database: 'master',
options: {
trustedConnection: true,
encrypt: true,
trustServerCertificate: true,
},
}
async function getConnection() {
const pool = sql.connect(dbSettings)
const result = await sql.query("SELECT 1")
console.log(result)
}
getConnection()
i also tried this as well but didn't work either
async function getConnection() {
const pool = await sql.connect(dbSettings)
const result = await pool.request().query("SELECT 1")
console.log(result)
i also checked if the SQL Server authentication is enabled with windows and SQL Server and it is, i can log in into SQL Server with that info, but somehow is having trouble creating the connection, by the way, this is the error message it is showing me:
node_modules\mssql\lib\tedious\connection-pool.js:70
err = new ConnectionError(err)
^
ConnectionError: Failed to connect to localhost:1433 - 186B0000:error:0A000102:SSL routines:ssl_choose_client_version:unsupported protocol:c:\ws\deps\openssl\openssl\ssl\statem\statem_lib.c:1986
any tips or solution you can give me to solve this problem would be really helpful to me, thank you very much in advance.
EDIT
I noticed that the connection error only appears when i call the function getConnection if i remove it it doesn't appear, however i need to make sure that the connection was properly established and see the response from the database to move on
change encrypt: true to encrypt: false
I am currently using node-postgres to create my pool. This is my current code:
const { Pool } = require('pg')
const pgPool = new Pool({
user: process.env.PGUSER,
password: process.env.PGPASSWORD,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
port: process.env.PGPORT,
ssl: {
rejectUnauthorized: true,
// Would like to add line below
// ca: process.env.CACERT,
},
})
I found another post where they read in the cert using 'fs' which can be seen below.
const config = {
database: 'database-name',
host: 'host-or-ip',
user: 'username',
password: 'password',
port: 1234,
// this object will be passed to the TLSSocket constructor
ssl: {
ca: fs.readFileSync('/path/to/digitalOcean/certificate.crt').toString()
}
}
I am unable to do that as I am using git to deploy my application. Specifically Digital Oceans new App Platform. I have attempted reaching out to them with no success. I would prefer not to commit my certificate in my source control. I see a lot of posts of people suggesting to set
ssl : { rejectUnauthorized: false}
That is not the approach I want to take. My code does work with that but I want it to be secure.
Any help is appreciated thanks.
Alright I finally was able to figure it out. I think the issue was multiline and just unfamiliarity with dotenv for my local developing environment.
I was able to get it all working with my code like this. It also worked with the fs.readFileSync() but I didn't want to commit that to my source control.
const { Pool } = require('pg')
const fs = require('fs')
const pgPool = new Pool({
user: process.env.PGUSER,
password: process.env.PGPASSWORD,
host: process.env.PGHOST,
database: process.env.PGDATABASE,
port: process.env.PGPORT,
ssl: {
rejectUnauthorized: true,
// ca: fs.readFileSync(
// `${process.cwd()}/cert/ca-certificate.crt`.toString()
// ),
ca: process.env.CA_CERT,
},
})
.on('connect', () => {
console.log('connected to the database!')
})
.on('error', (err) => {
console.log('error connecting to database ', err)
})
Now in my config.env I had to make it look like this:
CA_CERT="-----BEGIN CERTIFICATE-----\nVALUES HERE WITH NO SPACES AND A \n
AFTER EACH LINE\n-----END CERTIFICATE-----"
I had to keep it as a single line string to have it work. But I was finally to connect with
{rejectUnauthorized:true}
For the digital ocean app platform environment variable, I copied everything including the double quotes and pasted it in there. Seems to work great. I do not think you will be able to have this setting set to true with their $7 development database though. I had to upgrade to the managed one in order to find any CA cert to download.
I dont seem to be able to connect to Heroku Redis using TLS on Node.
These docs arent really much help: https://devcenter.heroku.com/articles/securing-heroku-redis
Does anyone have a working example? Should I be using REDIS_URL or REDIS_TLS_URL?
Im using node_redis v3
I found the Redis 6 add-on by Heroku generated an Error: self signed certificate in certificate chain error when when connecting to REDIS_URL without any parameters with ioredis on Node. You can avoid this error by passing in TLS options with rejectUnauthorized set to false.
The rejectUnauthorized of false allows for self-signed certificates, which would be an issue if concerned about MITM attacks. See TLS options for more background.
This is working for me with the latest ioredis package with rediss:// and redis:// URL's...
const REDIS_URL = process.env.REDIS_URL;
const redis_uri = url.parse(REDIS_URL);
const redisOptions = REDIS_URL.includes("rediss://")
? {
port: Number(redis_uri.port),
host: redis_uri.hostname,
password: redis_uri.auth.split(":")[1],
db: 0,
tls: {
rejectUnauthorized: false,
},
}
: REDIS_URL;
const redis = new Redis(redisOptions);
Here's my approach. It's easier to pass URL and TLS options separately.
const redisUrl = process.env.REDIS_TLS_URL ? process.env.REDIS_TLS_URL : process.env.REDIS_URL;
const redisDefaults = {
tls: {
// Heroku uses self-signed certificate, which will cause error in connection, unless check is disabled
rejectUnauthorized: false,
},
};
const defaultClient = redis.createClient(redisUrl, redisDefaults);
If you have test env running with hobby version, the TLS is URL is set in REDIS_TLS_URL, while production normally runs with premium and the env is REDIS_URL. So, to be compatible with the both, I first look for REDIS_TLS_URL and after that REDIS_URL to support both test and prod env.
For devs using node-redis, you'll need to set TLS to true when initializing your client.
redis.createClient({
url: REDIS_URL,
socket: {
tls: true,
rejectUnauthorized: false,
},
}
I don't know why you can't connect to this Redis Add-on unfortunately.
In the event you want to test on another Add-On, I have developped a Redis Add-On that is in the "alpha" phrase (free) on Heroku. I'll be able to provide you some support if you can't connect to it.
If you are interested, give me your Heroku email in private and I'll send you an invitation :)
For people using Bull, this implementation worked for me. Thanks #Tom McLellan.
const Queue = require('bull');
const redisUrlParse = require('redis-url-parse');
const REDIS_URL = process.env.REDIS_URL || 'redis://127.0.0.1:6379';
const redisUrlParsed = redisUrlParse(REDIS_URL);
const { host, port, password } = redisUrlParsed;
const bullOptions = REDIS_URL.includes('rediss://')
? {
redis: {
port: Number(port),
host,
password,
tls: {
rejectUnauthorized: false,
},
},
}
: REDIS_URL;
const workQueue = new Queue('work', bullOptions);
This worked for me using node-redis v3.0.0
const opts = config.REDIS_URL.includes('rediss://') ? {
url: config.REDIS_URL,
tls: {
rejectUnauthorized: false
}
} : config.REDIS_URL;
const client = redis.createClient(opts);
Use tls not socket. Thanks to this.
I use this script to connect node.js with Azure Postgresql.
But the ssl verification of our firewall blocks the connection, so in the past I need to use a proxy. Where in the code can I add the proxy settings as like host and port?
Means when I start the code, vscode should connect through the proxy to postgresql.
const pg = require('pg');
const config = {
host: '<your-db-server-name>.postgres.database.azure.com',
// Do not hard code your username and password.
// Consider using Node environment variables.
user: '<your-db-username>',
password: '<your-password>',
database: '<name-of-database>',
port: 5432,
ssl: true
};
const client = new pg.Client(config);
client.connect(err => {
if (err) throw err;
else { queryDatabase(); }
});
function queryDatabase() {
console.log(`Running query to PostgreSQL server: ${config.host}`);
const query = 'SELECT * FROM inventory;';
client.query(query)
.then(res => {
const rows = res.rows;
rows.map(row => {
console.log(`Read: ${JSON.stringify(row)}`);
});
process.exit();
})
.catch(err => {
console.log(err);
});
}
To configure proxy for Visual Studio Code
Edit the settings.json file
Depending on your platform, the user settings file is located here:
Windows: %APPDATA%\Code\User\settings.json
macOS: $HOME/Library/Application Support/Code/User/settings.json
Linux: $HOME/.config/Code/User/settings.json
Modify and Add the below lines to configure your proxy
"http.proxy": "http://user:pass#proxy.com:portnumber",
"https.proxy": "http://user:pass#proxy.com:portnumber",
"http.proxyStrictSSL": false
If your proxy doesn't require authentication, you could simply use
"http.proxy": "http://proxy.com:portnumber",
"https.proxy": "http://proxy.com:portnumber"
"http.proxyStrictSSL": false
Restart VS Code
The documentation related to settings and schema of the settings.json file is here for reference