When I leave my application idle for a few minutes, it errors out. This is my first webapp I created and it is sending and receiving data from a mysql database - somewhat of a sign up form. Is there a way to stop this from happening? Thanks!
var connection = mysql.createPool({
host: host,
user: user,
password: password,
database: db,
ssl: ssl
});
My original code was var connection = mysql.createConnection...
I replaced that and removed connection.connect();
This seems to happen when establish a single use connection, it can be avoided by establishing a connection pool instead.
Related
I am opening up a project from a year and a half ago and my local PostgreSQL database is not working. I am using a react front end, with a Node.js back end. The Node module I am using to connect with the local database is pg.pool. The error I get when I submit the data through pg.pool is:
Error with query for user error: no PostgreSQL user name specified in startup packet
I was reading on the internet and apparently the client has to submit a username to the backend for the Database to open up. The internet suggested opening the pg_hba.conf file and checking that the users are correct, but I checked the file and the users for the PostgreSQL is set to all, so I don't think that is the problem. Here is that file:
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all scram-sha-256
# IPv4 local connections:
host all all 127.0.0.1/32 scram-sha-256
# IPv6 local connections:
host all all ::1/128 scram-sha-256
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all scram-sha-256
host replication all 127.0.0.1/32 scram-sha-256
host replication all ::1/128 scram-sha-256
Here is my pool.js file that connects the node server to the postgresql instance running locally, it uses a if else statement incase I want to connect locally or in case I want to connect to heroku:
const pg = require('pg');
const url = require('url');
let config = {};
if (process.env.DATABASE_URL) {
// Heroku gives a url, not a connection object
// https://github.com/brianc/node-pg-pool
const params = url.parse(process.env.DATABASE_URL);
const auth = params.auth.split(':');
config = {
user: auth[0],
password: auth[1],
host: params.hostname,
port: params.port,
database: params.pathname.split('/')[1],
ssl: { rejectUnauthorized: false },
max: 10, // max number of clients in the pool
idleTimeoutMillis: 30000, // how long a client is allowed to remain idle before being closed
};
} else {
config = {
host: 'localhost', // Server hosting the postgres database
port: 5432, // env var: PGPORT
database: "workout_app_prime", // CHANGE THIS LINE! env var: PGDATABASE, this is likely the one thing you need to change to get up and running
max: 10, // max number of clients in the pool
idleTimeoutMillis: 30000, // how long a client is allowed to remain idle before being closed
};
}
// this creates the pool that will be shared by all other modules
const pool = new pg.Pool(config);
// the pool with emit an error on behalf of any idle clients
// it contains if a backend error or network partition happens
pool.on('error', (err) => {
console.log('Unexpected error on idle client', err);
process.exit(-1);
});
module.exports = pool;
Other relevant facts: yesterday morning working on a different project my computer crashed, PostgreSQL wasn't working when I tried to reopen that project because the project didn't terminate correctly, So I deleted the .pid file and restarted the computer then it started working.
When I made this database a year and a half ago I was running postgreSQL 12, now I am on 13.
I am running on Mac OS 10.15, idk if that is relevant.
Anybody have any ideas? Thanks!
I'm a bit stuck here and was hoping to get some help.
My node application has a seperate module where I connect to postgres and export the pool as so
const {Pool,Client} = require('pg');
const pool = new Pool({
user: process.env.POSTGRES_USER,
host: process.env.POSTGRES_URL,
database: process.env.POSTGRES_DATABASE,
password: process.env.POSTGRES_PASSWORD,
port: process.env.POSTGRES_PORT,
keepAlive: 0,
ssl:{ rejectUnauthorized: false,
sslmode:require},
connectionTimeoutMillis: 10000, // 10 seconds
allowExitOnIdle:true,
max: 10
});
pool.connect()
.then(() => console.log('postgress connected'))
.catch(err => console.error(err))
module.exports = pool
On my route, I have redis cache as middleware, this works as expected and can confirm it is being served up by redis, the logic in the route does not run when the request is cached, however I was doing some load testing to see how everything would handle spikes and noticed I started to get errors from postgres
Error: timeout exceeded when trying to connect
I also got errors talking about max connections etc.
I have tried to increase the max pool connection but still seem to get this error when running some larger load tests.
My question is, why, would PG be trying to connect if the connection should be shared? Additionally, why is it even trying to connect if the request is cached?
Any help would be appreciated!
Apparently some of your stress test cases are missing the redis cache. You haven't shown any code relevant to that, so what more can be said?
The error you show is not generated by PostgreSQL, it is generated by node's 'pg' module. You configured it to only allow 10 simultaneous connections. If more than that are requested, they have to line up and wait. And you also configured it to wait only for 10 seconds before bombing out with an error, and that is exactly what you are seeing.
You vaguely allude to other errors, but you would have to share the actual error message with us if you want help.
The system seems to be operating as designed. You did a stress test to see what would happen, and you have seen what happens.
I followed a guide on how to create a Node JS backend on PostgreSQL and a ReactJS frontend, which is working well, see here for more.
The js pool object of the quide looks as follows:
const Pool = require('pg').Pool
const pool = new Pool({
user: 'my_user',
host: 'localhost',
database: 'my_database',
password: 'postgres',
port: 5432,
});
I tried doing the same with a PostgreSQL Webserver using SSL encryption. This question is not tagged postgresql since the pool looks the same for mysql and other databases. It is also not tagged Linux since it should work the same on other OS.
How to create a pool object from the Pool class in javascript that can connect to a PostgreSQL backend using SSL?
You need to save the certificate, call it server-certificate.pem, in a place with a static path starting with / (~ for your user's home directory does not work). Therefore, you need to move it to a static place.
I copied the file to a newly created directory certificates, but that is of course up to you. You will need super user rights for this.
/etc/certificates/server-certificate.pem
You need to load fs and use it in the SSL part, see How to establish a secure connection (SSL) from a Node.js API to an AWS RDS.
const Pool = require('pg').Pool
const fs = require('fs');
const pool = new Pool({
user: 'my_user',
host: 'my_host',
database: 'my_database',
password: 'my_password',
port: 22222,
ssl: {
ca: fs
.readFileSync("/etc/certificates/server-certificate.pem")
.toString()
}
});
We recently moved to Heroku and upon attempting to connect our apps to the DB, it kept rejecting our queries with the message "Self signed certificate". Passing in rejectUnauthorized solved for this but now I'm wondering, should we be doing this in production? If not, what is the appropriate way for us to be connecting to our Heroku PG Databases?
const pgp = require('pg-promise')(/*initOptions*/);
const {ConnectionString} = require('connection-string');
const cnObj = new ConnectionString(process.env.DATABASE_URL);
const cn = {
host: cnObj.hostname,
port: cnObj.port,
database: cnObj.path?.[0],
user: cnObj.user,
password: cnObj.password,
ssl: {
rejectUnauthorized: false,
},
};
const db = pgp(cn);
The risk you are running is that somebody gets between you and the Heroku server and impersonates the latter. They can then present their own certificate to you and negotiate a connection with you. The man in the middle can also pass the challenge from the server down to you and use your response to log into the database server in your stead.
All that assumes that the attacker has control over one of the network nodes between you and the Heroku server.
So I would say that while there is a residual risk, I wouldn't lose too much sleep over it, unless you are working with really sensitive data, in which case paranoia is a virtue.
My setup is the following:
I have a Virtual Machine running all of my Database processes, let's call it DB-VM.
I'm currently developing at my own workstation (completely detached from DB-VM, except that we are under the same network.
I've created a valid connection string, validated by another database connection service throughout IIS and through a Data Link Properties file (.udl) and the connection.
This connection is described by the connection string as:
Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Data Source=DB-VM\MY_DATABASE.
I tried to insert it into my Sequelize configuration as following:
const sequelize = new Sequelize({
dialect: 'mssql',
dialectModulePath: 'sequelize-msnodesqlv8',
dialectOptions: {
connectionString: 'Provider=SQLOLEDB.1;Integrated Security=SSPI;Persist Security Info=False;Data Source=DB-VM\MY_DATABASE',
trustedConnection: true,
}
});
And then proceeded to try and authenticate through:
sequelize.authenticate().then(() => {
console.log('Connection stablished successfully!');
}).catch(err => {
console.log(err);
});
And this the error is as follows:
Notice: The database uses dynamic ports, therefore I can't specify the port through the port property.
Notice 2: The Named Pipes are disabled on my database settings, and I'm not sure if I will be able to enabled it.
Notice 3: The database is already setup to allow remote connections (it is currently used through a Webpage and works fine!
According to this line the sequelize-msnodesqlv8 library expects "Driver" to be a part of the connection string, otherwise it tries to guess. Besides that, all the examples of connection strings here1 and here2 are using either Server=myServerName\theInstanceName or just Server=myServerName. Instead of the Data Source=....
So step 1 is to fix you connection string. You could try one of the examples like:
dialectOptions: {
connectionString: 'Driver={SQL Server Native Client 10.0};Server=DB-VM;Database=MY_DATABASE;Trusted_Connection=yes;'
},
After that if you get a new error, please update the question.