I want to add timeout to pg-promise queries so they will fail after some amount of time if database have not yet responded.
Is there any recommended way to do that or should I make custom wrapper that will handle timer and reject promise if it's too late?
From the author of pg-promise...
pg-promise doesn't support query cancellation, because it is a hack to work-around incorrect database design or bad query execution.
PostgreSQL supports events that should be used when executing time-consuming queries, so instead of waiting, one can set an event listener to be triggered when specific data/view becomes available. See LISTEN/NOTIFY example.
You can extend pg-promise with your own custom query method that will time out with a reject (see example below), but that's again another work-around on top of a design problem.
Example using Bluebird:
const Promise = require('bluebird');
Promise.config({
cancellation: true
});
const initOptions = {
promiseLib: Promise,
extend(obj) {
obj.queryTimeout = (query, values, delay) => {
return obj.any(query, values).timeout(delay);
}
}
};
const pgp = require('pg-promise')(initOptions);
const db = pgp(/* connection details */);
Then you can use db.queryTimeout(query, values, delay) on every level.
Alternatively, if you are using Bluebird, you can chain .timeout(delay) to any of the existing methods:
db.any(query, values)
.timeout(500)
.then(data => {})
.catch(error => {})
See also:
extend event
Bluebird.timeout
UPDATE
From version 8.5.3, pg-promise started supporting query timeouts, via property query_timeout within the connection object.
You can either override the defaults:
pgp.pg.defaults.query_timeout = 3000; // timeout every query after 3 seconds
Or specify it within the connection object:
const db = pgp({
/* all connection details */
query_timeout: 3000
});
Related
I'm trying to save some json files inside my database using a custom function I've wrote. To achieve that I must connect to the database which I'm trying to do using this piece of code at the start of the function:
let url = "mongodb://localhost:27017/database";
(async () => {
const directory = await fs.promises.readdir(__dirname + '/files')
let database = await mongoose.createConnection(url, {useNewUrlParser:true, useUnifiedTopology:true});
database.on('error', error => {
throw console.log("Couldn't Connect To The Database");
});
database.once('open', function() {
//Saving the data using Schema and save();
Weirdly enough, when executing database.once('open', function()) the callback function isn't being called at all and the program just skips the whole saving part and gets right to the end of the function.
I've searched the web for a solution, and one solution suggested to use mongoose.createConnection instant of mongoose.connect.
As you can see it didn't really fixed the issue and the callback function is still not being called.
How can I fix it, and why it happens?
Thanks!
mongoose.createConnection creates a connection instance and allows you to manage multiple db connections as the documentation states. In your case using connect() should be sufficient (connect will create one default connection which is accessible under mongoose.connection).
By awaiting the connect-promise you don't actually need to listen for the open event, you can simply do:
...
await mongoose.connect(url, {useNewUrlParser:true, useUnifiedTopology:true});
mongoose.model('YourModel', YourModelSchema);
...
When working with a big application that has several tables and several DB operations it's very difficult to keep track of what transactions are occurring. To workaround this we started by passing around a trx object.
This has proven to be very messy.
For example:
async getOrderById(id: string, trx?: Knex.Transaction) { ... }
Depending on the function calling getOrderById it will either pass a trx object or not. The above function will use trx if it is not null.
This seems simple at first, but it leads to mistakes where if you're in the middle of a transaction in one function and call another function that does NOT use a transaction, knex will hang with famous Knex: Timeout acquiring a connection. The pool is probably full.
async getAllPurchasesForUser(userId: string) {
..
const trx = await knex.transaction();
try {
..
getPurchaseForUserId(userId); // Forgot to make this consume trx, hence Knex timesout acquiring connection.
..
}
Based on that, I'm assuming this is not a best practice, but I would love if someone from Knex developer team could comment.
To improve this we're considering to instead use knex.transactionProvider() that is accessed throughout the app wherever we perform DB operations.
The example on the website seems incomplete:
// Does not start a transaction yet
const trxProvider = knex.transactionProvider();
const books = [
{title: 'Canterbury Tales'},
{title: 'Moby Dick'},
{title: 'Hamlet'}
];
// Starts a transaction
const trx = await trxProvider();
const ids = await trx('catalogues')
.insert({name: 'Old Books'}, 'id')
books.forEach((book) => book.catalogue_id = ids[0]);
await trx('books').insert(books);
// Reuses same transaction
const sameTrx = await trxProvider();
const ids2 = await sameTrx('catalogues')
.insert({name: 'New Books'}, 'id')
books.forEach((book) => book.catalogue_id = ids2[0]);
await sameTrx('books').insert(books);
In practice here's how I'm thinking about using this:
SingletonDBClass.ts:
const trxProvider = knex.transactionProvider();
export default trxProvider;
Orders.ts
import trx from '../SingletonDBClass';
..
async getOrderById(id: string) {
const trxInst = await trx;
try {
const order = await trxInst<Order>('orders').where({id});
trxInst.commit();
return order;
} catch (e) {
trxInst.rollback();
throw new Error(`Failed to fetch order, error: ${e}`);
}
}
..
Am I understanding this correctly?
Another example function where a transaction is actually needed:
async cancelOrder(id: string) {
const trxInst = await trx;
try {
trxInst('orders').update({ status: 'CANCELED' }).where({ id });
trxInst('active_orders').delete().where({ orderId: id });
trxInst.commit();
} catch (e) {
trxInst.rollback();
throw new Error(`Failed to cancel order, error: ${e}`);
}
}
Can someone confirm if I'm understanding this correctly? And more importantly if this is a good way to do this. Or is there a best practice I'm missing?
Appreciate your help knex team!
No. You cannot have global singleton class returning the transaction for your all of your internal functions. Otherwise you are trying always to use the same transaction for all the concurrent users trying to do different things in the application.
Also when you once commit / rollback the transaction returned by provider, it will not work anymore for other queries. Transaction provider can give you only single transaction.
Transaction provider is useful in a case, where you have for example middleware, which provides transaction for request handlers, but it should not be started, since it might not be needed so you don't want yet allocate a connection for it from pool.
Good way to do your stuff is to pass transcation or some request context or user session around, so that each concurrent user can have their own separate transactions.
for example:
async cancelOrder(trxInst, id: string) {
try {
trxInst('orders').update({ status: 'CANCELED' }).where({ id });
trxInst('active_orders').delete().where({ orderId: id });
trxInst.commit();
} catch (e) {
trxInst.rollback();
throw new Error(`Failed to cancel order, error: ${e}`);
}
}
Depending on the function calling getOrderById it will either pass a trx object or not. The above function will use trx if it is not null.
This seems simple at first, but it leads to mistakes where if you're in the middle of a transaction in one function and call another function that does NOT use a transaction, knex will hang with famous Knex: Timeout acquiring a connection. The pool is probably full.
We usually do it in a way that if trx is null, query throws an error, so that you need to explicitly pass either knex / trx to be able to execute the method and in some methods trx is actually required to be passed.
Anyhow if you really want to force everything to go through single transaction in a session by default you could create API modules in a way that for each user session you create an API instance which is initialized with transaction:
const dbForSession = new DbService(trxProvider);
const users = await dbForSession.allUsers();
and .allUsers() does something like return this.trx('users');
I have a number of Node functions running on AWS Lambda. These functions have been using the Node 8 runtime but AWS sent out an end-of-life notice saying that functions should be upgraded to the latest LTS. With that, I upgraded one on my functions to use Node 12. After being in production for a bit, I'm starting to see a ton of connection terminated unexpectedly errors when querying the database.
Here are the errors that I'm seeing:
The connection terminated unexpectedly error
And Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed - this seems to happen on the 1st or second invocation after seeing the connection terminated unexpectedly error.
I'm using Knex.js for querying the database. I was running older version of knex and node-postgres and recently upgraded to see if it would resolve the issue, but no luck. Here are the versions of knex and node-postgres that I'm currently running:
"knex": "^0.20.8"
"pg": "^7.17.1"
The only change I've made to this particular function is the upgrade to Node 12. I've also tried Node 10, but the same issue persists. Unfortunately, AWS won't let me downgrade to Node 8 to verify that it is indeed an issue. None of my other functions running on Node 8 are experiencing this issue.
I've researched knex, node-postgres and tarn.js (the Knex connection pooling library) to see if any related issues or solutions popped up, but so far, I haven't had any luck.
UPDATE:
Example of a handler. Note that this is happening on many different Lambdas, all running Node 12.
require('../../helpers/knex')
const { Rollbar } = require('#scoutforpets/utils')
const { Email } = require('#scoutforpets/notifications')
const { transaction: tx } = require('objection')
const Invoice = require('../../models/invoice')
// configure rollbar for error logging
const rollbar = Rollbar.configureRollbar(process.env.ROLLBAR_TOKEN)
/**
*
* #param {*} event
*/
async function handler (event) {
const { invoice } = event
const { id: invoiceId } = invoice
try {
return tx(Invoice, async Invoice => {
// send the receipt
await Email.Customer.paymentReceipt(invoiceId, true)
// convert JSON to model
const i = Invoice.fromJson(invoice)
// mark the invoice as having been sent
await i.markAsSent()
})
} catch (err) {
return err
}
}
module.exports.handler = rollbar.lambdaHandler(handler)
Starting with node.js 10 aws lambda make the handler async, so you have to adapt your code.
Docs : https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
The runtime passes three arguments to the handler method. The first
argument is the event object, which contains information from the
invoker. The invoker passes this information as a JSON-formatted
string when it calls Invoke, and the runtime converts it to an object.
When an AWS service invokes your function, the event structure varies
by service.
The second argument is the context object, which contains information
about the invocation, function, and execution environment. In the
preceding example, the function gets the name of the log stream from
the context object and returns it to the invoker.
The third argument, callback, is a function that you can call in
non-async functions to send a response. The callback function takes
two arguments: an Error and a response. When you call it, Lambda waits
for the event loop to be empty and then returns the response or error
to the invoker. The response object must be compatible with
JSON.stringify.
For async functions, you return a response, error, or promise to the
runtime instead of using callback.
exports.handler = async function(event, context, callback) {
console.log("EVENT: \n" + JSON.stringify(event, null, 2))
return context.logStreamName
}
Thx!
I think you need to set the right connection pooling config.
See the docs here: https://github.com/marcogrcr/sequelize/blob/patch-1/docs/manual/other-topics/aws-lambda.md
const { Sequelize } = require("sequelize");
let sequelize = null;
async function loadSequelize() {
const sequelize = new Sequelize(/* (...) */, {
// (...)
pool: {
/*
* Lambda functions process one request at a time but your code may issue multiple queries
* concurrently. Be wary that `sequelize` has methods that issue 2 queries concurrently
* (e.g. `Model.findAndCountAll()`). Using a value higher than 1 allows concurrent queries to
* be executed in parallel rather than serialized. Careful with executing too many queries in
* parallel per Lambda function execution since that can bring down your database with an
* excessive number of connections.
*
* Ideally you want to choose a `max` number where this holds true:
* max * EXPECTED_MAX_CONCURRENT_LAMBDA_INVOCATIONS < MAX_ALLOWED_DATABASE_CONNECTIONS * 0.8
*/
max: 2,
/*
* Set this value to 0 so connection pool eviction logic eventually cleans up all connections
* in the event of a Lambda function timeout.
*/
min: 0,
/*
* Set this value to 0 so connections are eligible for cleanup immediately after they're
* returned to the pool.
*/
idle: 0,
// Choose a small enough value that fails fast if a connection takes too long to be established.
acquire: 3000,
/*
* Ensures the connection pool attempts to be cleaned up automatically on the next Lambda
* function invocation, if the previous invocation timed out.
*/
evict: CURRENT_LAMBDA_FUNCTION_TIMEOUT
}
});
// or `sequelize.sync()`
await sequelize.authenticate();
return sequelize;
}
module.exports.handler = async function (event, callback) {
// re-use the sequelize instance across invocations to improve performance
if (!sequelize) {
sequelize = await loadSequelize();
} else {
// restart connection pool to ensure connections are not re-used across invocations
sequelize.connectionManager.initPools();
// restore `getConnection()` if it has been overwritten by `close()`
if (sequelize.connectionManager.hasOwnProperty("getConnection")) {
delete sequelize.connectionManager.getConnection;
}
}
try {
return await doSomethingWithSequelize(sequelize);
} finally {
// close any opened connections during the invocation
// this will wait for any in-progress queries to finish before closing the connections
await sequelize.connectionManager.close();
}
};
It's actually for sequelize, not knex, but I'm sure under the hood they work the same way.
I had this problem too, in my case it was cause i tried to connect db in production.
so, I added ssl to Pool, like this:
const pool = new Pool({
connectionString: connectionString,
ssl: {rejectUnauthorized: false},
});
Hope it helps you too...
I want to add timeout to pg-promise queries so they will fail after some amount of time if database have not yet responded.
Is there any recommended way to do that or should I make custom wrapper that will handle timer and reject promise if it's too late?
From the author of pg-promise...
pg-promise doesn't support query cancellation, because it is a hack to work-around incorrect database design or bad query execution.
PostgreSQL supports events that should be used when executing time-consuming queries, so instead of waiting, one can set an event listener to be triggered when specific data/view becomes available. See LISTEN/NOTIFY example.
You can extend pg-promise with your own custom query method that will time out with a reject (see example below), but that's again another work-around on top of a design problem.
Example using Bluebird:
const Promise = require('bluebird');
Promise.config({
cancellation: true
});
const initOptions = {
promiseLib: Promise,
extend(obj) {
obj.queryTimeout = (query, values, delay) => {
return obj.any(query, values).timeout(delay);
}
}
};
const pgp = require('pg-promise')(initOptions);
const db = pgp(/* connection details */);
Then you can use db.queryTimeout(query, values, delay) on every level.
Alternatively, if you are using Bluebird, you can chain .timeout(delay) to any of the existing methods:
db.any(query, values)
.timeout(500)
.then(data => {})
.catch(error => {})
See also:
extend event
Bluebird.timeout
UPDATE
From version 8.5.3, pg-promise started supporting query timeouts, via property query_timeout within the connection object.
You can either override the defaults:
pgp.pg.defaults.query_timeout = 3000; // timeout every query after 3 seconds
Or specify it within the connection object:
const db = pgp({
/* all connection details */
query_timeout: 3000
});
I'm running Express on Node.js and am wondering how I can effectively pass a single database connection context object between distinct Node modules (think of them sort of like application models).
I'd like to do this to be able to start a database transaction in one model and preserve it across calls to other affected models, for the duration of a single HTTP request.
I've seen people attempt to solve this using per-request database connections exposed as middleware before my route is run (taking from a connection pool, then running another piece of middleware after the routes to return the connection to the pool). That unfortunately means explicitly passing around a context object to all the affected functions, which is inelegant and clunky.
I've also seen people talking about the continuation-local-storage and AsyncWrap modules, but I'm unclear how they can solve my particular problem. I tried working with continuation-local-storage briefly but because I primarily use promises and generators in my code, it wasn't able to pass state back from the run method (it simply returns the context object passed into its callback).
Here's an example of what I'm trying to do:
// player-routes.js
router.post('/items/upgrade', wrap(function *(req, res) {
const result = yield playerItem.upgrade(req.body.itemId);
res.json();
});
// player-item.js
const playerItem = {
upgrade: Promise.coroutine(function *(user, itemId) {
return db.withTransaction(function *(conn) {
yield db.queryAsync('UPDATE player_items SET level = level + 1 WHERE id = ?', [itemId]);
yield player.update(user);
return true;
});
})
};
module.exports = playerItem;
// player.js
const player = {
update(user) {
return db.queryAsync('UPDATE players SET last_updated_at = NOW() WHERE id = ?', [user.id]);
})
};
module.exports = player;
// db.js
db.withTransaction = function(genFn) {
return Promise.using(getTransactionConnection(), conn => {
return conn.beginTransactionAsync().then(() => {
const cr = Promise.coroutine(genFn);
return Promise
.try(() => cr(conn))
.then(res => {
return conn.commitAsync().thenReturn(res);
}, err => {
return conn.rollbackAsync()
.then(() => logger.info('Transaction successfully rolled back'))
.catch(e => logger.error(e))
.throw(err);
});
});
});
};
A couple of notes here:
The wrap function is just a little piece of wrapper middleware that allows me to use generators/yield in my routes.
The db module is also just a small wrapper around the popular mysql module, that has been promisified.
What I'd like to do, probably in db.queryAsync, is check if there's a conn object set on the current context (which I'd set around the return Promise... call in db.withTransaction). If so, use that connection to do all subsequent database calls until the context goes out of scope.
Unfortunately, wrapping the return Promise... call in the CLS namespace code didn't allow me to actually return the promise -- it just returned the context object, which is incorrect in my case. It looks like most usages of CLS rely on not actually returning anything from inside the run callback. I also looked at cls-bluebird, but that didn't seem to do what I need it to do, either.
Any ideas? I feel like I'm close, but it's just not all hooking up exactly how I need it to.