I am using node-pg with typescript.
I have a getPool utility from the doc https://node-postgres.com/features/pooling
export const getPool = (config?: PoolConfig) => {
const pool = new Pool(config);
pool.on('error', (err, client) => {
console.error('Unexpected error on idle client', err);
process.exit(-1);
});
return pool;
};
I use it like this in an async/await context
const pool = getPool();
await pool.query('my sql query here...');
When I have an invalid SQL query I get this kind of error:
error: null value in column "foo" violates not-null constraint
at Parser.parseErrorMessage (node_modules/pg-protocol/src/parser.ts:357:11)
at Parser.handlePacket (node_modules/pg-protocol/src/parser.ts:186:21)
at Parser.parse (node_modules/pg-protocol/src/parser.ts:101:30)
at Socket.<anonymous> (node_modules/pg-protocol/src/index.ts:7:48)
Note: I would understand if it was the pool.on('error')'s callback that stole my stack trace, but the errors are not prefixed with Unexpected error on idle client
Notice in the stack trace there is no line that points to a file in my codebase.
My problem is, I have hundereds of queries in my codebase, and I would like to be able to trace the line that called the failing pool.query(). This would help a lot to find which query triggered the error.
Expected :
error: null value in column "foo" violates not-null constraint
at ...
at mycodebase/src/myfile.ts:42
I use dirty hack (patch Pool.prototype), but it works for me:
const originalPoolQuery = Pool.prototype.query;
Pool.prototype.query = async function query(...args) {
try {
return await originalPoolQuery.apply(this, args);
} catch (e) {
// All magic is here. new Error will generate new stack, but message will copyid from e
throw new Error(e)
}
}
// After hack create pool and good luck
const pool = new Pool({})
await pool.query('SELECT * FROM ...')
Stacktrace in this case will be more informative.
I think that pool.on('error', cb) is not for catching query errors, it for connection errors (i am not sure)
Related
I am working with Knex to write a simple transaction. I am inserting an object into one table and then insert an array of objects. Since I am using Knex in my API, I would like to know which insert failed. Unfortunately the error objects only shares this information:
{
"length":130,
"name":"error",
"severity":\"ERROR\",
"code":\"22P02\",
"file":\"uuid.c\",
"line":\"137\",
"routine":\"string_to_uuid\"
}
I would like to know though which query exactly failed to send back a proper error message to the Frontend. I am using a lambda setup with a RDS connection.
This is my transaction:
const test = await client.transaction(async (trx) => {
try {
await trx<any>('schema.table')
.insert({
id: answers.myId,
addition: 'some text',
});
const test = answers.answers.map((answer) => ({
id: answer.id,
my_id: answers.myId,
offered_answer_id: answer.offered_answer_id,
value: answer.value !== '' ? answer.value : null,
}));
await trx<any>('schema.table2').insert(test);
} catch (error) {
return error;
}
I am triggering an error by manipulating an uuid. So the error object is technically correct but I cannot figure out which insert caused it with this information.
Any help is very appreciated!
Note: I already found a solution to this problem, posting it here for posterity. See the selected answer.
The following (simplified) code throws an uncatchable "write EPIPE" (and in some scenarios "write EOF") error:
const { exec } = require("child_process");
const veryLargeString = "x".repeat(10 * 1024 * 1024);
const p = exec("gibberishThatWillFailImmediately");
p.stdin.write(veryLargeString);
My failed attempts at the problem:
Checking the stdin.destroyed flag before writing
Checking the stdin.writeableEnded flag before writing
Chunking the input and checking stdin.writeableEnded before each chunk. This one lead to undeterministic behavior.
Wrapping the stdin.write(data) line with try-catch
Calling stdin.end(data) instead of stdin.write(data)
Pass stdin.write() a callback that should get any error that occurs. The callback got the error, but didn't prevent it from being thrown.
Registering an 'error' handler to the stdin stream seems to prevent the error from being thrown. Like this:
const { exec } = require("child_process");
const veryLargeString = "x".repeat(10 * 1024 * 1024);
const p = exec("gibberishThatWillFailImmediately");
p.stdin.on('error', (error) => console.log("error caught: ", error));
p.stdin.write(veryLargeString);
Here's an example that returns a promise containing the error or null if no error occured:
const { exec } = require("child_process");
const veryLargeString = "x".repeat(10 * 1024 * 1024);
function safelyWriteDataToStdin(stdin, data) {
// Register an awaitable callback that will capture any error occuring during the write operation
const promise = new Promise((resolve, _reject) => {
// Using once() and not on() to remove the listener after the first catch.
stdin.once("error", (error) => resolve(error));
// stdin.end(data, callback) can probably be used here, but I keep the `write()` just in case `end()`'s callback is called before the 'error' event, since the docs are not clear about that. (docs say: "The callback is invoked before 'finish' or on error." for node version 15.0.0. Is "on error" how node people say "after error"? idk.)
stdin.write(
data,
(error) => {
if (!error) resolve(null); // The condition is necessary because when an error occurs, the callback is called before the 'error' event handler
} // Signal the promise to complete when the write operation is complete with no errors. I don't simply use this `error` parameter because the exception will still be thrown if I don't listen to the 'error' event, and the docs say: "If an error occurs, the callback may or may not be called with the error as its first argument. To reliably detect write errors, add a listener for the 'error' event.". Also, I tested it myself and got two different errors in this callback and in the 'error' event handler.
);
});
return promise;
}
const p = exec("gibberishThatWillFailImmediately");
safelyWriteDataToStdin(p.stdin, veryLargeString).then((error)=>console.log("The error is:", error ));
I'm using Firebase Cloud Functions, and in one of my projects I have a simple function that looks like the following:
exports.responseGiven = functions.https.onCall(async (data, context) => {
if (!context.auth) {
throw new functions.https.HttpsError(
'permission-denied',
'Must be an user to execute this action'
);
}
const studentPath = data.studentPath;
const eventPath = data.eventPath;
const isUpdating = data.isUpdating;
const studentDoc = await admin.firestore().collection('students').doc('studentPath').get();
const eventDoc = await studentDoc.ref.collection('entries').doc(eventPath).get();
});
I know where the error is through other methods and why it is, I'm using an invalidStudentPath. But the bigger issue is that the error that gets thrown is this for all of my errors in this project:
Unhandled error function error(...args) {
write(entryFromArgs('ERROR', args));
}
How can I get the actual error instead of this obscure message? Thanks for any help.
Update
I have found a workaround for now. Currently I'm wrapping my whole function in a try catch block that looks like this:
exports.responseGiven = functions.https.onCall(async (data, context) => {
try {
...
} catch (e) {
console.log('There was an error');
console.log(e);
}
});
The issue was fixed in firebase-functions v3.9.1. Alternatively you could downgrade to v3.8.0.
See https://github.com/firebase/firebase-functions/issues/757#issuecomment-673080726
I had a case recently where I forgot to await the result of an async (Promise-returning) function inside the relevant try / catch, so it escaped to the outer level. In the code I was working on, the logs look similar in the escaped case and catch error handler case. It is something that evaded my notice during unit testing because the functions were still triggering the node assert rejection hooks with the proper error messages.
I just got the same error on a new project. Took me a few hours to realize I forgot to initialize firebase admin after I imported it in my functions
import * as admin from 'firebase-admin';
admin.initializeApp();
I am struggling to figure out what the error is in this async function. I keep getting this error message:
"Unexpected token catch"
and if that error is fixed I get this error Message:
"UnhandledPromiseRejectionWarning"
"DeprecationWarning"
router.get('/Views', async (req, res) => {
console.log("Google auth ", googleAuth)
const organizationId = req.verifiedToken.OrganizationId;
console.log("Got here")
const url = myUrl;
try{
const client = await auth.getClient({
scopes: [
'https://www.googleapis.com/auth/analytics.readonly'
]
})catch(err){
console.log(err);
throw err;
};
const outcome = await client.request({ url })catch(err){
console.log(err);
throw err;
};
};
console.log("Successfully connected!", organizationId);
return responses.success(res, outcome.data);
});
The line
const outcome = await client.request({ url })catch(err){
introduces a catch exception handler without a prior try block. You appear to have caught (no pun intended) this syntax error (though you haven't detailed the code changes to get rid of it).
Unfortunately you haven't posted a self-contained code fragment and you haven't specified which framework/libraries you use on top of node.js. If you are using the Express framework and an ajax library, it might be that your try block is missing an exception handler and the catch statements are meant to be method calls:
router.get('/Views', async (req, res) => {
console.log("Google auth ", googleAuth)
const organizationId = req.verifiedToken.OrganizationId;
console.log("Got here")
const url = myUrl;
try{
const client = await auth.getClient({
scopes: [
'https://www.googleapis.com/auth/analytics.readonly'
]
})
.catch(err){ //*** syntax error in your code sample
console.log(err);
throw err;
};
const outcome = await client.request({
url
})
.catch(err){ //*** syntax error in your code sample
console.log(err);
throw err;
};
} catch (e) { //*** exception handler missing from your original code missing
// exception handling code, possibly empty; there are the .catch calls after all
}
console.log("Successfully connected!", organizationId);
return responses
.success ( res, outcome.data )
.error ( )
//*** This call is missing from your code.
// It is assumed that 'responses' holds a "Promise", see explanation
;
});
A 'Promise' in asynchronous programming is an abstraction of a value not yet known (think of it as a placeholder for that value). There are 2 basic possible scenarios: either that value will eventually be computed (#1) or it is ascertained that it will never be computed at all (#2). A promise library handles these scenarios. The reference API sports promise objects with a .then ( fn_ok, fn_fail ) method taking 2 functions as arguments, one being associated the first scenario, one with the second. As soon as a scenario is established, the respective function will be called. Promise libraries may add additional layers of abstraction, possibly producing the .success/.error calls from the code sample. Note that you promise libraries do usually support 'chaining': In the code sample, the call to .success (.error) would actually make sure that res and outcome.data (nothing) will be preserved and fed to the handler for the scenario #1 (#2) and would return a promise (technically the same object with some properties being redefined).
The second error you have received ( UnhandledPromiseRejectionWarning ) would thus stem from not handling scenario #2. However, the case that the future computation of a value will fail is a very possible outcome and should be accounted for in general. Otherwise your code is amenable to run-time errors that are very hard to track down as you will neither be notified of the code section the error occurs nor (due to async programming) will you have guarantees on the execution order of code fragments. Moreover this coding style easily rsults in a deadlock or the eventual exhaustion of some resource (memory, file handles, ...)
This explanation is deliberately written following a (hopefully) intuitive approach and is technically somewhat sloppy. For a detailed explanation consult https://www.promisejs.org/, MDN: Promises, or similar resources.
Caveat
There is some guesswork involved in this answer. However the general outline of the problem's origin should remain valid.
I am struggling to get my head wrapped around what the proper pattern is for handling errors within nested await/async routines, yet keeping the code clean and simple. (despite reading countless articles and blogs)
I have a set of functions that are (fundamentally) similar to the following:
async validate(params) {
const recCount = await this._getCount(db, params);
if( recCount > 0 )
return "Record already exists";
}
_getCount is a wrapper that creates the sql
async _getCount(conn, regdata) {
const sql = "SELECT count(*) AS 'count' FROM myTable WHERE product = ? and category = ?";
let rows = await this._execSQL(conn, sql, [ regdata.product, regdata.category ]);
return rows[0].count;
}
and the actual query is executed as follows:
async _execSQL(conn, sql, data) {
const [ rows ] = await conn.query(sql, data);
return rows;
}
The method conn.query (from the mysql2/promise library) will reject the promise if the query fails.
So, my question becomes what is the proper pattern for handling the exceptions?
In a synchronous world, I could nothing to the _execSQL nor the _getCount and just catch the exception in validate; just naturally letting the exception bubble up.
However, in the async world how can I do the equivalent without getting the 'Unhandled Promise' exception?
Am I stuck with having to catch the error at every single async routine all the way through the levels?
Or is there a better way without using something like process.on('unhandledRejection',...) which feels like I am circumventing the problem?
EDIT: Added example and stack trace
Ok, so I have actually added this code to my application and put the try/catch in the validate function. Verbatim code is:
async validate(db, params) {
let recCount;
try {
recCount = await this._getCount(db, params);
} catch (err) {
console.log('Caught error', err);
}
if (recCount > 0) return 'Record already exists';
}
async _getCount(conn, regdata) {
const sql = "SELECT count(*) AS 'count' FROM myTable WHERE product = ? and category = ?";
let rows = await this._execSQL(conn, sql, [ regdata.product, regdata.category ]);
return rows[0].count;
}
async _execSQL(conn, sql, data) {
const [ rows ] = await conn.query(sql, data);
return rows;
}
I have a event handler for unhandledRejection, which reports out the event along with the inner exception along with the stack trace. This is what it dumps out:
Stack Trace:
AppError: Unhandled promise rejection. Plugin may not be properly handling error.
at process.on (D:\Development\website\service\server.js:73:5)
at emitTwo (events.js:126:13)
at process.emit (events.js:214:7)
at emitPendingUnhandledRejections (internal/process/promises.js:108:22)
at process._tickCallback (internal/process/next_tick.js:189:7)
Inner Error:
{ "message": "connect ECONNREFUSED 127.0.0.1:3306", "code": "ECONNREFUSED", "errno": "ECONNREFUSED" }
Error: connect ECONNREFUSED 127.0.0.1:3306
at PromisePool.query (D:\Development\website\webhooks\node_modules\mysql2\promise.js:323:22)
at Registration._execSQL (D:\Development\website\webhooks\plugins\registration.js:108:31)
at Registration._logRequest (D:\Development\website\webhooks\plugins\registration.js:179:14)
at Registration.register (D:\Development\website\webhooks\plugins\registration.js:52:8)
at Router.exec (D:\Development\website\service\router.js:119:20)
at IncomingMessage.request.on (D:\Development\website\service\server.js:292:47)
at emitNone (events.js:106:13)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
You can always let rejections bubble up and choose the best level to catch them:
async function f1() { return await f2(); }
async function f2() { return await f3(); }
async function f3() {
return Promise.reject('no way!');
// or
throw 'no way!';
}
async function f_await() {
try {
console.log('never succeeds here', await f1());
} catch (err) {
console.log('I check for errors at the level I prefer!');
throw 'I can even intercept and rethrow!';
}
return 'or i can keep going like eveything is fine';
}
function f_then() {
f1().then(console.log.bind(null, 'never succeeds here'))
.catch(function (err) {
console.log('I check for errors at the level I prefer!');
throw 'I can even intercept and rethrow!';
}).then(function () {
return 'or i can keep going like eveything is fine';
});
}
If you trigger an unhandled rejection warning, it's because... you didn't handle some rejection at any point in the chain, whereas you always need to: even in synchronous code, if an exception is raised and never caught, the computer will tell you how unhappy it is.
If you think the best way to deal with an SQL query being rejected in your code is in validate, then go for it: surround this await with a try/catch block and "handle" the error in the catch the way you think is best... Not sure I see the problem here!