I have a question, in the PDO manuial somewhere I read that errors reveal the db connect with username and password (due to a flaw in the zend engine). I see several examples of catching the pdo like this:
catch(PDOException $exception){
return $exception;
}
if the exception is returned, doesn't the user see the error?
Is it better to have disabled the error reporting in the php.ini file, or even do something like
setAttribute(PDO::ERRMODE_SILENT)
instead of the catch statement, or is it better to do a combination of above and redo the catch statement so it doesn't return the error to the user.
This is referring to the pink paragraph on the manual page that says: Warning: If your application does not catch the exception thrown from the PDO constructor, the default action taken by the zend engine is to terminate the script and display a back trace. This back trace will likely reveal the full database connection details, including the username and password. It is your responsibility to catch this exception, either explicitly (via a catch statement) or implicitly via set_exception_handler(). php.net/manual/en/pdo.connections.php.
The user "YOUR COMMON SENSE" marked this as duplicate which is not correct. I don't have an issue with using PDO, Its just a question of dealing with error responses, and correct methodology of error handling.
Related
I'm currently writing a public REST service in Node.js that interfaces with a Postgres-database (using Sequelize) and a Redis cache instance.
I'm now looking into error handling and how to send informative and verbose error messages if something would happen with a request.
It struck me that I'm not quite sure how to handle internal server errors. What would be the appropriate way of dealing with this? Consider the following scenario:
I'm sending a post-request to an endpoint which in turn inserts the content to the database. However, something went wrong during this process (validation, connection issue, whatever). An error is thrown by the Sequelize-driver and I catch it.
I would argue that it is quite sensitive information (even if I remove the stack trace) and I'm not comfortable with exposing references of internal concepts (table-names, functions, etc.) to the client. I'd like to have a custom error for these scenarios that briefly describes the problem without giving away too detailed information.
Is the only way to approach this by mapping every "possible" error in the Sequelize-driver to a generic one and send that back to the client? Or how would you approach this?
Thanks in advance.
Errors are always caused by something. You should identify and intercept these causes before doing your database operation. Only cases that you think you've prepared for should reach the database operation.
If an unexpected error occurs, you should not send an informative error message for security reasons. Just send a generic error for unexpected cases.
Your code will look somewhat like this:
async databaseInsert(req, res) {
try {
if (typeof req.body.name !== 'string') {
res.status(400).send('Required field "name" was missing or malformed.')
return
}
if (problemCase2) {
res.status(400).send('Error message 2')
return
}
...
result = await ... // database operation
res.status(200).send(result)
} catch (e) {
res.status(500).send(debugging ? e : 'Unexpected error')
}
}
I am trying to understand how to build my error handling system for my api.
Let's say I have a the following line in a controller method :
var age = json.info.age;
with
json = {"id":1, "name":"John", info": {"age":27, "sex":"m"}}
Let's say that the object doesn't contain an info field, I'll get the following error TypeError: Cannot read property 'info' of undefined and my server will crash.
Is there a way to make a higher level abstraction and catch all the potential errors that I could have? Or should I have a try/catch system for each of the methods of my controllers?
BEWARE OF THE CODE BELOW, IT WILL BITE YOU WHENEVER IT CAN!
Don't use the code snippet below if you do not understand its
implications, please read the whole answer.
You can use the node way for uncaught errors. Add this in your config/bootstrap.js
Updated the snippet below to add what was said in the comments, also added a warning about using a global to respond to the user.
process.on('uncaughtException', function (err) {
// Handle your errors here
// global.__current__ is added via middleware
// Be aware that this is a bad practice,
// global.__current__ being a global, can change
// without advice, so you might end responding with
// serverError() to a different request than the one
// that originated the error if this one happened async
global.__current__.res.serverError();
})
Now, can doesn't mean should. It really depends on your needs, but do not try to catch BUGS in your code, try to catch at a controller level the issues that might not happen every time but are somehow expected, like a third-party service that responded with empty data, you should handle that in your controller. The uncaughtException is mainly for logging purposes, its better to let your app crash if there is a bug. Or you can do something more complicated (that might be better IMHO), which is to stop receiving requests, respond to the error 500 (or a custom one) to user that requested the faulty endpoint, and try to complete the other requests that do not relate to that controller, then log and shutdown the server. You will need several instances of sails running to avoid zero downtime, but that is material for another question. What you asked is how to get uncaught exceptions at a higher lvl than the controllers.
I suggest you read the node guide for error handling
Also read about domains, even thought they are deprecated you can use them, but you would have to deal with them per controller action, since sails does not provide any help with that.
I hope it helps.
You can check this way if you want to:
if (object != null && object.response != null && object.response.docs != null){
//Do your stuff here with your document
}
I don't really get what is your "object" variable in the first place, so i don't know if you can check it at a different level, is it a sails parameter to your controller ?
So that's how I did it, thanks to Zagen's answer.
module.exports.bootstrap = function(cb) {
process.on('uncaughtException', function (err) {
//Handle your errors here
logger.fatal(err);
global.__current__.res.serverError();
})
cb();
};
I send a generic error 500 to the user if any uncaught exception is thrown, and I log the error to the fatal level. On that way, my server is still accessible 24/7 and I can monitor the logs at another level and trigger an alarm on a fatal error. I can then fix the exception that was thrown.
Helllo,
My application is a web server that fires many requests to other servers. We set up a maximum timeout on those requests, and whenever the timeout is reached, the connection is closed and a ESOCKETTIMEDOUT rises.
Error: socket hang up
at createHangUpError (http.js:1472:15)
at Socket.socketCloseListener (http.js:1522:23)
at Socket.EventEmitter.emit (events.js:117:20)
at TCP.close (net.js:465:12)
I want to exclude these errors from the New Relic Dashboard, since they distort the error rate and other metrics. Hiding them doesn't work either, because they still count in the error rate.
How can remove specific errors (that do not have a HTTP status code) from my Dashboard?
You can pass status codes to ignore to the error collector. If you are configuring the New Relic agent using environment variables you can use a comma separated list of codes as the value for NEW_RELIC_ERROR_COLLECTOR_IGNORE_ERROR_CODES.
See the README.
If you are using newrelic.js to do so you can set the error_collector.ignore_codes value to an Array of status codes to ignore:
See the example config.
Important caveat: when setting this value manually you are overriding the default value of 404 which means that if you do not specify 404 in your manual configuration the Error Collector will start logging all 404 errors in your application (which you probably do not want).
I noticed you have javascript, I'm not sure if my solution can help you but I'll answer in the hope it does.
I use Java agent, and we have the same kind of problem. So far the only way that I found that can do something near what I want is having the specific errors wrapped in a dedicated exception ("NewRelicIgnorableException") and wrap whatever error I don't want to see in it.
Then I'd have to go into the dashboard/application and select "error collection". Last, I'd fill in the "Ignore these errors" with the full package name AND exception class name, like com.mypackage.NewRelicIgnorableException. Save and enjoy. These particular errors should not impact your apdex, but they will still count towards RPM and other metrics.
Other solutions have drawbacks. For example if I call ignoreexception the RPM and time metrics will not count. If you click the "hide error" button you only hide them from the error panel, but everything else will be as usual. If you ignore by status code you can get more or less the same results as ignoring the specific exception, but without any hope for fine control.
It's a pity that there's so little documentation on their site, I had to run tests to find these out.
I am trying to implement Paul Calhoun's Apache FOP solution for creating PDF's from Xpages (from Notes In 9 #102). I am getting the following java exception when trying to run the xAgent that does the processing --> Can't get a Writer while an OutputStream is already in use
The only changes that I have done from Paul's code was to change the package name. I have isolated when the exception happens to the SSJS line: var jce: DominoXMLFO2PDF = new DominoXMLFO2PDF(); All that line does is instantiate the class, there is no custom constructor. I don't believe it is the code itself, but some configuration issue. The SSJS code is in the beforeRenderResponse event where it should be, I haven't changed anything on the xAgent.
I have copied the jar files from Paul's sample database to mine, I have verified that the build paths are the same between the two databases. Everything compiles fine (after I did all this.) This exception appears to be an xpages only exception.
Here's what's really going on with this error:
XPages are essentially servlets... everything that happens in an XPage is just layers on top of a servlet engine. There are basically two types of data that a servlet can send back to whatever is initiating the connection (e.g. a browser): text and binary.
An ordinary XPage sends text -- specifically, HTML. Some xAgents also send text, such as JSON or XML. In any of these scenarios, however, Domino uses a Java Writer to send the response content, because Writers are optimized for sending Character data.
When we need to send binary content, we use an OutputStream instead, because streams are optimized for sending generic byte data. So if we're sending PDF, DOC/XLS/PPT, images, etc., we need to use a stream, because we're sending binary data, not text.
The catch (as you'll soon see, that's a pun) is that we can only use one per response.
Once any HTTP client is told what the content type of a response is, it makes assumptions about how to process that content. So if you tell it to expect application/pdf, it's expecting to only receive binary data. Conversely, if you tell it to expect application/json, it's expecting to only receive character data. If the response includes any data that doesn't match the promised content type, that nearly always invalidates the entire response.
So Domino in its infinite wisdom protects us from making this mistake by only allowing us to send one or the other in a single request, and throws an exception if we disobey that rule.
Unfortunately... if there's any exception in our code when we're trying to send binary content, Domino wants to report that to the consumer... which tries to invoke the output writer to send HTML reporting that something went wrong. Except we already got a handle on the output stream, so Domino isn't allowed to get a handle on the output writer, because that would violate its own rule against only using one per response. This, in turn, throws the exception you reported, masking the exception that actually caused the problem (in your case, probably a ClassNotFoundException).
So how do we make sure that we see the real problem, and not this misdirection? We try:
try {
/*
* Move all your existing code here...
*/
} catch (e) {
print("Error generating dynamic PDF: " + e.toString());
} finally {
facesContext.responseComplete();
}
There are two reasons this is a preferred approach:
If something goes wrong with our code, we don't let Domino throw an exception about it. Instead, we log it (instead of using print to send it to the console and log, you could also toss it to OpenLog, or whatever your preferred logging mechanism happens to be). This means that Domino doesn't try to report the error to the user, because we've promised that we already reported it to ourselves.
By moving the crucial facesContext.responseComplete() call (which is what ultimately tells Domino not to send any content of its own) to the finally block, this ensures it will get executed. If we left it inside the try block, it would get skipped if an exception occurs, because we'd skip straight to the catch... so even though Domino isn't reporting our exception because we caught it, it still tries to invoke the response writer because we didn't tell it not to.
If you follow the above pattern, and something's wrong with your code, then the browser will receive an incomplete or corrupt file, but the log will tell you what went wrong, rather than reporting an error that has nothing to do with the root cause of the problem.
I almost deleted this question, but decided to answer it myself since there is very little out on google when you search for the exception.
The issue was in the xAgent, there is a line importPackage that was incorrect. Fixing this made everything work. The exception verbage: "Can't get a Writer while an OutputStream is already in use" is quite misleading. I don't know what else triggers this exception, but an alternative description would be "Java class ??yourClass?? not found"
If you found this question, then you likely have the same issue. I would ignore what the exception actually says, and check your package statements throughout your application. The java code will error on its own, but your SSJS that references the java will not error until runtime, focus on that code.
Update the response header after the body can solve this kind of problem, example :
HttpServletResponse response = (HttpServletResponse) facesContext.getExternalContext().getResponse();
response.getWriter().write("<html><body>...</body></html>");
response.setContentType("text/html");
response.setHeader("Cache-Control", "no-cache");
response.setCharacterEncoding("UTF-8");
I'm having trouble understanding how to handle something that it seems like would be a pretty basic aspect of express. If I have some code that throws an exception in an async callback, there is no way I can catch that exception because the try/catch block is no longer in scope by the time the callback is running. In these scenarios the browser will hang until it eventually give up stating that the server is unresponsive. This is a very bad user experience. I would much rather be able to immediately return a 500 error to the client. The default express error handler apparently does not handle this situation. Here is some sample code:
var express = require("express");
var app = express();
app.use(app.router);
//express error handler (never called)
app.use(function(err, req, res, next) {
console.log(err);
res.send(500);
});
app.get("/test", function(req, res, next) {
require("fs").readFile("/some/file", function(err, data) {
a.b(); //blow up
});
});
app.listen(8888);
In the above code, the line a.b() throws a "ReferenceError: a is not defined" exception. The defined error handler is never called. Notice that the err object returned by fs.readFile() is null in this case because the file was correctly read. The bug is the code inside the async handler.
I have read this post about using node's uncaughtExpception even, but the documentation says not use that method. Even if I did use it, how would I then send the 500 response back to the user? The express response object is no longer around for me to use.
So how do you handle this scenario?
OK, I'm just going to post an entirely different answer since my first answer has some valuable information but is off topic in hindsight.
The short answer: the correct thing to do is what already happens: your program should print a stack trace and exit with an error.
The underlying thinking:
So I think you need to think about errors in different categories. My first answer deals with data-related errors that a well-written program can and should handle cleanly. What you are describing is a CRASH. If you read the node.js documentation you linked to, it is correct. The only useful thing your program can do at this point is exit with a stack trace and allow a process supervisor to restart it and attain an understood state. Once your program has crashed, it is essentially unrecoverable due to the extremely wide range of errors that could be the root cause of an exception getting to the top of the stack. In your specific example, this error is just going to continue happening every time until the source code bug is fixed and the application is redeployed. If you are worried that untested and buggy code is going to get into your application, adding more untested and buggy error handling code isn't really addressing the right problem.
But in short, no, there's no way to get a reference to the HTTP request object that caused this exception so AFAIK you cannot change the way this is perceived by the end user in the browser, outside of handling this at an intermediate reverse proxy layer where you could configure a crude timeout and send a more friendly error page (which of course would be useless for any request that isn't for a full HTML document).
The Bible of Error Handling in Node
Error Handling in Node.js by Dave Pacheco is the definitive work on this topic in my opinion. It is comprehensive, extensive, and thorough. I recommend reading and re-reading periodically.
To address #asparagino's comments, if an unhandled exception is easily reproducible or happening with high frequency, it's not an edge case, it's a bug. The correct thing to do is improve your code to not generate uncaught exceptions in face of this situation. Actually handle the condition, thus converting a programmer error into an operational error, and your program can continue without a restart and without an uncaught exception.
You should use express's error handling middleware via app.use(error, req, res, next). Express maintains a separate middleware stack that it uses when the normal middleware stack throws an uncaught exception. (Side note that express just looks at the callback's arity (number of expected arguments) to categorize it as regular middleware or error handling middleware, which is a bit magical, so just keep in mind you must declare the arguments as above for express to understand this is an error handling middleware).
And based on your question and comments, just understand that exceptions aren't all that useful in node.js because every async call gets a new stack, which is why callbacks are used everywhere and the 1st argument is an error universally. Your try/catch block in the example is only going to catch an exception thrown directly by findById (like if id were undefined, as it is in your snippet), but once an actual call to the database is made, that's it, the call stack is over and no further exceptions can happen until a totally different call stack starts when node invokes the async IO callback.
Thanks for the answer, but this only works if I put the try/catch inside the async callback and have the catch do next(exp). I'd like to avoid having separate try/catch blocks in every single async callback.
Nope, that's not true. You don't have to manually call next(exp). Express will catch the error and trigger the error handling middleware for you (that's how express does it's developer-friendly exception report pages in dev mode anyway). And async libraries don't throw exceptions even under "normal" error conditions. They pass an error to the callback, so in general you don't have to bother with try/catch that much in node. Just never ignore an error parameter passed to a callback function, and you're fine.
You don't see this boilerplate in node:
someDb.query(someCriteria, function (error, result) {
try {
//some code to deal with result
} catch (exception) {
callback(exception);
}
});
You do see this though:
someDb.query(someCriteria, function (error, result) {
if (error) {
callback(error);
return;
}
//some code to deal with result
});
Node handles IO differently, which means the call stack works differently, which means exceptions work differently, which means error handling works differently. You can write a stable node/express app that handles errors without crashing without writing a single try/catch. (express has one that handles uncaught errors that bubble all the way to the top). It's not a functional limitation, it's just a consquence of async IO that means you have to write your code differently and handle errors with callbacks instead of exceptions. Thinking of it as a "limitation" as opposed to the "way it is" is putting a negative connotation on something that is really just a technical reality. There are clean and robust patterns for exception handling in both a synchronous and asynchronous paradigm.