I was trying to get my hands dirty on advanced NodeJS concepts by Stephen Grinder.
Trying to teach the mere basics of redis, Stephen did something like this
app.get('/api/blogs', requireLogin, async (req, res) => {
//This time we are setting
const redis = require('redis')
const redisURL = 'redis://127.0.0.1:6379';
const client = redis.createClient(redisURL);
const util = require('util')
client.get = util.promisify(client.get)
//We are checking if we have ever fetched any blogs related to the user with req.user.id
const cachedBlog = await client.get(req.user.id)
//if we have stored list of blogs, we will return those
if (cachedBlog) {
console.log(cachedBlog)
console.log("Serving from Cache")
return res.send(JSON.parse(cachedBlogs))
} //this is JSONIFIED as well so we need to convert it into list of arrays
console.log("serving from Mongoose")
//if no cache exsist
const blogs = await Blog.find({_user: req.user.id})
//blogs here is an object so we would need to stringfy it
res.send(blogs);
client.set(req.user.id, JSON.stringify(blogs))
})
And it works without any error but in last two lines, if we change the order
client.set(req.user.id, JSON.stringify(blogs))
res.send(blogs);
it does not display my blog.
Since inside the API, I am considering both of them to run asynchronously, I thought order won't matter.
Can anyone tell me what am I missing or unable to comprehend?
Since OP asks to understand the difference, not fix the code:
express runs the request handler function and catches synchronous errors (they become http500 errors). It doesn't do anything with the promise returned from the async function and doesn't use await internally, so you don't get error handling for async functions for free. All asynchronous errors need to be caught inside and passed to the next callback or handled in your code by sending an appropriate status code and error body.
When an error occurs, JS stops and doesn't execute any more lines in the function. So if an error is thrown from client.set placed before res.send, the line with send won't run and no response is sent. The browser should continue waiting for the response until timeout.
The other way around - you send response before the error, so you get the page, but the response doesn't end (I'd assume the connection remains open as if the backend was going to send more) but ever since early versions of Firefox browsers start rendering HTML as it's downloaded, so you see a page even though the browser is still waiting for the response to finish.
The order of these two lines doesn't matter but that res.send isn't called in case client.set goes first means that there's an error. If an error occurs in async function, this may result in UnhandledPromiseRejectionWarning warning that will be visible in console.
There are several problems with this snippet.
That the error occurs even when though client.set is asynchronous suggests that client.set causes synchronous error which wasn't caught.
client.set wasn't promisified but it should for correct control flow. That it wasn't provided with callback argument could be a reason why it caused an error.
As explained in this answer, Express doesn't support promises, all rejections should be explicitly handled for proper error handling.
All common code like require goes outside middleware function. It should be:
const redis = require('redis')
const redisURL = 'redis://127.0.0.1:6379';
const client = redis.createClient(redisURL);
const util = require('util')
client.get = util.promisify(client.get)
client.set = util.promisify(client.set)
app.get('/api/blogs', requireLogin, async (req, res, next) => {
try {
const cachedBlog = await client.get(req.user.id)
if (cachedBlog) {
return res.send(JSON.parse(cachedBlogs))
}
const blogs = await Blog.find({_user: req.user.id});
await client.set(req.user.id, JSON.stringify(blogs));
res.send(blogs);
} catch (err) {
next(err);
}
})
Most popular libraries have promise counterparts that allow to skip boilerplate promisification code, this applies to redis as well.
The two task will runs asynchronously but the order of execution matters.
client.set(req.user.id, JSON.stringify(blogs)) execution starts first, but as you are not using await, the promise will not be resolved but execution has already started.
After that res.send() will execute.
You are not getting the response implies that there is some error in the execution of client.set(req.user.id, JSON.stringify(blogs)).
Use Try catch block to trace this error (as mentioned in other answers).
You can also add these lines in your code to catch other "unhandledRejection" or "uncaughtException" error (if any).
process.on('unhandledRejection', (err) => {
logger.error('An unhandledRejection error occurred!');
logger.error(err.stack)
});
process.on('uncaughtException', function (err) {
logger.error('An uncaught error occurred!');
logger.error(err.stack);
});
Related
Working on a MERN stack project. When trying to access the site on localhost on iPhone Posts are posts are not loading. Working fine on adnroid devices.
Screenshot of error
const fetchFeedPosts = async () => {
const URL = `${BASE__URL}api/posts/feed/${LOGGED__IN__USER}`;
await axios.get(URL)
.then((response) => setFeedPosts([...response.data].reverse()))
.catch((e) => console.log(e.response));
}
fetchFeedPosts()
What the error means
When an Error is thrown (e.g. throw new Error()) it can
be catched locally (e.g. try{...} catch(err) { /** here */ })
or be passed on to the calling function. And in that fashion it bubbles up from caller to caller until it reaches a catch somewhere.
However, if it continues to bubble up, and there's nothing that captures it by the time it reaches the root of your application or code, then that will result in a so-called "Unhandled" error.
Now, as you may know, promises are more like jobs that drift around. They aren't called from the root of your application. But as they bubble up they can also reach a similar root-point, in which case they become "Unhandled Promise rejections".
What to do about it
Unhandled errors or rejections are bad practice though. Errors should be caught somewhere. And without catching them, you can't really know what has caused the error to happen in the first place.
In most cases, you can catch them with a .catch() function, (e.g. yourPromise.catch((err) => {console.err(err)}))
In case your promise is handled in an async function and waited for with an await keyword, then it's slightly different. In that case it makes more sense to use a try-catch block to capture your error.
How to apply it to your code
So, the first way of doing it would be to use the .catch() function
axios.get(URL)
.then((response) => setFeedPosts([...response.data].reverse()))
.catch((err) => console.error(err));
The alternative is to use the await syntax with a try-catch. If you want to use this syntax, you have to put the async keyword before your function.
try {
const response = await axios.get(URL)
setFeedPosts([...response.data].reverse()))
} catch (err) {
console.log(err);
}
Sure, you could mix the 2, but in most cases that would be rather strange.
Assume an Express route that makes a call to Mongoose and has to be async so it can await on the mongoose.find(). Also assume we are receiving XML but we have to change it to JSON, and that also needs to be async so I can call await inside of it.
If I do this:
app.post('/ams', async (req, res) => {
try {
xml2js.parseString(xml, async (err, json) => {
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
} catch(err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
}
The server hangs forever. I'm assuming the async/await means that the server hits a timeout before something concludes.
If I put this:
res.status(200);
res.send("Data saved")
on the line before the catch(), then that is returned, but it is the only thing every returned. The client gets a 200, even if an XMLException is thrown.
I can see the XMLException throw in the console, but I cannot get a 400 to send back. I cannot get anything I that catch block to execute in a way that communicates the response to the client.
Is there a way to do this?
In a nutshell, there is no way to propagate an error from the xml2js.parseString() callback up to the higher code because that parent function has already exited and returned. This is how plain callbacks work with asynchronous code.
To understand the problem here, you have to follow the code flow for xml2js.parseString() in your function. If you instrumented it like this:
app.post('/ams', async (req, res) => {
try {
console.log("1");
xml2js.parseString(xml, async (err, json) => {
console.log("2");
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
console.log("3");
} catch (err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
console.log("4");
});
Then, you would see this in the logs:
1 // about to call xml2js.parseString()
3 // after the call to xml2js.parseString()
4 // function about to exit
2 // callback called last after function returned
The outer function has finished and returned BEFORE your callback has been called. This is because xml2js.parseString() is asynchronous and non-blocking. That means that calling it just initiates the operation and then it immediately returns and the rest of your function continues to execute. It works in the background and some time later, it posts an event to the Javascript event queue and when the interpreter is done with whatever else it was doing, it will pick up that event and call the callback.
The callback will get called with an almost empty call stack. So, you can't use traditional try/catch exceptions with these plain, asynchronous callbacks. Instead, you must either handle the error inside the callback or call some function from within the callback to handle the error for you.
When you try to throw inside that plain, asynchronous callback, the exception just goes back into the event handler that triggered the completion of the asynchronous operation and no further because there's nothing else on the call stack. Your try/catch you show in your code cannot catch that exception. In fact, no other code can catch that exception - only code within the exception.
This is not a great way to write code, but nodejs survived with it for many years (by not using throw in these circumstances). However, this is why promises were invented and when used with the newer language features async/await, they provide a cleaner way to do things.
And, fortunately in this circumstance xml2js.parseString() has a promise interface already.
So, you can do this:
app.post('/ams', async (req, res) => {
try {
// get the xml data from somewhere
const json = await xml2js.parseString(xml);
// do something with json here
res.send("Data saved");
} catch (err) {
console.log(err);
res.status(400).send("Malformed XML error: " + err.message);
}
});
With the xml2js.parseString() interface, if you do NOT pass it a callback, it will return a promise instead that resolves to the final value or rejects with an error. This is not something all asynchronous interfaces can do, but is fairly common these days if the interface had the older style callback originally and then they want to now support promises. Newer interfaces are generally just built with only promise-based interfaces. Anyway, per the doc, this interface will return a promise if you don't pass a callback.
You can then use await with that promise that the function returns. If the promise resolves, the await will retrieve the resolved value of the promise. If the promise rejects, because you awaiting the rejection will be caught by the try/catch. FYI, you can also use .then() and .catch() with the promise, but in many cases, async and await are simpler so that's what I've shown here.
So, in this code, if there is invalid XML, then the promise that xml2js.parseString() returns will reject and control flow will go to the catch block where you can handle the error.
If you want to capture only the xml2js.parseString() error separately from other exceptions that could occur elsewhere in your code, you can put a try/catch around just it (though this code didn't show anything else that would likely throw an exception so I didn't add another try/catch). In fact, this form of try/catch can be used pretty much like you would normally use it with synchronous code. You can throw up to a higher level of try/catch too.
A few other notes, many people who first start programming with asynchronous operations try to just put await in front of anything asynchronous and hope that it solves their problem. await only does anything useful when you await a promise so your asynchronous function must return a promise that resolves/rejects when the asynchronous operation is complete for the await to do anything useful.
It is also possible to take a plain callback asynchronous function that does not have a promise interface and wrap a promise interface around it. You pretty much never want to mix promise interface functions with plain callback asynchronous operations because error handling and propagation is a nightmare with a mixed model. So, sometimes you have to "promisify" an older interface so you can use promises with it. In most cases, you can do that with util.promisify() built into the util library in nodejs. Fortunately, since promises and async/await are the modern and easier way to do asynchronous things, most newer asynchronous interfaces in the nodejs world come with promise interfaces already.
You are throwing exceptions inside the callback function. So you cant expect the catch block of the router to receive it.
One way to handle this is by using util.promisify.
try{
const util = require('util');
const parseString = util.promisify(xml2js.parseString);
let json = await parsestring(xml);
}catch(err)
{
...
}
I am struggling to figure out what the error is in this async function. I keep getting this error message:
"Unexpected token catch"
and if that error is fixed I get this error Message:
"UnhandledPromiseRejectionWarning"
"DeprecationWarning"
router.get('/Views', async (req, res) => {
console.log("Google auth ", googleAuth)
const organizationId = req.verifiedToken.OrganizationId;
console.log("Got here")
const url = myUrl;
try{
const client = await auth.getClient({
scopes: [
'https://www.googleapis.com/auth/analytics.readonly'
]
})catch(err){
console.log(err);
throw err;
};
const outcome = await client.request({ url })catch(err){
console.log(err);
throw err;
};
};
console.log("Successfully connected!", organizationId);
return responses.success(res, outcome.data);
});
The line
const outcome = await client.request({ url })catch(err){
introduces a catch exception handler without a prior try block. You appear to have caught (no pun intended) this syntax error (though you haven't detailed the code changes to get rid of it).
Unfortunately you haven't posted a self-contained code fragment and you haven't specified which framework/libraries you use on top of node.js. If you are using the Express framework and an ajax library, it might be that your try block is missing an exception handler and the catch statements are meant to be method calls:
router.get('/Views', async (req, res) => {
console.log("Google auth ", googleAuth)
const organizationId = req.verifiedToken.OrganizationId;
console.log("Got here")
const url = myUrl;
try{
const client = await auth.getClient({
scopes: [
'https://www.googleapis.com/auth/analytics.readonly'
]
})
.catch(err){ //*** syntax error in your code sample
console.log(err);
throw err;
};
const outcome = await client.request({
url
})
.catch(err){ //*** syntax error in your code sample
console.log(err);
throw err;
};
} catch (e) { //*** exception handler missing from your original code missing
// exception handling code, possibly empty; there are the .catch calls after all
}
console.log("Successfully connected!", organizationId);
return responses
.success ( res, outcome.data )
.error ( )
//*** This call is missing from your code.
// It is assumed that 'responses' holds a "Promise", see explanation
;
});
A 'Promise' in asynchronous programming is an abstraction of a value not yet known (think of it as a placeholder for that value). There are 2 basic possible scenarios: either that value will eventually be computed (#1) or it is ascertained that it will never be computed at all (#2). A promise library handles these scenarios. The reference API sports promise objects with a .then ( fn_ok, fn_fail ) method taking 2 functions as arguments, one being associated the first scenario, one with the second. As soon as a scenario is established, the respective function will be called. Promise libraries may add additional layers of abstraction, possibly producing the .success/.error calls from the code sample. Note that you promise libraries do usually support 'chaining': In the code sample, the call to .success (.error) would actually make sure that res and outcome.data (nothing) will be preserved and fed to the handler for the scenario #1 (#2) and would return a promise (technically the same object with some properties being redefined).
The second error you have received ( UnhandledPromiseRejectionWarning ) would thus stem from not handling scenario #2. However, the case that the future computation of a value will fail is a very possible outcome and should be accounted for in general. Otherwise your code is amenable to run-time errors that are very hard to track down as you will neither be notified of the code section the error occurs nor (due to async programming) will you have guarantees on the execution order of code fragments. Moreover this coding style easily rsults in a deadlock or the eventual exhaustion of some resource (memory, file handles, ...)
This explanation is deliberately written following a (hopefully) intuitive approach and is technically somewhat sloppy. For a detailed explanation consult https://www.promisejs.org/, MDN: Promises, or similar resources.
Caveat
There is some guesswork involved in this answer. However the general outline of the problem's origin should remain valid.
I spent too much time trying to figure out why my express.js controller did not respond to a simple query, and figured out that runtime errors fired from a Mongoose-promise callback were silently interrupting the callback process.
Here is a simplified version of my code:
server.get('/api/test', function (req, res, next) {
User.find({}).exec().then(function success(users){
console.log('SUCCESS');
typo[0] = 1; // throws a runtime error
res.json(users);
}, function error(err){
console.log('ERROR');
res.json({error: err});
});
});
This results in SUCCESS showing up in my console, but nothing happens then. No response is given to the user, the error caused by my typo is not appearing in my console, and the error callback is not called either.
I am aware that one should not throw exceptions from a callback function, but in that case, this was just a typo, and it would make sense to me to be warned (e.g. a stack trace in my standard output) whenever one makes this kind of mistake. (we're humans, after all...)
In your opinion, what's the best way to get feedback whenever this kind of mistakes are made in promise callbacks?
This is Mongoose's fault for using a bad promise implementation. Promises are throw-safe so exceptions are caught (so they can be later handled by future code) - the future code never comes and Mongoose never reports that it did not. Good promise implementations do not suffer from this issue.
Your options are two:
Use a library like Bluebird:
var Promise = require("bluebird");
var mongoose = Promise.promisifyAll(require("mongoose"));
User.findAsync({}).then(function(data){
JSON.prase("dsa"); // not a silent failure, will show up, easy debugging
});
This has the advantage of being faster than mongoose promises so there is no performance penalty. Alternatively, if you're super conservative and don't want the performance and API gains of bluebird - you can use native promises:
// Promise is the native promise
Promise.resolve(User.find({}).exec()).then(function(data){
JSON.prase("dsa");
});
And then, assuming you're running a modern variant of nodejs (read: io.js v 1.4.1 or upper), you can subscribe to promise rejections:
process.on("unhandledRejection", function(p, why){
console.log("FOUND ERROR!!!!", p , why);
});
So exceptions are not silently suppressed.
The exec() has two promises
.then(function)
.then(null , function)
try this, I think it will help
server.get('/api/test', function(req, res, next) {
User.find({}).exec()
.then(function success(users) {
console.log('SUCCESS');
typo[0] = 1; // throws a runtime error
res.json(users);
})
.then(null, function error(err) {
console.log('ERROR');
res.json({
error: err
});
});
});
I want to test the error in a request return. I'm using nock in my tests, how can I force Nock to provoke an error? I want to achieve 100% test coverage and need to test err branch for that
request('/foo', function(err, res) {
if(err) console.log('boom!');
});
Never enter in the if err branch. Even if hit err is a valid response, my Nock line in test looks like this
nock('http://localhost:3000').get('/foo').reply(400);
edit:
thanks to some comments:
I'm trying to mock an error in the request. From node manual:
https://nodejs.org/api/http.html#http_http_request_options_callback
If any error is encountered during the request (be that with DNS resolution, TCP level errors, or actual HTTP parse errors) an 'error' event is emitted on the returned request object
An error code (e.g. 4xx) doesn't define the err variable. I'm trying to mock exactly that, whatever error that defines the err variable and evaluates to true
Use replyWithError.
From the docs:
nock('http://www.google.com')
.get('/cat-poems')
.replyWithError('something awful happened');
When you initialise a http(s) request with request(url, callback), it returns an event emitter instance (along with some custom properties/methods).
As long as you can get your hands on this object (this might require some refactoring or perhaps it might not even be suitable for you) you can make this emitter to emit an error event, thus firing your callback with err being the error you emitted.
The following code snippet demonstrates this.
'use strict';
// Just importing the module
var request = require('request')
// google is now an event emitter that we can emit from!
, google = request('http://google.com', function (err, res) {
console.log(err) // Guess what this will be...?
})
// In the next tick, make the emitter emit an error event
// which will trigger the above callback with err being
// our Error object.
process.nextTick(function () {
google.emit('error', new Error('test'))
})
EDIT
The problem with this approach is that it, in most situations, requires a bit of refactoring. An alternative approach exploits the fact that Node's native modules are cached and reused across the whole application, thus we can modify the http module and Request will see our modifications. The trick is in monkey-patching the http.request() method and injecting our own bit of logic into it.
The following code snippet demonstrates this.
'use strict';
// Just importing the module
var request = require('request')
, http = require('http')
, httpRequest = http.request
// Monkey-patch the http.request method with
// our implementation
http.request = function (opts, cb) {
console.log('ping');
// Call the original implementation of http.request()
var req = httpRequest(opts, cb)
// In next tick, simulate an error in the http module
process.nextTick(function () {
req.emit('error', new Error('you shall not pass!'))
// Prevent Request from waiting for
// this request to finish
req.removeAllListeners('response')
// Properly close the current request
req.end()
})
// We must return this value to keep it
// consistent with original implementation
return req
}
request('http://google.com', function (err) {
console.log(err) // Guess what this will be...?
})
I suspect that Nock does something similar (replacing methods on the http module) so I recommend that you apply this monkey-patch after you have required (and perhaps also configured?) Nock.
Note that it will be your task to make sure you emit the error only when the correct URL is requested (inspecting the opts object) and to restore the original http.request() implementation so that future tests are not affected by your changes.
Posting an updated answer for using nock with request-promise.
Let's assume that your code calls request-promise like this:
require('request-promise')
.get({
url: 'https://google.com/'
})
.catch(res => {
console.error(res);
});
you can set up nock like this to simulate a 500 error:
nock('https://google.com')
.get('/')
.reply(500, 'FAILED!');
Your catch block would log a StatusCodeError object:
{
name: 'StatusCodeError',
statusCode: 500,
message: '500 - "FAILED!"',
error: 'FAILED!',
options: {...},
response: {
body: 'FAILED!',
...
}
}
Your test can then validate that error object.
Looks like you're looking for an exception on a nock request, this maybe can help you:
var nock = require('nock');
var google = nock('http://google.com')
.get('/')
.reply(200, 'Hello from Google!');
try{
google.done();
}
catch (e) {
console.log('boom! -> ' + e); // pass exception object to error handler
}