How do I optimise this callback hierarchy in my simple express app? - node.js

Question

I'm creating a simple rss feed aggregator. For which, I'm using a module called 'blind-parser'. This modules parses an rss/xml string and returns an object.
My aim was to obtain such objects from multiple rss feeds and render them. So I did this, initially :
var obj1 = {},obj2 = {}, obj3 = {};
app.get('/',function(req,res){
parser.parseURL('https://www.toptal.com/blog.rss', function (err, parsed1) {
obj1 = parsed1;
});
parser.parseURL('https://www.toptal.com/designers/blog.rss', function (err, parsed2) {
obj2 = parsed2;
});
parser.parseURL('http://jsfeeds.com/feed', function (err, parsed3) {
obj3 = parsed3;
});
parser.parseURL('http://www.mironov.com/feed/', function (err, parsed4) {
obj4 = parsed4;
});
res.render('index', {'toptal': obj1 , 'toptaldesign' : obj2 , 'jsf': obj3 , 'product': obj4 });
});
However it didn't work as obj1,obj2,obj3 and obj4 turned out to be undefined.
Which I coudn't understand. I tried declaring the variable inside the app.get and outside, both to no effect.
So I did this instead,
app.get('/',function(req,res){
parser.parseURL('https://www.toptal.com/blog.rss', function (err, parsed1) {
parser.parseURL('https://www.toptal.com/designers/blog.rss', function (err, parsed2) {
parser.parseURL('http://jsfeeds.com/feed', function (err, parsed3) {
parser.parseURL('http://www.mironov.com/feed/', function (err, parsed4) {
res.render('index', {'toptal': parsed1 , 'toptaldesign' : parsed2 , 'rs': parsed3 , 'product': parsed4 });
});
});
});
});
});
and voila it worked! However the major downside to this approach is that the time it takes increases 2x, every time I nest another rss-link.
already,the time it takes to load the page with three-tier nesting is ~20seconds.
Can someone please help me out and offer a solution to this, also I'm new to node so please be elaborate.
Thanks.

Answer 1

I'm assuming the parse.parseURL function is actually fetching the specified URL and parsing its contents. Each of your four parseURL requests is asynchronous, meaning that each function is invoked synchronously, but each of their callbacks is executed at some later point in time once the URL has been fetched etc. Meanwhile your res.render call has also been run synchronously, immediately after the preceding parseURL calls but before each of their callbacks have been invoked, and so before obj1 etc have been set.
When you nest the calls within each callback, as in your second example, then you are ensuring that each URL result is fetched before proceeding, but as you are fetching each URL sequentially (e.g. you don't fetch the second URL until the first has been returned) then the execution time will be slower.
You can possibly speed things up by fetching all four URLs in parallel (assuming that your requests are handled concurrently). To do this, it will be simpler if you first use a library like async, or alternatively, if you convert your code to use promises. If you use async, then your code could be changed to fetch the URLs in parallel like so:
async.parallel([
( cb ) => {
parser.parseURL('https://www.toptal.com/blog.rss', cb );
},
( cb ) => {
parser.parseURL('https://www.toptal.com/designers/blog.rss', cb );
},
( cb ) => {
parser.parseURL('http://jsfeeds.com/feed', cb );
},
( cb ) => {
parser.parseURL('http://www.mironov.com/feed/', cb );
}
],
( err, results ) => {
// Called once all requests have completed.
res.render('index', {'toptal': results[0] , 'toptaldesign' : results[1] , 'jsf': results[2] , 'product': results[3] });
});