I am using Blubird and Sequelize (which uses Blubird under the covers).
Suppose I have a code similar to:
Feed.findAll()
.map(function (feed) { // <---- this is what I'm interested in below
// do some stuff here
return some_promise_here;
})
.map(function (whatever) {
// What is the best way to access feed here?
})
....
I have found some replies which hinted at possible solutions, but I can't quite put my finger on it.
I have tried with Promise.all(), .spread(), but I never managed to make it work.
Feed.findAll()
.map(function (feed) { // <---- this is what I'm interested in below
// do some stuff here
return some_promise_here.then(function(result){
return { result: result, feed: feed};// return everything you need for the next promise map below.
});
})
.map(function (whatever) {
// here you are dealing with the mapped results from the previous .map
// whatever -> {result: [Object],feed:[Object]}
})
This looks very similar to How do I access previous promise results in a .then() chain?, however you're dealing with a .map call here and seem to want to access the previous result for the same index of the processed array. In that case, not all solutions do apply, and a closure seems to be the simplest solution:
Feed.findAll().map(function (feed) {
// do some stuff here
return some_promise_here.then(function (whatever) {
// access `feed` here
});
})
You can apply explicit pass-through as well, though, like outlined in #bluetoft's answer.
Related
i've worked with node now for 2 years but cannot solve the following requirements:
I have an array of ~ 50.000 Parameters
I need to loop through the array and make a get request to always the same url with the parameter added
I need to write the result of the url-call back to the array
It's needed to do this one by one, as i can not call the api with several threads.
I'm sure there is a simple solution for that but everything i tried didn't make the code wait for the get request to return. I know that doing things synchronous in node is not the way we should to things, but in this special situation it is by design that the process shall not go on till the result comes back.
Any hint appreciated
Regards
Use a for loop, use a means of doing the GET request that returns a promise (such as the got() library) and then use await to pause the for loop until your response comes back.
const got = require('got');
const yourArray = [...];
async function run() {
for (let [index, item] of yourArray.entries()) {
try {
let result = await got(item.url);
// do something with the result
} catch(e) {
// either handle the error here or throw to stop further processing
}
}
}
run().then(() => {
console.log("all done");
}).catch(err => {
console.log(err);
});
the console.log(workingWeekdaysVar) line; is outside the findOne's scope, and the variable was declared outside it too, yet it's giving me null ...
when i put console.log(workingWeekdaysVar); inside the findOne's scope, it does give the right output, but this is useless for me because I wanna use workingWeekdaysVar elsewhere below.
The two commented out lines are the 2nd approach i attempted to do, but it gave me an undesirable output because this whole code is inside a complicated for loop.
How can I simply pass the fetched value of workingWeekdaysVar out of the scope?
var workingWeekdaysVar = [];
buyerSupplierFisModel.findOne(filter).then(function (combo) {
workingWeekdaysVar = combo.workingWeekdays;
//server.getWorkingWeekdays = function () { return combo.workingWeekdays };
});
console.log(workingWeekdaysVar);
//console.log(server.getWorkingWeekdays());
findOne() is an asynchronous function (it returns a promise object). This means that it returns inmediately and your next code line is run (in this case, console.log(workingWeekdaysVar);. But, since the function isn't done yet, workingWeekdaysVar is empty, and it will be empty until findOne() has done its job and returned the results in the provided chained callback .then(function (combo) {....
So if you want to do anything with the results, you'll have to do it in the callback. One option to this would be to use async / await:
(async () => {
try {
const { workingWeekdaysVar } = await buyerSupplierFisModel.findOne(filter)
console.log(workingWeekdaysVar)
} catch (e) {
console.log(`Error: ${e}`);
}
})()
Re-arranging your code a bit:
let doToResponse = (combo)=>{
workingWeekdaysVar = combo.workingWeekdays;
console.log(workingWeekdaysVar);
}
buyerSupplierFisModel.findOne(filter).then(function (combo) {
doToResponse(combo)
});
good for re-usability
My personal favorite:
buyerSupplierFisModel.findOne(filter).then(combo=> {
workingWeekdaysVar = combo.workingWeekdays;
console.log(workingWeekdaysVar);
});
The important thing is keep in mind, as Miguel Calderón says.. findOne - returns a promise. At that point you have another thread with different local (Lexical?) scope
After a lot of googling I have not been able to confirm the correct approach to this problem. The following code runs as expected but I have a grave feeling that I am not approaching this in the correct way, and I am setting myself up for problems.
The following code is initiated by the main app.js file and is passed a location to start loading XML files from and processing into a mongoDB
exports.processProfiles = function(path) {
var deferrer = q.defer();
q(dataService.deleteProfiles()) // simple mongodb call to empty the Profiles collection
.then(function(deleteResult) {
return loadFilenames(path); // method to load all filenames in the given path using fs
})
.then(function(filenames) {
// now we have all the file names lets load and save
filenames.forEach(function(filename) {
// Here is where i think the problem is!
// kick off another promise chain for the dynamically sized array of files to process
q(loadFileContent(path, filename)) // first we load the data in the file
.then(function(inboundFile) {
// then parse XML structure to my new shiny JSON structure
// and ask Mongo to store it for me
return dataService.createProfile(processProfileXML(filename, inboundFile));
})
.done(function(result) {
console.log(result);
})
});
})
.catch(function(err) {
deferrer.reject('Unable to Process Profile records : ' + err);
})
.done(function() {
deferrer.resolve('Profile Processing Completed');
});
return deferrer.promise;
}
Whilst this code works these are my main concerns but cannot solve them on my own after a few hours of Google and reading.
1) Is this blocking? The read out to the console is difficult to understand if this is running asynchronously as i want it to - i think it is but advice on if I am doing something fundamentally wrong would be great
2) Is having a nested promise a bad idea, should I be linking it to the outter promise - I have tried but could not get anything to compile or run.
I haven't used Q in a really long time, but I think that you'd need to do is let it know you're about to hand back an array of promises that need to all be satisfied before moving on.
Additionally as you're waiting for multiple promises on one section of code, rather than nesting further, throw the 'set' of promises back up once they're all satisfied.
q(dataService.deleteProfiles()) // simple mongodb call to empty the Profiles collection
.then(function (deleteResult) {
return loadFilenames(path); // method to load all filenames in the given path using fs
})
.then(function (filenames) {
return q.all(
filenames.map(function (filename) {
return q(loadFileContent(path, filename)) { /* Do stuff with your filenames */ });
})
);
.then(function (resultsOfLoadFileContentsPromises) {
console.log('I did stuff with all the things');
)
.catch(function(err) {});
What you have is not 'blocking'. But really what you're doing with promises is moving things into a new 'block'ing section. The more blocks you have, the more async-ish your code will appear. If nothing else is running apart from this promise, it will still appear procedural.
But inner promises must still resolve before the parent promises resolve thereafter.
Inner promises like what you have aren't an inherently bad, personally I will break them out into seperate files to makes easier to reason about, but I wouldn't define that as 'bad' unless there's no need for that inner promise to exist, however where possible (and in your example here) I've adjusted so I throw back up the next set of promises for a new section to deal with the data after it's gotten it.
(I'm not great with Q though, this code will probably require a little further tweaking).
see codes below:
define(["dep1"], function (dep1, next) {
// necessary works here so that `dep2` is ready
// kind of circular dependency.
someNecessaryWork();
require(["dep2"], function (dep2) {
// return value in node-style, using `next()`
next(someValue);
}
// do not return value as normal, no `return someValue` here
}
Is require.js able to do this? For now I'm using functions to achieve this.
define(["dep1", "dep2Fn"], function (dep1, dep2Fn) {
someNecessaryWork();
dep2Fn();
return someValue;
});
but it feels not intuitive.
define does not allow you to set the return value of a module through a callback. What you show in your second snippet is how you do it. You just have to get used to it.
I'm using the Q library in nodejs and haven't worked too much with promises in the past, but I have semi complex logic that requires lots of nesting and thought Q would be a good solution, however I'm finding that it seems to be almost the same as just "callback hell".
Basically I have say 5 methods, all which require data from the previous or one of the previous. Here's an example:
We start with some binary data that has a sha1 hash generated based on the binary.
var data = {
hash : "XXX"
, binary: ''
}
First we want to see if we already have this, using this method:
findItemByHash(hash)
If we don't have it, we need to save it, using:
saveItem(hash)
Now we need to associate this to a user, but not only the results of the save. There's now a much larger hierarchy that we associate, so we need to get that first, doing:
getItemHierarchy(item_id), we use the item_id returned from our previous saveItem
Now, we can "copy" these results to a user:
saveUserHierarchy(hierarchy)
Now we're done, however, this assumes the item didn't exist yet. So we need to handle a case where the item did exist. This would be:
We need to check if the user may aleady have this:
getUserItemByItemId(item_id) - item_id was returned from findItemByHash
If it exists, we're done.
If it doesn't:
getItemHierarchy(item_id)
Then
saveUserHierarchy(hierarchy)
Ok, so right now we have callbacks that do these checks, which is fine. But we need to handle errors in each case along the way. That's fine too, just adds to the mess. Really, if any part of the flow throws an error or rejects then it can stop and just handle it in a single place.
Now with Q, we could do something like this:
findItemByHash(hash).then(function(res) {
if (!res) {
return saveItem(hash).then(function(item) {
return getItemHierarchy(item.id).then(function(hierarchy) {
return saveUserHierarchy(hierarchy);
});
})
} else {
return getUserItemByItemId(res.id).then(function(user_item) {
if (user_item) {
return user_item;
}
return getItemHierarchy(res.id).then(function(hierarchy) {
return saveUserHierarchy(hierarchy);
});
});
}
})
//I think this will only handle the reject for findItemByHash?
.fail(function(err) {
console.log(err);
})
.done();
So, I guess my question is this. Are there better ways to handle this in Q?
Thanks!
One of the reasons why I love promises is how easy it is to handle errors. In your case, if any one of those promises fail, it will be caught at the fail clause you have defined. You can specify more fail clauses if you want to handle them on the spot, but it isn't required.
As a quick example, sometimes I want to handle errors and return something else instead of passing along the error. I'll do something like this:
function awesomeFunction() {
var fooPromise = getFoo().then(function() {
return 'foo';
}).fail(function(reason) {
// handle the error HERE, return the string 'bar'
return 'bar';
});
return fooPromise;
}
awesomeFunction().then(function(result) {
// `result` will either be "foo" or "bar" depending on if the `getFoo()`
// call was successful or not inside of `awesomeFunction()`
})
.fail(function(reason) {
// This will never be called even if the `getFoo()` function fails
// because we've handled it above.
});
Now as for your question on getting out of "return hell" - as long as the next function doesn't require information about the previous one, you can chain .then clauses instead of nesting them:
doThis().then(function(foo) {
return thenThis(foo.id).then(function(bar) {
// `thenThat()` doesn't need to know anything about the variable
// `foo` - it only cares about `bar` meaning we can unnest it.
return thenThat(bar.id);
});
});
// same as the above
doThis().then(function(foo) {
return thenThis(foo.id);
}).then(function(bar) {
return thenThat(bar.id);
});
To reduce it further, make functions that combine duplicate promise combinations and we're left with:
function getItemHierarchyAndSave(item) {
return getItemHierarchy(item.id).then(function(hierarchy) {
return saveUserHierarchy(hierarchy);
});
}
findItemByHash(hash).then(function(resItem) {
if (!resItem) {
return saveItem(hash).then(function(savedItem) {
return getItemHierarchyAndSave(savedItem);
});
}
return getUserItemByItemId(resItem.id).then(function(userItem) {
return userItem || getItemHierarchyAndSave(resItem);
});
})
.fail(function(err) { console.log(err); })
.done();
Disclaimer: I don't use Q promises, I perfer when promises primarily for the extra goodies it comes with, but the principles are the same.