Using $.Deferred() with nested $.each statements - sharepoint

Question

I am working with SharePoint making JSOM calls. I need to get results and not move on until ALL data is retrieved. I have tried so many examples (none are complete enough for me to understand how to tailor to my issue with nested $.each loop statements. I seem to get close, but never anything working correctly.
I have edited my actual code (minus the top 3 variables as they are passed in from another page) to incorporate Tomalak's work so it we can figure it out better. Currently, the results are coming up with empty objects. Trying to figure out what I;m doing wrong.
[EDITED 06 AUG 2018]
Finally got it to work. there was only two tiny problems I found with the provided code :-). I will try and bold them.
var fya = [2008,2009]; //Fiscal Year Array which we use to know what lists to look at
var assignedRecords = []; //Global Reusable Variable
var assignedCourses = ['Math','Science','Reading']; //There might not be records who are associated with a particular course in each list. Wee need to pull the student records (assignedRecords) assoiciated with the above in 2008 and 2009.
SP.ClientContext.prototype.executeQueryPromise = function (items) {
var result = $.Deferred();
this.load(items);
this.executeQueryAsync(function (sender, args) { result.resolve(items) },
function (sender, args) { result.reject(args) });
return result.promise();
};
Move 'var arr = [];' outside of arrayFromCollection
Add 'var' to the e variable in that same function
var arr = [];
function arrayFromCollection(coll) {
var e = coll.getEnumerator();
while (e.moveNext()) arr.push(e.get_current());
return arr;
};
function queryListPromise(title, course) {
var list = hostWeb.get_lists().getByTitle(title);
var camlQuery = new SP.CamlQuery();
camlQuery.set_viewXml('<View><Query><Where>'
+ '<Eq><FieldRef Name="Course"/><Value Type="Text">' + course + '</Value></Eq>'
+ '</Where></Query></View>');
return context.executeQueryPromise(list.getItems(camlQuery)).then(arrayFromCollection);
};
function GetAssignedApplications() {
assignedRecords = []; //Need to start empty
var myCourses = assignedCourses;
$.each(myCourses, function (i, myCoursesItem) {
var courseName = myCoursesItem.get_lookupValue();
Change '$.forEach' to '$.each'
$.each(fya, function (n, fyaItem) {
var listTitle = "FY" + String(fyaItem).substring(2); //FY18 & FY19 are the names of the actual lists.
assignedRecords.push(queryListPromise(listTitle, courseName));
});
});
$.when(assignedRecords).then(function (results) {
return Array.prototype.concat.apply([], results);
}).then(function (items) {
items.forEach(function (item) {
var a = item; //item is empty and this actually runs before arrayFromCollection and it returns duplicate records (4) when there is only 2.
});
}).fail(onError);
};

Answer 1

So first things first, waiting on multiple promises in jQuery is done with $.when().
$.when(p1, p2, p3).then(function (results) {
// results is an array of length 3
});
In order to make this work with a variable argument count, we use Function#apply:
var promises = [p1, p2, p3]; // could be be of any length now
$.when.apply($, promises).then(function () {
var results = [].slice.call(arguments);
// results is an array of the same length as promises
});
In order to make this reusable, let's call that $.whenAll:
$.whenAll = function (promises) {
return $.when.apply($, promises).then(function () {
return [].slice.call(arguments);
});
};
Now we can do:
$.whenAll([p1, p2, p3]).then(function (results) {
// results is an array of length 3
});
Next, we need to turn the callback-based executeQueryAsync() into a promise-based function. In general, promisification follows this scheme:
function somethingPromise(args) {
var result = $.Deferred();
somethingAsync(/* onSuccess */ result.resolve, /* onError */ result.reject);
return result.promise();
}
This expects that the onSuccess and onError handlers receive data as arguments. That's not the case with executeQueryAsync - it modifies the items directly. However, we still want to resolve the promise with items, so we need to do that explicitly.
Let's slap that onto the SP.ClientContext prototype right-away for re-use.
SP.ClientContext.prototype.executeQueryPromise = function (items) {
var result = $.Deferred();
this.load(items);
this.executeQueryAsync(
function (sender, args) { result.resolve(items); },
function (sender, args) { result.reject(args); }
);
return result.promise();
};
Next we should set up a helper that turns those unwieldy collections into something usable.
function arrayFromCollection(coll) {
var arr = [], e = coll.getEnumerator();
while (e.moveNext()) arr.push(e.get_current());
return arr;
}
With that all done, we can extract a function that takes a title and returns a promise for an item array.
function queryListPromise(title) {
var list = web.get_lists().getByTitle(title);
var q = new SP.CamlQuery();
//...
return ctx.executeQueryPromise(list.getItems(q)).then(arrayFromCollection);
}
And finally, since you want a flat list of items, we should start with a flat list of queries:
function GetNextFunction() {
var queries = [];
// build flat list of queries
$.each(arr1, function(i, arr1Item) {
$.each(arr2, function(n, arr2Item) {
queries.push(queryListPromise(arr2Item));
});
});
// wait on all queries
$.whenAll(queries).then(function (results) {
// flatten incoming array of arrays
return Array.prototype.concat.apply([], results);
}).then(function (items) {
// items contains all items from all queries
items.forEach(function (item) { /* ... */ });
}).fail(onError);
}
Here queries becomes a flat array of promises, each promise will resolve to an array of items. Therefore, results will be an array of arrays of items.