I'm using the async library to help me with my control flow. I have a collection over which I want to iterate, for each element execute 1 asynchronous task and when all are done, call the callback.
I've decided to use an async.forEach loop, on each loop I call my asynchronous task but I get an error: callback was already called, but shouldn't the callback be called only when all callbacks are called? And I even wanted to understand properly how to handle errors, it is highly probable that some task will fail and others will succeed, I don't need to know which elements fail, but I would like, how can I do this?
This is my code:
async.forEach(fonts, function(font, callback) {
ftpm_module.installOsFont(font, callback);
}, function() {
console.log("finished");
});
EDIT: the error occurs only if I pass 2 or more fonts.
Related
I am still quite new to Node.js and can't seem to find anything to help me around this.
I am having an issue of getting the query from my last record and adding it to my variable.
If I do it like below: -
let lastRecord = Application.find().sort({$natural:-1}).limit(1).then((result) => { result });
Then I get the value of the variable showing in console.log as : -
Promise { <pending> }
What would I need to do to output this correctly to my full data?
Here is it fixed:
Application.findOne().sort({$natural:-1}).exec().then((lastRecord) => {
console.log(lastRecord); // "lastRecord" is the result. You must use it here.
}, (err) => {
console.log(err); // This only runs if there was an error. "err" contains the data about the error.
});
Several things:
You are only getting one record, not many records, so you just use findOne instead of find. As a result you also don't need limit(1) anymore.
You need to call .exec() to actually run the query.
The result is returned to you inside the callback function, it must be used here.
exec() returns a Promise. A promise in JavaScript is basically just a container that holds a task that will be completed at some point in the future. It has the method then, which allows you to bind functions for it to call when it is complete.
Any time you go out to another server to get some data using JavaScript, the code does not stop and wait for the data. It actually continues executing onward without waiting. This is called "asynchronisity". Then it comes back to run the functions given by then when the data comes back.
Asynchronous is simply a word used to describe a function that will BEGIN executing when you call it, but the code will continue running onward without waiting for it to complete. This is why we need to provide some kind of function for it to come back and execute later when the data is back. This is called a "callback function".
This is a lot to explain from here, but please go do some research on JavaScript Promises and asynchronisity and this will make a lot more sense.
Edit:
If this is inside a function you can do this:
async function someFunc() {
let lastRecord = await Application.findOne().sort({$natural:-1}).exec();
}
Note the word async before the function. This must me there in order for await to work. However this method is a bit tricky to understand if you don't understand promises already. I'd recommend you start with my first suggestion and work your way up to the async/await syntax once you fully understand promises.
Instead of using .then(), you'll want to await the record. For example:
let lastRecord = await Application.find().sort({$natural:-1}).limit(1);
You can learn more about awaiting promises in the MDN entry for await, but the basics are that to use a response from a promise, you either use await or you put your logic into the .then statement.
I needed to compare two arrays the first one a couple of filenames from a database, the second one a list of files I already downloaded to my client. The Idea was to load whatever files are missing on the client.
As the reading via fswas two slow, I tried using Promises to wait for one function to finish before the next starts. But somehow I got lost...
My code so far:
let filesIneed = [];
let filesIhave = [];
let filesToFetch = [];
getLocalFiles().then(getFilesIneed).then(getfilesToRetreive);
function getLocalFiles() {
fs.readdir(localPath, (err, files) => {
files.forEach(file => {
filesIhave.push(file)
});
})
return Promise.all(filesIhave);
}
function getFilesIneed () {
for (let x of docs) {//this is my JSON
filesIneed.push(y.NameOfFileIShouldHave);
}
}
return Promise.all(filesIneed);
}
function getfilesToRetreive() {
filesToFetch = _.difference(filesIneed, filesIhave);
return Promise.all(filesToFetch);
}
console.log(filesToFetch);
I do get the first and second array ("filesIneed" and "filesIhave"), but difference is always empty. So maybe I just mangled up the Promises, as this concept is completely new to me and I'm aware I only understood half of it.
This is completely wrong. You cannot run Promise.all on an array of filenames. You can only run it on an array of promises.
There is also no need to push every element of an array one at a time to an empty array just to return that array when you already have that array in the first place.
You cannot use promises to compare two arrays. You can use lodash to compare two arrays in a then handler of a promise, that resolves to an array.
If you want to get a promise of file names from the fs.readdir then use one of the following modules:
https://www.npmjs.com/package/mz
http://bluebirdjs.com/docs/api/promise.promisifyall.html
https://www.npmjs.com/package/fs-promise
https://www.npmjs.com/package/fs-promised
Also don't use global variables for everything because you will have problems with any concurrency.
Also, read about promises. Without understanding how promises work you will not be able to guess a correct way of using them. Even looking at some working code examples can help a lot and there are a lot of questions and answers on stack Overflow about promises:
promise call separate from promise-resolution
Q Promise delay
Return Promise result instead of Promise
Exporting module from promise result
What is wrong with promise resolving?
Return value in function from a promise block
How can i return status inside the promise?
Should I refrain from handling Promise rejection asynchronously?
Is the deferred/promise concept in JavaScript a new one or is it a traditional part of functional programming?
How can I chain these functions together with promises?
Promise.all in JavaScript: How to get resolve value for all promises?
Why Promise.all is undefined
function will return null from javascript post/get
Use cancel() inside a then-chain created by promisifyAll
Why is it possible to pass in a non-function parameter to Promise.then() without causing an error?
Implement promises pattern
Promises and performance
Trouble scraping two URLs with promises
http.request not returning data even after specifying return on the 'end' event
async.each not iterating when using promises
jQuery jqXHR - cancel chained calls, trigger error chain
Correct way of handling promisses and server response
Return a value from a function call before completing all operations within the function itself?
Resolving a setTimeout inside API endpoint
Async wait for a function
JavaScript function that returns AJAX call data
try/catch blocks with async/await
jQuery Deferred not calling the resolve/done callbacks in order
Returning data from ajax results in strange object
javascript - Why is there a spec for sync and async modules?
Return data after ajax call success
I realize that the standard practice for promises in Mongoose is to use exec(), but the following works (or at least appears to) and I want to understand the flow. I'm not against using exec, I'm just exploring this a bit to learn.
In the following Mongoose operation:
let id:string;
SomeDocument.remove({}, (err) => { //clears collection
someDoc = new SomeDocument(data); //creates a new doc for the collection. Id is created here from what I understand.
someDoc.save( (err, result) => { doSomething(); });//doSomething gets called sometime in the future.
id = someDoc._id.toString();
}).then( (result) => {doSomethingElse(id)});//This works - but will doSomethingElse always be called after the first annonymous callback completes?
I get that doSomething() will just get called at some future point - no problem. The question is, will the first callback to the remove call complete prior to doSomethingElse in the then call being called. It seems to be, in that the id is correctly populated in doSomethingElse, but I want to make sure that isn't just a fluke of timing - i.e. I want to know if I can rely on that callback completing prior to the then. I'm using standard ES6 promises (NodeJS.Global.Promise).
The alternative is that maybe then is called after the remove operation completes, but prior to the callback completing (doesn't seem to - but I want to confirm).
Set me straight if I'm explaining this incorrectly.
Yes, as #JaromandaX explained in the comments the order of callbacks is deterministic.
However, it's still bad code, and you should not rely on this behaviour. If you are using promises, don't pass a callback to remove at all; only pass callbacks to then!
SomeDocument.remove({})
.then(() => {
const someDoc = new SomeDocument(data);
someDoc.save().then(doSomething); // doSomething will get called in the future.
return someDoc._id.toString();
//^^^^^^
})
.then(doSomethingElse); // doSomethingElse will get passed the id
doSomethingElse will get called with the result of the previous callback, which is guaranteed to have been completed for that.
I'm learning node now and I'm confused about the err parameter.
I thought it's supposed to be the first argument of a callback function but I don't see it in many call back functions. Can anyone explain it to me? Thanks!
There's many different kinds of functions and callback functions in particular. The Node.js standard for callback functions is those of the form:
function(err, arg1, arg2, ...)
Where arg1 and so forth are only present if relevant but the err argument is always first. This is the reverse of a lot of historical JavaScript code where errors would be the last argument.
The Node.js method of forcing the error as the first argument even if there's no error makes ignoring errors harder, you rarely forget to declare that argument, and makes their location predictable.
Now this only applies in the case of a general-purpose callback. That is, there are occasions where calling a function will trigger a singular callback at some point in the future. You'll see them used like this:
doStuff(function(err, successValue) { ... });
There's also the style popularized by jQuery where one or more of your callbacks will be triggered depending on the outcome of the operation:
doStuff({
success: function(successValue) { ... },
error: function(err) { ... },
timeout: function() { ... }
});
Note that in this case you may have both the error and timeout callbacks being fired. You're not obligated to populate all of these, either.
The downside to this approach is the unpredictability of which ones get called and the risk of handling something twice inadvertently.
The error parameter is usually for asynchronous code.
node errors
Most asynchronous methods that accept a callback function will accept an Error object passed as the first argument to that function. If that first argument is not null and is an instance of Error, then an error occurred that should be handled.
app.get() sends get request and return an error like a 404
and you could do something like this res.status(404).render( in app.get()
Express error handling
error-handling functions have four arguments instead of three: (err, req, res, next)
The reason why some code uses err as the first parameter is because some code like fs.readFileis programmed to check if there was an error and to handle it. The author of the API specifically wrote code to check the first argument for an error and handle it.
That's why it is available to you for some methods an unavailable for other methods.
First: a callback is just a function. Different callbacks serve different purposes.
In general, a function that performs an asynchronous action and should "return" a value gets passed a callback function that will take (at least) two arguments: the first is used to pass errors (if any), the second (and following) are used to pass the value(s) that should be returned to the caller.
You noticed that net.createServer() will also take a callback function, but that function only has one argument.
That's because, in this case, the callback isn't used to pass errors and/or values. Instead, it's a function that gets called when a new connection is made to the server.
It's really a bit of a shortcut. This code:
var server = net.createServer(function(connection) {
...
});
Is short for this code:
var server = net.createServer();
server.on('connection', function(connection) {
...
});
This question already has answers here:
How do I convert an existing callback API to promises?
(24 answers)
Closed 8 years ago.
I have this source code:
UserSchema.post('save', function (next) {
doSomethingAsync(function(){
next()
});
});
myFunc = function(user){
Q.ninvoke(user, 'save').then(function(){
doThisAtTheEnd()
});
}
But then is called before "doSomethingAsync" calls is callback. How is it possible?!
How can I call "then" after all the saving stuff is done?
Thanks very much
EDIT:
the two functions are in different files, no way nor intention to use a global variable.
From the documentation for Q.ninvoke: https://github.com/kriskowal/q/wiki/API-Reference#qninvokeobject-methodname-args
Calls a Node.js-style method with the given variadic arguments,
returning a promise that is fulfilled if the method calls back with a
result, or rejected if it calls back with an error (or throws one
synchronously)
And looking at mongoose schema.post('save'): http://mongoosejs.com/docs/middleware.html
post middleware are executed after the hooked method and all of its
pre middleware have completed. post middleware do not directly receive
flow control, e.g. no next or done callbacks are passed to it. post
hooks are a way to register traditional event listeners for these
methods.
Which means that there is no next for you to call in the doSomethingAsync. Probably something is internally calling back into the ninvoke.
How about deferers? You could generate a deferer and resolve it. i.e.:
var saveDeferer = Q.defer();
UserSchema.post('save', function (next) {
doSomethingAsync(function(){
saveDeferer.resolve();
});
});
saveDeferer.promise.then( function() { doSomething(); } );
Update after the question edit:
It seems to me that you are trying to use schema.post('save', ... as an eventbus that carries flow variables around. I don't see any direct answer to your edit, other than either use a custom event bus, or do some refactoring so that you can pass the promise references around.