Where are people getting cb() from, is this a Node thing or vanilla JS thing?
For example:
Managing Node.js Callback Hell with Promises, Generators and Other Approaches
they're using cb() to I guess callback and return an error or a value or both in some cases depending on what the callback function sig is?
cb in the context you're describing it is how a vanilla callback function is passed into a (typically) asynchronous function, which is a common pattern in node.js (it's sometimes labelled next, but you can call it bananas if you so desire - it's just an argument).
Typically the first argument is an error object (often false - if all went as planned) and subsequent arguments are data of some form.
For example:
function myAsyncFunction(arg1, arg2, cb) {
// async things
cb(false, { data: 123 });
}
then using this function:
myAsyncFunction(10, 99, function onComplete(error, data) {
if (!error) {
// hooray, everything went as planned
} else {
// disaster - retry / respond with an error etc
}
});
Promises are an alternative to this design pattern where you would return a Promise object from myAsyncFunction
For example:
function myAsyncFunction2(arg1, arg2) {
return new Promise(function resolution(resolve, reject, {
// async things
resolve({ data: 123 });
});
}
then using this function:
myAsyncFunction2(10, 99)
.then(function onSuccess(data) {
// success - send a 200 code etc
})
.catch(function onError(error) {
// oh noes - 500
});
They're basically the same thing, just written slightly differently. Promises aren't supported especially widely in a native form, but if put through a transpiler (I'd recommend babel) during a build step they should perform reliably enough in a browser too.
Callbacks will always work in a browser with no shimming / transpilation.
node.js has lots of asynchronous operations that take a completion callback as an argument. This is very common in various node.js APIs.
The node.js convention for this callback is that the first argument passed to the callback is an error code. A falsey value for this first argument means that there is no error.
For example:
fs.readFile("test.txt", function(err, data) {
if (!err) {
console.log("file data is: " + data);
}
});
A function you create yourself may also define it's own callback in order to communicate the end of one or more asynchronous operations.
function getData(id, cb) {
var fname = "datafile-" + id + ".txt";
fs.readFile(fname, function(err, data) {
if (err) {
cb(err);
} else if (data.slice(0, 6) !== "Header"){
// proper header not found at beginning of file data
cb(new Error("Invalid header"));
} else {
cb(0, data);
}
});
}
// usage:
getData(29, function(err, data) {
if (!err) {
console.log(data);
}
});
From the Vanilla JS, you can declare a function and pass throuw parameters a declaration of another function, that can called async
https://developer.mozilla.org/en-US/docs/Glossary/Callback_function
Related
I am trying to return jsonObj below and it is not returning. Help would be appreciated. Here fileName is the path of the file which I need to read. And again, I need to return jsonObj. So I can access it outside fs.readFile function.``
Here is the code
return fs.readFile(fileName, 'utf-8', (err, fileContent) => {
if(err) {
console.log(err); // Do something to handle the error or just throw it
throw new Error(err);
}
let jsonObj = csvjson.toObject(fileContent);
console.log(jsonObj)
return jsonObj
});
In Node you don't return from a callback function, that gets discarded and ignored. You must either use Promises, possibly in conjunction with async/await, or you must accept a callback function of your own.
This is the founding principle of asynchronous code:
// The fs.readFile() function executes immediately
fs.readFile(..., (err, ...) => {
// This code runs in the distant future, like imagine ten years from now
// from the perspective of the computer.
});
// JavaScript continues here immediately.
The readFile function does not return anything in particular, the inner function has not yet been run. It's also worth noting that throwing exceptions in there is pointless, don't do it.
To chain:
function getData(fileName, cb) {
fs.readFileName(fileName, 'utf-8', (err, fileContent) => {
if (err) {
// Back-propagate the error
return cb(err);
}
// Engage the callback function with the data
cb(null, csvjson.toObject(fileContent));
}
}
Note that this code is a lot less convoluted and messy if you use promises instead.
I made my express application with the async+await.
However it took so long time to finish this task.
async.parallel(
{
cars: function (callback) {
Car.findById(req.params.id).exec(callback);
},
},
async (err, results) => {
if (err) {
return next(err);
} else {
var paramsDel = { Bucket: "test-s3-may", Key: results.cars.image_id };
s3.deleteObject(paramsDel, function (err, file) {
if (err) {
return next(err);
}
s3.upload(params, function (err, data) {
if (err) {
return next(err);
}
console.log("go to the next step"); //lineA
});
});
var updated_car = await Car.findByIdAndUpdate(req.params.id, car, {});
res.redirect(updated_car.url);
}
}
);
When I proceed this code, lineA(console.log("go to the next step) is displayed instantly.
After that, I have to wait almost 1-2 minutes for redirect commands.
Is it the problem of await? I'm sure that my internet speed is fast enough to upload any file.
You are assuming that the issue lies with the upload process, but following you method signature:
Car.findByIdAndUpdate
it seems rather to be local data storage based update call.
await is nothing but a syntactic sugar around Promise(s) with which you can write plain imperative statements in the old fashioned way and don't have to chain you completion callback into #then.
Thereby, awaited statement call will still cause the function to be paused in a non-blocking way until the underlying promise finishes (either fulfills or rejects).
In other words, the runtime is not blocked and further requests can be processed but the res.redirect(updated_car.url); won't be executed until the Car.findByIdAndUpdate function is resolved which thus needs to be profiled based on your storage system.
Currently i have a dashboard that lists a bunch of records in a table. users can select 1 record and hit execute and i send a AJAX POST request to my routes middleware which executes 3 functions inside async.waterfall and returns a 200 response back to my client if everything works correctly. this async waterfall usually takes about 40-55 seconds to finish executing (fn_1,fn_2 and fn_3) and works perfectly fine.
router.post('/url', function(req, res, next) {
try {
async.waterfall([
fn_1,
fn_2,
fn_3
], function (err, body) {
res.writeHead(200, {'Content-Type': 'application/json'});
res.end(JSON.stringify({"error":err, "result":body}));
});
function fn_1(callback) {
callback(null, response);
}
function fn_1(result, callback) {
callback(err, result);
}
function fn_2(result, callback) {
callback(null, result);
}
}
catch (err){
console.log(err)
}
});
But, If i were to give provision for users to select MULTIPLE records and send that as an array back to my route middleware. how can i execute multiple async.waterfall methods for each item in the array in parallel
i can run a loop and execute the waterfall inside the loop but it again will wait for each item to complete and only then start the next iteration. this is not what i want.
is this doable in node / express . whats the easiest way to achieve this ? or are there modules/plugins that can help solve this case ?
Here is an abbreviated version of your code and how it could be changed to suit your needs. If none of your calls need data from any of the other calls, you can just run them in parallel with promises and use Promise.all to capture the result.
function fn_1(callback) {
// See function fn_2 for structure
}
function fn_1(result, callback) {
// See function fn_2 for structure
}
function fn_2(result, callback) {
return new Promise(resolve, reject => {
resolve(result)
})
.then(d => {
// Instead of callbacks, use a "then"
// block/statement.
//
// Do something with D here.
})
}
Promise.all([fn_1(), fn_2(), fn_3()])
.then(v => {
// Do somthing with v;
})
.catch(e => {
// Do something with e
})
I tend to advocate the use of native Promises over libraries like async, however, since your already using async...
You can use parallel and map each item in the array to a waterfall handler e.g.
async.parallel(
myArray.map(val => cb => async.waterfall(fn_1, fn_2, fn_3, cb)
, (err, results) => {
// return consolidated response
})
You would need to rework your waterfall handlers to not send a response but instead just propagate any errors.
It should also be noted that parallel is only useful if you are infact running I/O bound code, if the code is anything like your example then you won't really gain anything from using parallel over something like async.each
So I'm new to Node.js and Im just wondering if the way I have my code setup makes sense. Im coming from a Java background so the nested callback structure is new. I have a Node program that runs a bunch of code that I broke down into different methods. The thing is that the methods need to be called in order. My code has this structure right now:
functionOne(data, callback(err) {
functionTwo(data, callback(err) {
functionThree(data, callback(err) {
functionFour(data, callback(err) {
//Code
});
});
});
});
This is very minimalistic, but is this structure ok? With Java, I'd take the return values of all the methods, then just pass them to the next function. From my understanding so far, the Java approach I just mentioned is one of the main things that Node.js was trying to eliminate. But anyway... Does that structure look ok, and is that how its intended to look? Just want to be sure that I'm not making any major errors with Node in general. Thanks!
Your code structure looks fine if you work with callback pattern.
But if you're interested in make your code cleaner and readable you would like to use Promises in your asynchronous function, so instead of pass a callback to your functions you could do something like this :
function asyncFunction (data){
return new Promise(function(resolve, reject){
// Do something with data
// Here you can call reject(error) to throw an error
resolve();
});
}
And instead of nested function callbacks you can call then method of Promise.
asyncFunction(data)
.then(function(){
// Promise resolved
// Something has been done with data
});
With Promises you can also execute async fuctions in parallel :
Promise.all([asyncFunctionA(data), asyncFunctionB(data), asyncFunctionC(data)])
.then(function(){...});
EDIT
If you need to pass values of one function to another, your code should look like this :
asyncFunctionA(data)
.then(function(dataA){
return asyncFunctionB(dataA);
})
.then(function(dataB){
return asyncFunctionC(dataB);
})
.then(function(dataC){
// ...
});
You should try to use promises to avoid your callback hell, so it could be something like these...
const Q = require('q'); // you can do a research for this module.
var myModule = {};
myModule.functionOne = (params) => {
const deferred = Q.defer(); // wait for this to complete
// body function
deferred.resolve(data); // this would be the result of this function
return deferred.promise; // data is the output on your function
}
myModule.functionTwo = (params) => {
const deferred = Q.defer(); // wait for this to complete
// body function
deferred.resolve(data); // this would be the result of this function
return deferred.promise; // data is the output on your function
}
myModule.doAll = (params) => {
myModule.functionOne(params)
.then((outputFunctionOne) => {
// this is called after functionOne ends
return myModule.functionTwo(outputFunctionOne);
})
.then((outputFunctionTwo) => {
// this is called after function 2 ends
if (outputFunctionTwo.success) {
// if everything ok, resolve the promise with the final output
deferred.resolve(outputFunctionTwo);
} else {
// reject the promise with an error message
deferred.reject('error');
}
})
.fail((err) => {
// this is call if the promise is rejected or an exception is thrown
console.log(err); // TODO: Error handling
})
.done();
}
module.exports = myModule;
You can Chain as many promises as you want really easily, that way you get rid of the callback hell. Best part, you can do promises on Javascript or Node.js
Reference Link https://github.com/kriskowal/q
Hope this helps
Most of the other answers give Promise/A as the answer to your callback woes. This is correct, and will work for you. However I'd like to give you another option, if you are willing to drop javascript as your working language.
Introducing Iced Coffee, a branch of the CoffeeScript project.
With Iced Coffee you would write:
await functionOne data, defer err
await functionTwo data, defer err2
await functionThree data, defer err3
//etc
This then compiles to the CoffeeScript:
functionOne data, (err) ->
functionTwo data, (err2) ->
functionThree data, (err3) ->
//etc
Which then compiles to your Javascript.
functionOne(data, callback(err) {
functionTwo(data, callback(err2) {
functionThree(data, callback(err3) {
//etc
});
});
});
User.prototype.isUnique = function () {
var result;
db.user.count({name:'abcdefg'}, function(err, count){
console.log('the count is: ' + count);
result = count;
});
return result;
}
this function is used to find out if this user is unique, but the return result is always 'undefined' since db operation is async. I believe it is a common problem in nodejs.
How do I solve it?
I try to use promise (https://npmjs.org/package/promise), but I don't have enough knowledge to understand their documentation. It's for very advanced developer.
Can you help me with some sample code? All I need to do is get the count() using mongoskin, and then return the result in this member method.
Thanks!
It's not a common problem, it's just how Node works and something that you'll have to get used to dealing with.
The simplest solution (also a very common one) is to provide a callback function from the calling code. That function will be called whenever the asynchronous action is finished:
User.prototype.isUnique = function(callback) {
db.user.count({name:'abcdefg'}, function(err, count){
console.log('the count is: ' + count);
callback(err, count);
});
}
// in your calling code:
user.isUnique(function(err, count) {
if (err) ...; // TODO: handle error
...
});
It's also common in Node to have callback functions accept at least one argument which contains an error object if an error occurred, or null if everything went okay.
In the code above I'm passing any errors that may have occurred during the call to db.user.count to the callback function. It's up to the calling code to deal with any errors.
There are several alternatives to dealing with asynchronous code. One would be to use promises, like you mention. There are also solutions like streamline that make asynchronous code look like it is synchronous, but it requires that you 'compile' your code (although this can also be done at runtime) before you can use it.
EDIT: if you want to use promises, you can use this:
var Promise = require('promise');
User.prototype.isUnique = function() {
return Promise(function(resolve, reject) {
db.user.count({name:'abcdefg'}, function(err, count){
if (err) {
console.log('an error occurred:', err);
reject(err);
} else {
console.log('the count is:', count);
resolve(count);
}
});
});
};
// in your calling code:
user.isUnique().then(function(count) {
...
}, function(err) {
...
});
(this requires that the promise package is installed)