How exactly does a callback function work in nodejs? - node.js

Question

I went through many examples of nodejs with callbacks, but didn't understand how exactly they work. I know that they're executed after the function, of which they're a part of is done, but I didn't understand callbacks as a function. Let me give an example:
function processData (callback) {
fetchData(function (err, data) {
if (err) {
console.log("An error has occured. Abort everything!");
callback(err);
}
data += 1;
callback(data);
});
}
Here, what how do callback(err) and callback(data) execute, as in, is it an alias for some other function to which we're passing the parameters - err/ data? If so, which function does it call when we write callback(parameter)?
Here's another example :
var express = require("express");
var bodyParser = require("body-parser");
var multer = require('multer');
var app = express();
app.use(bodyParser.json());
var storage = multer.diskStorage({
destination: function (req, file, callback) {
callback(null, './uploads');
},
filename: function (req, file, callback) {
callback(null, file.fieldname + '-' + Date.now());
}
});
var upload = multer({ storage : storage }).array('userPhoto',2);
app.get('/',function(req,res){
res.sendFile(__dirname + "/index.html");
});
app.post('/api/photo',function(req,res){
upload(req,res,function(err) {
//console.log(req.body);
//console.log(req.files);
if(err) {
return res.end("Error uploading file.");
}
res.end("File is uploaded");
});
});
app.listen(3000,function(){
console.log("Working on port 3000");
});
Again I sort of understood the other type of callback - suppose, here fs.readdir
fs.readdir(path, function (err, files) {
if (err) console.error(err);
for (var i = 0; i<files.length; i++) console.log(files[i];
}
console.log("done");
I know how this executes, most probably it'll print done first, and then prints the list of files, as soon as readdir is executed with files having the list of files.
But I didn't exactly understand the first and second code snippet. Can someone please explain that in simple terms, specific to the multer code snippet?

Answer 1

Callbacks and asynchronous behaviour are two different but related things. They're relate in the sense that asynchronous code can use callback to execute code in the intended sequence. But callbacks by themselves are just the result of the language treating functions as objects.
In your examples, you can see two sides of the callback story. One is the implementation and the other is the usage. The first code is an example of how to implement a function that accepts a callback. The last is an example of how to use a function that accepts a callback.
For simplicity let's ignore asynchronous behaviour for now and implement synchronous callbacks.
The following is an example of a function that loops through an array to construct a string:
function joiner (arr, callback) {
var return_value = "";
for (var i=0; i<arr.length; i++) {
return_value += callback(arr[i]); // use callback on each item
}
return return_value;
}
Once we've implemented the function above we can use it:
var result = joiner(['this','is','cool'],function(x) {
return "--> " + x + "!";
});
So, how do we know what arguments the callback accept? It accepts what the caller passes to it.
How do we know what arguments to pass to a callback function? We pass what the function is defined to accept?
If that sounds circular, it is. So how we really know what arguments to pass to a callback is:
If it's an API we created, we can do anything we want
If it's an API someone else created, read the documentation (or if your IDE can do it, check out the intellisense)
Callbacks gone meta
The above explanation is of course only the simplest example of callbacks: functions that accept a callback. The express example (and indeed your first example) shows how you can get even more complicated. You can write functions that accept a callback that accept a callback.
A simplified example of functions that accept a callback that accept a callback is something like this:
function processData (data) {
return "<" + data + ">";
}
function render (callback) {
return callback(processData);
}
So the render function above accepts a callback that will be passed a callback that returns a string. So if we now do:
var result = render(function(callback){
return "--" + callback('hello');
});
we will get:
--<hello>
As you can see, the callback we passed to render does not know how to process the data but render does (it calls processData). And render does not know what the data is but the callback we passed to it does ('hello'). Being able to pass a callback to a callback allows two pieces of code to cooperate without exposing each other's implementation details.
Async code uses this technique to order the sequence of code being executed. Because async code cannot return a value you can only pass it a callback so that once it gets its value your code (callback) can continue.
If you think about it, the Express API could have easily been implemented so that request handler callbacks must return the HTML page instead of calling res.send(). Indeed, this is how frameworks work in many languages. But that means that the request handler cannot execute asynchronous code to fetch data. Because Express doesn't want to limit you to using only synchronous code it passes an object with callback methods instead (res). This "design pattern" is called a monad. The one you see in Express is a continuation monad while the one you see in fs.readdir() is an I/O monad.

Answer 2

First, you got confused because you're not following the Node JS standards for passing parameters to asynchronous function. Every async callback must take 2 parameters. First should always be error and 2nd should always be data. Refer to how the callbacks for readdir and fetchData are defined.
So your code should ideally look like
function processData (callback) {
fetchData(function (err, data) {
if (err) {
console.log("An error has occured. Abort everything!");
callback(err, null); // here data will be null
}
data += 1;
callback(null, data); // here err will be null.
});
}
And you will define your callback function for processData at the time you call it like
processData(function(err, data){
err ? console.error(err) : console.log(data);
});
This way, in your callback, you know for sure whether the function has succeeded or failed.