i'm trying to understand asynchronous callbacks in NodeJS by following this guide and i have a question about the position of callback() in the code snippet below.
var fs = require('fs')
var myNumber = undefined
function addOne(callback) {
fs.readFile('number.txt', function doneReading(err, fileContents) {
myNumber = parseInt(fileContents)
myNumber++
callback()
})
}
function logMyNumber() {
console.log(myNumber)
}
addOne(logMyNumber)
Here, my file 'number.txt' contains the number 1 and the output of this entire code snippet is 2. This appears to be invoking callback() after the file is read, and the output is expected. However, moving callback() outside of fs.readFile() but inside addOne() as shown below has confused me as the output is now undefined.
var fs = require('fs')
var myNumber = undefined
function addOne(callback) {
fs.readFile('number.txt', function doneReading(err, fileContents) {
myNumber = parseInt(fileContents)
myNumber++
})
callback()
}
function logMyNumber() {
console.log(myNumber)
}
addOne(logMyNumber)
Does this mean that in the second example, callback() is invoked before fs.readFile() has completed?
Your thinking is right. Node.js executes fs.readFile but DOES NOT wait for it's completion.
So the execution moves to next statement, which invokes the callback and the result is undefined because the previous command has not yet finished.
Asynchronous programming is very interesting, but easy thing.
Asynchronous executions means that program won't run your code synchronously line by line waiting for completing each line of code WHEN YOU ARE EXECUTING BLOCKING CODE, but instead it will do concurrent execution (concurrency).
Blocking code is when you are doing for example http requests, reading files as in your example or DOM events in browser. Basically blocking code is code which is not depended on your code. Basically blocking code is when your program should wait unknown duration till their completion to continue.
On basic javascript operations like arithmetic and array operations, loops e.g.
Code runs synchronously because they are not depended on external resources.
That is why we have callbacks in JavaScript to not wait for their completion.
Callbacks helps us to run code after asynchronous code executed.
Asynchronous means that our code is NON-BLOCKING.
So in your code when you run
fs.readFile('number.txt', function doneReading(err, fileContents) {
myNumber = parseInt(fileContents)
myNumber++
})
callback()
It will start running fs.readFile and immediately will start execution of callback function.
But when when you run
fs.readFile('number.txt', function doneReading(err, fileContents) {
myNumber = parseInt(fileContents)
myNumber++
callback()
})
It will start executing fs.readFile and within his callback doneReading function will do operations with myNumber and will execute callback
Most of JavaScript asynchronous functions have callback functions.
You can read about blocking and non-blocking codes here, also here about callbacks
Also you can read about promises and async/await.
They are very cool things, that helps you to struct your code as you would in synchronous environment, but runs asynchronous code.
Related
I'm developing a simple discord bot and I am attempting to print some general data about a certain player. I recently learned about async/await and I attempted to implement it into my code. It does not seem to work however because when I first trigger this code, it prints null but on subsequent triggers it will print the correct data, indicating that my function did not wait for the Promise to resolve.
async function stats(){
data = await NBA.stats.playerInfo({ PlayerID: curry.playerId });
}
stats();
data = JSON.stringify(data);
console.log(data);
The variable data is a global variable declared at the top of my program and initially initialized to null.
If I understand your intention correctly, what you want is to asynchronously fetch some data into data and then display it on the console.
Your implementation of stats is doing the fetch correctly. But your problem is, the part where you log to console has no dependence on the fetch being completed.
When you call a function that has been declared async, you are saying that you want it to execute "in the background" so to speak. The interpreter won't wait for that function to finish executing.
stats(); // begin executing this function
data = JSON.stringify(data); // but don't wait for it to finish before moving on to this line
console.log(data);
Clearly, that's not what you want. Instead, you want to wait for stats to finish what it's doing before logging data. stats being an async function returns a Promise, so you can do this:
function logData() {
console.log(JSON.stringify(data));
}
stats().then(logData);
Are there some advantages of one of this options?
1.
const fs = require('fs')
const testFunc1 = async () => {
fs.writeFileSync('text.txt', 'hello world')
}
2.
const fs = require('fs')
const util = require('util')
const writeFilePromisified = util.promisify(fs.writeFile)
const testFunc2 = async () => {
await writeFilePromisified('text.txt', 'hello world')
}
I am aware the difference betweeen writeFile and writeFileSync. The question is are there some diffs between promisses that return testFunc1 and testFunc2. So ist it the same to calling
testFunc1.then(...) // or await testFunc1
or
testFunc2.then(...) // or await testFunc2
These both promisses will be fullfilled when file writing is done.
fs already contains promisified API that doesn't need promisify.
const fsPromises = require("fs/promises");
await fsPromises.writeFile(file, data[, options])
Asynchronous promise-based version requires to use it as a part of promise-based control flow, while synchronous version doesn't impose this requirement.
Asynchronous readFile/writeFile is non-blocking, while synchronous readFileSync/writeFileSync is blocking but allows to complete a job faster. This may be noticeable during intensive IO operations.
To illustrate the difference between two promises that returns functions:
const fs = require('fs')
const util = require('util')
const testFunc1 = async () => {
fs.writeFileSync('text.txt', 'hello world')
console.log('file write done with writeFileSync')
}
const writeFilePromisified = util.promisify(fs.writeFile)
const testFunc2 = async () => {
await writeFilePromisified('text.txt', 'hello world')
console.log('file write done with promisified writeFile')
}
console.log('start test1')
testFunc1().then(() => {
console.log('promise 1 is fullfiled')
})
console.log('start test2')
testFunc2().then(() => {
console.log('promise 2 is fullfiled')
})
console.log('stop')
Output would be:
start test1
file write done with writeFileSync
start test2
stop
promise 1 is fullfiled
file write done with promisified writeFile
promise 2 is fullfiled
So like estus said testFunc1 blocks the execution of the main thread. testFunc2 do not block.
fs.readFile takes a callback function, which means it will not block the execution of your script.
fs.readFileSync however does not take a callback, which means that the execution of your script will be paused untill the process is finished.
Using promisfy is one way to solve this issue, for small files it wont make a difference, but for large files you might want to transform fs.readFileSync into a promise so you wont block the execution.
Hope that helps.
The difference between writeFile() and writeFileSync() is as explained by others that writeFileSync() "blocks". So, what is the difference between "blocking" and "not blocking"?
I wrote a small test where I compared writeFile() and writeFileSync() in terms of speed. I measured the time it took for writeFileSync() to return a result vs. the time it took from calling writeFile() until its callback-argument got called. To my (initial) surprise there was no noticeable difference between the two. writeFile() did not seem any faster than writeFileSync(). So why should I use it when it seems to complicate my program-flow?
But then I measured the time writeFile() took if I didn't wait for its callback-argument to get called. There was a big difference in speed, maybe 10x.
Having a big or no difference thus depends on the rest of your program. If you make a call to writeFileSync() that takes a long time, your program can do nothing else while it is waiting for writeFileSync() to return. If you do writeFile() and wait for the callback to complete before doing anything else, the outcome is no different. But writeFile() is much faster if you don't (need to) wait for its callback to get called.
This would have a big difference if you write a server of some kind that calls writeFileSync(). The server could not service any new requests while it is waiting for a writeFileSync() to complete. And if most requests caused a call to writeFileSync() that could have a big impact on the performance of your server.
fs.writeFileSync( file, data, options )
fs.writeFile( file, data, options, callback)
The Asynchronous function has a callback function as the last
parameter which indicates the completion of the asynchronous function.
The ‘fs’ module implements the File I/O operation. Methods in the fs module can be synchronous as well as asynchronous.
Developers prefer asynchronous methods over synchronous methods as they never block a program during its execution, whereas the later does. Blocking the main thread is "malpractice" in Node.js, thus synchronous functions should only be used for "debugging" or when no other options are available.
The fs.writeFileSync() is a synchronous method & creates a new file if
the specified file does not exist while fs.writeFile() is an
asynchronous method.
I wrote the following function that takes a callback. I always thought that the content of a callback might be executed later. In this case it doesn't...
doesSomething(function(){
console.log("1");
var i = 0;
while (i < 10000)
{
console.log("hello");
i = i + 1;
}
});
console.log("2");
console.log("3");
Whatever I do, "2" and "3" always comes after "1" and a thousand of "hello".
Like this:
1
hello
hello
...
hello
hello
2
3
What I thought it would do:
2
3
1
hello
hello
...
hello
hello
Even if this behaviour makes my life simpler I don't really understand why the execution is procedural.
Do you think that in some case it might go reverse ?
It all depends upon how doSomething() calls its callback. If it calls the callback synchronously (e.g. before it returns), then everything in that function will execute before doSomething() returns. If it calls it asynchronously (sometime after it returns), then you will get a different order.
So, the order is determined by the code that you do not show us in doSomething().
Based on the order you observe, doSomething() must be calling its callback synchronously and thus it will execute in order just like any other synchronous function call.
For example, here are two scenarios:
function doSomething(callback) {
callback();
}
This calls the callback passed to it synchronously and thus it will execute before doSomething() returns and thus it will execute before code that follows.
Whereas something like this;
function doSomething(callback) {
fs.writeFile('foo.txt', callback);
}
or:
function doSomething(callback) {
setTimeout(callback, 50);
}
Will execute the callback asynchronously sometime later after the function has already returned and you will see a different execution order with your console.log() statements.
From the output that you mention, I can say that there is nothing asynchronous happening in your code inside doesSomething and is probably looking like this:
function doesSomething(callback) {
<maybe some synchronous operations, e.g. no ajax or filesystem calls>
callback();
}
, so the order of the calling functions will be always the same, like you posted above:
doesSomething(callback)
console.log(2)
console.log(3)
If you want the order you mention you should call them like this:
console.log(2)
console.log(3)
doesSomething(callback)
This does not have to do something with node.js in particular. It is the way methods are executed in JavaScript, which is synchronous since it is single threaded.
When you call the method doesSomething(callback) it immediately executes it and nothing else until it completes
javascript is a single threaded language, but not the node.js runtime or the browser. There are certain functions provided by node.js or the browser that would trigger for a function to be assigned to a task queue to be processed in a separate thread
lets take your example and make dosomething async
this is your synchronous code that would print, 1 hello...,2,3
function doesSomething(callback) {
callback();
}
doesSomething(function(){
console.log("1");
var i = 0;
while (i < 3)
{
console.log("hello");
i = i + 1;
}
});
console.log("2");
console.log("3");
and this is the async code, that prints out 3, 2, 1, hello ...
function doesSomething(callback) {
setTimeout(callback, 0);
}
doesSomething(function(){
console.log("1");
var i = 0;
while (i < 3)
{
console.log("hello");
i = i + 1;
}
});
console.log("2");
console.log("3");
to explain why the second code is async, you have to understand that setTimeout is not part of javascript, its an api provided by node.js and the browsers. setTimeout puts the callback function into a queue to be processed. At this time a separate thread runs the setTimeout and when the timer ends it puts the callback into a callback queue, and when the call stack is clear, whatever is in the callback queue will be moved to the call stack and processed.
in node you can use process.nextTick(yourFunction); to make it async, this is just another function that nodejs provides you which is better than using setTimeout (which I wont get into here) you can checkout https://howtonode.org/understanding-process-next-tick to understand more about process.nextTick
for more info check out this video https://youtu.be/8aGhZQkoFbQ?t=19m25s
I understand what a callback is and what asynchronous means, what I don't get is how to run asynchronous functions in node.
For example, how is this
var action = (function(data,callback) {
result = data+1;
callback(result);
});
http.createServer(function (req, res) {
action(5, function(r){
res.end(r.toString());
});
}).listen(80);
different from this
var action = (function(data) {
result = data+1;
return result;
});
http.createServer(function (req, res) {
var r = action(5);
res.end(r.toString());
}).listen(80);
?
I guess in the first example I'm doing it asynchronously, yet I don't know how Node knows when to do it sync or async... is it a matter of the return? or the fact that in the sync mode we're doing var x = func(data);?
And also: when to use sync or async? Because obviously you don't want to use it when adding +1... is it OK to use async just when performing IO tasks, such as reading from DB?
For example, I'm using the library crypto to encrypt a short string (50 chars at most), is this case a good example where I should already be using async?
I guess in the first example I'm doing it asynchronously...
Your first example isn't async :) Merely passing a callback and calling it when you're done doesn't make a function asynchronous.
Asynchronous means that, basically, you're telling Node: "here, do this for me, and let me know when you're done while I continue doing other stuff".
Your example is not handing anything to Node for future completion. It's doing a calculation and calling the callback immediately after that. That's functionally the same as your second example, where you return the result of the calculation.
However, you can change your first example to something that is asynchronous:
var action = (function(data,callback) {
setTimeout(function() {
result = data + 1;
callback(result);
}, 1000);
});
Here, you're telling Node to delay calling the callback for one second, by using setTimeout. In the mean time, Node won't get stuck waiting for a second; it will happily accept more HTTP requests, and each one will be delayed one second before the response is sent.
When to use sync or async?
Asynchronous code is "viral": if you rely on functions that are async, your own code that uses those functions will also have to be async (generally by accepting a callback, or using another mechanism to deal with asynchronicity, like promises).
For example, I'm using the library crypto to encrypt a short string (50 chars at most), is this case a good example where I should already be using async?
This depends on which function you're using. AFAIK, most encryption functions in crypto aren't asynchronous, so you can't "make" them asynchronous yourself.
Both examples will work synchronous. Simple async operations are setTimout and setInterval.
Node actually doesn't care what code are you running. You can block or not (blocking/non-blocking).
In other words - you have event loop. If your process is async he will pass the program control to the event loop, so it can execute any other action node needs to be done. If not - he wont.
if you want a function to work asynchronously, you can do that using promises, look at the code below :
function is_asynch(){
return new Promise((resolve,reject)=>{
resolve( here_your_synch_function() )
})
}
async.parallel([
function(callback) { //This is the first task, and callback is its callback task
db.save('xxx', 'a', function(err) {
//Now we have saved to the DB, so let's tell async that this task is done
callback();
});
},
function(callback) { //This is the second task, and callback is its callback task
db.save('xxx', 'b', callback); //Since we don't do anything interesting in db.save()'s callback, we might as well just pass in the task callback
}
], function(err) { //This is the final callback
console.log('Both a and b are saved now');
});
Hey I found this code online when I trying to understand async in node.js and my question I have with the code above is in the array of functions in async.parallel, what is the callback parameter that is passed in each function? Where does that callback parameter come from and what is the purpose of that variable? I am sorry if this is a dumb question, but I can't grasp a solid understanding of this callback's...
The contract that async.parallel() defines is that you pass it an array of functions. It will then call each of those functions and pass those function a single argument that is a function reference (I think of it as a completion callback). For async.parallel() to work properly, your function that you pass in the array has to call that function reference when your async operation is done. When it calls that function reference, it can pass an error argument and a result argument. If the error argument is truthy, then it will stop further processing and feed that error back to the calling code.
The function that async passes a reference to is internal to the async library and when you call it, the async library does some housekeeping to keep track of how many asynchronous operations are still going, whether they're all done now, whether further processing should stop due to error, etc...
You can essentially think of it like the async library completion function. When your asynchronous operation is done, you have to call that completion function so that the async library knows that your operation is now done. That's how it is able to manage and keep track of your multiple async operations. If you don't call that completion callback, then the async library never knows when your asynchronous operation is done.
The callback parameter is a function created by async. When you call it in each of the array functions, it signals async that the function completed. In that way, you can control when your function is done - when you call callback.
When callback is called from all the array functions, the final function is called.