I'm assuming NO because it returns a void instead of Promise<void>, however, the documentation states the following sentence:
Asynchronously reads the entire contents of a file.
How does this function read a file asynchronously if it's not an async function?
It is asynchronous.
It does not return anything (let alone a promise (and a promise that resolved as void (nothing) would be pointless).
It accepts a callback that will be called when the result is available.
See the documentation which says, explicitly:
Asynchronously reads the entire contents of a file.
and has an example of how to use it:
import { readFile } from 'fs';
readFile('/etc/passwd', (err, data) => {
if (err) throw err;
console.log(data);
});
A version that returns a promise is also available.
Yes, it is an asynchronous function because it takes a callback and calls callback function when the data is ready.
That's why it is asynchronous.
Example from the doc:
readFile('/etc/passwd', (err, data) => {
if (err) throw err;
console.log(data);
});
fs.readfile is asynchronous and can return a result later by calling a callback that you should indicate for it:
fs.readFile(path[, options], callback)
So if you are not interested in the result of readFile call immediately (at the next line) after this call you can use it. Otherwise use readFile from fs/promises package that returns Promise
Related
This question already has answers here:
How do I return the response from an asynchronous call?
(41 answers)
Closed 12 months ago.
I created these variables and to each of them assigned the text of each txt file. If i stored them in the the variables, will they still run in the background while executing the file or would this be Synchronously?
const getGiraffe = fs.readFile("./txt/giraffe.txt", "utf-8", (err, data) => {
return data;
});
const getRhino = fs.readFileSync("./txt/rhino.txt", "utf-8", (err, data) => {
return data;
});
Neither of your code examples is correct.
In the first code example:
const getGiraffe = fs.readFile("./txt/giraffe.txt", "utf-8", (err, data) => {
return data;
});
You are reading the file (with no error handling) and then you do return data back to the callback, but that doesn't go anywhere. So, getGiraffe will simply be undefined.
To use the asynchronous version of fs.readFile(), you must use the data value inside the callback or call some function from within that callback and pass it the data. You cannot simply return the data out.
fs.readFile("./txt/giraffe.txt", "utf-8", (err, data) => {
if (err) {
console.log(err);
} else {
console.log(data);
}
});
In the second code example:
const getRhino = fs.readFileSync("./txt/rhino.txt", "utf-8", (err, data) => {
return data;
});
You are passing a callback to a function fs.readFileSync() that does not accept a callback. That callback will be ignored.
The calling signature for fs.readFileSync() is this:
fs.readFileSync(path[, options])
It takes only two arguments, neither is a callback.
Your question
If i stored them in the the variables, will they still run in the background while executing the file or would this be Synchronously?
The first code example will run in the background asynchronously, but getGiraffe will be undefined. It will not have the data from your file in it. That is just not how fs.readFile() works. You must use the data it delivers INSIDE the callback itself.
Your second code example, is synchronous and blocking. It will block the event loop and getRhino will contain the data from the file, but your callback will be ignored and never called because that's not a supported parameter for fs.readFileSync().
For more information on communicating back a result to the caller from an asynchronous function like fs.readFile() see this reference:
How to return the response from an asynchronous call
Assume an Express route that makes a call to Mongoose and has to be async so it can await on the mongoose.find(). Also assume we are receiving XML but we have to change it to JSON, and that also needs to be async so I can call await inside of it.
If I do this:
app.post('/ams', async (req, res) => {
try {
xml2js.parseString(xml, async (err, json) => {
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
} catch(err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
}
The server hangs forever. I'm assuming the async/await means that the server hits a timeout before something concludes.
If I put this:
res.status(200);
res.send("Data saved")
on the line before the catch(), then that is returned, but it is the only thing every returned. The client gets a 200, even if an XMLException is thrown.
I can see the XMLException throw in the console, but I cannot get a 400 to send back. I cannot get anything I that catch block to execute in a way that communicates the response to the client.
Is there a way to do this?
In a nutshell, there is no way to propagate an error from the xml2js.parseString() callback up to the higher code because that parent function has already exited and returned. This is how plain callbacks work with asynchronous code.
To understand the problem here, you have to follow the code flow for xml2js.parseString() in your function. If you instrumented it like this:
app.post('/ams', async (req, res) => {
try {
console.log("1");
xml2js.parseString(xml, async (err, json) => {
console.log("2");
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
console.log("3");
} catch (err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
console.log("4");
});
Then, you would see this in the logs:
1 // about to call xml2js.parseString()
3 // after the call to xml2js.parseString()
4 // function about to exit
2 // callback called last after function returned
The outer function has finished and returned BEFORE your callback has been called. This is because xml2js.parseString() is asynchronous and non-blocking. That means that calling it just initiates the operation and then it immediately returns and the rest of your function continues to execute. It works in the background and some time later, it posts an event to the Javascript event queue and when the interpreter is done with whatever else it was doing, it will pick up that event and call the callback.
The callback will get called with an almost empty call stack. So, you can't use traditional try/catch exceptions with these plain, asynchronous callbacks. Instead, you must either handle the error inside the callback or call some function from within the callback to handle the error for you.
When you try to throw inside that plain, asynchronous callback, the exception just goes back into the event handler that triggered the completion of the asynchronous operation and no further because there's nothing else on the call stack. Your try/catch you show in your code cannot catch that exception. In fact, no other code can catch that exception - only code within the exception.
This is not a great way to write code, but nodejs survived with it for many years (by not using throw in these circumstances). However, this is why promises were invented and when used with the newer language features async/await, they provide a cleaner way to do things.
And, fortunately in this circumstance xml2js.parseString() has a promise interface already.
So, you can do this:
app.post('/ams', async (req, res) => {
try {
// get the xml data from somewhere
const json = await xml2js.parseString(xml);
// do something with json here
res.send("Data saved");
} catch (err) {
console.log(err);
res.status(400).send("Malformed XML error: " + err.message);
}
});
With the xml2js.parseString() interface, if you do NOT pass it a callback, it will return a promise instead that resolves to the final value or rejects with an error. This is not something all asynchronous interfaces can do, but is fairly common these days if the interface had the older style callback originally and then they want to now support promises. Newer interfaces are generally just built with only promise-based interfaces. Anyway, per the doc, this interface will return a promise if you don't pass a callback.
You can then use await with that promise that the function returns. If the promise resolves, the await will retrieve the resolved value of the promise. If the promise rejects, because you awaiting the rejection will be caught by the try/catch. FYI, you can also use .then() and .catch() with the promise, but in many cases, async and await are simpler so that's what I've shown here.
So, in this code, if there is invalid XML, then the promise that xml2js.parseString() returns will reject and control flow will go to the catch block where you can handle the error.
If you want to capture only the xml2js.parseString() error separately from other exceptions that could occur elsewhere in your code, you can put a try/catch around just it (though this code didn't show anything else that would likely throw an exception so I didn't add another try/catch). In fact, this form of try/catch can be used pretty much like you would normally use it with synchronous code. You can throw up to a higher level of try/catch too.
A few other notes, many people who first start programming with asynchronous operations try to just put await in front of anything asynchronous and hope that it solves their problem. await only does anything useful when you await a promise so your asynchronous function must return a promise that resolves/rejects when the asynchronous operation is complete for the await to do anything useful.
It is also possible to take a plain callback asynchronous function that does not have a promise interface and wrap a promise interface around it. You pretty much never want to mix promise interface functions with plain callback asynchronous operations because error handling and propagation is a nightmare with a mixed model. So, sometimes you have to "promisify" an older interface so you can use promises with it. In most cases, you can do that with util.promisify() built into the util library in nodejs. Fortunately, since promises and async/await are the modern and easier way to do asynchronous things, most newer asynchronous interfaces in the nodejs world come with promise interfaces already.
You are throwing exceptions inside the callback function. So you cant expect the catch block of the router to receive it.
One way to handle this is by using util.promisify.
try{
const util = require('util');
const parseString = util.promisify(xml2js.parseString);
let json = await parsestring(xml);
}catch(err)
{
...
}
I have a question about callback functions. Consider this simple code:
var array = [1,2,3];
console.log(array.map(function(val){
return val * 2
}))
This will correctly log back to the console [2,4,6] which means it is "returning" that value back.
Now consider this (NOTE: assume "ab" to be a valid directory)
fs.readdir("ab",function(err,data){
console.log(data)
})
This code works fine and will print an array of filenames in the directory "ab" to the terminal. However, the same code:
console.log(fs.readdir("ab",function(err,data){
return data
}))
This will print undefined.. why is this? if in the previous snippet I can log data to the terminal, why can't I return the value and log it to the console? And how is the first snippet including the map method working using this same logic?
Thank you.
fs is an Asynchronous function, it doesn't return the values as it doesn't know when the value will be available due to which it logs undefined.
When you say console.log(fs.readdir()), it reads the fs function and checks if it is returning anything which in this case is undefined and hence logs it. The execution didn't go inside the callback of fs.readdir() yet as it is asynchronous as takes time for that to complete.
fs.readdir() is like a promise, it calls to read the directory and forget about it till the reading operation is done and after which it's callback function is called.
Hence, we pass callback functions which will be called back when the asynchronous operation is done executing like below :
function readFile (callback){
fs.readdir("ab",function(err,data){
return callback(data);
});
}
readFile(function (data){
console.log(data);
});
Because the map() method by definition returns an array, and fs.readdir() returns undefined.
Why it returns undefined is because it is an asynchronous function, the result is not returned by the function but in the callback you have to wait, that's because you have to do the console.log inside that callback function.
Just for your information, the fs module has synchronous version of the methods as well, so in this case you could do:
console.log(fs.readdir("ab"))
But using the synchronous version you are blocking your code until you get the result, so it would not be recommended depending on the situation; that's up to your case.
I hope that the following will help someone else having similar problem with reading values from MongoDB using socket io.
In those cases you can use the following code (both functions on server side):
socket.on('readTopicMessages', (intGr, callback) => {
readTopicChats(intGr, function (data){
io.emit('topicMessages', data);
});
});
function readTopicChats (intGr, callback){
All_Chats
.find({ 'intGr': intGr.intGr })
.sort([['dateTime', 'ascending']])
.exec(function (err, data){
return callback(data);
});
}
function saveToTheDb(value) {
return new Promise(function(resolve, reject) {
db.values.insert(value, function(err, user) { // remember error first ;)
if (err) {
return reject(err); // don't forget to return here
}
resolve(user);
})
}
}
Here is the code which i see from here.
i am confused about return keyword.
For resolve(user);, do i need return?
For reject(user);, do i need return?
There is no need to use a return statement inside a new Promise() callback. The Promise constructor is not expecting any sort of return value from the callback.
So, the reason to use a return statement inside that callback is only to control the flow of execution in that function. If you want execution inside your callback to finish and not execute any more code within that callback, you can issue a return; at that point.
For example, you could have written your code like this with no return statement:
function saveToTheDb(value) {
return new Promise(function(resolve, reject) {
db.values.insert(value, function(err, user) {
if (err) {
reject(err);
} else {
resolve(user);
}
});
}
}
In this case, you used the if/else clause to make sure the flow of control in your function takes the correct path and no return was needed or used.
A common shortcut when promisifying async functions like this is:
function saveToTheDb(value) {
return new Promise(function(resolve, reject) {
db.values.insert(value, function(err, user) {
if (err) return reject(err);
resolve(user);
});
}
}
This is not functionally different than the previous code block, but it is less typing and more compact. The return statement in front of reject(err); is only for flow of control reasons to prevent from executing the resolve(user); statement in case of error since the desired flow of control is to call reject(err) and then not execute anything else in the callback.
In fact, the return statement in this last block is not actually even needed in this specific case because executing a resolve() after a reject() will not do anything since promises are latched to whichever happens first resolve or reject. But, it is generally considered poor practice to execute unnecessary code so many would argue that it is better to use flow of control structures such as if/else or return to only execute the code that is needed.
So, this would technically work too, but is not considered a best practice because it executes unnecessary code and isn't as clearly structured:
function saveToTheDb(value) {
return new Promise(function(resolve, reject) {
db.values.insert(value, function(err, user) {
if (err) reject(err);
resolve(user);
});
}
}
FYI, what you are doing here is called "promisifying" which makes a regular async function that works with a callback into a function that returns a promise. There are libraries and functions that will "promisify" a function or a whole object of functions (e.g. a whole API) for you in one function call so you don't have to do this manually. For example, I regularly use Bluebird which offers Promise.promisify() for promisifying a single function or Promise.promisifyAll() which will promisify all the methods on an object or prototype. This is very useful. For example, you could get promisified versions of the entire fs module with just this:
var Promise = require('bluebird');
var fs = Promise.promisifyAll(require('fs'));
Then, you can use methods that return a promise such as:
fs.readFileAsync("file.txt").then(function(data) {
// do something with file.txt data here
});
Generally, in NodeJS, you shouldn't use the promise constructor very much.
The promise constructor is for converting APIs that don't return promises to promises. You should consider using a library that provides promisification (even if you use native promises all-around) since it provides a safe alternative that does not have subtle errors with error-handling logic.
Automatic promisification is also considerably faster.
That said, the answer to your question is "Yes".
It is perfectly safe to do so, there is nothing special about promise constructors - they are just plain JavaScript. Domenic discusses the design of the promise constructor in his blog.
It is perfectly safe (just like any other function) to return early - it is actually quite common in regular asynchronous functions.
(Also, in your example code you should just use Promise.resolve, but I assume it was that simple only because it is an example).
Copied this answer from duplicate
As #JaromandaX said in this case the return statement does not make any diference.
From the docs:
In all cases where a promise is resolved (i.e. either fulfilled or rejected), the resolution is permanent and cannot be reset. Attempting to call resolve, reject, or notify if promise is already resolved will be a no-op.
Folks,
I have the following function, and am wondering whats the correct way to call the callback() only when the database operation completes on all items:
function mapSomething (callback) {
_.each(someArray, function (item) {
dao.dosomething(item.foo, function (err, account) {
item.email = account.email;
});
});
callback();
},
What I need is to iterate over someArray and do a database call for each element. After replacing all items in the array, I need to only then call the callback. Ofcourse the callback is in the incorrect place right now
Thanks!
The way you currently have it, callback is executed before any of the (async) tasks finish.
The async module has an each() that allows for a final callback:
var async = require('async');
// ...
function mapSomething (callback) {
async.each(someArray, function(item, cb) {
dao.dosomething(item.foo, function(err, account) {
if (err)
return cb(err);
item.email = account.email;
cb();
});
}, callback);
}
This will not wait for all your database calls to be done before calling callback(). It will launch all the database calls at once in parallel (I'm assuming that's what dao.dosomething() is). And, then immediately call callback() before any of the database calls have finished.
You have several choices to solve the problem.
You can use promises (by promisifying the database call) and then use Promise.all() to wait for all the database calls to be done.
You can use the async library to manage the coordination for you.
You can keep track of when each one is done yourself and when the last one is done, call your callback.
I would recommend options 1. or 2. Personally, I prefer to use promises and since you're interfacing with a database, this is probably not the only place you're making database calls, so I'd promisify the interface (bluebird will do that for you in one function call) and then use promises.
Here's what a promise solution could look like:
var Promise = require('bluebird');
// make promise version of your DB function
// ideally, you'd promisify the whole DB API with .promisifyAll()
var dosomething = Promise.promisify(dao.dosomething, dao);
function mapSomething (callback, errCallback) {
Promise.all(_.map(someArray, function(item) {
return dosomething(item.foo).then(function (account) {
item.email = account.email;
});
}).then(callback, errCallback);
}
This assumes you want to run all the DB calls in parallel and then call the callback when they are all done.
FYI, here's a link to how Bluebird promisify's existing APIs. I use this mechanism for all async file I/O in node and it saves a ton of coding time and makes error handling much more sane. Async callbacks are a nightmare for proper error handling, especially if exceptions can be thrown from async callbacks.
P.S. You may actually want your mapSomething() function to just return a promise itself so the caller is then responsible for specifying their own .then() handler and it allows the caller to use the returned promise for their own synchronization with other things (e.g. it's just more flexible that way).
function mapSomething() {
return Promise.all(_.map(someArray, function(item) {
return dosomething(item.foo).then(function (account) {
item.email = account.email;
});
})
}
mapSomething.then(mapSucessHandler, mapErrorHandler);
I haven't tried Bluebird's .map() myself, but once you've promisified the database call, I think it would simplify it a bit more like this:
function mapSomething() {
return Promise.map(someArray, function(item) {
return dosomething(item.foo).then(function (account) {
item.email = account.email;
});
})
}
mapSomething.then(mapSucessHandler, mapErrorHandler);