In my finis callback, I have:
finis((code, signal, error) => {
await myprocess.stop()
console.log(`finis(${code}, ${signal}, ${error})`)
})
However, I'm getting the error:
Can not use keyword 'await' outside an async function
So how to do await within finis' callback?
PS. finis()'s callback function will be run just before the node process exits, and I want to gracefully shutdown myprocess by myprocess.stop() before program exist.
The syntax error can be fixed by declaring your callback to be async.
But, it appears that you have bigger issues than that because you want the caller of your callback (the function finis) to wait for your asynchronous operation to complete before it does the rest of its work. await inside your callback by itself will not do that.
Here are a few notes about what await does:
await only does anything useful if you await a promise. It has no magic powers to await some mythical asynchronous operation. It only awaits a promise. So, in your particular code, myprocess.stop() must return a promise that it resolved when the underlying asynchronous operation in myprocess.stop() completes for the await in await myprocess.stop() to do anything useful.
await only blocks execution within the local async function. The containing function still returns as soon as you hit the first await. So, it won't block the caller at all.
An async function (that you can use await in) returns a promise so the only way the caller of that function (the caller of your callback in your case which is finis() would actually wait for your async operation to complete is if it was expecting the callback to return a promise and if it used either .then() or await on that callback itself.
If you control the code for the finis() function, then we could probably help you modify it to do what you want (to look for a returned promise before it shuts things down), but if you don't control that function, then you're probably out of luck.
Try the following.
finis(async (code, signal, error) => {
await myprocess.stop()
console.log(`finis(${code}, ${signal}, ${error})`)
})
Related
When I hover on the keyword 'function' the description says:
"(local function)(this: any, next: (err?: mongoose.CallbackError | undefined) => void): Promise<void>"
So does It return a Promise<void> or a simple <void>? I can't even understand what does this function returns? And to be honest I don't understand really well the concept of Promise<void>...
userSchema.pre('save', async function (next) {
let user = this as UserDocument;
if(!user.isModified('password')){
return next();
}
const salt = await bcrypt.genSalt(config.get<number>('saltWorkFactor'));
const hash = await bcrypt.hash(user.password, salt);
user.password = hash;
return next();
})
This question is really interesting. Your function returns a Promise<void>, which is compatible with the void return type that pre is expecting, but Mongoose is quietly smart enough to know what to do with your Promise so you don't even have to call next at all.
First some background:
void has a special meaning in TypeScript to mean that the return value could be any value; the value is frequently undefined (because that's what a function returns without a return statement) but it doesn't have to be. As in the TypeScript FAQ, this makes it convenient to accept or pass functions that return a value by indicating the return value is unused. If you need to supply a function with return type void, you could pass back a function that returns a string, Promise<void>, Promise<SomeObject>, null, undefined, or anything else.
All async functions return Promises, and this is no exception. A Promise<number> is a Promise that says that its then function will receive a number; a Promise<void> is a Promise that doesn't tell you anything about what its then function receives. The then function will still be called, unless it has an error to catch; you just don't know much about its argument.
In Mongoose's types, pre takes a PreSaveMiddlewareFunction<T> function, which is the type of the function you wrote. It accepts a function called next and returns void: Mongoose claims not to care what you return. Your middleware function is allowed to be asynchronous; when you're done you're expected to call next (with an error object, if you have one), and that call to next also returns void.
Your function passed to pre returns type Promise<void>: The function is async so it absolutely returns a promise, and your return next(); means that the Promise resolves to whatever next returns, which is defined as void. You don't know what next returns and shouldn't care about it. You don't even need to return next(), you just need to call it: It's just a callback so you can tell Mongoose your middleware is done and report any errors.
So your async function returns Promise<void>, but that works with the definition of pre: pre doesn't care what kind of return value your function has (void) as long as you call next to indicate you're done.
But wait! Reporting that your asynchronous function is done and whether or not there were errors is exactly the problem that Promises were designed to solve, and the next callback pattern is exactly the kind of pattern that Promises were designed to replace. If you're returning a Promise, why would you need to call next at all when Mongoose can just watch the promise you return?
In fact, in Mongoose 5.x or later, that's exactly what happens: If the function you pass into pre returns a Promise, then you can use that instead of calling next. You can still call next manually for compatibility's sake, but in your case you could delete return next() and everything would keep working. See the middleware docs:
In mongoose 5.x, instead of calling next() manually, you can use a function that returns a promise. In particular, you can use async/await.
schema.pre('save', function() {
return doStuff().
then(() => doMoreStuff());
});
// Or, in Node.js >= 7.6.0:
schema.pre('save', async function() {
await doStuff();
await doMoreStuff();
});
The docs further explain why return next() is a pattern at all:
If you use next(), the next() call does not stop the rest of the code in your middleware function from executing. Use the early return pattern to prevent the rest of your middleware function from running when you call next().
const schema = new Schema(..);
schema.pre('save', function(next) {
if (foo()) {
console.log('calling next!');
// `return next();` will make sure the rest of this function doesn't run
/*return*/ next();
}
// Unless you comment out the `return` above, 'after next' will print
console.log('after next');
});
In summary, the expected return type of void is compatible with the fact that you're returning a Promise<void>, but it hides the fact that recent versions of Mongoose are smart enough to check whether you're returning a Promise and do the right thing without needing a call to next. They're two different styles that both work.
Long answer short: It return a Promise<void>
Callbacks
To understand why, here are some details.
First one must understand Callbacks in node.js. Callbacks are one of the basic structure/feature of how node.js works.
You could say that node.js is basically an Event-Driven Programming "framework" (most people will frown to the framework word...). That means that you tell node that in the event of a certain thing happening, it should do a certain action/function (callback).
For node to understand us, we normally give the callback function as a parameter to another function that will do the work of "listening to the event" and executing the callback that we give it. So it is not "us" that execute the callback, it is the event listener.
In your case,
userSchema.pre('save', async function (next) {
pre is the function (a method in Mongoose's userSchema), save is the event that one must react to, async function (next) { is the callback or what must be done after the event.
You will note that your callback is returning next(), but next() returns void, which mean that your callback is returning void.
So why is it returning Promise<void>?
The fact is that in your case, your callback is an async function. And every async functions will return a promise. It is an async function because it is awaiting another promise (two promises even) inside of it. They are hidden because of the await
const salt = await bcrypt.genSalt(config.get<number>('saltWorkFactor'));
const hash = await bcrypt.hash(user.password, salt);
Note: The bcrypt methods are very expensive in terms of CPU and time (also a security feature among other things).
It also means that normally in your code
const hash = await bcrypt.hash(user.password, salt);
user.password = hash;
you couldn't have available "right away" the hash value for the user.password and, worse, you couldn't even know when it would come. Will your program stop and wait until bcrypt finish its business?
If you have many async functions, your program will be a great favourite for the slowest champion in the Olympics.
What is going on with those promises and how can we not be labelled as a geriatric program?
Promises
Here is a quick/long comment to try to explain the concept of promises.
In "normal" code, each lines of code is executed and "finished" before the next one. Ex: (with cooking)
Combine the Butter and Sugar,
Add Eggs One at a Time, etc.
Or in your code:
let user = this as UserDocument;
if(!user.isModified('password')){
return next();
}
A promise is a certain code that is executed but not finished before the next line of code. Ex:
while the cake is in the oven (promise),
you prepare the frosting,
but you can't put it until the cake in baked (the "then" action of promises).
Note: Your code is using await so there is no "explicit" then method.
You will have many example of "promises" things in everyday life. you may have heard of asynchronous code = not one after the other, not in sync, ...
Turning on an alarm to wake you in the morning, then you make the promise that you will not ignore it;
putting a reminder on the calendar then you make the promise that you will go to that job interview; etc.
All the while, you continue with your life after making those promises.
In code, a function that returns a promise will have a then method where you tell the computer what to do when when the "alarms goes off".
It is usually written like this
mypromise().then(doThisThingFunction)
const continueWithMyLife = true
In this way the then method is very similar to the callback of node.js. It is just expressed in a different way in the code and is not specific to node (callbacks are also not specific to node...).
One very important difference between them is that callbacks are something that the listener "do" and promises is something that resolves (hopefully) to a returning value.
Async/Await
Nowadays it is common to use async/await. Fortunately/unfortunately it basically hides the asynchronous behaviour. Better flow of reading the code, but also much worse understanding of promises for new programmers.
After a await, there is no then method (Or you could say that the following line of code is the then action). There is no "continuing with your life". There is only "waiting until the alarms goes off", So the next line after the await is essentially the "get out of the bed action".
That is why, in your code, the hash value is available in the next line. Basically in the "old way" to write promises
user.password = hash;
would be inside the then function.
And that is also why it is returning Promise<void>
But still, all these analogies won't really help. The best is to try it in everyday code. There is nothing like experience to understand anything.
Assume an Express route that makes a call to Mongoose and has to be async so it can await on the mongoose.find(). Also assume we are receiving XML but we have to change it to JSON, and that also needs to be async so I can call await inside of it.
If I do this:
app.post('/ams', async (req, res) => {
try {
xml2js.parseString(xml, async (err, json) => {
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
} catch(err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
}
The server hangs forever. I'm assuming the async/await means that the server hits a timeout before something concludes.
If I put this:
res.status(200);
res.send("Data saved")
on the line before the catch(), then that is returned, but it is the only thing every returned. The client gets a 200, even if an XMLException is thrown.
I can see the XMLException throw in the console, but I cannot get a 400 to send back. I cannot get anything I that catch block to execute in a way that communicates the response to the client.
Is there a way to do this?
In a nutshell, there is no way to propagate an error from the xml2js.parseString() callback up to the higher code because that parent function has already exited and returned. This is how plain callbacks work with asynchronous code.
To understand the problem here, you have to follow the code flow for xml2js.parseString() in your function. If you instrumented it like this:
app.post('/ams', async (req, res) => {
try {
console.log("1");
xml2js.parseString(xml, async (err, json) => {
console.log("2");
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
console.log("3");
} catch (err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
console.log("4");
});
Then, you would see this in the logs:
1 // about to call xml2js.parseString()
3 // after the call to xml2js.parseString()
4 // function about to exit
2 // callback called last after function returned
The outer function has finished and returned BEFORE your callback has been called. This is because xml2js.parseString() is asynchronous and non-blocking. That means that calling it just initiates the operation and then it immediately returns and the rest of your function continues to execute. It works in the background and some time later, it posts an event to the Javascript event queue and when the interpreter is done with whatever else it was doing, it will pick up that event and call the callback.
The callback will get called with an almost empty call stack. So, you can't use traditional try/catch exceptions with these plain, asynchronous callbacks. Instead, you must either handle the error inside the callback or call some function from within the callback to handle the error for you.
When you try to throw inside that plain, asynchronous callback, the exception just goes back into the event handler that triggered the completion of the asynchronous operation and no further because there's nothing else on the call stack. Your try/catch you show in your code cannot catch that exception. In fact, no other code can catch that exception - only code within the exception.
This is not a great way to write code, but nodejs survived with it for many years (by not using throw in these circumstances). However, this is why promises were invented and when used with the newer language features async/await, they provide a cleaner way to do things.
And, fortunately in this circumstance xml2js.parseString() has a promise interface already.
So, you can do this:
app.post('/ams', async (req, res) => {
try {
// get the xml data from somewhere
const json = await xml2js.parseString(xml);
// do something with json here
res.send("Data saved");
} catch (err) {
console.log(err);
res.status(400).send("Malformed XML error: " + err.message);
}
});
With the xml2js.parseString() interface, if you do NOT pass it a callback, it will return a promise instead that resolves to the final value or rejects with an error. This is not something all asynchronous interfaces can do, but is fairly common these days if the interface had the older style callback originally and then they want to now support promises. Newer interfaces are generally just built with only promise-based interfaces. Anyway, per the doc, this interface will return a promise if you don't pass a callback.
You can then use await with that promise that the function returns. If the promise resolves, the await will retrieve the resolved value of the promise. If the promise rejects, because you awaiting the rejection will be caught by the try/catch. FYI, you can also use .then() and .catch() with the promise, but in many cases, async and await are simpler so that's what I've shown here.
So, in this code, if there is invalid XML, then the promise that xml2js.parseString() returns will reject and control flow will go to the catch block where you can handle the error.
If you want to capture only the xml2js.parseString() error separately from other exceptions that could occur elsewhere in your code, you can put a try/catch around just it (though this code didn't show anything else that would likely throw an exception so I didn't add another try/catch). In fact, this form of try/catch can be used pretty much like you would normally use it with synchronous code. You can throw up to a higher level of try/catch too.
A few other notes, many people who first start programming with asynchronous operations try to just put await in front of anything asynchronous and hope that it solves their problem. await only does anything useful when you await a promise so your asynchronous function must return a promise that resolves/rejects when the asynchronous operation is complete for the await to do anything useful.
It is also possible to take a plain callback asynchronous function that does not have a promise interface and wrap a promise interface around it. You pretty much never want to mix promise interface functions with plain callback asynchronous operations because error handling and propagation is a nightmare with a mixed model. So, sometimes you have to "promisify" an older interface so you can use promises with it. In most cases, you can do that with util.promisify() built into the util library in nodejs. Fortunately, since promises and async/await are the modern and easier way to do asynchronous things, most newer asynchronous interfaces in the nodejs world come with promise interfaces already.
You are throwing exceptions inside the callback function. So you cant expect the catch block of the router to receive it.
One way to handle this is by using util.promisify.
try{
const util = require('util');
const parseString = util.promisify(xml2js.parseString);
let json = await parsestring(xml);
}catch(err)
{
...
}
I know the setInterval func is to run some periodic task. But I want to know what the difference between async and sync func. Here is an example.
const fetchFile = async ()=>{//dosometing}
setInterval(async()=>{
fetchFile()
}, 1000)
setInterval(async()=>{
await fetchFile()
}, 1000)
setInterval(()=>{
fetchFile()
}, 1000)
There is no practical difference to the interval itself between any of your three implementations. setInterval() does not pay any attention to the return value from the callback so it doesn't matter at all that the async version returns a promise.
You can declare it async if you want to use await inside the callback for your own implementation, but that won't affect the event loop whether you use await or .then(). With await, as soon as you hit the await on a promise, the callback will return immediately and the event loop will be able to process other events. The same is true if you do fn().then(). While the asynchronous operation is running and the promise is waiting to get notified of completion, the event loop can process other events.
You may also want to know that there's no difference between these two:
fn(async()=>{
fetchFile()
}, 1000)
fn(async()=>{
await fetchFile()
}, 1000)
even outside of the setInterval() world. Doing an await on the last line of an async function doesn't add anything. It's just superfluous. But, in the setInterval() case, since setInterval() doesn't pay any attention to the return value from the callback, there's no practical difference between any of your three implementations.
In fact, there's no difference between return fetchFile() and return await fetchFile() either in an async function.
In all your examples, you have NO error handling on fetchFile(). If it rejects, you will have an unhandled rejection. To catch those errors, you would either surround an await with try/catch or use .catch().
First of all you pass async function callback to setInterval function which not expects async function in result you lack of proper error handling;
So if you want to run some periodic function I propose something like this, where you have more control on your code with error handling. Keep in mind that this could block event loop as your code could run thru microstask queeue only.
const util = require('util');
const sleep = util.promisify(setTimeout);
const fetchFile = async () => {
console.log('some async function');
};
async function fetchSomeFile() {
try {
await fetchFile();
sleep(1000);
fetchSomeFile();
} catch (err) {
console.log(err);
}
}
fetchSomeFile().catch(err => console.log(err));
I propose to watch good presentation - Broken Promises - James Snell, NearForm https://youtu.be/XV-u_Ow47s0
Hey there so I'm trying to store the result of a promise in a variable because I have some code that doesn't work inside of the promise.
(I'm pretty new to node.js so a bit of explanation would be awesome!)
And how would I use a async await?
Heres my current code:
const rbx = require("noblox.js")
var myblurb;
rbx.getBlurb(166962499).then(function (blurb) {
myblurb = blurb;
});
console.log(myblurb);
The function that sets your variable will get called after your function completes, not during it. Therefore your variable is still undefined after calling rbx.getBlurb because it returns a promise that will complete later on.
Using async/await:
One way to get what you're after is to use await. This can only be done within a function that is declared as async, and that will mean the function now will also return a Promise. If you're okay with that, then you can do it like this:
async function doStuff() {
var myblurb;
myblurb = await rbx.getBlurb(166962499);
console.log(myblurb);
}
This tells your function to stop and wait until the promise from calling getBlurb has resolved, and assign the resolved value to your local variable.
Note that if there's an error (either a promise rejection or a thrown exception) then your function will pass that along. You could surround it all with a try/catch to handle that if you need.
Operating Within then
If you don't want to make your function return a promise then you could alternately do this:
rbx.getBlurb(166962499).then(function (blurb) {
const myblurb = blurb;
console.log(myblurb);
// myblurb is only useful within this 'then' resolve function
});
It just means that you have to do the rest of your functionality that relies on the myblurb variable within that resolve function of the then.
Note that whoever is calling this code needs to understand that it will be asynchronous, so it should either use a callback function or Promise to give its results back to the caller (if needed)
Your callback function where assignment is happening will be called after promise is called. Console.log will be executed before that. So it will always log undefined.
You have to do console.log inside function of you can also use async await so solve this.
exports.handler = async (event, context, callback) => {
try {
const { headers, body } = event;
//This is where I forgot the "await" keyword
const input = ValidateInput(body); //Returns Promise
callback(null, true);
}catch(err){
console.log(err);
callback(null, false);
}
}
When calling a function that returns a promise and forgetting to create an await expression promise function call, and that function rejects the promise, Lambda logs this error in cloudwatch
(node:1) UnhandledPromiseRejectionWarning: #<Object>
The fix is simple, don't forget the await expression
const input = await ValidateInput(body); //Return Promise
The fix is simple, don't forget the await expression
const input = await ValidateInput(body); //Return Promise
As already mentioned, the solution is to ensure you await the promise:
const input = await ValidateInput(body);
But I thought I'd add a little context around why this occurs.
Since Promises can be stored in variables and chained at any point, there's no way for the library to know whether or not a Promise chain will have a .catch associated with it some time in the future. Many libraries therefore have a default behaviour of writing to the console if the rejected Promise has not been handled within a number of passes of the event loop; which is why you see this in the log.
You should generally take this warning as implying that you haven't awaited something you should have. As in reality, it's rare that you'd see it on purpose.
The AWS doc explicitly says you don't use callback with an async function.
The third argument, callback, is a function that you can call in
non-async functions to send a response. The callback function takes
two arguments: an Error and a response. The response object must be
compatible with JSON.stringify.
For async functions, you return a response, error, or promise to the
runtime instead of using callback.
So, you may want to fix that in your lambda function.
See here : https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html#nodejs-prog-model-handler-callback