exports.handler = async (event, context, callback) => {
try {
const { headers, body } = event;
//This is where I forgot the "await" keyword
const input = ValidateInput(body); //Returns Promise
callback(null, true);
}catch(err){
console.log(err);
callback(null, false);
}
}
When calling a function that returns a promise and forgetting to create an await expression promise function call, and that function rejects the promise, Lambda logs this error in cloudwatch
(node:1) UnhandledPromiseRejectionWarning: #<Object>
The fix is simple, don't forget the await expression
const input = await ValidateInput(body); //Return Promise
The fix is simple, don't forget the await expression
const input = await ValidateInput(body); //Return Promise
As already mentioned, the solution is to ensure you await the promise:
const input = await ValidateInput(body);
But I thought I'd add a little context around why this occurs.
Since Promises can be stored in variables and chained at any point, there's no way for the library to know whether or not a Promise chain will have a .catch associated with it some time in the future. Many libraries therefore have a default behaviour of writing to the console if the rejected Promise has not been handled within a number of passes of the event loop; which is why you see this in the log.
You should generally take this warning as implying that you haven't awaited something you should have. As in reality, it's rare that you'd see it on purpose.
The AWS doc explicitly says you don't use callback with an async function.
The third argument, callback, is a function that you can call in
non-async functions to send a response. The callback function takes
two arguments: an Error and a response. The response object must be
compatible with JSON.stringify.
For async functions, you return a response, error, or promise to the
runtime instead of using callback.
So, you may want to fix that in your lambda function.
See here : https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html#nodejs-prog-model-handler-callback
Related
Assume an Express route that makes a call to Mongoose and has to be async so it can await on the mongoose.find(). Also assume we are receiving XML but we have to change it to JSON, and that also needs to be async so I can call await inside of it.
If I do this:
app.post('/ams', async (req, res) => {
try {
xml2js.parseString(xml, async (err, json) => {
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
} catch(err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
}
The server hangs forever. I'm assuming the async/await means that the server hits a timeout before something concludes.
If I put this:
res.status(200);
res.send("Data saved")
on the line before the catch(), then that is returned, but it is the only thing every returned. The client gets a 200, even if an XMLException is thrown.
I can see the XMLException throw in the console, but I cannot get a 400 to send back. I cannot get anything I that catch block to execute in a way that communicates the response to the client.
Is there a way to do this?
In a nutshell, there is no way to propagate an error from the xml2js.parseString() callback up to the higher code because that parent function has already exited and returned. This is how plain callbacks work with asynchronous code.
To understand the problem here, you have to follow the code flow for xml2js.parseString() in your function. If you instrumented it like this:
app.post('/ams', async (req, res) => {
try {
console.log("1");
xml2js.parseString(xml, async (err, json) => {
console.log("2");
if (err) {
throw new XMLException();
}
// assume many more clauses here that can throw exceptions
res.status(200);
res.send("Data saved")
});
console.log("3");
} catch (err) {
if (err instanceof XML2JSException) {
res.status(400);
message = "Malformed XML error: " + err;
res.send(message);
}
}
console.log("4");
});
Then, you would see this in the logs:
1 // about to call xml2js.parseString()
3 // after the call to xml2js.parseString()
4 // function about to exit
2 // callback called last after function returned
The outer function has finished and returned BEFORE your callback has been called. This is because xml2js.parseString() is asynchronous and non-blocking. That means that calling it just initiates the operation and then it immediately returns and the rest of your function continues to execute. It works in the background and some time later, it posts an event to the Javascript event queue and when the interpreter is done with whatever else it was doing, it will pick up that event and call the callback.
The callback will get called with an almost empty call stack. So, you can't use traditional try/catch exceptions with these plain, asynchronous callbacks. Instead, you must either handle the error inside the callback or call some function from within the callback to handle the error for you.
When you try to throw inside that plain, asynchronous callback, the exception just goes back into the event handler that triggered the completion of the asynchronous operation and no further because there's nothing else on the call stack. Your try/catch you show in your code cannot catch that exception. In fact, no other code can catch that exception - only code within the exception.
This is not a great way to write code, but nodejs survived with it for many years (by not using throw in these circumstances). However, this is why promises were invented and when used with the newer language features async/await, they provide a cleaner way to do things.
And, fortunately in this circumstance xml2js.parseString() has a promise interface already.
So, you can do this:
app.post('/ams', async (req, res) => {
try {
// get the xml data from somewhere
const json = await xml2js.parseString(xml);
// do something with json here
res.send("Data saved");
} catch (err) {
console.log(err);
res.status(400).send("Malformed XML error: " + err.message);
}
});
With the xml2js.parseString() interface, if you do NOT pass it a callback, it will return a promise instead that resolves to the final value or rejects with an error. This is not something all asynchronous interfaces can do, but is fairly common these days if the interface had the older style callback originally and then they want to now support promises. Newer interfaces are generally just built with only promise-based interfaces. Anyway, per the doc, this interface will return a promise if you don't pass a callback.
You can then use await with that promise that the function returns. If the promise resolves, the await will retrieve the resolved value of the promise. If the promise rejects, because you awaiting the rejection will be caught by the try/catch. FYI, you can also use .then() and .catch() with the promise, but in many cases, async and await are simpler so that's what I've shown here.
So, in this code, if there is invalid XML, then the promise that xml2js.parseString() returns will reject and control flow will go to the catch block where you can handle the error.
If you want to capture only the xml2js.parseString() error separately from other exceptions that could occur elsewhere in your code, you can put a try/catch around just it (though this code didn't show anything else that would likely throw an exception so I didn't add another try/catch). In fact, this form of try/catch can be used pretty much like you would normally use it with synchronous code. You can throw up to a higher level of try/catch too.
A few other notes, many people who first start programming with asynchronous operations try to just put await in front of anything asynchronous and hope that it solves their problem. await only does anything useful when you await a promise so your asynchronous function must return a promise that resolves/rejects when the asynchronous operation is complete for the await to do anything useful.
It is also possible to take a plain callback asynchronous function that does not have a promise interface and wrap a promise interface around it. You pretty much never want to mix promise interface functions with plain callback asynchronous operations because error handling and propagation is a nightmare with a mixed model. So, sometimes you have to "promisify" an older interface so you can use promises with it. In most cases, you can do that with util.promisify() built into the util library in nodejs. Fortunately, since promises and async/await are the modern and easier way to do asynchronous things, most newer asynchronous interfaces in the nodejs world come with promise interfaces already.
You are throwing exceptions inside the callback function. So you cant expect the catch block of the router to receive it.
One way to handle this is by using util.promisify.
try{
const util = require('util');
const parseString = util.promisify(xml2js.parseString);
let json = await parsestring(xml);
}catch(err)
{
...
}
i think both of them are giving me same promise but later one doesn't work as promise. it gives me "TypeError: data.then is not a function"
try{
await docClient.get(params).promise().then(x => console.log(x));//this one works
const data = await docClient.get(params).promise();
data.then(x =>console.log(x));//this.doesen't
}
catch(err){
console.log(err);
}
Function Logs:
START RequestId: 914e5709-7cf2-4978-9b58-d338ebee52dc Version: $LATEST
2019-11-08T04:39:22.259Z 914e5709-7cf2-4978-9b58-d338ebee52dc
INFO "Event: event"
2019-11-08T04:39:22.437Z 914e5709-7cf2-4978-9b58-d338ebee52dc
INFO { Item: { firstname: 'Bob', id: '12345', lastname: 'Johnson' } }
2019-11-08T04:39:22.487Z 914e5709-7cf2-4978-9b58-d338ebee52dc
INFO TypeError: data.then is not a function
at Runtime.exports.handler (/var/task/index2.js:21:11)
at process._tickCallback (internal/process/next_tick.js:68:7)
await docClient.get(params).promise().then(x => console.log(x));
Means that the entire expression docClient.get(params).promise().then(x => console.log(x)) returns a Promise and you await until it's resolved.
await docClient.get(params).promise()
Means you get a promise out of this part of the expression and await it thus unwrapping it into a plain value, which means that data is not a Promise any more and thus data.then(x =>console.log(x)) fails.
If you want to preserve the Promise in the second case, then you can not await it:
const data = docClient.get(params).promise();
await data.then(x =>console.log(x));
Although this is probably not very good, since you're better off using either await or Promises instead of breaking the chained Promise API to await part of it. You can just drop the Promises API altogether
const data = await docClient.get(params).promise();
console.log(data);
Worth noting that you can use use the alternative syntax with a callback but Promise or await are probably better:
const data = docClient.get(params, function(err, data) {
console.log(data);
});
In your first statement:
docClient.get(params).promise()
is a promise, so of course, you can add a .then() handler to it like:
docClient.get(params).promise().then(...)
Putting an await in front of it:
await docClient.get(params).promise().then(...)
doesn't affect the .then() itself, that's still a method call on docClient.get(params).promise(). It just awaits the result of the .then(). If you need some extra parens to see the order of evaluation, it would be like this:
await ( docClient.get(params).promise().then(...) )
though the extra parens are not needed in execution.
In your second statement:
const data = await docClient.get(params).promise();
Because of the await, the variable data contains the resolved value of the promise. It's not a promise. So, when you attempt:
data.then()
there's no .then() method on the value in the data variable so it's an error.
Like it says in the error, data.then is not a function. Since you awaited the promise the value is scalar at that point and you would need to console.log(data) to see it.
In the below code snippet I am using then to get the result of a promise. However, response is not returned. What is returned is Promise { <pending> }.
I've logged response and can see it correctly returned the data, not a pending promise. So why is it returning a pending promise? I've even added a then call to the call to describeTable, and it still comes back pending.
I've read the following questions and they were not helpful, so please do not mark as a duplicate of them:
How do I return the response from an asynchronous call?
async/await implicitly returns promise?
const AWS = require('aws-sdk');
AWS.config.update({region: 'eu-west-2'});
const docClient = new AWS.DynamoDB;
async function describeTable() {
const params = {
TableName: 'weatherstation_test',
};
let response;
try {
response = await docClient.describeTable(params).promise().then(result => result);
console.log(response); // Logs the response data
} catch (e) {
console.error(e)
throw e;
}
return response;
}
console.log(describeTable().then(result => result)); // Logs Promise { <pending> }
Update
So I've removed the first then (after promise()) because it is redundant. The answer from #libik works for me. It was the context in which then is run I wasn't understanding.
You cannot access the asynchronous content from synchronous one.
If you want to log it, you have to do it from inside like this
describeTable().then(result => console.log(result))
In your case you are logging the output of asynchronous function, which is always a promise.
What really happens: In Node.js all the synchronous content is executed first and any asynchronous content is put to event loop to be executed later.
So first this is executed
const AWS = require('aws-sdk');
AWS.config.update({region: 'eu-west-2'});
const docClient = new AWS.DynamoDB;
console.log(describeTable()
The function is called, so then it goes inside function. Because it is asynchronous function, it will execute it synchronously till the first await.
const params = {
TableName: 'weatherstation_test',
};
let response;
try {
response = await docClient.describeTable(params).promise()
Now this promise is added to Event Loop to be executed later and the function describeTable() returns to the synchronous context (the console log) the promise, that will be linked through all your function asynchronously later and you log this promise (which is indeed pending).
And now your synchronous context ends. Before the above promise is resolved your app can meanwhile executing other parts (which is always the same, take event from event loop, do the full synchronous part - which can push another events to event loop and then take another event and repeat)
In order to log the result you need to write the following:
describeTable().then(result => console.log(result));
Maybe I just don't understand promises but I've used this pattern before and never had issues. Using bluebird within node.
I have this function which is getting hit:
function getStores() {
return new Bluebird((resolve, reject) => {
return Api.Util.findNearbyStores(Address,(stores) => {
if (!stores.result) {
console.log('one')
reject('no response');
console.log('two')
}
const status = stores.results.status
})
})
}
then hits both of my logs, continues past the if and throws
'one'
'two'
TypeError: Cannot read property 'Status' of undefined
Basically it keeps going right past the resolve.
My impression was the promise should immediately short circuit on reject and pass the rejection through as the resolution to the promise. Am I misunderstanding this?
Yes, you are misunderstanding this. reject(…) is not syntax (just like resolve(…) isn't either) and does not act like a return statement that exits the function. It's just a normal function call that returns undefined.
You should be using
if (!stores.result) reject(new Error('no response'));
else resolve(stores.results.status);
The "short-circuiting" behaviour of rejections is attributed to promise chains. When you have
getStores().then(…).then(…).catch(err => console.error(err));
then a rejection of the promise returned by getStores() will immediately reject all the promises in the chain and trigger the catch handler, ignoring the callbacks passed to then.
Since March 2016, the aws sdk for javascript/nodejs does provide the .promise() method on AWS.Request link.
Now I'm running some unit test on the AWS.DynamoDB.DocumentClient(...).get() method
(which returns an AWS.Request object) where I also call the .promise() method afterwards to receive a Promise.
Now when I stub the .get() method, Im already returning a Promise, so I want the .promise() method to do nothing more than returning my existing Promise (via "return this").
my code is like
var docClient = AWS.DynamoDB.DocumentClient(...)
docClient.get(...).promise().then(...)
and my test file
var docClient = AWS.DynamoDB.DocumentClient(...)
var batchGetStub = sinon.stub(docClient, 'get').callsFake(function(){return Promise.resolve({id:1337})});
// now I need to also stub the .promise() method
var promiseStub = sinon.stub(docClient.get, 'promise').callsFake(functions(){return this}); // just return the promise back
This doesn't seem to work and I'm getting back the following error
TypeError: Cannot stub non-existent own property promise
I guess I somehow need to declare a .promise method on my fake function.
I tried this with:
this.protoype.promise = function(this){return this};
inside my fake function, but this throws
TypeError: Cannot set property 'promise' of undefined
The problem here is that you're treating the get function as returning a promise - which is doesn't.
get() returns an object that has a function called promise on it, which when called returns a promise object.
So what you actually need to do is something like:
var docClient = AWS.DynamoDB.DocumentClient(...)
var batchGetStub = sinon.stub(docClient, 'get').callsFake(function(){
return {
promise: function () {
return Promise.resolve({id:1337})
}
};
});
Tristan is right in the answer above: the get() function call does not return a promise, it returns an object that has a function called "promise" on it, which when called should return a promise object.
You can use sinon for this if you want to, but in simple cases it might be preferable to just stub the object manually, as follows:
const getStub = {
get: (params) => ({
promise: async () => ({id:1337}),
})
};
You don't need to return Promise.resolve if you use the async key word, it will return the {id: 1337} object as a resolved promise by default.
One other note: you're intermingling terminology around get and batchGet. These are different dynamo document client functions with different return signatures, so be careful. Good luck!