AWS Lambda for db trigger not getting completed - node.js

I have created a lambda for dynamodb trigger and it is working ok if I use lambda function with callback but as soon as I change it to async it stops working. It is getting called but call returns before the end of the internal function. I am using await for any async call.
const handler: Handler = async (event: any, context: Context, callback : Callback) => { console.log(event);
try{
// construct request handler
console.log(event.Records);
const createHandler = dummyHandler;
await createHandler.handleRequest(event.Records);
return callback(null, 'Successfully processed ${event.Records.length} records.');
}
catch(err){
console.log(err);
throw err;
}
finally{
console.log('Finally I am here');
}
}
So this code works. Though strangely finally is executed before the handleRequest completes. But still handleRequest does complete as expected.
But since callback is kinda older and unwanted version I am trying to remove it and add promise in my async handler and then it stops working as expected.
const handler: Handler = async (event: any, context: Context): Promise<boolean> =>
{
console.log(event);
try{
// construct request handler
console.log(event.Records);
const createHandler = dummyHandler;
const result = await createHandler.handleRequest(event.Records);
return result;
}
catch(err){
console.log(err);
throw err;
}
finally{
console.log('Finally I am here');
}
}
In my internal function, I am doing few db queries and updates. I am using await in all async calls. But still after my first db query, finally block is being executed. In handler with callback, other db calls still execute but in async version with no callback, after finally is called, nothing else happens and request is ended.
I have checked memory size (256MB) and timeout (300seconds) for lambda function and neither of them are exceeding.

So in case anyone else experiences this issue. I ended up making following changes to my async handler without callback.
const handler: Handler = async (event: any, context: Context):Promise<boolean> => {
console.log(event);
try{
// construct request handler
console.log(event.Records);
const createHandler = new ...();
return await createHandler.handleRequest(event.Records);
}
catch(err){
console.log(err);
throw err;
}
finally{
console.log('Event Record');
}
This seems to be working as expected. All my db queries and updates are completed (basically this call createHandler.handleRequest is ended properly) before the request ends. Though needs extensive testing. Still welcome any feedback.
Basically it was working with promise but jarmod was right, I just needed to return the await call.

Related

Firebase onCall method finishes before code execution completes

I have a rather intensive Node function that includes multiple aync tasks that waits for the previous action to complete prior to going to the next step. It's crucial that the entire task finishes when it is called. I'm seeing issues where the onCall function will end before the code execution finishes. I can see this in the logs, as I'm console logging the various methods called from top to bottom until the function is "complete". The parent function is here:
exports.myHandyFunction = functions.https.onCall((data, context) => {
return new Promise((resolve, reject) => {
const response = async () => {
try {
let result = await some_main_function_elsewhere_in_code();
resolve(result)
} catch (err) {
console.log('parent promise caught error:')
console.log(err)
reject(err)
}
}
response()
})
})
I've increased the function's timeout from 60s to 240s hoping that this is purely a function timeout issue? This is pretty troubling, as I have some complex logic being called that ends up not fully completing. Any help would be greatly appreciated.
Your response function is async. When you call it, you're not awaiting for it's execution. This is leading on the premature end of the oncall method execution.
Instead of wrapping the business logic inside an async function, wrapped inside a promise, you can change the code like this:
exports.myHandyFunction = functions.https.onCall((data, context) => {
return new Promise(async (resolve, reject) => {
try {
let result = await some_main_function_elsewhere_in_code();
resolve(result)
} catch (err) {
console.log('parent promise caught error:')
console.log(err)
reject(err)
}
})
})
Basically, the Promise is declared as async, so we can use the await constructor inside of it. Then, we just have to call the some_main_function_elsewhere_in_code method, awaiting for it.

Why would this AWS lambda cause error: WARNING: Callback/response already delivered

I created an API Gateway + lambda for signUp with amazon-cognito-identity-js.
Then I implemented a Cognito trigger function for preSignUp with Typescript
I use Serverless framework to pack and deploy. The runtime is Node 12
+++++++
const wrapperHandler: Handler<CognitoUserPoolEvent> = async (
event,
context,
callback
) => {
let error = null;
try {
await myAsyncFunc();
} catch (e) {
error = e;
}
callback(error, event);
};
Everything works fine, it can return the error to the actual endpoint lambda which will then be returned, if no error, the logic will be executed.
However, this warning is pretty annoying.
The code is for preSignUp in CloudWatch
WARNING: Callback/response already delivered. Did your function invoke the callback and also return a promise? For more details, see: https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
In the code, I didn't return anything before calling the callback, why would this happen? and how to solve it.
As your error message what did you get, you are mixing callback style with async/await style, then it throws a warning.
I prefer use async/await. This mean, handler function always is a async function (with async keyword), then instead of call callback function, just return the result, and you no need callback parameter in handler function.
In error case, just throw the error (without try/catch block).
const wrapperHandler: Handler<CognitoUserPoolEvent> = async (
event,
context,
// callback
) => {
// let error = null;
try {
await myAsyncFunc();
} catch (e) {
// error = e;
// Do something with your error
throw e;
}
// callback(error, event);
return event; // just return result for handler function
};
In simple:
const wrapperHandler: Handler<CognitoUserPoolEvent> = async (
event,
context,
) => {
await myAsyncFunc();
return event;
};

AWS SQS Node.js script does not await

Trying to send several messages (from AWS SQS lambda, if that matters) but it's never waiting for the promises.
function getEndpoint(settings){
return new Promise(function(resolve, reject) {
// [...] more stuff here
}
Which is then called in a loop:
exports.handler = async (event) => {
var messages = [];
event.Records.forEach(function(messageId, body) {
//options object created from some stuff
messages.push(getEndpoint(options).then(function(response){
console.log("anything at all"); //NEVER LOGGED
}));
});
await Promise.all(messages);
};
But the await seems to be flat out skipped. I'm not sure how I'm getting Process exited before completing request with an explicit await. I have similar async await/promise setups in other scripts that work, but cannot spot what I've done wrong with this one.
You forgot to return something to lambda:
exports.handler = async (event) => {
var messages = [];
event.Records.forEach(function(messageId, body) {
//options object created from some stuff
messages.push(getEndpoint(options));
});
await Promise.all(messages);
return 'OK'
};
this should also work:
exports.handler = (event) => { // async is not mandatory here
var messages = [];
event.Records.forEach(function(messageId, body) {
//options object created from some stuff
messages.push(getEndpoint(options));
});
return Promise.all(messages); // returning a promise
};
and you could use map:
exports.handler = (event) => { // async is not mandatory here
const messages = event.Records.map(function(messageId, body) {
//options object created from some stuff
return getEndpoint(options)
});
return Promise.all(messages); // returning a promise
};
To understand why this happens, you must dive a bit into lambda's implementation: it will essentially wait for the function stack to be cleared and since you did NOT return anything at all in there, the function stack got empty right after it queued all the stuff - adding a simple return after the await call makes the fn stack to NOT be empty which means lambda will wait for it to be finished.
If you run this on standard node, your function would also return before the promises were finished BUT your node process would NOT exit until the stack was cleared. This is where lambda diverges from stock node.

Asynchronous await In Node.js

How to use the asynchronous Await in Node.js by using these function and how
The request.get() function returns a Promise by which user will await...
I have tried the Below code so far and also gave the explanation below
async function fun1(req, res){
let response = await request.get('http://localhost:3000');
if (response.err) { console.log('error');}
else { console.log('fetched response');
}
The code above basically asks the javascript engine running the code to wait for the request.get() function to complete before moving on to the next line to execute it. The request.get() function returns a Promise for which user will await . Before async/await, if it needs to be made sure that the functions are running in the desired sequence, that is one after the another, chain them one after the another or register callbacks.
async function fun1(req, res){
let response = await request.get('http://localhost:3000');
if (response.err) { console.log('error');}
else { console.log('fetched response');
}
request package does not use return promise. Use the request-promise package which wraps the request with Promise.
You can use it like:
const rp = require('request-promise')
async function getSomeData() {
try {
const url = 'http://some.com'
// waits for promise to resolve
const data = await rp(url)
// data contains resolved value if successfull
// continue some other stuff
...
} catch (e) {
// handle error if error occurred
console.error(e)
}
}

Why my lambda is working only when i give a callback with some message?

I am trying to run a small snippet of lambda code where i am pushing data to S3 using firehose. Here is my snippet
const AWS = require( 'aws-sdk' );
var FIREhose = new AWS.Firehose();
exports.handler = async (event,context,callback) => {
// TODO implement
const response = {
statusCode:200,
Name:event.Name,
Value:event.Value
};
const params = {
DeliveryStreamName: 'kinesis-firehose',
Record: { Data: new Buffer(JSON.stringify(response)) }
};
FIREhose.putRecord(params, (err, data) => {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data);
});
};
Here are my events
{
"Name": "Mike",
"Value": "66"
}
When i run this lambda all i am getting response as null . Since i am not passing any callback lambda will default run the implicit callback and returns null. I see that no data is pushed to S3 bucket.
But when i add callback(null,"success") line at the end like this
FIREhose.putRecord(params, (err, data) => {
if (err) console.log(err, err.stack); // an error occurred
else console.log(data);
});
callback(null,"success")
};
I see the data is pushed to S3. Why is that ?
Does async functions always need a callback with some text appended to it ?
Any help is appreciated ?
Thanks
The problem here is that you're mixing your node.js lambda patterns.
Either you use an asynchronous function and return or throw:
exports.handler = async (event,context,callback) => {
// code goes here.
await FIREhose.putRecord(params).promise();
return null; // or whatever result.
};
Or you use the callback approach:
exports.handler = (event,context,callback) => {
// code goes here.
FIREhose.putRecord(params)
.promise();
.then((data) => {
// do stuff with data.
// n.b. you could have used the cb instead of a promise here too.
callback(null, null); // or whatever result.
});
};
(There's a third way using context. but that's a very legacy way).
This is all due to how lambda works and detects when there's been a response.
In your first example (no callback), lambda is expecting your handler to return a promise that it has to wait to resolve/reject, which, in turn, will be the response. However, you're not returning a promise (undefined) and so there's nothing to wait for and it immediately returns- quite probably before the putRecord call has completed.
When you used callback though, you explicitly told lambda that you're using the "old" way. And the interesting thing about the callback approach is that it waits for node's event loop to complete (by default). Which means that .putRecord will probably complete.

Resources