How to asynchronously make HTTP Requests in AWS Lambda Node.js runtime - node.js

I have created an API endpoint that returns an integer when it is successfully accessed with an HTTP Post request. I want an AWS CloudWatch scheduled process to run an AWS Lambda function every minute to check the API endpoint so make sure the value is not zero. I have set up the scheduled AWS CloudWatch process and the AWS Lambda function, where the runtime is Node.js 10. However, when I look at the AWS CloudWatch group's stream log, the logs seem to be out of order. I suspect this is because of the HTTP Request to the API endpoint is running asynchronously, but I ultimately do not know. The time that it takes to log the value of this HTTP Request to the API end point seems to be much longer than the actual time to process the request.
Here is the Node.js lambda function that is being run on a minutely basis:
exports.handler = async (event) => {
var datetime = new Date();
var request = require("request");
var options = {
method: 'POST',
url: 'https://website.com/api/getDataPoints',
headers:
{
'cache-control': 'no-cache',
'content-type': 'text/plain'
},
body: '{"token" : "yT7g8urUFmEZwQrJNHgQGRDA9zScpNzPM3rb"}'
};
await request(options, function (error, response, body) {
if (error)
{
throw new Error(error);
// Email and SMS message that this is having an error
}
if (body == 0)
{
// Email and SMS message that this is having an error
// Restart EC2 server
}
console.log(datetime.toString() + " - " + body + " Data Points!");
});
};
Here is the AWS CloudWatch log, where it's easier to see the delay in logging the response from the HTTP Request:
Any insight into the cause of this perceived log delay or suggestions on how to achieve similar results in a more efficient way would be greatly appreciated! Thank you!

Here is what is happening: the await is returning immediately because the request function does not return a promise, so the callback is happening after the lambda function exits. The reason why there is such a long delay in seeing the log is that when your lambda function becomes idle (meaning all handler functions have returned, even if there are callbacks waiting) AWS can suspend any executing code until a new invocation of the function. So in your case when the next minute rolls around and the lambda function is invoked again, AWS will un-suspend any executing code and the callback kicks off immediately and you see the log for the previous invocation.
To solve this you want to make sure the handler function doesn't return until all work has been completed (which is what I think you were intending to do with the await). So wrap the request call in a function that returns a promise which is resolved in the callback and use await on the wrapper function.
function doRequest() {
return new Promise((resolve, reject) => {
request(options, function (error, response, body) {
if (error){
throw new Error(error);
// Email and SMS message that this is having an error
}
if (body == 0){
// Email and SMS message that this is having an error
// Restart EC2 server
}
console.log(datetime.toString() + " - " + body + " Data Points!");
resolve();
});
});
}
await doRequest();

Related

how to get the first bytes of a file from a given url node.js

I want to a file from a given url, in a aws lambda function.
I wrote this code:
exports.handler = (event, context, callback) => {
var http = require('http');
var url= "https://mail.google.com/mail/u/0/?ui=2&ik=806f533220&attid=0.1&permmsgid=msg-a:r-8750932957918989452&th=168b03149469bc1f&view=att&disp=safe&realattid=f_jro0gbqh0"
//var client = http.createClient(80, url);
var request = http.request({
port: 80,
host: url
});
request.on('response', function( res ) {
res.on('data', function( data ) {
console.log(data);
});
});
request.end();
const result = {
statusCode: 200,
body: JSON.stringify('Hello from Lambda!'),
};
callback(null, result);
};
but I get an error saying:
"Response: {
"errorMessage": "RequestId: 52baec5e-60bc-47ea-911e-8e6cb1d2f1da Process exited before completing request"
}"
since I only need the first 2 bytes' I thought maybe I should read them, and not the whole file.
any ideas?
thanks a lot!
Did you increase your lambda execution Timeout limit? When you first created, lambda comes setted with only 3 seconds by default. You can change that under Basic settings. Change Timeout to 2 or 3 minutes to allow your lambda to finish execution. Also check if your memory is enough. You may need to increase it a little bit. I have mine with 256 MB.
When you test your lambda, pay attention at the Duration and Memory Size values. Lambda will print that in the last line of the log output. So if you set your lambda execution Timeout to 5 minutes and it only takes 2 minutes or if your lambda Memory Size is to close to your Memory you might want to increase it so your lambda do not fail execution due to a memory problem.
I usually do this with fetch or request. You can make it with request like this:
exports.handler = (event, context, callback) => {
var request = require('request');
var url= "https://mail.google.com/mail/u/0/?ui=2&ik=806f533220&attid=0.1&permmsgid=msg-a:r-8750932957918989452&th=168b03149469bc1f&view=att&disp=safe&realattid=f_jro0gbqh0"
request(url, { json: true, timeout: 1000 }, (err, response, body) => {
if (err) {
console.log(err);
callback(err, null);
} else {
console.log(body);
callback(null, "Hello from Lambda");
}
});
};
Just run npm install request to get module request and you are good to go.
It is always a good idea to start with your local development environment and when everything is ready, you just zip your files and then upload them to lambda. This way you know everything is just fine with your code and you can focus on the lambda configuration details. This way it much easier to test and you do not consume any lambda resource.
This is how to upload your file to lambda:

Sending an eMail with Lambda, NodeJs and Nodemailer doesn't work

I'm trying to send an email from Lambda with a previously SES authenticated account; with node.js using Nodemailer. There is no error, but it doesn't send neither.
This is the process I'm using for:
module.exports.eviarCorreoPrueba = (event, context, callback) => {
context.callbackWaitsForEmptyEventLoop = false;
console.log('inicia envio de correos');
var transporter = nodemailer.createTransport({
//pool: true,
host: 'email-smtp.us-east-1.amazonaws.com',
port: 465,
secure: true,
auth: {
user: 'user',
pass: 'pass'
}
});
console.log('se crea transporte ');
var mailOptions = {
from: 'test#email.com',
to: 'test#email.com',
subject: 'Prueba Lambda',
html: 'hello World'
};
console.log('se asignan las opciones de correo');
console.log('inicia envio de correo');
transporter.sendMail(mailOptions, function (error, info) {
if (error) {
callback(null, {
statusCode: 200,
body: JSON.stringify({
input: 'not send'
})
})
} else {
console.log('Email sent');
}
});
console.log('funcion finalizada');
};
And these are the log answer results:
Just in case anyone gets stuck with 'nodemailer not working in Lambda' issue:
The code in your post works in local server because:
In your local environment, there is a running server to schedule all call stacks including synchronous and asynchronous tasks. Once you called transporter.sendMail() in local environment, it will be placed to the end of the current call stack and will be executed until the last execution in your call stack finished. In your case, the asynchronous function transporter.sendMail() will be scheduled to be called after console.log('funcion finalizada');
Why it is not working in Lambda:
Every time when you call the lambda function, it will execute the code and then kill the process after the execution finished, which means it won't be able to call the asynchronous transporter.sendMail() because after console.log('funcion finalizada'); the lambda function process will be terminated and the scheduled asynchronous tasks will be cleared before get executed.
To make it work in Lambda:
1) Change your function to an async function
module.exports.eviarCorreoPrueba = async (event, context) => { ... }
2) await your transporter.sendMail() to be called before continue
const response = await new Promise((rsv, rjt) => {
transporter.sendMail(mailOptions, function (error, info) {
if (error) {
return rjt(error)
}
rsv('Email sent');
});
});
return {
statusCode: 200,
body: JSON.stringify({
input: response
})
}
3) Last thing to make your nodemailer emailbot work:
You need to turn Less secure app access on and Allow access to your Google Account because you are simply using username and password to connect your Gmail account.
*Note: If you want to use securer method to connect your Gmail (eg. OAuth 2.0), you can refer to my article: Create a Free Serverless Contact Form Using Gmail, Nodemailer, and AWS Lambda
Hope this helps anyone who gets stuck with this issue.
Cheers,
Your Lambda is timing out. See the last message in your Cloudwatch logs. You set your lambda timeout to 6 seconds I suppose which is not enough time for the lambda to send out the request (via transporter.sendMail) and get the response. Try increasing the lambda time. To about 30 seconds maybe?
The way I set timeouts for new lambdas I design is to run them several times till I get an average completion time. Then I add 20 seconds to that time.
Also, transporter.sendMail is asynchronous function. This means console.log('funcion finalizada') will probably run before your function completes (yet your function hasn't actually completed).
Read more on asynchronous javascript and callbacks: https://medium.com/codebuddies/getting-to-know-asynchronous-javascript-callbacks-promises-and-async-await-17e0673281ee
Also, If you want to write asynchronous code in a synchronous looking way use async/await

Will AWS Lambda wait for my response?

I'm using AWS lambda as SQS listener in my project. Some time no response got in my log, most of the time it's working.
code
console.log('Befor Req ' + post_data);
// Set up the request
var post_req = http.request(post_options, function(res) {
res.setEncoding('utf8');
res.on('data', function (response) {
console.log('Request data' + post_data);
console.log('Response ' + response);
});
res.on('error', function (err) {
console.log(err);
})
});
// post the data
post_req.write(post_data);
post_req.end();
Here I got all fine in before request, sometimes the log inside the request not get logging.
I check the other side request and response are logged and all fine.
Is this any specific issue with AWS LAMBDA?
The log inside the request is an asynchronous callback's either log inside your on success or your failure callback should prints.
If not There are two posibilities.
one is you are terminating lambda before that async code completed
posibility no 2 is, your lambda got timesout before that async request completed . see this scenario may works on sometime and fails on other time. It depends on our async execution time and lambda timeout
TO ensure this check your cloudwatch log for lines task timedout after x second. And you will be get 502 on that time if you are using lambda proxy in API gateway. see below

How to send http request with nodejs AWS Lambda?

I'm using AWS Lambda to drive an Alexa Skill Kit development. In an attempt to track events, I'd like the script to send an HTTP request on launch, however from the cloud logs it appears as though the http.get function is being skipped during the execution process.
The code is shown below (google.com replaces the analytics tracking url - which has been tested in the browser);
exports.handler = function (event, context) {
var skill = new WiseGuySkill();
var http = require('http');
var url = 'http://www.google.com';
console.log('start request to ' + url)
http.get(url, function(res) {
console.log("Got response: " + res.statusCode);
// context.succeed();
}).on('error', function(e) {
console.log("Got error: " + e.message);
// context.done(null, 'FAILURE');
});
console.log('end request to ' + url);
skill.execute(event, context);
};
The context objects have been commented out to allow for 'skill.execute' to function, yet either way this HTTP request is not executing. Only the 'start' and 'end' console.logs are recorded, those internal in the function do not.
Is this a async issue? Thanks.
You need to make sure the handler is being triggered. There are two ways of accomplishing this:
You could set up a new API endpoint and execute a request on that.
You could hit the Test button and your function would be invoked with the given data.
I copied and pasted your whole snippet except for the first and the last lines (because I don't have customSkill defined anywhere). I was able to get a 200 response code.
In order to successfully complete the http request, the http.get function must be incorporated into a callback function. Else the process will not be completed and will end prematurely, using a callback allows the http request to complete (with or without an error) before continuing with the rest of the function.
WiseGuySkill.prototype.eventHandlers.onLaunch = function (launchRequest, session, response) {
// Call requestFunction to make the http.get call.
// Get response from requestFunction using requestCallback
requestFunction(function requestCallback(err) {
// If error occurs during http.get request - respond with console.log
if (err) {
console.log('HTTP Error: request not sent');
}
ContinueIntent(session,response);
});
};
The function 'requestFunction' calls http.get and fires the callback.
function requestFunction(requestCallback){
var url = "http://www.google.com";
http.get(url, function(res) {
console.log("Got response: " + res.statusCode);
requestCallback(null);
}).on('error', function (e) {
console.log("Got error: ", e);
});
}
And obviously ensure you have required 'http' at the start of the script.
Hope this helps anybody else new to this!

AWS Lambda get context message

I am using the test function from AWS console:
console.log('Loading event');
exports.handler = function(event, context) {
console.log('value1 = ' + event.key1);
console.log('value2 = ' + event.key2);
console.log('value3 = ' + event.key3);
context.done(null, 'Hello World'); // SUCCESS with message
};
And calling it in nodejs as follows:
var params = {
FunctionName: 'MY_FUNCTION_NAME', /* required */
InvokeArgs: JSON.stringify({
"key1": "value1",
"key2": "value2",
"key3": "value3"
})
};
lambda.invokeAsync(params, function(err, data) {
if (err) {
// an error occurred
console.log(err, err.stack);
return cb(err);
}
// successful response
console.log(data);
});
and everything works fine:
//Console Output
{ Status: 202 }
But I was expecting to receive the message from context.done(null, 'Message') as well...
Any idea how to get the message?
As Eric mentioned, currently Lambda doesn't offer a REST endpoint to run the function and return its result, but may in the future.
Right now, your best bet would be to use a library like lambdaws, which wraps the function deployment and execution for you and handles returning results via an SQS queue. If you'd like more control by rolling your own solution, the process is straightforward:
Create an SQS queue
Have your Lambda function write its result to this queue
In your client, poll the queue for a result
You are calling invokeAsync, so your Lambda function is run asynchronously. This mean you get the success means back at the point your function is successfully started, and not after it completes.
As of this writing, AWS Lambda does not yet offer a way to invoke a function synchronously, returning information from the function directly back to the caller. However, this appears to be a common request and Amazon has publicly stated they are considering the feature.

Resources