AWS Lambda functions NodeJs for postgreSQL - timeout error - node.js

I am new to AWS. I am trying to connect to AWS RDS postgreSQL instance using Lambda functions. I followed the aws documentation. But it uses python for Lambda functions. Below is my code.
'use strict';
var pg = require('pg');
exports.handler = function (event, context) {
var dbConfig = {
username: '<username>',
password: '<password>',
database: '<database>',
host: '<db-endpoint>',
};
var client = new pg.Client(dbConfig);
try {
client.connect();
context.callbackWaitsForEmptyEventLoop = false;
client.end();
}
catch (err) {
console.log(err);
client.end();
}
};
I am getting timeout error as below
START RequestId: 368e619e-ed9d-4241-93a5-764ee01aa847 Version: $LATEST
2020-06-15T16:28:18.911Z 368e619e-ed9d-4241-93a5-764ee01aa847 INFO connected
END RequestId: 368e619e-ed9d-4241-93a5-764ee01aa847
REPORT RequestId: 368e619e-ed9d-4241-93a5-764ee01aa847 Duration: 20020.16 ms Billed Duration: 20000 ms Memory Size: 128 MB Max Memory Used: 70 MB Init Duration: 150.01 ms
2020-06-15T16:28:38.901Z 368e619e-ed9d-4241-93a5-764ee01aa847 Task timed out after 20.02 seconds
Please advise on the error.
I have few other questions to ensure if my code is correct
I gave db instance endpoint url for db-endpoint. is that right? or if not what should i use there?
is there any proper documentation available, for the beginners like me, about Lambda functions with nodejs to connect postgres on RDS?

You're not returning anything from the lambda. So the request keeps hanging without a response until it times out.
Use the third argument callback supplied to the handler to respond or return a Promise.
'use strict';
var pg = require('pg');
exports.handler = function (event, context,callback) {
var dbConfig = {
username: '<username>',
password: '<password>',
database: '<database>',
host: '<db-endpoint>',
};
var client = new pg.Client(dbConfig);
try {
client.connect();
context.callbackWaitsForEmptyEventLoop = false;
client.end();
//send the response
callback(null,"Some Response")
}
catch (err) {
console.log(err);
client.end();
callback(err)
}
};
AWS example : AWS Lambda NodeJS Connect to RDS Postgres Database
You can read the official js docs with all methods and properties here : https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/RDS.html
Hope this helps !

Here is the answer for async/await syntax
const { Client } = require("pg");
exports.handler = async (event, context, callback) => {
const dbConfig = {
host: process.env.RDS_HOSTNAME,
user: process.env.RDS_USERNAME,
password: process.env.RDS_PASSWORD,
port: process.env.RDS_PORT,
database: process.env.RDS_DATABASE,
};
const client = new Client(dbConfig);
try {
await client.connect();
const res = await client.query("SELECT * FROM your_table");
await client.end();
callback(null, res.rows);
} catch (err) {
await client.end();
callback(err)
}
};

Related

Locally run Lambda would not log all outputs

I have the following Lambda.
const AWS = require('aws-sdk');
exports.lambdaHandler = async (event, context) => {
console.log("START v5");
const SNSarn = process.env.SNS;
console.log(SNSarn);
const snsParams={
Message: "test",
TopicArn: SNSarn
};
const SNS = new AWS.SNS();
SNS.publish(snsParams, function(err,data){
console.log ("We are in the callback!");
if (err) {
console.log('Error sending a message', err);
context.done(err,'We have errors');
}
console.log('Sent message');
context.done(null,"Done!");
});
console.log('END');
};
When I run it locally, ther lambda performs as expected and pushes a message to SNS.
For some reason, I do not see the console.log that is in the callback.
How do I fix it so I can also see logs from callbacks?
Below is the output I do see:
START RequestId: ff2691cc-a7d0-40f2-a956-277bxxxx21d Version: $LATEST
2021-11-15T21:17:17.968Z ff2691cc-a7d0-40f2-a956-277bxxxx821d INFO START v5
2021-11-15T21:17:17.970Z ff2691cc-a7d0-40f2-a956-277bxxxx821d INFO arn:aws:sns:us-east-1:xxxxxxxxxxxxxxx:tooktook-topic
2021-11-15T21:17:17.992Z ff2691cc-a7d0-40f2-a956-277bxxxx821d INFO END
END RequestId: ff2691cc-a7d0-40f2-a956-277bxxxx821d
REPORT RequestId: ff2691cc-a7d0-40f2-a956-277bxxxx821d Init Duration: 0.19 ms Duration: 261.33 ms Billed Duration: 300 ms Memory Size: 128 MB Max Memory Used: 128 MB
In method SNS.publish, you are using callback which is asynchronous in nature and there is a possibility of successful lambda execution before callback is even called. SNS takes a bit of time for execution. you need to use async/ await in this case. AWS APIs support promises.
Here is an example:
const AWS = require('aws-sdk');
exports.lambdaHandler = async (event, context) => {
console.log("START v5");
const SNSarn = process.env.SNS;
console.log(SNSarn);
const snsParams={
Message: "test",
TopicArn: SNSarn
};
const SNS = new AWS.SNS();
try {
const data = await SNS.publish(snsParams).toPromise();
console.log('Sent Message');
context.done(null, 'Done');
} catch (error) {
console.log('We are in the callback!');
context.done(err, 'We have errors');
}
console.log('END');
};

Wrong aws request signature caused by opentelemetry https plugin

When using the #opentelemetry/plugin-https and the aws-sdk together in a NodeJS application, the opentelemetry plugin adds the traceparent header to each AWS request. This works fine if there is no need for retries in the aws-sdk. When the aws-sdk retries a request the following errors can occur:
InvalidSignatureException: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.
The first AWS request contains the following headers:
traceparent: '00-32c9b7adee1da37fad593ee38e9e479b-875169606368a166-01'
Authorization: 'AWS4-HMAC-SHA256 Credential=<credential>, SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-security-token;x-amz-target, Signature=<signature>'
Note that the SignedHeaders doesn't include traceparent.
The retried request contains the following headers:
traceparent: '00-c573e391a455a207469ffa4fb75b3cab-6f20c315628cfcc0-01'
Authorization: AWS4-HMAC-SHA256 Credential=<credential>, SignedHeaders=host;traceparent;x-amz-content-sha256;x-amz-date;x-amz-security-token;x-amz-target, Signature=<signature>
Note that the SignedHeaders does include traceparent.
Before the retry request is sent, the #opentelemetry/plugin-https sets new traceparent header and this makes the signature of the AWS request invalid.
Here is a code which reproduces the issue (you may need to run the script a few times before hitting the rate limit which causes the retries):
const opentelemetry = require("#opentelemetry/api");
const { NodeTracerProvider } = require("#opentelemetry/node");
const { SimpleSpanProcessor } = require("#opentelemetry/tracing");
const { JaegerExporter } = require("#opentelemetry/exporter-jaeger");
const provider = new NodeTracerProvider({
plugins: {
https: {
enabled: true,
path: "#opentelemetry/plugin-https"
}
}
});
const exporter = new JaegerExporter({ serviceName: "test" });
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register();
const AWS = require("aws-sdk");
const main = async () => {
const cwl = new AWS.CloudWatchLogs({ region: "us-east-1" });
const promises = new Array(100).fill(true).map(() => new Promise((resolve, reject) => {
cwl.describeLogGroups(function (err, data) {
if (err) {
console.log(err.name);
console.log("Got error:", err.message);
console.log("ERROR Request Authorization:");
console.log(this.request.httpRequest.headers.Authorization);
console.log("ERROR Request traceparent:");
console.log(this.request.httpRequest.headers.traceparent);
console.log("Retry count:", this.retryCount);
reject(err);
return;
}
resolve(data);
});
}));
const result = await Promise.all(promises);
console.log(result.length);
};
main().catch(console.error);
Possible solutions:
Ignore all calls to aws in the #opentelemetry/plugin-https.
Ignoring the calls to aws will lead to loosing all spans for aws requests.
Add the traceparent header to the unsignableHeaders in the aws-sdk - AWS.Signers.V4.prototype.unsignableHeaders.push("traceparent");
Changing the prototype seems like a hack and also doesn't handle the case if another node module uses different version of the aws-sdk.
Is there another solution which could allow me to keep the spans for aws requests and guarantees that the signature of all aws requests will be correct?
Update (16.12.2020):
The issue seems to be fixed in the aws sdk v3
The following code throws the correct error (ThrottlingException):
const opentelemetry = require("#opentelemetry/api");
const { NodeTracerProvider } = require("#opentelemetry/node");
const { SimpleSpanProcessor } = require("#opentelemetry/tracing");
const { JaegerExporter } = require("#opentelemetry/exporter-jaeger");
const { CloudWatchLogs } = require("#aws-sdk/client-cloudwatch-logs");
const provider = new NodeTracerProvider({
plugins: {
https: {
enabled: true,
path: "#opentelemetry/plugin-https"
}
}
});
const exporter = new JaegerExporter({ serviceName: "test" });
provider.addSpanProcessor(new SimpleSpanProcessor(exporter));
provider.register();
const main = async () => {
const cwl = new CloudWatchLogs({ region: "us-east-1" });
const promises = new Array(100).fill(true).map(() => new Promise((resolve, reject) => {
cwl.describeLogGroups({ limit: 50 })
.then(resolve)
.catch((err) => {
console.log(err.name);
console.log("Got error:", err.message);
reject(err);
});
}));
const result = await Promise.all(promises);
console.log(result.length);
};
main().catch(console.error);

Able to connect to redis but set/get times out

I'm trying to do a get() from my AWS Lambda (NodeJS) on ElastiCache Redis using node_redis client. I believe that I'm able to connect to redis but I'm getting Time out (Lambda 60 sec time out) when I'm trying to perform a get() operation.
I have also granted my AWS lambda Administrator access just to be certain that it's not a permissions issue. I'm hitting lambda by going to AWS console and clicking the Test button.
Here is my redisClient.js:
const util = require('util');
const redis = require('redis');
console.info('Start to connect to Redis Server');
const client = redis.createClient({
host: process.env.ElastiCacheEndpoint,
port: process.env.ElastiCachePort
});
client.get = util.promisify(client.get);
client.set = util.promisify(client.set);
client.on('ready',function() {
console.log(" subs Redis is ready"); //Can see this output in logs
});
client.on('connect',function(){
console.log('subs connected to redis'); //Can see this output in logs
})
exports.set = async function(key, value) {
console.log("called set!");
return await client.set(key, value);
}
exports.get = async function(key) {
console.log("called get!"); //Can see this output in logs
return await client.get(key);
}
Here's my index.js which calls the redisClient.js:
const redisclient = require("./redisClient");
exports.handler = async (event) => {
const params = event.params
const operation = event.operation;
try {
console.log("Checking RedisCache by calling client get") // Can see this output in logs
const cachedVal = await redisclient.get('mykey');
console.log("Checked RedisCache by calling client get") // This doesn't show up in logs.
console.log(cachedVal);
if (cachedVal) {
return {
statusCode: 200,
body: JSON.stringify(cachedVal)
}
} else {
const setCache = await redisclient.set('myKey','myVal');
console.log(setCache);
console.log("*******")
let response = await makeCERequest(operation, params, event.account);
console.log("CE Request returned");
return response;
}
}
catch (err) {
return {
statusCode: 500,
body: err,
};
}
}
This is the output (time out error message) that I get:
{
"errorMessage": "2020-07-05T19:04:28.695Z 9951942c-f54a-4b18-9cc2-119eed65e9f1 Task timed out after 60.06 seconds"
}
I have tried using Bluebird (changing get to getAsync()) per this: https://github.com/UtkarshYeolekar/promisify-redis-client/blob/master/redis.js but still got the same behavior.
I also changed the port to use a random value (like 8088) that I'm using to create client (to see the behavior of connect event for a failed connection) - in this case I still see a Timed Out error response but I don't see the subs Redis is ready and subs connected to redis in my logs.
Can anyone please point me in the right direction? I don't seem to understand why I'm able to connect to redis but the get() request times out.
I figured out the issue and posting here in case it helps anyone in future as the behavior wasn't very intuitive for me.
I had enabled AuthToken param while setting up my redis. I was passing the param to lambda with the environment variables but wasn't using it while sending the get()/set() requests. When I disabled the AuthToken requirement from redis configuration - Lambda was able to hit redis with get/set requests. More details on AuthToken can be found here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticache-replicationgroup.html#cfn-elasticache-replicationgroup-authtoken

Lambda function completes, but query to RDS is incomplete

I was trying to make a post query the db (RDS) using handler.async.
However, I ran into the following issues.
Half of the time, the lambda function completes but the query is not successfully sent to RDS. The other half of the time, it will be completely send to lambda. Tried adding a setTimeout function to increase lambda execution time by 3 secs and the query will be sent all the time.
Also the log will shows the error:
INFO Error: Cannot enqueue Query after fatal error.
The following are my code:
var mysql = require('mysql');
var connection = mysql.createConnection({
host : '***',
user : '***',
password : '***',
database : '***'
});
exports.handler = async (event) => {
const sql = `INSERT INTO forms VALUES(777,2,3,4,5,6,7,8,9,10,11);`;
const query = (x) => {
return new Promise ((resolve,reject)=>{
resolve(connection.query(x, function (error, results, fields) {
console.log(error)
console.log(results)
console.log(fields)
}))})}
await query(sql)
}
With the timeout function,
var mysql = require('mysql');
var connection = mysql.createConnection({
host : '***',
user : '***',
password : '***',
database : '***'
});
exports.handler = async (event) => {
const sql = `INSERT INTO forms VALUES(777,2,3,4,5,6,7,8,9,10,11);`;
const query = (x) => {
return new Promise ((resolve,reject)=>{
resolve(connection.query(x, function (error, results, fields) {
console.log(error)
console.log(results)
console.log(fields)
}))})}
await query(sql)
await wait(3000)
}
const wait = (x) => {
return new Promise ((resolve,reject)=>{
setTimeout(()=>{resolve(console.log("delay"))}, x);
})
}
The first value is a primary key. A constant 777 is sent to check, if error shows duplicate primary key, it will mean that the query is successfully sent. If there is no error, it means that the query is unsuccessfully sent although lambda finishes.
execution result succeeded but shows:
START RequestId: e541fe4b-6927-4fbb-90b4-750f77e5f460 Version: $LATEST
2019-12-19T01:54:45.212Z e541fe4b-6927-4fbb-90b4-750f77e5f460 INFO Error: **Cannot enqueue Query after fatal error**.
at Protocol._validateEnqueue (/var/task/node_modules/mysql/lib/protocol/Protocol.js:212:16)
at Protocol._enqueue (/var/task/node_modules/mysql/lib/protocol/Protocol.js:138:13)
at Connection.query (/var/task/node_modules/mysql/lib/Connection.js:201:25)
at /var/task/index.js:14:24
at new Promise (<anonymous>)
at query (/var/task/index.js:13:10)
at Runtime.exports.handler (/var/task/index.js:20:7)
at Runtime.handleOnce (/var/runtime/Runtime.js:66:25) {
code: 'PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR',
fatal: false
}2019-12-19T01:54:45.213Z e541fe4b-6927-4fbb-90b4-750f77e5f460 INFO undefined2019-12-19T01:54:45.213Z e541fe4b-6927-4fbb-90b4-750f77e5f460 INFO undefined2019-12-19T01:54:45.262Z e541fe4b-6927-4fbb-90b4-750f77e5f460 INFO delayEND RequestId: e541fe4b-6927-4fbb-90b4-750f77e5f460
REPORT RequestId: e541fe4b-6927-4fbb-90b4-750f77e5f460 Duration: 51.09 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 80 MB
May you please advise and also tell me which is the best way to execute it??
Managing RDBMS connections in any environment is not a trivial task. Lambda adds a layer of complexity here. You need to understand the distinction between warm and cold restarts, what it means for resources created outside of your handler function, when connection pools are appropriate, and when and how to release connections.
Persistent connections to a database are not particularly suitable in a microservices, FaaS environment like Lambda. That's one reason that Aurora Serverless supports an HTTP Data API (and hopefully other DB engines will too at some point).
Read How To: Manage RDS Connections from AWS Lambda Serverless Functions.
Also be aware of the new Amazon RDS Proxy with AWS Lambda.
In your particular case, the most obvious concern is that you are repeatedly creating DB connections but never releasing them (unless that is a built-in feature of the mysql package's query function that I'm not aware of).
You can increase the lambda timeout upto 15 minutes. But if you are calling the lambda through api gateway, the timeout is 29 seconds.
here is the code working for me.
const mysql = require('mysql');
const con = mysql.createConnection({
host: process.env.RDS_HOSTNAME,
user: process.env.RDS_USERNAME,
password: process.env.RDS_PASSWORD,
port: process.env.RDS_PORT,
connectionLimit: 10,
multipleStatements: true,// Prevent nested sql statements
debug: true
// ,database:'testdb1'
});
exports.handler = async (event) => {
try {
const data = await new Promise((resolve, reject) => {
con.connect(function (err) {
if (err) {
reject(err);
}
const sql = `INSERT INTO forms VALUES(777,2,3,4,5,6,7,8,9,10,11);`;
con.query(sql, function (err, result) {
if (err) {
console.log("Error->" + err);
reject(err);
}
resolve(result);
});
})
});
return {
statusCode: 200,
body: JSON.stringify(data)
}
} catch (err) {
return {
statusCode: 400,
body: err.message
}
}
};
reference: aws lambda with rds mysql DDL command not working

AWS RDSDataService times out whereas external mysql package works

I have a AWS Lambda (Node.js) talking to an Aurora database. Both belong to the same VPC, with internet access enabled via subnet. The RDS cluster also has a inbound rule that allows traffic from the VPC, used for the Lambda (which should be the same VPC). To my surprise, I found that the RDSDataService from AWS-SDK fails to connect to the database, whereas when I use mysql pacakge, it works. Following are the 2 code snippets.
I would like it very much to use AWS-SDK, as that will reduce the deployment bundle size, as I don't have to include that in the bundle that at all. Is there anyway to achieve that?
Failed attempt to use RDSDataService
const AWS = require("aws-sdk");
const rdsData = new AWS.RDSDataService({
params: {
dbClusterOrInstanceArn: 'rds.cluster.arn',
awsSecretStoreArn: 'rds.cluster.secret.arn',
database: 'mydb'
},
endpoint: 'mydb.endpoint'
});
return new Promise((resolve, reject) => {
try {
rdsData.executeSql({
dbClusterOrInstanceArn: 'rds.cluster.arn',
awsSecretStoreArn: 'rds.cluster.secret.arn',
database: 'mydb',
sqlStatements: "select 1 + 1 as result;"
}, (err, data) => {
if (err) {
reject(err);
}
const response = {
statusCode: 200,
body: JSON.stringify(data),
};
resolve(response);
});
} catch (er) {
reject(er);
}
});
Working implementation using mysql
const mysql = require('mysql');
const connection = mysql.createConnection({
host: 'mydb.endpoint',
user: 'user',
password: 'password',
port: 3306,
database: 'mydb',
debug: false
});
connection.connect(function (err) {
if (err) context.fail();
else {
connection.query('select 1 + 1 as result', function (error, results, fields) {
if (error) throw error;
resolve('The solution is: ' + JSON.stringify(results, undefined, 2));
});
}
});
connection.end();
As it turned out, Data API is not yet available for my region. The supported regions are listed here: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.html#data-api.regions

Resources