Connect to AWS S3 bucket from Nodejs app ( persistent connection required) - node.js

Question

I have nodejs/express app from which I want to connect to AWS S3.
I do have a temporary approach to make connection,
environment file
aws_access_key_id=XXX
aws_secret_access_key=XXXX/
aws_session_token=xxxxxxxxxxxxxxxxxxxxxxxxxx
S3-connection-service.js
const AWS = require("aws-sdk");
AWS.config.update({
accessKeyId: `${process.env.aws_access_key_id}`,
secretAccessKey: `${process.env.aws_secret_access_key}`,
sessionToken: `${process.env.aws_session_token}`,
region: `${process.env.LOCAL_AWS_REGION}`
});
const S3 = new AWS.S3();
module.exports = {
listBucketContent: (filePath) =>
new Promise((resolve, reject) => {
const params = { Bucket: bucketName, Prefix: filePath };
S3.listObjects(params, (err, objects) => {
if (err) {
reject(err);
} else {
resolve(objects);
}
});
}),
....
....
}
controller.js
const fetchFile = require("../../S3-connection-service.js");
const AWSFolder = await fetchFile.listBucketContent(filePath);
Fine it's works and I'm able to access S3 bucket files and play with it.
PROBLEM
The problem is connection is not persistent. Since, I use session_token, connection remains alive for sometime and again after sometime new tokens will be generated, I have to copy-paste them in env file and re-run the node app.
I really have no idea how can I make connection persistent ?
Where to store AWS confidential/secrets and how to use them to connect to S3 so connection remains alive ?

Answer 1

Just remove
AWS.config.update({
accessKeyId: `${process.env.aws_access_key_id}`,
secretAccessKey: `${process.env.aws_secret_access_key}`,
sessionToken: `${process.env.aws_session_token}`,
region: `${process.env.LOCAL_AWS_REGION}`
});
code block from lambda source in file S3-connection-service.js
Attach a role to lambda function with proper permissions. You will have same functionally.
For local development.
You can set environment variable before testing your application.
export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_DEFAULT_REGION=us-west-2
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
If you are using any IDE you can set these environment variables on it.
If you are testing from cli
$ AWS_ACCESS_KEY_ID=EXAMPLE AWS_SECRET_ACCESS_KEY=EXAMPLEKEY AWS_DEFAULT_REGION=us-west-2 npm start

Answer 2

connect to S3 so connection remains alive ?
You can't make one request to S3 and keep it alive forever.
These are your options:
Add a try/catch statement inside your code to handle credentials expired error. Then, generate new credentials and re-initialize the S3 client.
Instead of using a Role, use a User. (IAM Identities). User credentials can be valid forever. You won't need to update the credentials in this case.
Do not provide the credentials to AWS.config.update like you are doing right now. If you don't provide the credentials, the AWS client will try to read them from your ~/.aws/credentials file automatically. If you create a script to update them every hour (ex: a cronjob), then your credentials will be up-to-date at all times.