This is a question about how to manage a project with functions in separate files.
main file
const AWS = require('aws-sdk');
const monkeyDynamodb = require("../includes/dynamodb/monkey-dynamodb-functions.js")
...
exports.handler = async function(event, context) {
let billingDevices = monkeyDynamodb.scan("SensorNodeBilling");
....
monkey-dynamodb-functions.js
async function scan(dbName) {
var docClient = new AWS.DynamoDB.DocumentClient();
...
If you compile this, you have 2 separate objects called AWS & AWS2, and a warning since one of them is not declared. And if you run it, it fails.
On the other hand, if you add in const AWS = require('aws-sdk'); into monkey-dynamodb-functions.js, it now works, but the compiled code still creates multiple AWS objects. What is the right way to code this?
You should have this:
const AWS = require('aws-sdk');
in any module that wants to use this AWS SDK. The module system uses a cache so the module itself will only be loaded once (the first time it is required) and from then after will just return the same exports object that was originally loaded.
The rest of your question is not entirely clear as there's no AWS2 object in any of your code that you show so it's unclear what you think you have two of. If you're just talking about the monkey-dynamodb-functions.js file, then yes, it must also require in the AWS module so it can use it.
Unlike in the browser, each of your nodejs files runs in its own module. So, when you do something like this:
const AWS = require('aws-sdk');
in the main file, that AWS symbol is local to only this module. It cannot be accessed directly by any other module. So, in any other module that you also want to use the AWS sdk, just require it into that module also.
You can share objects or functions from one module to others by exporting them from one module and then requiring them into another module from that module and every module that wishes to have access to that shared object would require it in separately. That's what the aws-sdk module is itself doing.
but the compiled code still creates multiple AWS objects
It's not entirely clear what you mean by this. If you mean that when do require in the aws-sdk in both modules, you will then have two separate variables for the AWS sdk, that is true. That is the correct way to program it. Each module contains its own set of local variables. Those two variables will each contain the exact same aws-sdk object reference. This is the correct way to code with modules in nodejs.
Main file
const AWS = require('aws-sdk');
const monkeyDynamodb = require("../includes/dynamodb/monkey-dynamodb-functions.js")
...
exports.handler = async function(event, context) {
let billingDevices = monkeyDynamodb.scan("SensorNodeBilling");
....
monkey-dynamodb-functions.js
const AWS = require('aws-sdk');
async function scan(dbName) {
var docClient = new AWS.DynamoDB.DocumentClient();
...
}
exports.scan = scan;
Related
I am trying to create an S3Client using the #aws-sdk/client-s3 package, as shown below:
const { S3Client } = require('#aws-sdk/client-s3')
const client = new S3Client({ region: 'us-east-1' })
It runs fine locally, but when I upload the code to a MongoDB Realm function along with the client-s3 dependency, it throws the error: {"message":"'emitWarning' is not a function","name":"TypeError"}
What might be causing this error?
It turns out this was happening because the MongoDB Realm Functions environment does not have an emitWarning function defined on the global process variable, which is why the error kept saying 'emitWarning' is not a function.
I reached out to MongoDB Support about this, and apparently the AWS-SDK v3 just isn't compatible with MongoDB Realm Functions.
Instead, you must use the AWS-SDK v2. I eventually found this MongoDB Forums Article, which explained that they specifically support v2.737.0 of the SDK.
I recently discovered DynamoDB Local and started building it into my project for local development. I decided to go the docker image route (as opposed to the downloadable .jar file.
That being said I've gotten image up and running and have created a table and can successfully interact with the docker container via the aws cli. aws dynamodb list-tables --endpoint-url http://localhost:8042 successfully returns the table I created previously.
However, when I run my lambda function and set my aws config like so.
const axios = require('axios')
const cheerio = require('cheerio')
const randstring = require('randomstring')
const aws = require('aws-sdk')
const dynamodb = new aws.DynamoDB.DocumentClient()
exports.lambdaHandler = async (event, context) => {
let isLocal = process.env.AWS_SAM_LOCAL
if (isLocal) {
aws.config.update({
endpoint: new aws.Endpoint("http://localhost:8042")
})
}
(which I have confirmed is getting set) it actually writes to the table (with the same name of the local dynamodb instance) in the live AWS Webservice as opposed to the local container and table.
It's also worth mentioning I'm unable to connect to the local instance of DynamoDB with the AWS NoSQL Workbench tool even though it's configured to point to http://localhost:8042 as well...
Am I missing something? Any help would be greatly appreciated. I can provide any more information if I haven't already done so as well :D
Thanks.
SDK configuration changes, such as region or endpoint, do not retroactively apply to existing clients (regular DynamoDB client or a document client).
So, change the configuration first and then create your client object. Or simply pass the configuration options into the client constructor.
When using the SNS client found on AWS-SDK:
const sns = new AWS.SNS({});
Should I reuse this object across calls to save a handshake with the server?
This kind of object is usually stateless and benefits from pooling/cache; However the docs aren't really clear about that.
I believe you should initiate the class outside of your lambda.
AWS will reuse the instance when possible.
E.g.
const AWS = require('aws-sdk')
const sns = new AWS.SNS()
module.exports.handler = async input => {
// use sns class here
return input
}
EDIT:
Link to the official documentation that explains how the lambda execution context works: https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html
Any declarations in your Lambda function code (outside the handler
code, see Programming Model) remains initialized, providing additional
optimization when the function is invoked again. For example, if your
Lambda function establishes a database connection, instead of
reestablishing the connection, the original connection is used in
subsequent invocations. We suggest adding logic in your code to check
if a connection exists before creating one.
I'm trying to take advantage of db connection reuse in Lambda, by keeping the code outside of the handler.
For example - something like:
import dbconnection from './connection'
const handler(event, context, callback){
//use dbconnection
}
The issue is I don't decide what database to connect to until I do a lookup to see where they should be connecting. In my specific case I have 'customer=foo' in a query param then I can look to see that foo should connect to database1.
So what I need to do is something like this :
const dbconnection = require('./connection)('database1')
The way it is now I need to do this in every handler method which is expensive.
Is there some way I can pull the query parameter, look up my database and set it / switch it globally within the Lambda execution context?
I've tried this:
import dbconnection from './connection'
const handler(event, context, callback){
const client = dbconnection.setDatabase('database1')
}
....
./connection.js
setDatabase(database) {
if(this.currentDatabase !== database) {
// connect to different database
this.currentDatabase = database;
}
}
Everything works locally with sls offline but doesn't work through the AWS Lambda execution context. Thoughts?
You can either hardcode (or provide it via environment variable) it or not. If you can, then pull it out of then handler and it will not be executed each time. If you can't, as you have mentioned, then what you are trying to do is to make lambda stateful. Lambda was designed to be stateless and AWS intentionally doesn't expose specific informations about the underlying containers so that you don't start doing something like what you are trying to do now - introducing state to it.
In an effort to improve cold start latency in AWS Lambda, I am attempting to include only the necessary classes for each Lambda function. Rather than include the entire SDK, How can I include only the DynamoDB portion of the SDK?
// Current method:
var AWS = require('aws-sdk');
var dynamodb = new AWS.DynamoDB();
// Desired method:
var AWSdynamodb = require('aws-dynamodb-sdk');
The short answer is: you do not need to do this.
The AWS SDK for JavaScript uses dynamic requires to load services. In other words, the classes are defined, but the API data is only loaded when you instantiate a service object, so there is no CPU overhead in having the entire package around.
The only possible cost would be from disk space usage (and download time), but note that Lambda already bundles the aws-sdk package on its end, so there is no download time, and you're actually using less disk space by using the SDK package available from Lambda than using something customized.
I don't think this is possible.
The npm registry only has aws-sdk. https://www.npmjs.com/package/aws-sdk
There may be other npm packages available for dynamodb, but I would advice only using the sdk provided by aws team.
Be sure to instantiate the SDK outside of the handler.
This example is good, and will result in a cold start time of about 1s
const AWS = require("aws-sdk");
const SNS = new AWS.SNS({apiVersion: '2010-03-31'});
exports.handler = function(event, context) {
//do stuff
};
This example is bad and will result in a cold start time of about 5s
exports.handler = function(event, context) {
const AWS = require("aws-sdk");
const SNS = new AWS.SNS({apiVersion: '2010-03-31'});
//do stuff
};
https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html
Take advantage of execution context reuse to improve the performance
of your function. Initialize SDK clients and database connections
outside of the function handler, and cache static assets locally in
the /tmp directory. Subsequent invocations processed by the same
instance of your function can reuse these resources. This saves
execution time and cost.