AWS SAM Local + DynamoDB Local - node.js

I'm currently testing out AWS SAM with DynamoDB Local using Docker.
Here is the steps that I followed (mostly found in the internet)
Create new docker network using docker network create local-dev.
Run DynamoDB Local docker run -d -v "$PWD":/dynamodb_local_db -p 8000:8000 --network local-dev --name dynamodb amazon/dynamodb-local. Until this point, I'm being able to create and list tables using AWS CLI.
Then, I proceed with running AWS SAM sam local start-api --docker-network local-dev. Everything looks okay.
Invoked lambda.js, but it looks like no result for console.log(err)or console.log(data).
I'm not sure where could it be wrong. Please help me. Thank you in advance!
lambda.js
const services = require('./services.js');
const AWS = require('aws-sdk');
let options = {
apiVersion: '2012-08-10',
region: 'ap-southeast-1',
}
if(process.env.AWS_SAM_LOCAL) {
options.endpoint = new AWS.Endpoint('http://localhost:8000')
}
const dynamoDB = new AWS.DynamoDB(options);
exports.getUser = async (event, context) => {
let params = {};
dynamoDB.listTables(params, (err, data) => {
if(err) console.log(err)
else console.log(data)
})
return true;
}
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless Resources
Parameters:
FunctionsCodeBucket:
Type: String
Description: CodeBucket
FunctionsCodeKey:
Type: String
Description: CodeKey
FunctionsCodeVersion:
Type: String
Description: CodeVersion
NodeEnv:
Type: String
Description: NodeEnv
Globals:
Api:
Cors:
AllowMethods: "'OPTIONS,POST,GET,DELETE,PUT'"
AllowHeaders: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,Api-Key,api-key'"
AllowOrigin: "'*'"
Function:
Timeout: 300
Runtime: nodejs10.x
MemorySize: 128
CodeUri: ./
Resources:
DevResources:
Type: AWS::Serverless::Function
Properties:
Handler: "index.routes"
Environment:
Variables:
NODE_ENV: !Ref NodeEnv
# REGION: !Ref "AWS::Region"
Policies:
- Version: '2012-10-17'
Statement:
- Action:
- dynamodb:*
Effect: Allow
Resource: "*"
Events:
GetUser:
Type: Api
Properties:
Path: /user
Method: get

You lambda function does not wait for dynamoDB.listTables operation. You can fix this issue by using promisified version of dynamoDB.listTables as follows:
exports.getUser = async (event, context) => {
let params = {};
try {
const resp = await dynamoDB.listTables(params).promise();
console.log(resp);
} catch (err) {
console.log(err)
}
};
Another thing that you will likely need to do is to assign a network alias to your dynamodb container (you can do that using --network-alias=<container_name> option) for example, let's set the alias to dynamodb
docker run -d -v "$PWD":/dynamodb_local_db -p 8000:8000 --network local-dev --network-alias=dynamodb --name dynamodb amazon/dynamodb-local
After that you can use this network alias in your lambda function:
if(process.env.AWS_SAM_LOCAL) {
options.endpoint = new AWS.Endpoint('http://dynamodb:8000')
}

Related

Use Redis with AWS SAM (Redis Client Error)

Right now what I'm trying to do is that every time a request is made, a query is made to the Redis service. The problem is that when using a basic configuration, it would not be working. The error is the following:
INFO Redis Client Error Error: connec at TCPConnectWrap.afterConnect [as oncomplete] (node} port: 6379127.0.0.1',
I have as always running redis-server with its corresponding credentials listening to port 127.0.0.1:6379. I know that AWS SAM runs with a container, and the issue is probably due to a network configuration, but the only command that AWS SAM CLI provides me is --host. How could i fix this?
my code is the following, although it is not very relevant:
import { APIGatewayProxyEvent, APIGatewayProxyResult } from 'aws-lambda';
import { createClient } from 'redis';
import processData from './src/lambda-data-dictionary-read/core/service/controllers/processData';
export async function lambdaHandler(event: APIGatewayProxyEvent): Promise<APIGatewayProxyResult> {
const body: any = await processData(event.queryStringParameters);
const url = process.env.REDIS_URL || 'redis://127.0.0.1:6379';
const client = createClient({
url,
});
client.on('error', (err) => console.log('Redis Client Error', err));
await client.connect();
await client.set('key', 'value');
const value = await client.get('key');
console.log('----', value, '----');
const response: APIGatewayProxyResult = {
statusCode: 200,
body,
};
if (body.error) {
return {
statusCode: 404,
body,
};
}
return response;
}
My template.yaml:
Transform: AWS::Serverless-2016-10-31
Description: >
lambda-data-dictionary-read
Sample SAM Template for lambda-data-dictionary-read
Globals:
Function:
Timeout: 0
Resources:
IndexFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: app/
Handler: index.lambdaHandler
Runtime: nodejs16.x
Timeout: 10
Architectures:
- x86_64
Environment:
Variables:
ENV: !Ref develope
REDIS_URL: !Ref redis://127.0.0.1:6379
Events:
Index:
Type: Api
Properties:
Path: /api/lambda-data-dictionary-read
Method: get
Metadata:
BuildMethod: esbuild
BuildProperties:
Minify: true
Target: 'es2020'
Sourcemap: true
UseNpmCi: true
Im using:
"scripts": {
"dev": "sam build --cached --beta-features && sam local start-api --port 8080 --host 127.0.0.1"
}

Localstack lambda trying to reach AWS outside container

I have a localstack image :
version: '3'
services:
localstack:
image: localstack:latest
ports:
- 4566:4566
environment:
- SERVICES=ssm
- DEBUG=1
- DATA_DIR=/tmp/localstack/data
- LOCALSTACK_HOSTNAME=localstack
- LAMBDA_EXECUTOR=local
- LAMBDA_REMOTE_DOCKER=true
volumes:
- './.localstack:/tmp/localstack'
- '/var/run/docker.sock:/var/run/docker.sock'
I created a lambda which is running inside the docker container :
service: service-trigger-action-runner
provider:
name: aws
region: eu-central-1
runtime: nodejs12.x
plugins:
- serverless-offline
custom:
defaultStage: local
functions:
trigger_action_runner:
handler: ../src/trigger_action_runner.handler
environment:
URL_SSM: 'http://localhost:4566'
AWS_ACCESS_KEY_ID: 'ABCDEFGH123456'
AWS_SECRET_ACCESS_KEY: 'key'
Launching the all in a node.js integration test, I'am sending an event in order to trigger my lambda, this part is successfully working
const lambda = new Lambda({
apiVersion: '2031',
endpoint: 'http://localhost:3000'
})
const params = {
FunctionName: 'service-trigger-action-runner-dev-trigger_action_runner',
InvocationType: 'RequestResponse',
Payload: JSON.stringify({ eventTypeAliasName: "fake_event_alias"})
}
await lambda.invoke(params).promise()
Inside my handle script, I'm creating a lambda alias, and my issue is that this call is not staying inside my container, it's not mocked :
exports.handler = async (event) => {
const params = {
Description: 'alias name',
FunctionName: event.eventTypeAlias,
FunctionVersion: '007',
Name: event.eventTypeAliasName
}
console.log(`Creating alias with params=${JSON.stringify(params)}`)
return await lambdaAwsClient.createAlias(params).promise()
This below error is thrown :
UnrecognizedClientException: The security token included in the request is invalid.
How can I do to keep the call createAlias inside the container ?

Using dynamodb in local docker localhost:8000 with serverless-framework serverless-offline application running on localhost:4500

I'm looking to add state to serverless-framework node application running locally. I came across the official DynamoDb docker image, i'd like to use serverless framework with this dynamodb instance running on docker exposed at localhost:8000 without using the sls install dynamodb version.
I have tried using it normally with the nodejs aws-sdk with the endpoint and region configured to local. The new user table is lready created and database is accessible via aws-cli --endpoint localhost:8000 but can't access the dynamodb instance through nodejs aws-sdk
// server.js
const AWS = require('aws-sdk');
AWS.config.update({
region: 'localhost',
endpoint: "http://127.0.0.1:8000"
});
const ddb = new AWS.DynamoDB.DocumentClient();
const params = {
"TableName":tableName,
"IndexName":"email-index",
"KeyConditions":{
"email":{
"ComparisonOperator": "EQ",
"AttributeValueList": [{"S":email}]
}
}
};
ddb.query(params, (err,data) => {
console.log('query', data); // returns query null
}
//handler.js
const server = require('./server');
const http = require('serverless-http');
module.exports.client = http(server);
// serverless.yml
provider:
name: aws
runtime: nodejs10.16.0
region: ca-central-1
profile: default
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:DescribeTable
- dynamodb:Query
- dynamodb:Scan
- dynamodb:CreateTable
- dynamodb:ListTables
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource: "arn:aws:dynamodb:ddblocal:000000000000:table/user"
plugins:
- serverless-offline
functions:
client:
handler: handler.client
events:
- http: GET /
- http: 'GET /{param+}'
- http:
path: /signin
method: post
cors: true
- http:
path: /signup
method: post
cors: true
I expected to get a response from the dynamodb in docker local but the aws-sdk cannot connect to it. The above http events go to express.js which works well.
Try to update if local
let dynamoDb = new AWS.DynamoDB.DocumentClient();
if (process.env.STAGE === 'dev') dynamoDb = new AWS.DynamoDB.DocumentClient({
region: 'localhost',
endpoint: 'http://localhost:8000',
accessKeyId: 'DEFAULT_ACCESS_KEY',
secretAccessKey: 'DEFAULT_SECRET'
});

AWS::Lambda serverless invoke local function not reflecting my new wcode

Not sure what is happening with my code.
It never executes my updated code on my local. If I update my code and run sls invoke local , it still runs the old code. ALso, it does not send out SES EMail.
For some reason, it always executes the code which was already deployed to AWS platform rather than executing my local code. This is confusing.
Below is my serverless.yml:
service: lambda-test
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: ap-southeast-1
iamRoleStatements:
- Effect: Allow
Action:
- lambda:InvokeFunction
- lambda:InvokeAsync
Resource: "*"
functions:
hello:
handler: handler.hello
environment:
events:
- http:
path: /
method: get
trulyYours:
handler: handler.trulyYours
environment:
events:
- http:
path: /trulyYours
method: get
sendRegistrationEmail:
handler: handler.sendRegistrationEmail
environment:
events:
- http:
path: /sendRegistrationEmail
method: get
plugins:
- serverless-offline
I am not sure if I should continue to edit code in AWS web console itself or try
setting up local dev environment. Been tying since last two days, but turning out to be useless to spend time.
'use strict';
var aws = require("aws-sdk");
var nodeMailer = require("nodemailer");
//aws.config.loadFromPath('aws_config.json');
var ses = new aws.SES();
var s3 = new aws.S3();
module.exports.hello = async (event, context) => {
console.log("executing lambda function 'hello' XX...");
// return {
// statusCode: 200,
// body: JSON.stringify({
// message: 'v1.0',
// }),
// };
// Use this code if you don't use the http event with the LAMBDA-PROXY integration
// return { message: 'Go Serverless v1.0! Your function executed successfully!', event };
};
//
exports.trulyYours = async (event, context, callback) => {
console.log('Lambda trulyYours Received event:', JSON.stringify(event, null, 2));
//context.succeed('Hello from' + event.name);
return {
statusCode: 200,
body: JSON.stringify({
message: 'hello from trulyYours' + event.name,
}),
};
}
/*function trulyYours (foo, bar) {
// MyLambdaFunction logic here
}*/
module.exports.sendRegistrationEmail = (event, context) => {
console.log("Executing sendRegistrationEmail...");
var lambda = new aws.Lambda({
region: 'ap-southeast-1' //change to your region
});
var params = {
FunctionName: 'lambda-test-dev-trulyYours', // the lambda function we are going to invoke
InvocationType: 'RequestResponse',
LogType: 'Tail',
Payload: '{ "name" : "Alexa" }'
};
lambda.invoke(params, function (err, data) {
if (err) {
context.fail(err);
} else if (data.Payload) {
context.succeed('Lambda trulyYours ' + data.Payload);
}
});
//
// return {
// statusCode: 200,
// body: JSON.stringify({
// message: 'sent email successfully',
// }),
// };
// Use this code if you don't use the http event with the LAMBDA-PROXY integration
// return { message: 'Go Serverless v1.0! Your function executed successfully!', event };
};
Try to create a serverless-local.yml, for example :
config:
region: eu-west-1
environment:
VAR: local
database:
hostname: localhost
port: 8000
username: root
password: toor
database: db
url: http://localhost:3000
and in you serverless.yml in provider add this line :
stage: ${opt:stage, 'dev'}
then in you terminal try this cli :
sls invoke local -f functionName -s local

How to call an AWS Step Function using the definitions in the serverless-step-functions plugin?

I'm using Serverless Framework to create my Lambda functions and the serverless-step-functions plugin to define my step functions.
Is it possible to call an step function directly from one of the lambda functions using the name defined into the serverless.yml file?
I was trying to solve the same problem and this question and the self answer were very helpful. However, I want to add another answer with more details and a working example to help future readers.
There are two things that you may need:
1- Start a State Machine
2- Invoke one specific function from a State Machine (usually for testing purposes)
The following demo uses both cases.
First, we need to configure the serverless.yml file to declare the State Machine, the Lambda functions and the correct IAM permissions.
service: test-state-machine
provider:
name: aws
runtime: nodejs4.3
region: us-east-1
stage: dev
environment:
AWS_ACCOUNT: 1234567890 # use your own AWS ACCOUNT number here
# define the ARN of the State Machine
STEP_FUNCTION_ARN: "arn:aws:states:${self:provider.region}:${self:provider.environment.AWS_ACCOUNT}:stateMachine:${self:service}-${self:provider.stage}-lambdaStateMachine"
# define the ARN of function step that we want to invoke
FUNCTION_ARN: "arn:aws:lambda:${self:provider.region}:${self:provider.environment.AWS_ACCOUNT}:function:${self:service}-${self:provider.stage}-stateMachineFirstStep"
functions:
# define the Lambda function that will start the State Machine
lambdaStartStateMachine:
handler: handler.lambdaStartStateMachine
role: stateMachine # we'll define later in this file
# define the Lambda function that will execute an arbitrary step
lambdaInvokeSpecificFuncFromStateMachine:
handler: handler.lambdaInvokeSpecificFuncFromStateMachine
role: specificFunction # we'll define later in this file
stateMachineFirstStep:
handler: handler.stateMachineFirstStep
# define the State Machine
stepFunctions:
stateMachines:
lambdaStateMachine:
Comment: "A Hello World example"
StartAt: firstStep
States:
firstStep:
Type: Task
Resource: stateMachineFirstStep
End: true
# define the IAM permissions of our Lambda functions
resources:
Resources:
stateMachine:
Type: AWS::IAM::Role
Properties:
RoleName: stateMachine
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: stateMachine
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: "Allow"
Action:
- "states:StartExecution"
Resource: "${self:provider.environment.STEP_FUNCTION_ARN}"
specificFunction:
Type: AWS::IAM::Role
Properties:
RoleName: specificFunction
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: specificFunction
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource: "${self:provider.environment.FUNCTION_ARN}"
package:
exclude:
- node_modules/**
- .serverless/**
plugins:
- serverless-step-functions
Define the Lambda functions inside the handler.js file.
const AWS = require('aws-sdk');
module.exports.lambdaStartStateMachine = (event, context, callback) => {
const stepfunctions = new AWS.StepFunctions();
const params = {
stateMachineArn: process.env.STEP_FUNCTION_ARN,
input: JSON.stringify({ "msg": "some input" })
};
// start a state machine
stepfunctions.startExecution(params, (err, data) => {
if (err) {
callback(err, null);
return;
}
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'started state machine',
result: data
})
};
callback(null, response);
});
};
module.exports.lambdaInvokeSpecificFuncFromStateMachine = (event, context, callback) => {
const lambda = new AWS.Lambda();
const params = {
FunctionName: process.env.FUNCTION_ARN,
Payload: JSON.stringify({ message: 'invoked specific function' })
};
// invoke a specific function of a state machine
lambda.invoke(params, (err, data) => {
if (err) {
callback(err, null);
return;
}
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'invoke specific function of a state machine',
result: data
})
};
callback(null, response);
});
};
module.exports.stateMachineFirstStep = (event, context, callback) => {
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'state machine first step',
input: event
}),
};
callback(null, response);
};
Deploy executing:
serverless deploy stepf
serverless deploy
Test using:
serverless invoke -f lambdaStartStateMachine
serverless invoke -f lambdaInvokeSpecificFuncFromStateMachine
Solved using serverless environment variables:
environment:
MYFUNCTION_ARN: "arn:aws:states:${self:provider.region}:${self:provider.environment.AWS_ACCOUNT}:stateMachine:${self:service}-${self:provider.stage}-myFunction"
In the function:
var params = {
stateMachineArn: process.env.MYFUNCTION_ARN
};
Here is how you solve it nowadays.
In your serverless.yml, define your stepFunctions and also Outputs:
# define your step functions
stepFunctions:
stateMachines:
myStateMachine:
name: stateMachineSample
events:
- http:
path: my-trigger
method: GET
# make it match your step functions definition
Outputs:
myStateMachine:
Value:
Ref: StateMachineSample
Then you can set your state machine ARN as an environment using ${self:resources.Outputs.fipeStateMachine.Value}.

Resources