Localstack lambda trying to reach AWS outside container - node.js

I have a localstack image :
version: '3'
services:
localstack:
image: localstack:latest
ports:
- 4566:4566
environment:
- SERVICES=ssm
- DEBUG=1
- DATA_DIR=/tmp/localstack/data
- LOCALSTACK_HOSTNAME=localstack
- LAMBDA_EXECUTOR=local
- LAMBDA_REMOTE_DOCKER=true
volumes:
- './.localstack:/tmp/localstack'
- '/var/run/docker.sock:/var/run/docker.sock'
I created a lambda which is running inside the docker container :
service: service-trigger-action-runner
provider:
name: aws
region: eu-central-1
runtime: nodejs12.x
plugins:
- serverless-offline
custom:
defaultStage: local
functions:
trigger_action_runner:
handler: ../src/trigger_action_runner.handler
environment:
URL_SSM: 'http://localhost:4566'
AWS_ACCESS_KEY_ID: 'ABCDEFGH123456'
AWS_SECRET_ACCESS_KEY: 'key'
Launching the all in a node.js integration test, I'am sending an event in order to trigger my lambda, this part is successfully working
const lambda = new Lambda({
apiVersion: '2031',
endpoint: 'http://localhost:3000'
})
const params = {
FunctionName: 'service-trigger-action-runner-dev-trigger_action_runner',
InvocationType: 'RequestResponse',
Payload: JSON.stringify({ eventTypeAliasName: "fake_event_alias"})
}
await lambda.invoke(params).promise()
Inside my handle script, I'm creating a lambda alias, and my issue is that this call is not staying inside my container, it's not mocked :
exports.handler = async (event) => {
const params = {
Description: 'alias name',
FunctionName: event.eventTypeAlias,
FunctionVersion: '007',
Name: event.eventTypeAliasName
}
console.log(`Creating alias with params=${JSON.stringify(params)}`)
return await lambdaAwsClient.createAlias(params).promise()
This below error is thrown :
UnrecognizedClientException: The security token included in the request is invalid.
How can I do to keep the call createAlias inside the container ?

Related

API Gateway refuses to connect to S3,DynamodB using Localstack and serverless

I'm developing Localstack application where i need to connect s3 bucket and dynamodB to the API gateway, when i invoke my lambda using command line using
serverless invoke local -f helloagain
it just works fine, but when i invoke it through postman using api gateway e.g. http://localhost:4566/restapis/82wh1mpf11/local/_user_request_/helloagain it gives internal server error.
Read operations on s3 and ddb are working fine, it is getting a tedious task when it comes to dumping the data into these resources.
Sharing my code snippets below
this is docker-compose file
version: "3.1"
services:
localstack:
image: localstack/localstack:latest
environment:
- AWS_DEFAULT_REGION=us-east-1
- EDGE_PORT=4566
- SERVICES=lambda,s3,cloudformation,sts,apigateway,iam,route53
ports:
- "4566-4597:4566-4597"
volumes:
- "${TEMPDIR:-/tmp/localstack}:/temp/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
this is a serverless.yml
service: localstack-lambda
plugins:
- serverless-localstack
custom:
localstack:
debug: true
stages:
- local
- dev
endpointFile: localstack_endpoints.json
# frameworkVersion: "2"
provider:
name: aws
runtime: nodejs12.x
#functions are pretty important
functions:
hello:
handler: handler.hello
events:
- http:
path: hello
method: any
helloagain:
handler: handler.helloagain
events:
- http:
path: helloagain
method: any
createbucket:
handler: s3handler.createbucket
events:
- http:
path: createbucket
method: any
createDDB:
handler: ddb.createDDB
events:
- http:
path: createDDB
method: any
listTables:
handler: ddb.listTables
events:
- http:
path: listTables
method: any
this is helloagain lambda script
const aws = require('aws-sdk');
const fs = require('fs');
const s3 = new aws.S3({
apiVersion: '2006-03-01',
endpoint: `http://localhost:4566`, // This two lines are
s3ForcePathStyle: true, //only needed for LocalStack.
});
module.exports.helloagain = async (event) => {
const file_buffer = fs.readFileSync("demo2.txt");
const params = {
Bucket: "newbucket",
Key: "demo2.txt",
Body:file_buffer
}
const res = await s3.upload(params).promise();
return {
statusCode: 200,
body: JSON.stringify({
res
})
}
}
i've already checked invoking this lambda through command line that
file demo2.txt already exists
bucket exists
i'm getting this error in the console
localstack.services.awslambda.lambda_executors.InvocationException: Lambda process returned with error. Result: {"errorType":"NetworkingError","errorMessage":"connect ECONNREFUSED 127.0.0.1:4566"}.
and this in the postman

How to deploy NestJS services with graphql, serverless lambda in the Localstack?

Solution
I found the solution, thanks for your time. StackOverflow doesn't let me answer me. >:(
Description
I would like to know what kind of configuration I need to perform to deploy a lambda Localstack of an application in NesJS that has Graphql, not RestApi. This is my configuration.
Is correct this configuration?
How can I get the fake link to access the playground across lambda Localstack?
Implementation
Configuration the nestJS lambda function project
import { ValidationPipe } from '#nestjs/common';
import { NestFactory } from '#nestjs/core';
import { ExpressAdapter } from '#nestjs/platform-express';
import serverlessExpress from '#vendia/serverless-express';
import { APIGatewayProxyHandler, Handler } from 'aws-lambda';
import express from 'express';
import { AppModule } from './app.module';
let cachedServer: Handler;
const bootstrapServer = async (): Promise<Handler> => {
const expressApp = express();
const app = await NestFactory.create(
AppModule,
new ExpressAdapter(expressApp),
);
app.useGlobalPipes(new ValidationPipe());
app.enableCors();
await app.init();
return serverlessExpress({
app: expressApp,
});
};
export const handler: APIGatewayProxyHandler = async (
event,
context,
callback,
) => {
if (!cachedServer) {
cachedServer = await bootstrapServer();
}
return cachedServer(event, context, callback);
};
Serverless.yml Configuration
service: test
provider:
name: aws
runtime: nodejs14.x
stage: ''
# profile: local # Config your AWS Profile
environment: # Service wide environment variables
NODE_ENV: local
GLOBAL_PREFIX: graphql
PORT: 4000
plugins:
# - serverless-plugin-typescript
# - serverless-plugin-optimize
# - serverless-offline
- serverless-localstack
custom:
localstack:
debug: true
stages:
- local
- dev
endpointFile: localstack_endpoints.json
individually: true
# serverless-offline:
# httpPort: 3000
functions:
main:
handler: dis/index.handler
# local:
# handler: dist/main.handler
# events:
# - http:
# path: /
# method: any
# cors: true
Docker Compose Localstack
version: '3.8'
services:
localstack:
image: localstack/localstack:latest
environment:
- AWS_DEFAULT_REGION=us-east-1
- EDGE_PORT=4566
- SERVICES=lambda,s3,cloudformation,sts
ports:
- '4566-4597:4566-4597'
volumes:
- "${TMPDIR:-/tmp}/localstack:/tmp/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
Services
{
"CloudFormation" : "http://localhost:4566",
"CloudWatch" : "http://localhost:4566",
"Lambda" : "http://localhost:4566",
"S3" : "http://localhost:4566"
}
Commands
serverless deploy --stage local
When executing this command I get this error
serverless info --stage local
When executing this command I get this information
serverless invoke local -f main -l
When executing this command
Not sure, but probably you have a typo here at Serverless.yml:
handler: dis/index.handler

AWS SAM Local + DynamoDB Local

I'm currently testing out AWS SAM with DynamoDB Local using Docker.
Here is the steps that I followed (mostly found in the internet)
Create new docker network using docker network create local-dev.
Run DynamoDB Local docker run -d -v "$PWD":/dynamodb_local_db -p 8000:8000 --network local-dev --name dynamodb amazon/dynamodb-local. Until this point, I'm being able to create and list tables using AWS CLI.
Then, I proceed with running AWS SAM sam local start-api --docker-network local-dev. Everything looks okay.
Invoked lambda.js, but it looks like no result for console.log(err)or console.log(data).
I'm not sure where could it be wrong. Please help me. Thank you in advance!
lambda.js
const services = require('./services.js');
const AWS = require('aws-sdk');
let options = {
apiVersion: '2012-08-10',
region: 'ap-southeast-1',
}
if(process.env.AWS_SAM_LOCAL) {
options.endpoint = new AWS.Endpoint('http://localhost:8000')
}
const dynamoDB = new AWS.DynamoDB(options);
exports.getUser = async (event, context) => {
let params = {};
dynamoDB.listTables(params, (err, data) => {
if(err) console.log(err)
else console.log(data)
})
return true;
}
template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless Resources
Parameters:
FunctionsCodeBucket:
Type: String
Description: CodeBucket
FunctionsCodeKey:
Type: String
Description: CodeKey
FunctionsCodeVersion:
Type: String
Description: CodeVersion
NodeEnv:
Type: String
Description: NodeEnv
Globals:
Api:
Cors:
AllowMethods: "'OPTIONS,POST,GET,DELETE,PUT'"
AllowHeaders: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,Api-Key,api-key'"
AllowOrigin: "'*'"
Function:
Timeout: 300
Runtime: nodejs10.x
MemorySize: 128
CodeUri: ./
Resources:
DevResources:
Type: AWS::Serverless::Function
Properties:
Handler: "index.routes"
Environment:
Variables:
NODE_ENV: !Ref NodeEnv
# REGION: !Ref "AWS::Region"
Policies:
- Version: '2012-10-17'
Statement:
- Action:
- dynamodb:*
Effect: Allow
Resource: "*"
Events:
GetUser:
Type: Api
Properties:
Path: /user
Method: get
You lambda function does not wait for dynamoDB.listTables operation. You can fix this issue by using promisified version of dynamoDB.listTables as follows:
exports.getUser = async (event, context) => {
let params = {};
try {
const resp = await dynamoDB.listTables(params).promise();
console.log(resp);
} catch (err) {
console.log(err)
}
};
Another thing that you will likely need to do is to assign a network alias to your dynamodb container (you can do that using --network-alias=<container_name> option) for example, let's set the alias to dynamodb
docker run -d -v "$PWD":/dynamodb_local_db -p 8000:8000 --network local-dev --network-alias=dynamodb --name dynamodb amazon/dynamodb-local
After that you can use this network alias in your lambda function:
if(process.env.AWS_SAM_LOCAL) {
options.endpoint = new AWS.Endpoint('http://dynamodb:8000')
}

Using dynamodb in local docker localhost:8000 with serverless-framework serverless-offline application running on localhost:4500

I'm looking to add state to serverless-framework node application running locally. I came across the official DynamoDb docker image, i'd like to use serverless framework with this dynamodb instance running on docker exposed at localhost:8000 without using the sls install dynamodb version.
I have tried using it normally with the nodejs aws-sdk with the endpoint and region configured to local. The new user table is lready created and database is accessible via aws-cli --endpoint localhost:8000 but can't access the dynamodb instance through nodejs aws-sdk
// server.js
const AWS = require('aws-sdk');
AWS.config.update({
region: 'localhost',
endpoint: "http://127.0.0.1:8000"
});
const ddb = new AWS.DynamoDB.DocumentClient();
const params = {
"TableName":tableName,
"IndexName":"email-index",
"KeyConditions":{
"email":{
"ComparisonOperator": "EQ",
"AttributeValueList": [{"S":email}]
}
}
};
ddb.query(params, (err,data) => {
console.log('query', data); // returns query null
}
//handler.js
const server = require('./server');
const http = require('serverless-http');
module.exports.client = http(server);
// serverless.yml
provider:
name: aws
runtime: nodejs10.16.0
region: ca-central-1
profile: default
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:DescribeTable
- dynamodb:Query
- dynamodb:Scan
- dynamodb:CreateTable
- dynamodb:ListTables
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource: "arn:aws:dynamodb:ddblocal:000000000000:table/user"
plugins:
- serverless-offline
functions:
client:
handler: handler.client
events:
- http: GET /
- http: 'GET /{param+}'
- http:
path: /signin
method: post
cors: true
- http:
path: /signup
method: post
cors: true
I expected to get a response from the dynamodb in docker local but the aws-sdk cannot connect to it. The above http events go to express.js which works well.
Try to update if local
let dynamoDb = new AWS.DynamoDB.DocumentClient();
if (process.env.STAGE === 'dev') dynamoDb = new AWS.DynamoDB.DocumentClient({
region: 'localhost',
endpoint: 'http://localhost:8000',
accessKeyId: 'DEFAULT_ACCESS_KEY',
secretAccessKey: 'DEFAULT_SECRET'
});

How to call an AWS Step Function using the definitions in the serverless-step-functions plugin?

I'm using Serverless Framework to create my Lambda functions and the serverless-step-functions plugin to define my step functions.
Is it possible to call an step function directly from one of the lambda functions using the name defined into the serverless.yml file?
I was trying to solve the same problem and this question and the self answer were very helpful. However, I want to add another answer with more details and a working example to help future readers.
There are two things that you may need:
1- Start a State Machine
2- Invoke one specific function from a State Machine (usually for testing purposes)
The following demo uses both cases.
First, we need to configure the serverless.yml file to declare the State Machine, the Lambda functions and the correct IAM permissions.
service: test-state-machine
provider:
name: aws
runtime: nodejs4.3
region: us-east-1
stage: dev
environment:
AWS_ACCOUNT: 1234567890 # use your own AWS ACCOUNT number here
# define the ARN of the State Machine
STEP_FUNCTION_ARN: "arn:aws:states:${self:provider.region}:${self:provider.environment.AWS_ACCOUNT}:stateMachine:${self:service}-${self:provider.stage}-lambdaStateMachine"
# define the ARN of function step that we want to invoke
FUNCTION_ARN: "arn:aws:lambda:${self:provider.region}:${self:provider.environment.AWS_ACCOUNT}:function:${self:service}-${self:provider.stage}-stateMachineFirstStep"
functions:
# define the Lambda function that will start the State Machine
lambdaStartStateMachine:
handler: handler.lambdaStartStateMachine
role: stateMachine # we'll define later in this file
# define the Lambda function that will execute an arbitrary step
lambdaInvokeSpecificFuncFromStateMachine:
handler: handler.lambdaInvokeSpecificFuncFromStateMachine
role: specificFunction # we'll define later in this file
stateMachineFirstStep:
handler: handler.stateMachineFirstStep
# define the State Machine
stepFunctions:
stateMachines:
lambdaStateMachine:
Comment: "A Hello World example"
StartAt: firstStep
States:
firstStep:
Type: Task
Resource: stateMachineFirstStep
End: true
# define the IAM permissions of our Lambda functions
resources:
Resources:
stateMachine:
Type: AWS::IAM::Role
Properties:
RoleName: stateMachine
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: stateMachine
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: "Allow"
Action:
- "states:StartExecution"
Resource: "${self:provider.environment.STEP_FUNCTION_ARN}"
specificFunction:
Type: AWS::IAM::Role
Properties:
RoleName: specificFunction
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: specificFunction
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: "Allow"
Action:
- "lambda:InvokeFunction"
Resource: "${self:provider.environment.FUNCTION_ARN}"
package:
exclude:
- node_modules/**
- .serverless/**
plugins:
- serverless-step-functions
Define the Lambda functions inside the handler.js file.
const AWS = require('aws-sdk');
module.exports.lambdaStartStateMachine = (event, context, callback) => {
const stepfunctions = new AWS.StepFunctions();
const params = {
stateMachineArn: process.env.STEP_FUNCTION_ARN,
input: JSON.stringify({ "msg": "some input" })
};
// start a state machine
stepfunctions.startExecution(params, (err, data) => {
if (err) {
callback(err, null);
return;
}
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'started state machine',
result: data
})
};
callback(null, response);
});
};
module.exports.lambdaInvokeSpecificFuncFromStateMachine = (event, context, callback) => {
const lambda = new AWS.Lambda();
const params = {
FunctionName: process.env.FUNCTION_ARN,
Payload: JSON.stringify({ message: 'invoked specific function' })
};
// invoke a specific function of a state machine
lambda.invoke(params, (err, data) => {
if (err) {
callback(err, null);
return;
}
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'invoke specific function of a state machine',
result: data
})
};
callback(null, response);
});
};
module.exports.stateMachineFirstStep = (event, context, callback) => {
const response = {
statusCode: 200,
body: JSON.stringify({
message: 'state machine first step',
input: event
}),
};
callback(null, response);
};
Deploy executing:
serverless deploy stepf
serverless deploy
Test using:
serverless invoke -f lambdaStartStateMachine
serverless invoke -f lambdaInvokeSpecificFuncFromStateMachine
Solved using serverless environment variables:
environment:
MYFUNCTION_ARN: "arn:aws:states:${self:provider.region}:${self:provider.environment.AWS_ACCOUNT}:stateMachine:${self:service}-${self:provider.stage}-myFunction"
In the function:
var params = {
stateMachineArn: process.env.MYFUNCTION_ARN
};
Here is how you solve it nowadays.
In your serverless.yml, define your stepFunctions and also Outputs:
# define your step functions
stepFunctions:
stateMachines:
myStateMachine:
name: stateMachineSample
events:
- http:
path: my-trigger
method: GET
# make it match your step functions definition
Outputs:
myStateMachine:
Value:
Ref: StateMachineSample
Then you can set your state machine ARN as an environment using ${self:resources.Outputs.fipeStateMachine.Value}.

Resources