I have the following Terraform code. How can I implement the same in Serverless framework?
resource "aws_cloudwatch_log_group" "abc" {
name = logGroupName
tags = tags
}
resource "aws_cloudwatch_log_stream" "abc" {
depends_on = ["aws_cloudwatch_log_group.abc"]
name = logStreamName
log_group_name = logGroupName
}
My Serverless.yml file looks more like this. Basically I need to create a Log Group and Log Stream with names.
provider:
name: aws
runtime: python3.7
cfnRole: arn:cfnRole
iamRoleStatements:
- Effect: 'Allow'
Action:
- lambda:InvokeFunction
Resource: 'arn....'
functions:
handle:
handler: handler.handle
events:
- schedule:
rate: rate (2 hours)
resources:
Resources:
IamRoleLambdaExecution:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action: sts:AssumeRole
In your Resources you have to add AWS::Logs::LogGroup and AWS::Logs::LogStream.
But tags on AWS::Logs::LogGroup are not supported.
Related
how to reference the iam role (created in the parent stack) in a nested stack
Here are my yml files for the parent and child stack
I used the !Ref and !GetAtt, none of them are working.
Parent Stack:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description:
SAM Template for Nested application resources
Resources:
Layer:
Type: AWS::Lambda::LayerVersion
Properties:
CompatibleRuntimes:
- nodejs16.x
Content:
S3Bucket: bucketName
S3Key: Key
Description: My layer
LayerName: lambdaLayer
LicenseInfo: MIT
SourceIAMRole:
Type: AWS::IAM::Role
Properties:
RoleName: source-lambda-iam-omni-agent-role
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: ''
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: translate-policy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- comprehend:DetectDominantLanguage
- translate:TranslateText
Resource: '*'
- PolicyName: invokeLambda-sns-sqs-sm-policy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- lambda:InvokeAsync
- lambda:InvokeFunction
- sns:Publish
- iam:ListRoles
- iam:GetRole
- secretsmanager:GetSecretValue
- secretsmanager:ListSecrets
- secretsmanager:UpdateSecret
- sqs:*
Resource: '*'
- PolicyName: sts-policy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- sts:AssumeRole
Resource: '*'
- PolicyName: write-cloudwatch-logs-policy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:CreateLogGroup
- logs:PutLogEvents
Resource: '*'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaKinesisExecutionRole
- arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
- arn:aws:iam::aws:policy/AmazonS3FullAccess
- arn:aws:iam::aws:policy/AmazonAPIGatewayInvokeFullAccess
- arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
- arn:aws:iam::aws:policy/AmazonConnect_FullAccess
config:
Type: AWS::Serverless::Application
Properties:
Location: customerconfig.yml
Parameters:
LayerARN: !Ref Layer
IAMRole: !Ref SourceIAMRole
DependsOn:
- Layer
- SourceIAMRole
I did pass the IAM role to the nested stack in the parameter as you can see above in the parent stack. Now I need to pass the ARN created in the parent stack to child (nested) stack
Child stack (Nested)
AWSTemplateFormatVersion: 2010-09-09
Description:
Start from scratch starter project
Transform: AWS::Serverless-2016-10-31
Globals:
Function:
Runtime: nodejs16.x
Timeout: 15
CodeUri: ./
VpcConfig:
SecurityGroupIds:
- sg-005941cb59bd3c74e
SubnetIds:
- subnet-083b8c9bc31cefb69
- subnet-0f77f5b03c7fc1bc7
Layers:
- !Ref LayerARN
MemorySize: 128
Parameters:
LayerARN:
Type: String
IAMRole:
Type: String
Resources:
helloFromLambdaFunction:
Type: AWS::Serverless::Function
Properties:
Role: !GetAtt IAMRole.Arn
Handler: api/getapplicationconfig.handler
Description: A Lambda function that returns a static string.
I did manage to make it work,
While passing the parameters to nested stack, for the IAM role we will use
!GetAtt SourceIAMRole.Arn.
config:
Type: AWS::Serverless::Application
Properties:
Location: customerconfig.yml
Parameters:
LayerARN: !Ref Layer
IAMRole: !GetAtt SourceIAMRole.Arn
I have a use case where I need to call one API every 2 minutes to check the updates and store the result into the database. For the same, I am trying the AWS Schedule lambda function using AWS SAM CLI using Python but my Lambda function is not getting triggered. Below is my code:
app.py
def lambda_schedule(event, context):
print("Lambda Schedule event started Successfully......")
print("Lambda function ARN:", context.invoked_function_arn)
print("CloudWatch log stream name:", context.log_stream_name)
print("CloudWatch log group name:", context.log_group_name)
print("Lambda Request ID:", context.aws_request_id)
print("Lambda Schedule event ended Successfully......")
template.yaml
CronLambdaFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: hello_world/
Handler: app.lambda_schedule
Runtime: python3.8
Events:
PullBalanceScheduleRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- "aws.events"
PullBalanceScheduleRule:
Type: AWS::Events::Rule
Properties:
Description: "PullBalanceScheduleRule"
ScheduleExpression: "rate(2 minutes)"
State: "ENABLED"
Targets:
-
Arn: !GetAtt CronLambdaFunction.Arn
Id: "CronLambdaFunction"
PermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref "PullBalanceScheduleRule"
Action: "lambda:InvokeFunction"
Principal: "events.amazonaws.com"
SourceArn:
-
Arn: !GetAtt PullBalanceScheduleRule.Arn
Id: "PullBalanceScheduleRule"
can anyone tell me, what is wrong in my code OR what is missing in my code?
I got my mistakes. Mistakes were in the Permission sections. I am posting here correct yaml configurations so it could help to someone who are new to AWS SAM CLI.
app.py
def lambda_schedule(event, context):
print("Lambda Schedule event started Successfully......")
print("Lambda function ARN:", context.invoked_function_arn)
print("CloudWatch log stream name:", context.log_stream_name)
print("CloudWatch log group name:", context.log_group_name)
print("Lambda Request ID:", context.aws_request_id)
print("Lambda Schedule event ended Successfully......")
template.yaml
CronLambdaFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: hello_world/
Handler: app.lambda_schedule
Runtime: python3.8
Events:
PullBalanceScheduleRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- "aws.events"
PullBalanceScheduleRule:
Type: AWS::Events::Rule
Properties:
Description: "PullBalanceScheduleRule"
ScheduleExpression: "rate(2 minutes)"
State: "ENABLED"
Targets:
-
Arn: !GetAtt CronLambdaFunction.Arn
Id: "CronLambdaFunction"
PermissionForEventsToInvokeLambda:
Type: AWS::Lambda::Permission
Properties:
FunctionName: !Ref "CronLambdaFunction"
Action: "lambda:InvokeFunction"
Principal: "events.amazonaws.com"
SourceArn:
Fn::GetAtt:
- "PullBalanceScheduleRule"
- "Arn"
I am converting the following Cloudformation code to Terraform. I have converted the most of the following code except the first few lines upto TriggerConfigurations (Type, DependsOn, Properties):
Cloudformation:
CodeDeployDeploymentGroup:
Type: 'AWS::CodeDeploy::DeploymentGroup'
DependsOn:
- CodeDeployApplication
- CodeDeployRole
Properties:
ApplicationName: !Ref CodeDeployApplication
ServiceRoleArn: !GetAtt
- CodeDeployRole
- Arn
TriggerConfigurations:
- TriggerEvents:
- DeploymentStart
- DeploymentSuccess
- DeploymentFailure
- DeploymentStop
- DeploymentRollback
- DeploymentReady
TriggerName: SlackTarget
TriggerTargetArn: !ImportValue
'Fn::Sub': '${EnvName}CDNotificationTopicARN'
DeploymentGroupName: !Join
- '-'
- - !Ref AppName
- CodeDeploymentGroup
DeploymentConfigName: CodeDeployDefault.LambdaAllAtOnce
DeploymentStyle:
DeploymentOption: !Ref DGDeploymentOption
DeploymentType: !Ref DGDeploymentType
Terraform:
resource "aws_codedeploy_deployment_group" "CodeDeployDeploymentGroup" {
app_name = aws_codedeploy_app.example.name
deployment_config_name = "CodeDeployDefault.LambdaAllAtOnce"
deployment_group_name = "${var.AppName}-CodeDeploymentGroup"
service_role_arn = aws_iam_role.example.arn
deployment_style {
deployment_option = "${var.DGDeploymentOption}"
deployment_type = "${var.DGDeploymentType}"
}
trigger_configuration {
trigger_events = ["DeploymentStart","DeploymentSuccess","DeploymentFailure","DeploymentStop","DeploymentRollback","DeploymentReady"]
trigger_name = "SlackTarget"
trigger_target_arn = ["${EnvName}CDNotificationTopicARN"]
}
}
Please let me know how to convert these lines of code to terraform.
I coded the properties as below:
resource "aws_codedeploy_deployment_group" "CodeDeployDeploymentGroup" {
app_name = ${var.CodeDeployApplication}
deployment_config_name = "CodeDeployDefault.LambdaAllAtOnce"
deployment_group_name = "${var.AppName}-CodeDeploymentGroup"
service_role_arn = aws_iam_role.CodeDeployRole.arn
Let me know if this is right.
I have a serverless resource for SNS topic in the resources section in the serverless.yml something like this,
resources:
Resources:
SNSTopic:
Type: AWS::SNS::Topic
Properties:
DisplayName: SNS Topic
TopicName: ${self:service}-${self:provider.stage}-Topic
When I am trying to bind this SNS topic to my lambda event as given below, lambda is not triggering by the SNS event. When I check AWS console for that lambda function the SNS event bound with wrong ARN value.
Function:
handler: src/sample/file.lambdaHandler
role: s3FullAccessRole
events: SNSTopic
Properties:
Policies:
- AWSLambdaExecute
- Statement:
- Effect: Allow
Action:
- 'lambda:InvokeFunction'
I have tried with changing event with all the different ways mentioned in here, https://serverless.com/framework/docs/providers/aws/events/sns/. The only way I found is to hard code the SNS Topic ARN value in the lambda event, which is not ideal for my situation.
Any help is really appreciated.
You could actually create a variable in custom with the arn of the sns topic
custom:
region: ${opt:region, self:provider.region}
snsTopic: ${self:service}-${self:provider.stage}-Topic
snsTopicArn: { "Fn::Join" : ["", ["arn:aws:sns:${self:custom.region}:", { "Ref" : "AWS::AccountId" }, ":${self:custom.snsTopic}" ] ] }
then just use the arn on the places you need.
Or you can use the plugin https://github.com/silvermine/serverless-plugin-external-sns-events to basic reference the topic name.
If you have only 1 serverless.yml and don't want to have a separate cloudformation file I would use the first option
EDIT:
To use the arn follow the instructions on serverless https://serverless.com/framework/docs/providers/aws/events/sns#using-a-pre-existing-topic
functions:
dispatcher:
handler: <handler>
events:
- sns:
arn: ${self:custom.snsTopicArn}
since you have the sns topic on the same serverless.yml, you can even ignore the snsTopicArn variable and build it like one of the suggestions using !Ref which should be a better option for you:
functions:
dispatcher:
handler: <handler>
events:
- sns:
arn: !Ref SNSTopic
topicName: ${self:custom.snsTopic}
full example:
service: testsns
provider:
name: aws
runtime: nodejs12.x
region: eu-west-1
functions:
hello:
handler: handler.hello
events:
- sns:
arn: !Ref SuperTopic
topicName: MyCustomTopic
Properties:
Policies:
- AWSLambdaExecute
- Statement:
- Effect: Allow
Action:
- 'lambda:InvokeFunction'
resources:
Resources:
SuperTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: MyCustomTopic
Finally got it!
I end up removing my SNS TOPIC declaration from resource section of serverless.yml added under iamRoleStatements, something like this,
iamRoleStatements:
- Effect: Allow
Action:
- SNS:Publish
Resource: { "Fn::Join" : ["", ["arn:aws:sns:${self:provider.region}:", { "Ref" : "AWS::AccountId" }, ":${self:custom.mySnsTopic}" ] ] }
And added variables in the custom section
custom:
mySnsTopic: "${self:service}-${self:provider.stage}-sns-consume"
mySnsTopicArn: { "Fn::Join" : ["", ["arn:aws:sns:${self:provider.region}:", { "Ref" : "AWS::AccountId" }, ":${self:custom.mySnsTopic}" ] ] }
then mapped this to the lambda function events
Function:
handler: src/sample/file.lambdaHandler
role: s3FullAccessRole
events: ${self:custom.mySnsTopicArn}
Properties:
Policies:
- AWSLambdaExecute
For reference link
I've got a lambda function like this:
import boto3
import os
import json
step_functions = boto3.client('stepfunctions')
workers_topic = boto3.resource('sns').Topic(os.environ.get("WORKERS_TOPIC_ARN"))
def test_push_to_workers_sns(event, context):
activity_response = \
step_functions.get_activity_task(
activityArn=os.environ.get("ACKNOWLEDGE_ACTIVITY_ARN"),
workerName='test_push_to_workers_sns'
)
task_token, input_ = activity_response['taskToken'], activity_response['input']
print(f"Task token is {task_token}")
print(f"Input is {input}")
if not task_token:
print("No activity found")
return
workers_topic.publish(Message="blah blah")
When I set off an execution of the step function I have and it reaches the activity, I've repeatedly checked that running aws stepfunctions get-activity-task --activity-arn <ACKNOWLEDGE_ACTIVITY_ARN> on my terminal returns a taskToken and input that are both correct. However this lambda function seems to always time out regardless of whether or not the activity is running (I've got my timeout value set to 1 min 15 secs on the lambda function, and the activity state on the step function's timeout at 1 hour)
I checked this case using following CloudFormation template and it works:
AWSTemplateFormatVersion: "2010-09-09"
Description: Stack creating AWS Step Functions state machine and lambda function calling GetActivityTask.
Resources:
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Handler: "index.handler"
Role: !GetAtt LambdaExecutionRole.Arn
Code:
ZipFile: |
import boto3
import os
import json
step_functions = boto3.client('stepfunctions')
workers_topic = boto3.resource('sns').Topic(os.environ.get("WORKERS_TOPIC_ARN"))
def handler(event, context):
activity_response = step_functions.get_activity_task(
activityArn=os.environ.get("ACKNOWLEDGE_ACTIVITY_ARN"),
workerName='test_push_to_workers_sns'
)
if 'taskToken' not in activity_response:
return
task_token, task_input = activity_response['taskToken'], activity_response['input']
print(f"Task token is {task_token}")
print(f"Input is {input}")
workers_topic.publish(Message="blah blah")
step_functions.send_task_success(
taskToken=task_token,
output=task_input
)
Runtime: "python3.6"
Timeout: 25
Environment:
Variables:
WORKERS_TOPIC_ARN: !Ref WorkersTopic
ACKNOWLEDGE_ACTIVITY_ARN: !Ref AcknowledgeActivity
StateMachine:
Type: AWS::StepFunctions::StateMachine
Properties:
RoleArn: !GetAtt StatesExecutionRole.Arn
DefinitionString: !Sub
- >
{
"Comment": "State Machine for GetActivityTask testing purposes.",
"StartAt": "FirstState",
"States": {
"FirstState": {
"Type": "Task",
"Resource": "${ACKNOWLEDGE_ACTIVITY_ARN}",
"End": true
}
}
}
- ACKNOWLEDGE_ACTIVITY_ARN: !Ref AcknowledgeActivity
AcknowledgeActivity:
Type: AWS::StepFunctions::Activity
Properties:
Name: !Sub ${AWS::AccountId}-AcknowledgeActivity
WorkersTopic:
Type: AWS::SNS::Topic
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: StepFunctionsAccess
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- states:GetActivityTask
- states:SendTaskFailure
- states:SendTaskSuccess
Resource: arn:aws:states:*:*:*
- PolicyName: SNSAccess
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- SNS:Publish
Resource: arn:aws:sns:*:*:*
StatesExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- !Sub states.${AWS::Region}.amazonaws.com
Action: sts:AssumeRole
Path: "/"
Policies: []
I created execution manually from Step Functions console and executed Lambda manually from Lambda console.
Keep in mind that after you take taskToken using GetActivityTask the execution related with it is waiting for response (SendTaskSuccess or SendTaskFailure) until it reaches timeout (default is very long). So if you have taken token before then that execution isn't available for GetAcitivtyTask. You can lookup status of execution in Step Functions console looking at events for specific execution.
You should call SendTaskSuccess or SendTaskFailure from your code after getting token from GetActivityTask (otherwise execution will be hanging until it reaches timeout or is stopped).
Aside from original question: GetActivityTask is not designed to be called from Lambda. You can pass Lambda Function as resource to state machine (instead of activity) and it will be called when execution reaches specified state (event in handler will contain execution state). Activities should be used only for long-running jobs on dedicated machines (EC2, ECS). I should also point that there are service limits for GetActivityTask calls (25 RPS with bucket of size 1000) and Lambda-based states are limited basically only by transition count limit (400 per second with bucket of size 800). You can read more about step function limits here: https://docs.aws.amazon.com/step-functions/latest/dg/limits.html