LambdaFunction - Value of property Variables must be an object with String (or simple type) properties - node.js

I am using serverless to deploy my Lambda based application. It was deploying just fine, and then it stopped for some reason. I paired down the entire package to the serverless.yml below and one function in the handler - but I keep getting this error:
Serverless Error ---------------------------------------
An error occurred: TestLambdaFunction - Value of property Variables must be an object with String (or simple type) properties.
Stack Trace --------------------------------------------
Here is the serverless.yml
# serverless.yml
service: some-api
provider:
name: aws
runtime: nodejs6.10
stage: prod
region: us-east-1
iamRoleStatements:
$ref: ./user-policy.json
environment:
config:
region: us-east-1
plugins:
- serverless-local-dev-server
- serverless-dynamodb-local
- serverless-step-functions
package:
exclude:
- .gitignore
- package.json
- README.md
- .git
- ./**.test.js
functions:
test:
handler: handler.test
events:
- http: GET test
resources:
Outputs:
NewOutput:
Description: Description for the output
Value: Some output value
Test Lambda Function in Package
#handler.test
module.exports.test = (event, context, callback) => {
callback(null, {
statusCode: 200,
body: JSON.stringify({
message: 'sadfasd',
input: event
})
})
}

Turns out, this issue does not have any relationship to the Lambda function. Here is the issue that caused the error.
This does NOT work:
environment:
config:
region: us-east-1
This DOES work:
environment:
region: us-east-1
Simply put, I don't think you can have more than one level in your yaml environment variables.
Even if you try sls print as a sanity check, this issue will not pop up. Only in sls deploy.
You have been warned, and hopefully saved!

Other thing that might cause this kind of error is using invalid yaml syntax.
It's easy to get confused about this.
Valid syntax for environment variables
environment:
key: value
Invalid syntax for environment variables
environment:
- key: value
Notice little dash in the code below?
In yaml syntax - means array and therefore, code below is interpreted as array, not object.
So that's why error tells "Value of property Variables must be an object with String (or simple type) properties."
This can be easily fixed by removing - in front of all keys.

Related

Serverless AWS Lambda Edge: MalformedHandlerName

I'm following the below example to setup a Lambda Edge function:
https://www.serverless.com/blog/lambda-at-edge-support-added
The code: ./handler.js
// SAME CODE STRUCTURE AS ORIGINAL, JUST ADDING A HEADER TO TEST
'use strict';
exports.handler = (event, context, callback) => {
const response = event.Records[0].cf.response;
const headers = response.headers;
headers['test-lambda-edge'] = [{ key: 'test-lambda-edge', value: 'working' }]
callback(null, response);
};
The Config: ./serverless.yml
# SAME CONFIG AS ORIGINAL, JUST CHANGING eventType and origin
service: mylambda
provider:
name: aws
runtime: nodejs12.x
lambdaHashingVersion: 20201221
functions:
cfLambda:
handler: functions/handler.cloudfront
events:
- cloudFront:
eventType: origin-response
origin: https://example.org
The function deploys properly:
$ sls deploy
Serverless: Packaging service...
Serverless: Excluding development dependencies...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading mylambda.zip file to S3 (554 B)...
Serverless: Validating template...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
................
Serverless: Stack update finished...
Service Information
service: mylambda
stage: dev
region: us-east-1
stack: mylambda-dev
resources: 8
api keys:
None
endpoints:
CloudFront - ---------------.cloudfront.net
functions:
cfLambda: mylambda-dev-cfLambda
layers:
None
I am using the Amazon CloudFront Modify Response Headers test template for testing.
Attempt 1) with the same handlers naming as article
I get the below error, which I can understand as the article seems wrong, the cloudfront function isn't defined:
"errorMessage": "functions/handler.cloudfront is undefined or not
exported"
Attempt 2) Trying to fix serverless.yml
I replaced handler: functions/handler.cloudfront with handler: functions/handler to match the code and I get the below error:
{"errorType":"Runtime.MalformedHandlerName","errorMessage":"Bad
handler","stack":["Runtime.MalformedHandlerName: Bad handler"," at
_splitHandlerString
If I look at the official doc, the handler naming is the same (ie exports.handler = ):
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-how-it-works-tutorial.html
'use strict';
exports.handler = (event, context, callback) => {
Q: How should I name my handler in the code/config?
If the catalog structure that you're using looks like this:
├── handler.js
└── serverless.yml
and the name of the exported handler is handler and lives in handler.js file then the correct config will look like this:
service: mylambda
provider:
name: aws
runtime: nodejs12.x
lambdaHashingVersion: 20201221
functions:
cfLambda:
handler: handler.handler
events:
- cloudFront:
eventType: origin-response
origin: https://example.org
First part is the path to the module and after . you reference specific function exported in that module. For handler: functions/handler.cloudfront to work, you'd have to export cloudfront function in ./functions/handler.js file.

Serverless.yml: Reference existing environment variable in another

I have a serverless.yml which looks like this
service: my-service
provider:
name: aws
runtime: python3.7
versionFunctions: false
environment:
ACCOUNT_ID: "${file(./serverless.env.yml):${self:provider.stage}.account_id}"
ANOTHER_VARIABLE: "some text ${ACCOUNT_ID} some other text"
Here, I want to reference the existing environment ACCOUNT_ID in ANOTHER_VARIABLE. The ${ACCOUNT_ID} doesn't work. I also tried to look at the serverless documentation but I'm not able to find anything related to that.
You can simply use ${self:provider.environment.ACCOUNT_ID}.
service: my-service
provider:
name: aws
runtime: python3.7
versionFunctions: false
environment:
ACCOUNT_ID: "${file(./serverless.env.yml):${self:provider.stage}.account_id}"
ANOTHER_VARIABLE: "some text ${self:provider.environment.ACCOUNT_ID} some other text"

Environment Variables with Serverless and AWS Lambda

I am learning serverless framework and I'm making a simple login system.
Here is my serverless.yml file
service: lms-auth
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: ap-south-1
environment:
MONGODB_URI: $(file(../env.yml):MONOGDB_URI)
JWT_SECRET: $(file(../env.yml):JWT_SECRET)
functions:
register:
handler: handler.register
events:
- http:
path: auth/register/
method: post
cors: true
login:
handler: handler.login
events:
- http:
path: auth/login/
method: post
cors: true
plugins:
- serverless-offline
As you can see, I have two environment variables and both of them are referencing to a different file in the same root folder.
Here is that env.yml file
MONOGDB_URI: <MY_MONGO_DB_URI>
JWT_SECRET: LmS_JWt_secREt_auth_PasSWoRds
When I do sls deploy, I see that both the variables are logging as null. The environment variables aren't sent to lambda.
How can I fix this?
Also, currently I'm using this method and adding the env.yml to .gitignore and saving the values. Is there any other efficient way of hiding sensitive data?
I would do something like this to help you out with the syntax
service: lms-auth
custom: ${file(env.yml)}
provider:
name: aws
runtime: nodejs8.10
stage: dev
region: ap-south-1
environment:
MONGODB_URI: ${self:custom.mongodb_uri}
JWT_SECRET: ${self:custom.jwt_secret}
functions:
register:
handler: handler.register
events:
- http:
path: auth/register/
method: post
cors: true
login:
handler: handler.login
events:
- http:
path: auth/login/
method: post
cors: true
plugins:
- serverless-offline
Then in your env.yml you can do
mongodb_uri: MY_MONGO_DB_URI
jwt_secret: LmS_JWt_secREt_auth_PasSWoRds
Enviroment variables
1. Add useDotenv:true to your .yml file 2.Add your variables like this -> ${env:VARIABLE_NAME}3.Create a file called .env.dev and write the variables (You can add .env.prod but you have to change the stage inside your .yml file ) Example :
service: lms-auth
useDotenv: true
provider:
name: aws
runtime: nodejs12.x
stage: dev
region: us-east-1
environment:
MONGODB_URI: ${env:MONOGDB_URI}
JWT_SECRET: ${env:JWT_SECRET}
functions:
register:
handler: handler.register
events:
- http:
path: auth/register/
method: post
cors: true
login:
handler: handler.login
events:
- http:
path: auth/login/
method: post
cors: true
plugins:
- serverless-offline
.env.dev
MONOGDB_URI = The URI Value
JWT_SECRET = The JWT Scret Value
I ended up solving it. I had set up my Dynamo DB in AWS us-west region. reinitialized in US-East-2, and reset the region under 'provider' within the .yml file.

Serverless variable from external file nested property

I have serverless yml and a config file
config file
app: {
port: 3000,
db: {
connectionString: 'xxxxx'
},
lambdaDeploy:{
stage : "DEV",
region : "es-west-1"
}
Trying to use these variables in yml like below
yml
provider:
name: aws
runtime: nodejs6.10
stage: ${file(./appconfiguration.json).app.stage}
region: ${file(./appconfiguration.json).app.region}
But its reading and taking default
Please advise.
Thanks
The syntax used here is not correct.
stage: ${file(./appconfiguration.json).app.stage}
Use colon instead:
stage: ${file(./appconfiguration.json):app.stage}
More details here: https://www.serverless.com/framework/docs/providers/aws/guide/variables/#reference-variables-in-other-files

Python Step Functions API: get_activity_task seems to always timeout

I've got a lambda function like this:
import boto3
import os
import json
step_functions = boto3.client('stepfunctions')
workers_topic = boto3.resource('sns').Topic(os.environ.get("WORKERS_TOPIC_ARN"))
def test_push_to_workers_sns(event, context):
activity_response = \
step_functions.get_activity_task(
activityArn=os.environ.get("ACKNOWLEDGE_ACTIVITY_ARN"),
workerName='test_push_to_workers_sns'
)
task_token, input_ = activity_response['taskToken'], activity_response['input']
print(f"Task token is {task_token}")
print(f"Input is {input}")
if not task_token:
print("No activity found")
return
workers_topic.publish(Message="blah blah")
When I set off an execution of the step function I have and it reaches the activity, I've repeatedly checked that running aws stepfunctions get-activity-task --activity-arn <ACKNOWLEDGE_ACTIVITY_ARN> on my terminal returns a taskToken and input that are both correct. However this lambda function seems to always time out regardless of whether or not the activity is running (I've got my timeout value set to 1 min 15 secs on the lambda function, and the activity state on the step function's timeout at 1 hour)
I checked this case using following CloudFormation template and it works:
AWSTemplateFormatVersion: "2010-09-09"
Description: Stack creating AWS Step Functions state machine and lambda function calling GetActivityTask.
Resources:
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Handler: "index.handler"
Role: !GetAtt LambdaExecutionRole.Arn
Code:
ZipFile: |
import boto3
import os
import json
step_functions = boto3.client('stepfunctions')
workers_topic = boto3.resource('sns').Topic(os.environ.get("WORKERS_TOPIC_ARN"))
def handler(event, context):
activity_response = step_functions.get_activity_task(
activityArn=os.environ.get("ACKNOWLEDGE_ACTIVITY_ARN"),
workerName='test_push_to_workers_sns'
)
if 'taskToken' not in activity_response:
return
task_token, task_input = activity_response['taskToken'], activity_response['input']
print(f"Task token is {task_token}")
print(f"Input is {input}")
workers_topic.publish(Message="blah blah")
step_functions.send_task_success(
taskToken=task_token,
output=task_input
)
Runtime: "python3.6"
Timeout: 25
Environment:
Variables:
WORKERS_TOPIC_ARN: !Ref WorkersTopic
ACKNOWLEDGE_ACTIVITY_ARN: !Ref AcknowledgeActivity
StateMachine:
Type: AWS::StepFunctions::StateMachine
Properties:
RoleArn: !GetAtt StatesExecutionRole.Arn
DefinitionString: !Sub
- >
{
"Comment": "State Machine for GetActivityTask testing purposes.",
"StartAt": "FirstState",
"States": {
"FirstState": {
"Type": "Task",
"Resource": "${ACKNOWLEDGE_ACTIVITY_ARN}",
"End": true
}
}
}
- ACKNOWLEDGE_ACTIVITY_ARN: !Ref AcknowledgeActivity
AcknowledgeActivity:
Type: AWS::StepFunctions::Activity
Properties:
Name: !Sub ${AWS::AccountId}-AcknowledgeActivity
WorkersTopic:
Type: AWS::SNS::Topic
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: StepFunctionsAccess
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- states:GetActivityTask
- states:SendTaskFailure
- states:SendTaskSuccess
Resource: arn:aws:states:*:*:*
- PolicyName: SNSAccess
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- SNS:Publish
Resource: arn:aws:sns:*:*:*
StatesExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- !Sub states.${AWS::Region}.amazonaws.com
Action: sts:AssumeRole
Path: "/"
Policies: []
I created execution manually from Step Functions console and executed Lambda manually from Lambda console.
Keep in mind that after you take taskToken using GetActivityTask the execution related with it is waiting for response (SendTaskSuccess or SendTaskFailure) until it reaches timeout (default is very long). So if you have taken token before then that execution isn't available for GetAcitivtyTask. You can lookup status of execution in Step Functions console looking at events for specific execution.
You should call SendTaskSuccess or SendTaskFailure from your code after getting token from GetActivityTask (otherwise execution will be hanging until it reaches timeout or is stopped).
Aside from original question: GetActivityTask is not designed to be called from Lambda. You can pass Lambda Function as resource to state machine (instead of activity) and it will be called when execution reaches specified state (event in handler will contain execution state). Activities should be used only for long-running jobs on dedicated machines (EC2, ECS). I should also point that there are service limits for GetActivityTask calls (25 RPS with bucket of size 1000) and Lambda-based states are limited basically only by transition count limit (400 per second with bucket of size 800). You can read more about step function limits here: https://docs.aws.amazon.com/step-functions/latest/dg/limits.html

Resources