I am writing an async lambda on AWS using Node.js (14.x). This lambda is being called by an API Gateway (REST API, POST method, CORS enabled).
To allow the API Gateway to call an async lambda, I added the following HTTP Header to its Integration Request: Name = X-Amz-Invocation-Type and Mapped from (value) = 'Event' as specified here.
When I run Test, I get the following output:
Execution log for request BLAH
Fri Mar 26 07:30:34 UTC 2021 : Starting execution for request: BLAH
Fri Mar 26 07:30:34 UTC 2021 : HTTP Method: POST, Resource Path: BLAH
Fri Mar 26 07:30:34 UTC 2021 : Method request path: {}
Fri Mar 26 07:30:34 UTC 2021 : Method request query string: {}
Fri Mar 26 07:30:34 UTC 2021 : Method request headers: {}
Fri Mar 26 07:30:34 UTC 2021 : Method request body before transformations: {
"id": "BLAH",
"recaptcha": "BLAH"
}
Fri Mar 26 07:30:34 UTC 2021 : Request validation succeeded for content type application/json
Fri Mar 26 07:30:34 UTC 2021 : Endpoint request URI: https://lambda.BLAH.amazonaws.com/2015-03-31/functions/arn:aws:lambda:BLAH:function:BLAH/invocations
Fri Mar 26 07:30:34 UTC 2021 : Endpoint request headers: {X-Amz-Date=BLAH, x-amzn-apigateway-api-id=BLAH, Accept=application/json, User-Agent=AmazonAPIGateway_BLAH, Host=lambda.BLAH.amazonaws.com, X-Amz-Content-Sha256=BLAH, X-Amzn-Trace-Id=Root=BLAH, x-amzn-lambda-integration-tag=BLAH, Authorization=*****BLAH*****, X-Amz-Source-Arn=arn:aws:execute-api:BLAH/test-invoke-stage/POST/BLAH, X-Amz-Invocation-Type=Event, X-Amz-Security-Token=BLAH [TRUNCATED]
Fri Mar 26 07:30:34 UTC 2021 : Endpoint request body after transformations: {
"id": "BLAH",
"recaptcha": "BLAH"
}
Fri Mar 26 07:30:34 UTC 2021 : Sending request to https://lambda.BLAH.amazonaws.com/2015-03-31/functions/arn:aws:lambda:BLAH:function:BLAH/invocations
Fri Mar 26 07:30:34 UTC 2021 : Received response. Status: 202, Integration latency: 34 ms
Fri Mar 26 07:30:34 UTC 2021 : Endpoint response headers: {Date=Fri, 26 Mar 2021 07:30:34 GMT, Content-Length=0, Connection=keep-alive, x-amzn-RequestId=BLAH, x-amzn-Remapped-Content-Length=0, X-Amzn-Trace-Id=root=BLAH;sampled=0}
Fri Mar 26 07:30:34 UTC 2021 : Endpoint response body before transformations:
Fri Mar 26 07:30:34 UTC 2021 : Method response body after transformations:
Fri Mar 26 07:30:34 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=BLAH;Sampled=0, Access-Control-Allow-Origin=*, Content-Type=application/json}
Fri Mar 26 07:30:34 UTC 2021 : Successfully completed execution
Note that the response gives Status: 202 and that there is no response body. No matter what code I put in my lambda, this seems to be the case. Here are a couple basic examples that I've tried:
// Load the AWS SDK for Node.js and set the region.
var AWS = require('aws-sdk');
AWS.config.update({region: 'BLAH'});
// Main function, responds to AWS API Gateway.
exports.handler = async function (event, context) {
var response = {
httpStatus : 200,
message : "Success"
};
return JSON.stringify(response);
};
and
// Load the AWS SDK for Node.js and set the region.
var AWS = require('aws-sdk');
AWS.config.update({region: 'BLAH'});
// Main function, responds to AWS API Gateway.
exports.handler = async function (event, context) {
var promise = new Promise(function(resolve, reject) {
var response = {
httpStatus : 200,
message : "Success"
};
resolve(JSON.stringify(response));
});
return promise;
};
I made sure to deploy the API and Lambda. How can I get a non-202 status (i.e. get the API Gateway to wait on the async lambda)?
This is working as intended. The second sentence in AWS docs on "Asynchronous invocation" says:
When you invoke a function asynchronously, you don't wait for a response from the function code.
If you want a response with a result, asynchronous invocation will not work.
Related
Im testing functionality between AWS Lambda and API gateway. My lambda function is using a Node and I have defined the parameters that i want to be pulling from S3. This is my lambda:
console.log('Loading function');
const aws = require('aws-sdk');
const s3 = new aws.S3({ apiVersion: '2006-03-01' });
exports.handler = async (event, context) => {
const bucket = event.Records[0].s3.bucket.name;
const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
const params = {
Bucket: 'testing-api-lambda-s3-jfr',
Key: 'sample-file-testing.csv',
};
exports.sendRes = (body, status = 200) => {
const response = {
statusCode: status,
headers: {
"Content-Type": "application/json"
},
body: JSON.stringify(body)
};
return response;
};
The test event on lambda returns OK, so i think lambda its fine:
Test Event Name
test
Response
"text/csv"
Function Logs
START RequestId: 07d8ea41-976b-4e49-9fb7-ccfafca15dcf Version: $LATEST
2022-02-03T19:38:46.871Z 07d8ea41-976b-4e49-9fb7-ccfafca15dcf INFO CONTENT TYPE: text/csv
END RequestId: 07d8ea41-976b-4e49-9fb7-ccfafca15dcf
REPORT RequestId: 07d8ea41-976b-4e49-9fb7-ccfafca15dcf Duration: 499.10 ms Billed Duration: 500 ms Memory Size: 128 MB Max Memory Used: 80 MB
Request ID
07d8ea41-976b-4e49-9fb7-ccfafca15dcf
On my API gateway, i have it configured as GET with no query strings or path parameters defined:
And on the test the error is:
{
"message": "Internal server error"
}
Full API gateway execution logs:
Execution log for request 2866b8b2-bd57-49f7-aad9-1a9279612364
Thu Feb 03 19:44:42 UTC 2022 : Starting execution for request: 2866b8b2-bd57-49f7-aad9-1a9279612364
Thu Feb 03 19:44:42 UTC 2022 : HTTP Method: GET, Resource Path: /
Thu Feb 03 19:44:42 UTC 2022 : Method request path: {}
Thu Feb 03 19:44:42 UTC 2022 : Method request query string: {}
Thu Feb 03 19:44:42 UTC 2022 : Method request headers: {}
Thu Feb 03 19:44:42 UTC 2022 : Method request body before transformations:
Thu Feb 03 19:44:42 UTC 2022 : Endpoint request URI: https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:143578772304:function:getObjectNode/invocations
Thu Feb 03 19:44:42 UTC 2022 : Endpoint request headers: {X-Amz-Date=20220203T194442Z, x-amzn-apigateway-api-id=vzcwgufch1, Accept=application/json, User-Agent=AmazonAPIGateway_vzcwgufch1, Host=lambda.us-east-1.amazonaws.com, X-Amz-Content-Sha256=ec78e1a3cab036558869aced97bcd18b2213fcab99c7a38ddc15d6bf509a5f5b, X-Amzn-Trace-Id=Root=1-61fc30aa-f6d4b97e6327eed29d022b45, x-amzn-lambda-integration-tag=2866b8b2-bd57-49f7-aad9-1a9279612364, Authorization=*********************************************************************************************************************************************************************************************************************************************************************************************************************************************a0c1a3, X-Amz-Source-Arn=arn:aws:execute-api:us-east-1:143578772304:vzcwgufch1/test-invoke-stage/GET/, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEEQaCXVzLWVhc3QtMSJIMEYCIQDE4SYIymD/u99+1dzLRSk8U+DbUxCC9ygNy6aYr1SjhQIhAN9P2Oh/UyQl1L0mLSNTGhNRFGz2G5irLwGI7gHbs/4eKvoDCHwQABoMM [TRUNCATED]
Thu Feb 03 19:44:42 UTC 2022 : Endpoint request body after transformations: {"resource":"/","path":"/","httpMethod":"GET","headers":null,"multiValueHeaders":null,"queryStringParameters":null,"multiValueQueryStringParameters":null,"pathParameters":null,"stageVariables":null,"requestContext":{"resourceId":"dw7k4w6igh","resourcePath":"/","httpMethod":"GET","extendedRequestId":"M-yKsEC5IAMFegA=","requestTime":"03/Feb/2022:19:44:42 +0000","path":"/","accountId":"143578772304","protocol":"HTTP/1.1","stage":"test-invoke-stage","domainPrefix":"testPrefix","requestTimeEpoch":1643917482625,"requestId":"2866b8b2-bd57-49f7-aad9-1a9279612364","identity":{"cognitoIdentityPoolId":null,"cognitoIdentityId":null,"apiKey":"test-invoke-api-key","principalOrgId":null,"cognitoAuthenticationType":null,"userArn":"arn:aws:iam::143578772304:user/fernando","apiKeyId":"test-invoke-api-key-id","userAgent":"aws-internal/3 aws-sdk-java/1.12.138 Linux/5.4.156-94.273.amzn2int.x86_64 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Oracle_Corporation cfg/retry-mod [TRUNCATED]
Thu Feb 03 19:44:42 UTC 2022 : Sending request to https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:143578772304:function:getObjectNode/invocations
Thu Feb 03 19:44:42 UTC 2022 : Received response. Status: 200, Integration latency: 24 ms
Thu Feb 03 19:44:42 UTC 2022 : Endpoint response headers: {Date=Thu, 03 Feb 2022 19:44:42 GMT, Content-Type=application/json, Content-Length=261, Connection=keep-alive, x-amzn-RequestId=62424064-5199-48b7-ad4e-f8976d86e514, X-Amz-Function-Error=Unhandled, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-61fc30aa-f6d4b97e6327eed29d022b45;sampled=0}
Thu Feb 03 19:44:42 UTC 2022 : Endpoint response body before transformations: {"errorType":"TypeError","errorMessage":"Cannot read property '0' of undefined","trace":["TypeError: Cannot read property '0' of undefined"," at Runtime.exports.handler (/var/task/index.js:12:33)"," at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"]}
Thu Feb 03 19:44:42 UTC 2022 : Lambda execution failed with status 200 due to customer function error: Cannot read property '0' of undefined. Lambda request id: 62424064-5199-48b7-ad4e-f8976d86e514
Thu Feb 03 19:44:42 UTC 2022 : Method completed with status: 502
Any suggestions? thank you
The error message from the execution logs says that the event object does not have an array Records with some bucket information. The reason for this is that API Gateway request events do not have this array. API Gateway does not anything about your buckets as long as you do not pass the bucket information in a REST request body.
Moreover, you are expecting a bucket name and key from the API gateway request using these lines:
const bucket = event.Records[0].s3.bucket.name;
const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
But you never really use these variables anywhere, essentially hardcoding your own bucket name and key with these lines:
const params = {
Bucket: 'testing-api-lambda-s3-jfr',
Key: 'sample-file-testing.csv',
};
Just simply remove these two lines from your code, if you don't need them.
const bucket = event.Records[0].s3.bucket.name;
const key = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
I feel like you are confusing how the event object should look like when your Lambda is triggered by an S3 event, compared to what it would look like if the Lambda would be triggered by API Gateway proxy integration. If you want to know the how the event object is structured when the API Gateway invokes it, you may want to check out this page from the AWS docs.
As per the execution logs your API Gateway is not able to transform response received from lambda.
You need to provide your API a model defining response structure such that it can transform responses.
This falls under Mapping templates.
https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-gateway-response-using-the-console.html
I'm creating a simple Lambda function to serve a POST in AWS Gateway:
import json
import boto3
from datetime import datetime
def lambda_handler(event,context):
wf = event['WF']
if wf == 'start1':
body = 'Suceeded'
return {
"isBase64Encoded": False,
'statusCode': 200,
'body': json.dumps(body),
'headers': {
'Content-Type': 'application/json'
}
}
else:
body = 'Failed'
return {
"isBase64Encoded": False,
'statusCode': 400,
'body': json.dumps(body),
'headers': {
'Content-Type': 'application/json'
}
}
If I send a POST with this body is successful:
{
"WF": "start1",
"Context": "OK"
}
Returns:
Test Event Name
test
Response
{
"isBase64Encoded": false,
"statusCode": 200,
"body": "\"Suceeded\"",
"headers": {
"Content-Type": "application/json"
}
}
Function Logs
START RequestId: 91663a9d-d8b6-4aad-b8b6-eb2f15ff0354 Version: $LATEST
END RequestId: 91663a9d-d8b6-4aad-b8b6-eb2f15ff0354
REPORT RequestId: 91663a9d-d8b6-4aad-b8b6-eb2f15ff0354 Duration: 1.74 ms Billed Duration: 2 ms Memory Size: 128 MB Max Memory Used: 66 MB Init Duration: 300.73 ms
Request ID
91663a9d-d8b6-4aad-b8b6-eb2f15ff0354
But if I run a test in the API Gateway service in AWS I get:
Fri Jun 04 13:45:22 UTC 2021 : Sending request to https://XXXXXXX
Fri Jun 04 13:45:22 UTC 2021 : Received response. Status: 200, Integration latency: 15 ms
Fri Jun 04 13:45:22 UTC 2021 : Endpoint response headers: {Date=Fri, 04 Jun 2021 13:45:22 GMT, Content-Type=application/json, Content-Length=159, Connection=keep-alive, x-amzn-RequestId=d479fdac-6737-42fa-96a3-9e991056b48d, X-Amz-Function-Error=Unhandled, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-60ba2e72-ff62f051febbd21d6cc9d114;sampled=0}
Fri Jun 04 13:45:22 UTC 2021 : Endpoint response body before transformations: {"errorMessage": "'WF'", "errorType": "KeyError", "stackTrace": [" File \"/var/task/lambda_function.py\", line 7, in lambda_handler\n wf = event['WF']\n"]}
Fri Jun 04 13:45:22 UTC 2021 : Lambda execution failed with status 200 due to customer function error: 'WF'. Lambda request id: d479fdac-6737-42fa-96a3-9e991056b48d
Fri Jun 04 13:45:22 UTC 2021 : Method completed with status: 502
Why line 7 is wrong?
Is there another way to parse the body of the POST?
The solution was found by disabling Lambda Proxy Integration in the POST - Integration Request.
It looks like your API Gateway invocation is not populating the event parameter of your Lambda function. When you create a test event your lambda function's page, you get to create your own test event. API Gateway will not use this test event, it will instead send an empty event (probably null).
I.e.
import json
import boto3
from datetime import datetime
def lambda_handler(event,context): # Event is null here
wf = event['WF'] # Attempting to read the attribute 'WF' of null results in an error
In my instance, I had received this error when invoking my Lambda function via API gateway, it was also completely fine when invoking directly the lambda function expecting a simple JSON body.
my expected payload:
{"Email": "foo3#foo.com"}
It turns out Lambda Proxy Integration in the POST in the configuration API gateway changes the payload structure before passing into Lambda, and I was getting the error missing Key attribute Email. I noticed that line Endpoint request body after transformations: in cloudwatch. I learned that it was changing the payload.
How I debug this:
Invoke Lambda using test events ensuring the both the error scenario and success scenarios are working.
Ensure there are no permission issues via API gateway by making sure that your API gateway can invoke the Lambda and have appropriate IAM roles/policies
Ensure your lambda has the API gateway as the trigger
I tried removing all the security token first via API gateway to verify I am able to invoke the API
I tried both HTTP methods POST And GET, seeing the same error (Email was missing) was the key to figuring this out as I was already passing the payload in the request body in the POST method.
I am working on an event stream system. And my solution is: api-gatway->firehose data stream->data stream delivery->s3.
1.Send a json request(action is PUT) to api-getway
2.Integrate the request with Kinesis service
3.Valid data in the template mapping of the Integration Request
4.Delivery the data to data stream by firehose data delivery
5.Consume the stream and save into S3
It works with below data
1.Request data
{
"Data" : "this is a test"
}
2.Template mapping code of the Integration Request with firehose
{
"StreamName" : "Stream",
"Data" : "$util.base64Encode($input.json('$.Data'))",
"PartitionKey" : "1"
}
Execution log for request 4ad5f3a1-da7d-436f-bd3f-879ba045c622
Fri Jan 15 15:14:19 UTC 2021 : Starting execution for request: 4ad5f3a1-da7d-436f-bd3f-879ba045c622
Fri Jan 15 15:14:19 UTC 2021 : HTTP Method: PUT, Resource Path: /record
Fri Jan 15 15:14:19 UTC 2021 : Method request path: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request query string: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request headers: {}
Fri Jan 15 15:14:19 UTC 2021 : Method request body before transformations: {
"property":"1",
"name":"name"
}
Fri Jan 15 15:14:19 UTC 2021 : Endpoint request URI: https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:14:19 UTC 2021 : Endpoint request headers: {Authorization=***************************************************************************************************************************************************************************************************************************************************************************************9bb292, X-Amz-Date=20210115T151419Z, x-amzn-apigateway-api-id=450acvde3l, Accept=application/json, User-Agent=AmazonAPIGateway_450acvde3l, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEEAaCXVzLWVhc3QtMSJHMEUCIQC7W693nrtuUGrgUvOpn+PUKTRYTGAzRYVSWUPKoAc4eAIgGp/uH4yV/vynaMQUBNEBTwP79m0P9nVfCmw9CMWjY0Mq5QIIGBACGgw1MzUxMTE0MzMzNDAiDKy2z/4lnGJJJAWaOyrCAnANknvMUIbvf9diTZ4qYsdJ3LjETy9hE0f9pUZTBh7aKRjn2y4hbZc/oXGpsmHcgsgJAlIGD51gD0O5LR+N2EKjQeeo5GijkYvZvdbmxVRAZR27EhFzujCk3okePxAzVb3SeCQDWrBnEWn2VwqX4Y75QWwOgL8ZfcuxNKF/wbks0ukPFBo0p68KQ24PIXOYGMs9IAxHYa195+5mUMSi5Av62QKPpBbUAgZ2IJt9EKvRKyLYCs5XzS4lPCgsIbwouVB38f6UQKdQ9XsmD0cX04ODkdj2/0GHne6ufar7nANd3o08PmALd9mXK4z4gt/OxkZUe2AVLXvNQ4IodzW0WOUe1nFvq6YxpveDVfXKDRSK [TRUNCATED]
Fri Jan 15 15:14:19 UTC 2021 : Endpoint request body after transformations: {
"StreamName" : "EventStream",
"Data" : "IiI=",
"PartitionKey" : "1"
}
Fri Jan 15 15:14:19 UTC 2021 : Sending request to https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:14:19 UTC 2021 : Received response. Status: 200, Integration latency: 21 ms
Fri Jan 15 15:14:19 UTC 2021 : Endpoint response headers: {x-amzn-RequestId=c028bf47-0059-8b93-9853-0cccfda9a977, x-amz-id-2=L15V+8AS8gN3HxJwiz0qVqi/UJn2xSWueRMnXdhqgjFw6TeuaRlYX62DZK9pa+O1PcKopTP55aHRLdhX0cQwovKefVpiuRtv, Date=Fri, 15 Jan 2021 15:14:19 GMT, Content-Type=application/x-amz-json-1.1, Content-Length=110}
Fri Jan 15 15:14:19 UTC 2021 : Endpoint response body before transformations: {"SequenceNumber":"49614572211779959518343530489315214421429290892684951554","ShardId":"shardId-000000000000"}
Fri Jan 15 15:14:19 UTC 2021 : Method response body after transformations: {"SequenceNumber":"49614572211779959518343530489315214421429290892684951554","ShardId":"shardId-000000000000"}
Fri Jan 15 15:14:19 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=1-6001b14b-bbccae126175224317a10a4e, Content-Type=application/json}
Fri Jan 15 15:14:19 UTC 2021 : Successfully completed execution
Fri Jan 15 15:14:19 UTC 2021 : Method completed with status: 200
But does not work with below:
1.Request data
{
"Data" : {
"property":"1",
"name":"name"
}
}
2.Template mapping code of the Integration Request with firehose
{
"StreamName" : "Stream",
"Data" : {
"property" : $input.json('$.property'),
"name" : $input.json('$.name')
},
"PartitionKey" : "1"
}
Execution log for request 05b707cc-d95c-40bf-8b75-375048697414
Fri Jan 15 15:39:34 UTC 2021 : Starting execution for request: 05b707cc-d95c-40bf-8b75-375048697414
Fri Jan 15 15:39:34 UTC 2021 : HTTP Method: PUT, Resource Path: /record
Fri Jan 15 15:39:34 UTC 2021 : Method request path: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request query string: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request headers: {}
Fri Jan 15 15:39:34 UTC 2021 : Method request body before transformations: {
"property":"1",
"name":"name"
}
Fri Jan 15 15:39:34 UTC 2021 : Endpoint request URI: https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:39:34 UTC 2021 : Endpoint request headers: {Authorization=***************************************************************************************************************************************************************************************************************************************************************************************a43c70, X-Amz-Date=20210115T153934Z, x-amzn-apigateway-api-id=450acvde3l, Accept=application/json, User-Agent=AmazonAPIGateway_450acvde3l, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEEAaCXVzLWVhc3QtMSJFMEMCHw0tTLi/gwtxUjwAQSl/CIY8aie2nmayl+Qsm6/i520CID9iXCFafaQTh4YqE1/tzvKgMO5IlYgFJrcNbAQB2Nc+KuUCCBkQAhoMNTM1MTExNDMzMzQwIgxeE22ch5hdFy3nB+UqwgJyKnQpnLuEY3zpcbRdEO5jks7yfx2+o1xfIz9Kga0S1PojPfzxh5aD/PthhP8D0jutv96ZVe8p52TwfSnv/z3YeDCFzsnw/U9kGFzGVt1pY2JMB4sg1vU7li8pFP/qiUQ3QA8cXbp4nWeE3kQGlPG4pjH0MsOvowTxM8G6yKosvCdD8fVyCJxWIjFnn1+dK9GGV/MnZlnaVqc57z0n0nrHgLjxBzDcDKJ5/xrgcqcYmUETFj8NyDJ9ESzCp0PhKJV9tGF4LgxbAgffe2Yw/3qpQyB6JNqJrZEczADp3gL0rjIBXhbnx5Yizs9MBMtoB9L22mAwEeqJRx4lK12wOqQZ5+0homLCugauYVoy3juNv/zW [TRUNCATED]
Fri Jan 15 15:39:34 UTC 2021 : Endpoint request body after transformations: {
"StreamName" : "EventStream",
"Data" : {
"property" : "1",
"name" : "name"
},
"PartitionKey" : "1"
}
Fri Jan 15 15:39:34 UTC 2021 : Sending request to https://kinesis.us-east-1.amazonaws.com/?Action=PutRecord
Fri Jan 15 15:39:34 UTC 2021 : Received response. Status: 400, Integration latency: 2 ms
Fri Jan 15 15:39:34 UTC 2021 : Endpoint response headers: {x-amzn-RequestId=cc937c57-744b-fca2-94e8-c5214b4386dd, x-amz-id-2=fqg8MLIbHOfeeFXp6wpii3l4yl32mI/5RyTwYQyzw9/OqpdLNqCBvBbTp8x7Q4YWAroefbfHb5IUEYeD68SQqh2bq87nL4vp, connection=close, Date=Fri, 15 Jan 2021 15:39:34 GMT, Content-Type=application/x-amz-json-1.1, Content-Length=99}
Fri Jan 15 15:39:34 UTC 2021 : Endpoint response body before transformations: {"__type":"SerializationException","Message":"Start of structure or map found where not expected."}
Fri Jan 15 15:39:34 UTC 2021 : Method response body after transformations: {"__type":"SerializationException","Message":"Start of structure or map found where not expected."}
Fri Jan 15 15:39:34 UTC 2021 : Method response headers: {X-Amzn-Trace-Id=Root=1-6001b736-328415f2db20383946bffd9e, Content-Type=application/json}
Fri Jan 15 15:39:34 UTC 2021 : Successfully completed execution
Fri Jan 15 15:39:34 UTC 2021 : Method completed with status: 200
reference: https://www.youtube.com/watch?v=0UxiV5sUlcA
My bad and I forgot to encode in template mapping.
The code should be:
{
"StreamName" : "Stream",
"Data" : "$util.base64Encode(
{
"property" : $input.json('$.property'),
"name" : $input.json('$.name')
}
),
"PartitionKey" : "1"
}
So I have function in Lambda. Function is connected to the api gateway and it should add EC2 instance. When im reaching the endpoint by api gateway method test, it returns status 200 but no instance has been added. Maybe the instance params are wrong? Basically the function is modified version of documentation example.
var AWS = require('aws-sdk');
AWS.config.update({region: 'us-east-2'});
exports.handler = function index(event, context, callback) {
// Load the AWS SDK for Node.js
// Load credentials and set region from JSON file
// Create EC2 service object
var ec2 = new AWS.EC2({apiVersion: '2016-11-15'});
// AMI is amzn-ami-2011.09.1.x86_64-ebs
var instanceParams = {
InstanceType: 't2.micro',
KeyName: 'firstkeypair',
ImageId: 'ami-0bbe28eb2173f6167'
};
// Create a promise on an EC2 service object
var instancePromise = new AWS.EC2({apiVersion: '2016-11-15'}).runInstances(instanceParams).promise();
// Handle promise's fulfilled/rejected states
instancePromise.then(
function(data) {
console.log(data);
var instanceId = data.Instances[0].InstanceId;
console.log("Created instance", instanceId);
// Add tags to the instance
tagParams = {Resources: [instanceId], Tags: [
{
Key: 'Name',
Value: 'SDK Sample'
}
]};
// Create a promise on an EC2 service object
var tagPromise = new AWS.EC2({apiVersion: '2016-11-15'}).createTags(tagParams).promise();
// Handle promise's fulfilled/rejected states
tagPromise.then(
function(data) {
console.log("Instance tagged");
}).catch(
function(err) {
console.error(err, err.stack);
});
}).catch(
function(err) {
console.error(err, err.stack);
});
}
AWS test logs:
Execution log for request a83bae6e-2fbf-4d88-ad70-a683a83bdc41
Sun Aug 16 16:56:00 UTC 2020 : Starting execution for request: a83bae6e-2fbf-4d88-ad70-a683a83bdc41
Sun Aug 16 16:56:00 UTC 2020 : HTTP Method: GET, Resource Path: /
Sun Aug 16 16:56:00 UTC 2020 : Method request path: {}
Sun Aug 16 16:56:00 UTC 2020 : Method request query string: {}
Sun Aug 16 16:56:00 UTC 2020 : Method request headers: {}
Sun Aug 16 16:56:00 UTC 2020 : Method request body before transformations:
Sun Aug 16 16:56:00 UTC 2020 : Endpoint request URI: https://lambda.us-east-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-2:081348884123:function:hello/invocations
Sun Aug 16 16:56:00 UTC 2020 : Endpoint request headers: {x-amzn-lambda-integration-tag=a83bae6e-2fbf-4d88-ad70-a683a83bdc41, Authorization=**************************************************************************************************************************************************************************************************************************************************************************************59de14, X-Amz-Date=20200816T165600Z, x-amzn-apigateway-api-id=o2hkrbm1o4, X-Amz-Source-Arn=arn:aws:execute-api:us-east-2:081348884123:o2hkrbm1o4/test-invoke-stage/GET/, Accept=application/json, User-Agent=AmazonAPIGateway_o2hkrbm1o4, X-Amz-Security-Token=IQoJb3JpZ2luX2VjEAAaCXVzLWVhc3QtMiJIMEYCIQCPi2S8PtDGsVK3w101D8B05/BCFGyUCzHeX8CT6tC7pAIhAJZCgpbZN94qCVdAgrQGlIIE+ABsO9MDkzh6Lf3WGq3IKr0DCNn//////////wEQARoMNzE4NzcwNDUzMTk1IgxILUqxpu50pB1cJmcqkQP/g+OuOqP7/zXYq8IAzTMolDThuprxjuzwDbmtAmS3adcmmHO25YxBQrId1XiR7ZEU7mq52k4A0nIFhBPkz2dZZIfr8MiLVCDx5tLok8j3lPZJOW+I3n7BVglTMtfQDpPYRSUcIQhOfsSnEEc+FKPzHyrzGsLeazIUHItf5L3xY4QO9tyDWnTXfcM2pp [TRUNCATED]
Sun Aug 16 16:56:00 UTC 2020 : Endpoint request body after transformations:
Sun Aug 16 16:56:00 UTC 2020 : Sending request to https://lambda.us-east-2.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-2:081348884123:function:hello/invocations
Sun Aug 16 16:56:02 UTC 2020 : Received response. Status: 200, Integration latency: 1952 ms
Sun Aug 16 16:56:02 UTC 2020 : Endpoint response headers: {Date=Sun, 16 Aug 2020 16:56:02 GMT, Content-Type=application/json, Content-Length=4, Connection=keep-alive, x-amzn-RequestId=f84212ea-38f8-40cc-b5c6-c12885e78392, x-amzn-Remapped-Content-Length=0, X-Amz-Executed-Version=$LATEST, X-Amzn-Trace-Id=root=1-5f396520-4d9dfcb6b965192c5fea0df6;sampled=0}
Sun Aug 16 16:56:02 UTC 2020 : Endpoint response body before transformations: null
Sun Aug 16 16:56:02 UTC 2020 : Method response body after transformations: null
Sun Aug 16 16:56:02 UTC 2020 : Method response headers: {X-Amzn-Trace-Id=Root=1-5f396520-4d9dfcb6b965192c5fea0df6;Sampled=0, Content-Type=application/json}
Sun Aug 16 16:56:02 UTC 2020 : Successfully completed execution
Sun Aug 16 16:56:02 UTC 2020 : Method completed with status: 200
Policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iam:*",
"organizations:DescribeAccount",
"organizations:DescribeOrganization",
"organizations:DescribeOrganizationalUnit",
"organizations:DescribePolicy",
"organizations:ListChildren",
"organizations:ListParents",
"organizations:ListPoliciesForTarget",
"organizations:ListRoots",
"organizations:ListPolicies",
"organizations:ListTargetsForPolicy"
],
"Resource": "*"
}
]
}
Edit:
Solved by adding EC2 Full Access permission to Lambda Function.
There were 2 issues as discovered through the comments.
The first was that the RunInstances task was not including the MinCount and MaxCount properties which led to no instances being launched.
Once this was fixed the next issue was a permissions issue due to the lack of permissions to run ec2:RunInstance or e2:CreateTags.
It is worth stating the best practice with permissions is to scope down to the minimal permissions that you require to successfully run.
My purpose is to expire a token after 1hr (3600 secs). While trying with nodejs speakeasy the token is getting invalidated much before that. The below logs are for "1, 10 and 60 minutes" and that also getting invalidated muche before the 1 minute. Max of the time I am getting inconsistent results.
Partial code snippet
let secret = speakeasy.generateSecret({
length: 10
});
let seconds= 3600; //1Hr
let token = speakeasy.totp({
secret: secret.base32,
step: seconds
});
let otp = {
"secret": secret.base32.toString(),
"token": token
};
function checkOTP(otp) {
let verified = speakeasy.totp.verify({
secret: otp.secret,
token: otp.token,
step: seconds
});
return verified;
}
Am I doing something wrong? Few console logs from a sample script:
For 1 minute - Invalidated before 18secs
[ Fri Dec 08 2017 09:16:18 GMT-0800 (Pacific Standard Time) ](true) 9:16:59 AM
[ Fri Dec 08 2017 09:16:18 GMT-0800 (Pacific Standard Time) ](false) 9:17:00 AM
For 10Mins - Invalidated before 7minutes
[ Fri Dec 08 2017 09:18:28 GMT-0800 (Pacific Standard Time) ](true) 9:19:59 AM
[ Fri Dec 08 2017 09:18:28 GMT-0800 (Pacific Standard Time) ](true) 9:19:59 AM
[ Fri Dec 08 2017 09:18:28 GMT-0800 (Pacific Standard Time) ](true) 9:19:59 AM
[ Fri Dec 08 2017 09:18:28 GMT-0800 (Pacific Standard Time) ](true) 9:19:59 AM
[ Fri Dec 08 2017 09:18:28 GMT-0800 (Pacific Standard Time) ](false) 9:20:00 AM
For 1Hr - Invalidated before 7minutes
[ Fri Dec 08 2017 11:07:01 GMT-0800 (Pacific Standard Time) ](true) 11:56:41 AM
[ Fri Dec 08 2017 11:07:01 GMT-0800 (Pacific Standard Time) ](true) 11:56:43 AM
[ Fri Dec 08 2017 11:07:01 GMT-0800 (Pacific Standard Time) ](false) 12:00:37 PM
What is the appropriate way to validate within the above window?
From the readme of speakeasy it looks like your token parameters are wrong:
var token = speakeasy.totp({
secret: secret.base32,
encoding: 'base32',
time: 1453667708 // You have this as 'step' not 'time'
});