AWS Lambda: 503 Service Unavailable - node.js

Question

I am developing a REST API in NodeJs using AWS Lambda and AWS API Gateway. I am using AWS SAM Template as well.
Below is my NodeJS Code. There I am only trying to access a sample API over the internet and make a POST call.
const mysql = require('mysql2');
const errorCodes = require('source/error-codes');
const PropertiesReader = require('properties-reader');
const fetch = require('node-fetch');
const prop = PropertiesReader('properties.properties');
const con = mysql.createConnection({
host: prop.get('server.host'),
user: prop.get("server.username"),
password: prop.get("server.password"),
port: prop.get("server.port"),
database: prop.get("server.dbname")
});
exports.testApi = async (event, context) => {
context.callbackWaitsForEmptyEventLoop = false;
con.config.namedPlaceholders = true;
if (event.body == null && event.body == undefined) {
var response = errorCodes.missing_parameters;
return response;
}
let body = JSON.parse(event.body)
if (body.key == null ) {
console.log("fire 1");
var response = errorCodes.not_null_parameters;
return response;
}
try {
let key = body.key;
console.log("body", body);
var notificationMessage = {
"key": key
};
const notificationResponse = await fetch("https://reqbin.com/sample/post/json", {
method: 'post',
body: JSON.stringify(notificationMessage),
headers: {
'Content-Type': 'application/json'
}
});
const data = await notificationResponse.json();
//Return the response
var response = {
"statusCode": 200,
"headers": {
"Content-Type": "application/json"
},
"body": JSON.stringify({
"message": data
}),
"isBase64Encoded": false
};
return response;
} catch (error) {
console.log(error);
//Return the response
var response = {
"statusCode": 500,
"headers": {
"Content-Type": "application/json"
},
"body": JSON.stringify({
"error": error
}),
"isBase64Encoded": false
};
return response;
}
};
Below is my template.yaml file. It contains nested access to another template.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
xxx-restapi
Sample SAM Template for xxx-restapi
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 5
VpcConfig:
SecurityGroupIds:
- sg-xxxxx
SubnetIds:
- subnet-xxxx
- subnet-aaaa
- subnet-bbbb
- subnet-cccc
- subnet-dddd
- subnet-eeee
Parameters:
FirebaseProjectId:
Type: String
#Dont create this domain in the AWS Console manually, so it will fail here
DomainName:
Type: String
Default: api2.someapp.com
Resources:
# Authentication required HTTP API
AuthGatewayHttpApi:
Type: AWS::Serverless::HttpApi
Properties:
Domain:
DomainName: !Ref DomainName
EndpointConfiguration: REGIONAL
CertificateArn: arn:aws:acm:us-east-1:xxxxxx:certificate/bac44716-xxxx-431b-xxxx-xxxx
Route53:
HostedZoneId: xxxxxxx
IpV6: true
Auth:
Authorizers:
FirebaseAuthorizer:
IdentitySource: $request.header.Authorization
JwtConfiguration:
audience:
- !Ref FirebaseProjectId
issuer: !Sub https://securetoken.google.com/${FirebaseProjectId}
DefaultAuthorizer: FirebaseAuthorizer
# Authentication NOT required HTTP API
NoAuthGatewayHttpApi:
Type: AWS::Serverless::HttpApi
Properties:
Domain:
BasePath: noauth
DomainName: !Ref DomainName
CertificateArn: arn:aws:acm:us-east-1:xxxx:certificate/xxx-420d-xxx-xxx-xxxx
Route53:
HostedZoneId: xxxxxx
# Lambda settings
LambdaRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Policies:
- PolicyName: root
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- ec2:DescribeNetworkInterfaces
- ec2:CreateNetworkInterface
- ec2:DeleteNetworkInterface
- ec2:DescribeInstances
- ec2:AttachNetworkInterface
Resource: '*'
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
SharedValueOutput:
Value: !Ref FirebaseProjectId
Description: You can refer to any resource from the template.
# HelloWorldApi:
# Description: "API Gateway endpoint URL for Prod stage for functions"
# Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
Below is the nested template file
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
xxx-restapi
Sample SAM Template for xxx-restapi
Globals:
Function:
Timeout: 30
VpcConfig:
SecurityGroupIds:
- sg-xxxx
SubnetIds:
- subnet-xxx
- subnet-aaa
- subnet-ccc
- subnet-sss
- subnet-fff
- subnet-eee
Parameters:
FirebaseProjectId:
Type: String
DomainName:
Type: String
Resources:
NoAuthGatewayHttpApi2:
Type: AWS::Serverless::HttpApi
Properties:
StageName: Prod
MyApiMapping:
DependsOn: NoAuthGatewayHttpApi2
Type: AWS::ApiGatewayV2::ApiMapping
Properties:
ApiMappingKey: no-auth
DomainName: api2.xxx.com
ApiId: !Ref NoAuthGatewayHttpApi2
Stage: !Ref NoAuthGatewayHttpApi2.Stage
TestPostFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: xxx-restapi/
Handler: source/fcm/test-api.testApi
Runtime: nodejs14.x
Events:
GetRtcTokenAPIEvent:
Type: HttpApi
Properties:
Path: /fcm/test-api
Method: post
ApiId: !Ref NoAuthGatewayHttpApi2
When I executed the lambda function in local environment it works fine. But if i executed the same after uploading to AWS, it gives me the following error with 503 status code.
{
"message": "Service Unavailable"
}
Below is my Cloud Watch log.
START RequestId: e84242ea-xxx-4aa0-xxx-xxx Version: $LATEST
2022-06-12T05:55:52.914Z e84242ea-xxx-4aa0-xxx-xxx INFO body { key: 'value' }
END RequestId: e84242ea-xxx-xxx-dd37f2a005c0
REPORT RequestId: e84242ea-xxx-4aa0xxx993e-xxx Duration: 30032.56 ms Billed Duration: 30000 ms Memory Size: 128 MB Max Memory Used: 72 MB Init Duration: 315.65 ms
2022-06-12T05:56:22.936Z xxx-b568-xxx-993e-xxx Task timed out after 30.03 seconds
The timed out error you see above is just a mask to the real problem. I even tried this with time limit set to 2 minutes, same result. In my local environment with Docker this works in seconds.
After googling I figured out I "may" have not enabled internet connection from my Lambda functions to outside. Even in my API, all Lambda functions that require no Lambda to outside internet connection is working fine.
How can I fix this error?

Answer 1

I would first try to debug the networking aspect. Can anything in those subnets connect to the database? Are the security groups correct?
You might find it easier to create an EC2 instance and debug the networking from there. It certainly seems like a connection timing out.

Answer 2

I got the 503 error as well,
turns out to be the Api Gateway timeout problem.