I used s3 to trigger my lambda function every time I uploaded a file in s3 bucket, I've tested it and it works but I noticed that S3 is triggering my lambda function twice with unique request id, even if I removed the entire codes and just add console.info('something') it's still trigger twice so clearly it's not lambda making retries if there's error. I also set retry to 0.
1st request id : awsRequestId: '9f73e49f-6cc7-454e-a89f-7d88122a7166'
2nd request id : awsRequestId: '1c8572d5-61ee-4b0b-93d9-4f8a3dcd28bf'
here's my code
const AWS = require('aws-sdk');
const s3 = new AWS.S3({
region: process.env.region,
accessKeyId: process.env.accessKeyId,
secretAccessKey: process.env.secretAccessKey
});
const axios = require('axios');
const csvtojson = require('csvtojson');
const _ = require('lodash');
exports.handler = async (event, context) => {
const { normalizeAndHashData } = require('./hash');
const params = {Bucket: 'myBucket', 'myKey'}
const data = await s3.getObject(params).promise();
const eventsJson = await csvtojson().fromString(data.Body.toString());
const result1 = await axios.post(`https://myurl`, eventJson);
if (result1) {
const sampleDataToUpload = {id: 1, name: 'test'}
const result2 = await axios.post(`https://myurl2`, sampleDataToUpload);
}
return context.succeed()
};
I have an AWS Lambda function which is triggered by an API Gateway event. The API Gateway is configured to use X-Ray.
As the Lambda tracing configuration defaults to PassTrough it is also shown in X-Ray (service map, etc.).
The invoked Lambda uses the node.js aws-sdk to invoke another lambda. If I understand correctly the Tracing ID has to be passed on to the next Invocation in order to show this Lambda also in X-Ray. In the API of the SDK I found no option for this.
const result = await lambda
.invoke(lambdaParamsCreateUser)
.promise()
How can I achieve this? How can I trace also the invocation of the original request?
With the tips of #Balu Vyamajala I changed the AWS-SDK import to the following:
import AWS from "aws-sdk";
import AwsXRay from "aws-xray-sdk-core";
const aws = AwsXRay.captureAWS(AWS);
export default aws;
I use it when I invoice my second function like this:
import AWS from "aws";
const Lambda = AWS.Lambda;
// ...
const lambda = new Lambda({ region: "eu-central-1" });
const lambdaPromise = lambda
.invoke({
FunctionName: AUTH_CREATE_USER_FUNC,
InvocationType: "RequestResponse",
Qualifier: AUTH_CREATE_USER_FUNC_VERSION,
Payload: JSON.stringify({
eMail: eMail,
device: device,
customerId: customerId,
}),
LogType: "Tail",
})
.promise()
But in X-Ray there is no invocation chain :-(
https://imgur.com/wDMlNzb
Do I make a mistake?
if we enable X-Ray for both Lambda functions , trace-id is automatically passed and will be same for both Lambdas.
In the code, we can enable X-Ray simply by wrapping it around aws-sdk
JavaScript:
const AWSXRay = require("aws-xray-sdk-core");
const AWS = AWSXRay.captureAWS(require("aws-sdk"));
Typescript:
import AWSXRay from 'aws-xray-sdk';
import aws from 'aws-sdk';
const AWS = AWSXRay.captureAWS(aws)
Here is a sample test to confirm.
balu-test >> sample-test
Lambda 1 (balu-test) :
const AWSXRay = require("aws-xray-sdk-core");
const AWS = AWSXRay.captureAWS(require("aws-sdk"));
const lambda = new AWS.Lambda();
exports.handler = async function (event, context) {
var params = {
FunctionName: "sample-test",
InvocationType: "RequestResponse",
Payload: '{ "name" : "foo" }',
};
const response = await lambda.invoke(params).promise();
console.log('response',response);
return "sucess";
};
Lambda 2(sample-test):
const AWSXRay = require("aws-xray-sdk-core");
const AWS = AWSXRay.captureAWS(require("aws-sdk"));
let region = "us-east-1"
let secretName = "SomeSecret"
let secret
let decodedBinarySecret
var client = new AWS.SecretsManager({
region: region,
});
exports.handler = (event, context, callback) => {
client.getSecretValue({ SecretId: secretName }, function (err, data) {
if (err) {
callback(err);
} else {
if ("SecretString" in data) {
secret = data.SecretString;
} else {
let buff = new Buffer(data.SecretBinary, "base64");
decodedBinarySecret = buff.toString("ascii");
}
callback(null, secret);
}
});
};
TraceId is same and X-Ray points to same graph for both Lambda invocations. Same thing happens when first api is called from Api-Gateway. First time trace-id is generated and is passed along as http header to downstream processes.
I try to invoke a “child” lambda from a “parent” lambda
The example of code is very simple as below (I am using Serverless framework).
child_lambda
const mainHandler = async (event, context) => {
console.log('event: ', JSON.stringify(event));
return context.functionName;
};
export const handler = mainHandler;
parent_lambda
import AWS from 'aws-sdk';
const lambda = new AWS.Lambda();
const invokeLambda = async () => {
let sampleData = { number1: 1, number2: 2 };
let params = {
FunctionName: 'child_lambda',
Payload: JSON.stringify(sampleData),
Qualifier: '1'
};
try {
await lambda.invoke(params).promise();
return true;
} catch (e) {
console.log('invokeLambda :: Error: ' + e);
}
};
const mainHandler = async (event, context) => {
console.log('event: ', JSON.stringify(event));
await invokeLambda();
return context.functionName;
};
export const handler = mainHandler;
serverless.yml
parent_lambda:
handler: handlers/lambda/parent_lambda.handler
name: dev_parent_lambda
iamRoleStatements:
- Effect: "Allow"
Action:
- lambda: InvokeFunction
- lambda: InvokeAsync
Resource: "*"
events:
- http:
path: test/invokeLambda
method: GET
child_lambda:
handler: handlers/lambda/child_lambda.handler
name: dev_child_lambda
I run the parent from Postman and the result is
ResourceNotFoundException: Function not found:
arn:aws:lambda:xxxx:xxxxx:function:dev_child_lambda
I tried to trigger the child_lambda from an S3 event, it worked fine, but never work with invoke as AWS SDK.
Any suggestion is appreciated
From the comments, code given in the question is perfect except the Qualifier parameter
Qualifier is used to
Specify a version or alias to invoke a published version of the
function.
In this case, lambda is not versioned. Hence we just need to remove qualifier .
const invokeLambda = async () => {
let sampleData = { number1: 1, number 2: 2 };
let params = {
FunctionName: 'child_lambda',
Payload: JSON.stringify(sampleData)
};
try {
await lambda.invoke(params).promise();
return true;
} catch (e) {
console.log('invokeLambda :: Error: ' + e);
}
};
Lambda Asynchronous invocation
Amazon Simple Storage Service (Amazon S3) invoke functions asynchronously to process events. When you invoke a function asynchronously, you don't wait for a response from the function code. You hand off the event to Lambda and Lambda handles the rest.
In that case, I would simply chain the lambdas using AWS lambda destinations. Supported destinations:
Amazon SQS – sqs:SendMessage
Amazon SNS – sns:Publish
Lambda – lambda:InvokeFunction
EventBridge – events:PutEvents
Configuring destinations for asynchronous invocation
Introducing AWS Lambda Destinations
I've trying to call lambda function from another lambda function and get result to execute rest of the lambda.
Basic flow of function is below
X - main lambda function
- process A (independent)
- process C (need input from process B)
- process D
- return final dataset
Y - Child lambda function
- process B ( need input from process A and respond back to X )
This is my code so far
var AWS = require('aws-sdk');
AWS.config.region = 'us-east-1';
var lambda = new AWS.Lambda();
const GetUserCheckoutData: Handler = async (userRequest: EmptyProjectRequest, context: Context, callback: Callback) => {
const dboperation = new UserController();
const usercheckoutdata = new CheckOutInfo();
const addresscontroller = new AddressController();
const ordercontroller = new OrderController();
const paypalcreateorder = new PayPalController();
const userid = await dboperation.getUserID(userRequest.invokeemailAddress);
usercheckoutdata.useraddressdetails = await addresscontroller.GetListOfAddressByUserID(userid);
var orderlist = new Array<Order>();
orderlist = [];
orderlist = await ordercontroller.GetCurrentOrder(userid);
console.log("Order Complete");
var params = {
FunctionName: 'api-ENGG-SellItem', // the lambda function we are going to invoke
InvocationType: 'RequestResponse',
LogType: 'Tail',
Payload: '{ "orderlist" : xxxxxxx }'
};
lambda.invoke(params, (err:any, res:any) => {
if (err) {
callback(err);
}
console.log(JSON.stringify(res));
callback(null, res.Payload);
});
usercheckoutdata.orderID = await paypalcreateorder.CreateOrder(userid , orderlist);
usercheckoutdata.orderPreview = await ordercontroller.OrderPreview(userid);
//callback(null,usercheckoutdata);
};
export { GetUserCheckoutData }
I tried a few different ways but flow is not working properly. cross lambda function is executing. but cannot get the response on time.
My child lambda function demo code
import { Handler, Context } from "aws-lambda";
const SellItem: Handler = (event, context, callback) => {
context.callbackWaitsForEmptyEventLoop = false;
console.log("Other Lambda Function");
setTimeout(() => {
callback(null, "My name is Jonathan");
}, 1000 * 10); // 10 seconds delay
}
export {SellItem}
I think since I don't have much NodeJS knowledge this is happening. don't know how to put call back in right way I guess. Any help will be appreciated
You should make your call to the second lambda a promise, so you can await it.
const res = await lambda.invoke(params).promise();
// do things with the response
I have the following function which I use to invoke a Lambda function from within my code.
However when I try to use it within a Lambda function, I get the following error:
AWS lambda undefined 0.27s 3 retries] invoke({ FunctionName: 'my-function-name',
InvocationType: 'RequestResponse',
LogType: 'Tail',
Payload: <Buffer > })
How can I invoke a Lambda function from within a Lambda function?
My function:
'use strict';
var AWS = require("aws-sdk");
var lambda = new AWS.Lambda({
apiVersion: '2015-03-31',
endpoint: 'https://lambda.' + process.env.DYNAMODB_REGION + '.amazonaws.com',
logger: console
});
var lambdaHandler = {};
// #var payload - type:string
// #var functionName - type:string
lambdaHandler.invokeFunction = function (payload, functionName, callback) {
var params = {
FunctionName: functionName, /* required */
InvocationType: "RequestResponse",
LogType: "Tail",
Payload: new Buffer(payload, 'utf8')
};
var lambdaRequestObj = lambda.invoke(params);
lambdaRequestObj.on('success', function(response) {
console.log(response.data);
});
lambdaRequestObj.on('error', function(response) {
console.log(response.error.message);
});
lambdaRequestObj.on('complete', function(response) {
console.log('Complete');
});
lambdaRequestObj.send();
callback();
};
module.exports = lambdaHandler;
Invoking a Lambda Function from within another Lambda function is quite simple using the aws-sdk which is available in every Lambda.
I suggest starting with something simple first.
This is the "Hello World" of intra-lambda invocation:
Lambda_A invokes Lambda_B
with a Payload containing a single parameter name:'Alex'.
Lambda_B responds with Payload: "Hello Alex".
First create Lambda_B which expects a name property
on the event parameter
and responds to request with "Hello "+event.name:
Lambda_B
exports.handler = function(event, context) {
console.log('Lambda B Received event:', JSON.stringify(event, null, 2));
context.succeed('Hello ' + event.name);
};
Ensure that you give Lambda_B and Lambda_A the same role.
E.g: create a role called lambdaexecute which has AWSLambdaRole, AWSLambdaExecute and
AWSLambdaBasicExecutionRole (All are required):
Lambda_A
var AWS = require('aws-sdk');
AWS.config.region = 'eu-west-1';
var lambda = new AWS.Lambda();
exports.handler = function(event, context) {
var params = {
FunctionName: 'Lambda_B', // the lambda function we are going to invoke
InvocationType: 'RequestResponse',
LogType: 'Tail',
Payload: '{ "name" : "Alex" }'
};
lambda.invoke(params, function(err, data) {
if (err) {
context.fail(err);
} else {
context.succeed('Lambda_B said '+ data.Payload);
}
})
};
Once you have saved both these Lambda functions, Test run Lambda_A:
Once you have the basic intra-lambdda invocation working you can easily extend it to invoke more elaborate Lambda functions.
The main thing you have to remember is to set the appropriate ARN Role for all functions.
As of Dec 3, 2016, you can simply use an AWS Step function to put Lambda function Lambda_B as the sequential step of Lambda_A.
With AWS Step Functions, you define your application as a state
machine, a series of steps that together capture the behavior of the
app. States in the state machine may be tasks, sequential steps,
parallel steps, branching paths (choice), and/or timers (wait). Tasks
are units of work, and this work may be performed by AWS Lambda
functions, Amazon EC2 instances of any type, containers, or on
premises servers—anything that can communicate with the Step Functions
API may be assigned a task.
So the following state machine should meet your need.
Here is the code corresponding to the state machine.
{
"Comment": "A simple example of the Amazon States Language using an AWS Lambda Function",
"StartAt": "Lambda_A",
"States": {
"Lambda_A": {
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"Next": "Lambda_B"
},
"Lambda_B":{
"Type": "Task",
"Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:FUNCTION_NAME",
"End": true
}
}
}
Moreover, you can add much more sophisticated logics in a state machine, such as parallel steps and catch failures. It even logs the details of every single execution which makes debugging a much better experience, especially for lambda functions.
Everything mentioned by #nelsonic is correct, except for the roles.
I tried choosing the roles that he mentioned above:
AWSLambdaExecute
AWSLambdaBasicExecutionRole
But it did not allow me to invoke my other lambda function, so I changed the role to the below:
AWSLambdaRole
AWSLambdaBasicExecutionRole
The reason behind is AWSLambdaExecute only provides Put, Get access to S3 and full access to CloudWatch Logs.
but AWSLambdaRole provides Default policy for AWS Lambda service role.
if you observe its permission policy it will talk about the invokeFunction
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
"*"
]
}
]
}
Note: it is OK to proceed without AWSLambdaBasicExecutionRole policy as it only enables the logging in the cloud watch nothing much. But AWSLambdaRole is absolutely necessary.
It's easier to invoke a lambda using the AWS.Lambda promises interface in aws-sdk than using callbacks.
This example function lets you make a synchronous invocation of a lambda from another lambda (it uses 'RequestResponse' as InvocationType, so you'll can get the value returned by the invoked lambda).
If you use 'Event' (for asynchronous invocation), you can't get the value returned by the called lambda, only be able to detect whether the lambda could be invoked with success or not. It is intended for cases when you don't need to obtain a returned value from the invoked lambda.
//
// Full example of a lambda that calls another lambda
//
// (create a lambda in AWS with this code)
//
'use strict';
//
// Put here the name of the function you want to call
//
const g_LambdaFunctionName = 'PUT_HERE_THE_INVOKED_LAMBDA_NAME'; // <======= PUT THE DESIRED VALUE
const AWS = require('aws-sdk');
const lambda = new AWS.Lambda;
//
// Expected use:
//
// // (payload can be an object or a JSON string, for example)
// let var = await invokeLambda(lambdaFunctionName, payload);
//
const invokeLambda = async (lambdaFunctionName, payload) => {
console.log('>>> Entering invokeLambda');
// If the payload isn't a JSON string, we convert it to JSON
let payloadStr;
if (typeof payload === 'string')
{
console.log('invokeLambda: payload parameter is already a string: ', payload);
payloadStr = payload;
}
else
{
payloadStr = JSON.stringify(payload, null, 2);
console.log('invokeLambda: converting payload parameter to a string: ', payloadStr);
}
let params = {
FunctionName : lambdaFunctionName, /* string type, required */
// ClientContext : '', /* 'STRING_VALUE' */
InvocationType : 'RequestResponse', /* string type: 'Event' (async)| 'RequestResponse' (sync) | 'DryRun' (validate parameters y permissions) */
// InvocationType : 'Event',
LogType : 'None', /* string type: 'None' | 'Tail' */
// LogType : 'Tail',
Payload : payloadStr, /* Buffer.from('...') || 'JSON_STRING' */ /* Strings will be Base-64 encoded on your behalf */
// Qualifier : '', /* STRING_VALUE' */
};
//
// TODO/FIXME: add try/catch to protect this code from failures (non-existent lambda, execution errors in lambda)
//
const lambdaResult = await lambda.invoke(params).promise();
console.log('Results from invoking lambda ' + lambdaFunctionName + ': ' , JSON.stringify(lambdaResult, null, 2) );
// If you use LogType = 'Tail', you'll obtain the logs in lambdaResult.LogResult.
// If you use 'None', there will not exist that field in the response.
if (lambdaResult.LogResult)
{
console.log('Logs of lambda execution: ', Buffer.from(lambdaResult.LogResult, 'base64').toString());
}
console.log('invokeLambdaSync::lambdaResult: ', lambdaResult);
console.log('<<< Returning from invokeLambda, with lambdaResult: ', JSON.stringify(lambdaResult, null, 2));
// The actual value returned by the lambda it is lambdaResult.Payload
// There are other fields (some of them are optional)
return lambdaResult;
};
//
// We'll assign this as the calling lambda handler.
//
const callingFunc = async (event) => {
//
// in this example We obtain the lambda name from a global variable
//
const lambdaFunctionName = g_LambdaFunctionName;
// const payload = '{"param1" : "value1"}';
const payload = event;
//
// invokeLambda has to be called from a async function
// (to be able to use await)
//
const result = await invokeLambda(lambdaFunctionName, payload);
console.log('result: ', result);
};
// Assing handler function
exports.handler = callingFunc;
Notice that you should use await before invokeLambda:
...
//
// Called from another async function
//
const result = await invokeLambda(lambdaFunctionName, payload);
...
Some relevant links with additional information:
AWS Reference about invoke call: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html#invoke-property
AWS documentation about invoking a lambda: https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html
Promises interface in AWS:
https://aws.amazon.com/es/blogs/compute/node-js-8-10-runtime-now-available-in-aws-lambda/
Examples (explanations in Spanish): https://www.it-swarm.dev/es/node.js/invocar-aws-lambda-desde-otra-lambda-de-forma-asincronica/826852446/
Handling errors, avoiding coupling between lambdas: https://www.rehanvdm.com/serverless/13-aws-lambda-design-considerations-you-need-to-know-about-part-2/index.html
Invoke Lambda AWS SDK Typescript
I wrote my own class to do this, parse the response and check errors. I've posted it here to save anyone else who wants the effort :)
This requires aws-sdk and ts-log.
import { AWSError, Lambda } from 'aws-sdk'
import { Logger } from 'tslog';
export class LambdaClient {
awsLambda: Lambda;
logger: Logger;
constructor(region: string) {
this.awsLambda = new Lambda({ region })
this.logger = new Logger({ name: "LambdaClient" })
}
trigger({ functionName, payload }): Promise<any> {
return new Promise(
(resolve, reject) => {
const params = {
FunctionName: functionName,
InvocationType: 'RequestResponse',
LogType: 'Tail',
Payload: JSON.stringify(payload)
};
this.awsLambda.invoke(params, (err: AWSError, data: Lambda.InvocationResponse) => {
if (err) {
this.logger.error({ message: "error while triggering lambda", errorMessage: err.message })
return reject(err)
}
if (data.StatusCode !== 200 && data.StatusCode !== 201) {
this.logger.error({ message: "expected status code 200 or 201", statusCode: data.StatusCode, logs: base64ToString(data.LogResult) })
return reject(data)
}
const responsePayload = data.Payload
return resolve(JSON.parse(responsePayload.toString()))
})
}
)
}
}
function base64ToString(logs: string) {
try {
return Buffer.from(logs, 'base64').toString('ascii');
} catch {
return "Could not convert."
}
}