Lambda Invocation error with callback called: metrics show no errors - node.js

Question

I have created a Lambda function in AWS that export logs from Cloudfront to Elasticsearch.
From the AWS console, I still have a warining in front of Invocation error, though the metrics show there is none for more than 24hours.
A typical workflow of logs looks like
START RequestId: 302f0b95-7856-11e8-9486-55b3f10e7d4e Version: $LATEST
Request complete
END RequestId: 302f0b95-7856-11e8-9486-55b3f10e7d4e
REPORT RequestId: 302f0b95-7856-11e8-9486-55b3f10e7d4e Duration: 794.93 ms Billed Duration: 800 ms Memory Size: 128 MB Max Memory Used: 52 MB
There is no error in the logs, and the only thing I guess could trigger this invocation error is that sometimes two request starts at the same time
09:01:47
START RequestId: 63cd00e1-7856-11e8-8f96-1f900def8e65 Version: $LATEST
09:01:47
START RequestId: 63e1e7f3-7856-11e8-97e6-3792244f6ab0 Version: $LATEST
Except from this, I don't understand where this error comes from.
Do I miss something? Or do I have to wait more than 24hours before the satus change? May be there is a way to pinpoint the error with AWS console/API that I did not find about?
Would be happy to hear what you think about this.
Edit: In case you'd like to take a look at the code itself.
var aws = require('aws-sdk');
var zlib = require('zlib');
var async = require('async');
const CloudFrontParser = require('cloudfront-log-parser');
var elasticsearch = require('elasticsearch');
var s3 = new aws.S3();
var client = new elasticsearch.Client({
host: process.env.ES_HOST,
log: 'trace',
keepAlive: false
});
exports.handler = function(event, context, callback) {
var srcBucket = event.Records[0].s3.bucket.name;
var srcKey = event.Records[0].s3.object.key;
async.waterfall([
function fetchLogFromS3(next){
console.log('Fetching compressed log from S3...');
s3.getObject({
Bucket: srcBucket,
Key: srcKey
},
next);
},
function uncompressLog(response, next){
console.log("Uncompressing log...");
zlib.gunzip(response.Body, next);
},
function publishNotifications(jsonBuffer, next) {
console.log('Filtering log...');
var json = jsonBuffer.toString();
var records;
CloudFrontParser.parse(json, { format: 'web' }, function (err, accesses) {
if(err){
console.log(err);
} else {
records = accesses;
}
});
var bulk = [];
records.forEach(function(record) {
bulk.push({"index": {}})
bulk.push(record);
});
client.bulk({
index: process.env.ES_INDEXPREFIX,
type: 'log',
body: bulk
}, function(err, resp, status) {
if(err) {
console.log('Error: ', err);
}
console.log(resp);
next();
});
console.log('CloudFront parsed:', records);
}
], function (err) {
if (err) {
console.error('Failed to send data: ', err);
} else {
console.log('Successfully send data.');
}
callback(null, 'success');
});
};

Answer 1

You need to explicitly return information back to the caller.
Here's the related documentation:
The Node.js runtimes v6.10 and v8.10 support the optional callback
parameter. You can use it to explicitly return information back to the
caller. The general syntax is:
callback(Error error, Object result); Where:
error – is an optional parameter that you can use to provide results
of the failed Lambda function execution. When a Lambda function
succeeds, you can pass null as the first parameter.
result – is an optional parameter that you can use to provide the
result of a successful function execution. The result provided must be
JSON.stringify compatible. If an error is provided, this parameter is
ignored.
If you don't use callback in your code, AWS Lambda will call it
implicitly and the return value is null.
When the callback is called (explicitly or implicitly), AWS Lambda
continues the Lambda function invocation until the event loop is
empty.
The following are example callbacks:
callback(); // Indicates success but no information returned to the caller.
callback(null); // Indicates success but no informatio returned to the caller.
callback(null, "success"); // Indicates success with information returned to the caller.
callback(error); // Indicates error with error information returned to the caller
https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html