I try to invoke an api gateway from an aws lambda, which is triggered by a sqs queue. And my api gateway call another lambda function to get some informations.
But when i send a message from my sqs queue, lambda is correctly executed but the invocation of my api gaetway is not executed.
My code below :
var https = require('https');
const URL = process.env.API_GATEWAY_URL;
exports.handler = async (event, context, callback) => {
console.log("hello");
https.get(URL, function (result) {
console.log('Success, with: ' + result.statusCode);
context.done(null);
}).on('error', function (err) {
console.log('Error, with: ' + err.message);
context.done("Failed");
});
};
And here are the logs I have in cloudwatch:
START RequestId: 4eeccd03-b7d2-5efc-aedb-8c3a14411101 Version: $LATEST
2020-04-21T13:14:01.524Z 4eeccd03-b7d2-5efc-aedb-8c3a14411101 INFO hello
END RequestId: 4eeccd03-b7d2-5efc-aedb-8c3a14411101
But i expect something like that:
START RequestId: 4eeccd03-b7d2-5efc-aedb-8c3a14411101 Version: $LATEST
2020-04-21T13:14:01.524Z 4eeccd03-b7d2-5efc-aedb-8c3a14411101 INFO hello
2020-04-21T13:14:01.524Z 4eeccd03-b7d2-5efc-aedb-8c3a14411101 Success, with: 200
END RequestId: 4eeccd03-b7d2-5efc-aedb-8c3a14411101
Or:
START RequestId: 4eeccd03-b7d2-5efc-aedb-8c3a14411101 Version: $LATEST
2020-04-21T13:14:01.524Z 4eeccd03-b7d2-5efc-aedb-8c3a14411101 INFO hello
2020-04-21T13:14:01.524Z 4eeccd03-b7d2-5efc-aedb-8c3a14411101 Error, with : message_error
END RequestId: 4eeccd03-b7d2-5efc-aedb-8c3a14411101
And all my lambda are in the same VPC.
Any ideas?
You have mixed the newer async function handler with the older context/callback pattern.
If you want to use the older style, then remove the async function decorator.
If you want to use the newer (preferred) style, then change your code to return a promise, for example:
const https = require('https');
const URL = 'https://dummy.restapiexample.com/api/v1/employees';
exports.handler = async (event, context) => {
return new Promise((resolve, reject) => {
https.get(URL,(res) => {
let body = "";
res.on("data", (chunk) => {
body += chunk;
});
res.on("end", () => {
console.log(body);
resolve(body);
});
}).on("error", (error) => {
reject(error);
});
});
};
And make the GET request to the correct URL, of course.
Related
I have the following Lambda.
const AWS = require('aws-sdk');
exports.lambdaHandler = async (event, context) => {
console.log("START v5");
const SNSarn = process.env.SNS;
console.log(SNSarn);
const snsParams={
Message: "test",
TopicArn: SNSarn
};
const SNS = new AWS.SNS();
SNS.publish(snsParams, function(err,data){
console.log ("We are in the callback!");
if (err) {
console.log('Error sending a message', err);
context.done(err,'We have errors');
}
console.log('Sent message');
context.done(null,"Done!");
});
console.log('END');
};
When I run it locally, ther lambda performs as expected and pushes a message to SNS.
For some reason, I do not see the console.log that is in the callback.
How do I fix it so I can also see logs from callbacks?
Below is the output I do see:
START RequestId: ff2691cc-a7d0-40f2-a956-277bxxxx21d Version: $LATEST
2021-11-15T21:17:17.968Z ff2691cc-a7d0-40f2-a956-277bxxxx821d INFO START v5
2021-11-15T21:17:17.970Z ff2691cc-a7d0-40f2-a956-277bxxxx821d INFO arn:aws:sns:us-east-1:xxxxxxxxxxxxxxx:tooktook-topic
2021-11-15T21:17:17.992Z ff2691cc-a7d0-40f2-a956-277bxxxx821d INFO END
END RequestId: ff2691cc-a7d0-40f2-a956-277bxxxx821d
REPORT RequestId: ff2691cc-a7d0-40f2-a956-277bxxxx821d Init Duration: 0.19 ms Duration: 261.33 ms Billed Duration: 300 ms Memory Size: 128 MB Max Memory Used: 128 MB
In method SNS.publish, you are using callback which is asynchronous in nature and there is a possibility of successful lambda execution before callback is even called. SNS takes a bit of time for execution. you need to use async/ await in this case. AWS APIs support promises.
Here is an example:
const AWS = require('aws-sdk');
exports.lambdaHandler = async (event, context) => {
console.log("START v5");
const SNSarn = process.env.SNS;
console.log(SNSarn);
const snsParams={
Message: "test",
TopicArn: SNSarn
};
const SNS = new AWS.SNS();
try {
const data = await SNS.publish(snsParams).toPromise();
console.log('Sent Message');
context.done(null, 'Done');
} catch (error) {
console.log('We are in the callback!');
context.done(err, 'We have errors');
}
console.log('END');
};
I am trying to use the request module in AWS Lambda for Node.js, but I am not having good results.
This is the function I have so far:
const request = require('request');
url = 'https://www.google.com'
exports.handler = async (event) => {
request(url, (error,res,body) => {
console.log(error); //this is not printing
console.log(res); //this is not printing
console.log(body); //this is not printing
console.log('Come on!!'); //this is not printing
});
console.log(url); //This is the only thing printing, you can see in the Function Logs below, second line.
};
and this is the response I am getting:
Response:
null
Request ID:
"3e46f401-f26d-435d-90be-ac848c6c3a39"
Function Logs:
START RequestId: 3e46f401-f26d-435d-90be-ac848c6c3a39 Version: $LATEST
2019-10-14T08:06:23.755Z 3e46f401-f26d-435d-90be-ac848c6c3a39 INFO https://www.google.com
END RequestId: 3e46f401-f26d-435d-90be-ac848c6c3a39
REPORT RequestId: 3e46f401-f26d-435d-90be-ac848c6c3a39 Duration: 368.93 ms Billed Duration: 400 ms Memory Size: 128 MB Max Memory Used: 92 MB Init Duration: 461.54 ms
I am wondering why I am not getting any response from inside the requests method. I tried on my laptop and the part of the code outside the handler works just fine.
I uploaded the project as a zip file, so the node_modules folder with the request module is indeed there.
I noted a couple of solutions using the http module instead of request, but I would like to understand first why this is not working, before moving to a different solution.
Your function is marked as async, yet you are using a callback to handle the response. So nothing is being awaited on.
This means that your function returns an implicit promise that is already fulfilled. This makes the lambda stop. So, either:
A) Write your code using async/await, which probably requires request-promise or similar:
exports.handler = async event => {
const res = await promisifiedRequest(url);
console.log(res.statusCode);
console.log(url);
};
or
B) Convert the function from async/await to use callbacks, like so:
exports.handler = (event, context, callback) => {
const res = request(url, (error, res, body) => {
console.log('Come on!!');
callback(null, res.statusCode);
});
console.log(url);
};
I am trying to write a lambda function in node 8.10 supported by aws lambda
this is with reference to my previous question
I am trying node async/await , same code work when I run it in my local(and is not even taking a second to get the response back) but when I am trying it in lambda getting time out error, here the code control is getting inside promise and printing url urltohit with the correct url but after it is getting stuck, have tried different thing like increasing the time in lambda config and even tried node request instead of http but still getting the same error.
'use strict';
var http = require('http');
var request = require('request');
exports.handler = async (event) => {
const response = await getRequest(urltohit);
console.log(response);
}
const getRequest = async (url) => {
console.log("Test log");
return new Promise((resolve, reject) => {
console.log(`url ${url}`);
http.get(url, function(res) {
console.log("Got response: " + res.statusCode);
resolve(res.statusCode);
}).on('error', function(e) {
console.log("Got error: " + e.message);
reject(e.message);
});
});
}
You need to call callback after function execution.
exports.handler = async (event, context, callback) => {
const response = await getRequest(urltohit);
console.log(response);
callback(null);
}
It's lambda behaviour to keep function running until callback called.
More could be found in official documentation
I have created a Lambda function in AWS that export logs from Cloudfront to Elasticsearch.
From the AWS console, I still have a warining in front of Invocation error, though the metrics show there is none for more than 24hours.
A typical workflow of logs looks like
START RequestId: 302f0b95-7856-11e8-9486-55b3f10e7d4e Version: $LATEST
Request complete
END RequestId: 302f0b95-7856-11e8-9486-55b3f10e7d4e
REPORT RequestId: 302f0b95-7856-11e8-9486-55b3f10e7d4e Duration: 794.93 ms Billed Duration: 800 ms Memory Size: 128 MB Max Memory Used: 52 MB
There is no error in the logs, and the only thing I guess could trigger this invocation error is that sometimes two request starts at the same time
09:01:47
START RequestId: 63cd00e1-7856-11e8-8f96-1f900def8e65 Version: $LATEST
09:01:47
START RequestId: 63e1e7f3-7856-11e8-97e6-3792244f6ab0 Version: $LATEST
Except from this, I don't understand where this error comes from.
Do I miss something? Or do I have to wait more than 24hours before the satus change? May be there is a way to pinpoint the error with AWS console/API that I did not find about?
Would be happy to hear what you think about this.
Edit: In case you'd like to take a look at the code itself.
var aws = require('aws-sdk');
var zlib = require('zlib');
var async = require('async');
const CloudFrontParser = require('cloudfront-log-parser');
var elasticsearch = require('elasticsearch');
var s3 = new aws.S3();
var client = new elasticsearch.Client({
host: process.env.ES_HOST,
log: 'trace',
keepAlive: false
});
exports.handler = function(event, context, callback) {
var srcBucket = event.Records[0].s3.bucket.name;
var srcKey = event.Records[0].s3.object.key;
async.waterfall([
function fetchLogFromS3(next){
console.log('Fetching compressed log from S3...');
s3.getObject({
Bucket: srcBucket,
Key: srcKey
},
next);
},
function uncompressLog(response, next){
console.log("Uncompressing log...");
zlib.gunzip(response.Body, next);
},
function publishNotifications(jsonBuffer, next) {
console.log('Filtering log...');
var json = jsonBuffer.toString();
var records;
CloudFrontParser.parse(json, { format: 'web' }, function (err, accesses) {
if(err){
console.log(err);
} else {
records = accesses;
}
});
var bulk = [];
records.forEach(function(record) {
bulk.push({"index": {}})
bulk.push(record);
});
client.bulk({
index: process.env.ES_INDEXPREFIX,
type: 'log',
body: bulk
}, function(err, resp, status) {
if(err) {
console.log('Error: ', err);
}
console.log(resp);
next();
});
console.log('CloudFront parsed:', records);
}
], function (err) {
if (err) {
console.error('Failed to send data: ', err);
} else {
console.log('Successfully send data.');
}
callback(null, 'success');
});
};
You need to explicitly return information back to the caller.
Here's the related documentation:
The Node.js runtimes v6.10 and v8.10 support the optional callback
parameter. You can use it to explicitly return information back to the
caller. The general syntax is:
callback(Error error, Object result); Where:
error – is an optional parameter that you can use to provide results
of the failed Lambda function execution. When a Lambda function
succeeds, you can pass null as the first parameter.
result – is an optional parameter that you can use to provide the
result of a successful function execution. The result provided must be
JSON.stringify compatible. If an error is provided, this parameter is
ignored.
If you don't use callback in your code, AWS Lambda will call it
implicitly and the return value is null.
When the callback is called (explicitly or implicitly), AWS Lambda
continues the Lambda function invocation until the event loop is
empty.
The following are example callbacks:
callback(); // Indicates success but no information returned to the caller.
callback(null); // Indicates success but no informatio returned to the caller.
callback(null, "success"); // Indicates success with information returned to the caller.
callback(error); // Indicates error with error information returned to the caller
https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html
I have this Lambda function:
exports.handler = async (event,context,callback) => {
// console.log('Received event:', JSON.stringify(event, null, 2));
var key=event.queryStringParameters.ref;
// console.log('index.handler started with key : '+key);
var RemoteServer='aaa.bbb.ccc.ddd';
var net = require('net');
var responce={
"statusCode": 200,
"headers": {
"Content-Type": "*/*"
}
};
var socket = new net.Socket();
socket.setEncoding('utf8');
socket.on('close', () => {
console.log('Close.');
console.log(JSON.stringify(responce));
callback(null, JSON.stringify(responce));
});
socket.on('connect', () => {
// console.log('Connect');
var senddata='1,'+key;
// console.log(('About to write :'+senddata));
socket.write(senddata);
console.log(('Wrote :'+senddata));
});
socket.on('data', (data) => {
console.log('Data.');
responce.body = data;
console.log(JSON.stringify(responce));
socket.end();
});
socket.on('end', () => {
console.log('End.');
console.log(JSON.stringify(responce));
});
socket.on('error', (error) => {
console.log('Error' + error);
socket.destroy;
callback(error);
});
socket.on('timeout', () => {
console.log('Timeout');
socket.close;
callback('Timeout');
});
socket.connect(11010, RemoteServer, () => {
// console.log('socket connect');
});
callback(null,responce); // This is here to get this to do anything.
};
It works, in as such that the console.log shows that I am getting the correct data in the socket.on('data') routine.
However, it fails to return the responce.body in the callback.
Also, it only works if I have a callback in the async function, not in the event listeners.
This is my output - console.log:
START RequestId: 7e1876fe-4255-12de-56d6-c187bcfff9ee Version: $LATEST
Wrote :1,R9H39X
Data.
{"statusCode":200,"headers":{"Content-Type":"*/*"},"body":"<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n<group>\r\n <data>\r\n <name>COLIN MANNING</name>\r\n <from>26/03/2018</from>\r\n <to>31/05/2018</to>\r\n <room>A31</room>\r\n <location>X,Y</location>\r\n </data>\r\n</group>\r\n"}
End.
{"statusCode":200,"headers":{"Content-Type":"*/*"},"body":"<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n<group>\r\n <data>\r\n <name>COLIN MANNING</name>\r\n <from>26/03/2018</from>\r\n <to>31/05/2018</to>\r\n <room>A31</room>\r\n <location>X,Y</location>\r\n </data>\r\n</group>\r\n"}
Close.
{"statusCode":200,"headers":{"Content-Type":"*/*"},"body":"<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n<group>\r\n <data>\r\n <name>COLIN MANNING</name>\r\n <from>26/03/2018</from>\r\n <to>31/05/2018</to>\r\n <room>A31</room>\r\n <location>X,Y</location>\r\n </data>\r\n</group>\r\n"}
END RequestId: 7e1876fe-4255-12de-56d6-c187bcfff9ee
REPORT RequestId: 7e1876fe-4255-12de-56d6-c187bcfff9ee Duration: 178.76 ms Billed Duration: 200 ms Memory Size: 1536 MB Max Memory Used: 33 MB
and callback:
{"statusCode":200,"headers":{"Content-Type":"*/*"}}
And this is the console.log without the last callback:
START RequestId: c3dc7e69-87aa-1608-76d4-093a0e28711b Version: $LATEST
END RequestId: c3dc7e69-87aa-1608-76d4-093a0e28711b
REPORT RequestId: c3dc7e69-87aa-1608-76d4-093a0e28711b Duration: 3.43 ms Billed Duration: 100 ms Memory Size: 1536 MB Max Memory Used: 33 MB
and callback:
null
So without the callback, nothing runs.
What I want to achieve is:
Open socket to server.
send data through socket
get reply from server through socket
close socket
return reply to API Gateway
This is my first attempt at Node.js async, Lambda and API Gateway.
I assume that I have an error in the way I understand Node.js async and callbacks.
How do I return the data from the socket.on('close') listener?
It appears to me that the callback in the listener is never called, I and cannot determine how to make the callback in the async function await for the data from the listener.
Am I supposed to use EventAsPromise and then block with a while (!done) {....}? This seams to go against the Node async philosophy.
OK, so I found this post:
How Can I Wait In Node.js (Javascript), l need to pause for a period of time
and went down to Aminadav's answer.
So I end up with this function declared ahead of my existing code:
function sleep(ms){
return new Promise(resolve=>{
setTimeout(resolve,ms)
})
};
and now the end of my original code looks like this:
socket.connect(11010, RemoteServer, () => {
// console.log('socket connect');
});
while (!socket.destroyed) {
await sleep(10);
}
callback(null,responce);
};
So now I get the data returned to my calling function.
I'm not sure that a sleep() is the best way to do this, but it gets me moving on. This is an answer, but I appreciate that it may not be the best answer.