I'm trying to query a DynamoDB table from a Lambda function (for an Alexa skill), but when I send the intent that calls require('aws-sdk'), the skill seems to hang and timeout. The Alexa test page just says "There was a problem with the requested skill's response", and I don't see any errors in the CloudWatch logs. I have the skill set up to catch any errors and return them as a spoken response, so I'm sure it's not an uncaught exception. I've also tried wrapping the require in a try/catch block, but that didn't work either.
This is the module that gets loaded with require if the test database intent request is received:
const AWS = require('aws-sdk');
module.exports = () => {
return 'Success!';
};
If I comment out require('aws-sdk'), the function works properly and Alexa responds with "Success".
Why does my skill break when all I'm doing is requiring the aws-sdk module?
I'm very new to AWS and this is my first experience trying to access a DynamoDB table in a Lambda function.
The Lambda function is uploaded as a zip that contains my source code, package.json (that includes aws-sdk as a dependency), and the node_modules folder.
After hours of debugging, I've found that changing import * as AWS from 'aws-sdk'; to import {DynamoDB} from 'aws-sdk'; (or {CloudFront} or whatever you actually use) made the timeout issue disappear. Mind you, the time to actually connect to DynamoDB was never an issue for me, it was always the import line where the timeout happened.
This can be fixed by either increasing the timeout or the memory allotted to the lambda function.
This is probably because the SDK is too big to be imported by the default timeout value of 3 seconds and the default memory value of 128 MB.
This is why it will probably work if you try importing smaller components like only DynamoDB.
Lambda, when using NodeJS, uses a callback continuation model. Your module should export a function that takes three parameters: event, context, and callback.
Event provides input parameters.
The other two are used for returning control from your handler function, depending on the NodeJS version you are using.
Try adding the three parameters that I mentioned and the, from within your exported handler function, call:
module.export = function(event, context, callback) {
callback(‘success’);
}
Bear in mind, I wrote this on mobile off the top of my mind, so you may need to make small afjustments to the code but the idea is the same. Don’t return directly from the function, but call the callback to supply the response as a continuation. And note that in earlier versions of NodeJS, prior to version 4, you would have to use the context to set success or failure, rather than calling the callback.
For more details, see the Lambda with NodeJS tech docs on AWS.
The other thing to keep in mind is that for Alexa specifically, the response should be in the correct format. That is a JSON response that contains all the necessary elements as explained in the Alexa Skills Kit tech docs.
The Alexa ASK sdk that you’ve included generates those responses but I thought I should point you to the actual docs in case you were going to try building the response by hand to understand how it works.
Related
I have an AWS Lambda application built upon an external library that contains an EventEmitter. On a certain event, I need to make a HTTP request. So I was using this code (simplified):
myEmitter.on("myEvent", async() => {
setup();
await doRequest();
finishingWork();
});
What I understand that happens is this:
My handler is called, but as soon as the doRequest function is called, a Promise is returned and the EventEmitter continues with the next handlers. When all that is done, the work of the handler can continue (finishingWork).
This works locally, because my NodeJS process keeps running and any remaining events on the eventloop are handled. The strange thing is that this doesn't seem to work on AWS Lambda. Even if context.callbackWaitsForEmptyEventLoop is set to true.
In my logging I can see my handler enters the doRequest function, but nothing after I call the library to make the HTTP call (request-promise which uses request). And the code doesn't continue when I make another request (which I would expect if callbackWaitsForEmptyEventLoop is set to false, which it isn't).
Has anyone experienced something similar and know how to perform an ansynchronous HTTP request in the handler of a NodeJS event emitter, on AWS Lambda?
I have similar issue as well, my event emitter logs all events normally until running into async function. It works fine in ECS but not in Lambda, as event emitter runs synchronously but Lambda will exit once the response is returned.
At last, I used await-event-emitter to solve the problem.
await emitter.emit('onUpdate', ...);
If you know how to solve this, feel free to add another answer. But for now, the "solution" for us was to put the eventhandler code elsewhere in our codebase. This way, it is executed asynchronously.
We were able to do that because there is only one place where the event is emitted, but the eventhandler way would have been a cleaner solution. Unfortunately, it doesn't seem like it's possible.
When using the SNS client found on AWS-SDK:
const sns = new AWS.SNS({});
Should I reuse this object across calls to save a handshake with the server?
This kind of object is usually stateless and benefits from pooling/cache; However the docs aren't really clear about that.
I believe you should initiate the class outside of your lambda.
AWS will reuse the instance when possible.
E.g.
const AWS = require('aws-sdk')
const sns = new AWS.SNS()
module.exports.handler = async input => {
// use sns class here
return input
}
EDIT:
Link to the official documentation that explains how the lambda execution context works: https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html
Any declarations in your Lambda function code (outside the handler
code, see Programming Model) remains initialized, providing additional
optimization when the function is invoked again. For example, if your
Lambda function establishes a database connection, instead of
reestablishing the connection, the original connection is used in
subsequent invocations. We suggest adding logic in your code to check
if a connection exists before creating one.
We are trying to migrate our zip microservice from regular application in nodejs Express to AWS API Gateway integrated with AWS Lambda.
Our current application sends request to our API, gets list of attachments and then visits those attachments and pipes their content back to user in form of zip archive. It looks something like this:
module.exports = function requestHandler(req, res) {
//...
//irrelevant code
//...
return getFileList(params, token).then(function(fileList) {
const filename = `attachments_${params.id}`;
res.set('Content-Disposition', `attachment; filename=${filename}.zip`);
streamFiles(fileList, filename).pipe(res); <-- here magic happens
}, function(error) {
errors[error](req, res);
});
};
I have managed to do everything except the part where I have to stream content out of Lambda function.
I think one of possible solutions is to use aws-serverless-express, but I'd like a more elegant solution.
Anyone has any ideas? Is it even possible to stream out of Lambda?
Unfortunately lambda does not support streams as events or return values. (It's hard to find it mentioned explicitly in the documentation, except by noting how invocation and contexts/callbacks are described in the working documentation).
In the case of your example, you will have to await streamFiles and then return the completed result.
(aws-serverless-express would not help here, if you check the code they wait for your pipe to finish before returning: https://github.com/awslabs/aws-serverless-express/blob/master/src/index.js#L68)
n.b. There's a nuance here that a lot of the language SDK's support streaming for requests/responses, however this means connecting to the stream transport, e.g. the stream downloading the complete response from the lambda, not listening to a stream emitted from the lambda.
Had the same issue, now sure how you can do stream/pipe via the native lambda + API Gateway directly... but it's technically possible.
We used Serverless Framework and were able to use XX.pipe(res) using this starter kit (https://github.com/serverless/examples/tree/v3/aws-node-express-dynamodb-api)
What's interesting is that this just wraps over native lambda + API Gateway so, technically it is possible as they have done it.
Good luck
If you have a error in code and did not callback, Azure Function just hangs.
AWS by default detects for empty loop and exits the lambda and logs the error, return the call to the caller with 5xx. It can also be controlled with callbackWaitsForEmptyEventLoop = false.
Is there is anything in Azure Functions that we can control how the function should exist in case of unhandled exceptions.
This currently isn't supported on Azure Functions.
It's the first I've heard this request, but it's a good idea. If you open a GitHub issue for it, I can take a look. (I'm the dev responsible for Node.js on Azure Functions) https://github.com/Azure/azure-functions
If you are using Azure Functions beta instead of using callbacks you may also export your functions as async which should allow the node worker to handle the exception without further intervention.
When exporting async functions calling the callback is redundant, and unnecessary.
Example:
module.exports = async function (context, req) {
throw new Error('Error')
}
Returns:
Request URL:http://localhost:7071/api/HttpTriggerJS
Request Method:GET
Status Code:500 Internal Server Error
Remote Address:[::1]:7071
Referrer Policy:no-referrer-when-downgrade
I'd like to use Postgresql as a database on my AWS lambda functions but I'm worried about performance.
I'm worried that Lambdas are stateless and only exist in the time they're executing so I imagine every time the Lambda is triggered it'll try to initiate a brand new PG connection.
I'm not sure if this decreases performance or causes issues with stale connections somehow. Anyone know more about this?
I know DynamoDB is more in line with Lambda but I really need a relational database but at the same time Lambda's scalability.
You can make use of the container execution model of AWS lambda. When a lambda is invoked, AWS spins up a container to run the code inside the handler function. So if you define the PG connection outside the handler function it will be shared among the invocations of Lambda functions. You can find that in the above link.
Any declarations in your Lambda function code (outside the handler code, see Programming Model) remains initialized, providing additional optimization when the function is invoked again. For example, if your Lambda function establishes a database connection, instead of reestablishing the connection, the original connection is used in subsequent invocations. You can add logic in your code to check if a connection already exists before creating one.
const pg = require('pg');
const client = new pg.Client(<connection_string>);
exports.handler = (event, context, cb) => {
client.query('SELECT * FROM users WHERE ', (err, users) => {
// Do stuff with users
cb(null); // Finish the function cleanly
});
};
Refer this blog post.
But there is a caveat.
When you write your Lambda function code, do not assume that AWS Lambda always reuses the container because AWS Lambda may choose not to reuse the container. Depending on various other factors, AWS Lambda may simply create a new container instead of reusing an existing container.
Additionally you can create a scheduled job to warm up lambda function. (runs in every 5mins)