i'm writing a serverless node rest api, and i have few functions and today i faced an issue on sending responds from lambda function to api gateway, my callback doesn't work as it expected, what i am doing wrong?
module.exports.create = (event, context, callback) => {
client.on('connect', () => {
console.log("connected to redis");
callback(null, {
statusCode: 200,
headers: { 'Content-Type': 'text/plain' },
body: 'connection established.',
});
return;
});
};
A common issue people have with Lambda and NodeJS is timing... I think what's happening here is that the Lambda Function terminates before your response comes back. Lambda does not wait around for an async response, so most of the time doesn't execute the responds events, so never hits your callback.
Try using a Promise, which keeps the code/Lambda running until the async call comes back and the callback is called.
This is a good article on how to achieve that:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function
Ok, I've encountered Internal server error a few times before and I suggest you to do this.
First, a little background knowledge you must have:
When you're deploying your serverless application, what is happening under the hood is that serverless framework creates necessary configurations and .zip file (your lambda functions code and dependencies) under .serverless folder.
So if you're missing necessary dependencies in your package.json or forget to include them in the .zip file, your lambda will return Internal server error.
And you should check whether you included dependencies into dev-dependencies in package.json, too. (This will prevent your necessary modules to be included in .zip file).
And secondly, if you're using serverless-webpack plugin, you should include these lines in your serverless.yaml file.
custom:
webpack:
includeModules: true
This worked for my case.
If you don't understand or have anything to ask, feel free to do that :)
Internal errors are when the return isn't hit for some reason, it can be a coding error or a timeout because the default serverless timeout is too low for what you are trying to do.
If you want to alter the timeout you can do something like this in the serverless.yml:
functions:
create:
handler: handler/create
timeout: 30
...
Related
Trying out the transloadit api, the template works when I use the testing mode on the transloadit website, but when I try to use it in Node JS with the SDK I'm getting an error:
INVALID_FORM_DATA - https://api2.transloadit.com/assemblies - INVALID_FORM_DATA: The form contained bad data, which cannot be parsed.
The relevant code: (_asset.content) is a Buffer object
async function getThumbnailUrl(_assetkey: string, _asset: I.FormFile): Promise<string> {
let tOptions = {
waitForCompletion: true,
params: {
template_id: process.env.THUMB_TRANSLOADIT_TEMPLATE,
},
};
const stream = new Readable({
read() {
this.push(_asset.content);
this.push(null);
},
});
console.log(_asset.content);
util.transloadit.addStream(_assetkey, stream);
return new Promise((resolve, reject) => {
util.transloadit.createAssembly(tOptions, (err, status) => {
if (err) {
reject(err);
}
console.log(status);
//return status;
resolve(status);
});
});
}
I noticed that you also posted this question on the Transloadit forums - so in the case that anyone else runs into this problem you can find more information on this topic here.
Here's a work-around that the OP found that may be useful:
Just to provide some closure to this topic, I just tested my
workaround (upload to s3, then use import s3 robot to grab the file)
and got it to work with the nodejs sdk so i should be good using that.
I have a suspicion the error I was getting was not to do with the
transloadit api, but rather the form-data library for node js
(https://github.com/form-data/form-data 1) and that’s somehow not
inputting the form data in the way that the transloadit api is
expecting.
But as there aren’t alternatives to that library that I could find, I
wasn’t really able to test that hypothesis.
The Transloadit core team also gave this response regarding the issue:
It may try to set his streams to be Tus streams which would mean that
they’re not uploaded as multipart/form data.
In either case it seems like the error to his callback would be
originating from the error out of _remoteJson
These could be the problem areas
https://github.com/transloadit/node-sdk/blob/master/src/TransloaditClient.js#L146
https://github.com/transloadit/node-sdk/blob/master/src/TransloaditClient.js#L606
https://github.com/transloadit/node-sdk/blob/master/src/TransloaditClient.js#L642
It is also possible that the form-data library could be the source of
the error
To really test this further we’re going to need to try using the
library he was using, make sure the output of it is good, and then
debug the node-sdk to see where the logic failure is in it, or if the
logic failure is on the API side.
We are trying to migrate our zip microservice from regular application in nodejs Express to AWS API Gateway integrated with AWS Lambda.
Our current application sends request to our API, gets list of attachments and then visits those attachments and pipes their content back to user in form of zip archive. It looks something like this:
module.exports = function requestHandler(req, res) {
//...
//irrelevant code
//...
return getFileList(params, token).then(function(fileList) {
const filename = `attachments_${params.id}`;
res.set('Content-Disposition', `attachment; filename=${filename}.zip`);
streamFiles(fileList, filename).pipe(res); <-- here magic happens
}, function(error) {
errors[error](req, res);
});
};
I have managed to do everything except the part where I have to stream content out of Lambda function.
I think one of possible solutions is to use aws-serverless-express, but I'd like a more elegant solution.
Anyone has any ideas? Is it even possible to stream out of Lambda?
Unfortunately lambda does not support streams as events or return values. (It's hard to find it mentioned explicitly in the documentation, except by noting how invocation and contexts/callbacks are described in the working documentation).
In the case of your example, you will have to await streamFiles and then return the completed result.
(aws-serverless-express would not help here, if you check the code they wait for your pipe to finish before returning: https://github.com/awslabs/aws-serverless-express/blob/master/src/index.js#L68)
n.b. There's a nuance here that a lot of the language SDK's support streaming for requests/responses, however this means connecting to the stream transport, e.g. the stream downloading the complete response from the lambda, not listening to a stream emitted from the lambda.
Had the same issue, now sure how you can do stream/pipe via the native lambda + API Gateway directly... but it's technically possible.
We used Serverless Framework and were able to use XX.pipe(res) using this starter kit (https://github.com/serverless/examples/tree/v3/aws-node-express-dynamodb-api)
What's interesting is that this just wraps over native lambda + API Gateway so, technically it is possible as they have done it.
Good luck
If you have a error in code and did not callback, Azure Function just hangs.
AWS by default detects for empty loop and exits the lambda and logs the error, return the call to the caller with 5xx. It can also be controlled with callbackWaitsForEmptyEventLoop = false.
Is there is anything in Azure Functions that we can control how the function should exist in case of unhandled exceptions.
This currently isn't supported on Azure Functions.
It's the first I've heard this request, but it's a good idea. If you open a GitHub issue for it, I can take a look. (I'm the dev responsible for Node.js on Azure Functions) https://github.com/Azure/azure-functions
If you are using Azure Functions beta instead of using callbacks you may also export your functions as async which should allow the node worker to handle the exception without further intervention.
When exporting async functions calling the callback is redundant, and unnecessary.
Example:
module.exports = async function (context, req) {
throw new Error('Error')
}
Returns:
Request URL:http://localhost:7071/api/HttpTriggerJS
Request Method:GET
Status Code:500 Internal Server Error
Remote Address:[::1]:7071
Referrer Policy:no-referrer-when-downgrade
I need to publish data from aws lambda through mqtt protocol using aws iot. i have created a lambda function with node.js code. like this
exports.handler = (event, context, callback) => {
var awsIot = require('aws-iot-device-sdk');
var device = awsIot.device({
keyPath: 'samplepath/test.pem.key',
certPath: 'samplepath/test.crt',
caPath: 'samplepath',
clientId: 'sampleId',
region: 'us-east-1'
});
device
.on('connect', function () {
console.log('connected');
device.publish('test_topic', JSON.stringify({ "test_name": "hello", "test_value": 1001 }));
console.log('published successfully');
callback(null, 'item added');
});
}
I got mqtt message on subscriber. but lambda produce error message like this
Task timed out after 10.00 seconds
I have used context.succeed() instead of callback, lambda is exited properly. i cant get any messages on subscriber.
In both cases console prints published successfully message properly.
What is the issue related with my publishing code?
I understand my lambda function is timing out when connecting to AWS
IoT. About the sdk we are using, the aws-iot-device-sdk is designed to
use inside of an embedded device. When we are using a Lambda function
or trying to publish in a computer, the best practice is use the
aws-sdk. Using the aws-sdk we don't need to use the certificates to
publish in the AWS IoT, we just use the AWS credentials to do this.
Also, with aws-sdk we can do administrative tasks in the IoT, we can
create a thing, create a certificate, etc.
Coming to my code, the reason the function does not end and times out
is because the callback must be waiting for an asynchronous call to
finish execution, which I assume is being help up by the connection
being maintained from the function to IoT. The reason
context.succeed() exited properly but we did not get any messages must
be because context.succeed does not wait for our async calls to finish
execution.
Make sure you disconnect from the device after you published the message, otherwise Lambda will wait while the connection stays alive (see http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html, look for callbackWaitsForEmptyEventLoop).
To disconnect when done, simply change callback(null, 'item added'); to
device.end((err) => {
callback(err, "item added");
});
We've written a RESTful web API which responds to GET and PUT requests using node.js.
We're having some difficulties testing the API.
First, we used Zombie.js, but it's not well documented so we couldn't get it to make PUT requests:
var zombie = require("zombie");
describe("description", function() {
it("description", function() {
zombie.visit("http://localhost:3000/", function (err, browser, status) {
expect(browser.text).toEqual("A")
});
});
});
After that we tried using a REST-client called restler, which would OK, since we don't need any advanced browser simulation. This fails due to the fact that the request seems to be asynchronous - i.e. the test is useless since it finishes before the 'on success' callback is called:
var rest = require('restler');
describe("description", function() {
it("description", function() {
rest.get("http://www.google.com").on('complete', function(data, response) {
// Should fail
expect(data).toMatch(/apa/i);
});
});
});
We'd grateful for any tips about alternative testing frameworks or synchronous request clients.
For node, jasmine-node from Misko Hevery has asynchronous support and wraps jasmine.
https://github.com/mhevery/jasmine-node
You add a 'done' parameter to the test signature, and call that when the asynchronous call completes. You can also customize the timeout (the default is 500ms).
e.g. from the Github README
it("should respond with hello world", function(done) {
request("http://localhost:3000/hello", function(error, response, body){
done();
}, 250); // timeout after 250 ms
});
jasmine regular also has support for asynchronous testing with runs and waitsFor, or can use 'done' with Jasmine.Async.
I was curious about this so I did a little more research. Other than zombie, you have a couple of options...
You could use vows with the http library like this guy.
However, I think a better approach might be to use APIeasy, which is apparently built on vows. There is an awesome article over at nodejitsu that explains how to use it.
Another interesting idea is to use expresso if you are using express.