Im creating a lambda function using Node.js, and Im looking to bundle the dependencies into my zip to be used in my Lambda function in AWS. Specifically Im using the ffmpeg library, and I've got it running locally using npm. Is there a way that I can bundle this dependency with the .zip file that I upload so that I don't have to configure the dependency in AWS?
Not only is there a way to bundle this into your deployment package, but you have to do it this way. Lambda functions can't download dependencies.
Here's the documenation: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-create-deployment-pkg.html
Also you might want to look at this project: https://github.com/binoculars/aws-lambda-ffmpeg and possibly this thread: https://forums.aws.amazon.com/thread.jspa?messageID=680948 for more specific information about running ffmpeg on Lambda.
Related
First time dealing with serverless here. Have successfully deployed using serverless deploy after following the guide to migrating an existing express app over to serverless. But aws lambda keeps throwing an error:
“errorType”: “Runtime.ImportModuleError”,
“errorMessage”: “Error: Cannot find module ‘serverless-http’\nRequire stack:\n- /var/task/app.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js”,
So I’m confused. What am I doing wrong? The guide to converting an existing express app didn’t say we’d need to create an AWS Lambda Layer, but since it seems like Lambda can’t find the serverless-http module, does it mean that creating a layer is the fix?
Edit: my .zip file seems to only be an express.js file. I'm not sure if that's unusual but reading online reveals that most people seem to have a node_modules folder zipped up as well? I presume the lack of a node_module folder in the .zip file is causing this runtime.ImportModuleError fail? How do I get serverless to add a node_modules folder if that's the case?
node_modules folder is generally packaged with your code -- provided it exists in your directory (does it?).
If node_modules isn't installed on your local machine, then you can create it by using the npm install command. This command will install all the dependencies listed in the package-lock.json file (or package.json) -- sorry I'm not a node guy :(.
But it definitely sounds like you're not uploading your node_modules folder because it's not on your local machine. You have to initialize the directory first.
I want to deploy my node app in AWS lambda. The npm packages I am using are not pre-installed in lambda. So how can I deploy the whole node app in lambda? There is a option of uploading the files as a zip file. But how can I build the correct lambda file system?
You need to zip the file contents. basically
somezip.zip
node_modules
index.js
anyotherffile.js
package.json
Upload the zip and call the exposed handler function.
You could also use framework such as - https://www.npmjs.com/package/serverless
I recommend you to have a look at the AWS reference documentation.
The following steps could be done:
~/my-function$ npm install
~/my-function$ zip -r function.zip .
~/my-function$ aws lambda update-function-code --function-name my-function --zip-file fileb://function.zip
Look for node environments available on the following page from AWS.
Be aware of the limitations applicable, as Dec 27 2019 , if the deployment package is larger than 50 MB, you must use Amazon S3.
I'm working on an AWS Lambda function in Visual Studio that calls for fast image resizing. Originally, I was using ImageMagick to resize these images, but the entire process is is taking too long to process. The alternative to ImageMagick seems to be Sharp, another image library. Sharp supposedly will use Native AWS Lambda code in order to resize the images faster. According to this post, Sharp needs to be built in an EC2 instance to do this (https://github.com/awslabs/serverless-image-resizing).
I haven't been able to get this to work yet. Has anyone here got this to work yet? If so, is there a pre-built library out there for Sharp built in EC2 or should I build it myself?
I did find a Sharp library in npm, but I wasn't able to get it to work. Is the Sharp library in npm even compatible with AWS Lambda functions? If so, does it have the increased speed by using native processing?
To get sharp to work for me on AWS Lambda, I was able to add an npm script in my package.json and then do a npm run dockerbuild using docker-lambda. This compiled on the Amazon Linux docker image, but left the outputs was in my project folder (so I could package the .zip for deployment to AWS Lambda).
"scripts": {
"dockerbuild": "docker run -v \"$PWD\":/var/task lambci/lambda:build-nodejs6.10"
},
I was also able to test inside the docker image after npm install docker-lambda - since the sharp native library was compiled for Linux (and not my host system), I could not test directly.
I'm having an issue getting a node module to load in AWS Lambda using the Serverless Framework. One of my node packages uses native libraries so I used an EC2 to create the node_module folders and then copied them to my Serverless project. Everything works if I manually zip the project and upload to AWS Lambda but if I use serverless deploy without an artifact specified, I get an error about the module (specifically: ELF file's phentsize not the expected size regarding a .node file)
I've tried adding excludeDevDependencies: false which makes the deployment larger but still gives me the error. Currently, it only works if I zip the contents of the project folder and specify that file as the artifact to upload. Is there a different way to get a node module with native bindings to deploy with Serverless?
UPDATE: After turning off the exclusion of dev dependencies, packaging using serverless package and examining the expanded zip file serverless creates, I discovered the file sizes of the .o and .a files are different in the packaged version as compared to the original. Is this normal?
I ran into this problem and did some digging. It turns out that v1.21.0 of serverless broke packaging of binaries.
https://forum.serverless.com/t/serverless-1-21-0-breaks-sharp-library/2606
https://github.com/serverless/serverless/issues/4182
The recommended fix is to upgrade to v1.21.1.
Since Lambda runs on a Linux container, you should be running serverless deploy from a Linux machine.
This way, your native modules will be compiled for your target architecture which is Linux.
To check the deployment package that serverless creates, you can use sls package or sls deploy --noDeploy (for older versions), and inspect the .serverless directory that it creates. You'll see a zip file in here, extract its contents, and test the code from there.
If the contents of this zip is not what you expect (not the same as when you manually copy them), then maybe something is wrong with your file structure and/or serverless.yml.
I am using AWS Lambda and I want to install some external resources
npm install textapi
...
var TextAPI = require('textapi');
how can I install this when running my Lambda function.
Note I know I have to upload a zip file just not sure what else is needed to get this running.
Your zip file should contain your entire app, from a Node perspective that will include your node_modules folder.
See the docs.