I'm working on an AWS Lambda function in Visual Studio that calls for fast image resizing. Originally, I was using ImageMagick to resize these images, but the entire process is is taking too long to process. The alternative to ImageMagick seems to be Sharp, another image library. Sharp supposedly will use Native AWS Lambda code in order to resize the images faster. According to this post, Sharp needs to be built in an EC2 instance to do this (https://github.com/awslabs/serverless-image-resizing).
I haven't been able to get this to work yet. Has anyone here got this to work yet? If so, is there a pre-built library out there for Sharp built in EC2 or should I build it myself?
I did find a Sharp library in npm, but I wasn't able to get it to work. Is the Sharp library in npm even compatible with AWS Lambda functions? If so, does it have the increased speed by using native processing?
To get sharp to work for me on AWS Lambda, I was able to add an npm script in my package.json and then do a npm run dockerbuild using docker-lambda. This compiled on the Amazon Linux docker image, but left the outputs was in my project folder (so I could package the .zip for deployment to AWS Lambda).
"scripts": {
"dockerbuild": "docker run -v \"$PWD\":/var/task lambci/lambda:build-nodejs6.10"
},
I was also able to test inside the docker image after npm install docker-lambda - since the sharp native library was compiled for Linux (and not my host system), I could not test directly.
Related
Is there one equivalent of Google's JIB or BuildPacks for Node.JS ?
It is my understanding that JIB allows to build OCI container images from within the project's build tool like Gradle or Maven, as a developer we only have to include a plugin into the build and are able to package up the application into a container and having JIB implement all the best practices of packing up a Java application into container with no questions asked.
I have search around but have not found something equivalent for the Node.JS ecosystem.
It should be possible just into a node developer time dependency and it take care on packaging up my javascript/typescript Express.js for example app into a docker container or OCI image.
Thank you, Oscar
For posterity, I'll list some NodeJS-native Docker image creation packages (these usually can be added to your project's package.json). In no particular order:
nodejs-container-image-builder - container registry client and image builder with no dependency on docker (by Google)
Dockta - A Docker image builder for researchers
EZDocker - build docker images in Javascript
I did try Dockta and it has SUPER simple one-liner docker file/image build (either a simple package.json script or direct command line), it works nicely.
Yes, Heroku has a Node.js Buildpack. You can run it using the Pack CLI like this:
$ pack build myimage --builder heroku/buildpacks:18 --buildpack heroku/nodejs
If you are using GitLab, you can simply use Kaniko as well.
I want to deploy my node app in AWS lambda. The npm packages I am using are not pre-installed in lambda. So how can I deploy the whole node app in lambda? There is a option of uploading the files as a zip file. But how can I build the correct lambda file system?
You need to zip the file contents. basically
somezip.zip
node_modules
index.js
anyotherffile.js
package.json
Upload the zip and call the exposed handler function.
You could also use framework such as - https://www.npmjs.com/package/serverless
I recommend you to have a look at the AWS reference documentation.
The following steps could be done:
~/my-function$ npm install
~/my-function$ zip -r function.zip .
~/my-function$ aws lambda update-function-code --function-name my-function --zip-file fileb://function.zip
Look for node environments available on the following page from AWS.
Be aware of the limitations applicable, as Dec 27 2019 , if the deployment package is larger than 50 MB, you must use Amazon S3.
I have deployed the Node.js code on Google Cloud using following command:-
gcloud app deploy
So, How to download Node.js project deployed on Google Cloud.
I'm not sure but if you need Tomcat server then set and deploy node.js application inside by creating a folder and add the dist folder files in it.
npm install --save #google-cloud/storage
At this point in time it is only possible to download your app's source code for Java, Python, PHP and Golang. The instructions are similar for Python, Golang and PHP:
appcfg.py -A [YOUR_PROJECT_ID] -V [YOUR_VERSION_ID] download_app [OUTPUT_DIR]
where:
[YOUR_PROJECT_ID] is your GCP project ID.
[YOUR_VERSION_ID] is the version ID of your application that you want to download.
[OUTPUT_DIR] is the full directory path to where you want your files downloaded.
In Java you use appcfg.sh:
appcfg.sh -A [YOUR_PROJECT_ID] -V [YOUR_VERSION_ID] download_app [OUTPUT_DIR]
See the link above and also the reference for Java and Python, PHP, Golang
I'm having an issue getting a node module to load in AWS Lambda using the Serverless Framework. One of my node packages uses native libraries so I used an EC2 to create the node_module folders and then copied them to my Serverless project. Everything works if I manually zip the project and upload to AWS Lambda but if I use serverless deploy without an artifact specified, I get an error about the module (specifically: ELF file's phentsize not the expected size regarding a .node file)
I've tried adding excludeDevDependencies: false which makes the deployment larger but still gives me the error. Currently, it only works if I zip the contents of the project folder and specify that file as the artifact to upload. Is there a different way to get a node module with native bindings to deploy with Serverless?
UPDATE: After turning off the exclusion of dev dependencies, packaging using serverless package and examining the expanded zip file serverless creates, I discovered the file sizes of the .o and .a files are different in the packaged version as compared to the original. Is this normal?
I ran into this problem and did some digging. It turns out that v1.21.0 of serverless broke packaging of binaries.
https://forum.serverless.com/t/serverless-1-21-0-breaks-sharp-library/2606
https://github.com/serverless/serverless/issues/4182
The recommended fix is to upgrade to v1.21.1.
Since Lambda runs on a Linux container, you should be running serverless deploy from a Linux machine.
This way, your native modules will be compiled for your target architecture which is Linux.
To check the deployment package that serverless creates, you can use sls package or sls deploy --noDeploy (for older versions), and inspect the .serverless directory that it creates. You'll see a zip file in here, extract its contents, and test the code from there.
If the contents of this zip is not what you expect (not the same as when you manually copy them), then maybe something is wrong with your file structure and/or serverless.yml.
Im creating a lambda function using Node.js, and Im looking to bundle the dependencies into my zip to be used in my Lambda function in AWS. Specifically Im using the ffmpeg library, and I've got it running locally using npm. Is there a way that I can bundle this dependency with the .zip file that I upload so that I don't have to configure the dependency in AWS?
Not only is there a way to bundle this into your deployment package, but you have to do it this way. Lambda functions can't download dependencies.
Here's the documenation: http://docs.aws.amazon.com/lambda/latest/dg/nodejs-create-deployment-pkg.html
Also you might want to look at this project: https://github.com/binoculars/aws-lambda-ffmpeg and possibly this thread: https://forums.aws.amazon.com/thread.jspa?messageID=680948 for more specific information about running ffmpeg on Lambda.