My project's structure is next:
MyProject
layers
myLayer1
nodejs
node_modules
myLayer1
myExtension.js
lambda1
handler.js
lambda2
handler.js
jsconfig.json
myExtension.js
module.exports.myTest = () => {
return 'My extension test';
};
handler.js
const myext = require('myLayer1');
module.exports.handler = async (event) => {
return {
statusCode: 200,
body: JSON.stringify({
message: myext.myTest()
})
}};
When I deploy to AWS - things are working.
But I'm unable to run/debug it on my local machine.
According to what I found the jsconfig.json file should help to map paths in this case, but VSCode/NodeJS ignore it whatever I wrote there (I tried to place it the MyProject root folder and within lambda folders).
I can run this lambdas locally if I change the 'require' within the handler.js to:
const myext = require('./layers/myLayer1/nodejs/node_modules/myLayer1');
which obviously breaks the code when I deploy it to AWS.
Overview
This is similar to another question that is looking for how to reference modules from multiple layers: How configure Visual Studio Code to resolve input paths for AWS Lambda Layers (javascript)
The simpler answer, for a lambda function with a single layer for node_modules, is to use a jsconfig.json file in the directory where the lambda's package.json file with no dependencies is located (this is not the root of the entire project, but the root of the lambda function project only).
The example project of the question does attempt to use a jsconfig.json file but it should be in the lambda1 and lambda2 directories, while the question has this file located at the top-level, which does not have the desired effect.
Example jsconfig.json file for VS Code
This file should be placed at both of the following locations for the example project in the question:
MyProject/lambda1/jsconfig.json
MyProject/lambda2/jsconfig.json
// https://code.visualstudio.com/docs/languages/jsconfig
{
"compilerOptions": {
// baseUrl is the path used for pathless / naked node module includes
"baseUrl": "../layer/node_modules/",
"moduleResolution": "node",
"module": "commonJS"
}
}
Complete Example Code / Project Structure
https://github.com/pwrdrvr/layers-js-demo
Related
I am trying to move my application's API to Vercel. It is written in Typescript and uses Express.
The index.ts is located in <root>/src. The npm run build compiles it into <root>/dist directory. The file contains the following:
const app = express();
app.use((req: Request, res: Response, next: NextFunction) => {
//blah, blah, there is a lot going on here
})
app.use('/', common);
//... other app.use(s)
const port = process.env.PORT || 5000;
app.listen(port, () => console.log(`Server running on ${port}, http://localhost:${port}`));
module.exports = app;
I've got the following in the vercel.json file which is located in the root directory where the package.json also is:
{
"version": 2,
"installCommand": "npm install",
"buildCommand": "npm run build",
"outputDirectory": "dist",
"builds": [
{
"src": "dist/index.js",
"use": "#vercel/node"
}
],
"routes": [
{
"src": "/(.*)",
"dest": "dist/index.js"
}
]
}
When locally I run npm run build, then vercel dev --listen 5000 I get Ready! Available at http://localhost:5000 and can go to http://localhost:5000/ping and get a response.
Now I commit the files to git, the deployment runs, but judging by the logs the npm install and npm run build commands are not running. No functions are created my /ping endpoint returns "Page not found".
Here is the deployment log:
This is what Build & Development Settings look like (the Root Directory is left blank):
I followed several recommendations I found online and according to them everything should work. I probably miss some setting somewhere. What is it?
If more information is needed, please let me know, I'll update my question.
Thank you.
--- UPDATE ---
I have set the Root Directory to src and checked the "Include source files outside of the Root Directory in the Build Step" checkbox. Now the npm install and npm run build are executing. As you can see some static files are deployed, but there are still no serverless functions and my /ping route returns 404 and "home" page, i.e. / route returns the content of the index.js file. In addition the local is not working either anymore, also returning 404 now.
Without that checkbox I was getting
Warning: The vercel.json file should exist inside the provided root directory
and still no install or build running.
Also worth noting that I had to change my tsconfig.json to have "outDir": "src/dist" instead of "outDir": "dist", otherwise I was getting
Error: No Output Directory named "dist" found after the Build completed. You can configure the Output Directory in your Project Settings.
Removed the Root directory and back to square one, no npm commands running but local is working with / route returning Cannot GET / and /ping returning correct response.
For everyone out there who's looking for an answer, maybe this will help you.
In my case, what I needed is to create a folder, called api in my src folder, i.e. the folder that is specified as Root Directory in Build & Development Settings in Vercel. Within this directory, each serverless function needs a file named the same as the path of the route. For example, the file named "my-route.js" will be accessible via https://my-app-name.vercel.com/api/my-route.
All this file needs is an import of index.js file and module.exports. For example:
import app from '../index';
module.exports = app;
The index.js should also live the Root and contain your express setup.
If you want to have dynamic path parameters, the files' names in the api directory should be wrapped in square brakets, like [my-param.js]. You can also have sub-directories in the api foler.
Here are a few links that helped me figure this out:
https://dev.to/andrewbaisden/how-to-deploy-a-node-express-app-to-vercel-2aa
https://medium.com/geekculture/deploy-express-project-with-multiple-routes-to-vercel-as-multiple-serverless-functions-567c6ea9eb36
https://ryanccn.dev/posts/vercel-framework/#path-segments
No changes were needed in my existing Express setup and routes files.
Hope this will help someone. Took me quite a while to figure it all out :)
I know there are already many similar questions and answers to same problem, but I've tried all possible solutions and it's still not working. I have a aws layer (coded in a separate git repo) that I'm trying to utilize in my lambda repo. Here's the structure-
src
code
studenthelper
index.js
shared
logger.ts
index.js
jsconfig.json
package.json
And here's my content-
jsconfig.json
{
"compilerOptions":{
"baseUrl":".",
"paths":{
"/opt/nodejs/*":["src/shared/*"]
}
}
}
index.js(inside studenthelper)
const {errorCodes} = require('/opt/nodejs/logger');
I keep getting error cannot find module '/opt/nodejs/logger'.
I have to use the aws layer inside shared folder which is a typescript file that I can't change and my existing code too is restricted to JavaScript. Please help.
I'm using Typescript to write AWS Lambda code (both handlers and shared code) and then I transpile the code, add node_modules to the dir where transpiled code exist and publish the Lambda. This works correctly.
Now, if I want to add shared code as part of AWS Lambda layer, this is where it gets tricky with Typescript.
Consider following code structure that I have:
.
├── index.ts
├── lib
│ └── userData.ts
├── node_modules
├── package-lock.json
├── package.json
└── tsconfig.json
I would like to have a layer which contains:
node_modules
userData.ts
I'm aware of the structure that needs to exist when publishing the layer and also how to reference it in the index.ts (handler method) code.
Before layers, my index.ts code looks like this:
import { UserData } from "./lib/userData";
export const handler = async (event: any = {}): Promise<any> => {
# some code here
}
Because I want to import UserData from the layer, import would need to look like this:
import { UserData } from "./opt/nodejs/userData";
However, if I specify this, and try to transpile code (before creating a aws lambda package), I will get an obvious error:
index.ts:1:26 - error TS2307: Cannot find module '/opt/nodejs/userData' or its corresponding type declarations.
1 import { UserData } from "/opt/nodejs/userData";
~~~~~~~~~~~~~~~~~~~~~~
There are some ways to avoid this:
Adding // #ts-ignore before the actual import will silent this error
Dirty/hacky way would be to edit this import on transpiled code which is not what I want to do.
Now, this seems like a chicken and egg problem and I'm new to the AWS Lambdas, so any advice on how to handle this situation without these hacky ways to avoid it?
I solved this using typescript path mappings. You can add this to your tsconfig.json.
"baseUrl": ".",
"paths": {
"/opt/nodejs/*": ["src/testLayer/nodejs/*"]
This specifies to the TS compiler that the /opt/nodejs/ path is really pointing to src/testLayer/nodejs/ in our local directory structure, so it gets the files from there
I have an http cloud function that returns some dynamic HTML. I want to use Handlebars as the templating engine. The template is sufficiently big that it's not practical to have it in a const variable on top of my function.
I've tried something like:
const template = fs.readFileSync('./template.hbs', 'utf-8');
But when deploying the function I always get an error that the file does not exist:
Error: ENOENT: no such file or directory, open './template.hbs'
The template.hbs is in the same directory as my index.js file so I imagine the problem is that the Firebase CLI is not bundling this file along the rest of files.
According to the docs of Google Cloud Functions it is possible to bundle local modules with "mymodule": "file:mymodule". So I've tried creating a templates folder at the root of the project and added "templates": "file:./templates" to the package.json.
My file structure being something like this:
/my-function
index.js
/templates
something.hbs
index.js //this is the entry point
And then:
const template = fs.readFileSync('../node_modules/templates/something.hbs', 'utf-8');
But I'm getting the same not found error.
What is the proper way of including and requiring a non JS dependencies in a Firebase Cloud Function?
The Firebase CLI will package up all the files in your functions folder, except for node_modules, and send the entire archive to Cloud Functions. It will reconstitue node_modules by running npm install while building the docker image that runs your function.
If your something.hbs is in /templates under your functions folder, you should be able to refer to it as ./templates/something.hbs from the top-level index.js. If your JS is in another folder, you might have to work you way out first with ../templates/something.hbs. The files should all be there - just figure out the path. I wouldn't try to do anything fancy is your package.json. Just take advantage of the fact that the CLI deploys everything but node_modules.
This code works fine for me if I have a file called 'foo' at the root of my functions folder:
import * as fs from 'fs'
export const test = functions.https.onRequest((req, res) => {
const foo = fs.readFileSync('./foo', 'utf-8')
console.log(foo)
res.send(foo)
})
The solution was to use path.join(__dirname,'template.hbs').
const fs = require('fs');
const path = require('path');
const template = fs.readFileSync(path.join(__dirname,'template.hbs'), 'utf-8');
As #doug-stevenson pointed out all files are included in the final bundle but for some reason using the relative path did not work. Forcing an absolute path with __dirname did the trick.
I'm a Linux & node noob. I'm trying to run FabricJS (which requires node-canvas) in AWS Lambda. I've been able to follow the instructions to get up and running on an AWS Linux EC2, however, Lambda has me at my wits end. Anyone have any tips or pointers on how to get this compiled for AW Lambda?
I found this issue in the Node Canvas GitHub site. The questioner was trying to run FabricJS in Lambda as well. Here is the relevant section with an answer:
Make sure you're compiling this on the same AMI that lambda currently uses:
http://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html
Lambda runs at /var/task (that's the path when you unzip), so something.so at the root of the zip file will be at /var/task/something.so.
We then want to build our libraries using an "rpath":
export LDFLAGS=-Wl,-rpath=/var/task/
Compile libraries according to: https://github.com/Automattic/node-canvas/wiki/Installation---Amazon-Linux-AMI-%28EC2%29
You may want to set the prefix= ~/canvas to have all the files in one place.
Install node-canvas with npm
cd node_modules/canvas; node-gyp rebuild
mkdir ~/pkg and cp the .so files (~/canvas/lib/*.so) there, using -L to dereference symlinks.
scp the pkg directory to the local lambda folder, possibly putting the files in the right places. (.so in root, node_modules/canvas with other libs). You'll probably want to rearrange this.
Here is a gulp plugin which could upload your files along with node-canvas and its dependencies binary specifically built for aws lambda.
NPM Package
'use strict';
//This is a sample gulp file that can be used.
//npm install --save gulp gulp-zip gulp-awslambda
const gulp = require('gulp');
const zip = require('gulp-zip');
const path = require('path');
const lambda = require('gulp-awslambda');
const aws_lambda_node_canvas = require('./');
let runtime = 'nodejs4.3' // nodejs or nodejs4.3
const lambda_params = {
FunctionName: 'NodeCanvas', /* Lambda function name */
Description: 'Node canvas function in aws lambda', //Description for your lambda function
Handler: 'main.lambda_handler', //Assuming you will provide main.py file with a function called handler.
MemorySize: 128,
Runtime: runtime,
Role : 'ROLE_STRING',//eg:'arn:aws:iam::[Account]:role/lambda_basic_execution'
Timeout: 50
};
var opts = {
region : 'ap-southeast-2'
}
gulp.task('default', () => {
return gulp.src(['main.js', '!node_modules/**/*','!dist/**/*','!node_modules/aws-lambda-node-canvas/**/*']) //Your src files to bundle into aws lambda
.pipe(aws_lambda_node_canvas({runtime : runtime})) //Adds all the required files needed to run node-canvas in aws lambda
.pipe(zip('archive.zip'))
.pipe(lambda(lambda_params, opts))
.pipe(gulp.dest('dist')); //Also place the uploaded file
});