async / await breaks my Webpack build for AWS Lambda; how can I migrate to Node 8.10? - node.js

Note: this is a Q&A on migrating AWS Lambda Webpack builds from v6.10 to v8.10 — no help is needed, but even better answers are of course always encouraged!
For a while now I have been a disciple of using Webpack to build my back-end Lambdas after reading James Long's excellent series entitled "Backend Apps with Webpack" (part 1, part2, and part3).
Up until recently, the only version of Node.js that Amazon Web Services offered was 6.10; you had to write your Lambda fn in the "callback" style. But on April 2, 2018 AWS announced that 8.10 was now supported, and with it the suggested pattern is async / await which is great! Until it immediately broke my Webpack build. After a bit of debugging, I can break my build just by adding one async fn to the Lambda handler (I don't even need to call it):
async function firstAsync() {
return true;
}
exports.handler = async (event) => {
// TODO implement
return 'Hello from Lambda!'
};
To be clear, doing this in the AWS Lambda console is perfectly fine, runs great. Webpack even builds it successfully, but upon uploading to AWS Lambda, I get the following error message: regeneratorRuntime is not defined. My code is below. What am I needing to do?
webpack.config.js
const nodeExternals = require('webpack-node-externals');
const path = require('path');
const UglifyJsPlugin = require('uglifyjs-webpack-plugin')
const webpack = require('webpack');
const config = {
entry: './src/index.js',
output: {
path: path.resolve(__dirname, 'dist'),
library: 'index',
libraryTarget: 'commonjs2',
filename: 'index.js'
},
target: 'node', // building for a Node environment
externals: [nodeExternals()], // in order to ignore all modules in node_modules folder
module: {
rules: [{
test: /\.(js|jsx)$/,
use: {
loader: 'babel-loader',
options: {
presets: ['env']
}
}
}]
},
plugins: [
new UglifyJsPlugin()
]
};
module.exports = config;
package.json
{
"name": "lambda-webpack",
"version": "1.0.0",
"description": "An empty project scaffold to enable webpack builds in AWS Lambda",
"main": "index.js",
"scripts": {
"build": "webpack",
"upload": "upload.bat"
},
"author": "Geek Stocks®",
"license": "MIT",
"devDependencies": {
"aws-sdk": "^2.179.0",
"babel-core": "^6.26.0",
"babel-loader": "^7.1.2",
"babel-preset-env": "^1.6.1",
"uglifyjs-webpack-plugin": "^1.1.6",
"webpack": "^3.10.0",
"webpack-node-externals": "^1.6.0"
}
}

Geek Stock's answer above is only necessary when using the Node v6.10 runtime. It allows you to use async/await syntax and babel-runtime/regenerator converts your async functions to ES6 generators which is supported on Node v4.3.2 and newer runtimes.
In Node v8.10 though, it is different. async/await is already supported so you don't need Babel to convert your async functions to generators. You just use it correctly and it works.
As for your specific situation, I assume you simply changed your Javascript code to use async/await code but you didn't tell Babel NOT to convert your async/await code to generators. That is why babel-runtime is complaining about a missing plugin.
If this is the case, you simply need to configure Babel to simply target Node v8.10. You can do it like this in your .babelrc...
{
"presets": [
[
"env",
{
"targets": {
"node": "8.10"
}
}
]
]
}
babel will then stop converting those async functions. And since it's not doing a conversion, it will not need the regenerator-runtime anymore.

In my case I'm using it with nodejs12.x. I'm receiving the error regeneratorRuntime is not defined.
I was using preset es2015 and stage-0 with a node12.x code. Also I'm using an older version of babel.
What I did, installed:
"#babel/cli": "^7.10.1",
"#babel/core": "^7.10.2",
"#babel/preset-env": "^7.10.2"
And then set this babel preset:
"presets": [
["#babel/preset-env", {"targets": { "node": "current" }}]
]
Thanks,

To begin using async / await in your Webpack-built code, you need a little help from Babel, specfically the babel-runtime and the babel-plugin-transform-runtime. Fortunately there is a really good writeup on the installation and use of both here on the Babel website. You need Babel to do the following for you:
Automatically requires babel-runtime/regenerator when you use
generators/async functions.
I won't repeat what has been written there, however, of particular note is HOW you need to install these, because while their writeup certainly works for most, there is a needed tweak for some AWS Lambda devs who have not needed runtime dependencies before now.
The first part is pretty normal, you need new devDependencies:
npm install --save-dev babel-plugin-transform-runtime
And also you need to tweak your .babelrc file like they describe:
{
"plugins": [
["transform-runtime", {
"helpers": false,
"polyfill": false,
"regenerator": true,
"moduleName": "babel-runtime"
}]
]
}
But here is the kicker, and this is new to the code above: the babel-runtime is a regular dependencies, not a devDependencies:
npm install --save babel-runtime
These 2 installs and the .babelrc tweaks do SOLVE the async / await problem described above, but it introduces a new problem to the build above: Cannot find module 'babel-runtime/regenerator'.
If you are at all concerned with keeping your Webpack built code small, you are likely also using webpack-node-externals like the above code does. For example, the aws-sdk for JavaScript in Node is very large and is already available in the Lambda environment, so its superfluous to bundle it again. The webpack-node-externals configuration above configures Webpack to ignore ALL the modules in node-modules by default, and there is the root of the new problem. The newly installed babel-runtime is a module that needs to be bundled in the build in order for Lambda to run correctly.
Understanding the problem then, the answer becomes simple: don't use the default configuration. Instead of passing in nothing to webpack-node-externals, configure it like this:
externals: [nodeExternals({
whitelist: [
'babel-runtime/regenerator',
'regenerator-runtime'
]
})], // ignore all modules in node_modules folder EXCEPT the whitelist
And that solves both the original async / await problem, and the (possible) new problem you may be encountering if you had no previous dependencies in your build. Hope that helps — happy Lambda awaiting!

Related

Jasmine on Node.js fails with ERR_UNSUPPORTED_DIR_IMPORT

I'm writing an Express v4.18.2 app on Node.js 18.12.1 on Windows. I'm testing a controller with Jasmine 4.5.0. When I run jasmine spec, it fails with an error message about resolving ES modules:
Error [ERR_UNSUPPORTED_DIR_IMPORT]: Directory import
'...\SimpleServiceJasmine\spec' is not supported resolving ES modules
imported from
...\SimpleServiceJasmine\node_modules\jasmine\lib\loader.js
...
code: 'ERR_UNSUPPORTED_DIR_IMPORT',
url: 'file:///D:/.../SimpleServiceJasmine/spec'
I use require, not import, everywhere in the code being tested or the spec.
Note that jasmine runs fine if I specify the spec file explicitly, even with wildcards, as long as the wildcard path resolves to a single file:
jasmine spec/service/contact-api.spec.js # ok
jasmine spec/*/c* # ok
I tried downgrading jasmine to 3.0.0 and 2.0.1 but got the same error. The behavior is the same on Windows 11 and Windows Server 2019.
Any suggestions for how can I run all the specs in this project?
Here's my package.json:
{
"name": "simpleservice",
"version": "1.0.0",
"description": "A simple CRUD API for contacts",
"main": "service/contact-api.js",
"scripts": {
"test": "jasmine spec/service/contact-api.spec.js",
"start": "node src/service/contact-api.js"
},
"author": "Puzzled Dev",
"license": "ISC",
"dependencies": {
"express": "^4.18.2"
},
"devDependencies": {
"jasmine": "^4.5.0"
}
}
Here's the spec/support/jasmine.json:
{
"spec_dir": "spec",
"spec_files": [
"**/*[sS]pec.?(m)js"
],
"helpers": [
"helpers/**/*.?(m)js"
],
"env": {
"stopSpecOnExpectationFailure": false,
"random": true
}
}
Jasmine uses glob for pattern matching, which means whenever you pass spec as a pattern, glob finds this particular directory and then returns it to jasmine. Unfortunately, path validation in jasmine does not seem in a very smart way, it continues executing with the given path, and then it throws an error by Node.js.
It is equivalent to this:
import spec from './spec'
You can take a look at the jasmine-runner code, and the line in which contributor put a comment:
The ES module spec requires import paths to be valid URLs. As of v14,
Node enforces this on Windows but not on other OSes. On OS X, import
paths that are URLs must not contain parent directory references.
So, once you have defined spec_files in jasmine.json, there is no need to give an extra pattern to execute all tests.
jasmine will execute all tests with the help of spec_dir, and spec_files fields in jasmine.json. All you need to do is just simply run jasmine or npx jasmine.

Jest moduleNameMapper and NPM package exports

I am developing fullstack NPM packages with multiple entrypoints (namely client, server and sometimes tests), using the exports property of package.json.
I also use a small hack to make TS work with exports until it's officially supported (see this Stack Overflow post, this hackish solution and this ticket).
The package.json ends up looking like this:
{
"name": "#vulcanjs/graphql",
"version": "0.4.7",
"main": "./dist/index.js",
"files": [
"dist/"
],
"exports": {
".": "./dist/index.js",
"./server": "./dist/server/index.js",
"./testing": "./dist/testing.js"
},
"types": "./dist/index.d.ts",
"typesVersions": {
"*": {
"server": [
"./dist/server/index.d.ts"
],
"testing": [
"./dist/testing.d.ts"
]
}
},
"description": "Vulcan graphQL schema generator",
...
You can then import using either #vulcanjs/graphql for shared code, and #vulcanjs/graphql/server for Node.js-only code.
It works perfect in my Next.js app. However, it seems to break Jest moduleNameMapper.
First, I had this:
moduleNameMapper: {
"#vulcanjs/(.*)": [
"<rootDir>/node_modules/#vulcanjs/$1",
],
},
The error is:
Configuration error:
Could not locate module #vulcanjs/graphql/server mapped as:
[
"/code/vulcan-next/node_modules/#vulcanjs/graphql/server",
].
The problem is that it tries to find a package named #vulcanjs/graphql/server: yet "server" is not a different package, it's an entrypoint of #vulcanjs/graphql.
I've also tried this:
moduleNameMapper: {
"#vulcanjs/(.*)/(.*)": [
"<rootDir>/node_modules/#vulcanjs/$1",
],
},
With this config, #vulcanjs/graphql/server is mapped to #vulcanjs/graphql. The problem is that now, server code is not found. I've checked and the problem is that this solution totally removes the /server: so #vulcanjs/graphql/server points to the main entrypoints instead of the server entrypoint.
Finally I did try to remove the moduleNameMapper, but then #vulcanjs/graphql/server package is not found by Jest. Note that I need the mapper for a use case I did not demonstrate here, so getting rid of it is possible but not the best solution.
The bug can be reproduced by installing the framework: https://github.com/VulcanJS/vulcan-next. You can clone, yarn install, unskip the test in src/models/tests/sampleModel.server.test.ts and run yarn run test:unit sampleModel. It will show the error.
Any idea how I could fix this?

Why does ESLint throw an error while using export/import statement on Node.js 12.13.0?

I have a project built on Node.js, DynamoDB and other AWS services, deployed on Serverless architecture. I also have the ESLint package installed.
I am getting the following error:
ESLint: Import and export declarations are not supported yet on Node 8.0.0. (node/no-unsupported-features)
Following are the details of my project:
Node version: 12.13.0
NPM version: 6.12.0
Serverless version: 1.40.0
ESLint: 6.8.0
I have double verified by node version of my project and my local. Both are the same (12.13.0).
I am able to use async/await but whenever I try using import/export, it gives me the error.
Following is my .eslintrc file :
{
"extends" : [
"eslint:recommended",
"plugin:node/recommended"
],
"plugins": [
"promise",
"node"
],
"env" : {
"browser" : false,
"node": true,
"es6": true
},
"parserOptions": {
"ecmaVersion": 9,
"sourceType": "module",
"impliedStrict": false
},
"globals" : {
},
"rules": {
"no-console": 0,
"no-unused-vars": ["error", { "vars": "all", "args": "none", "ignoreRestSiblings": false }],
"node/no-unpublished-require": [
"error",
{
"allowModules": [
"aws-sdk"
]
}
],
"node/no-unsupported-features": ["error", {
"version": 8,
"ignores": []
}]
}
}
`
Node.js by default uses Common.js style for import/export.
In the new versions of Node.js you can use different extensions or the --experimental-modules option in the command line.
*.mjs you use ES6 import/export
*.js or *.cjs you use commonjs
If you consider that ECMAScript modules import/export are still experimental, I would say that ESLint is quite correct.
You might want to override EsLint rules in order to make it work with that.
Firstly consider that the rule you mentioned is obsolete so you might want to use the new ones => https://github.com/mysticatea/eslint-plugin-node/blob/master/docs/rules/no-unsupported-features.md
Because those rules are obsolete I would suggest you to take a look at your eslint and its external rulesets version (if you use them) and try again.
In any case...
Check/create the .eslintrc.json file in the root of your project and override the project rules you might want to change.
Just stumpled upon a similiar error.
In my case i solved it by adding the engines definition to package.json
"engines": {
"node": ">=12.13.0"
}
I am not sure if this solves you problem but a quite similar issue can be found here.

Sharing code between Firebase Functions and React

I'm using Firebase functions with a React application. I have some non-trivial code that I don't want to duplicate, so I want to share it between the deployed functions and my React client. I've got this working locally in my React client (though I haven't tried deploying) - but I can't deploy my functions.
The first thing I tried was npm link. This worked locally, but the functions won't deploy (which makes sense, since this leaves no dependency in your package.json). Then I tried npm install ../shared/ - this looked promising because it did leave a dependency in package.json with a file: prefix - but Firebase still won't deploy with this (error below).
My project directory structure looks like this:
/ProjectDir
firebase.json
package.json (for the react app)
/src
* (react source files)
/functions
package.json (for firebase functions)
index.js
/shared
package.json (for the shared module)
index.js
My shared module package.json (extraneous details omitted):
{
"name": "myshared",
"scripts": {
},
"dependencies": {
},
"devDependencies": {
},
"engines": {
"node": "8"
},
"private": true,
"version": "0.0.1"
}
My firebase functions package.json (extraneous details omitted):
{
"name": "functions",
"scripts": {
},
"dependencies": {
"myshared": "file:../shared",
},
"devDependencies": {
},
"engines": {
"node": "8"
},
"private": true
}
When I try to deploy with:
firebase deploy --only functions
It's telling me it can't load the module:
Function failed on loading user code. Error message: Code in file index.js can't be loaded.
Did you list all required modules in the package.json dependencies?
And I don't think the issue is how I export/imported my code- but just in case:
The export:
exports.myFunc = () => { some code };
The import (functions/index.js)
const myFunc = require('myshared');
And in my react code:
import { myFunc } from 'myshared';
So far the searching I've done hasn't yielded anything that works. Someone did mention entering the shared module path in firebase.json, but I couldn't find any details (including in the firebase docs) that show what that would look like. Thanks for any tips to get this going.
I found a solution. I'm not sure if it's the only or even the best solution, but it seems to work for this scenario, and is easy. As Doug noted above, Firebase doesn't want to upload anything not in the functions directory. The solution was to simply make my shared module a subdirectory under functions (ie ./functions/shared/index.js). I can then import into my functions like a normal js file. However, my shared folder also has a package.json, for use as a dependency to the react app. I install it using:
npm install ./functions/shared
This creates a dependency in my react app, which seems to resolve correctly. I've created a production build without errors. I haven't deployed the react app yet, but I don't think this would be an issue.
Another solution is to create a symlink. In terminal, under /ProjectDir, execute:
ln -s shared functions/shared
cd functions
npm i ./shared

webpack import error with node-postgres ('pg'.Client)

Trying to bundle the following file with Webpack fails with
ERROR in ./~/pg/lib/native/index.js Module not found: Error: Cannot
resolve module 'pg-native' in
.../node_modules/pg/lib/native
# ./~/pg/lib/native/index.js 9:13-33
I tried several ignore statements in the .babelrc but didnt get it running...
The test-file i want to bundle: handler.js
const Client = require('pg').Client;
console.log("done");
webpack.config.js
module.exports = {
entry: './handler.js',
target: 'node',
module: {
loaders: [{
test: /\.js$/,
loaders: ['babel'],
include: __dirname,
exclude: /node_modules/,
}]
}
};
.babelrc
{
"plugins": ["transform-runtime"],
"presets": ["es2015", "stage-1"]
}
package.json
"dependencies": {
"postgraphql": "^2.4.0",
"babel-runtime": "6.11.6"
},
"devDependencies": {
"babel-core": "^6.13.2",
"babel-loader": "^6.2.4",
"babel-plugin-transform-runtime": "^6.12.0",
"babel-preset-es2015": "^6.13.2",
"babel-preset-stage-0": "^6.5.0",
"babel-polyfill": "6.13.0",
"serverless-webpack": "^1.0.0-rc.3",
"webpack": "^1.13.1"
}
Somewhat related github-issues:
https://github.com/brianc/node-postgres/issues/1187
https://github.com/serverless/serverless-runtime-babel/issues/8
This is indeed an old thread, but one that helped me nonetheless.
The solution provided by Steve Schafer 1 is good, but not the simplest.
Instead, the one provided by Marco Lüthy 2 in the linked issue is probably the easiest to set up because it is pure configuration, without even the need for a dummy file to be created.
It consists of modifying your Webpack config plugins array as follows:
const webpack = require('webpack');
const webpackConfig = {
...
resolve: { ... },
plugins: [
new webpack.IgnorePlugin(/^pg-native$/)
// Or, for WebPack 4+:
new webpack.IgnorePlugin({ resourceRegExp: /^pg-native$/ })
],
output: { ... },
...
}
Updated to include a change suggested in the comments.
This is an old thread but the problem still exists, so for anyone experiencing it, there is a workaround. The problem is an interaction between the way that node-postgres is written and how babel rewrites the code, which forces pg-native to be loaded even when you don't explicitly import/require it.
The simplest workaround is to add a couple of aliases to your webpack.config.js to cause it to link in a dummy do-nothing file instead:
{
...
resolve: {
alias: {
...
'pg-native': path-to-dummy-js-file,
'dns': path-to-dummy-js-file
}
}
...
}
where the dummy file contains a single line:
export default null
See https://github.com/brianc/node-postgres/issues/838 for further discussion and alternative workarounds.
I know that this is an old topic but I'm compelled to share how I solved it. It was maddening to deal with.
So, here is the readers digest version as based on the recollection from the last brain cell that I have.
Error:
Webpack Compilation Error ./node_modules/pg/lib/native/client.js Module not found: Error: Can't resolve 'pg-native'
The error above was thrown when attempting to run a Cypress test that required the npm package 'pg'.
Attempting to install the pg-native package was not successful and resulted in another error; namely ->
Call to 'pg_config --libdir' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
I found that executing pg_config --libdir in the VSCode cmd prompt resulted in that command failing.
However, I knew that it should work since running that command from the system command prompt resulted in this -> C:/PROGRA~1/POSTGR~1/9.3/lib
That is the path that contains a required dll.
So, instead of running npm install from the VSCode command prompt, I ran it from the command prompt as launched from windows.
The result...success!!! pg-native was installed successfully.
After, the Cypress test was able to run as well.
Errors in now way helped me to arrive at this solution. It was more just checking that things were installed that were required, etc.
You may have pg-native globally installed locally. Hence the packet manager does not include the pg-native in the lock file. That was a issue i experienced where it did run fine locally but every time i build in the cloud webpack complained about pg-native missing. I solved it by removing the lockfile in the files pushed to the cloud (In this case seed.run).

Resources