webpack import error with node-postgres ('pg'.Client) - node.js

Trying to bundle the following file with Webpack fails with
ERROR in ./~/pg/lib/native/index.js Module not found: Error: Cannot
resolve module 'pg-native' in
.../node_modules/pg/lib/native
# ./~/pg/lib/native/index.js 9:13-33
I tried several ignore statements in the .babelrc but didnt get it running...
The test-file i want to bundle: handler.js
const Client = require('pg').Client;
console.log("done");
webpack.config.js
module.exports = {
entry: './handler.js',
target: 'node',
module: {
loaders: [{
test: /\.js$/,
loaders: ['babel'],
include: __dirname,
exclude: /node_modules/,
}]
}
};
.babelrc
{
"plugins": ["transform-runtime"],
"presets": ["es2015", "stage-1"]
}
package.json
"dependencies": {
"postgraphql": "^2.4.0",
"babel-runtime": "6.11.6"
},
"devDependencies": {
"babel-core": "^6.13.2",
"babel-loader": "^6.2.4",
"babel-plugin-transform-runtime": "^6.12.0",
"babel-preset-es2015": "^6.13.2",
"babel-preset-stage-0": "^6.5.0",
"babel-polyfill": "6.13.0",
"serverless-webpack": "^1.0.0-rc.3",
"webpack": "^1.13.1"
}
Somewhat related github-issues:
https://github.com/brianc/node-postgres/issues/1187
https://github.com/serverless/serverless-runtime-babel/issues/8

This is indeed an old thread, but one that helped me nonetheless.
The solution provided by Steve Schafer 1 is good, but not the simplest.
Instead, the one provided by Marco Lüthy 2 in the linked issue is probably the easiest to set up because it is pure configuration, without even the need for a dummy file to be created.
It consists of modifying your Webpack config plugins array as follows:
const webpack = require('webpack');
const webpackConfig = {
...
resolve: { ... },
plugins: [
new webpack.IgnorePlugin(/^pg-native$/)
// Or, for WebPack 4+:
new webpack.IgnorePlugin({ resourceRegExp: /^pg-native$/ })
],
output: { ... },
...
}
Updated to include a change suggested in the comments.

This is an old thread but the problem still exists, so for anyone experiencing it, there is a workaround. The problem is an interaction between the way that node-postgres is written and how babel rewrites the code, which forces pg-native to be loaded even when you don't explicitly import/require it.
The simplest workaround is to add a couple of aliases to your webpack.config.js to cause it to link in a dummy do-nothing file instead:
{
...
resolve: {
alias: {
...
'pg-native': path-to-dummy-js-file,
'dns': path-to-dummy-js-file
}
}
...
}
where the dummy file contains a single line:
export default null
See https://github.com/brianc/node-postgres/issues/838 for further discussion and alternative workarounds.

I know that this is an old topic but I'm compelled to share how I solved it. It was maddening to deal with.
So, here is the readers digest version as based on the recollection from the last brain cell that I have.
Error:
Webpack Compilation Error ./node_modules/pg/lib/native/client.js Module not found: Error: Can't resolve 'pg-native'
The error above was thrown when attempting to run a Cypress test that required the npm package 'pg'.
Attempting to install the pg-native package was not successful and resulted in another error; namely ->
Call to 'pg_config --libdir' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
I found that executing pg_config --libdir in the VSCode cmd prompt resulted in that command failing.
However, I knew that it should work since running that command from the system command prompt resulted in this -> C:/PROGRA~1/POSTGR~1/9.3/lib
That is the path that contains a required dll.
So, instead of running npm install from the VSCode command prompt, I ran it from the command prompt as launched from windows.
The result...success!!! pg-native was installed successfully.
After, the Cypress test was able to run as well.
Errors in now way helped me to arrive at this solution. It was more just checking that things were installed that were required, etc.

You may have pg-native globally installed locally. Hence the packet manager does not include the pg-native in the lock file. That was a issue i experienced where it did run fine locally but every time i build in the cloud webpack complained about pg-native missing. I solved it by removing the lockfile in the files pushed to the cloud (In this case seed.run).

Related

How to bundle node module CSS into a vscode extension

My Visual Studio Code extension uses the node module highlight.js which comes with a folder full of CSS files. These provide colour schemes for syntax colouring. It has become necessary to bundle some of the CSS files.
It's about bundling an asset
The objective is to bundle a CSS file and at run-time access the file content as a string. If that can be achieved without an import statement that would be perfect. Normally, how exactly one accesses the content of the bundled file would be a separate question, but I have a feeling that content retrieval and how one should go about bundling the asset are closely entwined.
I freely admit to having a weak understanding of WebPack.
The story so far
The bundler is specified in package.json as "webpack": "^5.4.0" but I don't know how to ascertain what is actually present. It is conceivable that there is something wrong with my setup: when I try to run webpack --version on a command prompt in the project folder, it responds
CLI for webpack must be installed.
webpack-cli (https://github.com/webpack/webpack-cli)
We will use "npm" to install the CLI via "npm install -D webpack-cli".
Do you want to install 'webpack-cli' (yes/no):
The first time this happened I responded yes. After a flurry of installation and another try the same thing happened. However, vsce package has no trouble using webpack for a production build and pressing F5 to debug successfully puts together a development build in a dist folder with an unminified file I can examine (which is how I know the file mentioned below is being bundled).
Moving on from there I've modified webpack.config.js like so
//#ts-check
'use strict';
const path = require('path');
/**#type {import('webpack').Configuration}*/
const config = {
target: 'node', // vscode extensions run in a Node.js-context -> https://webpack.js.org/configuration/node/
entry: './src/extension.ts', // the entry point of this extension, -> https://webpack.js.org/configuration/entry-context/
output: {
// the bundle is stored in the 'dist' folder (check package.json), -> https://webpack.js.org/configuration/output/
path: path.resolve(__dirname, 'dist'),
filename: 'extension.js',
libraryTarget: 'commonjs2',
devtoolModuleFilenameTemplate: '../[resource-path]'
},
devtool: 'source-map',
externals: {
vscode: 'commonjs vscode' // the vscode-module is created on-the-fly and must be excluded. Add other modules that cannot be webpack'ed, -> https://webpack.js.org/configuration/externals/
},
resolve: {
// support reading TypeScript and JavaScript files, -> https://github.com/TypeStrong/ts-loader
extensions: ['.ts', '.js', '.css']
},
module: {
rules: [
{
test: /\.ts$/,
exclude: /node_modules/,
use: [
{
loader: 'ts-loader'
}
]
},
{
test: /\.css$/,
use: [
'style-loader',
'css-loader'
]
}
]
}
};
module.exports = config;
As you can see there are rules and loaders for CSS.
When I add this import
import "../node_modules/highlight.js/styles/atelier-dune-light.css";
webpack happily builds the bundle and when I inspect it I can find the bundled CSS.
However, when I try to load the extension in the extension debug host, it fails to load, with this message.
Activating extension 'pdconsec.vscode-print' failed: document is not defined.
Enabling break on caught exceptions reveals this rather surprising exception.
Exception has occurred: Error: Cannot find module 'supports-color'
Require stack:
- c:\Users\Peter\AppData\Local\Programs\Microsoft VS Code\resources\app\node_modules.asar\get-uri\node_modules\debug\src\node.js
- c:\Users\Peter\AppData\Local\Programs\Microsoft VS Code\resources\app\node_modules.asar\get-uri\node_modules\debug\src\index.js
- c:\Users\Peter\AppData\Local\Programs\Microsoft VS Code\resources\app\node_modules.asar\get-uri\dist\index.js
- c:\Users\Peter\AppData\Local\Programs\Microsoft VS Code\resources\app\node_modules.asar\vscode-proxy-agent\out\agent.js
- c:\Users\Peter\AppData\Local\Programs\Microsoft VS Code\resources\app\node_modules.asar\vscode-proxy-agent\out\index.js
- c:\Users\Peter\AppData\Local\Programs\Microsoft VS Code\resources\app\out\bootstrap-amd.js
- c:\Users\Peter\AppData\Local\Programs\Microsoft VS Code\resources\app\out\bootstrap-fork.js
OK, so activation failed because the loader barfed. But WTF does importing CSS have to do with support-color?
Remove the import and it runs just fine. I really don't know how to respond to this; it's not clear to me why a demand for a stylesheet should cause that error. At this point I look to others for guidance and advice.
Remove style-loader from webpack.config.js to fix the error.
Pull the CSS as a string like this. Note the abbreviated path.
const cssPath: string = "highlight.js/styles/atelier-dune-light.css";
const theCss: string = require(cssPath).default.toString();
You do not need the import statement, the use of require will cause Webpack to bundle the files, but you still have to remove style-loader to avoid the loader error.

(Nuxt)npm run dev have wrong,node_modules error

First of all
Please forgive me for not good English, I am using google translation.
Recently just got to work, I want to turn the code in the company into localhost, which is more convenient to develop.
I don't know what happened.
I tried removing node_modules and reinstalling.I also tried npm rebuild node-sass, I still don't know where there is a problem.
Failed to compile with 19 errors:
These relative modules were not found:
* ../../../../../../../../../assets/img/icon/clock.svg in
./node_modules/css-loader/dist/cjs.js??ref--7-oneOf-1-1!./node_modules/vue-
loader/lib/loaders/stylePostLoader.js!./node_modules/postcss-loader/src??
ref--7-oneOf-1-2!./node_modules/sass-loader/dist/cjs.js??ref--7-oneOf-1-
3!./node_modules/vue-loader/lib??vue-loader-
options!./pages/_catName/vendorList/_location_id/index.vue?
vue&type=style&index=0&id=2e9c7704&lang=scss&scoped=true&
...
...
And so on
Error message
my package.josn(1)
my package.josn(2)
my package.josn(3)
The problem is not clear but I guess you need to install css-loader first before using style-loader. Try :
npm install --save-dev style-loader
after that use style-loader and css-loader together in your config.js file. For example:
module.exports = {
module: {
rules: [
{
test: /\.css$/i,
use: ['style-loader', 'css-loader'],
},
],
},
};

async / await breaks my Webpack build for AWS Lambda; how can I migrate to Node 8.10?

Note: this is a Q&A on migrating AWS Lambda Webpack builds from v6.10 to v8.10 — no help is needed, but even better answers are of course always encouraged!
For a while now I have been a disciple of using Webpack to build my back-end Lambdas after reading James Long's excellent series entitled "Backend Apps with Webpack" (part 1, part2, and part3).
Up until recently, the only version of Node.js that Amazon Web Services offered was 6.10; you had to write your Lambda fn in the "callback" style. But on April 2, 2018 AWS announced that 8.10 was now supported, and with it the suggested pattern is async / await which is great! Until it immediately broke my Webpack build. After a bit of debugging, I can break my build just by adding one async fn to the Lambda handler (I don't even need to call it):
async function firstAsync() {
return true;
}
exports.handler = async (event) => {
// TODO implement
return 'Hello from Lambda!'
};
To be clear, doing this in the AWS Lambda console is perfectly fine, runs great. Webpack even builds it successfully, but upon uploading to AWS Lambda, I get the following error message: regeneratorRuntime is not defined. My code is below. What am I needing to do?
webpack.config.js
const nodeExternals = require('webpack-node-externals');
const path = require('path');
const UglifyJsPlugin = require('uglifyjs-webpack-plugin')
const webpack = require('webpack');
const config = {
entry: './src/index.js',
output: {
path: path.resolve(__dirname, 'dist'),
library: 'index',
libraryTarget: 'commonjs2',
filename: 'index.js'
},
target: 'node', // building for a Node environment
externals: [nodeExternals()], // in order to ignore all modules in node_modules folder
module: {
rules: [{
test: /\.(js|jsx)$/,
use: {
loader: 'babel-loader',
options: {
presets: ['env']
}
}
}]
},
plugins: [
new UglifyJsPlugin()
]
};
module.exports = config;
package.json
{
"name": "lambda-webpack",
"version": "1.0.0",
"description": "An empty project scaffold to enable webpack builds in AWS Lambda",
"main": "index.js",
"scripts": {
"build": "webpack",
"upload": "upload.bat"
},
"author": "Geek Stocks®",
"license": "MIT",
"devDependencies": {
"aws-sdk": "^2.179.0",
"babel-core": "^6.26.0",
"babel-loader": "^7.1.2",
"babel-preset-env": "^1.6.1",
"uglifyjs-webpack-plugin": "^1.1.6",
"webpack": "^3.10.0",
"webpack-node-externals": "^1.6.0"
}
}
Geek Stock's answer above is only necessary when using the Node v6.10 runtime. It allows you to use async/await syntax and babel-runtime/regenerator converts your async functions to ES6 generators which is supported on Node v4.3.2 and newer runtimes.
In Node v8.10 though, it is different. async/await is already supported so you don't need Babel to convert your async functions to generators. You just use it correctly and it works.
As for your specific situation, I assume you simply changed your Javascript code to use async/await code but you didn't tell Babel NOT to convert your async/await code to generators. That is why babel-runtime is complaining about a missing plugin.
If this is the case, you simply need to configure Babel to simply target Node v8.10. You can do it like this in your .babelrc...
{
"presets": [
[
"env",
{
"targets": {
"node": "8.10"
}
}
]
]
}
babel will then stop converting those async functions. And since it's not doing a conversion, it will not need the regenerator-runtime anymore.
In my case I'm using it with nodejs12.x. I'm receiving the error regeneratorRuntime is not defined.
I was using preset es2015 and stage-0 with a node12.x code. Also I'm using an older version of babel.
What I did, installed:
"#babel/cli": "^7.10.1",
"#babel/core": "^7.10.2",
"#babel/preset-env": "^7.10.2"
And then set this babel preset:
"presets": [
["#babel/preset-env", {"targets": { "node": "current" }}]
]
Thanks,
To begin using async / await in your Webpack-built code, you need a little help from Babel, specfically the babel-runtime and the babel-plugin-transform-runtime. Fortunately there is a really good writeup on the installation and use of both here on the Babel website. You need Babel to do the following for you:
Automatically requires babel-runtime/regenerator when you use
generators/async functions.
I won't repeat what has been written there, however, of particular note is HOW you need to install these, because while their writeup certainly works for most, there is a needed tweak for some AWS Lambda devs who have not needed runtime dependencies before now.
The first part is pretty normal, you need new devDependencies:
npm install --save-dev babel-plugin-transform-runtime
And also you need to tweak your .babelrc file like they describe:
{
"plugins": [
["transform-runtime", {
"helpers": false,
"polyfill": false,
"regenerator": true,
"moduleName": "babel-runtime"
}]
]
}
But here is the kicker, and this is new to the code above: the babel-runtime is a regular dependencies, not a devDependencies:
npm install --save babel-runtime
These 2 installs and the .babelrc tweaks do SOLVE the async / await problem described above, but it introduces a new problem to the build above: Cannot find module 'babel-runtime/regenerator'.
If you are at all concerned with keeping your Webpack built code small, you are likely also using webpack-node-externals like the above code does. For example, the aws-sdk for JavaScript in Node is very large and is already available in the Lambda environment, so its superfluous to bundle it again. The webpack-node-externals configuration above configures Webpack to ignore ALL the modules in node-modules by default, and there is the root of the new problem. The newly installed babel-runtime is a module that needs to be bundled in the build in order for Lambda to run correctly.
Understanding the problem then, the answer becomes simple: don't use the default configuration. Instead of passing in nothing to webpack-node-externals, configure it like this:
externals: [nodeExternals({
whitelist: [
'babel-runtime/regenerator',
'regenerator-runtime'
]
})], // ignore all modules in node_modules folder EXCEPT the whitelist
And that solves both the original async / await problem, and the (possible) new problem you may be encountering if you had no previous dependencies in your build. Hope that helps — happy Lambda awaiting!

Warnings when building backend Express/WS Node app with Webpack

I am getting some confusing warnings when building a backend Node server with Webpack. I want to use Webpack to build my backend primarily for two reasons:
Webpack creates a single executable file, which is easier to deploy
Webpack includes all of my app's dependencies, so I can deploy my app to any compatible Node environment without needing to install dependencies first
Here are the warnings I'm getting:
WARNING in ./~/ws/lib/BufferUtil.js
Module not found: Error: Can't resolve 'bufferutil' in .../node_modules/ws/lib
# ./~/ws/lib/BufferUtil.js 35:21-42
# ./~/ws/lib/Receiver.js
# ./~/ws/index.js
# ./src/main.js
WARNING in ./~/ws/lib/Validation.js
Module not found: Error: Can't resolve 'utf-8-validate' in .../node_modules/ws/lib
# ./~/ws/lib/Validation.js 10:22-47
# ./~/ws/lib/Receiver.js
# ./~/ws/index.js
# ./src/main.js
WARNING in ./~/express/lib/view.js
80:29-41 Critical dependency: the request of a dependency is an expression
For the Critical dependency warning, I've found a good example explaining the problem and some documentation on how to use the ContextReplacementPlugin, although it's still unclear to me how to apply it to this situation. It looks like the warning is being caused by line 80 in node_modules/express/lib/view.js:
opts.engines[this.ext] = require(mod).__express
It is clear to me that the dependency cannot be resolved at build time, so how can I use the ContextReplacementPlugin to fix this dependency?
As for the Module not found warnings in the ws package, it's unclear to me what's going on. It looks like those dependencies exist in my global node_modules, and maybe they're not being pulled in by Webpack. I've tried adding them to my project's devDependencies, but then I just get Critical dependency warnings for them instead.
My application still runs after being built, so I suppose I could technically ignore the warnings, but I figure that these are widely used Node packages and Webpack is a popular build tool, so there must be a reasonable solution available.
Here are my dependencies in my package.json:
"devDependencies": {
"#types/cassandra-driver": "^0.8.10",
"#types/express": "^4.0.35",
"#types/uuid": "^2.0.29",
"#types/ws": "0.0.40",
"nodemon": "^1.11.0",
"typescript": "^2.3.1",
"webpack": "^2.5.1"
},
"dependencies": {
"cassandra-driver": "^3.2.1",
"express": "^4.15.2",
"uuid": "^3.0.1",
"ws": "^2.3.1"
}
And here's my webpack.config.js:
const path = require('path');
module.exports = {
entry: './src/main.js',
output: {
path: path.join(__dirname, 'dist'),
filename: 'main.js'
},
target: 'node',
node: {
__dirname: false,
__filename: false
}
};
I like keeping things minimal if possible. Thanks for reading.
The short answer
webpack can work with node, but it cannot extract follow require() statements. Modifications have to be made to ignore require() in order for it to work.
The long answer
It is actually possible to pull some files into a master file and run it in some instances.
One instance is if all the modules required are written in typescript and the modules are written in a away that the typescript plugin can parse the module in.
Another instance would be if you are using es6 babel plugins and using es6 style imports.
Even in the above scenarios the blunder may choose not to pull in certain files
The ultimate answer
It really should not matter too much about trying to perform the long answer because modules are stored in memory at boot and then referenced later from the cache. See the article below for more information.
http://fredkschott.com/post/2014/06/require-and-the-module-system/

Proper use of karma-commonjs with Jasmine 2

I've spent a fair amount of time trying to debug this, and figured I would ask. I even created a GitHub repository but won't rely on it, so here goes. I'm trying to take advantage of CommonJS syntax within the Karma test runner using PhantomJS. For my module I created the simplest thing I could think of:
exports.returnYes = function() {
return "Yes";
};
The Jasmine test is:
var returnYes = require("../js/returnYes").returnYes;
describe("returnYes", function() {
it("should return Yes", function() {
expect(returnYes()).toBe("Yes");
});
});
And, if I do a jasmine init I can run it from the command line thanks to jasmine-npm by simply typing jasmine with output:
$ jasmine
Started
.
1 spec, 0 failures
Finished in 0.003 seconds
Now to try and get it to work inside karma:
I create my karma.conf.js with frameworks: jasmine,commonjs. And, I add commonjs as preprocessor.
I try to do a karma run and I find that it can't find global which is part of getJasmineRequireObj in jasmine.js where it declares jasmineGlobal = global;
The command line output is a little hard to read, but here it is:
$ karma run
[2015-06-27 17:41:35.266] [DEBUG] config - Loading config /Users/zen/Projects/karma-commonjs-test/karma.conf.js
##teamcity[enteredTheMatrix]
##teamcity[testSuiteStarted nodeId='1' parentNodeId='0' name='karma.conf.js' nodeType='config' locationHint='config:///Users/zen/Projects/karma-commonjs-test/karma.conf.js']
##teamcity[testSuiteStarted nodeId='2' parentNodeId='1' name='PhantomJS 1.9.8 (Mac OS X 0.0.0)' nodeType='browser']
##teamcity[testStarted nodeId='3' parentNodeId='2' name='Error' nodeType='browserError']
##teamcity[testFailed nodeId='3' error='yes' message='ReferenceError: Can|'t find variable: global|nat http://localhost:9876/base/node_modules/jasmine-core/lib/jasmine-core/jasmine.js?68f13ab3f93af5a219b9fe8409f8763b31998bba:27']
##teamcity[testSuiteFinished nodeId='2']
##teamcity[testSuiteFinished nodeId='1']
For good measure here are the devDependencies in my packages.json:
"devDependencies": {
"jasmine-core": "^2.3.4",
"karma": "^0.12.37",
"karma-commonjs": "0.0.13",
"karma-jasmine": "^0.3.5",
"karma-phantomjs-launcher": "^0.2.0",
"phantomjs": "^1.9.17"
}
I'm not sure why I can't find global. Any help would be greatly appreciated!!! :)
It seems like my whole problem came down to the line in karma.conf.js (not shown in my original question:
preprocessors: {
'**/*.js': ['commonjs']
},
For some reason, jasmine.js is not happy being pre-processed by commonjs, and "**/*.js" says to go through all subdirectories (which is probably overkill), including node_modules which has jasmine-core/jasmine.js
So I can either make my pre-processor more specific (best practice):
preprocessors: {
'spec/*.js': ['commonjs'],
'js/*.js': ['commonjs']
},
but as a test to see if any other files would give me a problem, I tried:
preprocessors: {
'**/!(jasmine).js': ['commonjs'],
},
And, everything worked as well. Bottom line. Do not process jasmine.js through commonjs preprocessor!

Resources