I have a node-js API that's part of an nx workspace and I'm trying to use the core crypto node-js module, however I think it's getting stripped out by webpack.
My code uses the following:
crypto.getRandomValues(new Uint32Array(1))[0].toString(16)
However I get a runtime error:
ReferenceError: crypto is not defined
Am I right in thinking this issue is likely being caused by webpack? And if so, how can I use crypto in my API?
I'm very new to webpack, so I'm pretty clueless here. My nx project uses the #nrwl/node:webpack executor (the default for node-js APIs), and I've followed the steps here on how to customize this projects webpack configuration, so my custom webpack config is being used, I just dont know what to do with it on order to allow my API to use crypto.
Any help on this would be appreciated.
For completeness, here's what my custom webpack config looks like, but it makes no difference to the problem. Note that his was crafted based on some of the suggested answers here:
const { merge } = require('webpack-merge');
/** #type { import('webpack').Configuration } */
const webpackConfig = {
target: 'node',
resolve: {
fallback: {
"crypto": false
}
}
};
module.exports = (config, context) => merge(config, webpackConfig);
Edit: As a workaround, I've imported the randomUUID function from crypto and can use this to do what I need to do. It seems having this import means crypto gets included in the webpack.
import { randomUUID } from 'crypto';
randomUUID().split('-')[0];
In your above code + runtime error, it seems like you haven't imported crypto correctly.
Furthermore you are mixing up some concepts. Webpack is normally used for frontend projects, as Node.js supports requiring modules out of the box and you don't need to compile your code, as you don't need to generate a javascript bundle that's servable by browsers.
Because your project runs in the browser, but your tests run in Node, which has a different global namespace, you get issues. In the browser crypto is defined globally, but in Node you need to import the crypto standard library, the "suggested answers" you are linking to talks about this concept.
To fill in the gap between browser and Node, to ensure the code you write works for test + browser, you use polyfills.
In your usecase you have a Node.js project. Crypto is a standard library for Node.js and you can just require it, why should you configure webpack to create fallbacks for it or use polyfills?
Related
Stripe recently added typings to their stripe-node library with version 8. This replaced the need for the separate #types/stripe from DefinitelyTyped. In a node TypeScript environment using these types is straightforward, but I'm wondering if it's feasible to make use of these types in a client-side TypeScript application (Angular) that compiles to browser JavaScript.
It seems to work to npm install --save stripe and import Stripe from 'stripe' wherever I want to add Stripe typings, but with this being a node library I want to make sure I'm not missing anything obvious. Is there any concern with importing a library designed for node in a client application?
EDIT:
To be clear, I am not asking if it's ok to make full use of stripe-node in my client. That would be a huge security risk. Now that the library has typings built-in, it would be nice if we could reference those types in client-side typescript.
The types for stripe-node won't work for Stripe.js, you'll want to use these instead: https://github.com/stripe/stripe-js
npm install #stripe/stripe-js
See this bit about TypeScript support: https://github.com/stripe/stripe-js#typescript-support
Concerns would be there, only if you use your server secret in your frontend code. As far as you are not doing that & using server library just for the type definitions in frontend, There is no issue at all.
I use a monorepo package to export the TypeScript interfaces from the stripe-node library.
project
└── packages
├── common
└── web
Prevents clashing with Stripe from #stripe/stripe-js
Avoids adding stripe-node to web, which could be used in a bad way (e.g. using secrets)
Allows extending types when stripe API calls are made with expand
// project/packages/common/stripe.ts
import { Stripe } from "stripe";
type StripeInvoice = Stripe.Invoice;
type StripeSubscription = Stripe.Subscription;
type StripeSubscriptionList = Stripe.Response<
Stripe.ApiList<Stripe.Subscription>
>;
type StripeCustomer = Stripe.Customer;
export {
StripeInvoice,
StripeCustomer,
StripeSubscription,
StripeSubscriptionList,
};
// project/packages/web/request.ts
import { StripeSubscriptionList } from "#project/common/stripe";
const listSubscriptions = async () => {
// Wrapper function for the fetch API
return await fetchAPI<StripeSubscriptionList>("subscriptions", {
method: "GET",
});
};
I have a universal React project configured in which I use Webpack to create bundles for both the client and the server code.
This works fine when I run the server directly via the targetting the bundled output by Webpack.
node ./build/server/main.js
However I am having issues though when I try to run a "live" development server. In this case I don't want to target the bundled server files, instead I just target the source files directly which will allow me to run the webpack hot middleware for live code changes. Below is a stripped down version of the main file for the server.
src/server/index.js
import express from 'express'
import universalReactAppMiddleware from './middleware/universalReactApp'
const server = express()
// Get the client bundle webpack configuration.
const webpackClientConfig = require('../../webpack.client.config.js')
// If we are in development mode we will add the webpack hot
// reloading middleware.
if (process.env.NODE_ENV === 'development') {
const webpack = require('webpack')
const clientCompiler = webpack(webpackClientConfig)
const createWebpackMiddleware = require('webpack-dev-middleware')
const createWebpackHotMiddleware = require('webpack-hot-middleware')
server.use(
createWebpackMiddleware(clientCompiler, {
noInfo: true,
publicPath: webpackClientConfig.output.publicPath,
stats: {
colors: true,
hash: false,
timings: true,
chunks: false,
chunkModules: true,
modules: false
}
})
)
server.use(
createWebpackHotMiddleware(clientCompiler)
)
}
// Configure static serving of our webpack bundled client files.
server.use(
webpackClientConfig.output.publicPath,
express.static(webpackClientConfig.output.path))
// Bind our universal react app middleware for all GET requests.
server.get('*', universalReactAppMiddleware)
server.listen(process.env.SERVER_PORT)
An example execution of this now being:
NODE_ENV=development babel-node ./src/server
It starts up okay, but the moment the universalReactAppMiddleware handles a request it will attempt to perform server rendering for a resolved Component. This then falls over because some of my components import images/css, for example:
src/shared/components/Foo
import './styles.css'
import background from './background.jpg'
function FooComponent() {
return <img src={background} />
}
Computer says no!
Upon execution my express server throws out an exception saying unexpected syntax. It basically tries to parse the css and image imports as Javascript. These types of imports depend on my webpack loaders to operate correctly.
So now I am trying to look for a mechanism of spoofing the Webpack behaviour so that I can execute these types of components. I am investigating Pete Hunts webpack-require but have been having difficulty with it.
Does anyone know of any other approaches that will work for this context?
Update 2016/06/15
Boom! I've managed to pull this off without any 3rd party libs to help me. universal-webpack is pretty cool and much cleaner than the previous webpack-isomorphic-tools but I am liking that I have a minimal configuration set up in which as little as possible of the universal webpack configuration bleeds into my production code.
I'm pretty stoked with the results. Client bundle is backed by the lastest react-hot-loader v3 beta which is giving me an awesome HMR experience, and my Server bundle gets rebuilt on any file changes so not having to restart my server manually either. Making for a pretty sweet development experience.
I am going to throw this into a boilerplate (yes I know, yet another) but perhaps it will be useful to someone else.
I created a working solution achieving everything that I wanted from the configuration.
It's all within the following boilerplate repo:
https://github.com/ctrlplusb/react-universally
I was just wondering, I started using Webpack for a new project and so far it's working fine. I almost would say I like it better than Grunt, which I used before. But now I'm quite confused how and or I should use it with my Express back-end?
See, I'm creating one app with a front-end (ReactJS) and a back-end (ExpressJS). The app will be published on Heroku. Now it seems like I should use Webpack with ExpressJS as well to get the app up and running with one single command (front-end and back-end).
But the guy who wrote this blogpost http://jlongster.com/Backend-Apps-with-Webpack--Part-I seems to use Webpack for bundling all back-end js files together, which is in my opinion really not necessary. Why should I bundle my back-end files? I think I just want to run the back-end, watch my back-end files for changes and use the rest of Webpack's power just for the front-end.
How do you guys bundle the front-end but at the same time run the back-end nodejs part? Or is there any good reason to bundle back-end files with Webpack?
Why to use webpack on node backend
If we are talking about react and node app you can build isomorphic react app. And if you are using import ES6 Modules in react app on client side it's ok - they are bundled by webpack on the client side.
But the problem is on server when you are using the same react modules since node doesn't support ES6 Modules. You can use require('babel/register'); in node server side but it transipile code in runtime - it's not effective. The most common way to solve this problem is pack backend by webpack (you don't need all code to be transpile by webpack - only problematic, like react stuff in this example).
The same goes with JSX.
Bundling frontend and backend at the same time
Your webpack config can have to configs in array: one for frontend and second for backend:
webpack.config.js
const common = {
module: {
loaders: [ /* common loaders */ ]
},
plugins: [ /* common plugins */ ],
resolve: {
extensions: ['', '.js', '.jsx'] // common extensions
}
// other plugins, postcss config etc. common for frontend and backend
};
const frontend = {
entry: [
'frontend.js'
],
output: {
filename: 'frontend-output.js'
}
// other loaders, plugins etc. specific for frontend
};
const backend = {
entry: [
'backend.js'
],
output: {
filename: 'backend-output.js'
},
target: 'node',
externals: // specify for example node_modules to be not bundled
// other loaders, plugins etc. specific for backend
};
module.exports = [
Object.assign({} , common, frontend),
Object.assign({} , common, backend)
];
If you start this config with webpack --watch it will in parallel build your two files. When you edit frontend specific code only frontend-output.js will be generated, the same is for backend-output.js. The best part is when you edit isomorphic react part - webpack will build both files at once.
You can find in this tutorial explanation when to use webpack for node (in chapter 4).
This is my second update to this answer, which is beyond outdated by now.
If you need full a stack web framework in 2023, I'd recommend nextjs (which is built on top of react). No need to go around setting up anything, it just works out of the box, and is full stack.
On the other hand, if you need to compile your nodejs project written in typescript (which you should use as much as you can for js projects), I would use tsup-node.
You don't need to be a genius to imagine that in 3-5 years I'll come back to this and say this is really outdated, welcome to javascript.
This answer is outdated by now since node now has better support for ES modules
There's only a couple of aspects I can redeem the need to use webpack for backend code.
ES modules (import)
import has only experimental support in node (at least since node 8 up to 15). But you don't need to use webpack for them work in node.
Just use esm which is very lightweight and has no build step.
Linting
Just use eslint, no need to use webpack.
Hot reloading/restart
You can use nodemon for this. It's not hot reloading but I think it's way easier to deal with.
I wished I could refer to a more lightweight project than nodemon, but it does do the trick.
The blog post you shared (which is dated by now) uses webpack for having hot reloading. I think that's an overkill, opens a can of worms because now you have to deal with webpack config issues and hot reloading can also lead to unexpected behaviour.
The benefits we get from using tools like webpack on the frontend don't really translate to backend.
The other reasons why we bundle files in frontend is so browsers can download them in an optimal way, in optimal chunks, with cache busting, minified. There's no need for any of these in the backend.
Old (and terrible, but maybe useful) answer
You can use webpack-node-externals, from the readme:
npm install webpack-node-externals --save-dev
In your webpack.config.js:
var nodeExternals = require('webpack-node-externals');
module.exports = {
...
target: 'node', // in order to ignore built-in modules like path, fs, etc.
externals: [nodeExternals()], // in order to ignore all modules in node_modules folder
...
};
to use Webpack for bundling all back-end js files together, which is in my opinion really not necessary.
I think you are absolutely right. It's not necessary at all. I've been researching on this topic for a while. I've asked lots of questions on this topic, and up to this day, I haven't found yet a single "real" reason for one to use webpack on a Node.js back-end.
I'm not saying you can't or shouldn't set up a webpack-dev-server to develop your back-end code locally. But you definitely don't need to bundle your backend code on your build process.
webpack bundles are meant for the browser. Take a look at its official docs: Why webpack?. Historically, browsers never had a built-in module system, that's the reason why you need webpack. It basically implements a module system on the browser. On the other hand, Node.js has a built-it module system out of the box.
And I do re-use all of my front-end code for SSR on my server. The very same front-end files are run on my server as-is without any bundling (they are just transpiled, but the folder structured and number of files is the same). Of course I bundle it to run on the browser, but that's all.
To run on your Node.js server, simply transpile it with babel, without webpack.
Just use ["#babel/preset-env", { targets: { node: "12" }}], on your babel config. Choose the Node version of your backend environment.
backend
dist_app // BABEL TRANSPILED CODE FROM frontend/src
dist_service // BABEL TRANSPILED CODE FROM backend/src
src
index.tsx
frontend
src
App.tsx
public // WEBPACK BUNDLED CODE FROM frontend/src
You can perfectly render your frontend code on the server, by doing:
backend/src/index.tsx
import { renderToString } from "react-dom/server";
import App from "../dist_app/App";
const html = renderToString(<App/>);
This would be considered isomorphic, I guess.
If you use path aliases on your code, you should use babel-plugin-module-resolver.
I am trying to build a web app where I want to store all html, js and css files on amazon s3, and communicate with a restful server through api.
I am trying to achieve lazy loading and maybe routing with react router. It seems that webpack has this feature code splitting that would work similarly as lazy loading.
However, all of the tutorial and examples I found involves webpack-dev-server, which is a small node express server. Is there anyway I could generate bundle at build time and upload everything to amazon s3 and achieve something similar to Angular's ocLazyLoading?
It's definitely possible to create a static bundle js file, which you can use in your production code that does not include webpack-dev-server.
See this example as a reference (note: I am the owner of this repo). webpack.prod.config.js does create a production ready bundle file using webpack via node.js which itself does not require node.js anymore. Because of that you can simply serve it as a simple static file (which is done in the live example).
The key difference is how the entry points are written in the dev- and production environments. For development webpack-dev-server is being used
module.exports = {
entry: [
'webpack-dev-server/client?http://localhost:3000',
'webpack/hot/only-dev-server',
'./src/index'
],
// ...
}
In the production environment you skip the webpack-dev-server and the hot reloading part
module.exports = {
entry: [
'./src/index'
],
// ...
}
If you want to split your code into more than one bundle, you might want to have a look at how to define multiple entry points and link the files accordingly.
I've managed to properly use webpack dev server alongside with a node server (express), using the plugin section inside webpack's config.
It all works fine but now I'm trying to go isomorphic and use client-side components inside the express application.
So far the only problem I'm encountering is that without webpack 'parsing' my server-side code I get to a situation where I require components but the paths are not solved
I.E.
Inside a component
'use strict';
import React from 'react';
import { RouteHandler, Link } from 'react-router';
import Header from 'components/header/main'; // <-- This line causes the error because webpack is not working when parsing this JSX server-side
export default React.createClass({
displayName: 'App',
render() {
return ( // ... More code
Shall I configure webpack in another way or do I have to change all the imports to be valid server-side?
the codebase is here in case you want to see the actual state https://github.com/vshjxyz/es6-react-flux-node-quickstart
In order to be able to require components in a way such as require('components/Header.js'); and avoid using long relative paths such as require('../../../../../../Header.js'); you can add this code to your node app before any require() calls:
process.env.NODE_PATH = __dirname;
require('module').Module._initPaths();
However, since this relies on a private Node.js core method, this is
also a hack that might stop working on the previous or next version of
node.
Other possible solutions to this problem can be found at https://gist.github.com/branneman/8048520
I see 2 options:
Compile client code with webpack as well. If client's entry
point is in the same dir as server's - it should work with your
present code. This looks natural to me.
Use relative paths i.e.
import Header from './components/header/main'