Smarter webpack bundling with react and express - node.js

I've got a react app going, and it's run on an express server and is bundling with webpack. My issue is that everytime I restart the server, like when i am making changes to it, it takes forever to rebuild the frontend bundle, even though i don't make any changes to the frontend.
It would be nice to just reload the server portion and leave the current frontend bundle in tact when just making server/api changes that don't involve the front end bundle.
Here is the code that run's in a dev environment:
const compiler = webpack(webpackConfig)
const middleware = webpackMiddleware(compiler, {
publicPath: webpackConfig.output.publicPath,
contentBase: 'src',
stats: {
colors: true,
hash: false,
timings: true,
chunks: false,
chunkModules: false,
modules: false
}
})
app.use(middleware)
app.use(webpackHotMiddleware(compiler))
app.get('*', (req, res) => {
res.write(middleware.fileSystem.readFileSync(path.join(__dirname, 'build/app.html')))
res.end()
})
Is there a smarter way to do this? is it possible to leave the current frontend bundle in memory and just reload the server? Or can I detect if the bundle needs to be updated and skip the process if it doesn't need to be updated?
Any tips, advice and suggestions are welcome! Let me know if you need any other info. Thanks!

Chokidar solution
If using webpack-dev-tools, a great library for watching changes is chokidar
Chokidar does still rely on the Node.js core fs module, but when using
fs.watch and fs.watchFile for watching, it normalizes the events it
receives, often checking for truth by getting file stats and/or dir
contents.
Here is a small example of using chokidar to watch only the targetfolder. By targeting just a specific folder you could leave the frontend intact. I haven't tried this for your specific use case but at first sight it seems that this may suit your needs.
var production = process.env.NODE_ENV === 'production'
if(!production) {
var chokidar = require('chokidar')
var watcher = chokidar.watch('./targetfolder')
watcher.on('ready', function() {
watcher.on('all', function() {
console.log("Clearing /targetfolder/ module cache from server")
Object.keys(require.cache).forEach(function(id) {
if (/[\/\\]targetfolder[\/\\]/.test(id)) delete require.cache[id]
})
})
})
}
There's a great example on Github, called Ultimate Hot Reloading Example
NB: The webpack-dev-server doesn't write files to disk, it serves the result from memory trough an Express instance. But webpack --watch does write files to disk.
Flag solution
You can use webpack's --watch flag.
In your package.json, the script block that starts your server (or the one that runs webpack), add this webpack --progress --colors --watch.
See Webpack documentation, it says:
We don’t want to manually recompile after every change…
When using watch mode, webpack installs file watchers to all files, which were used in the compilation process. If any change is detected, it’ll run the compilation again. When caching is enabled, webpack keeps each module in memory and will reuse it if it isn’t changed.
Example in package.json:
"scripts": {
"dev": "webpack --progress --colors --watch"
}

I have this problem in a SpringBoot app, where i can rebuild my bundle rapidly, but then that wouldn't necessarily make the server look inside an actual folder and find it live in realtime. So really what you need to do is have a way to configure your server to always look in a local folder for the bundle.js file instead of pulling it from the WAR/JAR or wherever it normally pulls it from. It's not "webpack issue". It's an issue of how to make servers read directly from the bundle.js on the folder. I would give you the Spring way of doing it but that's not your architecture.

Related

Laravel Mix not immediately exiting/terminating after compiled success

I have a project that is currently using Laravel Jetstream with Inertia (inertia-vue2). While at first, it was good, but now I have irritated issue that also affects my GitHub actions. When I run, say, yarn dev or yarn prod, then it compiled successfully, it didn't exit immediately unless I press ctrl+c then terminate it. It was fine that I need to do that every time I run yarn dev on my machine (but honestly, I'm not), but this has become the problem on GitHub actions because I can't manually terminate Laravel Mix on the workflow. I was waiting to finish the build step for 3 hours to terminate itself, but it didn't. Any way to fix this? Here's the screenshot and my webpack.mix.js and my webpack.config.js.
This is the 8th time I have run this job, still the same result. Modify the webpack.mix.js, remove the yarn.lock and node_modules on my machine, then push the yarn.lock, still same, I'm running out of ideas.
And here's my webpack.mix.js (and yes, I have PostCSS and SASS, I was intending to going full sass, but since Vuetify is struggling, so at the moment, I'll use the postcss instead for Vuetify stuff, but other than that, I'm using sass)
const mix = require('laravel-mix');
require('laravel-mix-favicon');
require('vuetifyjs-mix-extension');
/*
|--------------------------------------------------------------------------
| Mix Asset Management
|--------------------------------------------------------------------------
|
| Mix provides a clean, fluent API for defining some Webpack build steps
| for your Laravel applications. By default, we are compiling the CSS
| file for the application as well as bundling up all the JS files.
|
*/
mix.js('resources/js/app.js', 'public/js').vue()
.postCss('resources/css/app.css', 'public/css', [
require('postcss-import'),
require('autoprefixer'),
])
.sass('resources/sass/print.scss', 'public/css')
.webpackConfig(require('./webpack.config'))
.vuetify()
.sourceMaps()
.favicon({
blade: 'resources/views/favicon.blade.php',
})
.disableNotifications();
if (mix.inProduction()) {
mix.version();
}
And here's my webpack.config.js
const path = require('path');
module.exports = {
resolve: {
alias: {
'#': path.resolve('resources/js'),
},
},
output: {
chunkFilename: 'js/[name].js?id=[chunkhash]',
}
};
Any help would be appreciated. Thanks!
[EDIT]
In Github Actions, I always cancel workflow, and I don't want to wait for three hours++ anymore. This is a pain T-T.

How to map errors in transpiled babel files back to source?

I'm working with an express app and am using babel to transpile my code to be able to use some ES6/7/8 goodies.
THe command I'm running to transpile the files is: npx babel server --out-dir lib --watch. Then to start my server, I run nodemon lib/server.js.
The issue I'm currently running into is that all errors are happening from the transpiled files in /lib, so the trace doesn't quite match with what is actually in the source, making it hard to debug.
So let's say an exception is thrown on line 10 in a transpiled file in /lib, that error doesn't match up to where the error actually is in the source since the trace is with respect to the transpiled file.
Is there a way I can get it to map correctly?
Thanks!
#Brian i suggest you to use "babel-polyfill" and "babel-register" modules.
add these modules in your main entry file for example refer the below code.
in this way you do not need to transpile your code seprately and can debug in the same ES6+ original code.
just add the start command simply as shown in the below code snippet, it will run your node.js code and transpiles all your ES6+ features as well at run time on fly.
Example:
app.js
// Added for regenerator runtime!!
require('babel-polyfill');
// Transpile on the fly
require('babel-register')({
ignore: false,
only: /\/src/,
});
require('dotenv/config');
let server = require('./server');
server.listen(process.env.APP_PORT, () => {
console.info(`application started on port ${process.env.APP_PORT}`);
});
package.json
"scripts": {
"start": "node src/app.js",
Happy Coding :)

How to set up dev and API server from create-react-app?

I've started a new app with create-react-app, and ejected from that. I made a small express server as follows:
const express = require('express');
const app = express();
if(process.env.NODE_ENV === 'production') {
app.use(express.static('build'));
}
const port = process.env.PORT || 3000;
app.listen(port, () => {
console.log(`Server started at: http://localhost:${port}/`);
});
In package.json I've added a line, "proxy": http://localhost:3000", as well as switching the commands:
"scripts": {
"run": "npm-run-all -p watch-css start-js",
"start": "node server.js",
},
run used to be start.
However now of course when I run npm start and go to localhost:3000, I get Cannot GET /. I need this server to receive and return local API calls I'll be making from my app, but I also want it to run a hot-reloading dev server just like the old npm start (now npm run) command did. How do I do this?
Some time ago I made a fork of the create-react-app repository adding webpack watch option because of this same reason. It might help you.
Just to add more info, I really invested time looking on how to get webpackdevserver to build the "bundle.js", and found that it is not possible because it loads the bundle into memory but doesn't persist it, so the file is never created. The only way available is the webpack watch option but, I don't understand why the create-react-app team can't add it to the repo, it's a really requested feature, and there are more forks than mine that solves this issue. So, you have three options:
Use the proxy server in package.json (if it works)
Make your own fork and add the watch option, or use an existing one
Don't use create-react-app

Vue.js Webpack Template in a Docker Container: How do I add Webpack-Dev-Server --watch-poll flag?

I am running the webpack / webpack-dev-server portion of the base Vue.js Webpack template (https://github.com/vuejs-templates/webpack/) inside of a docker container I created. The container also contains the vue CLI in order to create new projects (you can get my container here if you want: https://hub.docker.com/r/ncevl/webpack-vue/).
Hot-reload does not work after moving from the webpack-simple template to this one.
Everything was working using the Webpack-Simple template which you can clone / see over here: https://github.com/vuejs-templates/webpack-simple
I was able to get the simple template running (with hot-reload working as intended) with the following webpack-development-server launch command:
webpack-dev-server --hot --inline --progress --host 0.0.0.0 --watch-poll
That said the full (not simple) version of the webpack template does not appear to use a webpack-dev-server launch command and instead appears to use additional middleware as referenced in build/dev-server.js (https://github.com/vuejs-templates/webpack/blob/master/template/build/dev-server.js) and the webpack dev config.
Since the --watch-poll was the key to getting the WDS hot-reload functionality to work within a docker container in the last project, my thinking is that I need to do something similar with the webpack-hot-middleware but I dont see anything in their docs (over here: https://github.com/glenjamin/webpack-hot-middleware) that talks about changing to a polling based approach.
I am not 100% sure the polling flag will do the trick since I can see the container recompile my source when I make a change. I can also see the change in my browser if I refresh it manually.
Whats stranger still is if I inspect my page in browser within chrome dev tools, and then head over to network / XHR I can see that the browser actually does receive information from the webpack-dev-server, but visually it does not update.
Give the above I assume websockets (or socket.io which I think is used) are working and communicating between the browser and the WDS so maybe this is a browser caching issue of some sort?
I checked in my console and found this so it is looking like a header issue:
For reference the text error from that image (to make it easier for anyone having the same issue to find this post) is:
EventSource cannot load http://__webpack_hmr/. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://0.0.0.0:8080' is therefore not allowed access.
Again the Hot-Reload / Hot Module Reload was working with this identical container setup when using the webpack-simple Vue.js template.
I am wondering if anyone has run into anything similar or has any ideas on how to add the polling option . I guess my alternative would be roll back to a more basic webpack config and rebuild that portion of things to use the traditional webpack-dev-server / webpack config but give the above I am not sure that is going to fix it.
I am adding this as a separate answer since it more specifically answers the question in the title, while my other answer more specifically explains what solved my actual problem.
The vue.js webpack template project (which can either be init'd from the Vue CLI or pulled from its repo over here: https://github.com/vuejs-templates/webpack) separates its config files into several different directories.
I am posting this answer so that anyone who runs into the need to add polling to their project will be able to understand how / where to do that.
The base project structure for a Vue.js webpack template project looks like this:
The files that you care about if you are messing with trying to get hot module reload working are related to creating your server primarily with webpack-dev-middleware. The most important files related to that are highlighted here:
Basically if you want to add the polling code to the webpack-dev-middleware server you need to be in the /build/dev-server.js file on lines 20 to 24 that look like this:
var devMiddleware = require('webpack-dev-middleware')(compiler, {
publicPath: webpackConfig.output.publicPath,
quiet: true
})
To add polling you would add it just before or after quiet: true. As a side note, if you are having trouble with HMR I would change "quiet:true" to queit false to get a more verbose read out of whats going on from webpack-dev-middleware. I have included verbose and polling modifications to the above code here:
var devMiddleware = require('webpack-dev-middleware')(compiler, {
publicPath: webpackConfig.output.publicPath,
quiet: false, //Changed to for additional verbosity
watchOptions: { //Add Polling
aggregateTimeout: 300,
poll: 1000
}
})
My other answer is in regards to what ended up solving my problem, not necessarily how to actually add polling (which might be necessary for someone else but did not end up being needed to make my dockerized setup work).
It should also be noted that sometimes when HMR (webpack hot module reload) is not detecting changes it is due to the fact that webpack-hot-middleware or webpack-dev-middleware is running into an issue whereby some invisible characters are / were added to the name of the base project directory (probably by someone building the base Vue project) and therefore webpack on certain OSes is not able to see the changes.
If that happens to you and you are on OSx or running webpack inside of a docker container and you can't get HMR to detect changes, try to rename your vue-webpack project directory and it should work.
Ok. So I can't really take credit for this one since it was actually answered by Discuss user Cristian Pallarés over here: http://webpack.github.io/docs/webpack-dev-server.html#combining-with-an-existing-server
Christian says:
I was just trying the same. I just use "php artisan serve" on localhost:8000, and Webpack Dev Server on localhost:3000. You should make this:
set your webpack config "output.publicPath" as "http://localhost:3000/static/" instead of "/static/"
make your php application load this:
The key is the output.publicPath being absolute. Now, you should run "php artisan serve" and launch your webpack dev server too (in my case I use gulp).
Basically I took that and dug through the Vue.js Webpack Template files to locate the config file where webpack was looking for the public path. the public path setting ended up being in the index.js file located in the /config directory of the template.
I changed my code to look like this:
assetsSubDirectory: 'http://localhost:8080/static/', //!!Changed from /static/
assetsPublicPath: 'http://localhost:8080/', //!!Changed from /
As opposed to the previous setting which DID NOT WORK and looked like this:
assetsSubDirectory: '/static/',
assetsPublicPath: '/',
After that I was able to see my changes hot reload while running the vue.js Webpack template from within my docker container.

Sails.js application not refreshing files from assets after start

I have a Sails.JS application with Angular.JS front-end.
The angular files are stored in /assets/linker and they are injected properly on start. My issue is that when I change css or js file from assets the change doesn't appear on the server, the loaded js file is the same as when the server started. I tried to clear my browser cache and tried in another browser, but still the same.
I also tried to run the application with forever -w and nodemon, but still nothing. The application is in dev mode, anyway starting with sails lift --dev does not solve the issue neither.
I have feeling that I miss something in configuration. Is there any way to force reloading of assets?
You need to check your Gruntfile configuration. It's where the magic happen in term of linker and livereload.
Specifically, you'll need to look at the watch task and the related tasks.
By default it looks like this :
watch: {
api: {
// API files to watch:
files: ['api/**/*']
},
assets: {
// Assets to watch:
files: ['assets/**/*'],
// When assets are changed:
tasks: ['compileAssets', 'linkAssets']
}
}
I found the problem. I made the Angular.js structure with angular generator
which adds not only the js structure, but also karma test environment containing shell and bat scripts, karma framework and more.
Building sails application with all these files in watched folder is breaking the refresh functionality. There's no errors in console and nothing in the running application, but the files from assets are not reloaded anymore.
Tip of the day: be careful with the files you have in assets and take a look what does generators generate!
I came here looking for livereload, after a little search
Live Reloading
Enabling Live Reload in Your HTML
in current version of Sails v0.10 there is a file for watch task: tasks/config/watch.js

Resources