Chokidar isn't picking up file changes inside docker container - node.js

I'm using docker for mac version 1.12.0-rc2 for a react project. My workflow is this:
src/ folder on OS X is mounted to the container
When a developer modifies a file in src/ it gets converted to ES5 and placed in public/ (this works).
When a file is changed in public/ another watcher triggers hot reloading (works on my localhost but not in the container).
Here's my step watcher code from step 3:
// root = "/src"
const watcher = chokidar.watch(root, {
usePolling: true,
awaitWriteFinish: {
pollInterval: 100,
stabilityThreshold: 250
},
ignored: /\.(git|gz|map)|node_modules|jspm_packages|src/,
ignoreInitial: true,
persistent: true
})
.on("change", function fileWatcher(filename) {
const modulePath = filename.replace(`${root}/`, "");
wss.clients.forEach(function sendFileChange(client) {
send("filechange", modulePath, client);
});
if (cache[filename]) {
wss.clients.forEach(function sendCacheFlush(client) {
send("cacheflush", filename, client);
});
delete cache[filename];
}
});
And my docker-compose.yml file:
version: '2'
services:
wildcat:
build:
context: .
args:
JSPM_GITHUB_AUTH_TOKEN:
image: "nfl/react-wildcat-example:latest"
environment:
NODE_ENV: development
PORT: 3000
STATIC_PORT: 4000
COVERAGE:
LOG_LEVEL:
NODE_TLS_REJECT_UNAUTHORIZED: 0
CHOKIDAR_USEPOLLING: 'true'
volumes:
- ./src:/src/src
- ./api:/src/api
ports:
- "3000:3000"
- "4000:4000"
ulimits:
nproc: 65535
entrypoint: "npm run"
command: "dev"

After spending a day or two on this I found the issue, the line:
ignored: /\.(git|gz|map)|node_modules|jspm_packages|src/,
Was causing chokidar to ignore /src which was the folder I was copying all the source code to in the docker container. I changed this path in the Dockerfile and docker-compose.yml to /code instead and everything worked as expected.

As a hopefully helpful note to those coming here by the title, Chokidar docs states:
It is typically necessary to set this (polling) to true to successfully watch files over a network, and it may be necessary to successfully watch files in other non-standard situations.
On Docker Desktop for Mac 4.0.0, I got Firebase Emulators (that uses Chokidar) to see host-side file changes by:
environments:
- CHOKIDAR_USEPOLLING=true

Related

How to configure webpack hot reload to work inside Docker?

I'm working on a Symfony 4 project for months, and I want to Dockerize it.
I make everything work except Webpack, I use it to compile my .scss and .js files with the npm run watch or npm run dev command.
Actually webpack does not listen changes I do in a .scss or .js file for example.
Here is my config, I surely miss something in my files.
My docker-compose.yml :
version: '3.8'
services:
mysql:
image: mysql:8.0
command: --default-authentication-plugin=mysql_native_password
restart: on-failure
environment:
MYSQL_ROOT_PASSWORD: rootpassword
phpmyadmin:
image: phpmyadmin/phpmyadmin
restart: on-failure
depends_on:
- mysql
ports:
- '8004:80'
environment:
PMA_HOSTS: mysql
php:
build:
context: .
dockerfile: php/Dockerfile
volumes:
- '../.:/usr/src/app'
restart: on-failure
env_file:
- .env
nginx:
image: nginx:1.19.0-alpine
restart: on-failure
volumes:
- '../public:/usr/src/app'
- './nginx/default.conf:/etc/nginx/conf.d/default.conf:ro'
ports:
- '80:80'
depends_on:
- php
node:
build:
context: .
dockerfile: node/Dockerfile
volumes:
- '../.:/usr/src/app'
command: npm run watch
My Dockerfile for Node Image :
FROM node:12.10.0
RUN apt-get update && \
apt-get install -y \
curl
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
WORKDIR /usr/src/app
CMD ["npm", "run", "watch"]
My webpack.config.js :
var Encore = require('#symfony/webpack-encore');
var CopyWebpackPlugin = require('copy-webpack-plugin');
if (!Encore.isRuntimeEnvironmentConfigured()) {
Encore.configureRuntimeEnvironment(process.env.NODE_ENV || 'dev');
}
Encore
.setOutputPath('public/build/')
.setPublicPath('/build')
.addEntry('app', './assets/js/app.js')
.splitEntryChunks()
.disableSingleRuntimeChunk()
.enableSassLoader()
.cleanupOutputBeforeBuild()
.enableBuildNotifications()
.enableSourceMaps(!Encore.isProduction())
.enableVersioning(Encore.isProduction())
.configureBabel(() => {}, {
useBuiltIns: 'usage',
corejs: 3
})
.addPlugin(new CopyWebpackPlugin([
{ from: './assets/pictures', to: 'pictures' }
]))
;
module.exports = Encore.getWebpackConfig();
// module.exports = {
// mode: 'development',
// devServer: {
// port: 80,
// host: '0.0.0.0',
// disableHostCheck: true,
// watchOptions: {
// ignored: /node_modules/,
// poll: 1000,
// aggregateTimeout: 1000
// }
// }
// }
As you can see I already tried some thing in webpack.config.js, I saw many things about watchOptions but I didn't get it.
And here is my project's organisation :
project's organisation
I want to be able to launch my Docker with Webpack listening any change I do in real time.
Here is the command console after running docker-compose up:
console command docker-compose up
If you have some advise to improve my Docker environment, I take it all !
Thank you !
i just use this:
docker-compose.yml:
node:
image: node:16-alpine3.13
working_dir: /var/www/app
user: "$USERID"
volumes:
- .:/var/www/app
tty: true
and docker-compose exec node yarn watch
working as expected.
Okay i think i solved my issue,
I followed #Rufinus answer; i had to docker-compose up in a first console command, open a second console command and execute winpty docker-compose exec node yarn watch but for some reason i had issue with node-sass compatibility : i mounted my node_module (windows 10) folder into the container (Linux).
So i opened my node CLI container and execute npm rebuild node-sass to solve this and finally it worked !
But i don't know why, my current solution is to execute npm run watch on my local folders (like i used to do it before Dockerizing all my application) and it re-builds assets when i change .scss or .js file.

docker-compose for attaching node.js debugger from VS Code to a node process in WSL docker

I'm really doing the best as I can but I don't know what am I doing wrong.
I have this existing project that can be started and debugged both on Win10 and Linux containers. But I want to be able to attach the debugger in WSL container on Win10 and I'm really loosing my patience now.
So, I have VS Code on Win10 and WSL2 installed. I have Remote-WSL extension (also Remote-Containers, but can't get that to work either, so let's skip that for the time being). And I would prefer to keep Dockerfile as-is, if possible.
When I try to attach to node debugger by running "Docker: Attach to Node" configuration, I get the following error:
Error processing attach: Error: Could not connect to debug target at http://localhost:9229: Promise was canceled
at e (/home/ozren_admin/.vscode-server/bin/3dd905126b34dcd4de81fa624eb3a8cbe7485f13/extensions/ms-vscode.js-debug/src/extension.js:1:92915)
Here are my files...
/docker-compose.yml
version: '3'
services:
web:
build:
context: .
dockerfile: ./Dockerfile
env_file: ./myapp.env
depends_on:
- db
volumes:
- "./app:/src/app"
ports:
- "5000:4000"
- "9229:9229"
expose:
- 9229
links:
- db:myapp_mobile_apis_db
db:
image: mysql:5.7
container_name: myapp_mobile_apis_db
restart: always
environment:
# see: https://hub.docker.com/_/mysql
MYSQL_DATABASE: 'myapp'
MYSQL_USER: 'myapp_user'
MYSQL_PASSWORD: 'myapp_user_pwd'
MYSQL_ROOT_PASSWORD: '****************'
ports:
- '3306:3306'
expose:
- '3306'
volumes:
- my-db:/mnt/c/Projects/MyApp/Local/data/
volumes:
my-db:
/Dockerfile
FROM node:alpine
RUN mkdir /src
WORKDIR /src
COPY app/ /src/app/
COPY gulpfile.js /src/
COPY .sequelizerc /src/
ADD app/package.json /src/package.json
RUN npm install --global gulp-cli
RUN npm install
COPY myapp.env /src/.env
EXPOSE 80
CMD gulp prod
/.vscode/launch.json
{
"version": "0.2.0",
"configurations": [
{
"type": "node",
"request": "attach",
"name": "Docker: Attach to Node",
"port": 9229,
"address": "localhost",
"localRoot": "${workspaceRoot}/app",
"remoteRoot": "/src/app",
"protocol": "inspector"
}
]
}
When I add the following to docker.compose.yml
command: sh -c "npm run start:debug:docker"
I get this on docker-compose up
web_1 | npm ERR! missing script: start:debug:docker
I tried googling for additional info and the idea is probably to run node in the debugging mode, but I obviously can't get it to work.
Any help appreciated!
There was this:
/gulpfile.js
'use strict';
let gulp = require('gulp');
let nodemon = require('gulp-nodemon');
let started = !1;
let paths = {
js: [
'**/*.js',
'!node_modules'
]
};
gulp.task('prod', (e) => {
nodemon({
script: './app/app.js',
ext: 'js',
watch: paths.js,
ignore: ['./app/node_modules/**']
}).on('start', () => {
started || (started = !0, e());
});
});
And when I added this line to nodemon options to enable debug over inspect:
nodeArgs: ['--inspect=0.0.0.0:9229'],
...all of a sudden the debugger started working.
DISCLAIMER: I started my journey to Node.js ecosystem from .Net world, where stuff like remote debugging and other basics usually aren't hidden behind... packages, but eh. Live and learn.

Why NextJS using Docker container did not reload after changed code for dev environment?

I'm trying to run the NextJS on a Docker container using Dockerfile and running via docker-compose, after I changed my code in a JS file (such as index.js) the Next server did not reload.
But when I've tried to run outside without using Docker (by executing the "npm run dev" command directly) the Next server did reload smoothly.
I've also tried to run the server by "nodemon" command (inside a container), it did not make it either.
Dockerfile:
FROM node:10.14.2-alpine
COPY . /home/next_app
WORKDIR /home/next_app
RUN npm install
docker-compose.yml:
version: "3.6"
services:
self_nextjs:
container_name: self_nextjs
build:
context: ./app
dockerfile: Dockerfile
ports:
- 3000:3000
volumes:
- ./app:/home/next_app
- /home/next_app/node_modules
networks:
- zen_frontend
restart: always
command: npm run dev
networks:
zen_frontend:
name: zen_frontend
driver: bridge
Any suggestions would be appreciated.
I had the same issue on Windows 10. I followed some of the instructions in this thread https://github.com/zeit/next.js/issues/6417. Basically, you have to add a next.config.js to poll for changes. I'm not sure if MacOS has the same problem.
module.exports = {
webpackDevMiddleware: config => {
config.watchOptions = {
poll: 800,
aggregateTimeout: 300,
}
return config
},
}
Have you tested by exposing webpack default hot reload port?
add to your Dockerfile
...
EXPOSE 49153
...
and update your docker-compose.yml
version: "3.6"
services:
self_nextjs:
container_name: self_nextjs
build:
context: ./app
dockerfile: Dockerfile
ports:
- 3000:3000
- 49153:49153
volumes:
- ./app:/home/next_app
- /home/next_app/node_modules
networks:
- zen_frontend
restart: always
command: npm run dev
networks:
zen_frontend:
name: zen_frontend
driver: bridge
Hope this help,
Regards
I had to change #davidatthepark solution a little bit to get it to work for me. It looks like webpackDevMiddleware is not supported any more.
module.exports = {
webpack: (config, _) => ({
...config,
watchOptions: {
...config.watchOptions,
poll: 800,
aggregateTimeout: 300,
},
}),
}

Node watch on a docker-compose volume does not register delete events

I have a simple Node.js HTTP server running inside a docker container. One of the basic structural folders uses volume from docker-compose.yml to mirror the host and container directory.
Within the Node server, I have a watcher set up to track changes within the volumed directory, using the NPM package chokidar (although I have tried multiple other watchers already with the same result).
const watcher = require("chokidar");
watcher
.watch("./app/experiments", { depth: 0, ignoreInitial: true })
.on("all", (event, path) => {
console.log(event);
// ... DO SOME EXPRESS AND WEBPACK STUFF
});
When I run the Node server locally, the watcher correctly picks up changes to the watched directory. In this case, chokidar is reporting these as addDir or unlinkDir, which correspond to a scaffolding script I run to add or remove new folders into the directory (which is served later via express.static()).
STDOUT:
> addDir
> EXPERIMENT ADDED!
> ...
> unlinkDir
> EXPERIMENT DELETED!
However, when I port the application into a docker container, the behavior changes in a really strange way. I continue to get addDir events when I create new folders in the volumed directory, but I no longer receive unlinkDir (delete) events!. Note that this only happens if I add / delete a file within the volumed directory on the host machine. If I add / delete a file within that directory inside the docker container, my watcher correctly reports all of these events.
In either case, the volumed directory correctly mirrors itself. E.G. the files are actually deleted or added, and I can verify their existence on the host and by shelling into the docker container and running ls.
Any docker geniuses out there with sage wisdom on why this is happening?
KEY STUFF:
OS X 10.13.6
Docker Toolbox:
Docker 18.03.0-ce
docker-maching 0.14.0
docker-compose 1.20.1
virtualbox 5.2.18r124319
Dockerfile:
FROM node:8.12.0
WORKDIR /usr/dd-labs
COPY package*.json ./
RUN npm install
COPY app/ ./app
COPY server.js ./
COPY webpack/ ./webpack
EXPOSE 8080
Docker-compose.yml:
version: "2"
services:
app:
image: #someImageName
build: .
ports:
- "8080:8080"
labels:
io.rancher.container.pull_image: always
environment:
VIRTUAL_HOST: labs.docker
volumes:
- ./app:/usr/dd-labs/app
command: [sh, -c, "npm run start:dev"]
You are probably using Docker for Windows which has a very well known lack of support for file system events propagation from host to containers.
A work around is to use polling when in dev environment. With chokidar you'd want usePolling: true option.

Enabling webpack hot-reload in a docker application

I have a docker app with the following containers
node - source code of the project. it serves up the html page situated in the public folder.
webpack - watches files in the node container and updates the public folder (from the node container) on the event of change in the code.
database
this is the webpack/node container setup
web:
container_name: web
build: .
env_file: .env
volumes:
- .:/usr/src/app
- node_modules:/usr/src/app/node_modules
command: npm start
environment:
- NODE_ENV=development
ports:
- "8000:8000"
webpack:
container_name: webpack
build: ./webpack/
depends_on:
- web
volumes_from:
- web
working_dir: /usr/src/app
command: webpack --watch
So currently , the webpack container monitors and updates the public folder. i have to manually refresh the browser to see my changes.
I'm now trying to incorporate webpack-dev-server to enable automatic refresh in the browser
these are my changes to the webpack config file
module.exports = {
entry:[
'webpack/hot/dev-server',
'webpack-dev-server/client?http://localhost:8080',
'./client/index.js'
],
....
devServer:{
hot: true,
proxy: {
'*': 'http://localhost:8000'
}
}
}
and the new docker-compose file file webpack
webpack:
container_name: webpack
build: ./webpack/
depends_on:
- web
volumes_from:
- web
working_dir: /usr/src/app
command: webpack-dev-server --hot --inline
ports:
- "8080:8080"
i seem to be getting an error when running the app
Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.
webpack | - configuration.entry should be one of these:
webpack | object { <key>: non-empty string | [non-empty string] } | non-empty string | [non-empty string] | function
webpack | The entry point(s) of the compilation.
webpack | Details:
webpack | * configuration.entry should be an object.
webpack | * configuration.entry should be a string.
webpack | * configuration.entry should NOT have duplicate items (items ## 1 and 2 are identical) ({
webpack | "keyword": "uniqueItems",
webpack | "dataPath": ".entry",
webpack | "schemaPath": "#/definitions/common.nonEmptyArrayOfUniqueStringValues/uniqueItems",
webpack | "params": {
webpack | "i": 2,
webpack | "j": 1
webpack | },
webpack | "message": "should NOT have duplicate items (items ## 1 and 2 are identical)",
webpack | "schema": true,
webpack | "parentSchema": {
webpack | "items": {
webpack | "minLength": 1,
webpack | "type": "string"
webpack | },
webpack | "minItems": 1,
webpack | "type": "array",
webpack | "uniqueItems": true
webpack | },
webpack | "data": [
webpack | "/usr/src/app/node_modules/webpack-dev-server/client/index.js?http://localhost:8080",
webpack | "webpack/hot/dev-server",
webpack | "webpack/hot/dev-server",
webpack | "webpack-dev-server/client?http://localhost:8080",
webpack | "./client/index.js"
webpack | ]
webpack | }).
webpack | [non-empty string]
webpack | * configuration.entry should be an instance of function
webpack | function returning an entry object or a promise..
As you can see , my entry object doesnt have any duplicate items.
Is there something additional i should be doing? anything i missed?
webpack-dev-server should basically proxy all requests to the node server.
I couldn't make webpack or webpack-dev-server watch (--watch) mode work even after mounting my project folder into container.
To fix this you need to understand how webpack detects file changes within a directory.
It uses one of 2 softwares that add OS level support for watching for file changes called inotify and fsevent. Standard Docker images usually don't have these (specially inotify for linux) preinstalled so you have to install it in your Dockerfile.
Look for inotify-tools package in your distro's package manager and install it. fortunately all alpine, debian, centos have this.
Docker & webpack-dev-server can be fully operational without any middleware or plugins, proper configuration is the deal:
devServer: {
port: 80, // use any port suitable for your configuration
host: '0.0.0.0', // to accept connections from outside container
watchOptions: {
aggregateTimeout: 500, // delay before reloading
poll: 1000 // enable polling since fsevents are not supported in docker
}
}
Use this config only if your docker container does not support fsevents.
For performance efficient way check out HosseinAgha answer #42445288: Enabling webpack hot-reload in a docker application
try doing this:
Add watchOptions.poll = true in webpack config.
watchOptions: {
poll: true
},
Configure host in devServer config
host:"0.0.0.0",
Hot Module Reload is the coolest development mode, and a tricky one to set up with Docker. In order to bring it to life you'll need 8 steps to follow:
For Webpack 5 install, in particular, these NPM packages:
npm install webpack webpack-cli webpack-dev-server --save-dev --save-exact
Write this command into 'scripts' section in 'package.json' file:
"dev": "webpack serve --mode development --host 0.0.0.0 --config webpack.config.js"
Add this property to 'webpack.config.js' file (it'll enable webpack's hot module reloading)
devServer: {
port: 8080,
hot: "only",
static: {
directory: path.join(__dirname, './'),
serveIndex: true,
},
},
Add this code to the very bottom of your 'index.js' (or whatever), which is the entry point to your app:
if (module.hot) {
module.hot.accept()
}
Expose ports in 'docker-compose.yml' to see the app at http://localhost:8080
ports:
- 8080:8080
Sync your app's /src directory with 'src' directory within a container. To do this use volumes in 'docker-compose.yml'. In my case, directory 'client' is where all my frontend React's files sit, including 'package.json', 'webpack.config.js' & Dockerfile. While 'docker-compose.yml' is placed one leve up.
volumes:
- ./client/src:/client/src
Inside the volume group you'd better add the ban to syncronize 'node_modules' directory ('.dockerignore' is of no help here).
volumes:
...
- /client/node_modules
Fire this whole bundling from Docker's CMD, not from RUN command inside your 'docker-compose.yml'.
WORKDIR /client
CMD ["npm", "run", "dev"]
P.S. If you use Webpack dev server, you don't need other web servers like Nginx in your development. Another important thing to keep in mind is that Webpack dev server does not recompile files from '/src' folder into '/disc' one. It performs the compilation in memory.
Had this trouble from a windows device. Solved it by setting WATCHPACK_POLLING to true in environment of docker compose.
frontend:
environment:
- WATCHPACK_POLLING=true
I had the same problem. it was more my fault and not webpack nor docker. In fact you have to check that the workdir in your Dockerfile is the target for your bindmount in docker-compose.yml
Dockerfile
FROM node
...
workdir /myapp
...
on your docker-compose.yml
web:
....
-volumes:
./:/myapp
It should work if you configure the reloading on your webpack.config.js

Resources