I need some help with nodemone and webpack. My team and I are working on a project, we are still in the early stage, but I've been fighting this issue for three days.
We are working on a remote physical host that runs multiple VMs. Each VM is serving different purpose. I have setup an NFS share between the VM and the host. We all work from home so we upload the files via FTP to the host (actually we use Visual Studio Code with ftp-sync plugin, so when I hit the save button, the file automatically uploads to the remote folder). However, nodemon and webpack are not detecting the change in the file.
I have mounted the nfs share with the following options:
mount -o noac,sync <host IP address>:/path/to/folder
I have checked the ctime and mtime and they are in perfect sync. However, for some reason the change in the file is never detected by webpack and nodemon. If I hit touch within the guest, it works.
Can anyone suggest any solution to this issue?
I actually managed to find a solution literally few minutes after I posted my answer. So basically the issue was fixed by adding "--watch-poll" argument to my package.json scripts.
"scripts": {
"dev:build": "webpack -w --watch-poll",
"dev": "cross-env NODE_ENV=development nodemon dist/index.bundle.js" }
Right now, when I save the file within Visual Studio Code, it is automatically uploaded to the host, synced with the VM development environment and automatically re-compiled and the server restarted with nodemon.
Related
I would like to use nodemon to restart my project when its files are changed. I think nodemon works by listening for inotify events to trigger reloading a node.js project.
The project runs in a docker container, and the project files are in a mounted volume.
When the project files are edited from inside the docker container, for example
docker-compose exec dev vim server.js
nodemon works correctly and restarts the server.
However, when an editor running on the host machine is used, nodemon does not pick up the changes and restart the program.
The contents of the files in the docker container do in fact change, so I suspect editing files this way just doesn't trigger an FS event.
Is it possible to set this up so that editing files on the host machine causes file system events to occur in the Docker container? Why does this not happen already?
Platform Info:
Docker for Windows (Hyper-V)
node docker container
WebStorm -- Host based editor
It looks like file system events just don't work when Docker is running in Hyper-V and the changes happen on the host. But, it's possible to work around that limitation by enabling polling in nodemon:
nodemon -L server.js
In WebStorm the full command that ends up getting used is
docker-compose run dev node node_packages/nodemon/bin/nodemon.js -L server.js
More info:
https://github.com/remy/nodemon#application-isnt-restarting
After setting up a full production CI pipeline for a docker / ghost.org blog based site, I am attempting to setup a local development environment to more quickly develop themes, however with Ghost running in "Development Mode" and while passing "NODE_ENV = development" changes are not visible on browser refresh.
I am running the official Ghost Docker container (https://hub.docker.com/_/ghost/) locally, with Ghost properly in Development mode, changes made to the local host code (which were piped into the Docker Container via the -v volumes tag) are not visible on browser refresh.
The Back Story
Since I was at first running an NGinx reverse proxy in-front of the ghost container I started by attempting to tune my nginx.conf file since I assumed that the issue was cache based.
I added the following to my nginx.conf to attempt to disable all caching on my local ghost.org blog to make sure I was not caching my pages:
expires off;
I about Docker / NGinx reverse proxies having some issues regarding sendfile related to virtualbox (Docker restart not showing the desired effect), so I set sendfile off:
sendfile off;
When the above had no effect I completely removed the Reverse Proxy for my local development setup (to try to narrow down the possible issues). I assumed that this would solve the problem and allow a browser refresh to show my local changes, it did not.
After NGinx Reverse Proxy was Removed
At this point I was running ONLY the official ghost docker image (with no reverse proxy out front).
Obviously the issue was with ghost so my first impulse was that Ghost was running in "Production Mode" even though Ghost said it was running in "development mode" at the end of its startup. To make SURE it was running in development mode I removed the "Production" block options from my config.js ghost configuration file (which would cause ghost to error out if it indeed was in production mode).
I also added an echo statement to the beginning of the config file so that when Ghost spins up it will Echo out the current value of NODE_ENV (to make sure that is set properly to development).
Removed the container with:
docker rm ghost -f
And re-ran it successfully, ghost echoed "NODE_ENV = development" and booted, but again changes were not visible.
Are Code changes changing inside the container?
My next thought was that my changes might not be doing anything / being passed through into my docker container. Therefore I docker exec -it'd into the container with:
docker exec -it ghost bash
I installed vim inside of the container and then opened up one of the ghost files to verify the source before exiting VIM without saving. I then modified the file on my local host and saved it.
When I re-open that same file from within the Ghost container I was able to successfully view my changes therefore proving that volume based files on my local host are properly finding their way into the Ghost container and Should be able to be served / viewed / updated with a browser refresh since Ghost is in development mode.
Try a Different Container?
Next I tried a different container from Docker Hub (even though changing this would mean I have to make changes to my production pipeline to sync everything back up).
I searched Docker hub and found two of the most popular alternative Ghost.org docker containers, then pulled those down to test:
docker pull ptimof/ghost/ (2.9k pulls)
and
docker pull gold/ghost/ (1.9k pulls)]
Obviously MOST people use the official ghost docker image with 1 million + pulls so I was not confident that this would have any impact, however upon looking at the Dockerfile(s) for gold/ghost and ptimof/ghost both referenced a modification to permissions in the following line:
RUN chown -R user $GHOST_SOURCE/content
The above appeared to reference the production ghost blog and then modify the permissions for the content sub-directory that should then contain the themes folder and my code changes. I was a little hopeful at this point, but alas, after replacing the official container with each alternative container (even though the Dockerfiles appear identical) the problem persisted.
Another version of Ghost?
The last thing I tried was to spec out an older version of ghost since I suppose it could be an issue in the latest released container, I tried two previous versions but this had no impact.
StackOverflow to the rescue?
Since I couldn't think of anything else to try I thought I would post here before continuing my search on my own in the hopes that someone with more ghost experience, more docker experience (hopefully both) might stumble onto an answer.
The docker run command I am using is;
docker run --env NODE_ENV=development --name ghost -d -p 80:2368 -v ~/blog/ghost-letsencrypt/config/config dev.js:/usr/src/ghost/config.example.js -v ~/blog/ghost-letsencrypt/themes/:/usr/src/ghost/content/themes ghost:0.11.3
To re-state the problem: Everything spins up properly and my local ghost docker container successfully connects to my remote mysql DB as intended. My site is visible and correctly displayed in by browser but when I make changes to my local code neither a container restart or a browser refresh shows any of those changes.
Changes to the local code base are not properly displayed on browser refresh because the volume is mapping to the /src/ (Source) directory as opposed to the /var/lib/ghost working directory.
Ghost File Structure
Basically the ghost blog (inside a container or otherwise) has two identical file structures that can both contain theme / blog content. The Source directory and Working Directory.
The first one is located within the following directory: /usr/src/ghost/
This directory contains the base source ghost files that are only used / copied to the working directory during blog start-up if no additional theme / config files are found in the working directory.
Default Config would be in: /usr/src/ghost/config.example.js
Default Themes would be in: /usr/src/ghost/content/themes
Essentially the /src Source Directory is like a template that exists within every ghost installation for backup / reversion purposes only. It is in this directory that we copy / overwrite the default configuration file since when ghost spins up the first time it will then pull our MODIFIED default config into the working ghost directory.
This is where I (and maybe you) might have gone wrong since many docker containers copy their config file into the /src directory.
Working Ghost Files
The second file structure is where Ghost looks to find the source files which it will actually interact with when serving / creating static or dynamically created pages. The working directory is at the following location within the Ghost container: /var/lib/ghost
Themes that are currently being used by the working ghost server would be at: /var/lib/ghost/themes
So, to compare the Themes for the working directory with the source directory: /usr/src/ghost/content/themes
/var/lib/ghost/themes
As you can see there is no "content" sub-directory within the /var/lib/ghost path while that "content" sub-directory does exist within the src / source file structure.
Our problem was that we were referencing the source directory / path from our docker volume as opposed to the working ghost directory. Therefore the blog would spin up properly (since the theme files DID exist on ghost launch, and would be successfully copied from there into our working ghost directory) but then would not be able to be modified since Ghost only looks at the source ONCE when starting for the first time.
That caused refreshing the browser or re-starting the containers to have no effect since ghost was already started for the first time. Ie. Ghost only copies from the source /src file structure ONCE when it first starts and then completely forgets / ignores that file structure from then on.
Any changes to files within the source file structure have no impact on Ghost once it is running so changes we made to files that were referenced by our volumes were not monitored by ghost and therefore were not displayed by the server / visible within the browser.
My goal is to set up a Docker container that automatically restarts a NodeJS server when file changes are detected from the host machine.
I have chosen nodemon to watch the files for changes.
On Linux and Mac environments, nodemon and docker are working flawlessly.
However, when I am in a Windows environment, nodemon doesn't restart the server.
The files are updated on the host machine, and are linked using the volumes parameter in my docker-compose.yml file.
I can see the files have changed when I run docker exec <container-name> cat /path/to/fileChanged.js. This way I know the files are being linked correctly and have been modified in the container.
Is there any reason why nodemon doesn't restart the server for Windows?
Use nodemon --legacy-watch to poll for file changes instead of listening to file system events.
VirtualBox doesn't pass file system events over the vboxfs share to your Linux VM. If you're using Docker for Windows, it would appear HyperV doesn't propagate file system events either.
As a 2021 side note, Docker for Mac/Windows new GRPCfuse file system for mounting local files into the VM should send file system events across now.
2022 note: Looks like Windows/WSL Docker doesn't share FS events to the Linux VM (see comments #Mohamed Mirghani and #Ryan Wheale and github issue).
It is simple, according to the doc you must change:
nodemon server.js
to:
nodemon --legacy-watch server.js
As mentioned by others, using node --legacy-watch will work, however, the default polling rate is quite taxing on your cpu. In my case, it was consuming 30% of my CPU just by looping through all the files in my project. I would advise you to specify the polling interval as mention by #Sandokan El Cojo.
You can do so by either adding "pollingInterval": 4000 (4 seconds in this example) to your nodemon.json file or specifying it with the -P or --polling-interval flag in the command.
This was an issue in the docker for Windows. Now it's fixed
https://www.docker.com/blog/new-filesharing-implementation-in-docker-desktop-windows/
Maybe I am overcomplicating this.
My goal is to create a Docker-based workflow on Windows for Node.js application development.
In development time, I'd be able to run my app locally inside a container and still see the latest version without too much hassle (I don't want to rebuild the image everytime to see the latest).
On the other hand, when I deploy to production, I want to have my source files "baked" into the container image with all the dependencies (npm install)
So I created two Vagrantfiles - one for the container and one for its host. Here's an extract of the latter:
Vagrant.configure(2) do |config|
config.vm.provider "docker" do |docker|
docker.vagrant_vagrantfile = "host/Vagrantfile" # it references the host Vagrantfile
docker.build_dir = "." # we have a Dockerfile in the same dir
docker.create_args = ['--volume="/usr/src/host:/usr/src/appcontainer:rw"']
end
end
/usr/src/host is a directory which contains all of my source code (without node_modules). During the build of the Dockerfile, Docker copies the package.json to /usr/src/appcontainer and issues an npm install there which is fine for my second requirement (deploy to production)
But my first requirement was to change the source during development, so I mounted /usr/src/appcontainer as a volume pointing to the host directory /usr/src/host. However, this is not working, because /usr/src/host doesn't have a node_modules folder - this folder only exists in the container.
This whole problem seems to be easy - changing a file under Windows, see its changing both under the Linux host VM and in its container and vica versa... But I've got a bit stuck.
What is the best practice to achieve this syncing behaviour?
However, this is not working, because /usr/src/host doesn't have a node_modules folder
You can use one of the approaches described in this question, for instance by using a data volume dedicated for node_modules.
Or mounting the host node_modules as a separate folder within the container.
I was getting tired of using FileZilla every time I wanted to push a change to my server hosting my website and so I set up a github repo and linked it to my server so that changes and pushed right to the server.
However, my backend is written in node and so each time I update my server.js file I have to restart the server. With "node server.js"
Is there a way to watch the file and programmatically restart the node server when an update is detected?
If it helps, my serve is Ubuntu Linux running apache2
You could try writing something yourself, or use one of the popular libraries that are already out there:
https://github.com/petruisfan/node-supervisor supervisor server.js
https://github.com/remy/nodemon nodemon server.js