Docker - Override content of linked volume - node.js

Having simple Node.js docker container
docker-compose.yml:
app:
build: ./dockerfiles/app
volumes:
- /Users/home/work/app:/usr/app
Dockerfile:
FROM node:6.7-slim
COPY package.json /tmp
RUN cd /tmp && npm install
RUN mkdir -p /usr/app
WORKDIR /usr/app
CMD ["node", "./src/app.js"]
What I want to achieve is container where I have package.json and installed node modules (npm install). Part where I copy package.json and install modules inside container is pretty straighforward, but problem occur, when I want to use these node_modules inside linked app. I can't find any way, how to copy /tmp/node_modules into /usr/app/node_modules
Is there any Docker way ho to do that? If not, can I tell my node app to look for node_modules somewhere else than in root directory?

You can achieve what you want by changing the CMD used when starting the container, either in your Dockerfile, or in the docker-compose.yml file.
Instead of just starting node ./src/app.js, you want to do two things:
Copy the node_modules over.
Start Node
Using the docker-compose.yml, I would do the following:
app:
build: ./dockerfiles/app
volumes:
- /Users/home/work/app:/usr/app
command: >
bash -c "
rm -rf /usr/app/node_modules
&& cp -R /tmp/node_modules /usr/app/node_modules
&& node ./src/app.js
"
This will delete the existing node modules on the mapped-in volume, then copy in the ones from container, and then finally starts the node app. This is going to happen every time the container is started.

As #schovi has mentioned in order to not override the contents of node_modules within the container and the contents of node_modules within the folder of the host machine, it is necessary to create another internal volume in the docker-compose.yml file:
volumes:
- ${APP_PATH}:/usr/app
- /usr/app/node_modules
Doing that makes it safe to copy the files from /tmp/node_modules into /usr/app/node_modules using this instructions.
FROM node
# Node modules
COPY *.json /tmp/
RUN cd /tmp && yarn
# App
RUN mkdir -p /usr/app
WORKDIR /usr/app
RUN cp -a /tmp/node_modules /usr/app/node_modules
ENV NODE_ENV docker
CMD ["run-node", "src/app.js"]
However, I would create first the app folder and install node_modules directly on it, reducing considerably the cache layers and increasing the building speed.
FROM node:12.8.1 #always mind the version
# Node modules
RUN mkdir -p /usr/app
WORKDIR /usr/app
#Mind that point (workdir)
COPY package*.json .
RUN yarn
ENV NODE_ENV docker
CMD ["run-node", "src/app.js"]
I hope it helps! :D

Thing that helped me is following usage of volumes
volumes:
- ${APP_PATH}:/usr/app
# Empty node_modules directory
- /usr/app/node_modules
Then in Dockerfile:
FROM node
# Node modules
COPY *.json /tmp/
RUN cd /tmp && yarn
ENV NODE_PATH /tmp/node_modules:${NODE_PATH}
# App
RUN mkdir -p /usr/app
WORKDIR /usr/app
ENV NODE_ENV docker
CMD ["run-node", "src/app.js"]
This allow me to have node_modules in another directory and app will look for them there.

Related

Command in dockerfile doesn't take effect

Below is my dockerfile. After the dependencies are installed, I want to delete a specific binary (ffmpeg) from the node_modules folder on the container, and then reinstall it using the install.js file that exists under the same folder in node_modules.
FROM node:16-alpine
WORKDIR /web
COPY package.json package-lock.json ./
ARG NODE_ENV
ENV NODE_ENV ${NODE_ENV:-development}
ARG _ENV
ENV _ENV ${_ENV:-dev}
RUN npm install
RUN rm /web/node_modules/ffmpeg-static/ffmpeg
RUN node /web/node_modules/ffmpeg-static/install.js
COPY . .
EXPOSE 8081
ENTRYPOINT [ "npm" ]
I want the rm and node commands to take effect on the container after the startup and when all the dependencies are installed and copied to the container, but it doesn't happen. I feel like the commands after RUN are only executed during the build process, not after the container starts up.
After the container is up, when I ssh to it and execute the two commands above directly (RUN rm... and RUN node...), my changes are taking effect and everything works perfectly. I basically want these commands to automatically run after the container is up and running.
The docker-compose.yaml file looks like this:
version: '3'
services:
serverless:
build: web
user: root
ports:
- '8081:8081'
env_file:
- env/web.env
environment:
- _ENV=dev
command:
- run
- start
volumes:
- './web:/web'
- './env/mount:/secrets/'
if this dockerfile builds, it means you have a package-lock.json. That is evidence that npm install was executed in the root directory for that image, which means that node_modules exists locally and is being copied in the last copy.
You can avoid this by creating a .dockerignore file that includes files and directories (aka node_modules) that you'd like to exclude from being passed to the Docker build context, which will make sure that they don't get copied to your final image.
You can follow this example:
https://stackoverflow.com/a/43747867/3669093
Update
Dockerfile: Replace the ENTRYPOINT with the following
ADD ./start.sh /start.sh
CMD ["/start.sh"]
Start.sh
rm /web/node_modules/ffmpeg-static/ffmpeg
node /web/node_modules/ffmpeg-static/install.js
npm
Try rm -rf /web/node_modules/ffmpeg-static/ffmpeg
I assume that is directory, not a file.

Docker: node_modules symlink not working for typescript

I am working on containerization of Express app in TS. But not able to link node_modules installed outside the container. Volume is also mounted for development.But still getting error in editor(vscode) Cannot find module 'typeorm' or its corresponding type declarations., similar for all dependencies.
volumes:
- .:/usr/src/app
Dockerfile:
FROM node:16.8.0-alpine3.13 as builder
WORKDIR /usr/src/app
COPY package.json .
COPY transformPackage.js .
RUN ["node", "transformPackage"]
FROM node:16.8.0-alpine3.13
WORKDIR /usr/src/app
COPY --from=builder /usr/src/app/package-docker.json package.json
RUN apk update && apk upgrade
RUN npm install --quiet && mv node_modules ../ && ln -sf ../node_modules node_modules
COPY . .
EXPOSE 3080
ENV NODE_PATH=./dist
RUN npm run build
CMD ["npm", "start"]
I've one workaround where I can install dependencies locally, and then use those, but need another solution where we should install dependencies only in the container and not the outside.
Thanks in advance.
Your first code section implies you use docker-compose. Probably the build (of the Dockerfile) is also done there.
The point is that the volume mappings in the docker-compose are not available during build-phase in that same Docker-service.

Private node module pull inside docker

I have a private repository of a node_module which I install by including it in package.json
ssh://git#github.com/iamsaquib/<pivate-repo>.git
When I am copying all server files inside docker image and try to do a npm install it is unable to install the package and throws I don't have proper access rights. I think I have to authorize by copying my id_rsa.pub inside Dockerfile and add it as authorized key, what is the correct way to do this?
Dockerfile
FROM node:12-slim
ENV NODE_ENV=development
WORKDIR /app
USER root
COPY . .
RUN ./install.sh
RUN ./build.sh
EXPOSE 8000
CMD ["./run.sh"]
You need to mount SSH private key (/home/yourname/.ssh/id_rsa).
You should avoid putting private key in Docker images. One work around could be multi-stage image (security might still be debatable).
FROM node:12-slim as installer
ENV NODE_ENV=development
WORKDIR /app
USER root
COPY /home/yourname/.ssh /home/root/.ssh
COPY /home/yourname/.gitconfig /home/root/.gitconfig
COPY . .
RUN ./install.sh
RUN ./build.sh
RUN rm -rf /home/root/.ssh
RUN rm -rf /home/root/.gitconfig
# Final image
FROM node:12-slim
WORKDIR /app
ENV NODE_ENV=development
USER root
COPY --from=installer /app .
EXPOSE 8000
CMD ["./run.sh"]

Running gulp in Docker compose - does not create files

I have issues where gulp is not making any files. It says finished, but no file is being created.
If I log in to my docker instance using:
docker exec -t -i myservice-service /bin/bash
and if I run the gulp command, then it creates it properly
Then all the files defined in the gulpfile.js are created. In other words, public/dist/ is populated with the main.js and other css files.
This is my Dockerfile.
FROM node:9
RUN mkdir -p /usr/src/app
RUN mkdir -p /usr/src/logs
WORKDIR /usr/src/app
# GULP Installation
RUN npm install -g gulp
RUN npm install gulp
COPY package*.json /usr/src/app/
COPY .npmrc /usr/src/app/
RUN cd /usr/src/app/ && npm install && npm install -g nodemon
COPY . /usr/src/app
RUN chown -R node:node /usr/src/app && chown -R node:node /usr/src/logs
USER node
EXPOSE 3000
RUN gulp
CMD ["npm", "run-script", "start" ]
And this is my composer file (development):
version: "3"
services:
myservice-service:
build: .
image: myservice-service
container_name: myservice-service
volumes:
- .:/usr/src/app
- /usr/src/app/node_modules
environment:
- NODE_ENV=dev
ports:
- 3000:3000
command: nodemon --delay 2 ./bin/www
I run it as:
docker-compose -f docker-compose.development.yml up --build
When I run it like that, it does not create any files. I get the same output on the screen, when I run the command manually.
I have spent hours trying to make it work, I tried with setting permissions and what not, but it just does not work.
My expectation was to have public/dist/ populated with files.
Any help is appreciated.
UPDATE. It works, but I have doubts:
I manage to make it work by using command inside the composerfile itself.
So in my case:
command: bash -c "gulp && nodemon --delay 2 ./bin/www"
In my reasoning, gulp should be done inside the Dockerfile itself, not on the composer files. But then again, it is out of my scope of knowledge.
The Dockerfile is run at build time and will COPY all the files in your local directory into the container, then run gulp and create any files.
You then mount the local folder over the docker containers file system, pretty much overwriting what was done in the docker file with the original files, as gulp ran on the files in the container, it did not effect the original files so you are undoing the changes.
The solutions are either to do as as you have mentioned in your question (add it to the command in docker-compose.yml or run it via docker-compose exec) or write a custom entrypoint script that will run gulp and then the command, something like:
bin/entrypoint.sh
#!/bin/sh
gulp
exec "$#"
Dockerfile
FROM node:9
COPY bin/entrypoint.sh /entrypoint.sh
RUN chmod 755 /entrypoint.sh
RUN mkdir -p /usr/src/app
RUN mkdir -p /usr/src/logs
WORKDIR /usr/src/app
# GULP Installation
RUN npm install -g gulp
RUN npm install gulp
COPY package*.json /usr/src/app/
COPY .npmrc /usr/src/app/
RUN cd /usr/src/app/ && npm install && npm install -g nodemon
COPY . /usr/src/app
RUN chown -R node:node /usr/src/app && chown -R node:node /usr/src/logs
USER node
EXPOSE 3000
ENTRYPOINT ["/entrypoint.sh"]
CMD ["npm", "run-script", "start" ]
This will make your build a little less predictable though as it will run gulp each time the container starts (e.g. after every deployment) if you use the same Dockerfile in dev and production.

New package.json packages are not showing in Docker container

I am using Docker with Docker Compose and these are my files:
#DOCKERFILE
FROM mhart/alpine-node
# Create app directory
RUN mkdir -p /home/app
# Bundle app soure
COPY . /home/app
# From now on we work in /home/app
WORKDIR /home/app
# Install yarn and node modules
RUN echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-
cdn.alpinelinux.org/alpine/edge/community\nhttp://dl-
cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories \
&& apk add --no-cache yarn \
&& yarn
EXPOSE 8080
This is the docker-compose file for dev:
app:
build: .
command: yarn start:dev
environment:
NODE_ENV: development
ports:
- '8080:8080'
volumes:
- .:/home/app
- /home/app/node_modules
The problem I am having is that this setup seems to work just once because no matter which new module I add to the package.json, whenever I run docker-compose build it will not install the new package.
The reason why I am using the volumes is because nodemon would not work without .:/home/app, but if the node modules are not installed in the host then it will fail, reason why I need /home/app/node_modules. I suspect this could be the cause of my error, but I am not sure how to circumvent that.
I solved this by moving my src code inside an src directory.
This means my docker-compose.yml file now looks like this:
app:
build: .
command: yarn start:dev
environment:
NODE_ENV: development
ports:
- '8080:8080'
volumes:
- ./src:/home/app/src
Since I am not mounting the whole dir with the node_modules, new ones seem to be installed correctly.
The package.json should be copied into app directory and "npm install" should be invoked in Dockerfile before copying the bundle line.
#DOCKERFILE
FROM mhart/alpine-node
# Create app directory
RUN mkdir -p /home/app
WORKDIR /home/app
# Install app dependencies
COPY package.json /home/app
RUN npm install
# Bundle app soure
COPY . /home/app
# Install yarn and node modules
RUN echo -e 'http://dl-cdn.alpinelinux.org/alpine/edge/main\nhttp://dl-
cdn.alpinelinux.org/alpine/edge/community\nhttp://dl-
cdn.alpinelinux.org/alpine/edge/testing' > /etc/apk/repositories \
&& apk add --no-cache yarn \
&& yarn
EXPOSE 8080
If there is any new dependency registers in package.json, it should be installed when the docker build command is invoked.

Resources