Why is docker image is too big? NodeJS app - node.js

I am building my app into docker image.
My docker file:
FROM node:12-alpine
WORKDIR /usr/app
COPY ./package.json ./package.json
RUN yarn
COPY ./src ./src
COPY ./gulpfile.js ./gulpfile.js
COPY ./tsconfig.json ./tsconfig.json
RUN yarn build
RUN rm -rf ./node_modules
RUN rm -rf ./src
RUN rm -rf ./gulpfile.js
RUN rm -rf ./yarn.lock
RUN rm -rf ./package.json
RUN rm ./tsconfig.json
RUN cd dist && yarn
CMD ["node", "./dist/boot.js"]
After build I opened docker image and found my app in /user/app/dist size is 264MB (including node_modules).
But docker image has 867MB.
Why?
is there anything wrong in my dockerfile script? I am using node alpine, it should be small.

Adding lines to a Dockerfile never makes an image smaller. Because of the way an image is constructed from layers, a RUN line generally results in everything from the previous layer, plus whatever changes result from that RUN command.
As a specific example in your Dockerfile:
# Build the contents of the dist/ directory
RUN yarn build
# Keep the entire contents of the previous layer
# PLUS add markers that the node_modules directory should be removed
RUN rm -rf ./node_modules
As #jonrsharpe points out in comments, you're probably looking for a multi-stage build. The basic concept here is that a second FROM line will cause docker build to completely start over from a new base image, but then you can COPY --from= a previous stage into the final stage.
You might rebuild your existing image like so:
# Add "AS build" for later use
FROM node:12-alpine AS build
# This is exactly what you had before
WORKDIR /usr/app
COPY ./package.json ./package.json
RUN yarn
COPY ./src ./src
COPY ./gulpfile.js ./gulpfile.js
COPY ./tsconfig.json ./tsconfig.json
RUN yarn build
# Now build the actual image, starting over.
FROM node:12-alpine
WORKDIR /usr/app
COPY --from=build /usr/src/app/dist .
# but not its node_modules tree or anything else
CMD ["node", "boot.js"]

Related

Nodejs - cannot find module (Docker)

Trying to test out multistage builds in Docker for my nodejs app and I keep running into
internal/modules/cjs/loader.js:983
throw err;
^
Error: Cannot find module '/service/dist/server/server.js'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:980:15)
at Function.Module._load (internal/modules/cjs/loader.js:862:27)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
at internal/main/run_main_module.js:18:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
My Dockerfile
FROM mybaseimage as dev
WORKDIR /service
COPY src ./src
COPY package*.json ./
COPY yarn.lock ./
COPY tsconfig.json ./
# This is required to build native modules
# https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#node-gyp-alpine
RUN apk add --no-cache \
g++ \
make \
py3-pip
# Not clearing the cache here saves us time in the next step
RUN yarn install \
&& yarn compile
# Re-use the dev container to create a production-ready node_modules dir
FROM dev AS build
WORKDIR /service
RUN rm -rf /service/node_modules\
&& yarn install --production=true \
&& yarn cache clean
FROM mybaseimage AS prod
WORKDIR /service
COPY --from=build /service/dist/ .
COPY --from=build /service/node_modules/ .
# https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#handling-kernel-signals
RUN apk add --no-cache dumb-init
EXPOSE 5678
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
CMD ["node", "dist/server/server.js"]
My package.json
"serve": "node dist/server/server.js",
"start": "concurrently \"docker-compose up\" \"yarn watch\"",
My earlier working Dockerfile was
FROM mybaseimage
#set working directory
WORKDIR /service
# Copy the needed files into the container
COPY src ./src
COPY package*.json ./
COPY yarn.lock ./
COPY tsconfig.json ./
RUN apk update
RUN apk add python3
RUN echo python3 --version
RUN yarn install
RUN yarn compile
EXPOSE 5678
ENTRYPOINT ["npm", "run", "serve"]
In the final stage, you are COPYing the dist and node_modules trees from the build stage into the current directory. You need to explicitly state the subdirectory names on the right-hand side of COPY.
COPY --from=build /service/dist/ ./dist/
COPY --from=build /service/node_modules/ ./node_modules/
Also see the Dockerfile reference on COPY: since the COPY source is a directory, the contents of the directory are copied to the destination and not the directory itself. This differs from the normal Unix cp or mv commands.
You should be able to verify this running a debugging container on top of your built image; for example
docker run --rm your-image ls
should show the server subdirectory from the build dist tree, as well as all of the individual installed Node packages, all directly in the image's /service directory and not in subdirectories.

How to correctly build optimized (faster builds) for Nextjs production image using Docker

I have struggled to find good examples of images in Nextjs for docker images. The images I have found have not suited my needs. Below is the current image that I am using currently. I am trying to speed it up, I think a way in where I dont have to install twice would be more ideal.
# Install dependencies only when needed
FROM node:14.8.0-alpine3.12 AS deps
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json ./
RUN apk add git
RUN npm install
RUN mkdir /app/.next
# Rebuild the source code only when needed
FROM node:14.8.0-alpine3.12 AS builder
WORKDIR /app
COPY . .
COPY --from=deps /app/node_modules ./node_modules
RUN npm run build && npm install --production --ignore-scripts --prefer-offline
# Production image, copy all the files and run next
FROM node:14.8.0-alpine3.12 AS runner
WORKDIR /app
RUN addgroup -g 1001 -S nodejs
RUN adduser -S nextjs -u 1001
# You only need to copy next.config.js if you are NOT using the default configuration
COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/public ./public
COPY --from=builder --chown=nextjs:nodejs /app/.next ./.next
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./package.json
COPY --from=builder /app/.env.production ./
USER nextjs
EXPOSE 3000
# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry.
# ENV NEXT_TELEMETRY_DISABLED 1
CMD ["npm", "start"]

how do a dockerfile for React JS, Next js , npm project

I'm making an app with React JS, Next Js, npm and also I would have to change the .npmrc for it to run.
I don't know how I could make a DockerFile for these technologies and at the same time this dockerfile has to change the .npmrc
my docker file
FROM node:lts as dependencies
WORKDIR /emercore-arg-manager
COPY package.json yarn.lock ./
RUN echo "#lala-lalal:registry=https://npm.pkg.github.com/" >> ~/.npmrc
RUN echo "//npm.pkg.github.com/:_authToken=asdasdasdasdasdsad" >> ~/.npmrc
RUN echo "//registry.npmjs.org/:_authToken=assdasdasdasdsaasd" >> ~/.npmrc
RUN yarn install --frozen-lockfile
FROM node:lts as builder
WORKDIR /emercore-arg-manager
COPY . .
COPY --from=dependencies /emercore-arg-manager/node_modules ./node_modules
RUN yarn build
FROM node:lts as runner
WORKDIR /emercore-arg-manager
ENV NODE_ENV production
# If you are using a custom next.config.js file, uncomment this line.
# COPY --from=builder /my-project/next.config.js ./
COPY --from=builder /emercore-arg-manager/public ./public
COPY --from=builder /emercore-arg-manager/.next ./.next
COPY --from=builder /emercore-arg-manager/node_modules ./node_modules
COPY --from=builder /emercore-arg-manager/package.json ./package.json
EXPOSE 3000
CMD ["yarn", "start:dev"]
It does not work for me, and I think it is a lot of content for a dockerfile with these technologies, could someone help me to put together a shorter one and make it work?
The command that i use in my deskpot is yarn install and yarn start:dev (and its working)

Multi stage Dockerfile leads to running out of space

As my code (nodeJS-application) is changing more often than the (npm) dependencies do, I've tried to build something like a cache in my CI.
I'm using a multi-stage Dockerfile. In that I run npm install for all, and only, prod dependencies. Later they are copied to the final image so that it is much smaller. Great.
Also the build get super fast if no dependency has been changed.
However, over time the hd gets full so I have to run docker prune ... to get the space back. But, when I do this, the cache is gone.
So if I run a prune after each pipeline in my CI, I do not get the 'cache functionality' of the multi-stage Dockerfile.
### 1. Build
FROM node:10.13 AS build
WORKDIR /home/node/app
COPY ./package*.json ./
COPY ./.babelrc ./
RUN npm set progress=false \
&& npm config set depth 0 \
&& npm install --only=production --silent \
&& cp -R node_modules prod_node_modules
RUN npm install --silent
COPY ./src ./src
RUN ./node_modules/.bin/babel ./src/ -d ./dist/ --copy-files
### 2. Run
FROM node:10.13-alpine
RUN apk --no-cache add --virtual \
builds-deps \
build-base \
python
WORKDIR /home/node/app
COPY --from=build /home/node/app/prod_node_modules ./node_modules
COPY --from=build /home/node/app/dist .
EXPOSE 3000
ENV NODE_ENV production
CMD ["node", "app.js"]
If your CI system lets you have multiple docker build steps, you could split this into two Dockerfiles.
# Dockerfile.dependencies
# docker build -f Dockerfile.dependencies -t me/dependencies .
FROM node:10.13
...
RUN npm install
# Dockerfile
# docker build -t me/application .
FROM me/dependencies:latest AS build
COPY ./src ./src
RUN ./node_modules/.bin/babel ./src/ -d ./dist/ --copy-files
FROM node:10.13-alpine
...
CMD ["node", "app.js"]
If you do this, then you can delete unused images after each build:
docker image prune
The most recent build of the dependencies image will have a label, so it won't be "dangling" and won't appear in the image listing. On each build its label will get "taken from" the previous build (if it changed) and so this sequence will clean up previous builds. This will also delete the "build" images, though as you note if anything changed to trigger a build it will probably be in the src tree and so forcing a rebuild there is reasonable.
In this specific circumstance, just using the latest tag is appropriate. If the final built images have some more unique tag (based on a version number or timestamp, say) and they're stacking up then you might need to do some more creative filtering of that image list to clean them up.

How to fix "Directory not empty" on move during docker build?

I have a working Dockerfile for a node application:
FROM node:8.8
ENV TERM=xterm-color NPM_CONFIG_LOGLEVEL=warn PATH="$PATH:/usr/src/app/node_modules/.bin/"
VOLUME ["/logs"]
WORKDIR /tmp/node
ADD package.json yarn.lock .npmrc ./
RUN yarn install --frozen-lockfile --ignore-platform --ignore-engines --quiet
WORKDIR /usr/src/app
ADD . /usr/src/app
RUN mv /tmp/node/* ./ && tsc && webpack
CMD ["node", "/usr/src/app/server"]
I wanted to re-created the caching behavior for the node_modules during build, hence I have updated the Dockerfile for another project to look very similar.
FROM node:9-alpine
WORKDIR /tmp/node
ADD package.json yarn.lock ./
RUN yarn install --frozen-lockfile --ignore-platform --ignore-engines --quiet
WORKDIR /app
ADD . /app/
RUN mv /tmp/node/* ./
EXPOSE 1337
CMD ["yarn", "start"]
Yet for that Dockerfile I get an unexpected error during:
$ docker build .
...
Step 7/9 : RUN mv /tmp/node/* ./
---> Running in 51543827cd89
mv: can't rename '/tmp/node/node_modules': Directory not empty
The command '/bin/sh -c mv /tmp/node/* ./' returned a non-zero code: 1
Why doesn't the mv command work here?
When you run docker build ., the current directory gets passed as its context. It's most likely the case that you have either run the yarn install command on your host already which is why you alrady have a
/app/node_modules
This is why it cannot be moved, as it already exists.
To avoid passing the folder along within the context, you can add:
node_modules/
in your .dockerignore file.

Resources