Docker is not loading Node image from cache - node.js

I'm using docker-compose in my local environment for development. I have 4 docker containers, 3 of them based on docker-node.
When I run docker-compose up --build, I see most of the building steps are loaded from cache, so they usually run really fast. But since a couple of days ago, I noticed they began to take a very long time to build (around 10 minutes in total), even if nothing had changed.
I also noticed that all the building steps seem to be loaded from cache, except for the first one: FROM node:8. And indeed, it always takes the longest to print the first step. Here's the complete output:
Building mongo-seed
Step 1/10 : FROM node:8
---> d575940ea42b
Step 2/10 : ENV NODE_ENV=development
---> Using cache
---> 64584d7c28d1
Step 3/10 : WORKDIR /seeder
---> Using cache
---> 2481554b20cc
Step 4/10 : COPY ./server/app/models ./app/models
---> Using cache
---> ad6c76419ffd
Step 5/10 : COPY ./server/config ./config
---> Using cache
---> 379dd9e19e67
Step 6/10 : COPY ./server/util ./util
---> Using cache
---> 1e6db5498a1a
Step 7/10 : COPY ./server/seed ./seed
---> Using cache
---> 659f531ef78d
Step 8/10 : COPY ./server/package.json ./package.json
---> Using cache
---> 4eb54f2548b9
Step 9/10 : RUN npm i mongoose
---> Using cache
---> c34967921c69
Step 10/10 : CMD npm run seed
---> Using cache
---> 0274c6635c1d
Successfully built 0274c6635c1d
Successfully tagged javorosas_mongo-seed:latest
Building server
^[Step 1/17 : FROM node:8
---> d575940ea42b
Step 2/17 : RUN apt-get update
---> Using cache
---> 4cb3797b4737
Step 3/17 : RUN apt-get install -y graphicsmagick
---> Using cache
---> a33dcf53e585
Step 4/17 : RUN npm i -g webpack node-gyp nodemon
---> Using cache
---> c961ca565c54
Step 5/17 : ENV NODE_ENV=development
---> Using cache
---> c63d1a245873
Step 6/17 : WORKDIR /server
---> Using cache
---> 7ac6181902ca
Step 7/17 : COPY ./server/package.json .
---> Using cache
---> ebba732f0210
Step 8/17 : COPY ./common ../common
---> Using cache
---> 47a9e631cc57
Step 9/17 : RUN npm install --quiet && npm cache clean -f
---> Using cache
---> 8d57a963d1ac
Step 10/17 : COPY ./server/constants ./constants
---> Using cache
---> a6a8884a1343
Step 11/17 : COPY ./server/util ./util
---> Using cache
---> dfd9de1db59e
Step 12/17 : COPY ./server/webpack.ssr.js .
---> Using cache
---> 547282293bc6
Step 13/17 : COPY ./server/src ./src
---> Using cache
---> e7821c59d012
Step 14/17 : RUN npm run build
---> Using cache
---> 184527650563
Step 15/17 : COPY . .
---> 8be425af57c7
Step 16/17 : EXPOSE 2018
---> Running in c662570856a5
Removing intermediate container c662570856a5
---> 1eb434d7e0ea
Step 17/17 : CMD npm run start:dev
---> Running in 613c7064436e
Removing intermediate container 613c7064436e
---> f37d511ffc78
Successfully built f37d511ffc78
Successfully tagged javorosas_server:latest
Building client
Step 1/10 : FROM node:8
---> d575940ea42b
Step 2/10 : RUN npm i -g webpack
---> Using cache
---> b8428ab7c232
Step 3/10 : ENV NODE_ENV=development
---> Using cache
---> bbfbbb480a46
Step 4/10 : WORKDIR /client
---> Using cache
---> 2203bdaa2cea
Step 5/10 : COPY ./client/package.json .
---> Using cache
---> fe5ead356992
Step 6/10 : COPY ./common ../common
---> Using cache
---> db9c274937cc
Step 7/10 : RUN npm install --quiet && npm cache clean -f
---> Using cache
---> 16be06b426b3
Step 8/10 : COPY . .
---> c96343b67244
Step 9/10 : EXPOSE 2017
---> Running in 7460e5f1d0c6
Removing intermediate container 7460e5f1d0c6
---> 8436f916f585
Step 10/10 : CMD npm run start
---> Running in fbcf8af08cbf
Removing intermediate container fbcf8af08cbf
---> 47958626245f
Successfully built 47958626245f
Successfully tagged javorosas_client:latest
javorosas_mongo_1 is up-to-date
Starting javorosas_mongo-seed_1 ... done
Recreating javorosas_server_1 ... done
Recreating javorosas_client_1 ... done
Attaching to javorosas_mongo_1, javorosas_mongo-seed_1,
javorosas_server_1, javorosas_client_1
Any ideas on why it could have started to slow down?

Related

Express js npm install fails reason: getaddrinfo EAI_AGAIN

I have a Node.JS / Express.JS server-side program, which I run inside a Node.JS docker container. It was working for the last ~2 years, now it produces a strange error upon installing.
I use docker-compose to install / run the program.
The Dockerfile:
FROM node:10
ARG PORT
ENV PORT ${PORT}
# Create app directory
ADD backend /app
WORKDIR /app
RUN npm ci --only=production
EXPOSE $PORT
CMD ["npm", "run", "start-prod"]
When I try to install it (with docker-compose, it fails every time, with this message:
Building backend
Step 1/10 : FROM node:10
---> a8441ebf4e4d
Step 2/10 : ARG PORT
---> Using cache
---> d985b4141806
Step 3/10 : ENV PORT ${PORT}
---> Using cache
---> e053f5bb49fa
Step 4/10 : ADD backend /app
---> Using cache
---> 409d0ab160ea
Step 5/10 : WORKDIR /app
---> Using cache
---> b7aad109486b
Step 6/10 : RUN cp environment-prod-docker.js environment.js
---> Using cache
---> 218df5ff1cff
Step 7/10 : RUN cat environment.js
---> Using cache
---> 87965e20e57e
Step 8/10 : RUN npm ci --only=production
---> Running in 9352da8c367e
npm WARN prepare removing existing node_modules/ before installation
npm ERR! code EAI_AGAIN
npm ERR! errno EAI_AGAIN
npm ERR! request to https://registry.npmjs.org/bcryptjs/-/bcryptjs-2.4.3.tgz failed, reason: getaddrinfo EAI_AGAIN registry.npmjs.org registry.npmjs.org:443
I have read that it might be possible that this is a proxy error. I have tried to change to my home wifi, my home cable, and mobile data: still produced the same error. If I copy the link, I can download the said .tgz file from the browser.
This problem disappeared after two days (same network, same computer etc.)
It is probably related to what the comments indicated.
Simple DNS issue, check /etc/resolv.conf and add this:
"nameserver 8.8.8.8"

Running a linux command in Dockerfile throwing error

This is my Dockerfile content and it keeps throwing me error in running this statement. Any idea what am I doing wrong. I just want to dump ENV VARS into a file for React build on the container.
FROM node:12
WORKDIR /usr/src/app
COPY . .
RUN printenv | grep REACT_APP_ > client/.env
# RUN ["printenv", "|", "grep", "REACT_APP_", ">", "client/.env"]
RUN npm run setup
RUN npm run build:all
EXPOSE 3005
CMD [ "npm", "run", "start:prod" ]
Error: ERROR: Service '<name>' failed to build: The command 'printenv | grep REACT_APP_ > client/.env' returned a non-zero code: 1
I have spent many hours with no luck. Help appreciated.
So there's a couple things to consider:
Make sure the client dir exists. It doesn't come in the node:12 image by itself
Make sure the env has a variable that will allow the grep REACT_APP_ to find something. If it does not, grep will return 1 and the docker image build will halt. 1 is normal return code when grep finds nothing, so it's not an error (and thus nothing more is printed). But the docker build treats it as an error since it's nonzero, and thus the build stops.
Also to read on EXIT CODES on man grep page,
EXIT STATUS Normally the exit status is 0 if a line is selected, 1 if
no lines were selected, and 2 if an error occurred.
Here's a test I did that passes (slightly modified, just based on the considerations above):
FROM node:12
WORKDIR /usr/src/app
COPY . .
RUN mkdir client
ENV REACT_APP_1 1
RUN printenv | grep REACT_APP_ > client/.env
RUN cat client/.env
The output I get is this:
$ docker build -t test .
Sending build context to Docker daemon 2.048kB
Step 1/7 : FROM node:12
---> 28faf336034d
Step 2/7 : WORKDIR /usr/src/app
---> Running in 753293fa6257
Removing intermediate container 753293fa6257
---> 3a04798b1b9f
Step 3/7 : COPY . .
---> 3dd0d465a6e2
Step 4/7 : RUN mkdir client
---> Running in d513df2a0a34
Removing intermediate container d513df2a0a34
---> d46aa5200ae7
Step 5/7 : ENV REACT_APP_1 1
---> Running in af81940a90fb
Removing intermediate container af81940a90fb
---> 6169ad694a4d
Step 6/7 : RUN printenv | grep REACT_APP_ > client/.env
---> Running in 365020eeb2e5
Removing intermediate container 365020eeb2e5
---> b6ef574c48c8
Step 7/7 : RUN cat client/.env
---> Running in a6a69d6ba6c2
REACT_APP_1=1
Removing intermediate container a6a69d6ba6c2
---> 0814306133f0
Successfully built 0814306133f0
Successfully tagged test:latest

Docker build showing invalid containerPort

I need help in debugging this image build:
My dockerfile is :
FROM node:latest
WORKDIR /app
COPY . .
ENV PORT = 3000
RUN npm install
EXPOSE $PORT
ENTRYPOINT [ "node", "app.js" ]
and when I run sudo docker build -t uddeshya/node1 . to build the image, the log shows the following:
Sending build context to Docker daemon 2.009MB
Step 1/7 : FROM node:latest
---> dcda6cd5e439
Step 2/7 : WORKDIR /app
---> Using cache
---> 9450405b180f
Step 3/7 : COPY . .
---> Using cache
---> 91689830af35
Step 4/7 : ENV PORT = 3000
---> Using cache
---> d99d55d0ae81
Step 5/7 : RUN npm install
---> Using cache
---> dac4854ec168
Step 6/7 : EXPOSE $PORT
Invalid containerPort: =
The error point is at "EXPOSE $PORT", how do I fix this?
The syntax of ENV should be
ENV PORT=3000
That is the build throw error as it receives = because of spaces.
Step 3/6 : ENV PORT = 3000
---> Using cache
---> 8edc1281a96c
Step 4/6 : RUN echo ${PORT}
---> Running in 090a69369847
= 3000
without spaces it should be
Step 3/6 : ENV PORT=3000
---> Using cache
---> dc29398a0ca6
Step 4/6 : RUN echo ${PORT}
---> Running in 9fc5d9b07342
3000
then yon can verify
docker inspect your_image
you can see
"ExposedPorts": {
"3000/tcp": {}
}

Unable to run NodeJS microservices through Docker containers

I was getting started with Docker, created couple of tiny Express(NodeJS) services.
Plan is to run the microservices inside Docker containers and then establish a inter-communication between them using Docker Compose service names.
Here is the Github repo of this simple project. Am able to build images with below commands :
cd books
docker build -t node-micro/books .
cd auth
docker build -t node-micro/auth .
Commands to start containers :
docker run -d -p 6677:6677 node-micro/auth
docker run -d -p 7766:7766 node-micro/books
But when i hit below URL's there is no response, which was working fine couple of day's before :
http://localhost:6677/
http://localhost:7766/
And have no clue what's happening with docker compose. No luck on accessing same URL's as mentioned above after stoping all containers, delete all images & ran this command :
docker-compose up -d
Need some help on bringing up the containers individually and also through docker-compose.
I can see in each of your micro-service, your application is running on ports 3000 in the container but you are exposing 7766 and 6677 in your docker-compose.yml
Please check the below docker-compose.yml
version: '3'
services:
books:
build: './books'
ports:
- "7766:3000"
depends_on:
- auth
auth:
build: './auth'
ports:
- "6677:3005"
and then run the below command
docker-compose up --build
--build will build the images as well.
Then, you should be able to access the service
http://localhost:6677/
http://localhost:7766/
Output
docker-compose up --build
Creating network "node_microservices_default" with the default driver
Building auth
Step 1/7 : FROM node:10-alpine
---> 0aa7bb41deca
Step 2/7 : WORKDIR /usr
---> Running in a1dc67b70538
Removing intermediate container a1dc67b70538
---> 5fc74fc80a14
Step 3/7 : COPY package*.json ./
---> 454f1b7aba87
Step 4/7 : RUN npm install
---> Running in a24eea8b79d4
npm WARN auth#1.0.0 No description
npm WARN auth#1.0.0 No repository field.
added 50 packages from 37 contributors and audited 50 packages in 8.58s
found 0 vulnerabilities
Removing intermediate container a24eea8b79d4
---> 31b31ff4516e
Step 5/7 : COPY . .
---> 1eeaa8e70300
Step 6/7 : EXPOSE 3000
---> Running in fc798167dbcd
Removing intermediate container fc798167dbcd
---> 4d964d25c099
Step 7/7 : CMD ["npm", "start"]
---> Running in 3c28d92f9ef6
Removing intermediate container 3c28d92f9ef6
---> 514f68d11d7c
Successfully built 514f68d11d7c
Successfully tagged node_microservices_auth:latest
Building books
Step 1/7 : FROM node:10-alpine
---> 0aa7bb41deca
Step 2/7 : WORKDIR /usr
---> Using cache
---> 5fc74fc80a14
Step 3/7 : COPY package*.json ./
---> 56addb6c75a5
Step 4/7 : RUN npm install
---> Running in 4864fb7a171c
npm WARN books#1.0.0 No description
npm WARN books#1.0.0 No repository field.
added 50 packages from 37 contributors and audited 50 packages in 5.111s
found 0 vulnerabilities
Removing intermediate container 4864fb7a171c
---> 82bb2cd54357
Step 5/7 : COPY . .
---> 12893a93e82e
Step 6/7 : EXPOSE 3000
---> Running in 1301e29dbd52
Removing intermediate container 1301e29dbd52
---> c26948ebcb3b
Step 7/7 : CMD ["npm", "start"]
---> Running in db948866a121
Removing intermediate container db948866a121
---> 703b901d7bc4
Successfully built 703b901d7bc4
Successfully tagged node_microservices_books:latest
Creating node_microservices_auth_1 ... done
Creating node_microservices_books_1 ... done
Attaching to node_microservices_auth_1, node_microservices_books_1
auth_1 |
auth_1 | > auth#1.0.0 start /usr
auth_1 | > node index.js
auth_1 |
auth_1 | Running on port 3005
auth_1 | --------------------------
books_1 |
books_1 | > books#1.0.0 start /usr
books_1 | > node index.js
books_1 |
books_1 | Running on port 3000
books_1 | --------------------------
You are exposing from both Dockerfiles the port 3000. Replace the port for each microservice in docker-compose.yml file.
- "7766:3000"
- "6677:3000"
The mapping for ports is wrong. In both Dockerfile you are exposing the port 3000.
So you must to map the ports 6677 and 7766 to the exposed port on the Dockerfile.
To fix this, on your docker-compose.yml you must to config ports like this:
version: '3'
services:
books:
build: './books'
ports:
- "7766:3000"
depends_on:
- auth
auth:
build: './auth'
ports:
- "6677:3000"

Autotesting docker images with docker-compose.test : missing environment variables

I have a Node/Express app and I am using https://www.npmjs.com/package/dotenv-safe to include environment variables for dev and testing.
I am pushing my Docker repository to https://cloud.docker.com/app/gkatsanos/ and during AutoTesting (I have a docker-compose.test.yml), It seems that the environment variables are not found:
Successfully built 69f35563f12e
Successfully tagged gkatsanos/server:latest
Starting Test in docker-compose.test.yml...
Building sut
Step 1/10 : FROM node:8-alpine
---> 4db2697ce114
Step 2/10 : EXPOSE 3000
---> Using cache
---> ef9e0a8a09e1
Step 3/10 : ARG NODE_ENV
---> Using cache
---> cc6143bf9bee
Step 4/10 : ENV NODE_ENV $NODE_ENV
---> Using cache
---> 6477a9e9657f
Step 5/10 : RUN mkdir /app
---> Using cache
---> e9fff66cf3da
Step 6/10 : WORKDIR /app
---> Using cache
---> da82362255c6
Step 7/10 : ADD package.json yarn.lock seed.js /app/
---> Using cache
---> 97a842faeb2a
Step 8/10 : RUN yarn --pure-lockfile
---> Using cache
---> 6745c0d8c64c
Step 9/10 : ADD . /app
---> Using cache
---> 06d46eb4a57b
Step 10/10 : CMD yarn start
---> Using cache
---> 69f35563f12e
Successfully built 69f35563f12e
Successfully tagged b5wqkysdhyuqf8uz4kyreyn_sut:latest
Creating network "b5wqkysdhyuqf8uz4kyreyn_default" with the default driver
Creating b5wqkysdhyuqf8uz4kyreyn_sut_1 ...
Creating b5wqkysdhyuqf8uz4kyreyn_sut_1
Creating b5wqkysdhyuqf8uz4kyreyn_sut_1 ... done
yarn run v1.3.2
$ NODE_ENV=test nyc --reporter=html --reporter=text mocha --timeout 20000 --recursive src/tests
/app/node_modules/dotenv-safe/index.js:37
throw new MissingEnvVarsError(allowEmptyValues, options.path || '.env', sample, missing, dotenvResult.error);
Locally the docker building and running works.
That's my dockerfile:
FROM node:8-alpine
EXPOSE 3000
ARG NODE_ENV
ENV NODE_ENV $NODE_ENV
RUN mkdir /app
WORKDIR /app
ADD package.json yarn.lock seed.js /app/
RUN yarn --pure-lockfile
ADD . /app
CMD ["yarn", "start"]
and my docker-compose.test.yml:
version: "3"
services:
sut:
build: .
command: yarn test
By the way, cloud.docker.com has a place in its UI that allows for environment variables to be added:

Resources