I have a Node/Express app and I am using https://www.npmjs.com/package/dotenv-safe to include environment variables for dev and testing.
I am pushing my Docker repository to https://cloud.docker.com/app/gkatsanos/ and during AutoTesting (I have a docker-compose.test.yml), It seems that the environment variables are not found:
Successfully built 69f35563f12e
Successfully tagged gkatsanos/server:latest
Starting Test in docker-compose.test.yml...
Building sut
Step 1/10 : FROM node:8-alpine
---> 4db2697ce114
Step 2/10 : EXPOSE 3000
---> Using cache
---> ef9e0a8a09e1
Step 3/10 : ARG NODE_ENV
---> Using cache
---> cc6143bf9bee
Step 4/10 : ENV NODE_ENV $NODE_ENV
---> Using cache
---> 6477a9e9657f
Step 5/10 : RUN mkdir /app
---> Using cache
---> e9fff66cf3da
Step 6/10 : WORKDIR /app
---> Using cache
---> da82362255c6
Step 7/10 : ADD package.json yarn.lock seed.js /app/
---> Using cache
---> 97a842faeb2a
Step 8/10 : RUN yarn --pure-lockfile
---> Using cache
---> 6745c0d8c64c
Step 9/10 : ADD . /app
---> Using cache
---> 06d46eb4a57b
Step 10/10 : CMD yarn start
---> Using cache
---> 69f35563f12e
Successfully built 69f35563f12e
Successfully tagged b5wqkysdhyuqf8uz4kyreyn_sut:latest
Creating network "b5wqkysdhyuqf8uz4kyreyn_default" with the default driver
Creating b5wqkysdhyuqf8uz4kyreyn_sut_1 ...
Creating b5wqkysdhyuqf8uz4kyreyn_sut_1
Creating b5wqkysdhyuqf8uz4kyreyn_sut_1 ... done
yarn run v1.3.2
$ NODE_ENV=test nyc --reporter=html --reporter=text mocha --timeout 20000 --recursive src/tests
/app/node_modules/dotenv-safe/index.js:37
throw new MissingEnvVarsError(allowEmptyValues, options.path || '.env', sample, missing, dotenvResult.error);
Locally the docker building and running works.
That's my dockerfile:
FROM node:8-alpine
EXPOSE 3000
ARG NODE_ENV
ENV NODE_ENV $NODE_ENV
RUN mkdir /app
WORKDIR /app
ADD package.json yarn.lock seed.js /app/
RUN yarn --pure-lockfile
ADD . /app
CMD ["yarn", "start"]
and my docker-compose.test.yml:
version: "3"
services:
sut:
build: .
command: yarn test
By the way, cloud.docker.com has a place in its UI that allows for environment variables to be added:
Related
I am trying to make a simple react app in docker-compose. I am using this reference What I have done is run npx create-react-app frontend to generate the default react app. Then I added the Dockerfile. This is all in a directory frontend.
#Dockerfile
FROM node:14.9
# set working directory
WORKDIR /app
# add `/app/node_modules/.bin` to $PATH
ENV PATH /app/node_modules/.bin:$PATH
# install app dependencies
COPY package.json ./
COPY package-lock.json ./
RUN npm install --silent
RUN npm install react-scripts#3.4.1 -g --silent
# add app
COPY . ./
# start app
CMD ["npm", "start"]
In the directory above, I have my docker-compose
#docker-compose.yml
version: '3.7'
services:
frontend:
container_name: frontend
build: ./frontend
volumes:
- './frontend:/app'
- '/app/node_modules'
ports:
- 3000:3000
environment:
- CHOKIDAR_USEPOLLING=true
I run docker-compose up --build and after the normal build process I get this message.
docker-compose up --build
Building frontend
Step 1/9 : FROM node:14.9
---> 1b2c72215052
Step 2/9 : WORKDIR /app
---> Using cache
---> 2bab04404275
Step 3/9 : ENV PATH /app/node_modules/.bin:$PATH
---> Using cache
---> ff4ce5f5ec47
Step 4/9 : COPY package.json ./
---> Using cache
---> b1d25c0b6c05
Step 5/9 : COPY package-lock.json ./
---> Using cache
---> 5b829feaf00d
Step 6/9 : RUN npm install --silent
---> Using cache
---> 835367c47253
Step 7/9 : RUN npm install react-scripts#3.4.1 -g --silent
---> Using cache
---> 015ccb2db237
Step 8/9 : COPY . ./
---> Using cache
---> e4a5285339b5
Step 9/9 : CMD ["npm", "start"]
---> Using cache
---> 3f91b16d34d6
Successfully built 3f91b16d34d6
Successfully tagged projectname_frontend:latest
Recreating frontend ... done
Attaching to frontend
frontend |
frontend | > frontend#0.1.0 start /app
frontend | > react-scripts start
frontend |
frontend | ℹ ｢wds｣: Project is running at http://172.29.0.2/
frontend | ℹ ｢wds｣: webpack output is served from
frontend | ℹ ｢wds｣: Content not from webpack is served from /app/public
frontend | ℹ ｢wds｣: 404s will fallback to /
frontend | Starting the development server...
frontend |
frontend exited with code 0
It seems the container just exits without an error message. I have no idea what is causing this.
Docker version 19.03.12, build 48a66213fe
It is worth noting that I have been able to build my react apps successfully without the solutions below for the past few months. The need for the commands given in the solution has only recently become an issue for me.
I think I solved the issue. Adding stdin_open: true to the docker-compose.yml was half the solution. I also added command: npm start. After this, the container stopped exiting. I thought that the command in the Dockerfile would be sufficient. It seems to be working now.
docker-compose.yml
version: '3.7'
services:
frontend:
container_name: frontend
build: ./frontend
volumes:
- './:/app'
- '/app/node_modules'
ports:
- 3000:3000
stdin_open: true
environment:
- CHOKIDAR_USEPOLLING=true
command: npm start
Dockerfile
FROM node:14.9
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD [ "npm", "start" ]
This simple Dockerfile always works for me:
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8083
CMD [ "npm", "start" ]
I also like to add a stdin_open: true to my container in docker-compose.yml.
version: "3.8"
services:
frontend:
build: .
container_name: your-container-name
ports: 4300:4300
volumes:
- ./src:/app/src
- ./public/assets:/app/public/assets
- /app/node_modules
stdin_open: true
tty: true
env-file:
- location/to/your/env-file
I can't comment to Seth Faulkner's solution because of lacking reputation.
I added the stdin_open: true to the docker-compose at first and it still gives the exit with code 0 issue.
However, adding the command: yarn run start in the docker-compose solve the problem for me
stdin_open: true
command: yarn run start
I need help in debugging this image build:
My dockerfile is :
FROM node:latest
WORKDIR /app
COPY . .
ENV PORT = 3000
RUN npm install
EXPOSE $PORT
ENTRYPOINT [ "node", "app.js" ]
and when I run sudo docker build -t uddeshya/node1 . to build the image, the log shows the following:
Sending build context to Docker daemon 2.009MB
Step 1/7 : FROM node:latest
---> dcda6cd5e439
Step 2/7 : WORKDIR /app
---> Using cache
---> 9450405b180f
Step 3/7 : COPY . .
---> Using cache
---> 91689830af35
Step 4/7 : ENV PORT = 3000
---> Using cache
---> d99d55d0ae81
Step 5/7 : RUN npm install
---> Using cache
---> dac4854ec168
Step 6/7 : EXPOSE $PORT
Invalid containerPort: =
The error point is at "EXPOSE $PORT", how do I fix this?
The syntax of ENV should be
ENV PORT=3000
That is the build throw error as it receives = because of spaces.
Step 3/6 : ENV PORT = 3000
---> Using cache
---> 8edc1281a96c
Step 4/6 : RUN echo ${PORT}
---> Running in 090a69369847
= 3000
without spaces it should be
Step 3/6 : ENV PORT=3000
---> Using cache
---> dc29398a0ca6
Step 4/6 : RUN echo ${PORT}
---> Running in 9fc5d9b07342
3000
then yon can verify
docker inspect your_image
you can see
"ExposedPorts": {
"3000/tcp": {}
}
I was getting started with Docker, created couple of tiny Express(NodeJS) services.
Plan is to run the microservices inside Docker containers and then establish a inter-communication between them using Docker Compose service names.
Here is the Github repo of this simple project. Am able to build images with below commands :
cd books
docker build -t node-micro/books .
cd auth
docker build -t node-micro/auth .
Commands to start containers :
docker run -d -p 6677:6677 node-micro/auth
docker run -d -p 7766:7766 node-micro/books
But when i hit below URL's there is no response, which was working fine couple of day's before :
http://localhost:6677/
http://localhost:7766/
And have no clue what's happening with docker compose. No luck on accessing same URL's as mentioned above after stoping all containers, delete all images & ran this command :
docker-compose up -d
Need some help on bringing up the containers individually and also through docker-compose.
I can see in each of your micro-service, your application is running on ports 3000 in the container but you are exposing 7766 and 6677 in your docker-compose.yml
Please check the below docker-compose.yml
version: '3'
services:
books:
build: './books'
ports:
- "7766:3000"
depends_on:
- auth
auth:
build: './auth'
ports:
- "6677:3005"
and then run the below command
docker-compose up --build
--build will build the images as well.
Then, you should be able to access the service
http://localhost:6677/
http://localhost:7766/
Output
docker-compose up --build
Creating network "node_microservices_default" with the default driver
Building auth
Step 1/7 : FROM node:10-alpine
---> 0aa7bb41deca
Step 2/7 : WORKDIR /usr
---> Running in a1dc67b70538
Removing intermediate container a1dc67b70538
---> 5fc74fc80a14
Step 3/7 : COPY package*.json ./
---> 454f1b7aba87
Step 4/7 : RUN npm install
---> Running in a24eea8b79d4
npm WARN auth#1.0.0 No description
npm WARN auth#1.0.0 No repository field.
added 50 packages from 37 contributors and audited 50 packages in 8.58s
found 0 vulnerabilities
Removing intermediate container a24eea8b79d4
---> 31b31ff4516e
Step 5/7 : COPY . .
---> 1eeaa8e70300
Step 6/7 : EXPOSE 3000
---> Running in fc798167dbcd
Removing intermediate container fc798167dbcd
---> 4d964d25c099
Step 7/7 : CMD ["npm", "start"]
---> Running in 3c28d92f9ef6
Removing intermediate container 3c28d92f9ef6
---> 514f68d11d7c
Successfully built 514f68d11d7c
Successfully tagged node_microservices_auth:latest
Building books
Step 1/7 : FROM node:10-alpine
---> 0aa7bb41deca
Step 2/7 : WORKDIR /usr
---> Using cache
---> 5fc74fc80a14
Step 3/7 : COPY package*.json ./
---> 56addb6c75a5
Step 4/7 : RUN npm install
---> Running in 4864fb7a171c
npm WARN books#1.0.0 No description
npm WARN books#1.0.0 No repository field.
added 50 packages from 37 contributors and audited 50 packages in 5.111s
found 0 vulnerabilities
Removing intermediate container 4864fb7a171c
---> 82bb2cd54357
Step 5/7 : COPY . .
---> 12893a93e82e
Step 6/7 : EXPOSE 3000
---> Running in 1301e29dbd52
Removing intermediate container 1301e29dbd52
---> c26948ebcb3b
Step 7/7 : CMD ["npm", "start"]
---> Running in db948866a121
Removing intermediate container db948866a121
---> 703b901d7bc4
Successfully built 703b901d7bc4
Successfully tagged node_microservices_books:latest
Creating node_microservices_auth_1 ... done
Creating node_microservices_books_1 ... done
Attaching to node_microservices_auth_1, node_microservices_books_1
auth_1 |
auth_1 | > auth#1.0.0 start /usr
auth_1 | > node index.js
auth_1 |
auth_1 | Running on port 3005
auth_1 | --------------------------
books_1 |
books_1 | > books#1.0.0 start /usr
books_1 | > node index.js
books_1 |
books_1 | Running on port 3000
books_1 | --------------------------
You are exposing from both Dockerfiles the port 3000. Replace the port for each microservice in docker-compose.yml file.
- "7766:3000"
- "6677:3000"
The mapping for ports is wrong. In both Dockerfile you are exposing the port 3000.
So you must to map the ports 6677 and 7766 to the exposed port on the Dockerfile.
To fix this, on your docker-compose.yml you must to config ports like this:
version: '3'
services:
books:
build: './books'
ports:
- "7766:3000"
depends_on:
- auth
auth:
build: './auth'
ports:
- "6677:3000"
I made a simple hello world node.js application with the dockerfile
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD node index.js
If I change the CMD to RUN in my dockerfile it still works.
It is documented in dockerfile to use CMD as it will then start the node server upon the running of the container.
I would like to know what will happen underhood if I use RUN cmd instead of CMD.
Basically what happens if I make a docker image which itself is in running state.
RUN will execute a command during the build process. CMD is used as the default command when executing a container, as opposed to building. If you run node index.js in a RUN instruction, your build will never finish and you don't have a container to share with others.
Refer to the dockerfile documentation for more detail: RUN and CMD.
Relevant bits from that documentation:
The RUN instruction will execute any commands in a new layer on top of the current image and commit the results. The resulting committed image will be used for the next step in the Dockerfile.
The main purpose of a CMD is to provide defaults for an executing container.
EDIT: using OP's index.json, package.json, and Dockerfile files, the docker image build does not complete when using RUN node index.js and does complete (as expected) when using CMD node index.js.
Contents of index.js:
//Load express module with `require` directive
var express = require('express')
var app = express()
//Define request response in root URL (/)
app.get('/', function (req, res) {
res.send('Hello World!')
})
//Launch listening server on port 8081
app.listen(8080, function () {
console.log('app listening on port 8080!')
})
Contents of package.json:
{
"name": "dummy_nodejs_app",
"version": "1.0.0",
"description": "Node.js on Docker",
"author": "Debojit",
"main": "server.js",
"scripts": {
"start": "node server.js"
},
"dependencies": {
"express": "^4.16.1"
}
}
When using the Dockerfile as follows:
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
RUN node index.js
then the build hangs. Here is the output:
jakub#dash:/tmp/test-node$ docker build -t test .
Sending build context to Docker daemon 4.096kB
Step 1/7 : FROM node:10
---> d5680e53a228
Step 2/7 : WORKDIR /usr/src/app
---> Using cache
---> a4b4547833e5
Step 3/7 : COPY package*.json ./
---> Using cache
---> 2b19cc3e48a3
Step 4/7 : RUN npm install
---> Using cache
---> fe1f1e72d17d
Step 5/7 : COPY . .
---> eb6fe0e3d1a7
Step 6/7 : EXPOSE 8080
---> Running in e573b923fcb2
Removing intermediate container e573b923fcb2
---> b3590153eed7
Step 7/7 : RUN node index.js
---> Running in 08b408e6e6f3
app listening on port 8080!
This hangs indefinitely.
When using the Dockerfile
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8080
CMD node index.js
the build output is:
jakub#dash:/tmp/test-node$ docker build -t test .
Sending build context to Docker daemon 4.096kB
Step 1/7 : FROM node:10
---> d5680e53a228
Step 2/7 : WORKDIR /usr/src/app
---> Using cache
---> a4b4547833e5
Step 3/7 : COPY package*.json ./
---> Using cache
---> 2b19cc3e48a3
Step 4/7 : RUN npm install
---> Using cache
---> fe1f1e72d17d
Step 5/7 : COPY . .
---> Using cache
---> fc036f428e34
Step 6/7 : EXPOSE 8080
---> Using cache
---> d1ede7276d34
Step 7/7 : CMD node index.js
---> Using cache
---> cf051929395b
Successfully built cf051929395b
Successfully tagged test:latest
The RUN step executes a temporary container with the provided command, waits for that command to exit, and then captures to changes to the container filesystem as another layer of the resulting image. It does not store the running processes, changes to environment variables, or any changes to the state of the shell since those are not written to the filesystem. It also does not capture changes to volumes since temporary containers are started with the volumes defined in the image, and changes to a volume are not applied the container filesystem. This is a build time step.
The CMD step replaces the existing default command that docker runs when the image is run as a container. Containers exist for as long as this command is running, and you can only have a single value for the command. If you define CMD a second time, the previous value is replaced. And if you start a container with an overridden command, the image value for CMD is ignored.
Therefore you want to separate the steps to modify the filesystem at image build time from the steps to perform when the container is run into RUN and CMD respectively.
First of all, if you start any long-running process in build stage with RUN command your build process will be stuck.
The RUN command execute at build time and this is designed for build-time configuration and installation packages and tools, with RUN command your prepare your Docker image, for example installing npm modules and some other application dependency that will be available when the process is up in the container.
The CMD execute when you start the container, it does not execute at build time, CMD should be a long-running process to keep your container.
I'm using docker-compose in my local environment for development. I have 4 docker containers, 3 of them based on docker-node.
When I run docker-compose up --build, I see most of the building steps are loaded from cache, so they usually run really fast. But since a couple of days ago, I noticed they began to take a very long time to build (around 10 minutes in total), even if nothing had changed.
I also noticed that all the building steps seem to be loaded from cache, except for the first one: FROM node:8. And indeed, it always takes the longest to print the first step. Here's the complete output:
Building mongo-seed
Step 1/10 : FROM node:8
---> d575940ea42b
Step 2/10 : ENV NODE_ENV=development
---> Using cache
---> 64584d7c28d1
Step 3/10 : WORKDIR /seeder
---> Using cache
---> 2481554b20cc
Step 4/10 : COPY ./server/app/models ./app/models
---> Using cache
---> ad6c76419ffd
Step 5/10 : COPY ./server/config ./config
---> Using cache
---> 379dd9e19e67
Step 6/10 : COPY ./server/util ./util
---> Using cache
---> 1e6db5498a1a
Step 7/10 : COPY ./server/seed ./seed
---> Using cache
---> 659f531ef78d
Step 8/10 : COPY ./server/package.json ./package.json
---> Using cache
---> 4eb54f2548b9
Step 9/10 : RUN npm i mongoose
---> Using cache
---> c34967921c69
Step 10/10 : CMD npm run seed
---> Using cache
---> 0274c6635c1d
Successfully built 0274c6635c1d
Successfully tagged javorosas_mongo-seed:latest
Building server
^[Step 1/17 : FROM node:8
---> d575940ea42b
Step 2/17 : RUN apt-get update
---> Using cache
---> 4cb3797b4737
Step 3/17 : RUN apt-get install -y graphicsmagick
---> Using cache
---> a33dcf53e585
Step 4/17 : RUN npm i -g webpack node-gyp nodemon
---> Using cache
---> c961ca565c54
Step 5/17 : ENV NODE_ENV=development
---> Using cache
---> c63d1a245873
Step 6/17 : WORKDIR /server
---> Using cache
---> 7ac6181902ca
Step 7/17 : COPY ./server/package.json .
---> Using cache
---> ebba732f0210
Step 8/17 : COPY ./common ../common
---> Using cache
---> 47a9e631cc57
Step 9/17 : RUN npm install --quiet && npm cache clean -f
---> Using cache
---> 8d57a963d1ac
Step 10/17 : COPY ./server/constants ./constants
---> Using cache
---> a6a8884a1343
Step 11/17 : COPY ./server/util ./util
---> Using cache
---> dfd9de1db59e
Step 12/17 : COPY ./server/webpack.ssr.js .
---> Using cache
---> 547282293bc6
Step 13/17 : COPY ./server/src ./src
---> Using cache
---> e7821c59d012
Step 14/17 : RUN npm run build
---> Using cache
---> 184527650563
Step 15/17 : COPY . .
---> 8be425af57c7
Step 16/17 : EXPOSE 2018
---> Running in c662570856a5
Removing intermediate container c662570856a5
---> 1eb434d7e0ea
Step 17/17 : CMD npm run start:dev
---> Running in 613c7064436e
Removing intermediate container 613c7064436e
---> f37d511ffc78
Successfully built f37d511ffc78
Successfully tagged javorosas_server:latest
Building client
Step 1/10 : FROM node:8
---> d575940ea42b
Step 2/10 : RUN npm i -g webpack
---> Using cache
---> b8428ab7c232
Step 3/10 : ENV NODE_ENV=development
---> Using cache
---> bbfbbb480a46
Step 4/10 : WORKDIR /client
---> Using cache
---> 2203bdaa2cea
Step 5/10 : COPY ./client/package.json .
---> Using cache
---> fe5ead356992
Step 6/10 : COPY ./common ../common
---> Using cache
---> db9c274937cc
Step 7/10 : RUN npm install --quiet && npm cache clean -f
---> Using cache
---> 16be06b426b3
Step 8/10 : COPY . .
---> c96343b67244
Step 9/10 : EXPOSE 2017
---> Running in 7460e5f1d0c6
Removing intermediate container 7460e5f1d0c6
---> 8436f916f585
Step 10/10 : CMD npm run start
---> Running in fbcf8af08cbf
Removing intermediate container fbcf8af08cbf
---> 47958626245f
Successfully built 47958626245f
Successfully tagged javorosas_client:latest
javorosas_mongo_1 is up-to-date
Starting javorosas_mongo-seed_1 ... done
Recreating javorosas_server_1 ... done
Recreating javorosas_client_1 ... done
Attaching to javorosas_mongo_1, javorosas_mongo-seed_1,
javorosas_server_1, javorosas_client_1
Any ideas on why it could have started to slow down?