docker-compose setup node app - deep dive - node.js

I am getting a npm ERR! enoent ENOENT: no such file or directory, open '/usr/src/app/package.json' error currently with the below docker setup or a error TS2307: Cannot find module 'Actions' or its corresponding type declarations- i think its a case that the paths are not found in tsconfig.json during the build or i am not COPYing the correct directory/volume as part of the Dockerfile. Have spent multiple days working through different path configs / setups, any help getting this to build would be greatly appreciated.
Would love to see a node / TS / docker / mysql project example if there are any in the community to share - have found it difficult to find opensource projects to compare this to for hints.
...
"paths": {
"Actions/*": [
"Actions/*"
],
}
docker-compose
version: '3.8'
services:
app:
image: app:latest
container_name: balanced-money-backend
build:
context: .
dockerfile: Dockerfile
# TODO investigate uid and gid, how does it get in - from a startup script? Think it needs to be added like user: $UID:$GID if my cmd calls a setup to id on host machine. Needs more investigation.
depends_on:
db:
condition: service_healthy
env_file:
- .env
restart: always
volumes:
- .:/var/www/
command: npm start
ports:
- $NODE_LOCAL_PORT:$NODE_DOCKER_PORT
environment:
- DB_HOST=$MYSQL_HOST
- DB_USERNAME=$MYSQL_USER
- DB_PORT=$MYSQL_DOCKER_PORT
- DB_PASSWORD=$MYSQL_PASSWORD
- DB_DATABASE=$MYSQL_DATABASE
db:
image: mysql:5.7
restart: always
container_name: balanced-money-database
environment:
- MYSQL_ROOT_PASSWORD=$MYSQL_ROOT_PASSWORD
- MYSQL_USER=$MYSQL_USER
- MYSQL_PASSWORD=$MYSQL_PASSWORD
- MYSQL_DATABASE=$MYSQL_DATABASE
ports:
- $MYSQL_LOCAL_PORT:$MYSQL_DOCKER_PORT
volumes:
- db:/var/lib/mysql
healthcheck: # mysql does not start immediatly, app needs to wait for mysql to start, having condition: service_healthy on app and a healthcheck makes sure db has started before app... i think.
test: mysqladmin ping -h 127.0.0.1 -u $$MYSQL_USER --password=$$MYSQL_PASSWORD
timeout: 20s
retries: 10
volumes:
db:
Dockerfile
############### Stage 1 - build the project
# use alpine version of node to keep the image size small as possible
FROM node:16-alpine AS build
# node docs recommend this
WORKDIR /usr/src/app
# docker caches per row as it builds, so copy those files which do not change often to the container first and following builds will not need copy as they are already cached by Docker.
COPY package*.json ./
COPY src tsconfig.json ./
RUN npm install
RUN npm run build
# TODO not sure about the stages - can i have a test / dev stage so test / dev is run in docker too.
############### Stage 2 - run the project
FROM build AS prod
EXPOSE 4000
# from stage 1, i.e. build take the code in the dist / package.json and copy to the container
COPY --from=build /usr/src/app/dist ./dist/
COPY --from=build /usr/src/app/package*.json ./
# npm ci will install exact versions from a package-lock file, and --production will only install dependencies, not dev dependencies.
RUN npm ci --production && npm cache clean --force
# make sure user is not root which could have security consequences.
USER node
CMD ["node", "dist/index.js"]
package.json scripts
"scripts": {
"build": "tsc",
"start": "node ./dist/index.js",
"node": "./dist/index.js",
"dev": "NODE_ENV=development DOTENV_CONFIG_PATH=.env.dev nodemon ts-node src/index.ts",
"format:prettier": "prettier --config .prettierrc 'src/**/*.ts' --write",
"lint": "eslint . --ext .ts",
"lint:fix": "eslint . --ext .ts --fix",
"test": "DOTENV_CONFIG_PATH=.env.test NODE_ENV=test jest --runInBand",
"test:coverage": "DOTENV_CONFIG_PATH=.env.test NODE_ENV=test jest --coverage",
},

Related

Cloud Run: The user-provided container failed to start and listen on the port defined provided by the PORT=8080

I am trying to deploy a containerized node-typescript-express app to cloud run but I am unable to do so, receiving the following error:
The user-provided container failed to start and listen on the port defined provided by the PORT=8080
Here is my Dockerfile config:
FROM node:18.13.0 as base
WORKDIR /home/node/app
COPY package*.json ./
RUN npm i
COPY . .
FROM base as production
ENV NODE_PATH=./dist
RUN npm run build
In my code, I'm declaring port as
const PORT = process.env.PORT || 8080;
I also have a .env file where I was setting port, but I deleted the port key - as far as I know, GCP cloud run injects the port variable anyway.
Here is a screenshot from my project settings on GCP. I uploaded my image by building it locally with docker-compose build, tagging it, and uploading it to the GCP container repository.
I've tried manually setting the port in the code, removing the env file completely, specifying a different port, etc. I'm not even sure if the port is specifically the error and it's just some kind of catch-all.
Here's my package.json:
{
"name": "weather-service",
"version": "0.0.0",
"description": "small node server that fetches openweather api data",
"engines": {
"node": ">= 18.12 <19"
},
"scripts": {
"start": "NODE_PATH=./dist node dist/src/index.js",
"clean": "rimraf coverage dist tmp",
"dev": "ts-node-dev -r tsconfig-paths/register src/index.ts",
"prebuild": "npm run lint",
"build": "ttsc -p tsconfig.release.json",
"build:watch": "ttsc -w -p tsconfig.release.json",
"build:release": "npm run clean && ttsc -p tsconfig.release.json",
"test": "jest --coverage --detectOpenHandles --forceExit",
"test:watch": "jest --watch --detectOpenHandles --forceExit",
"lint": "eslint . --ext .ts --ext .mts && tsc",
"lint:fix": "eslint . --ext .ts --ext .mts",
"prettier": "prettier --config .prettierrc --write .",
"prepare": "husky install",
"pre-commit": "lint-staged"
And lastly, here's my docker-compose file and how I'm executing the commands
docker-compose.yml
version: '3.7'
services:
weather-service:
build:
context: .
dockerfile: Dockerfile
target: base
volumes:
- ./src:/home/node/app/src
container_name: weather-service
expose:
- '8080'
ports:
- '8080:8080'
command: npm run dev
docker-compose.prod.yml
version: '3.7'
services:
weather-service:
build:
target: production
command: npm run start
docker.compose.dev.yml
version: '3.7'
services:
weather-service:
env_file:
- .env
environment:
- ${PORT}
- ${WEATHER_API_KEY}
Makefile
up:
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up
up-prod:
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up
down:
docker-compose down
build:
docker-compose build
If you are using Macbook, then below answer from Bk Lim in the below link might help you:
Cloud Run: "Failed to start and then listen on the port defined by the PORT environment variable." When I use 8080
Update: I managed to get it successfully deployed by changing my docker-compose files to a template I found on GitHub, here
My docker knowledge is minimal so if anyone has any idea why my old docker-compose wasn't working, I'd love to know.

Docker Nodemon not reloading on changes even though -L is set

Hi I'm trying to dockerize an app im currently working on. It uses nodejs and mariadb. I have some difficulties with figuring out how to make nodemon work.
I tried using --legacy-watch or -L which is the short form but it didn't change the result.
NPM installs all dependecies correct i even get the nodemon text but it doesn't restart the server when i make changes.
Would be gal if anyone could help
package.json:
{
"name": "nodejs_mariadb_docker_test",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node src/index.js",
"dev": "nodemon -L src/index.js"
},
"author": "",
"license": "ISC",
"dependencies": {
"express": "^4.17.2",
"mariadb": "^2.5.5",
"nodemon": "^2.0.15"
}
}
Dockerfile for nodejs:
# Specifies the image of your engine
FROM node:16.13.2
# The working directory inside your container
WORKDIR /app
# Get the package.json first to install dependencies
COPY package.json /app
# This will install those dependencies
RUN npm install
# Copy the rest of the app to the working directory
COPY . /app
# Run the container
CMD ["npm", "run", "dev"]
and the docker compose file:
version: "3"
services:
node:
build: .
container_name: express-api
ports:
- "80:8000"
depends_on:
- mysql
mysql:
image: mariadb:latest
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: "password"
volumes:
- mysqldata:/var/lib/mysql
- ./mysql-dump:/docker-entrypoint-initdb.d
volumes:
mysqldata: {}
So the obvious problem is that you do not mount your code into the container. That is why nodemon cannot see any changes, and react to them.
Additionally, it may be more straight forward to develop the application locally and only use docker as a mean to package/ship it.
If you still want to go down this route, I would suggest something like this.
services:
express-api:
build: ./
# overwrite the prod command
command: npm run dev
ports:
- "80:8000"
volumes:
# mount your code folder into the app folder
- .:/app
# mysql stuff ...
In your dockerfile you can swap the command for the production one, since in development, compose will override it.
FROM node:16.13.2
WORKDIR /app
COPY package.json package-lock.json ./
# use ci to install from the lock file,
# to avoid suprises in prod
RUN npm ci
COPY . ./
# use the prod command
CMD ["npm", "run", "start"]
This will do a bit of redundant work in development, like copying the code, but it should be OK.
Additionally, you may want to use a .dockerignore to ignore the mysqldump for example. Otherwise, it will be copied into the image, which is probably not desirable.
Also note that through npm ci your dependencies are locked, and won't update automatically. It will also throw errors if your lock file is not in sync with package.json. This is what you want for production. If you develop locally, you can run npm install locally or via docker exec to bump the dependencies, if required. Then you can check if nothing is broken, and be sure that for your prod image it will be fine since it's used from the lock file again.

Docker compose MongoDB with NodeJS using yarn

I'm trying to use docker compose to put in containers a node.js server and a mongodb database. I'm also trying to use yarn as my package-manager. My project is codded in typescript and my Mongodb is used as a service for my server.
For what I have at the moment, it works just fine, but I'm not sure if my implementation is even good. When I run docker-compose up, the containers and the images of node.js and mongodb are created, and the containers are running on the specified ports.
The problem is that when I kill the mongodb container, my nodejs server still has access to the database, like if my container mongodb container is still running. I would guess that it's because my mongodb server is also running on the same container as my node.js server.
This is my Dockerfile:
FROM node:lts-alpine
ADD package.json /app/package.json
ADD yarn.lock /app/yarn.lock
WORKDIR /app
# Installing packages
RUN yarn
ADD . /app
ENV NODE_ENV=production
# Building TypeScript files
RUN yarn build
CMD ["node", "./dist/app.js"]
These are the scripts in my package.json file:
"scripts": {
"lint": "eslint src/**/*.ts",
"format": "eslint src/**/*.ts --fix",
"start": "tsc && node --unhandled-rejections=strict ./dist/app.js",
"debug": "export DEBUG=* && npm run start",
"test": "echo \"Error: no test specified\" && exit 1",
"dev": "ts-node src/app.ts",
"build": "rm -rf build && tsc -p ."
},
And this is my docker-compose.yml file:
version: "3"
services:
app:
container_name: app
restart: always
build:
context: .
dockerfile: Dockerfile
env_file: .env
volumes:
- ./src:/app/src
environment:
- COSMOSDB_HOST=${COSMOSDB_HOST}
- COSMOSDB_PORT=${COSMOSDB_PORT}
- COSMOSDB_DBNAME=${COSMOSDB_DBNAME}
- COSMOSDB_USER=${COSMOSDB_USER}
- COSMOSDB_PASSWORD=${COSMOSDB_PASSWORD}
ports:
- "4000:4000"
links:
- mongo
mongo:
container_name: mongo
image: mongo
ports:
- "27017:27017"
volumes:
- ./data:/data/db
I don't seem to understand what's wrong with my files...
Can you help me ? Also, would you have some recommendation to give me to optimize the build or anything else ? Thank you !

docker - image not updating after building

After I run docker-compose up -d --build, I run docker images, it shows:
REPOSITORY TAG IMAGE ID CREATED SIZE
test-tets-test-server_my-web latest 2a3f05e387a7 1 minutes ago 2.81GB
But When I run docker run -it 2a3f05e387a7 sh and look for the files, it seems that the files are not updating and still in old version.
Dockerfile
FROM node:lts-alpine
RUN npm install --global sequelize-cli nodemon
WORKDIR /server
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3030
CMD ["npm", "run", "dev"]
docker-compose.yml
version: '2.1'
services:
test-db:
image: mysql:5.7
...
test-web:
environment:
- NODE_ENV=local
- PORT=3030
build: .
command: >
./wait-for-db-redis.sh test-db npm run dev
volumes:
- ./:/server
ports:
- "3030:3030"
depends_on:
- test-db
package.json
...
"scripts": {
"test": "npm run lint && npm run mocha",
"lint": "eslint src/. test/. --config .eslintrc.json --fix",
"dev": "nodemon --legacy-watch src/",
"start": "node src/",
},
...
Since docker-compose up -d --build does not recreate, you may still see the old file, or they may be cache.
Run docker-compose up -d --build --force-recreate to force it recreate the image.

How do I run a webpack build from a docker container?

The app I'm making is written in ES6 and other goodies is transpiled by webpack inside a Docker container. At the moment everything works from creating the inner directory, installing dependencies, and creating the compiled bundle file.
When running the container instead, it says that dist/bundle.js does not exist. Except if I create the bundle file in the host directory, it will work.
I've tried creating a volume for the dist directory at it works the first time, but after making changes and rebuilding it does not pick up the new changes.
What I'm trying to achieve is having the container build and run the compiled bundle. I'm not sure if the webpack part should be in the Dockerfile as a build step or at runtime since the CMD ["yarn", "start"] crashes but RUN ["yarn", "start"] works.
Any suggestions ands help is appreciated. Thanks in advance.
|_src
|_index.js
|_dist
|_bundle.js
|_Dockerfile
|_.dockerignore
|_docker-compose.yml
|_webpack.config.js
|_package.json
|_yarn.lock
docker-compose.yml
version: "3.3"
services:
server:
build: .
image: selina-server
volumes:
- ./:/usr/app/selina-server
- /usr/app/selina-server/node_modules
# - /usr/app/selina-server/dist
ports:
- 3000:3000
Dockerfile
FROM node:latest
LABEL version="1.0"
LABEL description="This is the Selina server Docker image."
LABEL maintainer="AJ alvaroo#selina.com"
WORKDIR "/tmp"
COPY ["package.json", "yarn.lock*", "./"]
RUN ["yarn"]
WORKDIR "/usr/app/selina-server"
RUN ["ln", "-s", "/tmp/node_modules"]
COPY [".", "./"]
RUN ["yarn", "run", "build"]
EXPOSE 3000
CMD ["yarn", "start"]
.dockerignore
.git
.gitignore
node_modules
npm-debug.log
dist
package.json
{
"scripts": {
"build": "webpack",
"start": "node dist/bundle.js"
}
}
I was able to get a docker service in the browser with webpack by adding the following lines to webpack.config.js:
module.exports = {
//...
devServer: {
host: '0.0.0.0',
port: 3000
},
};
Docker seems to want the internal container address to be 0.0.0.0 and not localhost, which is the default string for webpack. Changing webpack.config.js specification and copying that into the container when it is being built allowed the correct port to be recognized on `http://localhost:3000' on the host machine. It worked for my project; hope it works for yours.
I haven't included my src tree structure but its basically identical to yours,
I use the following docker setup to get it to run and its how we dev stuff every day.
In package.json we have
"scripts": {
"start": "npm run lint-ts && npm run lint-scss && webpack-dev-server --inline --progress --port 6868",
}
dockerfile
FROM node:8.11.3-alpine
WORKDIR /usr/app
COPY package.json .npmrc ./
RUN mkdir -p /home/node/.cache/yarn && \
chmod -R 0755 /home/node/.cache && \
chown -R node:node /home/node && \
apk --no-cache add \
g++ gcc libgcc libstdc++ make python
COPY . .
EXPOSE 6868
ENTRYPOINT [ "/bin/ash" ]
docker-compose.yml
version: "3"
volumes:
yarn:
services:
web:
user: "1000:1000"
build:
context: .
args:
- http_proxy
- https_proxy
- no_proxy
container_name: "some-app"
command: -c "npm config set proxy=$http_proxy && npm run start"
volumes:
- .:/usr/app/
ports:
- "6868:6868"
Please note this Dockerfile is not suitable for production it's for a dev environment as its running stuff as root.
With this docker file there its a gotcha.
Because alpine is on musl and we are on glib if we install node modules on the host the compiled natives won't work on the docker container, Once the container is up if you get an error we run this to fix it (its a bit of a sticking plaster right now)
docker-compose exec container_name_goes_here /bin/ash -c "npm rebuild node-sass --force"
ikky but it works.
Try changing your start script in the package.json to perform the build first (doing this, you won't need the RUN command to perform the build in your Dockerfile:
{
"scripts": {
"build": "webpack",
"start": "webpack && node dist/bundle.js"
}
}

Resources