NodeJs Docker deployment on google cloud run - node.js

I am trying to deploy my NodeJs app on google cloud run over last two days. I use cloudbuild.ymal and all the building process and deployment process were all done without error. But I somehow can't open the app after deployment.
Before I was getting error comming from the errorHandler middleware but It was gone after I set NODE_ENV = production. I still get this error
The app is working fine without error on my local machine with Docker and MongoDB.
Here is my cloudbuild.yaml
steps:
- name: 'docker/compose:1.19.0'
args: ['up', '-d']
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/ecommerce:latest', '-t', 'gcr.io/$PROJECT_ID/ecommerce:$COMMIT_SHA', '-t', 'gcr.io/$PROJECT_ID/ecommerce:$BUILD_ID', '.']
id: 'build-image-ecommerce'
timeout: 500s
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/ecommerce:$COMMIT_SHA']
id: 'push-image-to-container-registry'
waitFor: ['build-image-ecommerce']
- name: 'gcr.io/cloud-builders/gcloud'
args:
- 'run'
- 'deploy'
- 'ecommerce'
- '--image'
- 'gcr.io/$PROJECT_ID/ecommerce:$COMMIT_SHA'
- '--region'
- 'us-east4'
- '--platform'
- 'managed'
- '--allow-unauthenticated'
waitFor: ['push-image-to-container-registry']
id: 'deploy-to-cloud-run'
images:
- 'gcr.io/$PROJECT_ID/ecommerce:latest'
- 'gcr.io/$PROJECT_ID/ecommerce:$COMMIT_SHA'
- 'gcr.io/$PROJECT_ID/ecommerce:$BUILD_ID'
Here is my server.js
const express = require('express')
const dotenv = require('dotenv')
const morgan = require('morgan')
const colors = require('colors')
const connectDB = require('./config/db')
const authRoute = require('./Routes/authRoute')
const productRoute = require('./Routes/productRoute')
const orderRoute = require('./Routes/orderRoute')
const { errorHandler, notFound } = require('./Middlewares/errorMiddleware')
// DB Connection
connectDB()
dotenv.config()
const app = express()
if (process.env.NODE_ENV === 'development') {
app.use(morgan('dev'))
}
app.use(express.json())
//Routes
app.use('/api/users', authRoute)
app.use('/api/products', productRoute)
app.use('/api/orders', orderRoute)
// Error Middlewares
app.use(errorHandler)
app.use(notFound)
const PORT = process.env.PORT || 5000
app.listen(PORT, console.log(`App is running in ${process.env.NODE_ENV} on port ${PORT}`.blue.underline))
Here is my docker-compose.yml very simple setup
version: '3'
services:
app:
container_name: ecommerceApi
restart: always
build: .
ports:
- '80:5000'
links:
- mongodb
mongodb:
container_name: mongodb
image: mongo
ports:
- '27017:27017'
environment:
- MONGO_INITDB_ROOT_USERNAME=admin
- MONGO_INITDB_ROOT_PASSWORD=password
Here is Dockerfile
FROM node:14
WORKDIR /usr/src/app
COPY package*.json ./
# Install production dependencies.
RUN npm install --only=production
COPY . ./
EXPOSE 5000
CMD ["npm", "start"]
I am using MongoDB as database, I do have another as well. How do I migrate some data to the db while working with Docker i.e run command like npm run data:import to import data in the db. I tried some ways but I didn't work for me. I could do it docker exec -it sh and run the command but on production this process to be automated.
Please any help solving these issues, thanks in advance!!

Related

Docker - React app can not fetch from the server

Hello I am new to the docker and I am trying to dockerize my application that uses React as frontend, nodejs as backend and mySQL as database. However when I try to fetch data from server from my react app, it gives me error:
Access to fetch at 'http://localhost:3001/api' from origin 'http://localhost:3000' has been blocked by CORS policy: The 'Access-Control-Allow-Origin' header has a value 'http://127.0.0.1:3000' that is not equal to the supplied origin. Have the server send the header with a valid value, or, if an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
My react app is rendered and also when I go to http://localhost:3001/api I receive the data I would like to get. Just the communication between react and nodejs is somehow broken.
Here are my Docker files and env files:
.env:
DB_HOST=localhost
DB_USER=root
DB_PASSWORD=123456
DB_NAME=testdb
DB_PORT=3306
MYSQLDB_USER=root
MYSQLDB_ROOT_PASSWORD=123456
MYSQLDB_DATABASE=testdb
MYSQLDB_LOCAL_PORT=3306
MYSQLDB_DOCKER_PORT=3306
NODE_LOCAL_PORT=3001
NODE_DOCKER_PORT=3001
CLIENT_ORIGIN=http://127.0.0.1:3000
CLIENT_API_BASE_URL=http://127.0.0.1:3001/api
REACT_LOCAL_PORT=3000
REACT_DOCKER_PORT=80
dockerfile for react:
FROM node:14.17.0 as build-stage
WORKDIR /frontend
COPY package.json .
RUN npm install
COPY . .
ARG REACT_APP_API_BASE_URL
ENV REACT_APP_API_BASE_URL=$REACT_APP_API_BASE_URL
RUN npm run build
FROM nginx:1.17.0-alpine
COPY --from=build-stage /frontend/build /usr/share/nginx/html
EXPOSE 80
CMD nginx -g 'daemon off;'
dockerfile for nodejs:
FROM node:14.17.0
WORKDIR /
COPY package.json .
RUN npm install
COPY . .
EXPOSE 3001
CMD [ "node", "server.js" ]
docker-compose.yml :
version: '3.8'
services:
mysqldb:
image: mysql
restart: unless-stopped
env_file: ./.env
environment:
- MYSQL_ROOT_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- MYSQL_DATABASE=$MYSQLDB_DATABASE
ports:
- $MYSQLDB_LOCAL_PORT:$MYSQLDB_DOCKER_PORT
volumes:
- db:/var/lib/mysql
networks:
- backend
server-api:
depends_on:
- mysqldb
build: ./
restart: unless-stopped
env_file: ./.env
ports:
- $NODE_LOCAL_PORT:$NODE_DOCKER_PORT
environment:
- DB_HOST=mysqldb
- DB_USER=$MYSQLDB_USER
- DB_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- DB_NAME=$MYSQLDB_DATABASE
- DB_PORT=$MYSQLDB_DOCKER_PORT
- CLIENT_ORIGIN=$CLIENT_ORIGIN
networks:
- backend
- frontend
frontend-ui:
depends_on:
- server-api
build:
context: ./frontend
args:
- REACT_APP_API_BASE_URL=$CLIENT_API_BASE_URL
ports:
- $REACT_LOCAL_PORT:$REACT_DOCKER_PORT
networks:
- frontend
volumes:
db:
networks:
backend:
frontend:
My project folder structure is a bit weird as my server its things(node_modules, package.json...) are in the root where docker-compose, .env and Dockerfile for server is located.
React app and frontend is in /frontend folder where also Dockerfile for react is located.
In react I call fetch("http://localhost:3001/api").
Server is created with express :
const express = require('express');
const cors = require('cors');
const server = express();
var mysql = require('mysql2');
require("dotenv").config();
const port = 3001
server.use(express.static('public'));
var corsOptions = {
origin: "http://127.0.0.1:3000"
}
server.use(cors(corsOptions));
var con = mysql.createConnection({
host: process.env.DB_HOST,
port: process.env.DB_PORT,
user: process.env.DB_USER,
password: process.env.DB_PASSWORD
});
server.get('/api', async (req, res) => {
console.log("START");
con.connect(function (err) {
if (err) throw err;
console.log("connected !");
con.query("use testdb;", function (err, result, fields) {
if (err) throw err;
console.log(result);
});
con.query("select * from records;", function (err, result, fields) {
if (err) throw err;
res.send(result);
});
});
});
server.listen(port, () => {
console.log(`Server listening on port ${port}`)
})
I created this thanks to This tutorial
Thanks for any help.
change this: origin: "http://127.0.0.1:3000"
to this: origin: "http://localhost:3000"

Dockerfile not working on backend with react

I'm trying to run a react app with 2 node servers. One for the front end and one for the back-end connected with a mysql data-base.
I'm trying to use docker for the container and I managed to get the database and the front-end server up. However,When the back-end server is fired it seems like it doesn't acknowledge the Dockerfile.
node_server | npm WARN exec The following package was not found and will be installed: nodemon
node_server | Usage: nodemon [nodemon options] [script.js[args]
node_server |
node_server | See "nodemon --help" for more.
node_server |
node_server exited with code 0
Dockerfile - client:
FROM node:latest
RUN mkdir -p /usr/src/app
WORKDIR /usr/scr/app
EXPOSE 3000
COPY package.json .
RUN npm install express body-parser nano nodemon cors
COPY . .
Dockerfile - server
FROM node:latest
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
RUN npm init -y
RUN npm install express body-parser nano nodemon cors
EXPOSE 5000
CMD ["npx", "nodemon", "src/server.js"]
docker-compose
version: '3'
services:
backend:
build:
context: ./server
dockerfile: ./Dockerfile
depends_on:
- mysql
container_name: node_server
image:
raff/node_server
ports:
- "5000:5000"
volumes:
- "./server:/usr/src/app"
frontend:
build:
context: ./client
dockerfile: ./Dockerfile
container_name: node_client
image:
raff/node_client
ports:
- "3000:3000"
volumes:
- "./client:/usr/src/app"
mysql:
image: mysql:5.7.31
container_name: db
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: admin
MYSQL_DATABASE: assignment
The server side is not done yet, but i don't believe it's causing this error.
Server.js
"use strict";
const path = require("path");
const express = require("express");
const app = express();
const bodyParser = require("body-parser");
app.use(bodyParser.urlencoded({ extended: true }));
app.use(express.json());
const mysql = require("mysql");
let con = mysql.createConnection({
host: "mysql",
port: "3306",
user: "root",
password: "admin",
});
const PORT = 5000;
const HOST = "0.0.0.0";
app.post("/posting", (req, res) => {
var topic = req.body.param1;
var data = req.body.param2;
sql_insertion(topic, data);
});
// Helper
const panic = (err) => console.error(err);
// Connect to database
con.connect((err) => {
if (err) {
panic(err);
}
console.log("Connected!");
con.query("CREATE DATABASE IF NOT EXISTS assignment", (err, result) => {
if (err) {
panic(err);
} else {
console.log("Database created!");
}
});
});
//select database
con.query("use assignment", (err, result) => {
if (err) {
panic(err);
}
});
// Create Table
let table =
"CREATE TABLE IF NOT EXISTS posts (ID int NOT NULL AUTO_INCREMENT, Topic varchar(255), Data varchar(255), Timestamp varchar(255), PRIMARY KEY(ID));";
con.query(table, (err) => {
if (err) {
panic(err);
} else {
console.log("Table created!");
}
});
app.get("*", (req, res) => {
res.sendFile(path.join(__dirname, "client/build" , "index.html"));
});
app.listen(PORT, HOST);
console.log("up!");
Modify this line
CMD ["npx", "nodemon", "src/server.js"]
By
CMD ["npx", "nodemon", "--exec", "node src/server.js"]
While putting the command in package.json under scripts section is better.
Your volumes: declarations are hiding everything that's in the image, including its node_modules directory. That's not normally required, and you should be able to trim the frontend: container definition down to
backend:
build: ./server # default `dockerfile:` location
depends_on:
- mysql
image: raff/node_server # only if you plan to `docker-compose push`
ports:
- "5000:5000"
The image then contains a fixed copy of the application, so there's no particular need to use nodemon; just run the application directly.
FROM node:latest
WORKDIR /usr/src/app # also creates the directory
COPY package.json package-lock.json .
RUN npm ci # do not `npm install` unmanaged packages
COPY . . # CHECK: `.dockerignore` must include `node_modules`
EXPOSE 5000
CMD ["node", "src/server.js"]
This apparently isn't a problem for your frontend application, because there's a typo in WORKDIR -- the image installs and runs its code in /usr/scr/app but the bind mount is over /usr/src/app, so the actual application's /usr/scr/app/node_modules directory isn't hidden.

Node js and mongodb cannot connect docker-compose [duplicate]

This question already has answers here:
MongoDB on with Docker "failed to connect to server [localhost:27017] on first connect "
(4 answers)
Closed 1 year ago.
Both these services work individually, Yet when I go to the docker endpoint through the container the app crashes as it fails to connect to my database. Can anyone see why?
After running docker-compose i have inspected each container individually and they work fine but just not together. Unsure as to why as for others it seems to work
Docker compose
version: '3.3'
services:
db:
container_name: db
image: mongo
# volumes:
# - ./db:/data/db
ports:
- '27017:27017'
# frontend:
# build:
# context: './frontend/'
# dockerfile: Dockerfile
# container_name: reactfront
# # depends_on: [server]
# ports:
# - '3000:3000'
# restart: always
server:
build:
context: ./backend/
dockerfile: Dockerfile
container_name: nodeserver
ports:
- '4000:4000'
restart: always
depends_on: [db]
links:
- db
volumes:
db_data: {}
Node js app
const express = require('express')
const app = express()
const port = 4000
var cors = require('cors')
const { MongoClient } = require("mongodb");
const uri = 'mongodb://localhost:27017'
const client = new MongoClient(uri);
app.use(cors())
app.get('/', (req, res) => {
res.send('Hello Worldd!')
})
app.get('/docker', async (req, res) => {
let myres;
client.connect().then(async (db, b) => {
myres = await db.db().admin().listDatabases()
console.log(await db.db().admin().listDatabases());
res.json(myres)
})
})
app.listen(port, () => {
console.log(`Example app listening at http://localhost:${port}`)
})
Docker-compose will create a bridge network and the containers will be able to resolve each others' names.
Replace
const uri = 'mongodb://localhost:27017' with const uri = 'mongodb://db:27017' (db is mongo's service name).
You cant use localhost apparently as they are in different containers (seems obvious now) - I checked the IP of the container and put that into the mongo url

how to resolve Cannot GET / in a kubernetes deployment?

I have an node.js application called "process-core" which has the main logic to run another node.js app called "page-core".Through "Process-core" I am deploying the distribution folder(static contents) of the "page-core" application.I am deploying this using kubernetes.
dockerfile used to build image for "process-core"
FROM node:8
RUN mkdir -p usr/src/app
WORKDIR /usr/src/app
COPY . .
WORKDIR /usr/src/app
RUN npm install
CMD [ "npm","start" ]
my server.js goes like this-
var express = require('express');
var cors = require('cors')
var app = express();
var path = require('path');
var router = express.Router();
var request = require('request');
var api = require('./server/api.js')
const base_url = 'http://localhost:8080/api/';
const static_dir = '/home/vignesh/page-core-devbranch/dist'; // location of
const assets_dir = path.join(__dirname + '/server/assets/'); dist folder
const tools_dir = path.join(__dirname + '/server/tools/');
const index_file = path.join(static_dir + '/index.html');
/* static */
app.use(cors())
app.use(express.static(static_dir));
app.use('/assets', express.static(assets_dir));
app.use('/tools', express.static(tools_dir));
/* listen */
app.listen(5000);
node.yaml
apiVersion: v1
kind: Service
metadata:
labels:
node: test
name: nodetest
spec:
ports:
- nodePort: 30000
port: 5000
protocol: TCP
targetPort: 5000
selector:
node: test
type: NodePort
---
apiVersion: v1
kind: Pod
metadata:
name: nodetest
labels:
node: test
spec:
containers:
- name: node-test
image: suji165475/devops-sample:processcore
ports:
- containerPort: 5000
When I hit http://206.189.22.155:30000/ i get an error saying Cannot GET /
FROM node:8
RUN mkdir -p usr/src/app
WORKDIR /usr/src/app
COPY . .
WORKDIR /usr/src/app
RUN npm install
EXPOSE 5000
CMD [ "npm","start" ]
You have not expose port from docker image in docker file please add one line as EXPOSE 5000 to open the container port 5000 from docker container.

Pm2 don't bind docker ports

Im building a node api boilerplate with docker, babel, istanbul, pm2, eslint and other features. My project works fine in dev mode with nodemon and works fine in test mode with mocha too. However when I run the project in prod mode with pm2 the docker ports don't bind.
The full project can be find here https://github.com/apandrade/node-api-boilerplate
Docker ps result after run in production mode
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3d5362284957 node:latest "npm start" 15 seconds ago Up 15 seconds nodeapiboilerplate_provision_run_1
a2c79e3e47cc mongo "docker-entrypoint.s…" 52 seconds ago Up 51 seconds 0.0.0.0:27017->27017/tcp mongo
Base.yml file
version: "2"
services:
db_credentials:
environment:
- MONGODB_ADMIN_USER=*********
- MONGODB_ADMIN_PASS=*********
- MONGODB_APPLICATION_DATABASE=node_api_db
- MONGODB_APPLICATION_USER=*********
- MONGODB_APPLICATION_PASS=*********
common: &common
image: "node:latest"
working_dir: /usr/src/app
restart: always
volumes:
- ./:/usr/src/app
- ./scripts/waitforit:/usr/bin/waitforit
ports:
- "3000:3000"
base:
<<: *common
environment:
- MONGODB_ADMIN_USER=*********
- MONGODB_ADMIN_PASS=*********
- MONGODB_APPLICATION_DATABASE=node_api_db
- MONGODB_APPLICATION_USER=*********
- MONGODB_APPLICATION_PASS=*********
- APP_NAME=node-api-boilerplate
- PORT=3000
- DB_HOST=mongo
- DB_PORT=27017
base_test:
<<: *common
environment:
- MONGODB_ADMIN_USER=*********
- MONGODB_ADMIN_PASS=*********
- MONGODB_APPLICATION_DATABASE=node_api
- MONGODB_APPLICATION_USER=*********
- MONGODB_APPLICATION_PASS=*********
- PORT=3000
- DB_HOST=mongo
- DB_PORT=27017
docker-compose.yml file
version: "2"
services:
mongo:
container_name: mongo
image: mongo
volumes:
- ./data:/data/db
- ./scripts/mongo-entrypoint.sh:/docker-entrypoint-initdb.d/mongo-entrypoint.sh
ports:
- "27017:27017"
extends:
file: base.yml
service: db_credentials
command: "mongod --auth"
develop:
extends:
file: base.yml
service: base
environment:
- NODE_ENV=development
- LOG_LEVEL=debug
container_name: dev_node_api
command: "npm run dev"
depends_on:
- mongo
provision:
extends:
file: base.yml
service: base
environment:
- NODE_ENV=production
- LOG_LEVEL=info
container_name: prod_node_api
command: "npm start"
depends_on:
- mongo
test:
extends:
file: base.yml
service: base_test
environment:
- NODE_ENV=test
- LOG_LEVEL=debug
container_name: test_node_api
command: "npm run test"
depends_on:
- mongo
process.json file
{
"apps" : [{
"name" : "node-api-boilerplate",
"script" : "./src/server.js",
"exec_mode" : "cluster",
"exec_interpreter": "babel-node",
"instances" : "max",
"merge_logs" :true
}]
}
server.js file
require('pretty-error').start();
require('babel-register');// eslint-disable-line import/no-extraneous-dependencies
const express = require('express');
const morgan = require('morgan');
const methodOverride = require('method-override');
const bodyParser = require('body-parser');
const createError = require('http-errors');
require('./config/database');
const router = require('./config/router');
const logger = require('./config/logger');
const allowCors = require('./config/cors');
const PORT = process.env.PORT;
const app = express();
app.disable('x-powered-by');
app.use(methodOverride());
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(allowCors);
app.use(morgan('dev', {
skip: (req, res) => res.statusCode < 400,
stream: process.stderr,
}));
app.use(morgan('dev', {
skip: (req, res) => res.statusCode >= 400,
stream: process.stdout,
}));
/**
* Add and remove headers for all requests
*/
app.use((req, res, next) => {
res.setHeader('Content-Type', 'application/json');
res.setHeader('Accept', 'application/json');
next();
});
app.use('/api/v1', router);
/**
* Error Handler
*/
app.use((err, req, res, next) => {
logger.error(err.stack);
const error = createError(err);
res.status(error.status).json(error);
next();
});
app.listen(PORT, () => {
logger.info(`Listening on port ${PORT}`);
});
After a few days in search of the solution, I discovery that do not exists any problem, what happens is that to run my project I was run docker-compose run --rm <service_name> and the docker compose reference is clear
the docker-compose run command does not create any of the ports specified in the service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports to be created and mapped to the host, specify the --service-ports flag:
docker-compose run --service-ports <service_name>
However I chose to run docker-compose up <service_name>, it is enough for me because I don't have specific needs how to override a command or run only one container on different ports.

Resources