Node.js service Timeout issues - node.js

The Node.js API service hosted on AWS times out at intervals and crashes the server. When I checked Cloudwatch logs, I saw this error come up often.
npm ERR! path /app
npm ERR! command failed
npm ERR! signal SIGTERM
npm ERR! command sh -c -- npm run build && node dist/server.js
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2022-11-11T11_54_06_397Z-debug-0.log
Here's my docker file
#!/bin/sh
FROM node:16-bullseye-slim
RUN apt-get update; \
apt-get install -y python3; \
apt-get install -y python2 make; \
apt-get install -y g++; \
ln -s /usr/bin/python2.7 /usr/bin/python;
ARG app_env
ARG DOPPLER_TOKEN
ARG DOPPLER_PROJECT
ARG DOPPLER_CONFIG
ENV NODE_ENV=${app_env}
ENV DOPPLER_TOKEN=${DOPPLER_TOKEN}
ENV DOPPLER_PROJECT=${DOPPLER_PROJECT}
ENV DOPPLER_CONFIG=${DOPPLER_CONFIG}
WORKDIR /app
COPY package*.json ./
RUN npm install
RUN npm ci --only=production
COPY . .
CMD ["npm", "run", "start:server"]
EXPOSE 3000
My package.json file start script
"start:server": "npm run build && node dist/server.js"
I'm using Node version: v16.13.2
Npm version: 8.19.0
Not sure the cause of this issue.

Related

M1 chip, Install Puppeteer in Docker NodeJs, The chromium binary is not available for arm64

I got an error while building backend docker, specifically installing Puppeteer. I'm using M1 MacBook, and I found a solution on the local machine(https://github.com/puppeteer/puppeteer/issues/6622), but this didn't work on the docker. Has anyone who has the same Puppeteer issue on the docker?
#12 15.58 npm ERR! code 1
#12 15.58 npm ERR! path /app/node_modules/puppeteer
#12 15.58 npm ERR! command failed
#12 15.58 npm ERR! command sh -c node install.js
#12 15.58 npm ERR! The chromium binary is not available for arm64.
#12 15.58 npm ERR! If you are on Ubuntu, you can install with:
#12 15.58 npm ERR!
#12 15.58 npm ERR! sudo apt install chromium
#12 15.58 npm ERR!
#12 15.58 npm ERR!
#12 15.58 npm ERR! sudo apt install chromium-browser
#12 15.58 npm ERR!
#12 15.58 npm ERR! /app/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserFetcher.js:115
#12 15.58 npm ERR! throw new Error();
FROM --platform=linux/amd64 node:16-alpine
WORKDIR /app
EXPOSE 8000
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
ENV DOCKER_DEFAULT_PLATFORM "linux/amd64"
COPY . .
RUN apk --no-cache add --virtual builds-deps build-base python3 && \
npm install
CMD ["npm", "start"]
This is the dockerfile that worked for me.
# reference https://developers.google.com/web/tools/puppeteer/troubleshooting#setting_up_chrome_linux_sandbox
FROM node:current-alpine
# manually installing chrome
RUN apk add chromium
# skips puppeteer installing chrome and points to correct binary
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true \
PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium-browser
WORKDIR /app
COPY ["package.json", "package-lock.json*", "./"]
RUN npm ci
# the rest of your dockerfile here
When launching puppeteer in js, make sure to add the following flags on launch
puppeteer.launch({args: ['--no-sandbox', '--disable-setuid-sandbox']});
You can read more about this in google's docs
I had the same error, and this made it work for me: https://github.com/puppeteer/puppeteer/issues/7740#issuecomment-970490323
It seems the npm installation cannot find a chromium binary for M1. To not make npm try to install Chromium, you can add ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true. But then I needed to also install the binary before running npm install, by adding RUN apk add chromium to the Dockerfile (if you are running the node alpine image).
Result:
FROM node:16-alpine
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
RUN apk add chromium
# Add and install everything
Add puppeteer env after building dependencies. Worked for me.
FROM node:16-alpine3.11
WORKDIR /usr/app
COPY package*.json ./
COPY tsconfig.json ./
RUN apk --no-cache --virtual build-dependencies add \
python3 \
make \
g++
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD true
RUN npm install --quiet
RUN npm install -g pm2 --quiet
COPY ./ ./
RUN npm run build
RUN rm -rf ./src

Installing fresh react app with dockerfile

I'm trying to run a fresh installation of React using a Dockerfile.
FROM node:12.14-alpine
RUN mkdir /app
WORKDIR /app
RUN npm install -g nodemon \
npx create-react-app .
COPY package.json package.json
RUN npm install --silent
COPY . .
CMD [ "node", "start" ]
My root folder is empty so no package.json file exists etc. When I run docker-compose up --build I get the following error which indicates that's failing on npx create-react-app .
Step 5/9 : RUN npm install -g nodemon npx create-react-app .
---> Running in c4878af5f94d
npm ERR! code ENOLOCAL
npm ERR! Could not install from "" as it does not contain a package.json file.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2020-02-18T12_55_14_899Z-debug.log
ERROR: Service 'react' failed to build: The command '/bin/sh -c npm install -g nodemon npx create-react-app .' returned a non-zero code: 1
Any ideas what I'm doing wrong? I'm not having node or npm installed nativelly so all needs t run through docker
You need to delete the . at the end of the RUN :
RUN npm install -g nodemon \
npx \
create-react-app
You forgot "&&" at the beginning line 7
FROM node:12.14-alpine
RUN mkdir /app
WORKDIR /app
RUN npm install -g nodemon \
&& npx create-react-app .
COPY package.json package.json
RUN npm install --silent
...
CMD [ "node", "start" ]

NodeJS installing on Docker

I am trying to run my app on the Docker. One of the library I am using is https://www.npmjs.com/package/odbc.
In order to install that lib I need to meet the requirements described in the odbc readme:
unixODBC binaries and development libraries for module compilation
on Ubuntu/Debian sudo apt-get install unixodbc unixodbc-dev
on RedHat/CentOS sudo yum install unixODBC unixODBC-devel
odbc drivers for target database
properly configured odbc.ini and odbcinst.ini.
As per Microsoft doc in order to install ODBC Driver 13 for SQL Server https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-2017#ubuntu-1604-1
I manage to install all the stuff locally on my Mac and successfully connect with the SQL Server on Azure but still have some issues with installing them on the Docker and then run on VSTS.
My Dockerfile:
FROM ubuntu:16.04
USER root
RUN apt-get update
RUN apt-get install --yes curl
RUN curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
RUN apt-get install --yes nodejs
RUN apt-get install --yes build-essential
RUN apt-get install -y npm
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get install -y build-essential
RUN apt-get install -y make
RUN apt-get install apt-transport-https
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql unixodbc-dev
ADD . /var/www/app
WORKDIR /var/www/app
RUN npm install && \
npm cache clean --force
RUN npm run build
EXPOSE 3000:80
CMD ["npm", "start"]
But so far have an issue with installing NodeJS in line with
RUN curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash -
error: /bin/sh: 1: sudo: not found
I was trying to install only the driver and for installing NodeJs just use some existing Docker images:
FROM ubuntu:16.04
USER root
RUN apt-get update
RUN apt-get install --yes curl
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
RUN apt-get install -y build-essential
RUN apt-get install -y make
RUN apt-get install apt-transport-https
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql unixodbc-dev
FROM node:9-alpine
ADD . /var/www/app
WORKDIR /var/www/app
RUN npm install && \
npm cache clean --force
RUN npm run build
EXPOSE 3000:80
But that approach throws an error:
gyp ERR! configure error
gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack at PythonFinder.failNoPython (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:483:19)
gyp ERR! stack at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:397:16)
gyp ERR! stack at F (/usr/local/lib/node_modules/npm/node_modules/which/which.js:68:16)
gyp ERR! stack at E (/usr/local/lib/node_modules/npm/node_modules/which/which.js:80:29)
gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/which/which.js:89:16
gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/which/node_modules/isexe/index.js:42:5
gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/which/node_modules/isexe/mode.js:8:5
gyp ERR! stack at FSReqWrap.oncomplete (fs.js:170:21)
gyp ERR! System Linux 4.9.125-linuxkit
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "build"
gyp ERR! cwd /var/www/app/node_modules/odbc
gyp ERR! node -v v9.11.2
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents#1.2.7 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents#1.2.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! odbc#1.4.5 install: `node-gyp configure build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the odbc#1.4.5 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2019-03-08T20_51_17_496Z-debug.log
You are working off the ubuntu:16.04 image, and essentially doing a lot of footwork that the NodeJS guys have already done.
I would go for the image node:10-stretch-slim if I was you. And then install the drivers that you need with apt-get (if available, otherwise script the download and install in your Dockerfile).
The sudo command is not typically installed on docker images, because the user is root by default in the container sessions. If you see any errors concerning sudo, you can generally just remove sudo from the command line that is causing the issue.
Possible solution
Updating my answer here, with a possible solution for you.
This solution will put your application in a node 10 image, based on debian stretch 9. It will get the database drivers for you, from the debian 9 microsoft repository, and install all the packages that I see you are requiring from your question.
I have also added an ENTRYPOINT and CMD in the bottom of the script. But those lines are guesswork, since your question states nothing about how you actually start your application. (If you add that, then I will update my answer).
Note. Notice that I am passing --host 0.0.0.0 to the npm run start command. This is to avoid binding the live server to localhost, which will make in inaccessible from outside the container. Unless you start the container with --network="host".
You may have another means of starting your application that is more "production grade" than the live development server. If so, just replace the lines in the bottom of the Dockerfile, or ask me on this answer.
Dockerfile
# from debian stretch 9.8, node 10
FROM node:10-stretch-slim
# get apt-transport-https, etc., so that we can install by https protocol
RUN apt-get update \
&& apt-get install -y \
apt-transport-https \
build-essential \
make
# add and accept the microsoft signature
RUN curl -q https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
# retrieve the microsoft packagelist for debian 9
RUN curl -q https://packages.microsoft.com/config/debian/9/prod.list > /etc/apt/sources.list.d/mssql-release.list
# install the ms odbc sql driver and unixodbc header stuff
RUN apt-get update \
&& ACCEPT_EULA=Y apt-get install -y \
msodbcsql17 \
unixodbc-dev \
&& rm -rf /var/lib/apt/lists
# expose port 80 in containers of this image
EXPOSE 80
# copy working directory into the image and set as pwd
ADD . /var/www/app
WORKDIR /var/www/app
# install dependencies for the application
RUN npm install \
&& npm cache clean --force
# build the application
RUN npm run build
# i am just guessing how you want your app started here, npm?
ENTRYPOINT ["npm"]
# and then this, which makes "npm run start --host 0.0.0.0"
CMD ["run", "start", "--host", "0.0.0.0"]
Build the image with:
docker build -t mynodeapp:0.1 .
Run the application image with:
docker run -p 3000:80 --name mynodeapp mynodeapp:01
Finally visit: http://localhost:3000 to see it working.

Bitbucket Pipeline npm install fails with "npm ERR! fatal: unable to access"

I'm trying to run test cases in my node app using Bitbucket pipeline as follows.
image: channagayan/node_test:latest
pipelines:
default:
- step:
caches:
- node
script: # Modify the commands below to build your repository.
- cd NodeApi && pwd && npm install
- npm test
But it fails giving following error
+ cd NodeApi && pwd && npm install
/opt/atlassian/pipelines/agent/build/NodeApi
npm ERR! Error while executing:
npm ERR! /usr/bin/git ls-remote -h -t https://git#github.com/peterbraden/node-opencv.git
npm ERR!
npm ERR! fatal: unable to access 'https://git#github.com/peterbraden/node-opencv.git/': Problem with the SSL CA cert (path? access rights?)
npm ERR!
npm ERR! exited with error code: 128
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-05-01T09_15_55_789Z-debug.log
My Dockerfile is as follows,
FROM ubuntu:16.04
RUN apt-get update
RUN apt-get install -y --no-install-recommends \
build-essential \
cmake \
python-software-properties \
pkg-config \
wget \
&& rm -rf /var/lib/apt/lists/*
#node installation goes here....
RUN apt-get update && apt-get install -y git
#opencv installation goes here....
ENV LD_LIBRARY_PATH /usr/local/lib
WORKDIR /usr/src/app
EXPOSE 3001
It seems like Bitbucket is not accessing the internet from the docker container. Appreciate any help to resolve this.

Strange Git Error on Docker NPM Install

I had a working Dockerfile until literally a day ago when it just seemed to break. I didn't make any changes to my dependencies - but I am getting the following error:
[91mnpm ERR! code ENOGIT
[0m
[91mnpm ERR! No git binary found in $PATH
npm ERR!
npm[0m
[91m ERR! Failed using git.
npm ERR! Please check if you have git installed and in your PATH.
[0m
[91m
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2017-09-28T21_12_50_050Z-debug.log
[0m
Removing intermediate container be9d5bfe5521
The command '/bin/sh -c npm install' returned a non-zero code: 1
This is super strange because this wasn't happening before. I'm also attaching my Dockerfile. The things I've tried so far are adding git (third line), and also trying to export the path. Nothing seems to be working.
FROM ubuntu:latest
RUN apt-get update
RUN apt-get install -y git
FROM node:alpine
RUN npm install sails -g
#RUN npm install git -g
#RUN export PATH="$HOME/usr/bin/git:$PATH"
RUN mkdir -p /service/app
WORKDIR /service/app
COPY package.json /service/app
RUN npm install
COPY . /service/app
EXPOSE 80
CMD NODE_ENV=production sails lift
One reason for this could be that you are using the slim version of node in your Dockerfile:
FROM node:8-slim
I presume this does not include git, because when I changed to the full version the error went away:
FROM node:8.11.2
Try the following:
RUN apk update && \
apk add --update git && \
apk add --update openssh
The git binary within the docker container becomes available at /usr/bin/git
I had the same issue. It was due to lack of git. Below is how it worked
FROM node:alpine
RUN apk add git
RUN npm install ...

Resources