Openshift - 503 Service Temporarily Unavailable when i am accessing the app - node.js

I got an app hosted on Openshift here , using node.js and mongo db. It was running perfectly yesterday and today when i am accessing the same, i am getting
Service Temporarily Unavailable Error
I searched some threads and tried this command
rhc tail -a news
and result was
npm ERR! node -v v0.6.20 npm ERR! npm -v 1.1.37 npm ERR! code
ELIFECYCLE npm ERR! message OpenShift-Sample-App#1.0.0 start: `node
server.jsnpm ERR! messagesh "-c" "node server.js"` failed with 1
npm ERR! npm ERR! Additional logging details can be found in: npm ERR!
/var/lib/openshift/4eb0c449f78a4d7e937726185b5e636e/app-root/runtime/repo/npm-debug.log
npm ERR! not ok code undefined npm ERR! not ok code 1
I have also tried
rhc app status -a news
and result was
RESULT: Geargroup nodejs-0.6+mongodb-2.2 is started
also i tried to restart my app
rhc app restart
and it restarted properly.
Could you suggest me what is the issue.?

As mentioned by Hardy, i still dont know which resource got exceeded. But have fixed it now.
Here is what i did.
just get remote access to your app console as described here.
ssh into the console with a client like PuTTy or Cygwin.
just check the status of your app by typing ctl_app status. for me it came like :
App is currently inaccessible or stopped.
just start it by typing ctl_all start
Now app is working as previously.

Are you using a free OpenShift account (free grear)? If so, what is your app doing? Is it very CPU or memory intensive. Your gears are monitored using cgroups and once you exceed any resource limit (cpu, memory, threads) your app gets killed. In this case you would get the message you describe. Unfortunately there is no way of telling which resource you exceeded and you don't get a notification that this was the problem. At least that is something I experienced with my apps. Of course there could be other problems with your app.

Related

Unable to deploy React JS application on Azure App service

I was going through the some of the links to deploy React Js application on the azure app service. But i am facing some problem while deploying the application.
I have added all the necessary things like web.config file to public folder and also added build directory to the workspace.
Deploying web app on azure app service
Deploy Node.js to Azure App Service using Visual Studio Code
followed all the steps but getting below error when i try to deploy on azure app service. Before deploying i do run these commands
npm run build.
2020-08-19T10:44:45.762075166Z A P P S E R V I C E O N L I N U X
2020-08-19T10:44:45.762079567Z
2020-08-19T10:44:45.762083667Z Documentation: http://aka.ms/webapp-linux
2020-08-19T10:44:45.762088167Z NodeJS quickstart: https://aka.ms/node-qs
2020-08-19T10:44:45.762092268Z NodeJS Version : v12.16.3
2020-08-19T10:44:45.762096468Z Note: Any data outside '/home' is not persisted
2020-08-19T10:44:45.762100768Z
2020-08-19T10:44:45.789282727Z Found build manifest file at '/home/site/wwwroot/oryx-manifest.toml'. Deserializing it...
2020-08-19T10:44:45.792738514Z Build Operation ID: |OtQwveNuO0A=.83a2ec6c_
2020-08-19T10:44:47.255197638Z Writing output script to '/opt/startup/startup.sh'
2020-08-19T10:44:47.960307930Z Running #!/bin/sh
2020-08-19T10:44:47.960336532Z
2020-08-19T10:44:47.960345833Z # Enter the source directory to make sure the script runs where the user expects
2020-08-19T10:44:47.960355334Z cd "/home/site/wwwroot"
2020-08-19T10:44:47.960363235Z
2020-08-19T10:44:47.960370735Z export NODE_PATH=$(npm root --quiet -g):$NODE_PATH
2020-08-19T10:44:47.960378436Z if [ -z "$PORT" ]; then
2020-08-19T10:44:47.960386136Z export PORT=8080
2020-08-19T10:44:47.960393937Z fi
2020-08-19T10:44:47.960401238Z
2020-08-19T10:44:47.960408638Z echo Found tar.gz based node_modules.
2020-08-19T10:44:47.960416339Z extractionCommand="tar -xzf node_modules.tar.gz -C /node_modules"
2020-08-19T10:44:47.960424040Z echo "Removing existing modules directory from root..."
2020-08-19T10:44:47.960431740Z rm -fr /node_modules
2020-08-19T10:44:47.960439141Z mkdir -p /node_modules
2020-08-19T10:44:47.960446542Z echo Extracting modules...
2020-08-19T10:44:47.960453842Z $extractionCommand
2020-08-19T10:44:47.960461243Z export NODE_PATH="/node_modules":$NODE_PATH
2020-08-19T10:44:47.960468943Z export PATH=/node_modules/.bin:$PATH
2020-08-19T10:44:47.960476344Z if [ -d node_modules ]; then
2020-08-19T10:44:47.960483745Z mv -f node_modules _del_node_modules || true
2020-08-19T10:44:47.960491245Z fi
2020-08-19T10:44:47.960498446Z
2020-08-19T10:44:47.960505546Z if [ -d /node_modules ]; then
2020-08-19T10:44:47.960524748Z ln -sfn /node_modules ./node_modules
2020-08-19T10:44:47.960532849Z fi
2020-08-19T10:44:47.960540149Z
2020-08-19T10:44:47.960547550Z echo "Done."
2020-08-19T10:44:47.960554951Z npm start
2020-08-19T10:44:48.258132115Z Found tar.gz based node_modules.
2020-08-19T10:44:48.258154316Z Removing existing modules directory from root...
2020-08-19T10:44:48.260461807Z Extracting modules...
2020-08-19T10:44:48.262765098Z tar (child): node_modules.tar.gz: Cannot open: No such file or directory
2020-08-19T10:44:48.262778299Z tar (child): Error is not recoverable: exiting now
2020-08-19T10:44:48.262970515Z tar: Child returned status 2
2020-08-19T10:44:48.262983816Z tar: Error is not recoverable: exiting now
2020-08-19T10:44:48.290740216Z Done.
2020-08-19T10:44:48.512406278Z npm info it worked if it ends with ok
2020-08-19T10:44:48.512836614Z npm info using npm#6.14.4
2020-08-19T10:44:48.512976126Z npm info using node#v12.16.3
2020-08-19T10:44:48.578204629Z npm info lifecycle adal_appp#0.1.0~prestart: adal_appp#0.1.0
2020-08-19T10:44:48.584464048Z npm info lifecycle adal_appp#0.1.0~start: adal_appp#0.1.0
2020-08-19T10:44:48.589867495Z
2020-08-19T10:44:48.589881796Z > adal_appp#0.1.0 start /home/site/wwwroot
2020-08-19T10:44:48.589887297Z > react-scripts start
2020-08-19T10:44:48.589891697Z
2020-08-19T10:44:48.597331914Z sh: 1: react-scripts: not found
2020-08-19T10:44:48.598224588Z npm info lifecycle adal_appp#0.1.0~start: Failed to exec start script
2020-08-19T10:44:48.599091959Z npm ERR! code ELIFECYCLE
2020-08-19T10:44:48.599182267Z npm ERR! syscall spawn
2020-08-19T10:44:48.599258573Z npm ERR! file sh
2020-08-19T10:44:48.599314678Z npm ERR! errno ENOENT
2020-08-19T10:44:48.600738196Z npm ERR! adal_appp#0.1.0 start: `react-scripts start`
2020-08-19T10:44:48.600749897Z npm ERR! spawn ENOENT
2020-08-19T10:44:48.600754497Z npm ERR!
2020-08-19T10:44:48.600758798Z npm ERR! Failed at the adal_appp#0.1.0 start script.
2020-08-19T10:44:48.600763398Z npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
2020-08-19T10:44:48.605436585Z npm timing npm Completed in 125ms
2020-08-19T10:44:48.605621800Z
2020-08-19T10:44:48.605672405Z npm ERR! A complete log of this run can be found in:
2020-08-19T10:44:48.605750311Z npm ERR! /root/.npm/_logs/2020-08-19T10_44_48_601Z-debug.log
But the thing is the same application runs perfectly on local with below commands
npm install , npm start.
and just to verify whether the build which generated after npm run build works or not tried running the application from the build directory with below commands
npm install -g serve
then
serve -s build
then application opens up in browser.
After doing some googling found solution. We need to add the below command in start up command in app service configuration for the Linux machines.
pm2 serve /home/site/wwwroot --no-daemon
Steps:
- Go to App Service
- Navigate to Configuration
- Click on General Settings
- add the above command in Start up command, click on save
- then restart the server
From the log it looks like this is not a Node.js application, but a react application. Therefore react-scripts start is something the Azure App Service doesn't know anything about.
When you run a react app on localhost, it is powered by a development server which indeed is a Node.js server, but once you build it for production using npm run build it is nothing but an index.html file powered by a bunch of .js files and stylesheets. It has no web capabilities in itself.
serve on the other hand is a separate story. As per their description at npmjs.com: Assuming you would like to serve a static site, single page application or just a static file (no matter if on your device or on the local network), this package is just the right choice for you.
But this is not an Azure-like approach.
In production however, if you are using Azure, I recommend using Azure Blob Storage v2, which has static site hosting capabilities. Enable static site hosting in the blob storage and deploy the build folder in a container named $web. Ofcourse all of this is automatically done if you are using vscode with the Azure plugin. Assuming you have signed into Azure thru vscode, right-click on the build folder and select deploy to static site, follow the steps and you will be live with your react app.
However, if you do have a Node.js express backend alongside the react app, then you may put the build folder into the Node.js project at the same level as the node_modules folder and use static routing to have both frontend and backend work as a single package. Explicitly define a route to tell express to respond with the index.html file when asked for. Then you can deploy the whole package into an Azure App Service.
const express = require('express');
const path = require('path');
const port = process.env.PORT || 3006;
const app = express();
app.use(express.json())
app.use(bodyParser.urlencoded({extended: true}));
app.use(bodyParser.json({extended: true}));
app.use(express.static(__dirname + '/build'));
app.get('/', (req, res) => {
res.sendFile(path.join(__dirname + '/build/index.html'))
});
// Then prefix your API endpoints with /api
app.get('/api/user/:id', (req, res) => {
// Code to get user by id
});
app.post('/api/user', (req, res) => {
// Code to save user
});
app.listen(port, () => {
console.log(`App bootstrapped on port ${port}...`);
});
When / is hit, then index.html is served. API calls served as defined with /api/*. I find this mechanism useful many a times.
Good luck.

502 Bad Gateway nginx/1.19.0 in multi-container deployed on AWS Elastic Beanstalk

I have a react application using two instances of nginx as React server and one as router. In practice, a first instance of nginx is use to route to the frontend (to the React application) when the request is different to " /api" (thus, easy application routing between pages) and when endpoint of the request start witn "/api". In the second case Nginx routes to backend server (Express server) that links to postgres or redis (depends on the type of data). In my local machine it works fine. When I deploy project on AWS elastic beanstalk (I use a CI instrument called Travis to do that) I don't have any error, except in running time. In the application I can route through different pages, but when I make a backend call, i.e. http://multidocker-env.eba-jqvdmh2c.us-east-2.elasticbeanstalk.com/api/all (you can try it by clicking) I have the following error:
The excercise should be able to calculate Fibonacci's chain of a number chosen by user. (try it on http://multidocker-env.eba-jqvdmh2c.us-east-2.elasticbeanstalk.com/). THe number chosen by client will go in postgres db, the result of calculation instead, will go to redis. In production Elastic Beanstalk stored client project (react one), server project (backend express server with api endpoint), worker project (that it is able to make calculations) and nginx for routing between frontend and backend api. Postgres and redis are stored in two different AWS Service:
Redis uses AWS ElastiCache
Postgres uses AWS Relational Database Service (RDS)
Those are linked to Elastic Beanstalk using an environment variable settings:
I defined a Security group through every service also, in order to allow TCP traffic through ports from 5432 to 6379. This is my github repo
https://github.com/Aragorn1992gb/multi-docker4 and those are some the error logs:
events.js:292
throw er; // Unhandled 'error' event
^
ReplyError: MOVED 9411 172.31.10.162:6379
at parseError (/app/node_modules/redis-parser/lib/parser.js:193:12)
at parseType (/app/node_modules/redis-parser/lib/parser.js:303:14)
Emitted 'error' event on RedisClient instance at:
at Object.callbackOrEmit [as callback_or_emit] (/app/node_modules/redis/lib/utils.js:91:14)
at RedisClient.return_error (/app/node_modules/redis/index.js:706:11)
at JavascriptRedisParser.returnError (/app/node_modules/redis/index.js:196:18)
at JavascriptRedisParser.execute (/app/node_modules/redis-parser/lib/parser.js:572:12)
at Socket.<anonymous> (/app/node_modules/redis/index.js:274:27)
at Socket.emit (events.js:315:20)
at addChunk (_stream_readable.js:302:12)
at readableAddChunk (_stream_readable.js:278:9)
at Socket.Readable.push (_stream_readable.js:217:10)
at TCP.onStreamRead (internal/stream_base_commons.js:186:23) {
command: 'HSET',
args: [ 'values', '5', 'Nothing yet!' ],
code: 'MOVED'
}
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! # start: `node index.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the # start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2020-06-04T14_09_13_153Z-debug.log
complete logs:
https://pastebin.com/rGcKiseq
As you can see, I used some "console.log" in server/index.js that you can see also in logs. But I can't figure out

How to increase NodeJS heap (--max-old-space-size) in Bitbucket Pipelines for Typescript/Webpack?

I am running into memory issues during the webpack packaging step in a serverless/TypeScript project.
I have tried:
'increase-memory-limit' npm package to no avail
Calling the following command from an npm run script
Calling the following command directly through bitbucket-pipelines.yml.
node --max-old-space-size=4096 ./node_modules/.bin/serverless deploy
It works fine locally with this command, but in Bitbucket pipelines I get the following output:
Serverless: Bundling with Webpack...
internal/child_process.js:323
throw errnoException(err, 'spawn');
^
Error: spawn ENOMEM
at _errnoException (util.js:1022:11)
at ChildProcess.spawn (internal/child_process.js:323:11)
at exports.spawn (child_process.js:502:9)
at Object.exports.fork (child_process.js:103:10)
at fork (/opt/atlassian/pipelines/agent/build/node_modules/worker-farm/lib/fork.js:17:36)
at Farm.startChild (/opt/atlassian/pipelines/agent/build/node_modules/worker-farm/lib/farm.js:106:16)
at Farm.processQueue (/opt/atlassian/pipelines/agent/build/node_modules/worker-farm/lib/farm.js:279:10)
at Farm.<anonymous> (/opt/atlassian/pipelines/agent/build/node_modules/worker-farm/lib/farm.js:97:21)
at ontimeout (timers.js:475:11)
at tryOnTimeout (timers.js:310:5)
at Timer.listOnTimeout (timers.js:270:5)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! platform-state-machine-import#0.0.2 deploy: `node --max-old-space-size=4096 ./node_modules/.bin/serverless deploy "--stage" "feattsify" "--region" "us-east-1"`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the platform-state-machine-import#0.0.2 deploy script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2019-02-07T22_37_25_150Z-debug.log
I've done a lot of googling and searching here to try to find if Bitbucket has some kind of env. variable you can set for runtime args, etc., but can't seem to find anything that doesn't tie back into the aforementioned NPM package.
I encountered a similar issue some time ago in some Jenkins build tasks.
You might be able to solve it using the NODE_OPTION environment variable:
NODE_OPTIONS=--max_old_space_size=4096
From your error stacktrace, it looks like new node processes are beeing spawned.
If you can set this environment variable, it will be used by node to pass arguments to any new processes, and so to the spawned processes that run out of memory here.
https://nodejs.org/dist/latest-v8.x/docs/api/cli.html#cli_node_options_options
Defining a fixed amouth of memory could be tricky, since you will not use the full memory available on the runner machine.
I wrote a code, to get the total memory on the runner machine, deduct 400mb (to other server resources), and assign it as node memory limit.
FREE=$(free -m);
echo "FREE: ${FREE}"
FREE_FINAL=$(echo "$FREE" | grep -F Mem: | grep -o "[0-9]*" | grep -o -m1 "^[0-9]*")
echo "FREE_FINAL: ${FREE_FINAL}"
MEM_LIMIT=$(($FREE_FINAL-400))
echo "MEM_LIMIT: ${MEM_LIMIT}"
echo " - "
export MEMORY_LIMIT="${MEM_LIMIT:=3500}";
export NODE_OPTIONS="--max-old-space-size=${MEMORY_LIMIT}"
echo "NODE_OPTIONS: ${NODE_OPTIONS}"
You can adapt to your case and use the maximum memory to your script, but respect the limit as well.
If you use docker, it can be added to an entrypoint.sh file (that will run on build time, but on each docker start as well).
Increase it from your start command e.g
node --max-old-space-size=8192 server.js
There ended up being two workarounds for this issue. The ENOMEM error was appearing first because I needed more memory for the Node process, and second because the memory I was allotting to the Node process exceeded that which was available in the Bitbucket environment (default 4gb).
Option 1:
Using any of the other answers' options AND ensuring the Bitbucket pipeline environment has enough memory - for me, that meant cranking the size to 2x (see this link).
Option 2:
Adding the 'transpileOnly' flag to your webpack.config.js file, using ts-loader as your transpiler. This skips type-checking and saves memory at package/deploy time. WARNING: You still should type-check your project on build. For me, this meant running tsc before packaging/deploying.
rules: [
// all files with a `.ts` or `.tsx` extension will be handled by `ts-loader`
{
test: /\.tsx?$/,
loader: 'ts-loader',
options: {
transpileOnly: true
}
},
],

Log file (not found) / Heroku

While debugging an app on Heroku, I get this kind of message in the log console:
............
app[web.1]:
app[web.1]: npm ERR! A complete log of this run can be found in:
app[web.1]: npm ERR! /app/.npm/_logs/2017-08-19T03_00_34_428Z-debug.log
heroku[web.1]: Process exited with status 1
My problem is that I don't know how to find this file:
/app/.npm/_logs/2017-08-19T03_00_34_428Z-debug.log
The given path taken as is, is obviously wrong and wherever I look I do not see it.
Anyone knows where the file is?
I have the same question. Not able to find this log file.
However, you can view the log by setting it to verbose
heroku config:set NPM_CONFIG_LOGLEVEL=verbose
and then running git push heroku master

Running AtlasBoard on Azure Web App with iisnode

I´m trying to run AtlasBoard in an Azure Web App, but can´t get it to work. I have created a new board using the "Get started in 30 seconds" steps and the demo board runs fine when I start it locally. It works both by starting it with atlasboard start 3333or if I run node start.
I´ve added the node_modules directory to .gitignore.
I´m using git deployment on Azure and this seems to work fine. The deployment log also shows that the npm modules are installed.
This is the last ouput from the deployment:
Using start-up script start.js from package.json.
Generated web.config.
The iisnode.yml file explicitly sets nodeProcessCommandLine. Automatic node.js version selection is turned off.
Selected npm version 3.5.1
npm WARN Invalid name: "HRMTS AtlasBoard"
npm WARN wwwroot No description
npm WARN wwwroot No repository field.
npm WARN wwwroot No README data
npm WARN wwwroot No license field.
Finished successfully.
However, the app doesn´t seem to start and when I look at the log message in the Kudu console, I get this:
Error: error installing D:\home\site\wwwroot\packages\demo
at process.<anonymous> (D:\Program Files (x86)\iisnode\interceptor.js:73:106)
at emitOne (events.js:96:13)
at process.emit (events.js:188:7)
at process._fatalException (node.js:267:26)
Does anyone have a clue about what´s wrong here?
If you are using the repo of the demo application at https://bitbucket.org/atlassian/atlasboard/src to test on Azure, there several additional modifications we need do to make the test run on Azure Web Apps.
First of all, assume you have successfully deploy the application to Azure, and it failed when start running the application. Then you can leverage Kudu console site or Visual Studio Online extension (refer to the answer of How to install composer on app service? for how to enable extensions of Azure Web Apps) to check the error log at D:\home\site\wwwroot\packages\demo\npm-debug.log.
There are similar errors:
155 error node -v v0.6.20
156 error npm -v 1.1.37
157 error message SSL Error: CERT_UNTRUSTED
It seems that the atlasboard runs command with very low node version on Azure. We can manually modify the dependent scripts to bypass the errors.
If you get error during deployment, you should modify the npm version in package.json before deployment, for example:
"engines": {
"npm": ">2.0.0",
"node": ">=0.10"
},
After deployment:
check and modify the port in the start.js in the root directory:
atlasboard({port: process.env.port||3000 , install: true},
function (err) {
if (err) {
throw err;
}
});
modify the install function in D:\home\site\wwwroot\node_modules\atlasboard\lib\package-dependency-manager.js to use a higher npm version and remove --production param, e.g :
...
var npmCommand = isWindows ? "D:\\Program Files (x86)\\npm\\3.5.1\\npm.cmd" : "npm";
executeCommand(npmCommand, ["install", pathPackageJson], function(err, code){
...
})
Then restart your website.
Any further concern, please feel free to let me know.

Resources