Jenkins service stops completely during a node js build, getting a 502 bad gateway with an nginx setup on Ubuntu - node.js

I have install Jenkins on Ubuntu VPS and I am doing a nodejs build, which, at some point triggers a 502 bad gateway in nginx and the Jenkis service stops.
The build seems to stop at random times, sometimes when installing the npm models, other times when using gulp and so on.
I have to restart it manually (sudo service jenkins restart).
Here is the console output from a build:
+ jspm -v
0.16.15
Running against local jspm install.
[workspace] $ /bin/sh -xe /tmp/hudson1116016846111302145.sh
+ jspm install
Looking up npm:babel-core
Looking up npm:babel-runtime
Looking up npm:core-js
Updating registry cache...
ok Up to date - babel as npm:babel-core#^5.8.24 (5.8.34)
Looking up github:jspm/nodelibs-fs
Looking up github:jspm/nodelibs-path
Looking up github:jspm/nodelibs-process
Looking up github:systemjs/plugin-json
Looking up npm:process
Looking up npm:path-browserify
Looking up github:jspm/nodelibs-assert
Looking up npm:assert
Looking up npm:util
Looking up npm:inherits
Looking up github:jspm/nodelibs-util
ok Up to date - core-js as npm:core-js#^1.1.4 (1.2.6)
ok Up to date - babel-runtime as npm:babel-runtime#^5.8.24 (5.8.34)
ok Install tree has no forks.
ok Install complete.
[workspace] $ /bin/sh -xe /tmp/hudson4025483240884121474.sh
+ npm run build
> dxx-app#1.0.0 build /var/lib/jenkins/jobs/dxx-app/workspace
> gulp build
[15:31:11] Using gulpfile ~/jobs/dxx-app/workspace/gulpfile.js
[15:31:11] Starting 'build'...
[15:31:11] Starting 'clean'...
[15:31:11] Finished 'clean' after 19 ms
[15:31:11] Starting 'jspm-build'...
[15:31:11] Starting 'copy-assets'...
[15:31:11] Starting 'copy-fonts'...
[15:31:11] Starting 'less'...
[15:31:11] Finished 'copy-assets' after 226 ms
[15:31:12] Finished 'less' after 191 ms
[15:31:12] Finished 'copy-fonts' after 213 ms
502 Bad Gateway
nginx/1.4.6 (Ubuntu)
The proble is, when Jenkis fails, it does not record anything in my build logs, so I do not know what is happening.
I have seen online you should increase the timeout from the nginx.conf, so I have added the following to the config file:
http {
##
# Increased timeout to fix 502 bad gateway error
##
fastcgi_buffers 800 10006k;
fastcgi_buffer_size 1000k;
fastcgi_connect_timeout 999999;
fastcgi_send_timeout 999999;
fastcgi_read_timeout 999999;
Also, in nginx, sites-available, I have the following configuration:
server {
listen 80;
server_name jenkins.domain.com;
ignore_invalid_headers off;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8080;
proxy_connect_timeout 999999999999;
proxy_send_timeout 999999999999;
proxy_read_timeout 999999999999;
}
}
Do you have any ideas?

As explained in this thread you can increase the memory in the /etc/default/jenkins where you can adjust the following flag
JAVA_ARGS="-Xmx256m" # Choose your desired memory instead of 256

I had the same issue in Ubuntu server.
I updated the JAVA_ARGS
nano /etc/default/jenkins
JAVA_ARGS="-Xmx256m"
it worked for me.

I experienced the same issue on Ubuntu server. I had to change JAVA_ARGS value in
/etc/default/jenkins
to:
JAVA_ARGS="-Xmx128m"
before the react build process stopped throwing 502 gateway error message. This is on Jenkins 2.263.4

Related

Running .NET Core 5.0 on Debian Nginx

I wanted to move a project I have working on Windows to run on a Unix machine.
The machine running on Debian 9 with Nginx.
This project runs absolutely fine on Windows with IIS.
I've followed all the instructions on here created a service for this to run on the start of the machine and Nginx configuration to proxy the connection from the port I want to use to port 5000.
When I start the application running Dotnet Myddl.dll it starts and says it is only listening on port 5000.
Then when I try to access it, I can see a warning.
warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
Failed to determine the HTTPS port for redirect.
I know it is related to my app redirecting to HTTPS and not knowing where to redirect it, but how do I resolve this?
My service
[Unit]
Description=Myapp API
[Service]
WorkingDirectory=/var/www/myapp/publish
ExecStart=/usr/bin/dotnet /var/www/myapp/publish/myapp.dll
Restart=always
# Restart service after 10 seconds if the dotnet service crashes:
RestartSec=10
KillSignal=SIGINT
SyslogIdentifier=dotnet-example
User=www-data
Environment=ASPNETCORE_ENVIRONMENT=Production
Environment=DOTNET_PRINT_TELEMETRY_MESSAGE=false
[Install]
WantedBy=multi-user.target
server {
listen 6969;
server_name mysite.net *.mysite.net;
location / {
proxy_pass http://localhost:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
EDIT:
I've been trying to resolve this and still can't. When I start the app on the unix machine I get the following
root#myhost:/var/www/myapp/publish# dotnet Myapp.dll info: Microsoft.Hosting.Lifetime[0] Now listening on: localhost:5000 info: Microsoft.Hosting.Lifetime[0] Application started. Press Ctrl+C to shut down. info: Microsoft.Hosting.Lifetime[0] Hosting environment: Production info: Microsoft.Hosting.Lifetime[0] Content root path: /var/www/myapp/publish
Obviously it is missing https option, and I can't figure why.
EDIT2:
I've published the app as self contained for Linux-x64, and now I do not get the warning saying that it couldn't determine the https port, on my browser I get redirected to https://mydomain:5001 when I access it on http://mydomain:6969
Still I do not get the app listening on https on Unix, just on Windows.
EDIT3:
Noticed that if I go to one of my endpoints e.g. http://IP:6969/api/users I get a 500 response.
EDIT4:
When I was loading my application locally, I was getting straight through to the swagger page, such as /swagger/index.html, for some reason my API when complied for Linux does not accept this URL and returns me a 404, but if I get to one of my endpoints e.g. /api/users, it does return me the data I was expecting for.
Default https port should be 5001.
You can set https port following offical docs.
Or disable relatied middlware. Use nginx for https termination if needed.
I was trying to Migrate an API, and as on Windows it was returning me to my swagger page located on /swagger/index.html I was expecting the same to happen, which for some reason, doesn't.
So if I access one of my endpoints (e.g. /api/users) it does work fine.

Nginx 502 Bad Gateway when Jenkins pipeline is running a docker for React app

I have an Ubuntu 18.04 server running in a Droplet (DigitalOcean) secured with SSL and using an Nginx reverse proxy. Also Jenkins in running in my server (not in any docker) and configured to be accessed under the domain I created for it: jenkins.testdomain.com (all these steps following DO docs)
So the goal is to manage the deployment of a Node.js-React application to my testdomain.com later, by now, I just want to create the dist folder generated, after the 'npm build', within the /var/lib/jenkins/workspace/ , just that.
By now, I'm able to access my jenkins.testdomain.com site alright, trigger the pipeline to start the process after pushing to my repo, and start to run the stages; but it's here when start to fail nginx, when the pipeline reaches the Deliver phase (let's read 'npm build' phase), sometimes in the Build phase ('npm install').
It's at this point, reading the Jenkins console output where I see when it gets stuck and eventually shows a 502 Bad Gateway error. I will need to run the command systemctl restart Jenkins in my server console, to have access again. After restarting, the pipeline resume the work and seems to get the job done :/
In the /var/log/nginx/error.log for nginx I can read:
*1 connect() failed (111: Connection refused) while connecting to upstream, client: 85.146.85.194, server: jenkins.testdomain.com,
request: "GET /job/Basic%20NodeJS-React%20app/8/console HTTP/1.1",
upstream: "https:
//127.0.0.1:8080/job/Basic%20NodeJS-React%20app/8/console", host:
"jenkins.testdomain.com", referrer: "https:
//jenkins.testdomain.com/job/Basic%20NodeJS-React%20app/8/"
*1 SSL_do_handshake() failed (SSL: error:1408F10B:SSL routines:ssl3_get_record:wrong version number) while SSL handshaking
to upstream, client: 85.146.85.194, server: jenkins.testdomain.com,
request: "GET /favicon.ico HTTP/1.1", upstream: "https:
//127.0.0.1:8080/favicon.ico", host: "jenkins.testdomain.com",
referrer: "https:
//jenkins.testdomain.com/job/Basic%20NodeJS-React%20app/8/console" ...
In the Jenkinsfile of my node-js-react app (from jenkins repo), the agent looks like this:
pipeline {
agent {
docker {
image 'node:6-alpine'
args '-p 3000:80'
}
}
environment {
CI = 'true'
}
stages {
// Build, Test, and Deliver stages
}
}
And my jenkins.testdomain.com configuration (/etc/nginx/sites-available/jenkins.testdomain.com) is like this (pass tests from nginx -t):
server {
listen 80;
root /var/www/jenkins.testdomain.com/html;
server_name jenkins.testdomain.com www.jenkins.testdomain.com;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Fix the "It appears that your reverse proxy set up is broken" error.
proxy_pass http://localhost:8080;
# High timeout for testing
proxy_connect_timeout 1200s;
proxy_send_timeout 1200s;
proxy_read_timeout 1200s;
proxy_redirect http://localhost:8080 https://jenkins.testdomain.com;
# Required for new HTTP-based CLI
proxy_http_version 1.1;
proxy_request_buffering off;
# Required for HTTP-based CLI to work over SSL
proxy_buffering off;
}
# Certbot auto-generated lines...
}
Any help would be very welcomed 3 days struggling with this and playing around with the different proxy_ directives from nginx and so...
Thanks in advance!
OK just add an update that some days after my latest post, I realized that the main and only reason the server was going down was a lack of resources in the droplet.
So I was using a droplet with 1GB of RAM, 25GB HD, etc.. (the most basic one), so I chose to update it to use at least 2GB of RAM and indeed, that made it work as I was expecting. Everything until now works fine and that issue didn’t happen again.
Hope it helps if someone experiences the same issue.

How to set automatically call "npm start" in nodejs?

I am using nodejs for web application. In this I am frustrated with continuously firing command "npm start" when applying small changes on any files.
Question 1 : Is there any way for how to keep automatically call my "npm start" without opening command prompt?
Question 2 : And also I want to remove this port display in url.
I want like below result, example:
Current url: http://localhost:3200/login
Expected url: http://localhost/myprojectname/login
node-dev will automatically restart your application whenever you save a code change. Install it with npm install -g node-dev then run your application with node-dev server.js (or whatever the entrypoint file for your project is called if not server.js).
For development, it's easiest to just live with the port in the URL. To get rid of it, you need to run your application on port 80 which requires running as root which is insecure. There are ways to do it (via ngrok iptables, nginx, etc, but most developers opt to just deal with the port in the URL during local development
Try nodemon module, Nodemon is a utility that will monitor for any changes in your source and automatically restart your server.Perfect for development on local machine, you no longer need to deal with npm start every time when you change the code.It will automatically restart the server and changes will be reflect instantly.
install it globally
npm install -g nodemon
then try to run your app using nodemon app.js instead of npm start or node app.js
If you want to use port 80, you should run your application with root/administrator privileges, and make sure no other service is running on that port another solution is which I strongly recommend is to use nginx server reverse proxy settings .Install nginx server on your machine, Following are the settings that you can apply using nginx reverse proxy to remove any port number you want from the url.
server {
listen 80;
server_name yourdomain.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://localhost:3000/;
proxy_redirect http://localhost:3000/ https://$server_name/;
}
}
Hope this helps you.

Deploying ASP.NET 5 MVC 6 Application to Linux & IIS 7

I have asp.net application which and just be substituted with the default MVC 6 project for the purposes of this post. I am trying to decide whether or not to deploy it to my linux Debian 8.2 Jessie Server (Preferred) or an Windows Server 2008 R2 IIS 7.
As far is Linux is concerned I have followed the following sites for instructions and I am confused on how and where to deploy the site files.
Installing ASP.NET 5 On Linux
How to Install ASP.Net 5 on Ubuntu Linux
Also I'm not sure if i am suppose to be using Kestrel or Mono. Then i hear Docker is being tossed around as well. I just need to know the best way (not necessarily the easiest way) to host an MVC 6 application on linux.
As far is IIS 7 goes i have followed the video instructions Here ASP.NET5 MVC Deployment to IIS Web Server yet i get the following error when i try to view the site:
HTTP Error 403.14 - Forbidden
The Web server is configured to not list the contents of this directory.
Most likely cause:A default document is not configured for the requested URL, and directory browsing is not enabled on the server.
Please help me out or point me in the right direction for me to resolve this.
***** Edit *****
Okay I have been determined to get this working correctly. Since i have first asked the question i have progressed in a few areas.
Fist i have dnx, dnu, dnvm install properly. I am able to run a dnu restore to get all of my dependencies in. I have nginx installe which directs the incoming requests to the 127.0.0.1:5000 which i had set in my project.json file
project.json:
"commands": {
"web": "Microsoft.AspNet.Server.Kestrel --server Kestrel --server.urls http://localhost:5000",
"ef": "EntityFramework.Commands"
},
nginx:
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html;
index index.html index.htm;
# Make site accessible from http://localhost/
server_name aspnet.dev www.aspnet.dev;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:5004;
}
}
I then navigate to the root folder and dnx web which brings up the following:
Hosting environment: Production
Now listening on: http://localhost:5000
Application started. Press Ctrl+C to shut down.
now since nginx is directing the request to the asp project with
proxy_pass http://127.0.0.1:5004
i can see some request being handled.
info: Microsoft.AspNet.Hosting.Internal.HostingEngine[1]
Request starting HTTP/1.0 GET http://***.**.**.***/
info: Microsoft.AspNet.Mvc.Controllers.ControllerActionInvoker[1]
Executing action method AspNet5.Controllers.HomeController.Index with arguments () - ModelState is Valid'
info: Microsoft.AspNet.Mvc.ViewFeatures.ViewResultExecutor[1]
Executing ViewResult, running view at path /Views/Home/Index.cshtml.
info: Microsoft.AspNet.Mvc.Infrastructure.MvcRouteHandler[2]
Executed action AspNet5.Controllers.HomeController.Index in 0.1468ms
info: Microsoft.AspNet.Hosting.Internal.HostingEngine[2]
Request finished in 0.1849ms 200 text/html; charset=utf-8
But yet it doesn't server any html to the browser. Any suggestions?
If anyone is intersted in how I just corrected my issue see this post: Hiccups with Hosting ASP .NET 5 apps on Linux (RC1)

Install Ghost Blog

I've been trying to get Ghost.io installed on my web server for quite sometime. I have a VPS with Centos 6 and Cpanel.
Today I found a script at http://www.allaboutghost.com/one-click-ghost-install-script/ that said you could just enter a command into your ssh terminal and have it all installed for you.
Command
wget -O - https://raw.github.com/howtoinstallghost/installghost.sh/master/installGhost.sh | sudo bash
I did this and it appears to have worked, I didn't get any errors but now I am not able to find the install in either FileZilla or by using my web browser. The website says that it installs in the /var/www/ghost/ directory but I can't find that. If I use cd /var/www/ghost/ in the ssh it takes me right to it and even lets me edit the config.example.js file.
If I direct my browser to www.mydomain.com:80 since the site says it installs on port 80 it just takes me back to my home page.
What am I missing and what do I need to do?
As Per the comments, I did follow the instructions on the github page. Now All I get when I visit mydomainname.com/ghost/
Ghost is installed really easy. You'd better don't use 3rd party scripts for that as it might really vary from system to system. All you need is to get Node.js installed and then follow instruction for example from here: https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-ghost-on-ubuntu-16-04. They are very detailed and must work for Centos as well.
Most common errors are:
- Not configuring a reverse proxy (nginx or apache) to link to your ghost install on port 2368. Here is an example for Nginx:
server {
listen 80;
server_name your_domain_or_ip_address;
location / {
proxy_set_header HOST $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://127.0.0.1:2368;
}
}
If you now see default home page it means some web server is already running and intercept all the requests and parks them to a default location (e.g. /var/www)
- If you want Ghost to be the one and the only web server on your VPS, you have to remove or shut down currently installed web server and try to configure ghost to answer on port 80 like this:
server: {
host: '0.0.0.0',
port: '80'
}
I didn't test it but has to work. This kind of install is not recommended and insecure. I suppose you can configure reverse proxy from Cpanel, but not sure.
The best and easiest way to set up Ghost is using SSH.
Hope it helps. For any further help, you have to provide more details and possibly logs and configs. Most of the possible errors you'll catch when installing or starting your blog with npm
sudo npm install --production
sudo npm start --production
Good luck with your deployment.
Hey you don't need to do that much you can just launch ghost from digitalpress for free...
Know how to host ghost blog for free out here : https://treanches.digitalpress.blog/hosting-ghost-blog/
You can think this as self promotion but this article is so much understanding you won't regret reading it

Resources