I'm interested in hosting nodejs applications in a cloud and I'm looking for a free cloud hosting for my purpose. I've found that Amazon has one but I have the following question: Are there any tutorials around how I can set up and run nodejs application in Amazon EC2?
EDIT: Can you provide any good hostings for nodejs (except heroku)?
I've been using Node.js with Amazon EC2 for a while and was quite happy with both of them. For the moment AWS seems to be the cheapest and the most robust cloud provider, so picking up Amazon wouldn't be a mistake. There's nothing special about running Node.js in the cloud - you work with it like if it were your own PC. Below are some general steps to follow for the simplest Node.js application running on EC2 Ubuntu server:
Create Amazon EC2 account.
From AWS console start t1.micro instance with any Ubuntu AMI (example).
Login via SSH to your instance.
Install node.js: sudo apt-get install nodejs
Create new file test_server.js with the following content:
require("http").createServer(function(request, response){
response.writeHeader(200, {"Content-Type": "text/plain"});
response.write("Hello World!");
response.end();
}).listen(8080);
Start the server: node test_server.js
Check it's working from another console: curl http://localhost:8080
Check out these tutorials (updated for 2021)
How to Deploy a Node.js Application On AWS EC2 Server
How to Deploy a Node.js application in AWS EC2
How To Deploy Your Node.js App On AWS With NGINX And SSL
Based on this tutorial, here's an updated step by step:
1) Make an account on Amazon Web Services.
2) Create an EC2 instance; I chose Ubuntu micro.
3) Configure Security Group (name it "Node") and add ports:
HTTP (80), HTTPS (443), and a custom TCP port for your Node app (e.g. 3000)
4) Launch the instance and save the pem file (private key), e.g. "node.pem".
5) On Windows - install Cygwin + OpenSSH package. it is also recommended to install WinScp to have "explorer like" access to the linux.
6) Open Cygwin Terminal as Administrator, and set correct permissions to "node.pem" file:
chown :Users node.pem
chmod 400 node.pem
7) Find your EC2 instance public DNS name in the EC2 dasboard, and connect to it with SSH:
ssh -i node.pem ubuntu#{your EC2 public DNS name}
8) Update Ubuntu and install NodeJS:
sudo apt-get update
curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo apt-get install -y build-essential
9) Copy your NodeJS application into the EC2 instance (via Cygwin, or Winscp).
10) Install all of your Node app required modules:
cd /home/ubuntu/My_Node_App
npm install --save
11) Re-route ports with IPtables so that your app can be accessed on default http port 80:
sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 3000
To view the iptables routing entries, run:
sudo iptables -t nat -L
If you need to remove routing entry (first line), run:
sudo iptables -t nat -D PREROUTING 1
12) Run your app as a background process:
sudo nohup node app.js &
To kill your app process:
ps -ef | grep app.js
sudo kill {proccess id number}
My blog post on how to deploy Node-based apps on EC2: http://devblog.daniel.gs/2014/01/deploying-node-apps-on-aws-ec2-with.html
Explaining:
Deploying Node apps from your github repo (private+public)
Automating the deployment process using scripts
Reverse proxy using Nginx
and using Forever utility.
Hope this helps.
There are quite some hosting solutions for Node.js available, here are a couple of these:
Joyent
Joyent is the corporate sponsor and trademark owner of Node.js and provides an appealing alternative to Amazon EC2 for many things, not the least Node.js hosting of course, see the Joyent's Node.js Development Environment (please check the Node.js™ Development SmartMachine Terms of Service though).
Apparently they are just restructuring this development offering though:
For the past year, Joyent Cloud has provided a free development
sandbox for users of Node.js. Over time, the community has made it
clear that they want more tools and more capacity. To this end, we are
excited to announce a partnership with Nodejitsu to provide both of
these in a world-class Node.js development environment with
Nodejitsu's development and management tools running on Joyent Cloud's
Infrastructure-as-a-Service platform. The new service will launch very
shortly.
Accordingly, it is not entirely clear yet how the pricing options for a production hosting of a Node.js solution will end up, but given Joyent's competitive pricing, I'd expect a similar option at least.
Cloud Foundry
The Cloud Foundry Open Platform as a Service Project support Node.js as well, amongst many other frameworks (which makes the platform so exciting), The platform is getting quite some traction recently and is meanwhile used by several solution Platform as a service (PaaS) providers as their backend accordingly - amongst these are (in no particular order and not necessarily complete):
AppFog - Simple PaaS for Java, Node, .Net, Ruby, PHP, MySQL, Mongo, PostgreSQL, and more...
Freedom to move between IaaS at will with the easiest pricing in the cloud.
Cloud Foundry (VMware) (corporate sponsor of Cloud Foundry) - Deploy and scale applications in seconds, without locking yourself into a single cloud.
Iron Foundry - Iron Foundry is an open source project that extends Cloud Foundry™ to the .NET ecosystem by providing services, installers, and developer tools.
Most of these are in beta still and the pricing isn't settled yet, but given the competition I'd expect quite some interesting options here over time.
The easiest way to run node.js for free on EC2 is IMHO on Heroku.
check out this complete tutorial here.
This tutorial shows how to install Node.js on EC2 and configure HTTP ports and nginx for port forwarding as well as using supervisor to run the Node.js forever as it normally stops on closing your SSH console session.
I just went through the Heroku sign-up and application tutorial. Could not have been easier. What a delightful experience...
...right up to the point where you can't have a MongoDB instance as a free gear. The minimum cost (other than a free trial month) is $18/month per GB of storage.
Honestly, the better choice then is Openshift. It's got three free gears which is enough for a lot of beginner stuff like what I'm doing. Both Heroku and Openshift are within Amazon's space but their customer interface is different. I thought Heroku's was easier for beginners to get started but as I mentioned, there's no free lunch on the database side of things.
Related
I started with Grafana to monitor on-premise Linux Servers. I am using the Cloud Portal. On the Grafana Dashboard, I installed the Linux Server Integration using this tutorial -> https://grafana.com/docs/grafana-cloud/quickstart/agent_linuxnode/.
I used the command line on one server to setup the agent:
sudo ARCH=amd64 GCLOUD_STACK_ID="XXXXX" GCLOUD_API_KEY="xxxxx" GCLOUD_API_URL="https://integrations-api-eu-west.grafana.net" /bin/sh -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/release/production/grafanacloud-install.sh)"
sudo systemctl restart grafana-agent.service
It works perfectly with one server. However, when I added a new Remote Linux Server with the same command line, it replaced the previous server in the dashboard and I cannot select the other server. I feel I should not use the same command line, but I cannot find what parameters I should modify.
Did someone face the same issue and found a solution ?
Thank you in advance,
B.
PS: Ideally I would make it work using docker containers on each Linux Server, communicating to the Cloud Portal
Assume sudo systemctl restart grafana-agent.service is restarting a specific server with the execution command in /etc/systemd/system/grafana-agent.service
If you want to have another grafana-agent you need additional service file. For example: grafana-agent-2.service with different configuration.
When I ssh into the EC2 instance and look around the server configuration files, I see Apache config files, NGINX config files. Why are both servers there? Which one takes priority? Do both servers run simultaneously? Do they work together? Where would my server side code go for Node.js?
It is Node.js running on 64bit Amazon Linux/4.10.2 on an Elastic Beanstalk environment.
To quote the documentation:
The AWS Elastic Beanstalk Node.js platform is a platform version for
Node.js web applications that can run behind an nginx proxy server,
behind an Apache server, or standalone.
I would assume that to simplify deployment, Elastic Beanstalk chooses to deploy both NGINX and Apache, regardless of which mechanism you choose to serve content.
The configuration options are ProxyServer=apache/nginx/none.
I asked this question in Serverfault and got this answer. You can run the command ps ax | grep -E '(apache2|httpd|nginx)' to see which one is running.
I'm pretty new to DevOps and I'm trying to set up my Node.js app on a AWS server instance. Steps I've taken:
Set up Elastic IP
Launched EC2 instance with Ubuntu server
Connected IP to instance
Allowed incoming connections on port 3000
SSH'd into the server with a .pem file
Now I'm at the point where I need to get my files uploaded to the server. I've used FileZilla (and like it) in the past to upload files but the initial part was already set up. When I set up the site on FileZilla there is no /var/www folder on the remote site.
Don't know how to connect these dots.
Also not sure what I need to run once I successfully upload the files. I imagine npm install when I'm ssh'd into the server? Most of the tutorials out there only go through the basic instance setup.
Thanks!
You don't need to have /var/www. Also, it's better that you use a version control and a remote repository like Github and then SSH to your EC2 and then clone your repository there.
Then cd into your repo and run npm install and then start your app.
And check.
Once you connect to the EC2 instance then clone your code in there. It not mandatory to be in /var/www/html but, it's best practice to keep it there. Once you clone npm install into your project home directory so all the required packages get installed. Then for running your node application in production you have to run it on service as pm2, supervisor, forever, passenger, etc. You can use any of these services and configured appropriately to run your application on desired port. As with pm2, you can follow this guide, install pm2 Then you can run with the following command w.r.t. your environment, like I want to run my application on port 5555 for production
$ PORT=5555 pm2 start app.js --name API --env production -f
Check the status using pm2 list Now, your application is running on http://server-ip:5555/ But, you won't be typing port number every-time. So, you need to configure the web server in front of your application like apache or nginx which will forward all request to your application running port. You could find the best guide to their home page. Then your application is available at http://server-ip/ You can follow this for single configuration of multiple node apps
Hope this helps.
I have already taken a look at How to install mongodb in Elastic Beanstalk? dated 2014, which no longer works. as well as https://docs.mongodb.org/ecosystem/platforms/amazon-ec2/#manually-deploy-mongodb-on-ec2
I have set up a new elastic beanstalk environment running on node.js with 1 ec2 micro instance '64bit Amazon Linux 2016.03 v2.1.0 running Node.js'
I have already tried using ssh to connect into my instance and install the mongodb packages using yum command:
$ sudo yum install -y mongodb-org-server mongodb-org-shell mongodb-org-tools
and received this call back:
Loaded plugins: priorities, update-motd, upgrade-helper
No package mongodb-org-server available.
No package mongodb-org-shell available.
No package mongodb-org-tools available.
Error: Nothing to do
When I first ssh 'd into my instance, I received this error warning:
This EC2 instance is managed by AWS Elastic Beanstalk. Changes made via SSH
WILL BE LOST if the instance is replaced by auto-scaling. For more information
on customizing your Elastic Beanstalk environment, see our documentation here:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html
Currently my environment is set up as a single instance environment, to save on costs. However, in the future I will upgrade to an auto-scaling environment.
Because of this, I am asking is it recommendable to make any changes via ssh in ec2, or should I only be using EB CLI?
I have both EC2 and EB CLI installed locally, however I have never used EB CLI before. If I should be using EB, does anyone have a recommended way to install mongodb?
In case anyone is looking for an answer, here is the advice I received from aws business support.
All code deployed to Elastic Beanstalk needs to be "stateless" I.E. Never make changes directly to a running beanstalk instance using SSH or FTP.... As this will cause inconsistencies and or data lose!
- Elastic Beanstalk is not designed for application that are not stateless.
The environment is designed to scale up and down pending on your Network / CPU load and build new instances from a base AMI. If an instance has issues or the underlying hardware, Elastic Beanstalk will terminate these running instances and replace with new instances. Hence, why no code modification must be applied or done "directly" to an existing instance as new instances will not be aware of these direct changes. ALL changes / code needs to be either uploaded to Elastic Beanstalk console or the CLI tools and the pushed to all the running instances.
More information on Elastic Beanstalk design concepts can be read at the following link
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.concepts.design.html
Suggested Solution:
With the above in mind, if using MongoDB to store application data our recommendation would be to DE-couple the MongoDB environment from your Node.js application.
I.E Create a MongoDB Server outside of Elastic Beanstalk, example launching MongoDB directly on a EC2 instance and have your Elastic Beanstalk Node.js application connect to MongoDB Server using connection settings in your app.
-Creating MongoDB
Below is some example links that may be of use for your scenario for creating a MongoDB Server.
Deploy MongoDB on EC2,
https://docs.mongodb.org/ecosystem/platforms/amazon-ec2/
MongoDB node client
https://docs.mongodb.org/getting-started/node/client/
MongoDB on the AWS Cloud quick start guide
http://docs.aws.amazon.com/quickstart/latest/mongodb/architecture.html
-Adding environment variables to Elastic Beanstalk to reference your MongoDB server
Once you have created your MongoDB Server you can pass the needed connection settings to your Elastic Beanstalk environment using environment variables.
Example using .ebextensions .config which you can add Mongo URL / ports / users etc..
option_settings:
- option_name: MONGO_DB_URL
value: "Your MongoDB EC2 internal IP address"
Information on how to use environment properties and read them from within your application can be seen below.
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_nodejs.container.html#create_deploy_nodejs_custom_container-envprop
And information using .ebextensions .config can be found at the following link
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions.html
Alternatively you can also set environment variable using the cli or via the AWS Console
eb cli set environment variables can be read per the below link.
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb3-setenv.html
Using AWS Console
To set system properties (AWS Management Console)
Open the Elastic Beanstalk console.
Navigate to the management console for your environment.
Choose Configuration.
In the Software Configuration section, choose Edit.
Under Environment Properties, create your name / values ...
Accessing Environment Configuration Settings
Inside the Node.js environment running in AWS Elastic Beanstalk, you can access the environment variables using process.env.ENV_VARIABLE similar to the following example.
process.env.MONGO_DB_URL
process.env.PARAM2
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_nodejs.container.html#create_deploy_nodejs_custom_container-envprop
Summary:
In summary I would recommend the following steps to integrate MongoDB with Elastic Beanstalk environments.
Step 1) Create a MongoDB Server outside of Elastic Beanstalk
Step 2) Create your Node.js application in Elastic Beanstalk that connect to your MongoDB server
3 options:
1) SSH into an eb instance and install the mongo CLI manually:
sudo yum-config-manager --add-repo https://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/4.0/x86_64/
sudo yum install --nogpgcheck -y mongodb-org-shell
Disadvantage is that if EB scales down its number of instances and the instance you are currently on gets terminated, you get kicked out of the SSH session:
The system is going down for halt NOW!
Connection to 1.2.3.4 closed by remote host.
Connection to 1.2.3.4 closed.
ERROR: CommandError - An error occurred while running: ssh.
You then need to start all over again: connect to instance, install mongo CLI...
2) Pre-install mongo CLI on instances by using a .config file:
container_commands:
01-mongocli:
command: "sudo yum-config-manager --add-repo https://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/4.0/x86_64/;sudo yum install --nogpgcheck -y mongodb-org-shell"
ignoreErrors: true //use this the ensure instance deployment even if mongo CLI installation fails
Again, if the instance gets terminated by auto-scaler you'd have to connect again, but you don't have to the install mongo CLI manually.
3) Create a separate instance that hosts your mongo CLI, as described in #amyloula's answer. If your mongodb is within an VPC you need to create that separate instance also within the VPC. You will then need to create a Gateway to access the instance publicly, as you cannot connect directly to an instance in a VPC.
Is there a brief guide to explain how to start up a application when the instance starts up and running? If it were one of the services installed through yum then I guess I can use /sbin/chkconfig to add it to the service. (To make it sure, is it correct?)
However, I just want to run the program which was not installed through yum. To run node.js program, I will have to run script sudo node app.js at home directory whenever the system boots up.
I am not used to Amazon Linux AMI so I am having little trouble finding a 'right' way to run some script automatically on every boot.
Is there an elegant way to do this?
One way is to create an upstart job. That way your app will start once Linux loads, will restart automatically if it crashes, and you can start / stop / restart it by sudo start yourapp / sudo stop yourapp / sudo restart yourapp.
Here are beginning steps:
1) Install upstart utility (may be pre-installed if you use a standard Amazon Linux AMI):
sudo yum install upstart
For Ubuntu:
sudo apt-get install upstart
2) Create upstart script for your node app:
in /etc/init add file yourappname.conf with the following lines of code:
#!upstart
description "your app name"
start on started mountall
stop on shutdown
# Automatically Respawn:
respawn
respawn limit 99 5
env NODE_ENV=development
# Warning: this runs node as root user, which is a security risk
# in many scenarios, but upstart-ing a process as a non-root user
# is outside the scope of this question
exec node /path_to_your_app/app.js >> /var/log/yourappname.log 2>&1
3) start your app by sudo start yourappname
You can use forever-service for provisioning node script as a service and automatically starting during boots. Following commands will do the needful,
npm install -g forever-service
forever-service install test
This will provision app.js in the current directory as a service via forever. The service will automatically restart every time system is restarted. Also when stopped it will attempt a graceful stop. This script provisions the logrotate script as well.
Github url: https://github.com/zapty/forever-service
As of now forever-service supports Amazon Linux, CentOS, Redhat support for other Linux distro, Mac and Windows are in works..
NOTE: I am the author of forever-service.
Quick solution for you would be to start your app from /etc/rc.local ; just add your command there.
But if you want to go the elegant way, you'll have to package your application in a rpm file,
have a startup script that goes in /etc/rc.d so that you can use chkconfig on your app, then install the rpm on your instance.
Maybe this or this help. (or just google for "creating rpm packages")
My Amazon Linux instance runs on Ubuntu, and I used systemd to set it up.
First you need to create a <servicename>.service file. (in my case cloudyleela.service)
sudo nano /lib/systemd/system/cloudyleela.service
Type the following in this file:
[Unit]
Description=cloudy leela
Documentation=http://documentation.domain.com
After=network.target
[Service]
Type=simple
TimeoutSec=0
User=ubuntu
ExecStart=/usr/bin/node /home/ubuntu/server.js
Restart=on-failure
[Install]
WantedBy=multi-user.target
In this application the node application is started. You will need a full path here. I configured that the application should simply restart if something goes wrong. The instances that Amazon uses have no passwords for their users by default.
Reload the file from disk, and then you can start your service. You need to enable it to make it active as a service, which automatically launches at startup.
ubuntu#ip-172-31-21-195:~$ sudo systemctl daemon-reload
ubuntu#ip-172-31-21-195:~$ sudo systemctl start cloudyleela
ubuntu#ip-172-31-21-195:~$ sudo systemctl enable cloudyleela
Created symlink /etc/systemd/system/multi-user.target.wants/cloudyleela.service → /lib/systemd/system/cloudyleela.service.
ubuntu#ip-172-31-21-195:~$
A great systemd for node.js tutorial is available here.
If you run a webserver:
You probably will have some issues running your webserver on port 80. And the easiest solution, is actually to run your webserver on a different port (e.g. 4200) and then to redirect that port to port 80. You can accomplish this with the following command:
sudo iptables -t nat -A PREROUTING -i -p tcp --dport 80 -j REDIRECT --to-port 4200
Unfortunately, this is not persistent, so you have to repeat it whenever your server restarts. A better approach is to also include this command in our service script:
ExecStartPre to add the port forwarding
ExecStopPost to remove the port forwarding
PermissionStartOnly to do this with sudo power
So, something like this:
[Service]
...
PermissionsStartOnly=true
ExecStartPre=/sbin/iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 4200
ExecStopPost=/sbin/iptables -t nat -D PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 4200
Don't forget to reload and restart your service:
[ec2-user#ip-172-31-39-212 system]$ sudo systemctl daemon-reload
[ec2-user#ip-172-31-39-212 system]$ sudo systemctl stop cloudyleela
[ec2-user#ip-172-31-39-212 system]$ sudo systemctl start cloudyleela
[ec2-user#ip-172-31-39-212 system]$
For microservices (update on Dec 2020)
The previously mentioned solution gives a lot of flexibility, but it does take some time to set it up. And for each additional application, you need to go through this entire process again. By the time you'll be installing your 5th node application, you'll certainly start wondering: "there has to be a shortcut".
The advantage of PM2 is that it's just 1 service to install. Next it's PM2 which manages the actual applications.
Even the initial setup of PM2 is easy, because it automatically installs the pm2 service for you.
npm install pm2 -g
And adding new services is even easier:
pm2 start index.js --name "foo"`.
When everything's up and running, you can save your setup, to have it automatically start on reboot.
pm2 save
If you want an overview of all your running node applications,
you can run pm2 list
And PM2 also offers an online (webbased) dashboard to monitor your application remotely. You may need a license to access some of the dashboard functionality though (which is a bit over-priced imho).
You can create a script that can start and stop your app and place it in /etc/init.d; make the script adhere to chkconfig's conventions (below), and then use chkconfig to set it to start when other services are started.
You can pick an existing script from /etc/init.d to use as an example; this article describes the requirements, which are basically:
An executable script that identifies the shell needed (i.e., #!/bin/bash)
A comment of the form # chkconfig: where is often 345, startprio indicates where in the order of services to start, and stopprio is where in the order of services to stop. I generally pick a similar service that already exists and use that as a guide for these values (i.e., if you have a web-related service, start at the same levels as httpd, with similar start and stop priorities).
Once your script is set up, you can use
chkconfig --add yourscript
chkconfig yourscript on
and you should be good to go. (Some distros may require you to manually symlink to the script to /etc/init.d/rc.d, but I believe your AWS distro will do that for you when you enable the script.
Use Elastic Beanstalk :) Provides support for auto-scaling, SSL termination, blue/green deployments, etc
If you want the salty sysadmin way for a RedHat based linux distro (Amazon Linux is a flavor of RedHat), learn systemd, as mentioned by #bvdb in the answer above:
https://en.wikipedia.org/wiki/Systemd
Set everything up as described on an EC2 instance, snapshot a custom AMI, and use this custom AMI as your base for EC2 instances hosting your apps. This way you don't have to go through all that setup multiple times. You'll probably want to get acquainted with load balancers too, if you are running in a production environment with uptime requirements.
Or, yes, as mentioned by #bvdb, you could also use pm2 to interface with systemd. Though I don't think pm2 helps with running your app across multiple EC2 instances, which is definitely recommended for production environments with uptime requirements.
All of which is a very steep learning curve. Since the OP seemed to be new to all this, Elastic Beanstalk, Google App Engine, and others are a great way to get code running in the cloud without all that.
These days I dev in TypeScript, deploying to serverless function execution in the cloud for most things, and don't have to think about package installs or app startup at all.
You can use screen. Run crontab -e and add this line:
#reboot screen -d -m bash -c "cd /home/user/yourapp/; node app"
Have been using forever on AWS and it does a good job. Install using
[sudo] npm install forever -g
To add an application use
forever start path_to_application
and to stop the application use
forever stop path_to_application
This is a useful article that helped me with setting it up.