Excuse me if the subject is vague, but I tried to describe my problem to the best of my possibilities. I have my raspberry pi which I want to deploy to using codeship. Rsyncing the files works perfectly, but when I am to restart my application using pm2 my problem occurs.
I have installed node and pm2 using the node version manager NVM.
ssh pi#server.com 'source /home/pi/.bashrc; cd project; pm2 restart app.js -x -- --prod'0 min 3 sec
bash: pm2: command not found
I have even added:
shopt -s expand_aliases in the bottom of my bashrc but it doesn't help.
How can I make it restart my application after I have done a deploy? Thanks in advance for your sage advice and better wisdom!
EDIT 1: My .bashrc http://pastie.org/10529200
My $PATH: /home/pi/.nvm/versions/node/v4.2.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
EDIT 2: I added /home/pi/.nvm/versions/node/v4.2.0/bin/pm2 which is the full path to pm2 and now I get the following error: /usr/bin/env: node: No such file or directory
It seems that even if I provide the full path, node isn't executed.
I think the problem is the misinterpretation that the shell executing node has a full environment like an interactive ssh session does. Most likely this is not the case.
When a SSH session spawns a shell it goes through a lot of gyrations to build an environment suitable to work with interactively. Things like inheriting from the login process, reading /etc/profile, reading ~/.profile. But in the cases where your executing bash directly this isn't always guaranteed. In fact the $PATH might be completely empty.
When /usr/bin/env node executes it looks for node in your $PATH which in a non-interactive shell could be anything or empty.
Most systems have a default PATH=/bin:/usr/bin typically /usr/local/bin is not included in the default environment.
You could attempt to force a login with ssh using ssh … '/bin/bash -l -c "…"'.
You can also write a specialized script on the server that knows how the environment should be when executed outside of an interactive shell:
#!/bin/bash
# Example shell script; filename: /usr/local/bin/my_script.sh
export PATH=$PATH:/usr/local/bin
export NODE_PATH=/usr/local/share/node
export USER=myuser
export HOME=/home/myuser
source $HOME/.nvm/nvm.sh
cd /usr/bin/share/my_script
nvm use 0.12
/usr/bin/env node ./script_name.js
Then call it through ssh: ssh … '/usr/local/bin/my_script.sh'.
Beyond these ideas I don't see how to help further.
Like Sukima said, the likelihood is that this is due to an environment issue - SSH'ing into a server does not set up a full environment. You can, however, get around much of this by simply calling /etc/profile yourself at the start of your command using the . operator (which is the same as the "source" command):
ssh pi#server.com '. /etc/profile ; cd project; pm2 restart app.js -x -- --prod'
/etc/profile should itself be set up to call the .bashrc of the relevant user, which is why I have removed that part. I used to have to do this quite a lot for quick proof-of-concept scripts at a previous workplace. I don't know if it would be considered a nasty hack for a more permanent script, but it certainly works, and would require minimal modification to your existing script should that be an issue.
For me I have to load :nvm as I installed node and yarn using :nvm
To load :nvm when ssh remote execution, we call
ssh :user#:host 'source ~/.nvm/nvm.sh; :other_commands_here'
Try:
ssh pi#server.com 'bash -l -c "source /home/pi/.bashrc; cd project; pm2 restart app.js -x -- --prod"'
You should enable some environment values by "source" or dot command ".". Here is an example.
ssh pi#server.com '. /home/pi/.nvm/nvm.sh; cd project; pm2 restart app.js -x -- --prod'
What worked for me was adding this to my .bash_profile:
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
Source: https://stackoverflow.com/a/820533/1824444
Related
I want to remotely run a node.js script containing a shebang line through ssh, similarly as when running it locally.
myscript file:
#!/usr/bin/env node
var param = process.argv[2] || 'help';
//... other js code
When running locally on each host – e.g. myscript arg1 – it runs successfully. When running remotely on a "sister" node in a cluster (containing the same file and directory structure, including nodeand myscript):
ssh -o "PasswordAuthentication no" bob#123.1.2.3 /path/to/myscript arg1
I get /usr/bin/env: ‘node’: No such file or directory error.
Am I missing a ssh param / option?
Mode details: If I run
ssh -o "PasswordAuthentication no" bob#123.1.2.3 echo "hello"
It also works fine. Forgive me it this is obvious to you, I'm not an advanced Linux user, the ssh manual seemed a little bit overwhelming and tried a couple answers found here with no success:
What exactly does "/usr/bin/env node" do at the beginning of node files?
Run scripts remotely via SSH
how to run a script file remotely using ssh
If the node executable isn't already in your PATH environment variable at login, you could provide the full path to it in the shebang line of your script:
#!/usr/bin/env /full/path/to/node
As others have commented, you would have to update your script if the path to node ever changes. This is not ideal. Alternatively, you could force ssh to create a pseudo-terminal session by specifying the -t flag and run your script in an interactive bash shell:
ssh -t -o "PasswordAuthentication no" bob#123.1.2.3 'bash -ic "/path/to/myscript arg1"'
Sebastian's answer inspired me to find a solution that doesn't hardcode the full path to node on the script. Basically, I make sure the remote PATH is available before running the command:
ssh -o "PasswordAuthentication no" bob#123.1.2.3 "export PATH=$PATH;/path/to/myscript arg1"
But this only worked for me because both local and remote servers have the same PATH value, since the local PATH is being set onto the remote session.
Here there may be some ways to explore other solutions if your case is not like mine:
How do I set $PATH such that `ssh user#host command` works?
How to set PATH when running a ssh command?
I want NodeJs server to start at boot and be sure it stays up if it crashes with forever.
I've read a lot of posts on how to do it and the possible issues associated with it.
The problem is that I have a limited memory available on given machine so I'm forced to "contain" NodeJs.
Reading the forever documentation I found out that it can be done using the command:
forever start -c "node --max_old_space_size=512" myapp.js
In this way forever calls the node server passing the memory argument which, in this case, limits the heap to 512 MB.
So I wrote my sh script for rc.local like this:
#!/bin/sh
cd /forever_bin_dir
./forever start -c "node --max_old_space_size=512" /myapp_dir/myapp.js
Since rc.local works with limited environmental variables, "node" can't be find and the script fails.
I even tried with Cron adding to its file the line:
#reboot /forever_bin_dir/forever start -c "node --max_old_space_size=512" /myapp_dir/myapp.js
But the same issue persist.
My question is: how can i run forever at boot time passing at the same time the argument about memory ?
I found this init script used for same purpose starting Node script forever.
The best way I found It's just setting the PATH inside a script.
I created a sh script in the home folder called launchforever.sh
I got the environmental variables using the #printenv command.
The launchforever script is something like this:
#!/bin/sh
export PATH=<copied from printenv>:$PATH
forever start -c "node --max_old_space_size=30" > /dev/null
After saving I gave it the permissions:
#chmod 700 /home/<user>/launchforever.sh
Then I added it to Cron:
#crontab -u <user> -e
And inside its file I added:
#reboot /home/<user>/launchforever.sh
I made this shell script
/var/start-app.sh
#!/bin/sh
# file-name: app-start.sh
cd /var/www/html/app
bundle exec rake sunspot:solr:start RAILS_ENV=production
unicorn_rails -c config/unicorn.rb -E production -D
If I type /var/start-app.sh in SSH. It triggers, and everything works fine.
Then I did put sh /var/start-app.sh in the end line of /etc/rc.d/rc.local in order to make it run on Server's start up.
But this never work. Why? and How can I enable that?
From SSH, I called it with root permission.
Is that why? If so, how can I give root permission to rc.local?
Looks like starting services from rc.local is not the best practice http://bencane.com/2011/12/30/when-its-ok-and-not-ok-to-use-rc-local/. You would be better writing an init script :
http://refspecs.linuxbase.org/LSB_3.1.1/LSB-Core-generic/LSB-Core-generic/sysinit.html
I am using node's forever module to keep my node server running. Forever however terminates when there is a system restart. Is there any way I can automatically start the node server (with forever) when the system restarts?
I would suggest using crontab. It's easy to use.
How to
To start editing run the following replacing the "testuser" with your desired runtime user for the node process. If you choose a different user other than yourself, you will have to run this with sudo.
$ crontab -u testuser -e
If you have never done this before, it will ask you which editor you wish to edit with. I like vim, but will recommend nano for ease of use.
Once in the editor add the following line:
#reboot /usr/local/bin/forever start /your/path/to/your/app.js
Save the file. You should get some feedback that the cron has been installed.
For further confirmation of the installation of the cron, execute the following (again replacing "testuser" with your target username) to list the currently installed crons:
$ crontab -u testuser -l
Note that in my opinion, you should always use full paths when executing binaries in cron.
Also, if the path to your forever script is not correct, run which forever to get the full path.
Given that forever calls node, you may also want to provide the full path to node:
#reboot /usr/local/bin/forever start -c /usr/local/bin/node /your/path/to/your/app.js
Further Reading
crontab Man Page
Ubuntu Cron HowTo
You can use forever-service for doing this.
npm install -g forever-service
forever-service install test
This will provision app.js in the current directory as a service via forever. The service will automatically restart every time system is restarted. Also when stopped it will attempt a graceful stop. This script provisions the logrotate script as well.
Github url: https://github.com/zapty/forever-service
NOTE: I am the author of forever-service.
Install PM2 globally using NPM
npm install pm2 -g
Start your script with pm2
pm2 start app.js
generate an active startup script
pm2 startup
NOTE: pm2 startup is for startting the PM2 when the system reboots. PM2 once started, restarts all the processes it had been managing before the system went down.
In case you want to disable the automatic startup, simply use pm2 unstartup
If you want the startup script to be executed under another user, just use the -u <username> option and the --hp <user_home>:
This case is valid for Debian.
Add the following to /etc/rc.local
/usr/bin/sudo -u {{user}} /usr/local/bin/forever start {{app path}}
{{user}} replaces your username.
{{app path}} replaces your app path. For example, /var/www/test/app.js
An alternative crontab method inspired by this answer and this blog post.
1. Create a bash script file (change bob to desired user).
vi /home/bob/node_server_init.sh
2. Copy and paste this inside the file you've just created.
#!/bin/sh
export NODE_ENV=production
export PATH=/usr/local/bin:$PATH
forever start /node/server/path/server.js > /dev/null
Make sure to edit the paths above according to your config!
3. Make sure the bash script can be executed.
chmod 700 /home/bob/node_server_init.sh
4. Test the bash script.
sh /home/bob/node_server_init.sh
5. Replace "bob" with the runtime user for node.
crontab -u bob -e
6. Copy and paste (change bob to desired user).
#reboot /bin/sh /home/bob/node_server_init.sh
Save the crontab.
You've made it to the end, your prize is a reboot (to test) :)
Copied answer from the attached question.
You can use PM2, it's a production process manager for Node.js applications with a built-in load balancer.
Install PM2
$ npm install pm2 -g
Start an application
$ pm2 start app.js
If you using express then you can start your app like
pm2 start ./bin/www --name="app"
Listing all running processes:
$ pm2 list
It will list all process. You can then stop / restart your service by using ID or Name of the app with following command.
$ pm2 stop all
$ pm2 stop 0
$ pm2 restart all
To display logs
$ pm2 logs ['all'|app_name|app_id]
You need to create a shell script in the /etc/init.d folder for that. It's sort of complicated if you never have done it but there is plenty of information on the web on init.d scripts.
Here is a sample a script that I created to run a CoffeeScript site with forever:
#!/bin/bash
#
# initd-example Node init.d
#
# chkconfig: 345
# description: Script to start a coffee script application through forever
# processname: forever/coffeescript/node
# pidfile: /var/run/forever-initd-hectorcorrea.pid
# logfile: /var/run/forever-initd-hectorcorrea.log
#
# Based on a script posted by https://gist.github.com/jinze at https://gist.github.com/3748766
#
# Source function library.
. /lib/lsb/init-functions
pidFile=/var/run/forever-initd-hectorcorrea.pid
logFile=/var/run/forever-initd-hectorcorrea.log
sourceDir=/home/hectorlinux/website
coffeeFile=app.coffee
scriptId=$sourceDir/$coffeeFile
start() {
echo "Starting $scriptId"
# This is found in the library referenced at the top of the script
start_daemon
# Start our CoffeeScript app through forever
# Notice that we change the PATH because on reboot
# the PATH does not include the path to node.
# Launching forever or coffee with a full path
# does not work unless we set the PATH.
cd $sourceDir
PATH=/usr/local/bin:$PATH
NODE_ENV=production PORT=80 forever start --pidFile $pidFile -l $logFile -a -d --sourceDir $sourceDir/ -c coffee $coffeeFile
RETVAL=$?
}
restart() {
echo -n "Restarting $scriptId"
/usr/local/bin/forever restart $scriptId
RETVAL=$?
}
stop() {
echo -n "Shutting down $scriptId"
/usr/local/bin/forever stop $scriptId
RETVAL=$?
}
status() {
echo -n "Status $scriptId"
/usr/local/bin/forever list
RETVAL=$?
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart)
restart
;;
*)
echo "Usage: {start|stop|status|restart}"
exit 1
;;
esac
exit $RETVAL
I had to make sure the folder and PATHs were explicitly set or available to the root user since init.d scripts are ran as root.
Use the PM2
Which is the best option to run the server production server
What are the advantages of running your application this way?
PM2 will automatically restart your application if it crashes.
PM2 will keep a log of your unhandled exceptions - in this case, in a file at /home/safeuser/.pm2/logs/app-err.log.
With one command, PM2 can ensure that any applications it manages restart when the server reboots. Basically, your node application will start as a service.
ref: https://www.digitalocean.com/community/tutorials/how-to-use-pm2-to-setup-a-node-js-production-environment-on-an-ubuntu-vps
Forever was not made to get node applications running as services. The right approach is to either create an /etc/inittab entry (old linux systems) or an upstart (newer linux systems).
Here's some documentation on how to set this up as an upstart:
https://github.com/cvee/node-upstart
crontab does not work for me on CentOS x86 6.5. #reboot seems to be not working.
Finally I got this solution:
Edit: /etc/rc.local
sudo vi /etc/rc.local
Add this line to the end of the file. Change USER_NAME and PATH_TO_PROJECT to your own. NODE_ENV=production means the app runs in production mode. You can add more lines if you need to run more than one node.js app.
su - USER_NAME -c "NODE_ENV=production /usr/local/bin/forever start /PATH_TO_PROJECT/app.js"
Don't set NODE_ENV in a separate line, your app will still run in development mode, because forever does not get NODE_ENV.
# WRONG!
su - USER_NAME -c "export NODE_ENV=production"
Save and quit vi (press ESC : w q return). You can try rebooting your server. After your server reboots, your node.js app should run automatically, even if you don't log into any account remotely via ssh.
You'd better set NODE_ENV environment in your shell. NODE_ENV will be set automatically when your account USER_NAME logs in.
echo export NODE_ENV=production >> ~/.bash_profile
So you can run commands like forever stop/start /PATH_TO_PROJECT/app.js via ssh without setting NODE_ENV again.
I wrote a script that does exactly this:
https://github.com/chovy/node-startup
I have not tried with forever, but you can customize the command it runs, so it should be straight forward:
/etc/init.d/node-app start
/etc/init.d/node-app restart
/etc/init.d/node-app stop
The problem with rc.local is that the commands are accessed as root which is different than logging to as a user and using sudo.
I solved this problem by adding a .sh script with the startup commands i want to etc/profile.d. Any .sh file in profile.d will load automatically and any command will be treated as if you used the regular sudo.
The only downside to this is the specified user needs to loggin for things to start which in my situation was always the case.
I tried lots of the above answers. None of them worked for me. My app is installed in /home and as user, not as root. This probably means that when the above mentioned start scripts run, /home is not mounted yet, so the app is not started.
Then I found these instructions by Digital Ocean:
https://www.digitalocean.com/community/tutorials/how-to-use-pm2-to-setup-a-node-js-production-environment-on-an-ubuntu-vps
Using PM2 as explained was very simple and works perfectly: My virtual servers had two physical crashes since - downtime was only about a minute.
complete example crontab (located at /etc/crontab) ..
#!/bin/bash
# edit this file with .. crontab -u root -e
# view this file with .. crontab -u root -l
# put your path here if it differs
PATH=/root/bin:/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
# * * * * * echo "executes once every minute" > /root/deleteme
#reboot cd /root/bible-api-dbt-server; npm run forever;
#reboot cd /root/database-api-server; npm run forever;
#reboot cd /root/mailer-api-server; npm run forever;
I have found my own solution by using serve & npm as follows:
Install serve package: npm install -g serve
Then have the command serve -s /var/www/sitename to execute on reboot.
This is what works for me on my VPS.
You can use the following command in your shell to start your node forever:
forever app.js //my node script
You need to keep in mind that the server on which your app is running should always be kept on.
I have a node.js script which need to start at boot and run under the www-data user. During development I always started the script with:
su www-data -c 'node /var/www/php-jobs/manager.js
I saw exactly what happened, the manager.js works now great. Searching SO I found I had to place this in my /etc/rc.local. Also, I learned to point the output to a log file and to append the 2>&1 to "redirect stderr to stdout" and it should be a daemon so the last character is a &.
Finally, my /etc/rc.local looks like this:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
su www-data -c 'node /var/www/php-jobs/manager.js >> /var/log/php-jobs.log 2>&1 &'
exit 0
If I run this myself (sudo /etc/rc.local): yes, it works! However, if I perform a reboot no node process is running, the /var/log/php-jobs.log does not exist and thus, the manager.js does not work. What is happening?
In this example of a rc.local script I use io redirection at the very first line of execution to my own log file:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.
exec 1>/tmp/rc.local.log 2>&1 # send stdout and stderr from rc.local to a log file
set -x # tell sh to display commands before execution
/opt/stuff/somefancy.error.script.sh
exit 0
On some linux's (Centos & RH, e.g.), /etc/rc.local is initially just a symbolic link to /etc/rc.d/rc.local. On those systems, if the symbolic link is broken, and /etc/rc.local is a separate file, then changes to /etc/rc.local won't get seen at bootup -- the boot process will run the version in /etc/rc.d. (They'll work if one runs /etc/rc.local manually, but won't be run at bootup.)
Sounds like on dimadima's system, they are separate files, but /etc/rc.d/rc.local calls /etc/rc.local
The symbolic link from /etc/rc.local to the 'real' one in /etc/rc.d can get lost if one moves rc.local to a backup directory and copies it back or creates it from scratch, not realizing the original one in /etc was just a symbolic link.
I ended up with upstart, which works fine.
In Ubuntu I noticed there are 2 files. The real one is /etc/init.d/rc.local; it seems the other /etc/rc.local is bogus?
Once I modified the correct one (/etc/init.d/rc.local) it did execute just as expected.
You might also have made it work by specifying the full path to node. Furthermore, when you want to run a shell command as a daemon you should close stdin by adding 1<&- before the &.
I had the same problem (on CentOS 7) and I fixed it by giving execute permissions to /etc/local:
chmod +x /etc/rc.local
if you are using linux on cloud, then usually you don't have chance to touch the real hardware using your hands. so you don't see the configuration interface when booting for the first time, and of course cannot configure it. As a result, the firstboot service will always be in the way to rc.local. The solution is to disable firstboot by doing:
sudo chkconfig firstboot off
if you are not sure why your rc.local does not run, you can always check from /etc/rc.d/rc file because this file will always run and call other subsystems (e.g. rc.local).
I got my script to work by editing /etc/rc.local then issuing the following 3 commands.
sudo mv /filename /etc/init.d/
sudo chmod +x /etc/init.d/filename
sudo update-rc.d filename defaults
Now the script works at boot.
I am using CentOS 7.
$ cd /etc/profile.d
$ vim yourstuffs.sh
Type the following into the yourstuffs.sh script.
type whatever you want here to execute
export LD_LIBRARY_PATH=/usr/local/cuda-7.0/lib64:$LD_LIBRARY_PATH
Save and reboot the OS.
I have used rc.local in the past. But I have learned from my experience that the most reliable way to run your script at the system boot time is is to use #reboot command in crontab. For example:
#reboot path_to_the_start_up_script.sh
This is most probably caused by a missing or incomplete PATH environment variable.
If you provide full absolute paths to your executables (su and node) it will work.
It is my understanding that if you place your script in a certain RUN Level, you should use ln -s to link the script to the level you want it to work in.
first make the script executable using
sudo chmod 755 /path/of/the/file.sh
now add the script in the rc.local
sh /path/of/the/file.sh
before exit 0
in the rc.local,
next make the rc.local to executable with
sudo chmod 755 /etc/rc.local
next to initialize the rc.local use
sudo /etc/init.d/rc.local start
this will initiate the rc.local
now reboot the system.
Done..
I found that because I was using a network-oriented command in my rc.local, sometimes it would fail. I fixed this by putting sleep 3 at the top of my script. I don't know why but it seems when the script is run the network interfaces aren't properly configured or something, and this just allows some time for the DHCP server or something. I don't fully understand but I suppose you could give it a try.
I had exactly same issue, the script was running fine locally but when I reboot/power-on it was not.
I resolved the issue by changing the file path. Basically need to give the complete path in the script. While running locally, file can be accessed but when running on reboot, local path will not be understood.
1 Do not recommend using root to run the apps such as node app.
Well you can do it but may catch more exceptions.
2 The rc.local normally runs as root user.
So if the your script should runs as another user such as www U should make sure the PATH and other environment is ok.
3 I find a easy way to run a service as a user:
sudo -u www -i /the/path/of/your/script
Please prefer the sudo manual~
-i [command]
The -i (simulate initial login) option runs the shell specified by the password database entry of the target user as a loginshell...
rc.local only runs on startup. If you reboot and want the script to execute, it needs to go into the rc.0 file starting with the K99 prefix.