How to run .sh file on Ec2 Instance Start - linux

I am trying to run a start.sh file when I start my AWS EC2 instance. Currently I have to start the instance connect to it create a screen and start the start.sh file within it and disconnect.
I have tried putting the below code within the User data, which should run on start-up however when I connect to see if its working I can't see evidence of any screens active or the start.sh file running. I have looked in the /var/log/cloud-init.log but cannot see evidence of it trying to run (though I do not know what to look for)
#!/bin/bash
sudo su
cd
screen
sudo ./start.sh
The start.sh file is located in the root users default directory

The easiest way is using a cron job for this. Add your script to the cron file with:
crontab -e
add:
#reboot sh ./start.sh
with the #reboot your job will be executed after startup.

Related

Is there a way to make crontab run a gnu screen session?

I have a discord bot running on a raspberry pi that i need to restart every day, I'm trying to do this through crontab but with no luck.
It manages to kill the active screen processes but never starts an instance, not that I can see with "screen -ls" and I can tell that it doesn't create one that I can't see as the bot itself does not come online.
Here is the script:
#!/bin/bash
sudo pkill screen
sleep 2
screen -dmS discordBot
sleep 2
screen -S "discordBot" -X stuff "node discordBot/NEWNEWNEWN\n"
Here is also the crontab:
0 0 * * * /bin/bash /home/admin/discordBot/script.sh
Is it possible to have crontab run a screen session? And if so how?
Previously I tried putting the screen command stright into cron but now I have it in a bash script instead.
If I run the script in the terminal it works perfectly, it’s just cron where it fails. Also replacing "screen" with the full path "/usr/bin/screen" does not change anything.
So the best way of doing it, without investigating the underlying problem would be to create a systemd service and putting a restart command into cron.
 
/etc/systemd/system/mydiscordbot.service:
[Unit]
Description=A very simple discord bot
[Service]
Type=simple
ExecStart=node /path/to/my/discordBot.js
[Install]
WantedBy=multi-user.target
Now you can run your bot with systemctl start mydiscordbot and can view your logs with journalctl --follow -u mydiscord bot
Now you only need to put
45 2 * * * systemctl restart discordbot
into root's crontab and you should be good to go.
You probably should also write the logs into a logfile, maybe /var/log/mydiscordbot.log, but how and if you do that is up to you.
OLD/WRONG ANSWER:
cron is run with a minimal environment, and depending on your os, /usr/bin/ is probably not in the $PATH var. screen is mostlikely located at /usr/bin/screen, so cron can't run it because it can't find the screen binary. try replacing screen in your script with /usr/bin/screen
But the other question here is: Why does your discord bot need to be restarted every day. I run multiple bots with 100k+ users, and they don't need to be restarted at all. Maybe you should open a new question with the error and some code snippets.
I don't know what os your rpi is running, but I had a similar issue earlier today trying to get vms on a server to launch a terminal and run a python script with crontab. I found a very easily solution to automatically restart it on crashes, and to run something simply in the background. Don't rely on crontab or an init service. If your rpi as an x server running, or anything that can be launched on session start, there is a simple shell solution. Make a bash script like
while :; do
/my/script/to/keep/running/here -foo -bar -baz >> /var/log/myscriptlog 2>&1
done
and then you would start it on .xprofile or some similar startup operation like
/path/to/launcher.sh &
to have it run the background. It will restart automatically everytime it closes if started in something like .xprofile, .xinitrc, or anything ran at startup.
Then maybe make a cronjob to restart the raspberry pi or whatever system is running the script but this script wil restart the service whenever it's closed anyway. Maybe you can put that cronjob on a script but I don't think it would launch the GUI.
Some other things you can do to launch a GUI terminal in my research with cronjob that you can try, though they didn't work for my situation on these custom linux vms, and that I read was a security risk to do this from a cronjob, is you can specify the display.
* * * * * DISPLAY=:0 /your/gui/stuff/here
but you would would need to make sure the user has the right permissions and it's considered an unsafe hack to even do this as far as I have read.
for my issue, I had to launch a terminal that stayed open, and then changed to the directory of a python script and ran the script, so that the local files in directory would be called in the python script. here is a rough example of the "launcher.sh" I called from the startup method this strange linux distro used lol.
#!/bin/sh
while :; do
/usr/bin/urxvt -e /bin/bash -c "cd /path/to/project && python loader.py"
done
Also check this source for process management
https://mywiki.wooledge.org/ProcessManagement

run sh script to stop and restart nginx as root from crontab?

I need a script to run a bash script that stops nginx dows a bunch of other stuff and then starts nginx
the crontab looks like this
5 10 * * 0 sh ~/runnow/script.sh
and I edited the crontab well logged in as root.
the script runs fine however the lines for stop and start of nginx give the error service not found.
echo "Stopping Nginx"
service nginx-sp stop
gives
Stopping Nginx
/root/run/script.sh: 2: /root/run/script.sh: service: not found
if I manually run the script logged on as root it runs fine.
my thought is that crontab is not being run as root but searching for that it seems if you exited your crontab when logged in as root with crontab -e it should be run as root. and it must be since I am using ~/run/script.sh to run the script and the real path is /root/run/script.sh
so... im lost....
As cyrus pointed ...
use which to find service path in this way:
which service
Then add full path in your script.

what is the best way to start a script in boot time on linux

I want to start a script I have on when the system start and looking for the best way, my way is:
vi /etc/systemd/system/myscript.service
[Service]
Type=simple
ExecStart=/usr/bin/myscript
CPUSchedulingPolicy=rr
CPUSchedulingPrioty=27
[Install]
WantedBy=multi-user.target graphical.target
systemctl daemon-reload; systemctl enable myscript; systemctl start rmyscript
it's working good but just wondered if there another and better way.
There are a couple of ways to achieve this, but you will need root privileges for any the following. To get root, open a terminal and run the command:
sudo su
and the command prompt will change to '#' indicating that the terminal session has root privileges.
Alternative #1. Add an initscript
Create a new script in /etc/init.d/myscript:
vi /etc/init.d/myscript
(Obviously it doesn't have to be called "myscript".) In this script, do whatever you want to do. Perhaps just run the script you mentioned:
#!/bin/sh
/path/to/my/script.sh
Make it executable:
chmod ugo+x /etc/init.d/myscript
Configure the init system to run this script at startup:
update-rc.d myscript defaults
Alternative #2. Add commands to /etc/rc.local
vi /etc/rc.local
with content like the following:
# This script is executed at the end of each multiuser runlevel
/path/to/my/script.sh || exit 1 # Added by me
exit 0
Alternative #3. Add an Upstart job
Create /etc/init/myjob.conf:
vi /etc/init/myjob.conf
with the following content:
description "my job"
start on startup
task
exec /path/to/my/script.sh
BTW:
You don't need to be root if you can edit your crontab (crontab -e) and create an entry like this:
#reboot /path/to/script.sh
This way, you can run it as a regular user. #reboot just means it's run when the computer starts up (not necessarily just when it's rebooted).

crontab #reboot does not execute bash script when server is rebooted

I'm using #reboot ~/www/example.com/bin/server in my user's crontab...but when I reboot the server, the web server (this script) does not come up. (script works fine from command line).
My guess is the /home/user directory has not been mounted yet...does anyone know if its possible to get a script to run out of a home directory using this crontab #reboot method?
If you think /home/user hasn't been mounted (or some required systems aren't running) yet, in your crontab line, you can always wait before executing a command like:
#reboot sleep 60; /home/user/www/example.com/bin/server
It should definitely be due to the environment scenarios as given in comments. Try the following and check once by doing a reboot
#reboot (date > /tmp/date-check.txt)
To be sure cron is able to run the jobs.
My problem was that the crontab did not have a full environment. I made the script it was pointing to source my .bashrc.
#reboot /home/user/www/example.com/bin/server
./server does . /home/user/.bashrc to get a working environment.
In Ubuntu if you are using the Home Directory Encryption feature turned on then #reboot in your crontab file won't work as the file system is still encrypted when the system is starting up and cron runs its #reboot jobs.
Your options are to place your files in an unencrypted location (/usr/local/bin or something?) or disable Home Directory encryption on your home directory.

Automatically start forever (node) on system restart

I am using node's forever module to keep my node server running. Forever however terminates when there is a system restart. Is there any way I can automatically start the node server (with forever) when the system restarts?
I would suggest using crontab. It's easy to use.
How to
To start editing run the following replacing the "testuser" with your desired runtime user for the node process. If you choose a different user other than yourself, you will have to run this with sudo.
$ crontab -u testuser -e
If you have never done this before, it will ask you which editor you wish to edit with. I like vim, but will recommend nano for ease of use.
Once in the editor add the following line:
#reboot /usr/local/bin/forever start /your/path/to/your/app.js
Save the file. You should get some feedback that the cron has been installed.
For further confirmation of the installation of the cron, execute the following (again replacing "testuser" with your target username) to list the currently installed crons:
$ crontab -u testuser -l
Note that in my opinion, you should always use full paths when executing binaries in cron.
Also, if the path to your forever script is not correct, run which forever to get the full path.
Given that forever calls node, you may also want to provide the full path to node:
#reboot /usr/local/bin/forever start -c /usr/local/bin/node /your/path/to/your/app.js
Further Reading
crontab Man Page
Ubuntu Cron HowTo
You can use forever-service for doing this.
npm install -g forever-service
forever-service install test
This will provision app.js in the current directory as a service via forever. The service will automatically restart every time system is restarted. Also when stopped it will attempt a graceful stop. This script provisions the logrotate script as well.
Github url: https://github.com/zapty/forever-service
NOTE: I am the author of forever-service.
Install PM2 globally using NPM
npm install pm2 -g
Start your script with pm2
pm2 start app.js
generate an active startup script
pm2 startup
NOTE: pm2 startup is for startting the PM2 when the system reboots. PM2 once started, restarts all the processes it had been managing before the system went down.
In case you want to disable the automatic startup, simply use pm2 unstartup
If you want the startup script to be executed under another user, just use the -u <username> option and the --hp <user_home>:
This case is valid for Debian.
Add the following to /etc/rc.local
/usr/bin/sudo -u {{user}} /usr/local/bin/forever start {{app path}}
{{user}} replaces your username.
{{app path}} replaces your app path. For example, /var/www/test/app.js
An alternative crontab method inspired by this answer and this blog post.
1. Create a bash script file (change bob to desired user).
vi /home/bob/node_server_init.sh
2. Copy and paste this inside the file you've just created.
#!/bin/sh
export NODE_ENV=production
export PATH=/usr/local/bin:$PATH
forever start /node/server/path/server.js > /dev/null
Make sure to edit the paths above according to your config!
3. Make sure the bash script can be executed.
chmod 700 /home/bob/node_server_init.sh
4. Test the bash script.
sh /home/bob/node_server_init.sh
5. Replace "bob" with the runtime user for node.
crontab -u bob -e
6. Copy and paste (change bob to desired user).
#reboot /bin/sh /home/bob/node_server_init.sh
Save the crontab.
You've made it to the end, your prize is a reboot (to test) :)
Copied answer from the attached question.
You can use PM2, it's a production process manager for Node.js applications with a built-in load balancer.
Install PM2
$ npm install pm2 -g
Start an application
$ pm2 start app.js
If you using express then you can start your app like
pm2 start ./bin/www --name="app"
Listing all running processes:
$ pm2 list
It will list all process. You can then stop / restart your service by using ID or Name of the app with following command.
$ pm2 stop all
$ pm2 stop 0
$ pm2 restart all
To display logs
$ pm2 logs ['all'|app_name|app_id]
You need to create a shell script in the /etc/init.d folder for that. It's sort of complicated if you never have done it but there is plenty of information on the web on init.d scripts.
Here is a sample a script that I created to run a CoffeeScript site with forever:
#!/bin/bash
#
# initd-example Node init.d
#
# chkconfig: 345
# description: Script to start a coffee script application through forever
# processname: forever/coffeescript/node
# pidfile: /var/run/forever-initd-hectorcorrea.pid
# logfile: /var/run/forever-initd-hectorcorrea.log
#
# Based on a script posted by https://gist.github.com/jinze at https://gist.github.com/3748766
#
# Source function library.
. /lib/lsb/init-functions
pidFile=/var/run/forever-initd-hectorcorrea.pid
logFile=/var/run/forever-initd-hectorcorrea.log
sourceDir=/home/hectorlinux/website
coffeeFile=app.coffee
scriptId=$sourceDir/$coffeeFile
start() {
echo "Starting $scriptId"
# This is found in the library referenced at the top of the script
start_daemon
# Start our CoffeeScript app through forever
# Notice that we change the PATH because on reboot
# the PATH does not include the path to node.
# Launching forever or coffee with a full path
# does not work unless we set the PATH.
cd $sourceDir
PATH=/usr/local/bin:$PATH
NODE_ENV=production PORT=80 forever start --pidFile $pidFile -l $logFile -a -d --sourceDir $sourceDir/ -c coffee $coffeeFile
RETVAL=$?
}
restart() {
echo -n "Restarting $scriptId"
/usr/local/bin/forever restart $scriptId
RETVAL=$?
}
stop() {
echo -n "Shutting down $scriptId"
/usr/local/bin/forever stop $scriptId
RETVAL=$?
}
status() {
echo -n "Status $scriptId"
/usr/local/bin/forever list
RETVAL=$?
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status
;;
restart)
restart
;;
*)
echo "Usage: {start|stop|status|restart}"
exit 1
;;
esac
exit $RETVAL
I had to make sure the folder and PATHs were explicitly set or available to the root user since init.d scripts are ran as root.
Use the PM2
Which is the best option to run the server production server
What are the advantages of running your application this way?
PM2 will automatically restart your application if it crashes.
PM2 will keep a log of your unhandled exceptions - in this case, in a file at /home/safeuser/.pm2/logs/app-err.log.
With one command, PM2 can ensure that any applications it manages restart when the server reboots. Basically, your node application will start as a service.
ref: https://www.digitalocean.com/community/tutorials/how-to-use-pm2-to-setup-a-node-js-production-environment-on-an-ubuntu-vps
Forever was not made to get node applications running as services. The right approach is to either create an /etc/inittab entry (old linux systems) or an upstart (newer linux systems).
Here's some documentation on how to set this up as an upstart:
https://github.com/cvee/node-upstart
crontab does not work for me on CentOS x86 6.5. #reboot seems to be not working.
Finally I got this solution:
Edit: /etc/rc.local
sudo vi /etc/rc.local
Add this line to the end of the file. Change USER_NAME and PATH_TO_PROJECT to your own. NODE_ENV=production means the app runs in production mode. You can add more lines if you need to run more than one node.js app.
su - USER_NAME -c "NODE_ENV=production /usr/local/bin/forever start /PATH_TO_PROJECT/app.js"
Don't set NODE_ENV in a separate line, your app will still run in development mode, because forever does not get NODE_ENV.
# WRONG!
su - USER_NAME -c "export NODE_ENV=production"
Save and quit vi (press ESC : w q return). You can try rebooting your server. After your server reboots, your node.js app should run automatically, even if you don't log into any account remotely via ssh.
You'd better set NODE_ENV environment in your shell. NODE_ENV will be set automatically when your account USER_NAME logs in.
echo export NODE_ENV=production >> ~/.bash_profile
So you can run commands like forever stop/start /PATH_TO_PROJECT/app.js via ssh without setting NODE_ENV again.
I wrote a script that does exactly this:
https://github.com/chovy/node-startup
I have not tried with forever, but you can customize the command it runs, so it should be straight forward:
/etc/init.d/node-app start
/etc/init.d/node-app restart
/etc/init.d/node-app stop
The problem with rc.local is that the commands are accessed as root which is different than logging to as a user and using sudo.
I solved this problem by adding a .sh script with the startup commands i want to etc/profile.d. Any .sh file in profile.d will load automatically and any command will be treated as if you used the regular sudo.
The only downside to this is the specified user needs to loggin for things to start which in my situation was always the case.
I tried lots of the above answers. None of them worked for me. My app is installed in /home and as user, not as root. This probably means that when the above mentioned start scripts run, /home is not mounted yet, so the app is not started.
Then I found these instructions by Digital Ocean:
https://www.digitalocean.com/community/tutorials/how-to-use-pm2-to-setup-a-node-js-production-environment-on-an-ubuntu-vps
Using PM2 as explained was very simple and works perfectly: My virtual servers had two physical crashes since - downtime was only about a minute.
complete example crontab (located at /etc/crontab) ..
#!/bin/bash
# edit this file with .. crontab -u root -e
# view this file with .. crontab -u root -l
# put your path here if it differs
PATH=/root/bin:/root/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
# * * * * * echo "executes once every minute" > /root/deleteme
#reboot cd /root/bible-api-dbt-server; npm run forever;
#reboot cd /root/database-api-server; npm run forever;
#reboot cd /root/mailer-api-server; npm run forever;
I have found my own solution by using serve & npm as follows:
Install serve package: npm install -g serve
Then have the command serve -s /var/www/sitename to execute on reboot.
This is what works for me on my VPS.
You can use the following command in your shell to start your node forever:
forever app.js //my node script
You need to keep in mind that the server on which your app is running should always be kept on.

Resources