ubuntu: start (upstart) second instance of mongodb - linux

the standard upstart script that comes with mongodb works fine:
# Ubuntu upstart file at /etc/init/mongodb.conf
limit nofile 20000 20000
kill timeout 300 # wait 300s between SIGTERM and SIGKILL.
pre-start script
mkdir -p /var/lib/mongodb/
mkdir -p /var/log/mongodb/
end script
start on runlevel [2345]
stop on runlevel [06]
script
ENABLE_MONGODB="yes"
if [ -f /etc/default/mongodb ]; then . /etc/default/mongodb; fi
if [ "x$ENABLE_MONGODB" = "xyes" ]; then exec start-stop-daemon --start --quiet --chuid mongodb --exec /usr/bin/mongod -- --config /etc/mongodb.conf; fi
end script
if i want to run a second instance of mongod i thought i just copy both /etc/mongodb.conf -> /etc/mongodb2.conf and /etc/init/mongodb.conf -> /etc/init/mongodb2.conf and change the std port in the first conf-file. then adjust the script above to start with the newly created /etc/mongodb2.conf.
i can then just say start mongodb2and the service starts ... but it is killed right after starting. what do i change, to get both processes up and running?
# Ubuntu upstart file at /etc/init/mongodb2.conf
limit nofile 20000 20000
kill timeout 300 # wait 300s between SIGTERM and SIGKILL.
pre-start script
mkdir -p /var/lib/mongodb2/
mkdir -p /var/log/mongodb2/
end script
start on runlevel [2345]
stop on runlevel [06]
script
ENABLE_MONGODB="yes"
if [ -f /etc/default/mongodb ]; then . /etc/default/mongodb; fi
if [ "x$ENABLE_MONGODB" = "xyes" ]; then exec start-stop-daemon --start --quiet --chuid mongodb --exec /usr/bin/mongod -- --config /etc/mongodb2.conf; fi
end script

i couldn't get the "standard" upstart script to work (as described above), so i changed it like this:
# Ubuntu upstart file at /etc/init/mongodb.conf
limit nofile 20000 20000
kill timeout 300 # wait 300s between SIGTERM and SIGKILL.
pre-start script
mkdir -p /var/lib/mongodb/
mkdir -p /var/log/mongodb/
end script
start on runlevel [2345]
stop on runlevel [06]
script
exec sudo -u mongodb /usr/bin/mongod --config /etc/mongodb.conf
end script
and if you want to run other instances of mongodb just copy the *.conf files and make the changes to /etc/mongodb2.conf and /etc/init/mongodb2.conf
# Ubuntu upstart file at /etc/init/mongodb2.conf
limit nofile 20000 20000
kill timeout 300 # wait 300s between SIGTERM and SIGKILL.
pre-start script
mkdir -p /var/lib/mongodb2/
mkdir -p /var/log/mongodb2/
end script
start on runlevel [2345]
stop on runlevel [06]
script
exec sudo -u mongodb /usr/bin/mongod --config /etc/mongodb2.conf
end script
i think the only thing that is not working is restart mongodb - you have to stop and then start again ...

I know there's already an accepted solution but I think this one is more elegant.
The other way is to use start-stop-daemon's pid file creation. For example, I have 2 mongos running on the same server with 2 different upstart scripts, and the two magic lines are:
exec start-stop-daemon --make-pidfile --pidfile /var/run/mongodb-router.pid --start --startas /data/bin/mongos --chuid mongo -- --logappend --logpath /mnt/log/mongos.log --configdb mongo2-config01,mongo2-config02,mongo2-config03
exec start-stop-daemon --make-pidfile --pidfile /var/run/mongodb-routerrt.pid --start --startas /data/bin/mongos --chuid mongo -- --logappend --logpath /mnt/log/mongos-rt.log --configdb mongort-config01,mongort-config02,mongort-config03 --port 27027
Note that one has '--pidfile /var/run/mongodb-router.pid' and the other has '--pidfile /var/run/mongodb-routerrt.pid' and a different port.

Yeah I ran into this same issue today. The reason is that the default script uses the start-stop-daemon to start mongo, which is specifically designed to ensure that only one version of a process is running. You already figured out that one way to fix this is to not use start-stop-daemon and to start the binary yourself. That's the way I do it too but I'd be curious to hear if there's a better way.

This is how I do it.
2 instances of mongodb, with start-stop-daemon, on the same server
that are my start-stop-daemon configs
exec start-stop-daemon --make-pidfile --pidfile /var/lib/mongodb/db1.pid --start --quiet --chuid mongodb --name mongod1 --exec /usr/bin/mongod -- --config etc/mongodb1.conf
exec start-stop-daemon --make-pidfile --pidfile /var/lib/mongodb/db2.pid --start --quiet --chuid mongodb --name mongod2 --exec /usr/bin/mongod -- --config etc/mongodb2.conf
pay attention to the --name option. That did the trick for me

the two daemons cannot listen on the same tcp port, thus you have to change the --port parameter of mongod2 in order to listen to a different port.
the two daemons cannot share the same data dir, thus you have to change the --data-dir parameter of mongod2.

I find the below upstart works for me
# Ubuntu upstart file at /etc/init/mongodb.conf
description "manage mongodb instance"
start on runlevel [2345]
stop on runlevel [06]
limit nofile 20000 20000
kill timeout 300 # wait 300s between SIGTERM and SIGKILL.
env MONGODB_USER=mongodb
env MONGODB_DATA=/var/lib/mongodb/
env MONGODB_LOG=/var/log/mongodb/
env MONGODB_PID=/var/run/mongodb.pid
pre-start script
if [ ! -d $MONGODB_DATA ]; then
mkdir -p $MONGODB_DATA
chown $MONGODB_USER:$MONGODB_USER $MONGODB_DATA
fi
if [ ! -d $MONGODB_LOG ]; then
mkdir -p $MONGODB_LOG
chown $MONGODB_USER:$MOGODB_USER $MONGODB_LOG
fi
end script
exec start-stop-daemon --start --pidfile $MONGODB_PID --chuid $MONGODB_USER:$MONGODB_USER --exec /usr/bin/mongod -- --config /etc/mongodb/mongodb.conf
pre-stop exec start-stop-daemon --signal QUIT --stop --quiet --pidfile $MONGODB_PID --chuid $MONGODB_USER:$MONGODB_USER --exec /usr/bin/mongod -- --config /etc/mongodb/mongodb.conf

Related

Linux Debian 9 daemon stdin (bash)

Is there a way to pipe a value into a daemon so that when the daemon starts it populates the service options required to start successfully. I'm running Debian 9 and here is a snippet of code:
start-stop-daemon --start --chuid $USER \
$START_STOP_OPTIONS --exec $DAEMON -- \
--username $VPN_USER --password $VPN_PASSWORD --domain $VPN_DOMAIN \
$OPTIONS $VPN_SERVER:$VPN_SERVER_PORT&
Ideally I need to parse Y into the daemon otherwise the service being executed by the daemon cannot start correctly. I've tried a mixture of stdin from echo >>> <<< inside and outside of --exec but no luck...
TLDR;
How do I get this to work?
echo "Y" | start-stop-daemon ...
The proper way to do this is to have a -y switch(for example) in the program arguments that will start the daemon without prompts. In general it's a very bad practice to not have this option.
Otherwise I just tested with a simple python script that prompts for "Y" and it works.
Script:
#!/usr/bin/python3
from time import sleep
k = input("prompt:")
if k == "Y":
print("exit!")
exit(1)
while True:
sleep(100)
I run it like echo "Y" | start-stop-daemon --start --exec /home/user/tests/startd.py and it prints exit! and exits.
If it is the case that it requires multiple "Y"s you can try the yes program like
yes "Y" | start-stop-daemon --start --exec yourdaemon.

Supervisord on linux CentOS 7 only works when run with root

I am trying to run a process in the background as a deamon but it only works when I use root as user.
This is what I did.
Installed supervisor as told on their website
$ yum -y install python-setuptools
$ easy_install supervisor
created the config folders
$ mkdir -p /etc/supervisor/conf.d
populate with default settings
$ echo_supervisord_conf > /etc/supervisor/supervisord.conf
add a new user
$ useradd gogopher
on CentOS 7 to make it start automatically I had to do this
$ vim /usr/lib/systemd/system/supervisord.service
added the code below
[Unit]
Description=supervisord - Supervisor process control system for UNIX
Documentation=http://supervisord.org
After=network.target
[Service]
Type=forking
ExecStart=/usr/bin/supervisord -c /etc/supervisor/supervisord.conf
ExecReload=/usr/bin/supervisorctl reload
ExecStop=/usr/bin/supervisorctl shutdown
User=gogopher
[Install]
WantedBy=multi-user.target
now I can enable it so that it starts on reboot. this all works fine.
$ systemctl enable supervisord
$ systemctl start supervisord
$ systemctl status supervisord
OK
editing the config file to include files from conf.d folder
$ vim /etc/supervisor/supervisord.conf
adding at the end of file
[include]
files = /etc/supervisor/conf.d/*.conf
adding a simple program
$ vim /etc/supervisor/conf.d/goapp.conf
[program:main]
command=/srv/www/websiteurl.com/bin/main
autostart=true
autorestart=true
startretries=10
user=gogopher
$ systemctl restart supervisord
no error, but the process does not work
if I reboot nothing happens
$ systemctl status supervisord
shows that it supervisord is running but not the daemon program.
if I run
$ supervisorctl reload
I get the error
error: <class 'socket.error'>, [Errno 111] Connection refused: file: /usr/lib64/python2.7/socket.py line: 571
if I run
$ supervisorctl status main
I get the error
http://localhost:9001 refused connection
I have already disabled selinux.
but the weird part is that if I change both of them to root, it works.
The executable is able to be executed by user group and others.
So I have no idea what is going on. I have heard that I should not use
root as user that is running a webserver for security reasons.
For all the people out there having the same problem, this works for me.
cd
echo_supervisord_conf > /etc/supervisord.conf
# content of /etc/supervisord.conf ...
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
[inet_http_server] ; inet (TCP) server disabled by default
port=*:9001 ; (ip_address:port specifier, *:port for all iface) - I had all this wrong from my original config.
username=user
password=passwd
Paste this content into /etc/rc.d/init.d/supervisord ( I´m not the owner of this script, by now i don´t remember where i got it from )
#!/bin/sh
#
# /etc/rc.d/init.d/supervisord
#
# Supervisor is a client/server system that
# allows its users to monitor and control a
# number of processes on UNIX-like operating
# systems.
#
# chkconfig: - 64 36
# description: Supervisor Server
# processname: supervisord
# Source init functions
. /etc/rc.d/init.d/functions
prog="supervisord"
prefix="/usr/local/"
exec_prefix="${prefix}"
prog_bin="${exec_prefix}/bin/supervisord -c /etc/supervisord.conf"
PIDFILE="/var/run/$prog.pid"
start()
{
echo -n $"Starting $prog: "
daemon $prog_bin --pidfile $PIDFILE
sleep 1
[ -f $PIDFILE ] && success $"$prog startup" || failure $"$prog startup"
echo
}
stop()
{
echo -n $"Shutting down $prog: "
[ -f $PIDFILE ] && sleep 1 && killproc $prog || success $"$prog shutdown"
echo
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status $prog
;;
restart)
stop
start
;;
*)
echo "Usage: $0 {start|stop|restart|status}"
;;
esac
Make the script executable and register it as a service
sudo chmod +x /etc/rc.d/init.d/supervisord
sudo chkconfig --add supervisord
sudo chkconfig supervisord on
# Start the service
sudo service supervisord start
# Stop the service
sudo service supervisord stop
# Restart the service
sudo service supervisord restart

How do I run GAE as a background-service

I found a init.d script template -- filled in the blanks and tried to invoke GAE using something like:
start-stop-daemon -S --background python
/opt/google_appengine/dev_appserver.py --host=0.0.0.0
--admin_host=0.0.0.0 --php_executable_path=/usr/bin/php-cgi /var/www
This doesn't work...but if I run from the command line works fine but hangs the input...
How do I invoke this command at startup using init.d and change to the user "gae" -- similar to Apache runs as www-data
I also (briefly) tried to use start-stop-daemon to control Google App Engine (without any luck), so I ended up using /etc/rc.local to launch the daemon.
Add the following to /etc/rc.local (before any exit command):
sudo -i -u gae python /opt/google_appengine/dev_appserver.py --host=0.0.0.0 \
--storage_path /var/cache/appengine/gae \
--admin_host=0.0.0.0 --php_executable_path=/usr/bin/php-cgi /var/www > /dev/null 2> /dev/null &
Note, I included a storage_path in the launch options. Make sure you do the following:
sudo mkdir -p /var/cache/appengine/gae
sudo chown gae: /var/cache/appengine/gae
To restart the process (after an update), I just kill python and manually execute rc.local:
sudo killall -9 python
sudo /etc/rc.local
I have finally figured out how and why the start-stop-daemon was not working...it all boiled down to some simple syntactical errors and a (still?) misunderstanding on my behalf:
https://unix.stackexchange.com/questions/154692/start-stop-daemon-wont-start-my-python-script-as-service
In brief, when I use this init.d script and register it accordingly, GAE starts and stops accordingly:
#!/bin/sh
### BEGIN INIT INFO
# Provides: Google App Engine daemon management
# Required-Start:
# Required-Stop:
# Default-Start:
# Default-Stop:
# Short-Description: Google App Engine initscript
# Description: Start/stop appengine web server
### END INIT INFO
# Author: Alex Barylski <alex.barylski#gmail.com>
. /lib/lsb/init-functions
#
# Initialize variables
#
name=appengine
desc="Google App Engine"
bind=0.0.0.0
docroot=/var/www
phpexec=/usr/bin/php-cgi
pidfile=/var/run/$name.pid
args="--host=$bind --admin_host=$bind --php_executable_path=$phpexec"
prog='/usr/bin/python /opt/google_appengine/dev_appserver.py'
#
# TODO: Figure out how to switch user (ie: --chuid www-data)
#
case "${1}" in
start)
echo "Starting...$desc"
start-stop-daemon --start --make-pidfile --background --oknodo \
--pidfile $pidfile \
--name $name \
--exec $prog \
-- $args $docroot
;;
stop)
echo "Stopping...$desc"
start-stop-daemon --stop --quiet --oknodo --retry=0/30/KILL/5 --exec $prog
;;
restart)
${0} stop
sleep 1
${0} start
;;
*)
echo "Usage: ${0} {start|stop|restart}"
exit 1
;;
esac
exit 0
I cannot figure out how to start the service as www-data and I am sure I could make this script more robust but for development purposes it is sufficient and runs as a daemon.
Hope this helps someone in the future,
Alex

Minecraft server script to start on reboot

I am using a script I found on a minecraft wiki to automatically start my minecraft server after I reboot.
console log
exec start-stop-daemon --stop "stop" --start --chdir /minecraft --chuid minecraft \
--exec /usr/bin/java -- -Xms1536m -Xmx2048M -jar minecraft_server.jar nogui 2>&1
start on runlevel [2345]
stop on runlevel [^2345]
respawn
respawn limit 20 5
Every time I try to start the script I get this error.
root#bcserv:/# tail /var/log/upstart/minecraft-server.log
start-stop-daemon: only one command can be specified
Try 'start-stop-daemon --help' for more information.
Anyone know what I am doing wrong in my exec syntax? I am running Ubuntu Linux 13.10
Tried removing the --stop "stop" now I am getting this.
root#bcserv:/home/chris# tail /var/log/upstart/minecraft-server.log
/usr/bin/java already running.
And server does not appear to start.
root#bcserv:/home/chris# ps -aux |grep mine
root 4564 0.0 0.0 9452 904 pts/2 S+ 17:21 0:00 grep --color=auto mine –
Any other suggestions? It doesn't seem to be picking up my minecraft options.
Remove ' --stop "stop" ' from your exec.
You can only have one Command, --stop or --start, and it looks as if you just need the latter.

Upstart script for node.js app

I'm having trouble starting an Upstart script.
Here's the script (app.conf in /etc/init/)
description "node.js server"
author "kvz"
start on startup
stop on shutdown
script
# We found $HOME is needed. Without it, we ran into problems
export HOME="/root"
exec sudo -u /usr/local/bin/node \
/var/www/vhosts/travelseguro.com/node/app.js \
2>&1 >> /var/log/node.log
end script
When I run sudo start app, I get:
start: Unknown job: app
How can I make this work?
I was having the same problem running on the latest Amazon (AWS) linux which is Redhat based.
I have my upstart file in /etc/init called node.conf and when I ran sudo start node I would get a similar error to you start: Unknown job: node.
It turns out that the job won't start if there's an error in your .conf file. So I started out by commenting out all the lines and slowly building up to find the error. The error message isn't very clear and makes it look like upstart can't find your conf file.
Tailing your '/var/log/messages' will help you debug as Upstart logs to there (It may be somewhere different on Ubuntu. Mine said init: /etc/init/node-upstart.conf:8: Unknown stanza which helped me get to the bottom of it. In my particular case I was declaring variables incorrectly.
See on AskUbuntu for a similar thread.
Here's my edited working script:
<!-- language: lang-sh -->
#!upstart
# using upstart http://upstart.ubuntu.com/getting-started.html and node forever https://github.com/nodejitsu/forever/
# to run server
# redhat has additional sudo restrictions, you must comment out 'Default requiretty' from /etc/sudoers
#startup monitoring script derived from http://stackoverflow.com/questions/11084279/node-js-setup-for-easy-deployment-and-updating
description "node.js server"
author "jujhar"
env PROGRAM_NAME="node"
env FULL_PATH="/home/httpd/buto-middleman/public"
env FILE_NAME="forever.js"
env NODE_PATH="/usr/local/bin/node"
env USERNAME="springloops"
start on startup
stop on shutdown
script
export HOME="/root"
export NODE_ENV=staging #development/staging/production
echo $$ > /var/run/$PROGRAM_NAME.pid
cd $FULL_PATH
#exec sudo -u $USERNAME $NODE_PATH $FULL_PATH/$FILE_NAME >> /var/log/$PROGRAM_NAME.sys.log 2>&1
exec $NODE_PATH $FULL_PATH/$FILE_NAME >> /var/log/$PROGRAM_NAME.sys.log 2>&1
end script
pre-start script
# Date format same as (new Date()).toISOString() for consistency
echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Starting" >> /var/log/$PROGRAM_NAME.sys.log
end script
pre-stop script
rm /var/run/$PROGRAM_NAME.pid
echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Stopping" >> /var/log/$PROGRAM_NAME.sys.log
end script
-- Edit 2013-06-01 --
If you're on Centos or Amazon Linux like me, take a look at this init.d script.
-- Edit 2013-10-14 --
Here's a link to a gist of an init.d script that I actually use in production on Amazon Linux(Redhat Based). I simply keep it in my project under an init.d folder and then symlink to it in the /etc/init.d folder and now it's a daemon/service!
-- Edit 2014-06-05 --
Check out this awesome blog artcile by Jeff Dickey on Node.js in production using systemd which is much cleaner and easier than all the stuff we've been doing here (IMHO). He also uses Ansible to control his cluster (which I love) but you don't have to go that far if you're not ready.
After a few attempts I implemented working .conf file for upstart which works as a service with automatic start after reboot and restart (respawn) in case of crash. Also it can start my app with unprivileged user permissions. The name of the file is /etc/init/my-app.conf.
To start / stop service please use
sudo start my-app / sudo stop my-app
If you have an error like
start: Unknown job: my-app
exec the following command
sudo initctl reload-configuration
My /etc/init/my-app.conf file:
#my-app
description "node.js my-app website"
env FULL_PATH="/home/myuser/app.prod/app"
env NODE_PATH="/usr/bin/node"
start on filesystem or runlevel [2345]
stop on [!2345]
script
export HOME="/root"
export NODE_ENV=production
echo $$ > /var/run/my-app.pid
cd $FULL_PATH
#Use exec below if you want to launch it under myuser,
#don't forget to create /var/log/my-app.sys.log with appropriate permissions
#exec sudo -u myuser sh -c "$NODE_PATH server.js >> /var/log/my-app.sys.log 2>&1"
exec $NODE_PATH server.js >> /var/log/my-app.sys.log 2>&1
end script
pre-start script
echo "[`date`] (sys) Starting" >> /var/log/my-app.sys.log
end script
pre-stop script
rm /var/run/my-app.pid
echo "[`date`] (sys) Stopping" >> /var/log/my-app.sys.log
end script
#uncomment respawn if you want to restart your service in case of crash
#respawn
#respawn limit 50 30
I do recommend to uncomment respawn after you will make sure that everything works ok.
UPDATE
I improved my script (please keep in mind that it works not under root but under regular user zn ):
#znapi.conf
description "node.js zn api"
env FULL_PATH="/home/zn/app.prod"
env NODE_PATH="/usr/bin/node"
env LOG="/var/log/znapi.log"
env PIDFILE="/var/run/znapi.pid"
# Start up when the system hits any normal runlevel, and
#start on filesystem or runlevel [2345]
#start when mongod started
start on started mongod
# shuts down when the system goes to shutdown or reboot.
stop on runlevel [06]
respawn
respawn limit 50 5
pre-start script
# check that log file exist or create it
test -f $LOG || { touch $LOG; chown zn:zn $LOG; }
# Date format same as (new Date()).toISOString() for consistency
echo "[`date`] (sys) Starting" >> $LOG
end script
script
export NODE_ENV=production
exec start-stop-daemon --start -m -p $PIDFILE -c zn -d $FULL_PATH -x server.js >> $LOG 2>&1
end script
pre-stop script
rm $PIDFILE
echo "[`date`] (sys) Stopping" >> $LOG
end script

Resources