My application is built with three distincts servers: each one of them serves a different purpose and they must stay separated (at least, for using more than one core). As an example (this is not the real thing) you could think about this set up as one server managing user authentication, another one serving as the game engine, another one as a pubsub server. Logically the "application" is only one and clients connect to one or another server depending on their specific need.
Now I'm trying to figure out the best way to run a setup like this in a production environment.
The simplest way could be to have a bash script that would run each server in background one after the other. One problem with this approach would be that in the case I need to restart the "application", I should have saved each server's pid and kill each one.
Another way would be to use a node process that would run each servers as its own child (using child_process.spawn). Node spawning nodes. Is that stupid for some reason? This way I'd have a single process to kill when I need to stop/restart the whole application.
What do you think?
If you're on Linux or another *nix OS, you might try writing an init script that start/stop/restart your application. here's an example.
Use specific tools for process monitoring. Monit for example can monitor your processes by their pid and restart them whenever they die, and you can manually restart each process with the monit-cmd or with their web-gui.
So in your example you would create 3 independent processes and tell monit to monitor each of them.
I ended up creating a wrapper/supervisor script in Node that uses child_process.spawn to execute all three processes.
it pipes each process stdout/stderr to the its stdout/stderr
it intercepts errors of each process, logs them, then exit (as it were its fault)
It then forks and daemonize itself
I can stop the whole thing using the start/stop paradigm.
Now that I have a robust daemon, I can create a unix script to start/stop it on boot/shutdown as usual (as #Levi says)
See also my other (related) Q: NodeJS: will this code run multi-core or not?
Related
In my project I have several servers which run NodeJS applications using PM2, those were not created by me. I am not that familiar with the PM2. Now I need to start a new server, which is simply a CRON process that queries an ElasticSearch instance.
There are no routes or anything in it, just a CRON with some logging.
Here is my dilemma. I have played with PM2 and I become somewhat familiar with what is it, and what it does. But the question is how shall I run it?
The previous projects do have PM2 config.json with many parameters, and they are started in cluster mode (handled with Nginx), and when I start them I see all process's becoming daemons. But in my case I don't need that. I just need it to run as a single service.
In other words if I use the configuration file to run the PM2, I see it spawned in cluster mode, and it creates chaos as my CRON is fired many times. I don't need that. If I start it in Fork mode, it also spawns instances, but all of them die, except one (due to which they are using same port). I also don't need that.
I just need single service.
I managed to run the my CRON app.js with the singe line, simple as:
PM2 start app.js. It runs in single thread, and I can see it's info with PM2 status. All fine.
If I run it with the single line(as in my case), is it considered ok? Based in my knowledge if I use config.json, it will always run it in fork or cluster.
Is it ok to run it in single line, or do I need still to use a config.json file.
If you only need one process to be run, as is the case, you're doing the right thing.
Requirements:
I want to run my application on linux in the background (at startup of course).
I want to be able to call start/stop/restart commands directly from console (it have to be simple just like for /etc/init.d - just call simple command directly from console).
I want to be able to call status - and I want that this command will somehow get the actual status of application returned by itself. I thought that I can call some method which returns String or just use stdin to send command but when I do noup .. &, or start-stop-daemon, then the stdin is detached. Is there a simple way to attach stdin back to the application (I've seen that I can create a pipe, but this is pretty complitated). Or what is the best way to communicate with application after it is started as a daemon (I can make a socket and connect through telnet for example, but I am looking for simpler solution and possibility to do it directly from console, without starting telnet first)? Ideally it will be great to get the possibility to send any command, but simple status will be sufficient (but again - it have to communicate with the application to get that status somnehow)
I have found many different answers. Some of them says to simply use nohup and &, and some others says that nohup and & is old fashion. Some answers says to use start-stop-daemon or JSvc (for java). But it seems that none of them will suffice this 3 requirements from me.
So... What are the simplest possibilities for all of 3 requirements to be met?
PS. I don't want to use screen. The application must be run as a linux daemon.
PPS. Application is written in Java but I am looking for generic soluction which is not limited to java.
You should create a command line tool for communicate with a daemon in way you need. The tool itself can use TCP/IP or named pipes.
And then use cli-tool start|stop|restart|status from console.
If you need to start a daemon at startup sequence (before user login) you have to deal with init system (init.d, systemd, OpenRC, etc...).
Dragons be here:
Be sure that init doesn't restart your daemon after manual stop via cli.
Command line tool itself runs with unprivileged user rights, so restart may be hard if first startup script use superuser rights or application-specific user and, especially in case deep init integration, you might have to use sudo cli-tool start.
To avoid this one possible solution is to make wrapper daemon, that runs forever via init and control the underlying application (start-stop) with proper rights.
Cons: Develop two additional tools for a daemon.
Pros: Wrapper daemon can operate as a circuit breaker between superuser/specific user and userspace.
My application is composed of 4 unique processess. For HA reason, i am going to start 3 instances of each such that 2 instances of each process would run on a single linux host and one another set on a different linux host.
I am trying to write a monitoring script (bash script) that would periodically poll for these processess. My main challenge is that it sounds kludgy to write a script which is host name and process name dependent. For example , i dont want to write a script which monitors process-A-1, process-B-1, process-A-2, process-B-2 on linux host with IP Address A and process-A-3 on linux IP host Address B.
One way to write a monitoring script which is independent of host and process name is that when each of these process starts, they create a mutex name. For example, process-A-1 will create a mutex called mutex.process-A-1 and process-A-2 will create a mutex called mutex.process-A-2. Then all the script has to do is look for mutexes on a system with name like mutex.process-A*. The script then can use a ps command to check if those processess are running.
My question is that coupling with mutex name okay? Is there another way you have solved this problem on linux?
I would personally write a bash script which runs all this processes in background, then you can store their PIDs directly after calling them, storing process1_pid=$! after you send each one to the background and then trigger another script for monitoring using those pids.
Other way to get their PIDs is using the jobs command which will list all the jobs you have set to the background, jobs -p will list all the PIDs you have on the background. You can also make use of jobs to know if they're still running or not.
http://www.gnu.org/software/bash/manual/bashref.html#Job-Control
I'd start from there, if it's more complicated and your processes are being created from other places you can always use a particular user to run them all and use ps -u to filter them by user.
Is there a universally accepted means of deamonizing (and of course, later communicating through signals or some abstraction thereon) a node script?
That is, is there a Node equivalent of:
if (fork())
// parent process, die
exit(1);
// we're a daemon
The following is a list of ways to run Node as a background daemon on
different platforms:
nodejs-autorestart manages a Node instance on Linux which uses Upstart (Ubuntu, Debian, and so on).
fugue watches a Node server, restarting it if it crashes.
forever is a small commandline Node script which ensures a script will run "forever".
node-init is a Node script which turns your Node application into a LSB-compliant init script. LSB being a specification of Linux
compatibility.
There is not a built-in way to do this in Node. Take a look at Writing Daemon's in JavaScript with Node.js for one implementation (warning: this is relatively old and Node moves fast--I haven't tested it. :)
Upstart works well for me, though I'm having an issue when I serve over https. Here's the tutorial I used:
http://kevin.vanzonneveld.net/techblog/article/run_nodejs_as_a_service_on_ubuntu_karmic/
You can use node's process object to send/handle signals.
As others have pointed out there really isn't a way to do this in Node directly. You really need to run it using foreverjs. The reason you need to run it using a monitor like forever, is because error thrown by your code will often result in the entire Node process to quit and exit. A monitor will look for this occurring and restart the process immediately.
It is also important to note that while the process is restarting, your server will not respond to request so plan ahead if you expect this to be a problem and make sure that you have a few servers processes running under a load-balancer.
I am trying to set up a development environment for node.js. I assumed at first that it requires something similar to the traditional, "localhost" server approach. But I found myself at a loss. I managed to start a node.js hello world app from the terminal. Which doesn't looked like a big deal - having to start an app from the console isn't that hard. But, after some tweaking, I found out that the changes aren't shown in the browser immediately - you need to "node [appName here]" it again to run.
So, my question is:
Is there a software or a tutorial on how to create a more "traditional" development server on your local machine? Along with port listening setup, various configurations, root directories etc (things that are regular in stacks like XAMMP, BitNami or even the prepackaged Ubuntu LAMP). Since I'm new at node.js, I can't really be sure I'm even searching for the right things on google.
Thanks.
Take a look at :
https://github.com/remy/nodemon
it'll allow you to do - nodemon app.js
and the server will restart automatically in case of failure.
To do this I built a relatively small tool in NodeJS that allows me to start/stop/restart a NodeJS child process (which contains the actual server) and see/change configuration option and builds/versions of the application, with admin options available on a different tcp port. It also monitors said child process to automatically respawn it if there was a error (and after x failed attempts stops trying and contacts me).
While I'm prohibited from sharing source code, this requires the (built-in) child_process module, which has a spawn method that returns a child process I guess, which contains a pid (process id) which you can use with the kill method to kill said child process. Instead of killing it you could also work with SIGINT an catch it within your child application to first clean up some stuff and then exit. It's relatively easy to do.
Some nice reading material regarding this.
http://nodejs.org/docs/v0.5.5/api/child_processes.html