I'm working on something, using NodeJS, that's intended to run as a service that I can connect with.
Let's say I'm working on a Calculator npm module.
I will need to run it within my repo as following:
./node_modules/.bin/calculator start
And I want to keep it running forever, and i can connect it somehow (on port maybe?)
So, I can send/receive messages with the calculator using another node module, let's say 'calculator-connector', for example as following:
var calcConnector = require('calculator-connector'),
calc = calcConnector.connect();
calc.add(1, 2);
Any idea how can achieve this design?
I'd make it like this:
Calculator by itself shouldn't be opinionated, where and when it will run. I'd just create it in the moment I need it:
var calculator = require('calculator');
calculator.listen('localhost', 8000); // create the service listening on port 8000
// create client capable of submitting the tasks
calcClient = createCalculatorClient('localhost', 8000)
calcClient.add(1,2)
I believe that such setup is optimal for quick development and debugging.
When you'll need the things to be really separated (say, calculator itself will run on a separate server), you can do simple node script which'll run the calculator (basically, it's first 2 lines of the snippet above) and then create simple upstart job (in case of debianish server, or some alternative on other platforms), that'll keep the script alive.
PS:
check out how express works, it's beautifully designed:
http://expressjs.com/
read more about upstart:
http://upstart.ubuntu.com/getting-started.html
To run calculator forever, you should use PM2 or Forever.
PM2 allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks.
For connection, you could create a http or TCP server.
Related
I have a website hosted on Heroku and Firebase (front (react) and backend(nodejs)) and I have some "long running scripts" that I need to perform. I had the idea to deploy a node process to my raspberry pi to execute this (because I need resources from inside my network).
How would I set this up securely?
I think I need to create a nodejs process that checks the central server regularly if there are any jobs to be done. Can I use sockets for this? What technology would you guys use?
I think the design would be:
1. Local agent starts and connects to server
2. Server sends messages to agent, or local agent polls with time interval
EDIT: I have multiple users that I would like to serve. The user should be able to "download" the agent and set it up so that it connects to the remote server.
You could just use firebase for this right? Create a new firebase db for "tasks" or whatever that is only accessible for you. When the central server (whatever that is) determines there's a job to be done, it adds it to your tasks db.
Then you write a simple node app you can run on your raspberry pi that starts up, authenticates with firebase, and listens for updates on your tasks database. When one is added, it runs your long running task, then removes that task from the database.
Wrap it up in a bash script that'll automatically run it again if it crashes, and you've got a super simple pubsub setup without needing to expose anything on your local network.
I have a node server, which needs to:
Serve the web pages
Keep querying an external REST API and save data to database and send data to clients for certain updates from REST API.
Task 1 is just a normal node tasks. But I don't know how to implement the task 2. This task won't expose any interface to outside. It's more like a background task.
Can anybody suggest? Thanks.
To make a second node.js app that runs at the same time as your first one, you can just create another node.js app and then run it from your first one using child_process.spawn(). It can regularly query the external REST API and update the database as needed.
The part about "Send data to clients for certain updates from REST API" is not so clear what you're trying to do.
If you're using socket.io to send data to connected browsers, then the browsers have to be connected to your web server which I presume is your first node.js process. To have the second node.js process cause data to be sent through the socket.io connections in the first node.js process, you need some interprocess way to communicate. You can use stdout and stdin via child_process.spawn(), you can use some feature in your database or any of several other IPC methods.
Because querying a REST API and updating a database are both asynchronous operations, they don't take much of the CPU of a node.js process. As such, you don't really have to do these in another node.js process. You could just have a setInterval() in your main node.js process, query the API every once in a while, update the database when results are received and then you can directly access the socket.io connections to send data to clients without having to use a separate process and some sort of IPC mechanism.
Task 1:
Express is good way to accomplish this task.
You can explore:
http://expressjs.com/
Task 2:
If you are done with Expressjs. Then you can write your logic with in Express Framework.
This task then can be done with node module forever. Its a simple tool that runs your background scripts forever. You can use forever to run scripts continuously (whether it is written in node.js or not)
Have a look:
https://github.com/foreverjs/forever
I am still pretty new to NodeJS and want to know if I am looking at this in the wrong way.
Background:
I am making an app that runs once a week, generates a report, and then emails that out to a list of recipients. My initial reason for using Node was because I have an existing front end already built using angular and I wanted to be able to reuse code in order to simplify maintenance. My main idea was to have 4+ individual node apps running in parallel on our server.
The first app would use node-cron in order to run every Sunday. This would check the database for all scheduled tasks and retrieve the stored parameters for the reports it is running.
The next app is a simple queue that would store the scheduled tasks and pass them to the worker tasks.
The actual pdf generation would be somewhat CPU intensive, so this would be a cluster of n apps that would retrieve and run individual reports from the queue.
When done making the pdf, they would pass to a final email app that would send the file out.
My main concerns are communication between apps. At the moment I am setting up the 3 lower levels (ie. all but the scheduler) on separate ports with express, and opening http requests to them when needed. Is there a better way to handle this? Would the basic 'net' work better than the 'http' package? Is Express even necessary for something like this, or would I be better off running everything as a basic http/net server? So far the only real use I've made of Express is to specifically listen to a path for put requests and to parse the incoming json. I was led to asking here because in tracking logs so far I see every so often the http request is reset, which doesn't appear to affect the data received on the child process, but I still like to avoid errors in my coding.
I think that his kind of decoupling could leverage some sort of stateful priority queue with features like retry on failure, clustering, ...
I've used Kue.js in the past with great sucess, it's redis backed and has nice documentation and interface http://automattic.github.io/kue/
User case:
My nodejs server start with a configuration wizard that allow user to change the port and scheme. Even more, update the express routes
Question:
Is it possible to apply the such kind of configuration changes on the fly? restart the server can definitely bring all the changes online but i'm not sure how to trigger it from code.
Changing core configuration on the fly is rarely practiced. Node.js and most http frameworks do not support it neither at this point.
Modifying configuration and then restarting the server is completley valid solution and I suggest you to use it.
To restart server programatically you have to execute logics outside of the node.js, so that this process can continue once node.js process is killed. Granted you are running node.js server on Linux, the Bash script sounds like the best tool available for you.
Implementation will look something like this:
Client presses a switch somewhere on your site powered by node.js
Node.js then executes some JavaScript code which instructs your OS to execute some bash script, lets say it is script.sh
script.sh restarts node.js
Done
If any of the steps is difficult, ask about it. Though step 1 is something you are likely handling yourself already.
I know this question was asked a long time ago but since I ran into this problem I will share what I ended up doing.
For my problem I needed to restart the server since the user is allowed to change the port on their website. What I ended up doing is wrapping the whole server creation (https.createServer/server.listen) into a function called startServer(port). I would call this function at the end of the file with a default port. The user would change port by accessing endpoint /changePort?port=3000. That endpoint would call another function called restartServer(server,res,port) which would then call the startServer(port) with the new port then redirect user to that new site with the new port.
Much better than restarting the whole nodejs process.
I'm developing a simple 2d online game and now I'm designing my server. The server will be run on linux vps and I need a way to communicate with it (for example to close it, and as it will be run on vps, simply closing terminal won't work). So I think there are 2 options:
1) Write 2 apllications - server which doesn't say anything and doesn't accept console input and the second application is console which sends commands to server (like exit, get online players etc).
2) Write 1 application which has 2 threads - one is the real server, the second thread will be used for cin and cout. However I'm not sure if this will work on vps...
Or maybe there is better aproach? What is the usual way of doing this?
Remember that it must be vps-compatible way (only ssh access to it).
Thanks
I would go for a "daemon" (server) for the main server function and then use a secondary application that can connect to the server and send it commands.
Or just use regular signals, like most other servers do - when you reconfigure your Apache server, for example, you send it a SIGHUP signal that restarts the server. That way, you don't need a second application at all - just "kill -SIGHUP your_server_pid".