i have project in sails.js, i want to write a polling service that check some record in some interval and after that send email. my example code is:
module.exports.bootstrap = function(cb) {
cb();
var refresh = function() {
setTimeout(doWork, //someInterval);
};
var doWork = function() {
if (//check some condition) {
sendEmail();
}
refresh();
};
doWork();
}
i use pm2 libary and start my project with cluster mode. example code is:
pm2 start app.js -i 4
this command run app.js in cluster mode with 4 process.
the problem is my polling service run in all process because i run my polling service in config/bootstrap.js file and this is very bad.
my question is how can i run my service once in all process?
You can check if process is master and then run script only on that case.
var cluster = require('cluster');
if(cluster.isMaster) // rest of your service...
But for me... This is strange logic... You should queue your tasks to shared db, and when task is pooled remove it from it etc.
Related
I am using Meteorjs
I have one heavy query which I need to run in background
e.g:
Meteor.users.update({}, {$set:{status: 1}});
I need this query to run in background.
For that I used worker threads.
I created a worker (when my server launches in config.js):
Meteor.startup(function () {
workerThread = Npm.require('worker_threads');
worker = new workerThread.Worker('worker.js');
});
Contents of worker.js:
Meteor.startup(function () {
Meteor.users.update({}, {$set:{status: 1}});
});
The above code gives me an error:
ReferenceError: Meteor is not defined
Does anyone have idea how can I run a background task in Meteorjs?
What is Process, Worker, Thread, Task, Pool in Node.js from a programmer point of view?
I went through a lot of material, but difficult to understand quickly for a beginner programmer. Here is a quick summary
A Node.js process is created when run a Node.js program like node app.js (or the child process created through child_process or cluster modules). Each process will have its own memory and resources
Worker is a Node.js built-in module which takes your module (.js) as an input and creates worker object, inside a process, that executes asynchronously.
//app.js
//TODO add modules
const { Worker } = require('worker_threads');
//TODO wrap the below code into your code
const worker = new Worker('./task_processor.js');
const workerMaxLifetime = 10000;
//Send message to worker
worker.postMessage('Message to thread');
//Receive message from worker
worker.on('message', (message) => { console.log(' App:', message); });
//Terminate worker
setTimeout(() => { worker.terminate(); }, workerMaxLifetime);
Task is your module (.js) where you write the code to run as a Thread. Actually, we should call it 'Task Processor'
//task_processor.js
//TODO add modules
const { parentPort } = require('worker_threads');
//TODO wrap the below code into your code
//Receive message from App
parentPort.on('message', (task_input) => {
//Send message to App
parentPort.postMessage(task_input.a + task_input.b);
});
Thread is nothing but the worker in execution.
Pool is a wrapper .js file which create/terminate worker objects and facilitates communication between App and worker. Worker pool is not mandatory though most real world scenarios implements pools where worker thread concept is implemented. Example
Node.js module is a .js file
App: The main(or default) thread in a process is also referred as App
Process vs Worker: Each process will have its own memory and resources, whereas worker uses the same memory and resources of the process from which it is created.
I have an express application that predictably spikes CPU after running for a while under load. I'd like to proactively restart it every N minutes in order to avoid the spikes. It's currently running under forever, but I could use pm2 or some other process manager. Is there any process manager that can do periodic restarts? How can I go about accomplishing this with the minimum of added structure?
You can do it programmatically with PM2 with the following code.
var pm2 = require('pm2');
pm2.connect(function(err) {
if (err) throw err;
setTimeout(function worker() {
console.log("Restarting app...");
pm2.restart('app', function() {});
setTimeout(worker, NUM_MILLI_SECONDS);
}, NUM_MILLI_SECONDS);
});
This will restart it every number of given milliseconds. There is also a CRON library you can use.
I'm currently trying to use Node.js Kue for processing jobs in a queue, but I believe I'm not doing it right.
Indeed the way I'm working now, I have two different services (which in this case I'm running with Docker Compose): one Web API built with Express with sends jobs to the queue and one processing module. The issue here is with the processing module.
I've coded it as follows:
var kue = require('kue');
var config = require('./config');
var queue = kue.createQueue({
prefix: config.redis.queuePrefix,
redis: {
port: config.redis.port,
host: config.redis.host
}
});
queue.process('jobType', function (job, done) {
// do processing here...
});
When we run this with Node, it sits there waiting for things to be placed on the queue to do the processing.
There are two issues however:
It needs that Redis be available before running this module. If we run this without Redis already available, it crashes because the host is not accessible and ends the process.
If Redis suddenly becomes unavailable, the processing module also crashes because it cannot stablish the connection and the process is killed.
How can I avoid these problems?
My guess is that I should somehow make the code "wait" for Redis, but I have no idea on how to do this.
How can this be done in this case?
You can use promise to wait until redis is loaded. Then run your module.
loadRedis().then(() => {
//load your module
})
Or you can use generator to "stop" until redis is loaded.
function*(){
const redisLoaded = yield loadRedis();
//load your module
}
I'm developing some really simple node.js libraries for learning purposes.
It's about functions like HexToBase64 and things like that.
Ideally, I'd like to program in a text editor, and play with it on the node repl, having the code automatically reloaded on the repl on every save.
Any module or tool to interactively play with node?
There are modules such as supervisor, nodemon and forever that can reload your application on a code change. Otherwise, you can create your own implementation like this:
var fs = require('fs');
var cluster = require('cluster');
if (cluster.isMaster) {
var worker = cluster.fork();
fs.watch(process.argv[1], function(event, filename) {
worker.kill();
worker = cluster.fork();
});
}
if (cluster.isWorker) {
// put your application logic here that will
// run when this file changes
}
As for using Node interactively, you can just run node in a terminal and you have an interactive console. If you needed to load a script and use it interactively, then you would use .load script.js.