Lets say I have a bunch of tasks in an object, each with a date object. I was wondering if it's even possible to have tasks within the object be run within a single process and trigger when the date is called.
Here's an example:
var tasks = [
"when": "1501121620",
"what": function(){
console.log("hello world");
},
"when": "1501121625",
"what": function(){
console.log("hello world x2");
},
]
I'm fine with having these stored within a database and the what script being evaled from a string. I need a point in the right direction. I've never seen anything like this in the node world.
I'm thinking about using hotload and using the file system so I don't need to deal with databases.
Should I just look into setInterval or is there something out there that is more sophisticated? I know things like cron exist, the thing is I need all of these tasks to occur within an already existing running process. I need to be able to add a new task to the queue without ending the process.
To add a little context I need some way of queuing up socket.io .emit() functions.
Do not reinvent the wheel. Use cron package from npm. He is written pure on js (using second variant from bellow). So all of these tasks will occur within an already existing running process. For example your can create CronJob like this:
var CronJob = require('cron').CronJob;
var job = new CronJob(1421110908157);
job.addCallback(function() { /* some stuff to do */ });
In pure javascript you can do it only through setTimeout and setInterval methods. There are two variants:
1) Set interval callback, which will check your task queue and execute callbacks in appropriate time:
setInterval(function() {
for (var i = 0; ii = tasks.length; ++i) {
var task = tasks[i];
if (task.when*1000 < Date.now()) {
task.what();
tasks.splice(i,1);
--i;
}
};
}, 1000);
As you see accuracy of callback calling time will be dependent on interval time. Less interval time => more accuracy, but also more CPU usage.
2) Create wrapper around your tasks. So when you want to add new task you're calling some method addTask, that will be calling setTimeout with your task callback. Beware that maximum time for setTimeout is 2147483647ms (around 25 days). So if your time exceeds max time, you must set timeout on the maximum time with callback which will be set new timeout with remaining time. For example:
var MAX_TIME = 2147483647;
function addTask(task) {
if (task.when*1000 < MAX_TIME) {
setTimeout(task.what, task.when);
}
else {
task.when -= MAX_TIME/1000;
setTimeout(addTask.bind(null, task), MAX_TIME);
}
}
Related
I have a cron job running on pm2 that sends notifications on a 5 second interval. Although it should never happen, I'm concerned that the script will take longer than 5 seconds to run. Basically, if the previous run takes 6 seconds, I don't want to start the next run until the first one finishes. Is there a way to handle this solely in pm2? Everything I've found says to use shell scripting to handle it, but it's not nearly as easy to replicate and move to new servers when needed.
As of now, I have the cron job just running in a never ending while loop (unless there's an error) that waits up to 5 seconds at the end. If it errors, it exits and reports the error, then restarts because it's running via pm2. I'm not too excited about this implementation though. Are there other options?
edit for clarification of my current logic -
function runScript() {
while (!err) {
// do stuff
wait(5 seconds - however long 'do stuff' took) // if it took 1 second to 'do stuff', then it waits 4 seconds
}
}
runScript()
This feels like a hacky way to get around the cron limits of pm2. It's possible that I'm just being paranoid... I just wanna make sure I'm not using antipatterns.
What do you mean you have the cron job running in a while loop? PM2 is starting a node process which contains a never-ending while loop that waits 5 seconds? Your implementation of a cron seems off to me, maybe you could provide more details.
Instead of a cron, I would use something like setTimeout method. Run your script using PM2 and in the script is a method like such:
function sendMsg() {
// do the work
setTimeout(sendMsg, 5000); // call sendMsg after waiting 5 seconds
}
sendMsg();
By doing it this way, your sendMsg function can take all the time it needs to run, and the next call will start 5 seconds after that. PM2 will restart your application if it crashes.
If you're looking to do it at specific 5 second intervals, but only when the method is not running, simply add a tracking variable to the equation, something like:
let doingWork = false;
function sendMsg() {
if (!doingWork) {
doingWork = true;
// do the work
doingWork = false;
}
}
setInterval(sendMsg, 5000); // call sendMsg every 5 seconds
You could replace setInterval with PM2 cron call on the script, but the variable idea remains the same.
To have exactly 5000 ms between the end your actions:
var myAsyncLongAction = function(cb){
// your long action here
return cb();
};
var fn = function(){
setTimeout(function(){
// your long action here
myAsyncLongAction(function(){
console.log(new Date().getTime());
setImmediate(fn);
});
}, 5000)
};
fn();
To have exactly 5000 ms between the start of your actions :
var myAsyncLongAction = function(cb){
// your long action here
setTimeout(function(){
return cb();
}, 1000);
};
var fn = function(basedelay, delay){
if(delay === undefined)
delay = basedelay;
setTimeout(function(){
// your long action here
var start = new Date().getTime();
myAsyncLongAction(function(){
var end = new Date().getTime();
var gap = end - start;
console.log("Action took "+(gap)+" ms, send next action in : "+(basedelay - gap)+" ms");
setImmediate(fn, basedelay, (gap < basedelay ? 1 : basedelay - gap));
});
}, delay);
};
fn(5000);
How can I run a single method multiple times multi-threaded when called as a method of a class?
At first I tried to use the cluster module, but I realize it just re-runs the whole process from the start, rightfully so.
How can I achieve something like what's outlined below?
I want a class's method to spawn n processes, and when the parallel tasks are completed, I can resolve a promise which the method returns.
The problem with the code below is that calling cluster.fork() will fork index.js process.
index.js
const Person = require('./Person.js');
var Mary = new Person('Mary');
Mary.run(5).then(() => {...});
console.log('I should only run once, but I am called 5 times too many');
Person.js
const cluster = require('cluster');
class Person{
run(distance){
var completed = 0;
return new Promise((resolve, reject) => {
for(var i = 0; i < distance; i++) {
// run a separate process for each
cluster.fork().send(i).on('message', message => {
if (message === 'completed') { ++completed; }
if (completed === distance) { resolve(); }
});
}
});
}
}
I think the short answer is impossible. It's even worse - this has nothing to do with js. To multi (process or thread) in your particular problem you will essentially need a copy of the object in every thread, since it needs (maybe) access to fields - in this case you would need to either initialize it in every thread or share memory. That last one I don't think is provided in cluster, and not trivial in other languages in every use case.
If the calculation is independent of the Person I suggest you extract it, and use the usual (in index.js):
if(cluster.isWorker) {
//Use the i for calculation
} else {
//Create Person, then fork children in for loop
}
You then collect the results and change the Person as needed. You will be copying index.js, but this is standard and you only run what you need.
The problem is if results are dependent on Person. If these are constant for all i you can still send them to your forks independently. Otherwise what you have is the only way to fork. In general forking in cluster is not meant for methods, but for the app itself, which is the standard forking behavior.
Another solution
Following your comment, I suggest you checkout child_process.execFile or child_process.exec on same file.
This way you can spawn a totally independent process on the fly. Now instead of calling cluster.fork you call execFile. You can use either the exit code or stdout as return values (stderr etc.). Promise is now replaced with:
var results = []
for(var i = 0; i < distance; i++) {
// run a separate process for each
results.push(child_process.execFile().child.execFile('node', 'mymethod.js`,i]));
}
//... catch the exit event from all results or return a callback using results.
Inside mymethod.js Have your code that takes i and returns what you want either in the exit code or through stdout, both properties of the returned child_process. This is a bit un-node.js-y since you're waiting on asynchronous calls, but you're requirements are non standard. Since I'm not sure how you use this perhaps returning a callback with the array is a better idea.
How can I correctly perform something like sleep function using getTimer()? I need to do an action every 15 seconds. The code below doesn't work. I compile it with mtasc compiler on Linux.
class Tuto
{
static var lastMsg = 0;
static var msgInt = 15000;
static function main(mc)
{
if(getTimer() > lastMsg + msgInt)
{
trace("something");
lastMsg = getTimer();
}
}
}
The main instruction will be executed just once. You have to build some kind of loop or rely on the tick events sent by the player to execute your code continuously.
The basic options are:
while (true) { doSomething() }
this will execute forever, but remember that the flashplayer is single threaded so while that runs everything else will be frozen, UI and user inputs included. this is only "good" if you are building some heavy-processing tool that has no need of interacting with the user.
setInterval(doSomething, 15000)
this creates an interval that will call your function every X milliseconds. This is the simplest option and probably what you're looking for.
addEventListener(Event.ENTER_FRAME, doSomething)
this registers a listener for the ENTER_FRAME event of the Flash Player, which will be dispatched 30 times per second (by default). Inside that function you can check the current time with getTimer() and decide if it's time to execute your logic.
I'd like to monitor how long each run of the event loop in node.js takes. However I'm uncertain about the best way to measure this. The best way I could come up with looks like this:
var interval = 500;
var interval = setInterval(function() {
var last = Date.now();
setImmediate(function() {
var delta = Date.now() - last;
if (delta > blockDelta) {
report("node.eventloop_blocked", delta);
}
});
}, interval);
I basically infer the event loop run time by looking at the delay of a setInterval. I've seen the same approach in the blocked node module but it feels inaccurate and heavy. Is there a better way to get to this information?
Update: Changed the code to use setImmediate as done by hapi.js.
"Is there a better way to get this information?"
I don't have a better way to test the eventloop than checking the time delay of SetImmediate, but you can get better precision using node's high resolution timer instead of Date.now()
var interval = 500;
var interval = setInterval(function() {
var last = process.hrtime(); // replace Date.now()
setImmediate(function() {
var delta = process.hrtime(last); // with process.hrtime()
if (delta > blockDelta) {
report("node.eventloop_blocked", delta);
}
});
}, interval);
NOTE: delta will be a tuple Array [seconds, nanoseconds].
For more details on process.hrtime():
https://nodejs.org/api/all.html#all_process_hrtime
"The primary use is for measuring performance between intervals."
Check out this plugin https://github.com/tj/node-blocked I'm using it now and it seems to do what you want.
let blocked = require("blocked");
blocked(ms => {
console.log("EVENT LOOP Blocked", ms);
});
Will print out how long in ms the event loop is blocked for
Code
this code will measure the time in nanoseconds it took for the event loop to trigger. it measures the time between the current process and the next tick.
var time = process.hrtime();
process.nextTick(function() {
var diff = process.hrtime(time);
console.log('benchmark took %d nanoseconds', diff[0] * 1e9 + diff[1]);
// benchmark took 1000000527 nanoseconds
});
EDIT: added explanation,
process.hrtime([time])
Returns the current high-resolution real time in a [seconds, nanoseconds] tuple Array. time is an optional parameter that must be the result of a previous process.hrtime() call (and therefore, a real time in a [seconds, nanoseconds] tuple Array containing a previous time) to diff with the current time. These times are relative to an arbitrary time in the past, and not related to the time of day and therefore not subject to clock drift. The primary use is for measuring performance between intervals.
process.nextTick(callback[, arg][, ...])
Once the current event loop turn runs to completion, call the callback function.
This is not a simple alias to setTimeout(fn, 0), it's much more efficient. It runs before any additional I/O events (including timers) fire in subsequent ticks of the event loop.
You may also want to look at the profiling built into node and io.js. See for example this article http://www.brendangregg.com/flamegraphs.html
And this related SO answer How to debug Node.js applications
Suppose you've got a 3rd-party library that's got a synchronous API. Naturally, attempting to use it in an async fashion yields undesirable results in the sense that you get blocked when trying to do multiple things in "parallel".
Are there any common patterns that allow us to use such libraries in an async fashion?
Consider the following example (using the async library from NPM for brevity):
var async = require('async');
function ts() {
return new Date().getTime();
}
var startTs = ts();
process.on('exit', function() {
console.log('Total Time: ~' + (ts() - startTs) + ' ms');
});
// This is a dummy function that simulates some 3rd-party synchronous code.
function vendorSyncCode() {
var future = ts() + 50; // ~50 ms in the future.
while(ts() <= future) {} // Spin to simulate blocking work.
}
// My code that handles the workload and uses `vendorSyncCode`.
function myTaskRunner(task, callback) {
// Do async stuff with `task`...
vendorSyncCode(task);
// Do more async stuff...
callback();
}
// Dummy workload.
var work = (function() {
var result = [];
for(var i = 0; i < 100; ++i) result.push(i);
return result;
})();
// Problem:
// -------
// The following two calls will take roughly the same amount of time to complete.
// In this case, ~6 seconds each.
async.each(work, myTaskRunner, function(err) {});
async.eachLimit(work, 10, myTaskRunner, function(err) {});
// Desired:
// --------
// The latter call with 10 "workers" should complete roughly an order of magnitude
// faster than the former.
Are fork/join or spawning worker processes manually my only options?
Yes, it is your only option.
If you need to use 50ms of cpu time to do something, and need to do it 10 times, then you'll need 500ms of cpu time to do it. If you want it to be done in less than 500ms of wall clock time, you need to use more cpus. That means multiple node instances (or a C++ addon that pushes the work out onto the thread pool). How to get multiple instances depends on your app strucuture, a child that you feed the work to using child_process.send() is one way, running multiple servers with cluster is another. Breaking up your server is another way. Say its an image store application, and mostly is fast to process requests, unless someone asks to convert an image into another format and that's cpu intensive. You could push the image processing portion into a different app, and access it through a REST API, leaving the main app server responsive.
If you aren't concerned that it takes 50ms of cpu to do the request, but instead you are concerned that you can't interleave handling of other requests with the processing of the cpu intensive request, then you could break the work up into small chunks, and schedule the next chunk with setInterval(). That's usually a horrid hack, though. Better to restructure the app.