I need to run code in Node.js every 24 hours. I came across a function called setTimeout. Below is my code snippet
var et = require('elementtree');
var XML = et.XML;
var ElementTree = et.ElementTree;
var element = et.Element;
var subElement = et.SubElement;
var data='<?xml version="1.0"?><entries><entry><TenantId>12345</TenantId><ServiceName>MaaS</ServiceName><ResourceID>enAAAA</ResourceID><UsageID>550e8400-e29b-41d4-a716-446655440000</UsageID><EventType>create</EventType><category term="monitoring.entity.create"/><DataCenter>global</DataCenter><Region>global</Region><StartTime>Sun Apr 29 2012 16:37:32 GMT-0700 (PDT)</StartTime><ResourceName>entity</ResourceName></entry><entry><TenantId>44445</TenantId><ServiceName>MaaS</ServiceName><ResourceID>enAAAA</ResourceID><UsageID>550e8400-e29b-41d4-a716-fffffffff000</UsageID><EventType>update</EventType><category term="monitoring.entity.update"/><DataCenter>global</DataCenter><Region>global</Region><StartTime>Sun Apr 29 2012 16:40:32 GMT-0700 (PDT)</StartTime><ResourceName>entity</ResourceName></entry></entries>'
etree = et.parse(data);
var t = process.hrtime();
// [ 1800216, 927643717 ]
setTimeout(function () {
t = process.hrtime(t);
// [ 1, 6962306 ]
console.log(etree.findall('./entry/TenantId').length); // 2
console.log('benchmark took %d seconds and %d nanoseconds', t[0], t[1]);
//benchmark took 1 seconds and 6962306 nanoseconds
},1000);
I want to run the above code once per hour and parse the data. For my reference I had used one second as the timer value. Any idea how to proceed will be much helpful.
There are basically three ways to go
setInterval()
The setTimeout(f, n) function waits n milliseconds and calls function f.
The setInterval(f, n) function calls f every n milliseconds.
setInterval(function(){
console.log('test');
}, 60 * 60 * 1000);
This prints test every hour. You could just throw your code (except the require statements) into a setInterval(). However, that seems kind of ugly to me. I'd rather go with:
Scheduled Tasks
Most operating systems have a way of sheduling tasks. On Windows this is called "Scheduled Tasks" on Linux look for cron.
Use a libary As I realized while answering, one could even see this as a duplicate of that question.
Related
This question already has answers here:
How do I measure the execution time of JavaScript code with callbacks?
(12 answers)
Closed 1 year ago.
I have a node file which contains many methods including a Async call method which fetches data from db. I need to find the exact execution time of it.
So i tried,
.
.
var start = Date.now();
await dbFetching()
var end = Date.now();
console.log(end - start)
.
Then I tried with an external shell script which executes the file and find the actual time execution of the entire file execution. But the issue is i need to calculate only the time taken for the Async call (dbFetching). Below is my shell script,
#!/bin/sh
START=$(date +%s)
node ./s3-glacier.js
END=$(date +%s)
DIFF=$(( $END - $START ))
echo "It took $DIFF seconds"
But i though like, if i can run entire node script in the shell script, May be i can calculate the time. Therefore suggest your thought on this to calculate only the Async time consumption.
Thanks in advance
If you have this possibility - use async/await. It would look like:
var start = Date.now();
await dbFetching(); // blocks execution of code
var end = Date.now(); // this happens AFTER you've fetched the data
console.log(end - start)
That would require your function to be marked as async though, or return a Promise.
Otherwise you'd need to pass a callback to a function, so that the function calls it when it's done and you know that it's now time to calculate the dime. Something along the lines of:
var start = Date.now();
dbFetching(function done() { // done should be called AFTER the data is fetched
var end = Date.now();
console.log(end - start)
})
I want to execute a piece code each Sunday 23:59 (11 pm) (basically at the end of each week). However, it should only be fired once per week.
A setInterval() function won't cut it here, as the app might be restarted meanwhile.
If this will help anyhow, I had this basic idea:
Set an interval (with setInterval) for every 5-10 seconds and check if it's Sunday and hour 23 (11 pm). However, this solution will be inconsistent and may fire more than once a week. I need a more bullet-proof solution to this.
You can use any cron module (like https://www.npmjs.com/package/cron) and set job for 59 23 * * 0 (ranges)
const { CronJob } = require('cron');
const job = new CronJob('59 23 * * 0', mySundayFunc);
job.start();
How about calculating the remaining time on start, like this code
const WEEK_IN_MS = 604800000;
const ONE_HOUR_IN_MS = 3600000;
const FOUR_DAYS_IN_MS = 4 * WEEK_IN_MS / 7;
function nextInterval() {
return WEEK_IN_MS - ((Date.now() + FOUR_DAYS_IN_MS) % WEEK_IN_MS) - ONE_HOUR_IN_MS;
}
const interval = nextInterval();
console.log(`run after ${interval} ms`);
setTimeout(
() => console.log('Do it!!!'),
interval
)
Working on a scheduler with RxJS that every second checks the array of jobs. When job is finished it is removed from array. I would like to run that with the .mergeAll(concurrency) parameter so for example there are only two jobs running at the same time.
Currently I have an workaround which can be seen here.
What I am trying is something like
Observable
.interval(1000)
.timeInterval()
.merge(...jobProcesses.map(job => Observable.fromPromise(startJob(job.id))))
.mergeAll(config.concurrency || 10)
.subscribe();
which obviously doesn't work. Any help would be appreciated.
From the comments, it seems you are simply trying to limit concurrency, and this interval stuff is just a detour. You should be able to get what you need with:
const Rx = require('rxjs/Rx')
let startTime = 0
const time = () => {
if (!startTime)
startTime = new Date().getTime()
return Math.round((new Date().getTime() - startTime) / 1000)
}
const jobs = new Rx.Subject() // You may additionally rate-limit this with bufferTime(x).concatAll()
const startJob = j => Rx.Observable.of(undefined).delay(j * 1000).map(() => time())
const concurrency = 2
time()
jobs
.bufferCount(concurrency)
.concatMap(buf => Rx.Observable.from(buf).flatMap(startJob))
.subscribe(x => console.log(x))
Rx.Observable.from([3, 1, 3]).subscribe(jobs)
// The last job is only processed after the first two are completed, so you see:
// 1
// 3
// 6
Note that this technically isn't squeezing out the maximum amount of concurrency possible, since it breaks the jobs up into constant batches. If your jobs have significantly uneven processing times, the longest job in the batch will delay pulling work from the next batch.
I am using the scheduled script which will create the custom records based on criteria. every time when the schedule script runs it should create approx. 100,000 records but the script is timing out after creating 5000 or 10000 records. I am using the below script to prevent the script execution usage limit but even with this also the script is not working. can any one please suggest some thing or provide any information. any suggestions are welcome and highly appreciated.
In my for loop iam using the below script. with this below script included the scheduled script is able to create up to 5000 or 10000 records only.
if (nlapiGetContext().getRemainingUsage() <= 0 && (i+1) < results.length )
{
var stateMain = nlapiYieldScript();
}
If you are going to reschedule using the nlapiYieldScript mechanism, then you also need to use nlapiSetRecoveryPoint at the point where you wish the script to resume. See the Help documentation for each of these methods, as well as the page titled Setting Recovery Points in Scheduled Scripts
Be aware that nlapiSetRecoveryPoint uses 100 governance units, so you will need to account for this in your getRemainingUsage check.
#rajesh, you are only checking the remaining usage. Also do check for execution time limit, which is 1 hour for any scheduled script. Something like below snippet-
var checkIfYieldOrContinue = function(startTime) {
var endTime = new Date().getTime();
var timeElapsed = (endTime * 0.001) - (startTime * 0.001);
if (nlapiGetContext().getRemainingUsage() < 3000 ||
timeElapsed > 3500) { //3500 secs
nlapiLogExecution('AUDIT', 'Remaining Usage: ' + nlapiGetContext().getRemainingUsage() + '. Time elapsed: ' + timeElapsed);
startTime = new Date().getTime();
var yieldStatus = nlapiYieldScript();
nlapiLogExecution('AUDIT', 'script yielded.' + yieldStatus.status);
nlapiLogExecution('AUDIT', 'script yielded reason.' + yieldStatus.reason);
nlapiLogExecution('AUDIT', 'script yielded information.' + yieldStatus.information);
}
};
Inside your for loop, you can call this method like-
var startTime = new Date();
if ((i+1) < results.length ) {
//do your operations here and then...
checkIfYieldOrContinue(startTime);
}
I have a script that lets you process an array like a forEach. The script checks each iteration and calculates the maximum usage and yields when there is not enough usage left to cover the max.
Head over to https://github.com/BKnights/KotN-Netsuite and download simpleBatch.js
I have a simple node.js parser that has to push data to a remote server during work hours only and sleep for the rest of the time.
Looking at the available modules, schedule and node-cron (https://github.com/ncb000gt/node-cron) seems to do part of my requirement.
I am using the PM2 module to restart the process, when it goes down
Here is what I have so far in coffee script:
runParser = (callback) ->
#...
console.log 'waking up parser...'
parseAll()
return
_jobs = [ {
name: 'Start parser'
cronTime: '00 34 16 * * 1-5'
onTick: runParser
start: true
id: 'parsedbf'
#timeZone: 'Europe/London'
} ]
_cronJobs = {}
schedule = ->
_jobs.map (job) ->
_cronJobs[job.id] = new cronJob(job)
console.log util.format('%s cronjob scheduled at %s on timezone', job.name, job.cronTime)
return
return
run = ->
start = moment('08:30','HH:mm').valueOf()
now = moment().valueOf()
end = moment('18:00','HH:mm').valueOf()
if start < now and now < end
runParser()
else
schedule(console.info 'scheduler started...')
run(console.info 'sync code statrted after a hard reboot...')
my question, how do i change the script so that at 18:30 the parser is just idle?
should i use schedule.js (http://bunkat.github.io/schedule/index.html) how do i modify the code for this?
any advice much appreciated
Is there any reason you can't just run this in cron? You have the question tagged for cron which refers to the unix utility, it was made to do this sort of thing. You could use a combination of cron and forever: one call starts it in the am and another stops the script in the evening but it runs continuously otherwise.