I wanted to perform some data processing in parallel using Bull NPM and start processing each job at the given cron Time
const Queue = require("bull"),
/**
* initialize the queue for executing cron jobs
**/
this.workQueue = new Queue(this.queueOptions.name, {
redis: redisConfig
});
this.workQueue.process((job, done) => {
done();
this.processJob(job)
.then(data => {
global.logger.info(`successfully-executed-job ${job.id}`);
})
.catch(error => {
global.logger.error(`JSON.stringify(error)}-in-executing-job-${job.id}`);
});
});
// here I have included Unique JobId
this.workQueue.add({}, {repeat: { cron:"5 * * * *",jobId:Date.now()});
Any suggestions to achieve the same?
The issue is resolved now if you're facing the same issue make sure that you're referring to the correct timezone.
Cheers!!
I also faced this same issue. One thing to note with respect to the above code is that a Queuescheduler instance is not initialized. Ofcourse timezone also plays a crucial role. But without a Queuescheduler instance (which has the same name as the Queue), the jobs doesnt get added into the queue. The Queuescheduler instance acts as a book keeper. Also take care about one more important parameter "limit". If you dont set the limit to 1, then the job which is scheduled at a particular time will get triggered unlimited number of times.
For example: To run a job at german time 22:30 every day the configuration would look like:
repeat: {
cron: '* 30 22 * * *',
offset: datetime.getTimezoneOffset(),
tz: 'Europe/Berlin',
limit: 1
}
Reference: https://docs.bullmq.io/guide/queuescheduler In this above link, the documentation clearly mentions that the queuescheduler instance does the book keeping of the jobs.
In this link - https://docs.bullmq.io/guide/jobs/repeatable, the documentation specifically warns us to ensure that we instantiate a Queuescheduler instance.
Related
I want to do something in node at, say, midnight each day.
I see a lot of stuff pointing me to node-cron, and I see this article configuring a docker container to execute a script per a crontab
I want to 1. not use any external packages and 2. keep the script being executed inside the server code itself (i.e. I couldn't have the docker container execute some other file on a schedule)
The use case is I want to update a cache on the server every day around midnight, and then, at more frequent intervals, use that cache for various things.
You can use setInterval to run the code every hour and check if it's around midnight
setInterval(() => {
if (new Date().getHours() === 0) {
// do stuff
}
}, 1000 * 60 * 60 * 60)
config/functions/cron.js
async function logger() {
console.log("Hello, Im Async");
}
module.exports = {
'*/10 * * * * *': async () => {
console.log("Before);
await strapi.services.collectionName.someMethodName();
console.log("After);
},
'*/10 * * * * *': async () => {
await logger();
},
};
In the example above logger gets called properly. But someMethodName doesn't get called at all. Before and After are also not printed. I dont know what is wrong and how to check it.
Same code works in staging site but not on production server.
I dont understand what is happening and how to solve this.
Does anyone know a solution to this?
Thanks!
Found the solution. Posting it here so that it will help someone else in the future.
Time format for both of the cron tasks is the same. When I checked the list of active cron in the strapi config; there was only one cron task available. It seems that as the key for the export was same, second entry was overriding the first one. When I wrote diff time formats for diff task, list of active cron jobs showed all the cron tasks.
Below commands will help you check the list of active cron tasks.
npm run strapi console
const strapi = require('strapi');
strapi().config
I am very new to NodeJS and trying to develop an application which acts as a scheduler that tries to fetch data from ELK and sends the processed data to another ELK. I am able to achieve the expected behaviour but after completing all the processes, scheduler job does not exists and wait for another scheduler job to come up.
Note: This scheduler runs every 3 minutes.
job.js
const self = module.exports = {
async schedule() {
if (process.env.SCHEDULER == "MinuteFrequency") {
var timenow = moment().seconds(0).milliseconds(0).valueOf();
var endtime = timenow - 60000;
var starttime = endtime - 60000 * 3;
//sendData is an async method
reports.sendData(starttime, endtime, "SCHEDULER");
}
}
}
I tried various solutions such Promise.allSettled(....., Promise.resolve(true), etc, but not able to fix this.
As per my requirement, I want the scheduler to complete and process and exit so that I can save some resources as I am planning to deploy the application using Kubernetes cronjobs.
When all your work is done, you can call process.exit() to cause your application to exit.
In this particular code, you may need to know when reports.sendData() is actually done before exiting. We would have to know what that code is and/or see the code to know how to know when it is done. Just because it's an async function doesn't mean it's written properly to return a promise that resolves when it's done. If you want further help, show us the code for sendData() and any code that it calls too.
I've been working with node for the first time in a while again and stumbled upon node-schedule, which for the most part has been a breeze, however, I've found resuming a scheduled task after canceling it via job.cancel() pretty difficult.
For the record, I'm using schedule to perform specific actions at a specific date (non-recurring) and under some circumstances cancel the task at a specific date but would later like to resume it.
I tried using job.cancel(true) after cancelling it via plain job.cancel() first as the documentation states that that would reschedule the task, but this has not worked for me. Using job.reschedule() after having cancelled job first yields the same result.
I could probably come up with an unelegant solution, but I thought I'd ask if anyone knows of an elegant one first.
It took me a while to understand node-schedule documentation ^^
To un-cancel a job, You have to give to reschedule some options.
If you don't pass anything to reschedule, this function returns false (Error occured)
For exemple, you can declare options, and pass this variable like this :
const schedule = require('node-schedule');
let options = {rule: '*/1 * * * * *'}; // Declare schedule rules
let job = schedule.scheduleJob(options, () => {
console.log('Job processing !');
});
job.cancel(); // Cancel Job
job.reschedule(options); // Reschedule Job
Hope it helps.
Does anyone know if it's possible to make a Heroku Scheduler job that would send an email to all of my users once per day? I'm using Meteor and MongoDB.
I can see that the Heroku Scheduler can run a command such as "node somefile.js" but I can't seem to figure out how to make a connection to the mongodb in a file like this. Can I somehow tap into the DB without involving Meteor in this?
Any help would be appreciated!
I eventually found a package to do so: synced-cron. Basically, you need to setup a method in which use the package to fire a recurring job.
The package website also has a sample code:
SyncedCron.add({
name: 'Crunch some important numbers for the marketing department',
schedule: function(parser) {
// parser is a later.parse object
return parser.text('every 2 hours');
},
job: function() {
var numbersCrunched = CrushSomeNumbers();
return numbersCrunched;
}
});
Here you just need to replace the code in the job function to send out the email.
The job supports schedules like "every 5 minutes", "at 5:00pm", etc. The package relies on the text parser in Later.js to parse the schedule. You can refer to the Later.js doc.
Two different options.
The first is to use Heroku's scheduler,
In which you create a text file in your bin directory:
#! /app/.heroku/node/bin/node
var test = require('./jobToDo') //put your job in this file (jobToDo.js)
Now you don't have to put the job in another .js file, but it makes it easier to work with, rather than coding in a plain text file. (put again that is up to you)
The first line #! /app/.heroku/node/bin/node may be different for you depending on how your configuration is set up, depending on your OS and node/npm set up.
The second option is a cron style library. This will allow you to decide when you want your code to run.
This is pretty easy, and for me the preferred method.
var CronJob = require('cron').CronJob;
var fn = function(){
// Do Something
}
var job = new CronJob({
cronTime: "00 00 02 * * 1-5",
onTick: fn,
start: true,
timeZone: 'America/Los_Angeles'
});
You can look at documentation on github