I am new to Agenda jobs (https://github.com/rschmukler/agenda) and fail to understand how I can schedule a job to run every day at a given time. I have tried the following:
agenda.every('everyday at 00:01', ...) - runs only one time.
agenda.schedule('at 00:01', ...) and then job.repeatEvery('1 day') but without much effect.
Agenda internally uses Human Interval which is inspired by date. i checked demo of date here and found that everyday at 00:00 is accepted but could not use that very well with Agenda.
Any help will be appreciated. Thank you.
//You can use something like this...
agenda.define('first', (job, done) => {
Somefunction()
job.repeatEvery('24 hours', {
skipImmediate: true
});
job.save()
done()
})
agenda.start()
agenda.on('ready', function () {
agenda.schedule('everyday at 00:00','first')
})
// This worked for me..
I think you can use repeatAt() for this purpose. Like repeatAt('12am')
or you can also use 24 hour format:- repeatAt('00:00')
The solution proposed by others, I think is a bit funky as calling job.repeatEvery() within the job handler feels kind of out of its place.
agenda.every accepts cron format, therefore you can apply it to any defined job and run the handler according to the cron sequence
You have a defined job:
agenda.define('do-something', async (job, done) => {
// Doing some heavy async stuff here
async heavyStuff();
done();
})
After server initialisation you can call it anywhere in your code to setup a repeated job. Just make sure await agenda.start() was already called and agenda has an established mongo connection
await agenda.every("0 1 * * *", "do-something");
This will run the do-something job and call the handler every day at 00:01am
Related
I'm working with node and typescript, using node-cron 2.0 to schedule a background operation to run every hour.
cron.schedule("0 0 * * * *", () => {
purgeResponsesSurveys();
});
I'm concerned about what happens if the method doesn't finish within 1 hour since I don't want two instances of my method to run at the same time.
What are the best practices to prevent the scheduler from invoking the function purgeResponsesSurveys if it is already running from the previous hourly invocation?
You can use a Semaphore to prevent parallel calls.
You will need to know when purgeResponsesSurveys is done. So if it's asynchronous you will need to return Promise or receive a callback that will be called when purgeResponsesSurveys is done.
I used semaphore npm package.
Here is a small example/simulation.
const semaphore = require('semaphore');
const sem = semaphore(1);
simulateCron(function() {
console.log('cron was triggered')
// wrap task with mutex
sem.take(function() {
longTask(function(){
sem.leave();
})
})
})
function longTask(cb) {
console.log("Start longTask")
setTimeout(function(){
cb()
console.log("Done longTask")
}, 3000)
}
function simulateCron(cb) {
setInterval(cb, 500)
}
// output
cron was triggered
Start longTask
cron was triggered
cron was triggered
cron was triggered
cron was triggered
cron was triggered
Done longTask
Start longTask
cron was triggered
...
So, I found an old question about this (here: How to stop a node cron job) but it doesn't work for me (and I can't realize the reason why).
I'm using cron-job-manager (since I plan to use more than one scheduled job at the same time) but as far as I know it should be built on node-cron (but I'm a newbie, so...)
So, I'm asking again: how do I deal with starting and stopping a cron job under certain conditions?
I'm actually doing this for a discord bot: basically, the user input a command for starting and one for stopping.
First try was something like:
job = cron('keyTask','* * * * * *', ()=>{
//Do the actual job
},
{
start: false,
timeZone:"Europe/London",
});
switch args[1]
case go:
job.start();
break;
case stop:
job.stop();
break;
This start the job successfully when the user gives the 'go' command, but when it gives the 'stop' command, the scheduled job just won't stop.
Second try is:
var x = args [1];
new cron('keyTask' , '* * * * * *', job(doTheThing(x)) ,
{
start: false,
timeZone:"Europe/London",
});
Where job() is a function defined beforehand that contains the actual job and DoTheThing() is something that gives true or false depending on what the user is saying in input.
This executes the scheduled job once and then stops.
Somehow I suspect that the issue here is related to the fact that I'm defining function externally, while in most examples I saw the function is always written within the cron().
So, I'm out of ideas: thanks in advance for any help!
I want to have a cron job/scheduler that will run every 30 minutes after an onCreate event occurs in Firestore. The cron job should trigger a cloud function that picks the documents created in the last 30 minutes-validates them against a json schema-and saves them in another collection.How do I achieve this,programmatically writing such a scheduler?
What would also be fail-safe mechanism and some sort of queuing/tracking the documents created before the cron job runs to push them to another collection.
Building a queue with Firestore is simple and fits perfectly for your use-case. The idea is to write tasks to a queue collection with a due date that will then be processed when being due.
Here's an example.
Whenever your initial onCreate event for your collection occurs, write a document with the following data to a tasks collection:
duedate: new Date() + 30 minutes
type: 'yourjob'
status: 'scheduled'
data: '...' // <-- put whatever data here you need to know when processing the task
Have a worker pick up available work regularly - e.g. every minute depending on your needs
// Define what happens on what task type
const workers: Workers = {
yourjob: (data) => db.collection('xyz').add({ foo: data }),
}
// The following needs to be scheduled
export const checkQueue = functions.https.onRequest(async (req, res) => {
// Consistent timestamp
const now = admin.firestore.Timestamp.now();
// Check which tasks are due
const query = db.collection('tasks').where('duedate', '<=', new Date()).where('status', '==', 'scheduled');
const tasks = await query.get();
// Process tasks and mark it in queue as done
tasks.forEach(snapshot => {
const { type, data } = snapshot.data();
console.info('Executing job for task ' + JSON.stringify(type) + ' with data ' + JSON.stringify(data));
const job = workers[type](data)
// Update task doc with status or error
.then(() => snapshot.ref.update({ status: 'complete' }))
.catch((err) => {
console.error('Error when executing worker', err);
return snapshot.ref.update({ status: 'error' });
});
jobs.push(job);
});
return Promise.all(jobs).then(() => {
res.send('ok');
return true;
}).catch((onError) => {
console.error('Error', onError);
});
});
You have different options to trigger the checking of the queue if there is a task that is due:
Using a http callable function as in the example above. This requires you to perform a http call to this function regularly so it executes and checks if there is a task to be done. Depending on your needs you could do it from an own server or use a service like cron-job.org to perform the calls. Note that the HTTP callable function will be available publicly and potentially, others could also call it. However, if you make your check code idempotent, it shouldn't be an issue.
Use the Firebase "internal" cron option that uses Cloud Scheduler internally. Using that you can directly trigger the queue checking:
export scheduledFunctionCrontab =
functions.pubsub.schedule('* * * * *').onRun((context) => {
console.log('This will be run every minute!');
// Include code from checkQueue here from above
});
Using such a queue also makes your system more robust - if something goes wrong in between, you will not loose tasks that would somehow only exist in memory but as long as they are not marked as processed, a fixed worker will pick them up and reprocess them. This of course depends on your implementation.
You can trigger a cloud function on the Firestore Create event which will schedule the Cloud Task after 30 minutes. This will have queuing and retrying mechanism.
An easy way is that you could add a created field with a timestamp, and then have a scheduled function run at a predefined period (say, once a minute) and execute certain code for all records where created >= NOW - 31 mins AND created <= NOW - 30 mins (pseudocode). If your time precision requirements are not extremely high, that should work for most cases.
If this doesn't suit your needs, you can add a Cloud Task (Google Cloud product). The details are specified in this good article.
I have written a NodeJS app that calls an API and posts to an endpoint only on weekdays at a specific time.
I could setup a cron job to run the app at the specified time but I'd prefer to run it with node index.js and have it run constantly, doing nothing until it's the right day and time and then going back to "sleep" until the following day.
How do I achieve that? I tried with a while loop:
while (true) {
myApp.run();
}
Obviously that didn't go too well.
What's the right way to do it? Should I rewrite my modules to use events, so that I emit one when it's time and there is a listener that reacts to it?
--edit: To be more specific, I would like it to run in a similar way to an app that has a webserver in it. When you start the app, it's running and waiting for connections; it doesn't exit when the request & connection end, it stays running waiting for more requests & connections.
--edit 2: The reason I'd rather not use a cron is because the days and time to run on are configurable in a config.json file that the app parses. I'd rather avoid messing with cron and just change the schedule by editing the config.json file.
--edit 3: I'd like to code this myself and not use a module. My brain always hurts when trying to write an app that would run forever and I'd like to understand how it's done rather than using a module.
--edit 4: Here is what I ended up using:
function doStuff() {
// code to run
};
function run() {
setInterval(doStuff, 30000);
};
run();
well you could always use a simple setInterval:
function execute(){
//code to execute
}
setInterval(execute,<calculate time interval in milliseconds you want to execute after>);
this is a very bare-bones basic way to do it. Hope this helps.
You can use cron module. And for time and data that you can load from your config file, without hard coding it in the code.
Example:
var CronJob = require('cron').CronJob;
var job = new CronJob({
cronTime: '00 30 11 * * 1-5',
onTick: function() {
/*
* Runs every weekday (Monday through Friday)
* at 11:30:00 AM. It does not run on Saturday
* or Sunday.
*/
},
start: false,
timeZone: 'America/Los_Angeles'
});
job.start();
So here is how I solved it, as #AJS suggested:
function myApp() {
while (!isItTimeYet) {
setInterval(myApp, 59000);
}
}
It was simpler than I thought and hopefully it doesn't leak memory.
--edit: This is wrong and leaks memory because each loop makes it one level deeper. This is the way:
function doStuff() {
// code to run
};
function run() {
setInterval(doStuff, 30000);
};
run();
You could try this tool https://www.npmjs.com/package/forever. It's pretty simple to use:
Just run your script with forever start app.js,
NPM to the rescue ;)
node-schedule
Adding to the setInterval() approach, using setTimeout().
const sleep = (milliseconds) => {
return new Promise((resolve, reject) => {
setTimeout(() => resolve(), millisecond)
})
}
const doStuff = () => {}
const main = async () => {
while(true) {
doStuff()
await sleep(5000)
}
}
(async () => {
await main()
})()
Note: If doStuff() has promises and async code then you would need to take care of re-entry related issues.
Using Kue, how do I schedule a job to be executed once every Thursday? The Kue readme mentions that I can delay a Job, but what about repeatedly executing the Job at a specific time?
I can do what I want with a cron job, but I like Kue's features.
What I want is to process a Job once anytime on Thursday, but only once.
I had a similar question and I basically came up with the following. If anyone else has a different solution I would love to see some other ideas.
var jobQueue = kue.createQueue();
// Define job processor
jobQueue.process('thursday-jobs', function (job, done) {
var milisecondsTillThurs = // TODO: Get the time until next thursday. For this I used moment.js
// do this job again next Thursday
jobQueue.create('thursday-jobs').delay(milisecondsTillThurs).save();
// For Example purpose this job waits then calls done
setTimeout(function () {
done();
}, 10000);
});
// Use some initialization code to check if the job exists yet, and create it otherwise
kue.Job.rangeByType('thursday-jobs','delayed', 0, 10, '', function (err, jobs) {
if (err) {return handleErr(err);}
if (!jobs.length) {
jobQueue.create('thursday-jobs').save();
}
// Start checking for delayed jobs. This defaults to checking every 5 seconds
jobQueue.promote();
});
Kue has minimal documentation, but the source is well commented and easy to read
Take a look at kue-scheduler. I'm pretty sure that you should be able to do something like this:
var kue = require('kue-scheduler');
var Queue = kue.createQueue();
//create a job instance
var job = Queue
.createJob('every', data)
.attempts(3)
.backoff(backoff)
.priority('normal');
//schedule it to run every Thursday at 00:00:00
var thursday = '0 0 0 * * 4';
Queue.every(thursday, job);
//somewhere process your scheduled jobs
Queue.process('every', function(job, done) {
...
done();
});
kue-scheduler docs: https://github.com/lykmapipo/kue-scheduler;
link in their docs to cron stuff: https://github.com/kelektiv/node-cron;