I'm planning to host an express app on Heroku (I'm already experimenting with a single dyno).
I want to use node-cron for maintenance tasks (doing some MongoDB updates). The question is, what's the simplest way to make sure the maintenance only runs once? All dynos would try to run the maintenance at the same time.
My current approach uses MongoDB's atomic upserts as some sort of semaphore (every dyno tries to set the flag for the current maintenance). But that's kinda ugly.
I'd like to not have a separate worker instance since it's really just a simple task that needs to be run once a day.
I think that's what the Heroku Scheduler is good for (https://devcenter.heroku.com/articles/scheduler). If the execution of your update isn't taking too long it's the way to go. You write a JS script (e.g. schedule.js) that knows how to update your MongoDB, put it in your root and schedule it using the Heroku Scheduler (it comes with a trivial frontend) to invoke it (node scheduler.js) at the time desired.
Related
I have been looking for a time based persistent scheduler. I looked into some applications (Agenda, node-cron, node-schedule). But I couldn't find anything that satisfies my criteria.
So my applications sends out reminders to our customers based on their event timings. I am hesitating to run a regular cronjob because I have to run every 15 mins or so in this case. And for each cronjob, I have to make a database call. I am trying not to use resources unnecessarily.
In addition to that, I am already running a lot of cronjobs. But in my case, when the job is completed, I want the cron to get cancelled/finished; not live on memory until the server restart happens.
I tried using the above specified applications by setting exact timestamps (agenda, node-cron, node-schedule). But the cron lives on forever even after the job is completed, and if i restart the server, all the scheduled jobs are cron. So persistence is also an issue I am facing.
My server uses node js. If there are any other languages/tools to make this work, I am all ears.
Looking forward to your help.
I tried following this solution. But this solution is for one predefined event. In my case, the number of reminders to be sent out are dynamic and jobs are to be scheduled on the fly.
So, I see that heroku provides the option to run a command at a specific time. Information on the scheduler here.
LINK: https://elements.heroku.com/addons/scheduler
However, if you go through the steps when setting it up, they do not provide a lot of flexibility on when you can run your code daily. For example, you can only run code at 4:00pm or 4:30pm, not 4:10pm.
How can I make it so that a node.js file would run on heroku at a specific time (like 4:10pm or 2:15pm, some time outside the options given on heroku) on a daily basis?
There appears to be no support for node.js explaining this either on their website.
Might be just a work around, but you could start the process at the nearest time slot, that is before your desired time, let it run and wait passively until your desired time and let it do the actual task just then.
However notice as Heroku mentions in the documentation Heroku Scheduler isn't guaranteed to run the task, even though it's very reliable. If you need something critical or have something that has to been run everyday for sure, you should probably make a separate process, which handles the scheduling.
We've added a simple Heroku add-on called Cron To Go that does exactly that - you can use Cron expressions for accuracy and schedule one-off Dynos, just like with Heroku Scheduler.
There's also a simple Node example here.
I'm creating a project in Node & Express that allows users to schedule the server to run test scripts e.g. once every ten minutes. I looked into node-schedule which looks great however it seems that all scheduled tasks disappear if the server ever restarts Node.
Cron looks good too but it has the problem that it doesn't seem to have a way to delete scheduled tasks after they have been set up.
If you were doing this, how would you go about it? I really don't want anything that's going to be complex, just need to schedule tasks, be able to delete individual tasks, and keep tasks in the event of a server reboot.
Simplest solution is to store the configurations for Cron in a database (since it takes a string as a parameter). Load the jobs from the db every time the app starts.
Our node app gets quite big and one job takes quite some time to execute. We run this job with a cronjob, but by calling the URL. Now Heroku has problems with this, because the job takes more than 30 seconds to finish. So we receive a time-out and after that it tries to execute it immediately again, and again, till our Memory quota is about 300% and the app crashes.
Now I want to fix this. Locally we don't have any problems running this script at all. It takes about a minute (for now, but in the future if we have more users it may take more time) to finish and memory stays stable.
Now running this script on the background should fix the problem according https://devcenter.heroku.com/articles/request-timeout#debugging-request-timeouts
Overe here https://devcenter.heroku.com/articles/asynchronous-web-worker-model-using-rabbitmq-in-node#getting-started I read about JackRabbit. But it seems like it's used for systems like RabbitMQ https://github.com/hunterloftis/jackrabbit
So my question: anyone who has experience with background tasks in node? Can and should I use JackRabbit for my background tasks, or are there better solutions? My background task just contains a very complex ExpressJS task, which takes some time to execute so....
I'm the Node.js platform owner at Heroku (and I actually wrote the web worker article you referenced).
Your use case sounds like it may fit the scheduler very well:
https://devcenter.heroku.com/articles/scheduler
It's a great replacement for cron-type jobs.
I have a node app running on Heroku.
I want some jobs to run periodically every few seconds, in order to fetch data from an external MySQL DB to my MongoDB.
After extensive search I have found a lot of ways to do this.
My problem is I cannot fully understand the difference between cron-module and the Heroku Scheduler and the pros and cons.
Some differences I spotted:
If I use Heroku Scheduler there is a limit of 10 minutes minimum.
If I use node-cron module, I will run it in my main index.js file and it will run every few seconds if I want it to.
But how are those two methods affected when running multiple dynos?
In which case the scripts will run multiple duplicated times?
node-cron will run the function at the time specified within the instance of your app that is currently running. You can schedule these with much greater precision.
Heroku Scheduler will spin up a new dyno and run the function. Once the function finishes, the dyno will spin down (i.e. shut down). You can schedule these with less precision.
If you're using multiple dynos:
Heroku Scheduler will not run duplicate commands.
node-cron will run on each instance which it has been called (likely every instance) so there will be concurrent (duplicate) functions running.