I have a node app running on Heroku.
I want some jobs to run periodically every few seconds, in order to fetch data from an external MySQL DB to my MongoDB.
After extensive search I have found a lot of ways to do this.
My problem is I cannot fully understand the difference between cron-module and the Heroku Scheduler and the pros and cons.
Some differences I spotted:
If I use Heroku Scheduler there is a limit of 10 minutes minimum.
If I use node-cron module, I will run it in my main index.js file and it will run every few seconds if I want it to.
But how are those two methods affected when running multiple dynos?
In which case the scripts will run multiple duplicated times?
node-cron will run the function at the time specified within the instance of your app that is currently running. You can schedule these with much greater precision.
Heroku Scheduler will spin up a new dyno and run the function. Once the function finishes, the dyno will spin down (i.e. shut down). You can schedule these with less precision.
If you're using multiple dynos:
Heroku Scheduler will not run duplicate commands.
node-cron will run on each instance which it has been called (likely every instance) so there will be concurrent (duplicate) functions running.
Related
I have been looking for a time based persistent scheduler. I looked into some applications (Agenda, node-cron, node-schedule). But I couldn't find anything that satisfies my criteria.
So my applications sends out reminders to our customers based on their event timings. I am hesitating to run a regular cronjob because I have to run every 15 mins or so in this case. And for each cronjob, I have to make a database call. I am trying not to use resources unnecessarily.
In addition to that, I am already running a lot of cronjobs. But in my case, when the job is completed, I want the cron to get cancelled/finished; not live on memory until the server restart happens.
I tried using the above specified applications by setting exact timestamps (agenda, node-cron, node-schedule). But the cron lives on forever even after the job is completed, and if i restart the server, all the scheduled jobs are cron. So persistence is also an issue I am facing.
My server uses node js. If there are any other languages/tools to make this work, I am all ears.
Looking forward to your help.
I tried following this solution. But this solution is for one predefined event. In my case, the number of reminders to be sent out are dynamic and jobs are to be scheduled on the fly.
I am working on a nestjs app that makes heavy use of task scheduling using the #nestjs/schedule package that integrates with the node-cron npm lib.
At the moment, the app has been in development for over 6 months and has over 30 cron tasks running in the background simultaneously. although most of them have distinct intervals, some crons have the same interval (e.g. runs EVERY 30 SECONDS).
All cron tasks more or less follow the same behavior;
send request to external APIs to get data.
query mongo db to run some checks and update records accordingly.
some crons emit events to the client when certain condition is met by other cron tasks.
my question is:
How can I measure the performance of the node process while running all these background tasks in my local development PC? and what effect it might have on requests that comes from the client?
another point is: Is it possible to detect a memory leak before it happen?
basically I have concerns about the app performance and I want to try to prevent the problem before it happen.
Thanks.
I'm creating a project in Node & Express that allows users to schedule the server to run test scripts e.g. once every ten minutes. I looked into node-schedule which looks great however it seems that all scheduled tasks disappear if the server ever restarts Node.
Cron looks good too but it has the problem that it doesn't seem to have a way to delete scheduled tasks after they have been set up.
If you were doing this, how would you go about it? I really don't want anything that's going to be complex, just need to schedule tasks, be able to delete individual tasks, and keep tasks in the event of a server reboot.
Simplest solution is to store the configurations for Cron in a database (since it takes a string as a parameter). Load the jobs from the db every time the app starts.
I am starting with Heroku and I have a webapp that has a part that needs to run once every week (Mondays preferably). I had been reading something about workers: here and here and here... But I still have many doubts:
1) This workers, runs on background without a strict control, can´t be scheduled to run once a week. or am I wrong? If I am wrong how can I schedule it?
2) To make them work, what exactly do I need to do? Type
web: node webApp.js
worker: node worker.js
in the Procfile (where worker.js is the part of the program that needs to run only once a week). And that is all?? nothing else?? so easy??
3) And the last one... but the most important. The "squamous matter of money"... One dyno is the same as one worker, so if you have a dyno running for the web you need to buy another for the worker... no? And on the list of prices a extra dyno cost 34.5$ (27.87€). It isn´t cheap... so I want to know if I am right, is it necessary buy a dyno if you want to run a worker?
You might find that the Heroku Scheduler add-on (https://devcenter.heroku.com/articles/scheduler) is a 'good enough' low cost option. You are charged for the hours that your scheduled tasks run for so if you have a regular job that only takes a short time to run it would work out much cheaper than a continuous worker process.
Its not as flexible with regard to scheduling as other options. It can be set up to run a task at a specific time every day or hourly. So if you need to have your task run say only on Mondays then you would need to have the scheduler run daily then check the day within your worker.js and exit immediately on other days.
I'm planning to host an express app on Heroku (I'm already experimenting with a single dyno).
I want to use node-cron for maintenance tasks (doing some MongoDB updates). The question is, what's the simplest way to make sure the maintenance only runs once? All dynos would try to run the maintenance at the same time.
My current approach uses MongoDB's atomic upserts as some sort of semaphore (every dyno tries to set the flag for the current maintenance). But that's kinda ugly.
I'd like to not have a separate worker instance since it's really just a simple task that needs to be run once a day.
I think that's what the Heroku Scheduler is good for (https://devcenter.heroku.com/articles/scheduler). If the execution of your update isn't taking too long it's the way to go. You write a JS script (e.g. schedule.js) that knows how to update your MongoDB, put it in your root and schedule it using the Heroku Scheduler (it comes with a trivial frontend) to invoke it (node scheduler.js) at the time desired.