I am starting with Heroku and I have a webapp that has a part that needs to run once every week (Mondays preferably). I had been reading something about workers: here and here and here... But I still have many doubts:
1) This workers, runs on background without a strict control, can´t be scheduled to run once a week. or am I wrong? If I am wrong how can I schedule it?
2) To make them work, what exactly do I need to do? Type
web: node webApp.js
worker: node worker.js
in the Procfile (where worker.js is the part of the program that needs to run only once a week). And that is all?? nothing else?? so easy??
3) And the last one... but the most important. The "squamous matter of money"... One dyno is the same as one worker, so if you have a dyno running for the web you need to buy another for the worker... no? And on the list of prices a extra dyno cost 34.5$ (27.87€). It isn´t cheap... so I want to know if I am right, is it necessary buy a dyno if you want to run a worker?
You might find that the Heroku Scheduler add-on (https://devcenter.heroku.com/articles/scheduler) is a 'good enough' low cost option. You are charged for the hours that your scheduled tasks run for so if you have a regular job that only takes a short time to run it would work out much cheaper than a continuous worker process.
Its not as flexible with regard to scheduling as other options. It can be set up to run a task at a specific time every day or hourly. So if you need to have your task run say only on Mondays then you would need to have the scheduler run daily then check the day within your worker.js and exit immediately on other days.
Related
So, I see that heroku provides the option to run a command at a specific time. Information on the scheduler here.
LINK: https://elements.heroku.com/addons/scheduler
However, if you go through the steps when setting it up, they do not provide a lot of flexibility on when you can run your code daily. For example, you can only run code at 4:00pm or 4:30pm, not 4:10pm.
How can I make it so that a node.js file would run on heroku at a specific time (like 4:10pm or 2:15pm, some time outside the options given on heroku) on a daily basis?
There appears to be no support for node.js explaining this either on their website.
Might be just a work around, but you could start the process at the nearest time slot, that is before your desired time, let it run and wait passively until your desired time and let it do the actual task just then.
However notice as Heroku mentions in the documentation Heroku Scheduler isn't guaranteed to run the task, even though it's very reliable. If you need something critical or have something that has to been run everyday for sure, you should probably make a separate process, which handles the scheduling.
We've added a simple Heroku add-on called Cron To Go that does exactly that - you can use Cron expressions for accuracy and schedule one-off Dynos, just like with Heroku Scheduler.
There's also a simple Node example here.
I'm creating a project in Node & Express that allows users to schedule the server to run test scripts e.g. once every ten minutes. I looked into node-schedule which looks great however it seems that all scheduled tasks disappear if the server ever restarts Node.
Cron looks good too but it has the problem that it doesn't seem to have a way to delete scheduled tasks after they have been set up.
If you were doing this, how would you go about it? I really don't want anything that's going to be complex, just need to schedule tasks, be able to delete individual tasks, and keep tasks in the event of a server reboot.
Simplest solution is to store the configurations for Cron in a database (since it takes a string as a parameter). Load the jobs from the db every time the app starts.
Our node app gets quite big and one job takes quite some time to execute. We run this job with a cronjob, but by calling the URL. Now Heroku has problems with this, because the job takes more than 30 seconds to finish. So we receive a time-out and after that it tries to execute it immediately again, and again, till our Memory quota is about 300% and the app crashes.
Now I want to fix this. Locally we don't have any problems running this script at all. It takes about a minute (for now, but in the future if we have more users it may take more time) to finish and memory stays stable.
Now running this script on the background should fix the problem according https://devcenter.heroku.com/articles/request-timeout#debugging-request-timeouts
Overe here https://devcenter.heroku.com/articles/asynchronous-web-worker-model-using-rabbitmq-in-node#getting-started I read about JackRabbit. But it seems like it's used for systems like RabbitMQ https://github.com/hunterloftis/jackrabbit
So my question: anyone who has experience with background tasks in node? Can and should I use JackRabbit for my background tasks, or are there better solutions? My background task just contains a very complex ExpressJS task, which takes some time to execute so....
I'm the Node.js platform owner at Heroku (and I actually wrote the web worker article you referenced).
Your use case sounds like it may fit the scheduler very well:
https://devcenter.heroku.com/articles/scheduler
It's a great replacement for cron-type jobs.
I have a node app running on Heroku.
I want some jobs to run periodically every few seconds, in order to fetch data from an external MySQL DB to my MongoDB.
After extensive search I have found a lot of ways to do this.
My problem is I cannot fully understand the difference between cron-module and the Heroku Scheduler and the pros and cons.
Some differences I spotted:
If I use Heroku Scheduler there is a limit of 10 minutes minimum.
If I use node-cron module, I will run it in my main index.js file and it will run every few seconds if I want it to.
But how are those two methods affected when running multiple dynos?
In which case the scripts will run multiple duplicated times?
node-cron will run the function at the time specified within the instance of your app that is currently running. You can schedule these with much greater precision.
Heroku Scheduler will spin up a new dyno and run the function. Once the function finishes, the dyno will spin down (i.e. shut down). You can schedule these with less precision.
If you're using multiple dynos:
Heroku Scheduler will not run duplicate commands.
node-cron will run on each instance which it has been called (likely every instance) so there will be concurrent (duplicate) functions running.
I've seen that Heroku charges $15/mo to run Delayed Job, and $3/mo to run cron tasks daily. Is it possible to skip that entirely and run my own cron tasks manually? Or are they somehow figuring out that I'm running cron tasks?
I'm not entirely sure what you mean by "run my own cron tasks manually". For cron specifically, you need access to crontab, which they can control, as they're their servers. If you have another way of doing it, it would probably be fine, but bear in mind that your app is not tied to a specific server when running under Heroku, and that the server will change between executions.
Also, unless they've changed it since last time I checked, you can run daily cron tasks for free, but hourly costs $3/mo.
EDIT: Yes, daily crons are free. See http://addons.heroku.com/.
If you install the Heroku gem on your computer, you can then run your cron tasks manually as follows:
$ heroku rake cron
(in /disk1/home/slugs/xxxxxx_aa515b2_6c4f/mnt)
Running cron at 2010/04/25 10:28:54...
This will execute the exact same code as Heroku's daily/hourly cron add-on does; that is, for this to work, your application must have a Rakefile with a cron task, for example:
desc "Runs cron maintenance tasks."
task :cron do
puts "Running cron at #{Time.now.strftime('%Y/%m/%d %H:%M:%S')}..."
# TODO: your cron code goes here
end
Now, just add the heroku rake cron command to a crontab on any Unix server of yours, or even directly to your personal computer's crontab if you're running Linux or Mac OS X, and you can be scheduling cron jobs for your Heroku application as you please and without being charged for it.
Updating the answer for 2020:
You can use Heroku Scheduler which is Heroku's own add-on that lets you schedule commands using one-off dynos (so that you only pay for the run time of your jobs). The add-on itself is free, but doesn't really allow you to use cron but rather plain frequency: every day, every hour or every 10 minutes. Also, there's no guarantee that your job will execute at the scheduled time or at all..
There are other 3rd party add-ons that can help you run one-off dynos using cron expressions for better flexibility and are more resilient than Heroku Scheduler (proper disclosure, my company is the creator of one such add-on).
You can also use custom clock process (see here for more info) which essentially means that you have one dyno or process spawn tasks that run on other dynos. This usually costs more than using the aforementioned add-ons, but you have more granular control over your processes and since you only rely on Heroku, it may be more stable.
Yes, I've successfully used a cron job on my local server which essentially runs
$ heroku rake <rake task>
at whatever intervals I've required. I've used in on both aspen and bamboo stacks.
You can also just install a gem like rufus-scheduler if you're running a rails app and setup scheduling that way. I don't know if this is bad practice for some reason, but it's what I do with my app, and it seems to work fine.
If you want to have scheduled jobs you can also use http://guardiano.getpeople.in that is a free service (for 10 jobs) for job scheduling.
You just need to setup an HTTP endpoint in your application to recieve event notifications on POST or GET and you can also set some additional params to prevent not authorized actions.
So you set a job in Guardiano that will cal http://yourapp.com/youraction and leave "minutes" blank if you want you action to run once in the future or set minutes to what you want to run your action every X minutes. In that way you only have to create your endpoint for you app and when this EP is called you execute something.
So your app can sleep and you don't need to spend money and time setting up jobs and taking care that they are working properly.
IMHO if you need something fast for an MVP or you need to setup a lot of jobs for different apps than a free service like that where you can actually outsource cronjobs is quite good.
There was aslo an Heroku Addon called Temporize to do that but I'm not sure is still alive and working