I have a node.js app hosted on Heroku. I am paying the $7 a month hosting for the better plan which has me running with the next tier dynos and SSL. My problem is, I have a cronjob running in my app that runs every minute. It is VERY important this runs every minute and pretty much never misses. However, it happens to not run sometimes, and after debugging a little bit, I believe it to be that it restarts itself. like so:
So I was wondering if there is a way to schedule the app to restart instead of having it do it whenever, or if my cronjob is actually the problem and I can't do what I'm looking for. any ideas?
EDIT: here's the cronjob code:
var sendTexts = new CronJob('*/1 * * * *', function() {
// code that sends Texts if event is true
}, null, true)
and it should run every 1 minute. it does locally when my server is up, but again the issue seems to be with restarting dynos
Dynos are restarted (cycled) at least every 24 hours, if you restart manually (with heroku CLI for example) will reset the 24 hour period.
You could consider restarting you app every X hours to try to manage that, however you must consider:
Dynos can be restarted randomly by Heroku (after a platform error)
upon restart your chronojob starts immediately, so you are going to have executions before a whole minute is passed
You might want to consider an architectural change using a DB or a queue which allow you not to rely on the application always running.
In cloud-based architecture it is never a good idea to assume a single instance (container) is always available.
Related
I have a timer trigger Function App ("version": "2.0") in azure which runs every 5 min. Cron Expression Used- 0 */5 * * * *
Its working as expected but sometimes it suddenly stops running. If I disable the function app and re-enable it, its starts working again.
If you see the screenshot below, It stopped working from 2021-04-14 16:54:59.998 to 2021-04-14 20:55:12.139
Any Help will be appreciated.
There could be different reasons for this issue and I will suggest you to review the below document to troubleshoot the issue and see if you are able to find the root cause.
Timer triggered function app uses TimerTriggerAttribute. This attribute consists of the Singleton Lock feature which ensures that only a single instance of the function is running at any given time. If any process runs longer than the scheduled timer, the new incoming process waits for the older process to finish and then uses the same instance. If you are using the same storage account across different timer trigger functions then this could be one of the reasons as mentioned here.
The other reason could be a restart and I will suggest you to check the Web App Restart detection section.
https://github.com/Azure/azure-functions-host/wiki/Investigating-and-reporting-issues-with-timer-triggered-functions-not-firing
https://github.com/Azure/azure-webjobs-sdk-extensions/wiki/TimerTrigger#troubleshooting
This question actually means a couple of things.
First of all I want to ask what exactly happens when a dyno sleeps?
If I have global variables stored in an array in my bot, does it get
wiped when it sleeps (so that means that I have to actually save
everything to external files)? - Basically does my memory data cleared
when it sleeps, when it wokes up my bot won't be having that data?
Secondly, for the 550 free dyno hours, can I schedule a sleeping
schedule (e.g., 01:00 - 7:00 am) or is it not a daily limit (18h/day)
but a monthly limit (so a 24/7 uses up the hours until I have 0 for
the rest of the month)?
Adding on to what #Beppe C said in his answer, in order to understand dyno sleeping, you need to understand the difference between web and worker dynos.
Web dynos allow you to show a webpage, some file content, etc. This dyno goes to sleep periodically (after 30 minutes of inactivity). This is where you're getting the phrase dyno sleeping. If you insist on using a web dyno (as in you need to show content), the best way to schedule it is by using the Heruko Scheduler add-on. However, I don't recommend this, as it requires you to enter your credit card information. Otherwise, you could use a cron job service.
Worker dynos are different. As long as you don't use up your 550 hours in a given month (the hour limit is monthly), they will stay running without sleeping. With a worker dyno, any global variables will be saved for the duration of the time that the process stays running, unless you stop the process. The downside is that they work in the background, meaning that you can't show/display web content with them. If you need to show web content, stick with a web dyno.
To start a worker process for an app, scale up the app like so:
# scale up
heroku ps:scale worker=1 -a {appname}
Read more documentation about worker dynos here: https://devcenter.heroku.com/articles/background-jobs-queueing
My advice? If you don't need to show web content, use a worker dyno that periodically shuts off at a given time every day. Since you seem to be using node.js, maybe use setInterval to check the time?
A sleeping dyno is when the virtual node shuts down stopping the application which runs on it. The application memory is cleared (yes, all your variables and arrays) and it cannot process any request until the dyno restarts.
Any data which needs to survive should be persisted to an external storage (external file system, database).
A Web dyno goes to sleep after 30 min inactivity (ie no http requests have arrived), you cannot schedule this.
A Worker does not go to sleep and it runs until your free quota has run out.
You can always scale down (shutdown) and scale up (restart) using the command line.
# scale down
heroku ps:scale web=0 -a {appname}
I've changed a long running process on an Node/Express route to use Bull/Redis.
I've pretty much copied this tutorial on Heroku docs.
The gist of that tutorial is: The Express route schedules the Job, immediately returns 200 to the client, and browser long polls for the job status (a ping route on Express). When the client gets a completed status, it displays it in the UI. The Worker is a separate file and is run with an addtional yarn run worker.js.
Notice the end where it recommends using Throng for clustering your workers.
I'm using this Bull Dashboard to monitor jobs/queues. This dashboard shows the available Workers, and their status (Idle when not running/ Not in Idle when running).
I've got the MVP working, but the operation is super slow. The average time to complete is 1 minute 30 second. Whereas before (before adding Bull) it was seconds to complete.
Another strange it seems to take at least 30 seconds for a Workers status to change from Idle to not Idle. Seems that a lot of the latency is waiting for the worker.
Being that the new operation is a separate file (worker.js) and throng is enabling clustering, I was expecting this operation to be super fast, but it is just the opposite.
Does anyone have an experience with this? Or pointers to help figure out what is causing this to be so slow?
I am using cron in node.js to schedule a function that sends text messages at user-determined times. It works on my local server, but when i deploy to heroku the functions never get called.
I'll elaborate a little bit more on what #rsp said above, so that if anyone else finds this question they'll understand why using Heroku Scheduler is the correct answer here.
When you're running software on Heroku, what happens is that Heroku will take your project, and run it on a random dyno (server) in the Heroku collection of servers on Amazon.
For a number of reasons (including to help distribute application load evenly across a large number of Amazon servers), Heroku will periodically move your dyno from one Amazon server to another. This happens many times per day, automatically, behind the scenes.
This means that your application code will be periodically restarting all the time when running on Heroku.
Now -- this isn't a bad thing from an end-user perspective, because while your application code is restarting, Heroku will queue up incoming requests, then just send them to your application once it's been successfully restarted on a new host. So to the end user, this behavior is 100% transparent.
What's important to know here, though, is that since your application code may randomly restart, you shouldn't use it to do things like run long tasks that take a while to finish, or queue up jobs to be executed in the future (what the cron module does).
Instead: Heroku created a free scheduler addon that you can use to basically say "Hey Heroku, run this Node script every minute|hour|etc."
The scheduler addon Heroku provides will reliably execute your cron task because Heroku keeps track of that timing stuff separately (outside of your application logic).
Hopefully this helps!
I'm using a cron job on heroku with node. Here is my top level stuff
var CronJob = require('cron').CronJob;
var dailyJob = new CronJob({
cronTime: '0 0 0 * * *',
onTick: function () {
// Do daily function
console.log('I get called 1 time a day.');
},
start: false
});
dailyJob.start();
On Heroku you may need to use the Heroku Scheduler.
See: https://devcenter.heroku.com/articles/scheduler
So, I need to run my node.js app on heroku, it works very well, but when my app crashes, i need something to restart it, so i added forever to package.json, and created a file named forever.js with this:
var forever = require('forever');
var child = new (forever.Monitor)('web.js', {
max: 3,
silent: false,
options: []
});
//child.on('exit', this.callback);
child.start();
forever.startServer(child);
on my Procfile (that heroku uses to know what to start) i put:
web: node forever.js
alright! Now everytime my app crashes it auto restarts, but, from time to time (almost every 1 hour), heroku starts throwing H99 - Platform error, and about this error, they say:
Unlike all of the other errors which will require action from you to correct, this one does not require action from you. Try again in a minute, or check the status site.
But I just manually restart my app and the error goes away, if I don't do that, it may take hours to go away by itself.
Can anyone help me here? Maybe this is a forever problem? A heroku issue?
This is an issue with free Heroku accounts: Heroku automatically kills unpaid apps after 1 hour of inactivity, and then spins them back up the next time a request comes in. (As mentioned below, this does not apply to paid accounts. If you scale up to two servers and pay for the second one, you get two always-on servers.) - https://devcenter.heroku.com/articles/dynos#dyno-sleeping
This behavior is probably not playing nicely with forever. To confirm this, run heroku logs and look for the lines "Idling" and " Stopping process with SIGTERM" and then see what comes next.
Instead of using forever, you might want to try the using the Cluster API and automatically create a new child each time one dies. http://nodejs.org/api/cluster.html#cluster_cluster is a good example, you'd just put your code into the else block.
The upshot is that your app is now much more stable, plus it gets to use all of the available CPU cores (4 in my experience).
The downside is that you cannot store any state in memory. If you need to store sessions or something along those lines, try out the free Redis To Go addon (heroku addons:add redistogo).
Here's an example that's currently running on heroku using cluster and Redis To Go: https://github.com/nfriedly/node-unblocker
UPDATE: Heroku has recently made some major changes to how free apps work, and the big one is they can only be online for a maximum of 18 hours per day, making it effectively unusable as a "real" web server. Details at https://blog.heroku.com/archives/2015/5/7/heroku-free-dynos
UPDATE 2: They changed it again. Now, if you verify your ID, you can run 1 free dyno constantly: https://blog.heroku.com/announcing_heroku_free_ssl_beta_and_flexible_dyno_hours#flexible-free-dyno-hours