I have a function that checks every minute if it is 7:30am, and if it is time, it sends me a text message.
But since I upload to heroku and my time zone is different, I have to adjust the time ahead to look like this:
setInterval(function() {
let date = new Date();
if(date.getHours() == 11 && date.getMinutes() == 30){
sendDaily()
}
}, 60000)
But when I upload to heroku, I get alerts on my phone at random times. I have also tried using the schedule package that helps with this, but the same error occurs, the function does not run when supposed to. I have also tried changing the timezone on heroku but it doesnt help because when I enter my city/location it still gives the same default time zone time.
Heroku is built on AWS and create containers on any center it deems fit. Which means your application can be running anywhere in the world as shown on this AWS infrastructure map. With that being said using new Date on a server will have a very different outcome based on which server it is created in, which you have no control over.
How do you solve this problem?
You need to decouple the dependency of new Date on the OS.
Options
You can use an API that returns a standard time of day which would create a single source of truth for all AWS containers no matter where they are. Here is an Example of this.
You can manually set the Time Zone assuming all the AWS OS clocks are in sync. You can do this with moment and moment timezone
Using a dedicated server that is not created randomly throughout the world. Like any tradeoffs, this creates its own issues if you are trying to scale but it is an option.
Related
I have built a function that involves pulling a timestamp for Firebase Firestore for data analytic purposes.
When I run the logic locally using Node.js the timestamps already reflect the correct time at the point of them being recorded.
However, when I deploy the same logic to a Firebase function I have to offset the DST depending on when the timestamp was taken in order for it to display the correct time that was recorded.
Why is this happening?
I am trying to deploy a node application that will execute code periodically (I was hoping to execute once every 1-5 minutes). This code will make several API calls to different platforms and route some lightweight json data (mostly text and numbers).
I was hoping to deploy this project using either Heroku, Amazon EC2, or Amazon Lambda and would like to stay within their free tiers and I am not sure how my code will interact with their servers.
If I deploy this code to run using a setTimeout, does this mean that I am requesting the servers to always be "on" and I will essentially be charged for 100% uptime? I could see this being an issue as Heroku only provides 550 free hours per month and Amazon EC2 provides 750 free hours per month.
The code is pretty lightweight and should only be running momentarily so it would be a shame to have a server running 24/7 for only a few seconds of actual usage per day. How can I best deploy this code to run periodically? Is there another service that I can use here?
Also, below is the barebones of what my code would look like. Please let me know if there is something glaringly obvious that I am missing that could be an issue.
const axios = require('axios');
const asana = require('asana');
const { Dropbox } = require('dropbox');
function apiCallOne() {
...
}
function apiCallTwo() {
...
}
function runMyCode() {
apiCallOne();
apiCallTwo();
...
}
setTimeout(runMyCode, 1000);
With EC2 you will definitely get charged by uptime of the resource, whether you are using it or not. Although 750 free hours p/month is basically 1 free tier instance p/month.
Now I'd recommend Lambda as it seems a pretty simple code. And with Lambda you have 1M free requests per month and 400,000 GB-seconds of compute time per month.. But Lambda by itself can't run on schedules, you will need to use AWS CloudWatch Events for that.
I am serving my users with data fetched from an external API. Now, I don't know when this API will have new data, how would be the best approach to do that using Node, for example?
I have tried setInterval's and node-schedule to do that and got it working, but isn't it expensive for the CPU? For example, over a day I would hit this endpoint to check for new data every minute, but it could have new data every five minutes or more.
The thing is, this external API isn't ran by me. Would the only way to check for updates hitting it every minute? Is there any module that can do that in Node or any approach that fits better?
Use case 1 : Call a weather API for every city of the country and just save data to my db when it is going to rain in a given city.
Use case 2 : Send notification to the user when a given Philips Hue lamp is turned on at the time it is turned on without having to hit the endpoint to check if it is on or not.
I appreciate the time to discuss this.
If this external API has no means of notifying you when there's new data, then the only thing you can do is to "poll" it to check for new data.
You will have to decide what an "efficient design" for polling is in your specific application and given the type of data and the needs of the client (what is an acceptable latency for new data).
You also need to be sure that your service is not violating any terms of service with your polling scheme or running afoul of rate limiting that may deny you access to the server if you use it "too much".
Would the only way to check for updates hitting it every minute?
Unless the API offers some notification feature, there is no other scheme other than polling at some interval. Polling every minute is fairly quick. Do your clients really need information that is less than a minute old? Or would it really make no difference if the information was as much as 5 minutes old.
For example, in your example of weather, a client wouldn't really need temperature updates more often than probably every 10-15 minutes.
Is there any module that can do that in Node or any approach that fits better?
No. Not really. You'll probably just use some sort of timer (either repeated setTimeout() or setInterval() in a node.js app to repeatedly carry out your API operations.
Use case: Call a weather API for every city of the country and just save data to my db when it is going to rain in a given city.
Trying to pre-save every possible piece of data from an external API is probably a losing proposition. You're essentially trying to "scrape" all the data from the external API. That is likely against the terms of service and will likely also run afoul of rate limits. And, it's just not very practical.
Instead, you will probably want to fetch data upon demand (when a client requests data for Phoenix, then, and only then, do you start collecting data for Phoenix) and then once a demand for a certain type of data (temperatures in a particular city) is established, then you might want to pre-cache that data more regularly so you can notify clients of changes. If, after awhile, no clients are asking for data from Phoenix, you stop requesting updates for Phoenix any more until a client establishes demand again.
I have tried setInterval's and node-schedule to do that and got it working, but isn't it expensive for the CPU? For example, over a day I would hit this endpoint to check for new data every minute, but it could have new data every five minutes or more.
Making a remote network request is not a CPU intensive operation, even if you're doing it every minute. node.js uses non-blocking networking so most of the time during a network request, node.js isn't doing anything and isn't using the CPU at all. The only time the CPU would be briefly used is when you first send the API request and then when you receive back the result from the API call and need to process it.
Whether you really need to "poll" every minute depends upon the data and the needs of the client. I'd ask yourself if your app will work just fine if you check for new data every 5 minutes.
The method I would use to update would be contained outside of the code in a scheduled batch/powershell/bash file. In windows you can schedule tasks based upon time of day or duration since last run, so what you could do is run a simple command that will kill your application for five minutes, run npm update, and then restart your application before closing the shell.
That way you're staying out of your API and keeping code to a minimum, and if your code is inside that Node package in the update, it'll be there and ready once you make serious application changes or you need to take the server down for maintenance and updates to the low-level code.
This is a light-weight solution for you and it's a method I've used once or twice at my workplace. There are lots of options out there, and if this isn't what you're looking for I can keep looking out for you.
I have made a web real time application that connected to Node.js server through a websocket. In my website I can turn on/off an LED connected to Arduino Uno.
What I want to do is, I want my website have capability to turn on/off led at certain date and time dynamically. What I mean 'dynamically' is I can add new or remove current schedule task.
I have been trying using node-schedule, cron, but it's just a static schedule task. I can't change or add new task.
Use a db / file. You can store the dates in a json and then edit it as per your convenience. Use node-cron to create events of what you wanna do from the data. Create function that removes entry from json when you want to and it also remove it from the upcoming tasks by task.destroy() method of node-cron.
https://github.com/kdichev/Green-Systems/blob/development/PumpController.js
check what I have done with my pump. on line 19 I have an array of times that will run the pump according to the entries given.
So, I need to run my node.js app on heroku, it works very well, but when my app crashes, i need something to restart it, so i added forever to package.json, and created a file named forever.js with this:
var forever = require('forever');
var child = new (forever.Monitor)('web.js', {
max: 3,
silent: false,
options: []
});
//child.on('exit', this.callback);
child.start();
forever.startServer(child);
on my Procfile (that heroku uses to know what to start) i put:
web: node forever.js
alright! Now everytime my app crashes it auto restarts, but, from time to time (almost every 1 hour), heroku starts throwing H99 - Platform error, and about this error, they say:
Unlike all of the other errors which will require action from you to correct, this one does not require action from you. Try again in a minute, or check the status site.
But I just manually restart my app and the error goes away, if I don't do that, it may take hours to go away by itself.
Can anyone help me here? Maybe this is a forever problem? A heroku issue?
This is an issue with free Heroku accounts: Heroku automatically kills unpaid apps after 1 hour of inactivity, and then spins them back up the next time a request comes in. (As mentioned below, this does not apply to paid accounts. If you scale up to two servers and pay for the second one, you get two always-on servers.) - https://devcenter.heroku.com/articles/dynos#dyno-sleeping
This behavior is probably not playing nicely with forever. To confirm this, run heroku logs and look for the lines "Idling" and " Stopping process with SIGTERM" and then see what comes next.
Instead of using forever, you might want to try the using the Cluster API and automatically create a new child each time one dies. http://nodejs.org/api/cluster.html#cluster_cluster is a good example, you'd just put your code into the else block.
The upshot is that your app is now much more stable, plus it gets to use all of the available CPU cores (4 in my experience).
The downside is that you cannot store any state in memory. If you need to store sessions or something along those lines, try out the free Redis To Go addon (heroku addons:add redistogo).
Here's an example that's currently running on heroku using cluster and Redis To Go: https://github.com/nfriedly/node-unblocker
UPDATE: Heroku has recently made some major changes to how free apps work, and the big one is they can only be online for a maximum of 18 hours per day, making it effectively unusable as a "real" web server. Details at https://blog.heroku.com/archives/2015/5/7/heroku-free-dynos
UPDATE 2: They changed it again. Now, if you verify your ID, you can run 1 free dyno constantly: https://blog.heroku.com/announcing_heroku_free_ssl_beta_and_flexible_dyno_hours#flexible-free-dyno-hours