I'm using REST GET calls from a google script to build a temperature profile of my house during the day. The function triggers every 15min. last_connection (and the rest of the data) will sometimes be the same 3-4 calls in a row, other times can be different each time for several hours running, suggesting variable rates at which the thermostat sends data up to the server.
Does anyone know what governs the thermostat's connections to nest.com or if there is a way to force a connection in order to get an up to date profile?
The thermostat connects to Nest's cloud under the following circumstances:
A 'significant' even has occurred (ie the furnace turning on)
A timeout has occurred (ie a scheduled check in appointment)
A thermostat will be considered offline if it misses its check in window, there is more detail on why that would happen in Nest's Troubleshooting Offline Status in the Nest apps support article.
You can force a thermostat to come online by sending a change to it, for example changing the target temperature will necessarily force the thermostat to wake up so the new value can be set, while awake the thermostat will update the cloud service with updated information. Forcing a thermostat to wake as way to get updated data from the thermostat is not recommended as you will run into an API rate limit designed to protect the battery on the thermostat. Charging rates on thermostats are rather limited, wake it too often and it will go offline for a while, annoying the user.
Rest assured, if the ambient temperature or humidity changes by a 'significant' amount, the thermostat will wake up and update the cloud service. The thresholds of what signifies a significant amount are harder to predict as they are partially determined by charging rate. If you want to know why that can vary, Nest has filed a patent which goes into great detail.
Related
I'm a bit confused by the documentation : In order for Stripe to compute the number of units consumed during the billing cycle, you must report the customer’s usage by creating usage records
then : The usage reporting endpoint is rate-limited, so you might need to exercise caution and avoid making too many separate usage records.
So what is it saying exactly? after adding some usage for a few customers, my app will stop working? Then what should I use? line items? invoice items?
So far I've created a customer and subscribed him to a plan. How do I increment his usage without limit and risking my app to break for no apparent reason?
This just means that, if you're in danger of hitting the rate limiting, you should do something to batch up your calls to https://api.stripe.com/v1/subscription_items/{SUBSCRIPTION_ITEM_ID}/usage_records
Instead of POSTing there every time your customer's usage increases, for example, just keep track of it on your side and do one POST daily, at the end of the billing cycle, or at some other interval.
I am serving my users with data fetched from an external API. Now, I don't know when this API will have new data, how would be the best approach to do that using Node, for example?
I have tried setInterval's and node-schedule to do that and got it working, but isn't it expensive for the CPU? For example, over a day I would hit this endpoint to check for new data every minute, but it could have new data every five minutes or more.
The thing is, this external API isn't ran by me. Would the only way to check for updates hitting it every minute? Is there any module that can do that in Node or any approach that fits better?
Use case 1 : Call a weather API for every city of the country and just save data to my db when it is going to rain in a given city.
Use case 2 : Send notification to the user when a given Philips Hue lamp is turned on at the time it is turned on without having to hit the endpoint to check if it is on or not.
I appreciate the time to discuss this.
If this external API has no means of notifying you when there's new data, then the only thing you can do is to "poll" it to check for new data.
You will have to decide what an "efficient design" for polling is in your specific application and given the type of data and the needs of the client (what is an acceptable latency for new data).
You also need to be sure that your service is not violating any terms of service with your polling scheme or running afoul of rate limiting that may deny you access to the server if you use it "too much".
Would the only way to check for updates hitting it every minute?
Unless the API offers some notification feature, there is no other scheme other than polling at some interval. Polling every minute is fairly quick. Do your clients really need information that is less than a minute old? Or would it really make no difference if the information was as much as 5 minutes old.
For example, in your example of weather, a client wouldn't really need temperature updates more often than probably every 10-15 minutes.
Is there any module that can do that in Node or any approach that fits better?
No. Not really. You'll probably just use some sort of timer (either repeated setTimeout() or setInterval() in a node.js app to repeatedly carry out your API operations.
Use case: Call a weather API for every city of the country and just save data to my db when it is going to rain in a given city.
Trying to pre-save every possible piece of data from an external API is probably a losing proposition. You're essentially trying to "scrape" all the data from the external API. That is likely against the terms of service and will likely also run afoul of rate limits. And, it's just not very practical.
Instead, you will probably want to fetch data upon demand (when a client requests data for Phoenix, then, and only then, do you start collecting data for Phoenix) and then once a demand for a certain type of data (temperatures in a particular city) is established, then you might want to pre-cache that data more regularly so you can notify clients of changes. If, after awhile, no clients are asking for data from Phoenix, you stop requesting updates for Phoenix any more until a client establishes demand again.
I have tried setInterval's and node-schedule to do that and got it working, but isn't it expensive for the CPU? For example, over a day I would hit this endpoint to check for new data every minute, but it could have new data every five minutes or more.
Making a remote network request is not a CPU intensive operation, even if you're doing it every minute. node.js uses non-blocking networking so most of the time during a network request, node.js isn't doing anything and isn't using the CPU at all. The only time the CPU would be briefly used is when you first send the API request and then when you receive back the result from the API call and need to process it.
Whether you really need to "poll" every minute depends upon the data and the needs of the client. I'd ask yourself if your app will work just fine if you check for new data every 5 minutes.
The method I would use to update would be contained outside of the code in a scheduled batch/powershell/bash file. In windows you can schedule tasks based upon time of day or duration since last run, so what you could do is run a simple command that will kill your application for five minutes, run npm update, and then restart your application before closing the shell.
That way you're staying out of your API and keeping code to a minimum, and if your code is inside that Node package in the update, it'll be there and ready once you make serious application changes or you need to take the server down for maintenance and updates to the low-level code.
This is a light-weight solution for you and it's a method I've used once or twice at my workplace. There are lots of options out there, and if this isn't what you're looking for I can keep looking out for you.
I created a QnA bot using the Azure Bot service, and now I'm seeing data transfers out of my subscription of over 1 GB a day! I cannot figure out why, but since it's billable, I'd like to know why and how I can stop it.
The bot isn't being used yet, so no one is sending queries to it. I'm confused how this is happening.
Here's a screen shot of the graph for use in the last hour as well as a screen shot of the billing for the last few days showing the sudden jump in use.
Is this normal?
If you add AzureWebJobsDisableHomepage with a value of true, to the App settings, the data out will stop.
The setting itself is documented here: https://github.com/Azure/azure-webjobs-sdk-script/wiki/Configuration-Settings (although it doesn't provide an explanation for how this setting affects a bot specifically)
The reasoning behind what is happening is a little complex. Azure Functions are not normally "in memory" and available all the time. There is a small spinup time that is not ideal within a bot. So, apparently there is a job setup with consumption plan bots to ping it every 10 seconds (and by 'ping', i mean retrieve the root of the site). If you open the Log Stream, you'll see an http get request every 10 seconds. Adding AzureWebJobsDisableHomepage doesn't disable the request, but changes the status of what is returned from "OK" to "NoContent".
This will be added to the Bot Service arm template soon (so future consumption plan bots do not automatically accrue these data usages).
I am trying to get an interval function, pre-programmed on separate devices, to sync up with each other. So several mobile devices all running the same interval function in sync. At first I thought I could just use the devices internal clocks and start the functions on the 0 of the minute. I realize now that mobile clocks aren't really all that accurate and are not synced to each other. Now I need a new solution.
I'm using heroku, node.js, socket.io, and ionic if that helps at all.
Conceptually, you could do the following:
Have each client establish a time reference vs. a common server.
Send a color changing message to each client with a specific timestamp a short time in the future when the color changing effect is to be started.
When each client receives the color changing message, it looks at the scheduled timestamp a short time in the future, corrects for the time reference and schedules the event vs. its own clock.
When that time arrives at each client, each client starts playing the event.
This will be as accurate as the time reference you establish in step 1 and that's where the tricky portion is and where the accuracy of lack of accuracy is established. There's a description of one method for doing that in this post: Measuring time difference between networked devices.
Once you establish the time delta between the client clock and the reference clock, you store that delta locally and you can apply that delta to any future time directives from the server. So, if you receive a directive to carry out some operation at 12:30:05.00, but your client clock has been measured to be +12.33 seconds fast from the server reference, then you would subtract that +12.33 seconds from the scheduled time and set a timer that would fire at 12:29:52.67 on your local clock.
You handle the fact that the transit time to each client may not be the same by sending a directive for a specific time in the future. You can pick any time in the future, but it must not be a longer time into the future than the longest transit time to any client. You can also measure that from each client and report it to the server or if you have enough time to schedule in advance, you can just send the message with the directive at least several seconds ahead of time (longer than any transit time is likely to be in normal operating conditions).
I am looking at building an app that monitors the public transport buses for a major city:
I did a quick prototype using pubnub. The buses have a phone transmitting gps signals to a channel and bus users have phones subscribed to channels. I have questions:
I am planning for each bus route there is a channel. The city has 50 routes so there will be 50 routes. Does this adhere to the best practice?
Is there an api to list channels ?
I am sending a message to a channel every second. Assume, there are 50 routes with 5 buses each running 24 hours. There will be 216000000 daily messages. what will i be charged for a day?
Does your Android client open a network connection everytime a publish is call? I want to minimize the bandwith used by the phone that is transmitting the GPS signal.
Bus users may want to see location of multiples buses. I know best practice is to subscribe to one public and one private channel. What is the best way to do it?
I would appreciate if you could answer the above questions.
Full disclosure up front - I work for PubNub Customer Success so responses for pricing related questions are informational in nature only and not to be construed as a promotional. Asker specifically mentions PubNub and the information provided below is publicly available from the PubNub website.
Anant, also as an FYI StackOverflow would normally ask that each of these questions gets asked as a separate thread. Moving forward please do your best to adhere to community guidelines.
1 Every implementation will be different as far as the specific architecture and design pattern strategy though your proposed approach seems to be a sensible utilization of channel methodology. PubNub does not limit the total number of channels in use, however as a practical limitation for most mobile development frameworks subscribing to more than 50 channels simultaneously would be around the upper limit. Adding more than that and both iOS and Android will begin exhibiting performance limitations. If new bus lines are added the subscriptions can be managed to only subscribe to nearby routes, etc.
Question 1 the second with the indent. Yes that can be done with the here_now API
2 PubNub charges $1 per million messages (without SSL enabled) so based on your hypothetical your message charges would be $216 per day. That being said, there is significant room here for design pattern optimization so that busses only publish a new location whenever there is a change - repeated publishes while the bus is standing still are unnecessary. This optimization on it's own will bring the message usage figure down significantly, and there are other strategies which can be utilized to further optimize depending on your specific implementation approach. If you anticipate needing more than 1 billion messages per month, a deployment to Global Cloud would make sense so as to avail yourself of volume discount pricing not otherwise available on Go Cloud.
3 Rather than opening a new connection with every publish, PubNub keeps an active socket connection open until unsubscribed or disconnected via loss of network connection/app force close. The bandwidth utilization to keep this connection active over a period of several hours and absent any other publish/subscribe activity typically measures less than 1K depending on your configuration parameters. Android supports background threading so even when the app is not in focus the connection can remain open to facilitate data push alerts which can be used to prompt the user to bring the app back into the foreground to review any updated information.
4 This question is not clear, assuming that the bus locations are published to the public channel what would the purpose of the private channel serve? If you meant a private channel to receive alerts for the arrival of the user's selected bus, then yes that would be an appropriate implementation strategy. Please clarify if you meant something different.