I know there are many questions relative to this, but I can't find exactly what I am looking for..
I am creating an iOS Rideshare app and am utilizing the Google Distance Matrix. What I am looking to do is taking the current date and set it to midnight. For example: currentDate = 12/6/2018 12:00:00.
I want to take this currentDate value and convert is to Epoch and set it up as a baseEpoch value. This way, I can take the user's time input, and add the difference to get the date/time they entered in Epoch form.
I've tried solutions such as:
function convertToEpoch(time)
{
var sep = time.split(':');
var seconds = (+sep[0]) * 60 * 60 + (+sep[1]) * 60 + (+sep[2]);
return seconds;
}
function currentDateAsEpoch(time) {
var time = new Date();
time.setHours(0,0,0,0);
convertToEpoch(time);
}
const baseEpoch = currentDateAsEpoch();
But am getting the error: TypeError: time.split is not a function
I want the baseEpoch to be set as the current date so Google Distance Matrix doesn't return the departure_time error saying time can only be equal to or in the future.
Thank you in advance for your help!
You can use momentjs package for nodejs. This package helps you in dealing date and time effortlessly. You may need to dig more into the docs for better understanding of the moment module(Documentation is simple to understand).
Here is some snippet from momentjs docs.
moment().unix();
//moment#unix outputs a Unix timestamp (the number of seconds since the Unix Epoch).
moment(1318874398806).unix(); // 1318874398
Also using a library like moment which is well tested is better than writing your own functions to handle date and time for following reasons.
The code would be very well tested due to large number of people using them in a day to day basis.
No need to waste your time in reinventing the wheel.
Additional functionalities for future development work.(Like formatting and other date operations)
Easy to understand and implement even for new team members (due very well written documents and good support due to large number of developers using the library)
Related
So I have a NodeJS backend and using MongoDB as the DB. I want to run a certain function every time the current time equals a certain value of time as mentioned in the DB object.
For example:-
I have a record in the DB with the timestamp of 1623263299944, and there is a certain function named cancelIntent which performs some kind of task, now I want to call this function when the current time equals the mentioned time in the DB record.
How can I achieve this?
Any help is highly appreciated!
A quick google search came up with this blog post, for this library, which looks like what you're looking for
Here's another question that looks like it gives you what you're looking for
It looks like you just need to calculate how many days, hours, minuets, etc. in the future you want to schedule your task, and then input that into the agenda.schedule function
Alternatively you could use node-schedule, which has the benefit of allowing you to input a date directly, like this:
const job = schedule.scheduleJob(date, function(){
/* your code */
})
I quickly tested this and it seems to work without issues
I'm trying to set a maxWallClockTime of 72 hours using the ISO 8601 Duration format. The documentation for this property is useless, so I'm basing my guess on using the 8601 format on that being the way to set the same property at the Batch Job level when using the CLI. My constraints object is as follows:
const taskConstraints = {
maxWallClockTime: 'P3D' //ISO 8601 Duration Format e.g. P3Y6M4DT12H30M5S represents a duration of three years, six months, four days, twelve hours, thirty minutes, and five seconds.
};
However, this results in the following error:
task.constraints.maxWallClockTime must be a TimeSpan/Duration.
I cannot find any examples that set this property and use the Javascript API, any pointers to better documentation or example code would be greatly appreciated.
Agreed the docs are lacking here. I haven't tested this out locally yet, but from looking at the code I believe the answer depends on whether you are using the older Node.js-specific azure-batch package or the newer #azure/batch which also runs in web browsers.
For the "azure-batch" package, it looks like it takes a Moment.js duration object. Here's the related JSDoc string:
* #property {moment.duration} [maxWallClockTime] The maximum elapsed time
* that the Task may run, measured from the time the Task starts. If the Task
* does not complete within the time limit, the Batch service terminates it.
* If this is not specified, there is no time limit on how long the Task may
* run.
For the newer "#azure/batch" package, it should take an ISO-8601 duration string. If you're using that package then the value you're trying to use looks right to me, and maybe it's a bug (I'd have to try to repro it).
From the Google Cloud Firestore documentation:
https://cloud.google.com/nodejs/docs/reference/firestore/0.15.x/Timestamp#toDate
Timestamp
CLASS
A Timestamp represents a point in time independent of
any time zone or calendar, represented as seconds and fractions of
seconds at nanosecond resolution in UTC Epoch time. It is encoded
using the Proleptic Gregorian Calendar which extends the Gregorian
calendar backwards to year one. It is encoded assuming all minutes are
60 seconds long, i.e. leap seconds are "smeared" so that no leap
second table is needed for interpretation. Range is from
0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z.
Bold text is my emphasis
What exactly does it mean by leap seconds are "smeared"?
In practice, day to day, let's say storing a created Timestamp in Firestore, and using it to order records whilst querying,
let querySnap = await colRef.orderBy('created', 'asc').limit(10).get();
do I need to consider it?
Read Google's documentation about time smearing:
Since 2008, instead of applying leap seconds to our servers using
clock steps, we have "smeared" the extra second across the hours
before and after each leap. The leap smear applies to all Google
services, including all our APIs.
You and your users are highly unlikely to notice this effect, and it removes the need to write special code to handle sudden shifts in time that would normally be required to account for a leap second.
I have to limit my API request to 10 calls per minute, how can I modify the for loops to accomplish this?
I am trying to add in time.sleep(8) in the for observation loop without any luck... Any ideas?
import arrow # learn more: https://python.org/pypi/arrow
from WunderWeather import weather # learn more: https://python.org/pypi/WunderWeather
import time
api_key = ''
extractor = weather.Extract(api_key)
zip = '53711'
# get 20170101 00:00
begin_date = arrow.get("2017","YYYY")
# get 20171231 23:00
end_date = arrow.get("2018","YYYY").shift(hours=-1)
for date in arrow.Arrow.range('hour',begin_date,end_date):
# get date object for feature
# http://wunderweather.readthedocs.io/en/latest/WunderWeather.html#WunderWeather.weather.Extract.date
date_weather = extractor.date(zip,date.format('YYYYMMDD'))
# use shortcut to get observations and data
# http://wunderweather.readthedocs.io/en/latest/WunderWeather.html#WunderWeather.date.Observation
for observation in date_weather.observations:
time.sleep(8)
print("Date:",observation.date_pretty)
print("Temp:",observation.temp_f)
A possible explanation of why you are still exceeding the API limit might have to do with the line on which you are adding the time wait. If the API response you are getting contains no observations, the inner loop won't execute. So first I would try to move the time wait in the outer loop right after the API call.
You might also consider using something like loopingCall from twisted to schedule your task to run every X seconds
http://twistedmatrix.com/documents/9.0.0/core/howto/time.html
Depending on how realtime you want your data or you can afford to be a day behind, you could get all observations for a date in the past which would be one API call to retrieve data for a day(or it could be an end of day summary for the current day's observations).
Alternatively, if you're trying to get the current weather every x minutes or so (under the limit)
I'd use some sort of loop with a timer (or possibly twisted which seems to abstract the "loop") but make a call to one of the following (depending on what you're looking for). Your current code is looking for dates in the past but these other endpoints are for the current day.
You don't want the timer in the observations loop since, as mentioned above, there might be none.
http://wunderweather.readthedocs.io/en/latest/WunderWeather.html#WunderWeather.weather.Extract.hourly_daycast
http://wunderweather.readthedocs.io/en/latest/WunderWeather.html#WunderWeather.weather.Extract.today_now
which can be called similar to the following examples
http://wunderweather.readthedocs.io/en/latest/index.html#additional-examples
I would like to process/update every document in a Mongodb collection periodically (every 5 mins or so) and save the results back to the DB. The update function requires actual code to execute on each document (as far as I know) because it needs to perform computations such as taking the difference in timestamps and taking exponents with Math.pow, which the standard MongoDB update operators do not cover.
What is the best way to do this in NodeJS?
Full context: I am trying to implement the Hacker News ranking algorithm, which is time-dependent. The discussion I've seen around this involves using a separate thread/process to periodically update the scores on documents.
without wasting back and forth investigation it seems you have fields that i will call points, time of initial creation created_date and, then the ycombinator result of (p - 1) / (t + 2)^1.5
the easiest is to write a very simple 3 liner mongo shell script.
db.ycombinator.find().forEach(function(doc) {
var diff = ISODate() - doc.created_date; // subtract date using some form of date ISODate is available in mongo shell
var hours = diff.tomagicalhours(); // some regulr javascript
var result = (doc.points - 1) / Math.pow((hours + 2), 1.5); // perform yc algo
db.ycombinator.update({"_id":doc._id}, {$set:{"result": result} }); // write back into same collection and field, result
})
that goes into a file ycombinator_update.js and then do a 5 minute crontab.
*/5 * * * * mongo ycombinator_update.js
the performance of your reads will be noticeably slower during the writes operation contingent on the number of records in that collection.
you could assign scores based on the document timestamp at lookup time, and only keep the raw timestamps in the database. Since the score is a function of the timestamp anyway, the scoring algorithm can incorporate the exponential decay logic on the unmodified data. Scores can be converted to timestamps if to search by score.
Another option that isn't represented here is the MongoDB MapReduce or Aggregation frameworks.
Both these frameworks provide a way to iterate over all elements in a collection and output some results into a different collection. The aggregation API does not directly include the primitives we need to calculate the 1.5 exponent in the HN algorithm (no $sqrt or $pow), but there is a workaround.
I'm not certain at this point which approach is the most performant for this use case (and how it compares to the MongoDB shell script suggested by Gabe Rainbow).
I believe the next step is to run the update operations in a separate process, which is either scheduled with something like cron, or it could be kicked off via the node app itself using fork with the following logic:
On request for front page:
# when did we last update the scores for the front page?
if last_update was within last X minutes:
return list sorted by score right away
else
fork a process to sort the front page
last_update := Date.Now
return list sorted by score (either right away [stale], or after the update completes [takes a while])