Is it possible to get milliseconds from Python 3's native time library? - python-3.x

I have been trying to find a nice way to format a timestamp of the current date & time with milliseconds using Python 3's native time library.
However there's no directive for milliseconds in the standard documentation https://docs.python.org/3/library/time.html#time.strftime.
There's undocumented directives though, like %s which will give the unix timestamp. Is there any other directives like this?
Code example:
>>> import time
>>> time.strftime('%Y-%m-%d %H:%M:%S %s')
'2017-08-28 09:27:04 1503912424'
Ideally I would just want to change the trailing %s to some directive for milliseconds.
I'm using Python 3.x.
I'm fully aware that it's quite simple to get milliseconds using the native datetime library, however I'm looking for a solution using the native time library solely.

If you insist on using time only:
miliSeconds = time.time()%1*1000
time() returns accurately the time since the epoch. Since you already have the the date up to a second, you don't really care this is a time delta, since the remaining fraction is what you need to add anyway to what you have already to get the accurate date. %1 retrieves the fraction and then I convert the numbers to millisecond by multiplying by 1000.
note
Note that even though the time is always returned as a floating point
number, not all systems provide time with a better precision than 1
second. While this function normally returns non-decreasing values, it
can return a lower value than a previous call if the system clock has
been set back between the two calls.
Taken from https://docs.python.org/3/library/time.html#time.time. But this means there is no way to do what you want. You may be able to do something more robust with process_time, but that would have to be elaborate.

Related

How to speed up a CLOCK_MONOTONIC list of timestamps

I have a list of SECONDS.MICROSECONDS CLOCK_MONOTONIC timestamps like those below:
5795.944152
5795.952708
5795.952708
5795.960820
5795.960820
5795.969092
5795.969092
5795.977502
5795.977502
5795.986061
5795.986061
5795.994075
5795.994075
5796.002382
5796.002382
5796.010860
5796.010860
5796.019241
5796.019241
5796.027452
5796.027452
5796.035709
5796.035709
5796.044158
5796.044158
5796.052453
5796.052453
5796.060785
5796.060785
5796.069053
They each represent a particular action to be made.
What I need to do, in python preferably, but the programming language doesn't really matter, is to speed up the actions - something like being able to do a 2X, 3X, etc., speed increment on this list. So those values need to decrease in such a way to match the speed incrementation of ?X.
I thought of dividing each timestamp with the speed number I want, but it seems it doesn't work this way.
As described and suggested by #RobertDodier I have managed to find a quick and simple solution to my issue:
speed = 2
speedtimestamps = [(t0 + (t - t0)/speed for t in timestamps]
Just make sure to remove the first line containing the first t0 timestamp.

How to set base timestamp for relative date parsing in NLPCraft?

I'm working with nlpcraft to build a parsing system for scheduling. Users are asked when they will be doing certain activities and they can respond with relative or absolute dates, such as "tuesday and wednesday" or "not until 8/15".
While nlpcraft has very nice relative date parsing, near as I can tell it always parses dates relative to the current system time in UTC. Not only does this complicate testing (because the input is relative while the output is absolute), it means that if the server does not parse the input close to the time the user wrote it, relative dates may be parsed incorrectly. For example, if the user says "tomorrow" at 11PM on a Sunday, but the server doesn't parse it until 5AM on Monday, it might result in Tuesday instead of Monday.
I looked into NCDateEnricher where this all seems to happen and then parsing routine computes a base time as the current system time. I didn't see a way to override this with a config variable or request parameter -- am I missing something?
UTC time server on server-side allows users to easily convert times to local timezone. It's the simplest way to support different timezone users with one server.
If you aren't satisfied with nlpcart provided date NER, you can look at date/time NERS from opennlp/stanford/spacy/google, which can be simply used with nlpcraft system (https://nlpcraft.incubator.apache.org/integrations.html)

How to get accurate UTC nanoseconds time stamp in Node.js

python3 has possibility running following construct:
from datetime import datetime
print({'timestamp':str(int(datetime.utcnow().timestamp() * (10**9)))})
That would output JSON-like structore:
{'timestamp': '1567658264143923968'}
Now I want to get similar result in Node.JS 12.
I need a string
I need UTC stamp since 1970, but not since machine boot
Error no more than last 4 digits (microseconds precision or better)
So 32-bitness of int cant be a limitation. (That cant be a problem in Q3 2019, since Node 10 has bigint support).
OS also cant pose such a limitation, since WinAPI has GetSystemTimePreciseAsFileTime and *nix has clock_gettime(CLOCK_MONOTONIC) both are better than 1us
How could I get nanoseconds UTC timestamp string in NodeJS?
Date.now()+"000000"
shows that it lacks last 6 digits (millis), but the task was 4 (micros or better)
This 7yo question seems to address general timestamp question, and hrtime will get you just machine uptime, not real UTC time. results will be irrelevant with that of python.
What are other options?
UPD
Now, I can reword question a bit. From very simple perspective, Nodejs can be seen as a OS-independent runtime library. You write one js code and it will work quite same on all platforms. So it MUST bind most viable functions to js interface; timer functions are among them. As we saw, monotonic clock QueryPerformanceCounter() call is bound through process.hrtime()
Which function in node.js would lead to a clock_gettime(3) call on linux, or GetSystemTimePreciseAsFileTime() call on windows, and return result to js, with microseconds precision or better?
If the resolution you are looking for is microseconds then process.hrtime() should be enough.
const hrTime = process.hrtime()
console.log(hrTime[0] * 1000000 + hrTime[1] / 1000)
Alternatively you could use timestamp-nano
If you need an even higher precision you could build yourself a native node module: https://medium.com/the-node-js-collection/n-api-next-generation-node-js-apis-for-native-modules-169af5235b06
Or if you really like Python, you could execute a python script from Node: How to call a Python function from Node.js
For recent versions of Node, process.hrtime.bigint() seems to provide this.

Open trip planner script slower on days other than today

I'm making use of open trip planner using the jython scripting method explained here: http://docs.opentripplanner.org/en/latest/Scripting/
(specifically 'Using OTP as a library') and am using a script very similar to their example script
For testing purposes I have two csv files containing 40 locations each. The locations are inside the Netherlands and I have loaded both the dutch gtfs and map. The strange thing is that the code that calculates the public transport trip times (line 32 in the example script: res = spt.eval(colleges), using modes WALK,TRANSIT) takes longer when I specify a day other than today.
An example:
req.setDateTime(2018, 12, 8, 16, 00, 00) # today
spt.eval(my_data) # -> takes ~7 - 10 seconds
req.setDateTime(2018, 12, 7, 16, 00, 00) # yesterday
spt.eval(my_data) # -> takes ~30 - 40 seconds
When not setting req.setDateTime(), spt.eval() is even faster. Note that I ran the script on the 6th, for the 6th, as well, and it was fast then too, so it's certainly related to "today" and not specifically the 8th.
Of course my primary question is, how do I make it fast for days other than today? (my main interest is actually tomorrow)
Is it related to when the OTP instance is started or is it some internal optimization? I don't think it's related to the building of the graph because that was built a couple of days ago. I was looking into providing a day or datetime setting when initializing OTP but am unable to find that in the docs.
(I haven't tried messing with my system time yet, but that's also an option I'm not very fond of). Any ideas or comments are welcome. If necessary I will provide a reproducible sample tomorrow.
This problem was actually caused because of how I used req.setDateTime() in combination with req.setMaxTimeSec().
Basically, setMaxTimeSec() uses the date set by setDateTime() as a starting point, and defines a worstTime (aka the last possible time) to that date time + the maxTimeSec. However, if setDateTime() was not yet set when calling setMaxTimeSec(), the current date time is used instead. This will consequently cause problems when you happen to call setDateTime() AFTERWARDS. Example:
setMaxTimeSec(60*60) # Sets worst time to now + 1 hour
setDateTime(yesterday) # Sets departure time to yesterday
This example has a very long time window to search for solutions! Instead of only looking within an hour time, we are now looking in a window of 25 hours!
Anyway, a simple solution is to first call setDateTime(), and then setMaxTimeSec():
setDateTime(yesterday) # Sets departure time to yesterday
setMaxTimeSec(60*60) # Sets worst time to yesterday + 1 hour
Alternatively, if, for some reason, you can't switch these methods, you can always correct the setMaxTimeSec() with the time difference between now and your setDateTime()-value:
date = datetime.strptime('2019-01-08 21:00', '%Y-%m-%d %H:%M')
date_seconds = time.mktime(date.timetuple())
now_seconds = time.mktime(datetime.now().timetuple())
date_diff_seconds = int(round(date_seconds - now_seconds))
req.setMaxTimeSec(60*60 + date_diff_seconds)
req.setDateTime(date.year, date.month, date.day, date.hour, date.minute, 00)

How to get the runTime of an active task according to its GMT lanuchTime in Spark?

I want to get the runtime of an active task. In the REST API page of /applications/[app-id]/stages/[stage-id], I can get tasks info in detail.
enter image description here
You can see, the executorRunTime is 0 when a task is not completed. I think I can get the runTime according to launchTime. Suppose the launchTime is 2017-12-21T03:15:31.106GMT. I use the following code to compute the runTime.
val format = new SimpleDateFormat("yyyy-MM-dd'T'hh:mm:ss'.'sss'GMT'", Locale.ENGLISH)
format.setTimeZone(TimeZone.getTimeZone("GMT"))
val launchTime = format.parse("2017-12-21T03:15:31.106GMT").getTime
val runTime = Calendar.getInstance(TimeZone.getTimeZone("GMT")).getTimeInMillis -
launchTime
But I get a negative number. Is the time format wrong? Or what?
TL;DR
String launchTimeString = "2017-12-21T03:15:31.106GMT";
launchTimeString = launchTimeString.replaceFirst("GMT$", "Z");
long launchTime = Instant.parse(launchTimeString).toEpochMilli();
long runTime = System.currentTimeMillis() - launchTime;
(Sorry, I can write only Java 8 code, I will have to trust you to translate.)
java.time and ISO 8601
The date and time classes from Java 1.0 and 1.1 are long outdated, and SimpleDateFormat in particular notoriously troublesome. I recommend you stop using them and use java.time, the modern Java date and time API instead. It is so much nicer to work with.
Your launch time string is a bit funny. It resembles the ISO 8601 standard format with the characteristic T between the date and the time, but has a non-standard GMT in the end, where strings following the standard would usually have Z (or a positive or negative offset from UTC). Since the modern date and time classes parse ISO 8601 as their default, that is, without any explicit formatter, and since writing format pattern strings seems to be an endless source of bugs (it certainly isn’t just you), I found it tempting to modify your string to fit the standard and then parse it.
What was wrong in your format pattern?
There are two bugs in you format pattern string:
You want uppercase HH for hour of day. Lowercase hh is for hour within AM or PM, in the interval 1 through 12. With SimpleDateFormat this bug usually “just” means that an hour of 12 is understood as 00 (which would have given you a very long run time) (the modern DateTimeFormatter more eagerly tells you you have a bug if you try the same with that class).
While lowercase ss is correct for seconds, milliseconds are uppercase SSS. This must have been what hit you: 106 in you string was taken to be seconds rather than milliseconds, so if running your code before 03:16:46, you got a negative run time.
So both bugs boil down to: Format pattern strings are case sensitive, so you need to beware the correct case.
Links
Oracle tutorial: Date Time, explaining how to use java.time.
Wikipedia article on ISO 8601.

Resources