How to convert any date format in milliseonds in linux/centos/rhel with shell script
Date formats like:
Thu, 23 Jun 2022 08:27:26 +0000,
Thu Jun 23 11:11:43 UTC 2022,
2022-06-23T08:28:23Z
This can be converted in milliseconds with command:
date -d "2022-06-23T08:28:23Z" +"%s%N"
Any other format of date can also be converted with above command like:
date -d "Thu, 23 Jun 2022 08:27:26 +0000" +"%s%N"
date -d "Thu Jun 23 11:11:43 UTC 2022" +"%s%N"
Given all possible time formats I can get from my user (chatbot):
09:03
9:23A.M.
9:23 A.m.
13:44 pm
20:00 P.m
15:40
00:00
12:33
4:33p.M
...
I want to convert them to appropriate time. There won't be one format I can use.
Please advise how to solve such issue?
I have tried with strptime from the datetime package but I need to specify format and it can be different every time.
Find my answer thanks to this amazing source!
Posting for others:
dateutil
The dateutil module is an extension to the datetime module. We don't need to pass any parsing code to parse a string. For example:
from dateutil.parser import parse
datetime = parse('2018-06-29 22:21:41')
print(datetime)
This parse function will parse the string automatically and store it in the datetime variable. Parsing is done automatically. You don't have to mention any format string. Let's try to parse different types of strings using dateutil:
from dateutil.parser import parse
date_array = [
'2018-06-29 08:15:27.243860',
'Jun 28 2018 7:40AM',
'Jun 28 2018 at 7:40AM',
'September 18, 2017, 22:19:55',
'Sun, 05/12/1999, 12:30PM',
'Mon, 21 March, 2015',
'2018-03-12T10:12:45Z',
'2018-06-29 17:08:00.586525+00:00',
'2018-06-29 17:08:00.586525+05:00',
'Tuesday , 6th September, 2017 at 4:30pm'
]
for date in date_array:
print('Parsing: ' + date)
dt = parse(date)
print(dt.date())
print(dt.time())
print(dt.tzinfo)
print('\n')
Output:
$ python3 dateutil-1.py
Parsing: 2018-06-29 08:15:27.243860
2018-06-29
08:15:27.243860
None
Parsing: Jun 28 2018 7:40AM
2018-06-28
07:40:00
None
Parsing: Jun 28 2018 at 7:40AM
2018-06-28
07:40:00
None
Parsing: September 18, 2017, 22:19:55
2017-09-18
22:19:55
None
Parsing: Sun, 05/12/1999, 12:30PM
1999-05-12
12:30:00
None
Parsing: Mon, 21 March, 2015
2015-03-21
00:00:00
None
Parsing: 2018-03-12T10:12:45Z
2018-03-12
10:12:45
tzutc()
Parsing: 2018-06-29 17:08:00.586525+00:00
2018-06-29
17:08:00.586525
tzutc()
Parsing: 2018-06-29 17:08:00.586525+05:00
2018-06-29
17:08:00.586525
tzoffset(None, 18000)
Parsing: Tuesday , 6th September, 2017 at 4:30pm
2017-09-06
16:30:00
None
You can see that almost any type of string can be parsed easily using the dateutil module.
Maya
Maya also makes it very easy to parse a string and for changing timezones. Some simple examples are shown below:
import maya
dt = maya.parse('2018-04-29T17:45:25Z').datetime()
print(dt.date())
print(dt.time())
print(dt.tzinfo)
Output:
$ python3 maya-1.py
2018-04-29
17:45:25
UTC
For converting the time to a different timezone:
import maya
dt = maya.parse('2018-04-29T17:45:25Z').datetime(to_timezone='America/New_York', naive=False)
print(dt.date())
print(dt.time())
print(dt.tzinfo)
Output:
$ python3 maya-2.py
2018-04-29
13:45:25
America/New_York
Now isn't that easy to use? Let's try out maya with the same set of strings we have used with dateutil:
import maya
date_array = [
'2018-06-29 08:15:27.243860',
'Jun 28 2018 7:40AM',
'Jun 28 2018 at 7:40AM',
'September 18, 2017, 22:19:55',
'Sun, 05/12/1999, 12:30PM',
'Mon, 21 March, 2015',
'2018-03-12T10:12:45Z',
'2018-06-29 17:08:00.586525+00:00',
'2018-06-29 17:08:00.586525+05:00',
'Tuesday , 6th September, 2017 at 4:30pm'
]
for date in date_array:
print('Parsing: ' + date)
dt = maya.parse(date).datetime()
print(dt)
print(dt.date())
print(dt.time())
print(dt.tzinfo)
Output:
$ python3 maya-3.py
Parsing: 2018-06-29 08:15:27.243860
2018-06-29 08:15:27.243860+00:00
2018-06-29
08:15:27.243860
UTC
Parsing: Jun 28 2018 7:40AM
2018-06-28 07:40:00+00:00
2018-06-28
07:40:00
UTC
Parsing: Jun 28 2018 at 7:40AM
2018-06-28 07:40:00+00:00
2018-06-28
07:40:00
UTC
Parsing: September 18, 2017, 22:19:55
2017-09-18 22:19:55+00:00
2017-09-18
22:19:55
UTC
Parsing: Sun, 05/12/1999, 12:30PM
1999-05-12 12:30:00+00:00
1999-05-12
12:30:00
UTC
Parsing: Mon, 21 March, 2015
2015-03-21 00:00:00+00:00
2015-03-21
00:00:00
UTC
Parsing: 2018-03-12T10:12:45Z
2018-03-12 10:12:45+00:00
2018-03-12
10:12:45
UTC
Parsing: 2018-06-29 17:08:00.586525+00:00
2018-06-29 17:08:00.586525+00:00
2018-06-29
17:08:00.586525
UTC
Parsing: 2018-06-29 17:08:00.586525+05:00
2018-06-29 12:08:00.586525+00:00
2018-06-29
12:08:00.586525
UTC
Parsing: Tuesday , 6th September, 2017 at 4:30pm
2017-09-06 16:30:00+00:00
2017-09-06
16:30:00
UTC
As you can see, all date formats were parsed, but did you notice the difference? If we are not providing the timezone info it automatically converts it to UTC. So, it is important to note that we need to provide to_timezone and naive parameters if the time is not in UTC.
Is it possible to convert a date/time in Excel such as Mon Nov 11 2019 22:12:22 UTC time to and EST date/time value? Essentially subtract 5 hours from it? Some of the formulas I've been playing with are:
=C2-5/24
=(SUBSTITUTE(LEFT(A1,27),"T"," "))+(MID(A1,28,3)/24)
I have a column of timestamp converted to human readable form.
I have tried to sort it from epochtime as well as after converting. It's giving me
Fri, 08 Feb 2019 17:24:16 IST
Mon, 11 Feb 2019 02:19:40 IST
Sat, 09 Feb 2019 00:22:43 IST
which is not sorted.
I have used sort_values()
each_tracker_df = each_tracker_df.sort_values(["timestamp"],ascending=True)
why it isn't working??
Since all the time is in IST. Replace the string IST with NULL.
>>import datetime
>>times=['Fri, 10 Feb 2010 17:24:16','Fri, 11 Feb 2010 17:24:16','Fri, 11 Feb 2019 17:24:16']
>>change_format=[]
>> for time in times:
change_format.append(datetime.datetime.strptime(time, '%a, %d %b %Y %H:%M:%S'))
>>change_format.sort()
My very smart android phone has produced emails where the date field is formatted with the german locale, e.g. Date: Di., 20 Dez. 2011 23:28:49 +0100. I switched the phone to english to stop it doing this but a number of emails have been written already.
Some other mail program can't handle this because it doesn't have all those locales installed. As a result it set the main date-time to Thu Jan 1 01:00:00 1970 and the emails appear to disappear when I sort by date :-(
So I want to fix those emails by rewriting the header. But how can I parse the german date? the date command supports locales on output (date +%c) but seems to ignore the locale on input:
$ date -d "08 Dec 2015"
Tue Dec 8 00:00:00 CET 2015
$ LC_TIME=de_DE.utf8 date -d "08 Dez 2015"
date: invalid date ‘08 Dez 2015’
Any ideas?
If you have Python and a de locale installed, one way would be to call
% python /path/to/convert.py "08 Dez 2015"
2015-12-08 00:00:00
where convert.py is this python script:
import sys, locale, datetime as DT
locale.setlocale(locale.LC_ALL, 'de_DE.utf8')
print(DT.datetime.strptime(sys.argv[1], '%d %b %Y'))