This question already has an answer here:
Not able to convert "00:30 AM" to 24 hours by strptime python
(1 answer)
Closed 5 years ago.
I am trying to create date time functions using the following code:
d1 = datetime.strptime('1/1/1960 0:00 AM', '%m/%d/%Y %I:%M %p')
d2 = datetime.strptime('1/1/2000 0:00 AM', '%m/%d/%Y %I:%M %p')
I get the following error:
ValueError: time data '1/1/1960 0:00 AM' does not match format
'%m/%d/%Y %I:%M %p'
I would appreciate help with this as I have tried tweaking the parameters to no avail.
0:00 AM doesn't match %I:%M %p, because %I doesn't include the hour 0, just 1 to 12 like on an analogue clock face:
%I Hour (12-hour clock) as a zero-padded decimal number. 01, 02, ..., 12
Midnight in a 12-hour clock is 12:00 AM:
>>> datetime.strptime('1/1/1960 12:00 AM', '%m/%d/%Y %I:%M %p')
datetime.datetime(1960, 1, 1, 0, 0)
Related
I am exploring different date formats and trying to convert date formats to others. Currently, I m stuck in a scenario where I have input dates and times as below:
I was able to convert it to a date timestamp using concatenation
concat_ws(' ',new_df.transaction_date,new_df.Transaction_Time)
While I m trying to use
withColumn("date_time2", F.to_date(col('date_time'), "MMM d yyyy hh:mmaa")) with ('spark.sql.legacy.timeParserPolicy','LEGACY')
It is displayed as 'undefined'
I am looking for pointers/code snippets to extract YYYY-MM-DD HH:MM:SS in CET (input is in PST) as below
input_date_time
output (in CET)
Mar 1, 2022 01:00:00 PM PST
2022-03-01 22:00:00
Parse PST string to timestamp with timezone in UTC. Then convert to "CET" time:
import pyspark.sql.functions as F
df = spark.createDataFrame(data=[["Mar 1, 2022 01:00:00 PM PST"]], schema=["input_date_time_pst"])
df = df.withColumn("input_date_time_pst", F.to_timestamp("input_date_time_pst", format="MMM d, yyyy hh:mm:ss a z"))
df = df.withColumn("output_cet", F.from_utc_timestamp("input_date_time_pst", "CET"))
[Out]:
+-------------------+-------------------+
|input_date_time_pst|output_cet |
+-------------------+-------------------+
|2022-03-01 21:00:00|2022-03-01 22:00:00|
+-------------------+-------------------+
Note - The 2022-03-01 21:00:00 above is Mar 1, 2022 01:00:00 PM PST displayed in UTC.
I have 2 columns with different date formats. In every row string dates are formatted differently.
I want to convert the columns to Date type. However, I am wondering if there is any built in method that will do the parsing for me:
What I tried
from datetime import datetime
newFrame = newDF.assign(Effective_Date=newDF['Effective_Date'].apply(lambda element: datetime.strptime(element,'%b %d %Y %H %M %S')), Paid_Off_Time=newDF['Paid_Off_Time'].apply(lambda element: datetime.strptime(element,'%b %d %Y %H %M %S')))
error when I run code above
line 359, in _strptime
(data_string, format))
ValueError: time data '09/08/2016' does not match format '%b %d %Y %H %M %S'
Example Date formats in .csv:
10/07/2016
10/07/2016 09:00
Data
newDF=pd.DataFrame({'Effective_Date':['10/07/2016','10/07/2016 09:00','09 August 2016'],'Paid_Off_Time':['10 July 2016','10/08/2016','10/09/2016 01:00:30']})
Effective_Date Paid_Off_Time
0 10/07/2016 10 July 2016
1 10/07/2016 09:00 10/08/2016
2 09 August 2016 10/09/2016 01:00:30
Solution
newDF.assign(Effective_Date=pd.to_datetime(newDF['Effective_Date']).dt.date,Paid_Off_Time=pd.to_datetime(newDF['Paid_Off_Time']).dt.date)
Effective_Date Paid_Off_Time
0 2016-10-07 2016-07-10
1 2016-10-07 2016-10-08
2 2016-08-09 2016-10-09
Checkout the Pandas documentation on Data Functionality for more details:
https://pandas.pydata.org/docs/user_guide/timeseries.html#converting-to-timestamps
I want to convert the following 12 hour time format to 24 time format using Azure Kusto language. I would expect the output to be converted from 07:00:00 AM to 07:00:00 and 07:00:00 PM to 19:00:00. Executing the below query is not resulting into correct output, sure i am missing something. Can anyone help please.
datatable (Date:string, Event:string)
['07:00:00 AM', "Morning",
'07:00:00 PM', "Evening"]
| extend val = todatetime(Date), val2 = format_datetime(todatetime(strcat('1900-01-01, ',Date)),'HH:mm:ss')
a. if you want to create a datetime value, you could try something like this:
parse the parts out of the string (hours, minutes, ...).
adjust the hours part to be in 24-hour format
create a datetime value out of those, using make_datetime(), or using datetime / timespan arithmetic.
for example:
datatable (TimeOfDay:string, Event:string)
[
'07:00:01 AM', "Morning",
'07:01:23 PM', "Evening"
]
| parse TimeOfDay with hours:int ":" minutes:int ":" seconds:int " " am_pm
| extend hours = case(am_pm == "AM" and hours == 12, hours - 12,
am_pm == "AM", hours,
hours == 12, hours,
hours + 12)
| project dt = make_datetime(1970, 1, 1, hours, minutes, seconds) // just used 1970-01-01 for the example
b. otherwise, if you just want to create a string value, you could do something somewhat similar:
datatable (TimeOfDay:string, Event:string)
[
'07:00:01 AM', "Morning",
'07:01:23 PM', "Evening"
]
| parse TimeOfDay with hours:int ":" minutes_seconds " " am_pm
| extend hours = case(am_pm == "AM" and hours == 12, tostring(hours),
am_pm == "AM", strcat("0", hours),
hours == 12, tostring(hours),
tostring(hours + 12))
| project str = strcat(hours, ":", minutes_seconds)
I have a DataFrame column that has string values for date/time (Input data). I need to convert it into a semi-timestamp format (Desired output data). There are rows that are blank and need to remain blank. I use quotes for illustrative purposes. I am using strptime but getting an error (see below).
Input data (String):
Mar 8 12:00 PM 2020
' '
Mar 8 1:00 PM 2020
Mar 8 6:00 PM 2020
Mar 9 8:00 AM 2020
Desired output data:
3/8/2020 12:00:00
' '
3/8/2020 13:00:00
3/8/2020 18:00:00
3/9/2020 08:00:00
Code:
import datetime as dt
df['date'].apply(lambda x: dt.datetime.strptime(x, '%b %d %H:%M %p %Y'))
Error:
ValueError: time data '' does not match format '%b %d %H:%M %p %Y'
How can I rewrite this code to get the desired output?
For me working to_datetime with format similar like yoour with %I for select hours in 12H format, also is added errors='coerce' for missing values (NaT) if some value not matching:
df['date'] = pd.to_datetime(df['date'], format='%b %d %I:%M %p %Y', errors='coerce')
print (df)
date
0 2020-03-08 12:00:00
1 NaT
2 2020-03-08 13:00:00
3 2020-03-08 18:00:00
4 2020-03-09 08:00:00
Last for custom format use Series.dt.strftime with Series.replace:
df['date'] = (pd.to_datetime(df['date'], format='%b %d %I:%M %p %Y', errors='coerce')
.dt.strftime('%m/%d/%y %H:%M:%S')
.replace('NaT', ''))
print (df)
date
0 03/08/20 12:00:00
1
2 03/08/20 13:00:00
3 03/08/20 18:00:00
4 03/09/20 08:00:00
Or replace multiple spoaces to one space:
df['date'] = (pd.to_datetime(df['date'].replace('\s+', ' ', regex=True), format='%b %d %I:%M %p %Y', errors='coerce')
.dt.strftime('%m/%d/%y %H:%M:%S')
.replace('NaT', ''))
print (df)
date
0 03/08/20 12:00:00
1
2 03/08/20 13:00:00
3 03/08/20 18:00:00
4 03/09/20 08:00:00
I have an output "Wed Mar 1 00:00:00 2000". I want to convert this into the format '08/11/2019 05:45PM'. How to achieve this format?
You could use something like below
import datetime
datetime.datetime.strptime('Wed Mar 1 00:00:00 2000', '%a %b %d %H:%M:%S %Y').strftime('%d/%m/%Y %I:%M%p')