I have an output "Wed Mar 1 00:00:00 2000". I want to convert this into the format '08/11/2019 05:45PM'. How to achieve this format?
You could use something like below
import datetime
datetime.datetime.strptime('Wed Mar 1 00:00:00 2000', '%a %b %d %H:%M:%S %Y').strftime('%d/%m/%Y %I:%M%p')
Related
I am exploring different date formats and trying to convert date formats to others. Currently, I m stuck in a scenario where I have input dates and times as below:
I was able to convert it to a date timestamp using concatenation
concat_ws(' ',new_df.transaction_date,new_df.Transaction_Time)
While I m trying to use
withColumn("date_time2", F.to_date(col('date_time'), "MMM d yyyy hh:mmaa")) with ('spark.sql.legacy.timeParserPolicy','LEGACY')
It is displayed as 'undefined'
I am looking for pointers/code snippets to extract YYYY-MM-DD HH:MM:SS in CET (input is in PST) as below
input_date_time
output (in CET)
Mar 1, 2022 01:00:00 PM PST
2022-03-01 22:00:00
Parse PST string to timestamp with timezone in UTC. Then convert to "CET" time:
import pyspark.sql.functions as F
df = spark.createDataFrame(data=[["Mar 1, 2022 01:00:00 PM PST"]], schema=["input_date_time_pst"])
df = df.withColumn("input_date_time_pst", F.to_timestamp("input_date_time_pst", format="MMM d, yyyy hh:mm:ss a z"))
df = df.withColumn("output_cet", F.from_utc_timestamp("input_date_time_pst", "CET"))
[Out]:
+-------------------+-------------------+
|input_date_time_pst|output_cet |
+-------------------+-------------------+
|2022-03-01 21:00:00|2022-03-01 22:00:00|
+-------------------+-------------------+
Note - The 2022-03-01 21:00:00 above is Mar 1, 2022 01:00:00 PM PST displayed in UTC.
I have 2 columns with different date formats. In every row string dates are formatted differently.
I want to convert the columns to Date type. However, I am wondering if there is any built in method that will do the parsing for me:
What I tried
from datetime import datetime
newFrame = newDF.assign(Effective_Date=newDF['Effective_Date'].apply(lambda element: datetime.strptime(element,'%b %d %Y %H %M %S')), Paid_Off_Time=newDF['Paid_Off_Time'].apply(lambda element: datetime.strptime(element,'%b %d %Y %H %M %S')))
error when I run code above
line 359, in _strptime
(data_string, format))
ValueError: time data '09/08/2016' does not match format '%b %d %Y %H %M %S'
Example Date formats in .csv:
10/07/2016
10/07/2016 09:00
Data
newDF=pd.DataFrame({'Effective_Date':['10/07/2016','10/07/2016 09:00','09 August 2016'],'Paid_Off_Time':['10 July 2016','10/08/2016','10/09/2016 01:00:30']})
Effective_Date Paid_Off_Time
0 10/07/2016 10 July 2016
1 10/07/2016 09:00 10/08/2016
2 09 August 2016 10/09/2016 01:00:30
Solution
newDF.assign(Effective_Date=pd.to_datetime(newDF['Effective_Date']).dt.date,Paid_Off_Time=pd.to_datetime(newDF['Paid_Off_Time']).dt.date)
Effective_Date Paid_Off_Time
0 2016-10-07 2016-07-10
1 2016-10-07 2016-10-08
2 2016-08-09 2016-10-09
Checkout the Pandas documentation on Data Functionality for more details:
https://pandas.pydata.org/docs/user_guide/timeseries.html#converting-to-timestamps
I am trying to convert this column of my dataframe 'Sat Feb 02 12:50:00 IST 2019' to regular datetime format ie(2019-05-02 12:00:00) in python
How do i convert all the rows to this format?
Assuming you don't need your datetime Python object to be timezone aware, you could just use strptime as follows:
dt = "Sat Feb 02 12:50:00 IST 2019"
out = datetime.strptime(dt, "%a %b %d %H:%M:%S IST %Y")
print(out)
This prints:
2019-02-02 12:50:00
My very smart android phone has produced emails where the date field is formatted with the german locale, e.g. Date: Di., 20 Dez. 2011 23:28:49 +0100. I switched the phone to english to stop it doing this but a number of emails have been written already.
Some other mail program can't handle this because it doesn't have all those locales installed. As a result it set the main date-time to Thu Jan 1 01:00:00 1970 and the emails appear to disappear when I sort by date :-(
So I want to fix those emails by rewriting the header. But how can I parse the german date? the date command supports locales on output (date +%c) but seems to ignore the locale on input:
$ date -d "08 Dec 2015"
Tue Dec 8 00:00:00 CET 2015
$ LC_TIME=de_DE.utf8 date -d "08 Dez 2015"
date: invalid date ‘08 Dez 2015’
Any ideas?
If you have Python and a de locale installed, one way would be to call
% python /path/to/convert.py "08 Dez 2015"
2015-12-08 00:00:00
where convert.py is this python script:
import sys, locale, datetime as DT
locale.setlocale(locale.LC_ALL, 'de_DE.utf8')
print(DT.datetime.strptime(sys.argv[1], '%d %b %Y'))
the date command of linux return the following date format
root#root:~# date
Sat Jun 14 06:36:42 CEST 2014
The current date time stamp could be printed if I add +%s
root#root:~# date +%s
1402720624
Now If I want to print the time stamp of the date returned by the date command, I get the following error
root#Inteno:~# date -d"Sat Jun 14 06:36:42 CEST 2014" +%s
date: invalid date 'Sat Jun 14 06:36:42 CEST 2014'
How I can make the date return the time stamp of the date format Sat Jun 14 06:36:42 CEST 2014 ?
Note: I m using date from BusyBox v1.19.4
Unfortunately busybox's date has limitations but if you can consider using timestamps instead you can do:
TS=$(date '+%s')
date -d "#${TS}"
Still if CEST is your current local time, you can do:'
DATE=$(date)
date -d "$(echo "$DATE" | cut -d ' ' -f 2,3,4,6)" '+%s'
As removing the timezone would still apply.