I converted the datetime from a format '2018-06-22T09:38:00.000-04:00'
to pandas datetime format
i tried to convert using pandas and got output but the output is
o/p: 2018-06-22 09:38:00-04:00
date = '2018-06-22T09:38:00.000-04:00'
dt = pd.to_datetime(date)
expected result: 2018-06-22 09:38
actual result: 2018-06-22 09:38:00-04:00
There is timestamps with timezones, so if convert to UTC by Timestamp.tz_convert, times are changed:
date = '2018-06-22T09:38:00.000-04:00'
dt = pd.to_datetime(date).tz_convert(None)
print (dt)
2018-06-22 13:38:00
So possible solution is remove last 6 values in datetimes:
dt = pd.to_datetime(date[:-6])
print (dt)
2018-06-22 09:38:00
Related
Pandas by default represent dates with datetime64 [ns], so I have in my columns this format [2016-02-05 00:00:00] but I just want the date 2016-02-05, so I applied this code for a few columns:
df3a['MA'] = pd.to_datetime(df3a['MA'])
df3a['BA'] = pd.to_datetime(df3a['BA'])
df3a['FF'] = pd.to_datetime(df3a['FF'])
df3a['JJ'] = pd.to_datetime(df3a['JJ'])
.....
but it gives me as result this error: TypeError: type unhashable: 'numpy.ndarray'
my question is: why i got this error and how do i convert datetime to date for multiple columns (around 50)?
i will be grateful for your help
One way to achieve what you'd like is with a DatetimeIndex. I've first created an Example DataFrame with 'date' and 'values' columns and tried from there on to reproduce the error you've got.
import pandas as pd
import numpy as np
# Example DataFrame with a DatetimeIndex (dti)
dti = pd.date_range('2020-12-01','2020-12-17') # dates from first of december up to date
values = np.random.choice(range(1, 101), len(dti)) # random values between 1 and 100
df = pd.DataFrame({'date':dti,'values':values}, index=range(len(dti)))
print(df.head())
>>> date values
0 2020-12-01 85
1 2020-12-02 100
2 2020-12-03 96
3 2020-12-04 40
4 2020-12-05 27
In the example, just the dates are already shown without the time in the 'date' column, I guess since it is a DatetimeIndex.
What I haven't tested but might can work for you is:
# Your dataframe
df3a['MA'] = pd.DatetimeIndex(df3a['MA'])
...
# automated transform for all columns (if all columns are datetimes!)
for label in df3a.columns:
df3a[label] = pd.DatetimeIndex(df3a[label])
Use DataFrame.apply:
cols = ['MA', 'BA', 'FF', 'JJ']
df3a[cols] = df3a[cols].apply(pd.to_datetime)
How to extract month and year from a string in DateTime stamp format?
sales_close_7 = 2018-12-07
import datetime
sales_close_7 = '2018-12-07'
date_object = datetime.datetime.strptime(sales_close_7, '%Y-%m-%d').date()
print(date_object.year)
Output: 2018
print(date_object.month)
Output: 12
My problem:
I have a datetime columns, with formats like
'27SEP18:05:02:11'
When trying to convert the datetime values I started like
df['dtimes'] = pd.to_datetime(df['dtimes'],format = '%d%b%Y:%H:%M:%S')
and ran into the problem that 'SEP' is not of the form 'Sep'. Surely I would not like to loop these columns.
Any speed code suggestions, please!?
Use %y for match year in format YY, %Y is used for YYYY format:
#YY format of year - %y
df = pd.DataFrame({'dtimes':['27SEP18:05:02:11','27JAN18:05:02:11']})
df['dtimes'] = pd.to_datetime(df['dtimes'],format = '%d%b%y:%H:%M:%S')
print (df)
dtimes
0 2018-09-27 05:02:11
1 2018-01-27 05:02:11
#YYYY format of year - %Y
df = pd.DataFrame({'dtimes':['27SEP2018:05:02:11','27JAN2018:05:02:11']})
df['dtimes'] = pd.to_datetime(df['dtimes'],format = '%d%b%Y:%H:%M:%S')
print (df)
dtimes
0 2018-09-27 05:02:11
1 2018-01-27 05:02:11
I have two date variables, and I tried to compute the difference in days between them with:
from datetime import date, timedelta,datetime
date_format = "%Y/%m/%d"
a = datetime.strptime(df.D1, date_format)
b = datetime.strptime(df.D2, date_format)
df['delta'] = b - a
print delta.days
But I'm getting this error:
TypeError: strptime() argument 1 must be str, not Series
How could I do this? The variables are objects, should I transform them in Datatime64?
Since you're working with pandas, you can use pd.to_datetime instead of the datetime package:
# Convert each date column to datetime:
df['D1'] = pd.to_datetime(df.D1,format='%Y/%m/%d')
df['D2'] = pd.to_datetime(df.D2,format='%Y/%m/%d')
# With 2 datetime Series, a simple subtraction will give you a Timedelta column:
df['delta'] = df.D1 - df.D2
For example:
>>> df
D1 D2
0 2015/05/18 2014/06/21
1 2015/10/18 2014/08/14
df['D1'] = pd.to_datetime(df.D1,format='%Y/%m/%d')
df['D2'] = pd.to_datetime(df.D2,format='%Y/%m/%d')
df['delta'] = df.D1 - df.D2
>>> df
D1 D2 delta
0 2015/05/18 2014/06/21 331 days
1 2015/10/18 2014/08/14 430 days
When I try to convert from number format to Date I'm not getting the same result what I get in Excel.
I need to convert a Number to date format and get the same result what I get in Excel.
For Example in Excel for the below Number I get the following:
Input - 42970.73819
Output- 8/23/2017 17:43
I tried using the date conversion in Pandas but not getting the same result as of Excel.
Thank you
Madan
I think you need convert serial date:
df = pd.DataFrame({'date':[42970.73819,42970.73819]})
print (df)
date
0 42970.73819
1 42970.73819
df = pd.to_datetime((df['date'] - 25569) * 86400.0, unit='s')
print (df)
0 2017-08-23 17:42:59.616
1 2017-08-23 17:42:59.616
Name: date, dtype: datetime64[ns]