Unable to convert DataFrame column to date time format.
from datetime import datetime
Holidays = pd.DataFrame({'Date':['2016-01-01','2016-01-06','2016-02-09','2016-02-10','2016-03-20'], 'Expenditure':[907.2,907.3,904.8,914.6,917.3]})
Holidays['Date'] = pd.to_datetime(Holidays['Date'])
type(Holidays['Date'])
Output: pandas.core.series.Series
Also tried
Holidays['Date'] = Holidays['Date'].astype('datetime64[ns]')
type(Holidays['Date'])
But same output
Output: pandas.core.series.Series
I think you are getting a bit mixed up. The dtypes of Holidays['Date'] is datetime64[ns]
Here's how I am checking.
from datetime import datetime
import pandas as pd
Holidays = pd.DataFrame({'Date':['2016-01-01','2016-01-06','2016-02-09','2016-02-10','2016-03-20'], 'Expenditure':[907.2,907.3,904.8,914.6,917.3]})
print ('Before converting : ' , Holidays['Date'].dtypes)
Holidays['Date'] = pd.to_datetime(Holidays['Date'])
print ('After converting : ' ,Holidays['Date'].dtypes)
The output is:
Before converting : object
After converting : datetime64[ns]
Thought I will also share some addition information for you around types and dtypes. See more info in this link for types-and-dtypes
Related
I have a column in my pandas data frame which is string and want to convert it to pandas date so that I will be able to sort
import pandas as pd
dat = pd.DataFrame({'col' : ['202101', '202212']})
dat['col'].astype('datetime64[ns]')
However this generates error. Could you please help to find the correct way to perform this
I think this code should work.
dat['date'] = pd.to_datetime(dat['col'], format= "%Y%m")
dat['date'] = dat['date'].dt.to_period('M')
dat.sort_values(by = 'date')
If you want to replace the sorted dataframe add in brackets inplace = True.
Your code didn't work because your wrong format to date. If you would have date in format for example 20210131 yyyy-mm-dd. This code would be enought.
dat['date'] = pd.to_datetime(dat['col'], format= "%Y%m%d")
I am trying to convert a pandas dataframe containing date in YYYYMM format to YYYYQ format as below
import pandas as pd
dat = pd.DataFrame({'date' : ['200612']})
pd.PeriodIndex(pd.to_datetime(dat.date), freq='Q')
However this generates output as 2012Q2, whereas correct output should be 2006Q4
What is the right way to get correct Quarter?
Explicitly specific the input format:
dat = pd.DataFrame({'date' : ['200612']})
pd.PeriodIndex(pd.to_datetime(dat.date, format='%Y%m'), freq='Q')
Output:
PeriodIndex(['2006Q4'], dtype='period[Q-DEC]', name='date')
I have pandas data frame that contains Month and Year values in a yyyy-mm format. I am using pd.to_sql to set the data type value to sent it to .db file.
I keep getting error:
sqlalchemy.exc.StatementError: (builtins.TypeError) SQLite Date type only accepts Python date objects as input.
Is there a way to set 'Date' Data type for 'MonthYear' (yyyy-mm) column? Or it should be set in a VARCHAR? I tried changing it to different types pandas's datetime data type, none of them seem to work.
I don't have any issues with 'full_date', it assigns it properly. Data type for 'full_date' is datetime64[ns] in pandas.
MonthYear full_date
2015-03 2012-03-11
2015-04 2013-08-19
2010-12 2012-06-29
2012-01 2018-01-01
df.to_sql('MY_TABLE', con=some_connection,
dtype={'MonthYear':sqlalchemy.types.Date(),
'full_date':sqlalchemy.types.Date()})
My opinion is that you shouldn't store unnecessarily the extra column in your database when you can derive it from the 'full_date' column.
One issue you'll run into is that SQLite doesn't have a DATE type. So, you need to parse the dates upon extraction with your query. Full example:
import datetime as dt
import numpy as np
import pandas as pd
import sqlite3
# I'm using datetime64[ns] because that's what you say you have
df = pd.DataFrame({'full_date': [np.datetime64('2012-03-11')]})
con = sqlite3.connect(":memory:")
df.to_sql("MY_TABLE", con, index=False)
new_df = pd.read_sql_query("SELECT * FROM MY_TABLE;", con,
parse_dates={'full_date':'%Y-%m-%d'})
Result:
In [111]: new_df['YearMonth'] = new_df['full_date'].dt.strftime('%Y-%m')
In [112]: new_df
Out[112]:
full_date YearMonth
0 2012-03-11 2012-03
I am reading in some excel data that contains datetime values stored as '8/13/2019 4:51:00 AM' and formatted as '4:51:00 AM' in excel. I would like to have a data frame that converts the value to a timestamp formatted as '4:51 AM' or H%:M% p%.
I have tried using datetime strptime but I don't believe I have been using it correctly. None of my attempts have worked so I have left it out of the code below. The two columns I would like to convert are 'In Punch' and 'Out Punch'
import pandas as pd
import pymssql
import numpy as np
import xlrd
import os
from datetime import datetime as dt
rpt = xlrd.open_workbook('OpenReport.xls', logfile=open(os.devnull,'w'))
rpt = pd.read_excel(rpt, skiprows=7)[['ID','Employee','Date/Time','In Punch','Out Punch',
'In Punch Comment','Out Punch Comment', 'Totaled Amount']]
rpt
Any suggestions will be greatly appreciated. Thanks
EDIT:
Working with the following modifications now.
rpt['In Punch'] = pd.to_datetime(rpt['In Punch']).dt.strftime('%I:%M %p')
rpt['Out Punch'] = pd.to_datetime(rpt['Out Punch']).dt.strftime('%I:%M %p')
Try working with datetime inside pandas. Convert Pandas Column to DateTime has some good suggestions that could help you out.
rpt['In Punch'] = pd.to_datetime(rpt['In Punch'])
Then you can do all sorts of lovely tweaks to a datetime. https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.to_datetime.html
I'm importing a csv files which contain a datetime column, after importing the csv, my data frame will contain the Dat column which type is pandas.Series, I need to have another column that will contain the weekday:
import pandas as pd
from datetime import datetime
data =
pd.read_csv("C:/Users/HP/Desktop/Fichiers/Proj/CONSOMMATION_1h.csv")
print(data.head())
all the data are okay, but when I do the following:
data['WDay'] = pd.to_datetime(data['Date'])
print(type(data['WDay']))
# the output is
<class 'pandas.core.series.Series'>
the data is not converted to datetime, so I can't get the weekday.
Problem is you need dt.weekday with .dt:
data['WDay'] = data['WDay'].dt.weekday
Without dt is used for DataetimeIndex (not in your case) - DatetimeIndex.weekday:
data['WDay'] = data.index.weekday
use the command data.dtypes to check the type of the columns.