I have a pandas column with timestamp strings in the format '00:00:00.000' (hours, minutes, seconds, micro seconds). I would like to convert them to datetime objects to work on the seconds.
I have seen many similar questions here for example. I am guessing that I should use strptime but I couldn't figure out how.
If convert values to datetimes in pandas, also there is added some default date by to_datetime:
df['col'] = pd.to_datetime(df['col'], format='%H:%M:%S.%f')
If need avoid it convert values to timedeltas by to_timedelta:
df['col'] = pd.to_timedelta(df['col'])
Related
I have the following dataframe:
The column Time is a string and I want to convert it either to timestamp or datetime formats. However, when I run df['Time'] = pd.to_datetime(df['Time']), I always get an error
OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 1-01-01 08:53:30
Are you sure you are getting the right column and values. Because running
time = pd.to_datetime("13:30:35.805")
Gives
Timestamp('2020-04-20 13:30:35.805000')
as output as expected.
If you can't solve the problem with pandas directly you can always manually split the string in hours, minutes and seconds with
h, m, s = map(float, x.split(':'))
And use those values to create a timestamp
I have a column of times expressed as seconds since Jan 1, 1990, that I need to convert to a DateTime. I can figure out how to do this for a constant (e.g. add 10 seconds), but not a series or column.
I eventually tried writing a loop to do this one row at a time. (Probably not the right way, and I'm new to python).
This code works for a single row:
def addSecs(secs):
fulldate = datetime(1990,1,1)
fulldate = fulldate + timedelta(seconds=secs)
return fulldate
b= addSecs(intag112['outTags_1_2'].iloc[1])
print(b)
2018-06-20 01:05:13
Does anyone know an easy way to do this for a whole column in a dataframe?
I tried this:
for i in range(len(intag112)):
intag112['TransactionTime'].iloc[i]=addSecs(intag112['outTags_1_2'].iloc[i])
but it errored out.
If you want to do something with column (series) in DataFrame you can use apply method, for example:
import datetime
# New column 'datetime' is created from old 'seconds'
df['datetime'] = df['seconds'].apply(lambda x: datetime.datetime.fromtimestamp(x))
Check documentation for more examples. Overall advice - try to think in terms of vectors (or series) of values. Most operations in pandas can be done with entire series or even dataframe.
Pandas: convert series of time YYYY-MM-DD hh:mm:ss.0 keeping the YYYY-MM-DD format only
python 3.6, pandas 0.19.0
timestamp
0 2013-01-14 21:19:42.0
1 2013-01-16 09:04:37.0
2 2013-03-20 12:50:49.0
3 2013-01-03 17:02:53.0
4 2013-04-13 16:44:20.0
I tried:
df['timestamp'] = df['timestamp'].dt.strftime('%Y-%m-%d')
`AttributeError: Can only use .dt accessor with datetimelike values.`
Any thoughts? Thank you!
convert the series into datetime datatype and try,
df['timestamp'] = pd.to_datetime(df['timestamp']).dt.strftime('%Y-%m-%d')
it may satisfy your demand
df['timestamp'] = pd.to_datetime(df['timestamp']).dt.strftime('%Y-%m-%d')
Using the below shown method also helps to achieve the same.
df['timestamp'] = pd.to_datetime(df['timestamp']).dt.date
You can refer to the documentation provided in the below link as a handy guide for date time handling:
https://pandas.pydata.org/pandas-docs/stable/api.html#datetimelike-properties
I am trying to compare a pandas to_datetime object to another to_datetime object. In both locations, I am entering the date as date = pd.to_datetime('2017-01-03'), but when I run a print statement on each, in one case I get 2017-01-03, but in another I get 2017-01-03 00:00:00. This causes a problem because if I use an if statement comparing them such as if date1 == date2: they will not compare as equal, when in reality they are. Is there a format statement that I can use to force the to_datetime() command to yield the 2017-01-03 format?
You can use date() method to just select date from pandas timestamp and also use strftimeme(format) method to convert it into string with different formats.
date = pd.to_datetime('2017-01-03').date()
print(date)
>datetime.date(2017, 1, 3)
or
date = pd.to_datetime('2017-01-03').strftime("%Y-%m-%d")
print(date)
>'2017-01-03'
try .date()
pd.to_datetime('2017-01-03').date()
You can use
pd.to_datetime().date()
For example:
a='2017-12-24 22:44:09'
b='2017-12-24'
if pd.to_datetime(a).date() == pd.to_datetime(b).date():
print('perfect')
I have a column called construction_year as numerical(int) year. I want to convert it to dd-mm-yyyy format in python. I have tried with datetime and pandas to_datetim and converting time stamp extracting the format but in vain.
Ex: I have year like 2013(int) I would like to convert it as 01-01-2013 in python 3.x.
Into a string
If you want to convert it into a string, you can simply use:
convert_string = '01-01-{}'.format
and then use it like:
>>> convert_string(2013)
'01-01-2013'
Into a datetime
If you want to convert it to a datetime object, you can simply use:
from datetime import date
from functools import partial
convert_to_date = partial(date,month=1,day=1)
Now convert_to_date is a function that converts a numerical year into a date object:
>>> convert_to_date(2013)
datetime.date(2013, 1, 1)