What I need to do is convert 'year-month-day' timestamp to Unix time stamp do somethings with it then change it back to date time series. I am working with '1999-09-07' as my timestamp. I am getting an error : invalid literal for int() with base 10: '1999-09-07'
df1['timestamp'] = df1['timestamp'].astype(np.int64) // 10**9
#Got back this
ERROR:invalid literal for int() with base 10: '1999-09-07'
df1 = pd.read_csv('stock_CSV/' + ticker + '.csv')
pd.to_datetime(df1['timestamp'],unit='ns', origin='unix')
df1['timestamp'] = df1['timestamp'].astype(np.int64) // 10**9
#
#....some code
#
pd.to_datetime(df1['timestamp'], unit='s')
What I am expecting is a my dates converted to unix timestamp then converted back
Calling astype('int64') on a Timestamp Series return the number of seconds even though the Timestamps may have a resolution of nanosecond.
You can do it the old way, by counting the seconds from Jan 1, 1970 to the timestamps:
# random epoch times, in nanoseconds
t = np.random.randint(1e9 * 1e9, 2e9 * 1e9, 10)
# convert to Timestamps
dates = pd.to_datetime(t, unit='ns')
# convert back to nanoseconds
epoch = (dates - pd.Timestamp('1970-01-01')).total_seconds() * 1e9
# verify that we did the conversion correctly
assert np.isclose(t, epoch).all()
Related
I have a pandas dataframe which has a column called timeElapsed in seconds.
I take input from user to get a specific timestamp.
I want to add this specific timestamp with the timeElapsed column value.
For example:
user enters: 2021-07-08 10:00:00.0000
First entry in timeElapsed column is 80.1234.
New Column should be 2021-07-08 10:01:20.1234
so far, this is my code
import time
import pandas as pd
from datetime import datetime
df1 = pd.DataFrame({'userData': [1, 2, 3, 4, 5, 6, 7],
'timeElapsed': [0, 1.6427, 2.5185,5.3293,6.6699,37.4221,67.4378]})
takeDateInput = str(datetime.strptime(input("Enter current timestamp: YYYY-MM-DD HH:MM:SS.MS"),'%Y-%m-%d %H:%M:%S.%f'))
def myfunc2(x):
time.gmtime(x)
print(df1['timeElapsed'].apply(myfunc2))
I am trying to convert the seconds value to get a formatted hh:mm:ss timestamp using the myfun2. But I am not able to convert it. Is this the current approach?
Any direction as to how to achieve my final goal, would be much appreciated. Thank you
The timeElapsed value you're trying to add is best represented with a Timedelta. Keep the inputted timestamp as Datetime object (not a string), then you can just add the seconds as Timedelta:
takeDateInput = datetime.strptime(input("Enter current timestamp: YYYY-MM-DD HH:MM:SS.MS"),'%Y-%m-%d %H:%M:%S.%f')
def myfunc2(x):
return takeDateInput + pd.Timedelta(x, unit='sec')
print(df1['timeElapsed'].apply(myfunc2))
I want to subtract two dates and convert the results to milliseconds, I can subtract two dates but not sure how to convert to milliseconds, for example the final output of below code is '0:11:27.581293', I want to convert this to unit in milliseconds, example 12400ms like that, please help.
>>> import dateutil.parser as dparser
>>> stime='2019-04-23 04:22:50.421406'
>>> etime='2019-04-23 04:34:18.002699'
>>> str((dparser.parse(etime, fuzzy=True) - dparser.parse(stime, fuzzy=True)))
'0:11:27.581293'
Expected results: convert '0:11:27.581293' to milliseconds.
Use total_seconds() * 1000
Ex:
import dateutil.parser as dparser
stime='2019-04-23 04:22:50.421406'
etime='2019-04-23 04:34:18.002699'
print((dparser.parse(etime, fuzzy=True) - dparser.parse(stime, fuzzy=True)).total_seconds() * 1000)
#or
print(int((dparser.parse(etime, fuzzy=True) - dparser.parse(stime, fuzzy=True)).total_seconds()* 1000))
Output:
687581.293
687581
Use the code below which returns the difference in time in microseconds. Divide by 1000 to get to milliseconds.
diff = dparser.parse(etime, fuzzy=True) - dparser.parse(stime, fuzzy=True)
print(diff.microseconds)
I have a column of a user's DOB data in datetime64[ns] format, and would like to calculate their current age. Every time I parse this date and try to subtract the same with the present date, it throws me an error of str and datetime data format invalidity.
from datetime import datetime
main_file['AD_DOB'] = pd.to_datetime(main_file['AD_DOB']).dt.date ##10/2/1943
now = datetime.now().strftime("%H:%M:%S")
main_file['Age'] = ((now - main_file['AD_DOB'])/365).dt.days
Error: TypeError: unsupported operand type(s) for -: 'str' and 'datetime.date'
Try this
from datetime import datetime
main_file['AD_DOB'] = pd.to_datetime(main_file['AD_DOB']).dt.date ##10/2/1943
now = datetime.datetime.now()
main_file['Age'] = ((now - main_file['AD_DOB'])/365).dt.days
It's because that datetime.now().strftime("%H:%M:%S") returns a string object
Below should work:
now = datetime.now()
main_file['Age'] = (now - main_file['AD_DOB']).days/365.0
For subtract in pandas is necessary convert values to datetimes, if want remove times use Series.dt.floor for AD_DOB column and also for now is used Timestamp.floor:
main_file = pd.DataFrame({'AD_DOB':['10/2/1943','10/8/1946','10/12/1983']})
main_file['AD_DOB'] = pd.to_datetime(main_file['AD_DOB']).dt.floor('d')
now = pd.to_datetime('now').floor('d')
main_file['Age'] = ((now - main_file['AD_DOB'])/365).dt.days
print (main_file)
AD_DOB Age
0 1943-10-02 76
1 1946-10-08 73
2 1983-10-12 36
How can I convert date from seconds to date format.
I have a table containing information about lat, long and time.
table
f_table['dt'] = pd.to_datetime(f_table['dt'])
f_table["dt"]
it results like this:
output
but the output is wrong actually the date is 20160628 but it converted to 1970.
My desired output:
24-April-2014
The unit needs to be nanoseconds, so you need to multiply with 1e9
f_table['dt'] = pd.to_datetime(f_table['dt'] * 1e9)
This should work.
#Split your string to extract timestamp, I am assuming a single space between each float
op = "28.359062 69.693673 5.204486e+08"
ts = float(op.split()[2])
from datetime import datetime
#Timestamp to datetime object
dt = datetime.fromtimestamp(ts)
#Datetime object to string
dt_str = dt.strftime('%m-%B-%Y')
print(dt_str)
#06-June-1986
I get the following error:
ValueError: unconverted data remains: 918+00:00
When I try to convert a datetime like: 2018-11-20 14:31:33.799661918+00:00
v_time=[]
for i in range (1,len(db)+1):
p = (datetime.strptime(db.Time[i], "%Y-%m-%d%H:%M:%S.%f").strftime("%Y-
%m-%d %H:%M:%S"))
v_time.append(p)
db.Time = v_time
Since you anyway only display in seconds precision later, you could either strip simply the nanoseconds and live with microseconds precision (the nanoseconds might be not meaningful anyway)
from datetime import datetime
time_0 = '2018-11-20 14:31:33.799661918+00:00'
time_0_cut = time_0[0:26]
print(datetime.strptime(time_0_cut,
'%Y-%m-%d %H:%M:%S.%f').strftime('%Y-%m-%d %H:%M:%S'))
gives
2018-11-20 14:31:33
or, if you have to keep the nanoseconds precision, use numpy.datetime which is able to convert datetimes down to attoseconds precision
import numpy as np
print(np.datetime_as_string(np.datetime64(time_0)))
print(np.datetime_as_string(np.datetime64(time_0), unit='s')) # display in sec precision
gives
2018-11-20T14:31:33.799661918
2018-11-20T14:31:33