I have some time series data from an API request and when I am doing some data wrangling this error pops up below. The data wrangling is just some simple Pandas series math (not shown).
TypeError: unsupported operand type(s) for -: 'str' and 'str'
But when I save the data to a CSV:
elecMeter_df.to_csv('C:\\Python Scripts\\elecMeter_df.csv', sep=',', header=True, index=True, na_rep='N/A')
And then parse the dates on a read_CSV:
elecMeter_dfCSV = pd.read_csv('C:\\Python Scripts\\elecMeter_df.csv', index_col='Date', parse_dates=True)
I do not get the original error described above.. Why is that? Am I getting the error because the time stamp is a string and I need to convert into an integer format?
When I get the error, the index is in this format:
print(elecMeter_df.index)
But when read the CSV file and Parse the date column (No error in the data wrangling processes, the index is in this format: (no Chicago Time zone reference)
print(elecMeter_df.index)
Any help/tips that can be explained to me about time stamps and why this error happens would be greatly appreciated. Utilimetely I am trying to not have to use the read/write CSV process, but if its the only method to not get any errors Ill just stick with that!
Not sure what code you are running to generate that error. However the time stamp probably needs to be converted from a string to a date time. Try using pd.to_datetime, additionally you can specify the format (list of options and meanings are provided below). The example I used for the format is year-month-day hour-minutes.
pd.to_datetime(df['column'], format = %Y-%m-%d %H:%M)
%a Locale’s abbreviated weekday name.
%A Locale’s full weekday name.
%b Locale’s abbreviated month name.
%B Locale’s full month name.
%c Locale’s appropriate date and time representation.
%d Day of the month as a decimal number [01,31].
%f Microsecond as a decimal number [0,999999], zero-padded on the left
%H Hour (24-hour clock) as a decimal number [00,23].
%I Hour (12-hour clock) as a decimal number [01,12].
%j Day of the year as a decimal number [001,366].
%m Month as a decimal number [01,12].
%M Minute as a decimal number [00,59].
%p Locale’s equivalent of either AM or PM.
%S Second as a decimal number [00,61].
%U Week number of the year (Sunday as the first day of the week)
%w Weekday as a decimal number [0(Sunday),6].
%W Week number of the year (Monday as the first day of the week)
%x Locale’s appropriate date representation.
%X Locale’s appropriate time representation.
%y Year without century as a decimal number [00,99].
%Y Year with century as a decimal number.
%z UTC offset in the form +HHMM or -HHMM.
%Z Time zone name (empty string if the object is naive).
%% A literal '%' character.
Related
I downloaded the data from tradingview which was exported csv format but the time format is unreadable. please see below.
1502942400
1502949600
1502956800
1502964000
1502971200
1502978400
1502985600
1502992800
1503000000
How can I change it to yyyy / mm / dd hh : mm format by excel ?
Grateful if you advise. Thanks a lot.
The numbers are in unix time format.
so just use this function on each number
put this function in a module .
Public Function fromUnix(ts) As String
fromUnix = Format(DateAdd("s", ts, "1/1/1970 00:00:00"), "yyyy / mm / dd mm:hh")
End Function
and use it like
=fromUnix ("1502949600") OR =fromUnix (Range("A1").value) ' or whatever cell.
credits >> the source for this function
Unix time stamps represent the number of seconds since 1 Jan 1970.
Excel stores date/time as days and fractions of a day since 1 Jan 1900.
There are 86400 seconds in a day (24*60*60)
So to convert:
=A1/86400+DATE(1970,1,1)
and format the result as you wish.
I want to covert a column with Excel date numbers (float) to datetime, below is the function I am trying to use:
import datetime
datetime.datetime.fromtimestamp(42029.0).strftime('%Y-%m-%d')
Bu the result I got is: '1970-01-01',I don't think it is right, I must missing a constant which should be added to the variable, because in the excel the number 42029.0 represents date: 1/25/2015.
Can anyone please advise how to improve the code?
datetime.fromtimestamp() follows the unix convention and expects seconds since '1970-01-01'. The Excel date number for '1970-01-01' is 25569, so you have to subtract this number from your given date number and multiply the result, which is in days, with 86400 (seconds per day):
datetime.datetime.fromtimestamp(
(42029.0 - 25569) * 86400).strftime('%Y-%m-%d')
'2015-01-25'
I am having this data and try to convert the string date to datetime format. and I have this time data does not match format error.
Date=stock['Trddt'].values
datetime.datetime.strptime(Date[0], "%y.%m.%d")
and it return this error:
ValueError: time data '2015/11/23' does not match format '%y.%m.%d'
Try datetime.datetime.strptime(Date[0], "%Y/%m/%d").
%y matches years without century, but you have years with century, so you need capital %Y.
I have a date that looks like this text. 17-OCT-16 03.24.11.000000000 AM
I need to format that text into a date that I can then manipulate. Ultimate I want to transform that time to millis from epoch but I'm not sure if that's even possible
Its definitely possible, you can make use of DATEVALUE and TIMEVALUE function which convert a time or date in string format to date/time format:
=DATEVALUE(LEFT(A1,9)) + TIMEVALUE(SUBSTITUTE(MID(A1,11,8), ".", ":")&MID(A1, 19, 9))
This will give you a single number (42660.1417939815) in this case which is the number of days since 1/1/1900 (including milliseconds)
It should just be some simple maths to get total milliseconds
I am trying to plot some data which has time (date) on the x-axis in the format 2015-12-20. I do not want to plot my data against the absolute date, but against the time difference between that date and another reference "zero" date. The reference date is in same format, eg: 2015-12-15.
Is it possible to have the data file in the format:
%Y-%m-%d Value
%Y-%m-%d Value
but have gnuplot create a plot in which the x-axis is number of days since a reference date?
This isn't ideal, and there probably is a better way, but this works (I assume that column 1 is your date column and that column 2 is your y-axis value):
plot "data.txt" u ((strptime("%Y-%m-%d",strcol(1))-strptime("%Y-%m-%d","2015-12-10"))/86400.0):2
The strptime function turns a time string to an internal time representation (the number of seconds since some reference date) and the strcol function reads the string from column 1. We take the difference from the "zero" date (here we use December 10th). This will give the difference in seconds, so we divide by the number of seconds in a day, 86400 (we use 86400.0 so that it doesn't truncate to integers in the division). This will make the x-axis the number of days since the reference date.