pd.to_datetime output is 1970-01-01 00:00:00.0000 - python-3.x

dt_corr = dt[["date","mean_press", "maxwind", "sun_dur","mean_rel_hum", "mean_temp", "mean_cloud", "max_temp_2m", ]].dropna()
dt_corr["year"] = dt_corr.date.dt.year
dt_corr.set_index("date")
dt_corr.index = pd.to_datetime(dt_corr.index, format='%Y/%m-%d') #####why appears format green here ????? i don't get it
dt_corr
The index in any output is always like this:
1970-01-01 00:00:00.000033417 ##last for digits differ
I cannot coerce my index to datetime, to finally reasample my data and plot it.
The format parameter doesn't change anything and appears in green in my codeline.
What can I do?

Related

How to format iso date in nodeJS

Here is my code
var newDateObj2 = moment("2020-04-29T14:05:00.000Z").format('YY/MM/DD,HH:mm:ss.S');
console.log(newDateObj2);
Output
20/04/29,19:35:00.0
Expected Output
20/04/29,14:05:00.0
Why 5.30 (h.mm) getting added in my final result, When i print
moment("2020-04-29T14:05:00.000Z")`=> Moment<2020-04-29T19:35:00+05:30>
How to get Output 20/04/29,14:05:00.0
You should trim the Z from the ISO date string or use the utc() function
moment("2020-04-29T14:06:00.000Z").utc().format('YY/MM/DD,HH:mm:ss.S');

To check if the continuity of dates are missing in a column

I want to check in my dataframe's column that if there is a missing date for a certain month then the code should output the following month in the format MMM- YYYY
The data set looks like this :
date_start_balance date_end_balance start_balance
22.02.16 22.03.16 3590838
22.04.16 22.05.16 69788
15.06.16 21.07.16 452165
Both date cols are in datetime format. Now in the above data set the dates are missing for March and May in the start col and this should be returned as MMM-YYYYY
I have tried the following code :
import datetime
dates = df1['date_start_balance'].tolist()
missing = []
for i in range(0,len(dates)-1):
if dates[i+1].month - dates[i+1].month != 1:
for j in range(dates[i].month+1,dates[i+1].month):
missing.append(datetime(dates[i].year, j,1))
print(missing)
You can first create a date range with pd.date_range
march = pd.date_range(start='2016-05-01', end='2016-05-31')
And then you will have the list with the dates that you already have, in the example there is only one date: 2016-05-15:
your_list = [datetime.datetime.strptime('15052016', "%d%m%Y").date()]
And then you can calculate the difference between the range and your list and get the dates that you are missing:
march.difference(your_list)
DatetimeIndex(['2016-05-01', '2016-05-02', '2016-05-03', '2016-05-04',
'2016-05-05', '2016-05-06', '2016-05-07', '2016-05-08',
'2016-05-09', '2016-05-10', '2016-05-11', '2016-05-12',
'2016-05-13', '2016-05-14', '2016-05-16', '2016-05-17',
'2016-05-18', '2016-05-19', '2016-05-20', '2016-05-21',
'2016-05-22', '2016-05-23', '2016-05-24', '2016-05-25',
'2016-05-26', '2016-05-27', '2016-05-28', '2016-05-29',
'2016-05-30', '2016-05-31'],
dtype='datetime64[ns]', freq=None)

Date of 7 days before today

I give current day in a string in MATLAB. For example if today is '20180703', I need 7 strings containing:
'20180702'
'20180701'
'20180630'
'20180629'
'20180628'
'20180627'
'20180626'
Simple:
t = datetime('20180703', 'InputFormat', 'yyyyMMdd')
t = t - days(1:7)
datestr(t, 'yyyymmdd')
Edit.
As excaza pointed out, datetime and datestr use different input format. Hence, 'MM' in the first function, and 'mm' in the second one.
I would go with something like:
lastSevenDays = arrayfun(#(offset) datestr(now-offset, 'yyyymmdd'), 1:7, 'UniformOutput', false)
or more matlaby:
datestr(now - days(1:7), 'yyyymmdd')

pd.to_datetime to solve '2010/1/1' rather than '2010/01/01'

I have a dataframe which contain a column 'trade_dt' like this
2009/12/1
2009/12/2
2009/12/3
2009/12/4
I got this problem
benchmark['trade_dt'] = pd.to_datetime(benchmark['trade_dt'], format='%Y-&m-%d')
ValueError: time data '2009/12/1' does not match format '%Y-&m-%d' (match)
how to solve it? Thanks~
Need change format for match - replace & and - to % and /:
benchmark['trade_dt'] = pd.to_datetime(benchmark['trade_dt'], format='%Y/%m/%d')
Also working with sample data removing format (but not sure with real data):
benchmark['trade_dt'] = pd.to_datetime(benchmark['trade_dt'])
print (benchmark)
trade_dt
0 2009-12-01
1 2009-12-02
2 2009-12-03
3 2009-12-04

Converting time format

There must be a quick solution for this, but after 30min I gave up and need help.
This is the format of source data
0h56m40s 0h57m10s 1h00m40s 1h02m15s 1h02m25s
52m47s 54m25s 54m52s 57m23s 57m43s
49m30s 54m31s 54m34s 56m35s 56m36s
47m45s 48m03s 51m02s 52m23s 53m05s
46m54s 49m29s 50m51s 51m02s 51m03s
46m09s 47m56s 50m16s 51m20s 51m53s
46m55s 47m08s 47m13s 48m16s 50m11s
and I need this in time format like 0h56m40s to 0:56:40
I tried search/replace, from h to :, m to : and removing s, works for when there's hour, but messes up for when only minutes are there.
Any tips?
You can concatenate 0: if the input string is too short:
=(IF(LEN(A1)<7,"0:","") & SUBSTITUTE(SUBSTITUTE(SUBSTITUTE(A1,"h",":"),"m",":"),"s",""))+0
The +0 part converts string to time value (change cell format to h:mm:ss). If you prefer to keep it in text format, remove +0.

Resources