I have a list of date objects X ("%Y,%m,%d") and a date Y and want to have a list Z with all dates in it that are +-10days away from date Y IGNORING the year.
Ex.:
timeDelta = 10days
X = [2017-10-10, 2014-09-31, 1999-05-10, 1992-10-18]
Y = 2019-10-05
Z = MyFunction(X, Y, delta)
MyFunction(X,Y, timeDelta):
for i in range((currentDate-500000),currentDate,10000): # last 50 years in 1 year steps
bottomBoundries.append(datetime.strptime(str(i),'%Y%m%d')-timedelta(timeDelta))
topBoundries.append(datetime.strptime(str(i),'%Y%m%d')+timedelta(timeDelta))
for i in range(0, pandas.shape[0], 1):
for j in range(0, len(bottomBoundries) , 1):
if ((pandas["MESS_DATUM"].iloc[i] > bottomBoundries[j]) & (pandas["MESS_DATUM"].iloc[i] < topBoundries[j])):
indices.append(i)
result = pandas.iloc[indices]
return result
This takes a lot of time and I was wondering how to make it more efficient.
you could use the timetuple().yday to get the day in the year
so something like this might work:
def diff_without_year(date, reference_date, timedelta):
ref_yday = reference_date.timetuple().yday
diff = d.timetuple().yday - ref_yday # + optionally code to take care of the difference in hours, minutes,...
return abs(diff) <= timedelta
here timedelta is in days. If you get it in a TimeDelta object, you can convert by dividing by pd.TimeDelta('1day')
df['result'] = df["MESS_DATUM"].apply(lambda x: diff_without_year(x, ref_date, timedelta))
If you have your dates in a pandas series, you can simplify this up by using Series.dt.dayofyear
df['result'] = (df["MESS_DATUM"].dt.dayofyear - reference_date.timetuple().yday) < timedelta
A more adhoc solution that checks for each date in X whether the date in the year of Y is within 10 days, as well as the date in the preceding and succeeding year. This runs in linear time in size of X. For a large list of X, you can adapt this to do it in a pandas dataframe, and parallelize this code.
import datetime
timeDelta = 10 # in days
X = ['2017-10-10', '2014-09-30', '1999-05-10', '1992-10-18']
Y = '2019-10-05'
Y_date = datetime.datetime.strptime(Y, '%Y-%m-%d') # convert to datetime
td = datetime.timedelta(timeDelta)
year = Y_date.year
output_dates = []
for date in X:
X_date = datetime.datetime.strptime(date, '%Y-%m-%d')
month = X_date.month
day = X_date.day
date_previous_year = datetime.datetime(year=year-1, month=month, day=day)
date_current_year = datetime.datetime(year=year, month=month, day=day)
date_next_year = datetime.datetime(year=year+1, month=month, day=day)
if abs(date_previous_year - Y_date) <= td or \
abs(date_current_year - Y_date) <= td or \
abs(date_next_year - Y_date) <= td:
output_dates.append(date)
print(output_dates)
Related
Say for the year of 2020, how do I iterate through the days in the months so that my outcome would be in the following format:
Jan1
Jan2
Jan3
....
Jan31
Feb1
I've tried so many things online but I couldnt find an answer. Please help :(
Both of these methods will handle leap years correctly out of the box.
Using a simple while loop:
from datetime import datetime, timedelta
def iter_days(year):
dt = datetime(year, 1, 1)
while dt.year == year:
yield dt
dt += timedelta(days=1)
Using date rules:
from datetime import datetime
from dateutil.rrule import rrule, DAILY
def iter_days(year):
first_date = datetime(year, 1, 1)
last_date = datetime(year, 12, 31)
return rrule(DAILY, dtstart=first_date, until=last_date)
Both would be used the same:
for dt in iter_days(2020):
print(dt.strftime('%b%-d'))
The format string '%b%-d' will give you the format you specified in your question. I don't know if that was a requirement or not.
This is crude but gets what you want for 2020. You'll need to change 366 to 365 for non-leap-years.
#!/usr/bin/python3
import datetime
startDate = '2020-01-01'
start = datetime.datetime.strptime(startDate, '%Y-%m-%d')
for dayNum in range(0,366):
dayOfYear = start + datetime.timedelta(days=dayNum)
print(dayOfYear.strftime('%b %d, %Y'))
The calendar module offers quite a bit of functionality.
Here is a solution that works for any given year
import calendar as cal
for mi in range(1,13):
_, days = cal.monthrange(2020, mi)
for d in range(1, days+1):
print(cal.month_name[mi], d)
I have two tables:
-event dates
-return dates
Some event dates are not at a trading day.
How can I change the event date to the next trading day?
So if event date is not in return dates, take the next day in return dates.
The approach to change weekend days to working days does not work because of days like Christmas.
The best would be to look up the next day in the return table.
for i in event['date']:
if i is not in return ['date'].values:
event ['date']=i+datetime.timedelta(days=1)
but this doenst work
I am working with dataframes and dates have the format datetime64[ns]. If the event date does not exist in return date than event date plus one day
Edit
After the clarifications concerning the desired logic, here is the new solution
from datetime import datetime, timedelta
import numpy as np
import pandas as pd
# Create two df
event_date = datetime.now()
event_dates = pd.DataFrame([datetime(2020, 2, _) for _ in range(1, 29)], columns=['date'])
print(event_dates.date[0])
# 2020-02-01 00:00:00
return_dates = pd.DataFrame([datetime(2020, 1, _) for _ in range(1, 32)], columns=['date'])
# Apply logic
event_dates.date = [_ if _ in return_dates.date else _ + timedelta(days=1) for _ in event_dates.date]
print(event_dates.date[0])
# 2020-02-02 00:00:00
Base Python
Here is a solution using the standard datetime library
from datetime import datetime
from typing import List
def get_next_trade_date(date: datetime, date_list: List[datetime]) -> datetime: # The annotations here are just to specify the types of the objects
if date in date_list: # Check if the date is contained in the list
return date
delta, res = None, None # Initialize both to None
for _ in date_list:
tmp = abs((date - _).days) # Time difference in current iteration
if not delta or tmp < delta: # See bullet point 1.
delta, res = tmp, _
return res
if __name__ == '__main__':
event_date = datetime.now()
return_dates = [datetime(2020, 1, _) for _ in range(1, 32)]
print(get_next_trade_date(event_date, return_dates))
# 2020-01-01 00:00:00
Notice that
The condition not delta or tmp < delta is twofold: in the first iteration delta, res are both None so we will overwrite them with tmp, _. We catch this by using not delta. The other part (tmp < delta) is more obvious: if we have a new minimal delta then we overwrite delta, res.
I only considered days intervals ((date - _).days), you could go further into details (see datetime.timedelta for more info)
coming from R I believe there must be a simpler solution using numpy - see below
Numpy
This solution uses numpy. (date_list - date) is an array of timedeltas, (date_list - date).argmin() returns the index of the minimal value.
from datetime import datetime
import numpy as np
def get_next_trade_date(date: datetime, date_list: np.ndarray) -> datetime:
return date_list[(date_list - date).argmin()]
if __name__ == '__main__':
event_date = datetime.now()
return_dates = np.array([datetime(2020, 1, _) for _ in range(1, 32)])
print(get_next_trade_date(event_date, return_dates))
# 2020-01-01 00:00:00
I am generating dates between 01-01-2010 and 31-01-2010 with a gap of 1 second as follows:
import datetime
dt = datetime.datetime(2010, 1, 1)
end = datetime.datetime(2010, 1, 31, 23, 59, 59)
step = datetime.timedelta(seconds=1)
result = []
while dt < end:
result.append(dt.strftime('%Y-%m-%d %H:%M:%S'))
dt += step
and the result that it gives it perfectly fine, but it takes about 10secs to give the result.
I was wondering if the same can be achieved using pandas so that it can be achieved a bit quickly
Use date_range with DatetimeIndex.strftime:
result = pd.date_range(dt, end, freq='S').strftime('%Y-%m-%d %H:%M:%S').tolist()
Or:
result = pd.date_range('2010-01-01',
'2010-01-30 23:59:59',freq='S').strftime('%Y-%m-%d %H:%M:%S').tolist()
try like below
l = (pd.DataFrame(columns=['NULL'],
index=pd.date_range('2010-01-01T00:00:00Z', '2010-01-31T00:00:00Z',
freq='1T'))
.index.strftime('%Y-%m-%dT%H:%M:%SZ')
.tolist()
)
print(l)
I am defining a function that is being applied to every row in my Data Frame that counts unique codes in a the column "Code" for every id in the set. The code I have works, but it is incredibly slow and I am using a large data set. I am looking for a different approach that speed up the operation.
from datetime import timedelta as td
import pandas as pd
df['Trailing_12M'] = df['Date'] - td(365) #current date - 1 year as new column
def Unique_Count(row):
"""Creating a new df for each id and returning unique count to every row in original df"""
temp1 = np.array(df['ID'] == row['ID'])
temp2 = np.array(df['Date'] <= row['Date'])
temp3 = np.array(df['Date'] >= row['Trailing_12M'])
temp4 = np.array(temp1 & temp2 & temp3)
df_Unique_Code_Count = np.array(df[temp4].Code.nunique())
return df_Unique_Code_Count
df['Unique_Code_Count'] = df.apply(Unique_Count, axis=1)
I'm trying to get two dates as input and convert is epoch time, but i need the two different dates given as input to be validated in correct format else recursively ask for correct input.
from datetime import date
import datetime
start_date = datetime.datetime.strptime(raw_input('Enter Start date in the format DD-MM-YYYY: '), '%d-%m-%Y')
end_date = datetime.datetime.strptime(raw_input('Enter Start date in the format DD-MM-YYYY: '), '%d-%m-%Y')
epoch_date = datetime.datetime(1970,1,1)
diff1 = (start_date - epoch_date).days
diff2 = (end_date - epoch_date).days
epoch1 = (diff1 * 86400)
epoch2 = (diff2 * 86400)
print('\nPTime_Start: %i' % diff1),
print("&"),
print('PTime_End: %i' % diff2)
print('Epoch_Start: %i' % epoch1),
print("&"),
print('Epoch_End: %i' % epoch2)
First of all, you are using Python 3.x and Python 3.x does not have any function that is called "raw_input()". It has been changed to "input()".
def take_date_input():
input_date = input('Enter date in the format DD-MM-YYYY: ')
try:
one_date = datetime.datetime.strptime(input_date, '%d-%m-%Y')
except ValueError:
return take_date_input()
return one_date
You can do this if you really want recursiveness in your code but it would be better with while loop.