I have list of US Holidays:
Holidays = ["January 01, 2019","January 21, 2019","February 14, 2019","February 18, 2019","April 19, 2019","April 21, 2019","May 12, 2019","May 27, 2019","June 16, 2019","July 04, 2019","September 02, 2019","October 14, 2019","October 31, 2019","November 11, 2019","November 28, 2019","December 25, 2019"]
I am converting it to datetinme stamp:
for i in Holidays:
print(datetime.strptime(i, "%B %d, %Y").date())
I am trying to append it to the another table but i am not getting the expected result
DATE = [datetime.strptime(i, "%B %d, %Y") for i in Holidays]
I assume that you're after a string value of the datetime object in your DATE list; based on the example for loop in the question.
Here's how to solve that:
date = [str(datetime.strptime(holiday, "%B %d, %Y").date()) for holiday in holidays]
print(date)
All I've done is change your variable names, add .date() to remove the time and convert it to a string.
Hope that helps
Check this.
from datetime import datetime
import pandas as pd
Holidays = ["January 01, 2019","January 21, 2019","February 14, 2019","February 18, 2019","April 19, 2019","April 21, 2019","May 12, 2019","May 27, 2019","June 16, 2019","July 04, 2019","September 02, 2019","October 14, 2019","October 31, 2019","November 11, 2019","November 28, 2019","December 25, 2019"]
converted_holidays = []
for i in Holidays:
converted_holidays.append(datetime.strptime(i, "%B %d, %Y").date())
data = {'Name': ["Person1","Person2","Person3","Person4","Person5","Person6","Person7","Person8","Person9","Person10","Person11","Person12","Person13","Person14","Person15","Person16"]}
df = pd.DataFrame(data)
#insert the holidays in dataframe using insert()
df.insert(0, "Holidays",converted_holidays , True)
print(df)
Related
I'm a newbie in creating Python Lambda function and I'm trying to figure out how to convert all the timestamps inside my JSON data into a date time format (2021-08-31T11:20:42.264+08:00). Sample data below:
{'Id': 'jr_e10ba9a2ab867b3abb3b1f8955cfd1815e1ce00cc981230702dfad45fb49a4b9',
'Attempt': 0,
'JobName': 'my_job_name',
'StartedOn': datetime.datetime(2021, 8, 31, 3, 17, 34, 49000, tzinfo=tzlocal()),
'LastModifiedOn': datetime.datetime(2021, 8, 31, 3, 19, 34, 855000, tzinfo=tzlocal()),
'CompletedOn': datetime.datetime(2021, 8, 31, 3, 19, 34, 855000, tzinfo=tzlocal()),
'JobRunState': 'FAILED'}
If you are writing your dictionary to a .json file, you can make your own serializer:
import json
...
def default_serialize(val):
if isinstance(val, datetime.datetime):
ret = val.isoformat()
else:
ret = str(val)
return ret
with open("my_data.json", "w") as fp:
json.dump(dct, fp, indent=4, default=default_serialize)
# Add empty line at EOF
fp.write("\n")
datetime luckily has a .isoformat function which will convert a datetime.datetime object to an isoformat, which is close to the one you want above.
You could just iterate through each key/value, and rewrite each key like:
my_json = {'Id': 'jr_e10ba9a2ab867b3abb3b1f8955cfd1815e1ce00cc981230702dfad45fb49a4b9',
'Attempt': 0,
'JobName': 'my_job_name',
'StartedOn': datetime.datetime(2021, 8, 31, 3, 17, 34, 49000, tzinfo=tzlocal()),
'LastModifiedOn': datetime.datetime(2021, 8, 31, 3, 19, 34, 855000, tzinfo=tzlocal()),
'CompletedOn': datetime.datetime(2021, 8, 31, 3, 19, 34, 855000, tzinfo=tzlocal()),
'JobRunState': 'FAILED'}
for key, value in my_json.items():
if isinstance(value, datetime.datetime):
my_json[key] = value.isoformat()
This iterates through the key and value of the json, checks if it is a datetime object, then converts it to the specified isoformat.
Please keep in mind that you need to load the JSON if you're loading it from an external file to do this, as JSON parsing in Python will lead to a dictionary, which is what I'm assuming you have here based off of what you gave above.
I have this function:
def function(start_date_arrow=None,end_date_arrow=None, date_concept=None):
list=[getattr(date, date_concept) for date in arrow.Arrow.range(date_concept, start_date_arrow, end_date_arrow)]
This function works well when iterating over date_concept='month' and date_concept='day'. On the other hand, date_concept='year' only returns a list of one item.
For example:
start_date_arrow= arrow.get('2021-11-05')
end_date_arrow= arrow.get('2022-02-05')
year_list=function(start_date_arrow=start_date_arrow,end_date_arrow=end_date_arrow, date_concept='year')
year_list is [2021]
month_list=function(start_date_arrow=start_date_arrow,end_date_arrow=end_date_arrow, date_concept='month')
month_list is [11, 12, 1, 2]
day_list=function(start_date_arrow=start_date_arrow,end_date_arrow=end_date_arrow, date_concept='day')
day_list is [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]
Second and third call are okei, but first one should return [2021,2022] instead of [2021].
Any idea of what is happening in the year call?
Found the issue.
If you use:
start_date_arrow= arrow.get('2021-11-05')
end_date_arrow= arrow.get('2022-02-05')
Year difference between both is less than 1, so it only returns the first one, so to return 2022 in the list end_date_arrow should be end_date_arrow= arrow.get('2022-11-05')
So I forced with an if statement the end date to be bigger just by one year, to force the return of both years.
I'm trying to convert timezone aware datetime object to UTC and then back to it's original timezone. I have a following snippet
t = datetime(
2013, 11, 22, hour=11, minute=0,
tzinfo=pytz.timezone('Europe/Warsaw')
)
now in ipython:
In [18]: t
Out[18]: datetime.datetime(
2013, 11, 22, 11, 0, tzinfo=<DstTzInfo 'Europe/Warsaw' WMT+1:24:00 STD>
)
and now let's try to do conversion to UTC and back. I would expect to have the same representation as:
In [19]: t.astimezone(pytz.utc).astimezone(pytz.timezone('Europe/Warsaw'))
Out[19]: datetime.datetime(
2013, 11, 22, 10, 36, tzinfo=<DstTzInfo 'Europe/Warsaw' CET+1:00:00 STD>
)
Yet we see that Out[18] and Out[19] differ. What's going on?
The documentation http://pytz.sourceforge.net/ states "Unfortunately using the tzinfo argument of the standard datetime constructors 'does not work' with pytz for many timezones." The code:
t = datetime(
2013, 5, 11, hour=11, minute=0,
tzinfo=pytz.timezone('Europe/Warsaw')
)
doesn't work according to this, instead you should use the localize method:
t = pytz.timezone('Europe/Warsaw').localize(
datetime(2013, 5, 11, hour=11, minute=0))
I want to write a code that prints the first and last number within a list as well as 5 other points. The points have to be equidistant from each other as well. So the numbers outputted would be 25, 19, 29, 16, 20.
list_ = [25, 23, 14, 22, 19, 13, 12, 10, 28, 29, 11, 15, 18, 27, 16, 21, 20, 17, 24, 26]
Is this something you are looking for？
>>> size = len(lst)
>>> size // 5 # five elements starting with first one.
4
>>>
>>> for i in range(0, size, size//5):
print(lst[i])
25
19
28
18
20
I am trying to retrieve the list of available option expiries for a given ticker on yahoo finance.
For instance using SPY as ticker on https://finance.yahoo.com/quote/SPY/options
The list of expiries are in the drop down list:
<div class="Fl(start) Pend(18px) option-contract-control drop-down-selector" data-reactid="4">
<select class="Fz(s)" data-reactid="5">
<option selected="" value="1576627200" data-reactid="6">December 18, 2019</option>
<option value="1576800000" data-reactid="7">December 20, 2019</option>
<option value="1577059200" data-reactid="8">December 23, 2019</option>
...
< / select >
< / div >
Using the div class name (or the select class name, but there seems to be several of these on the page), I get the list of values as a single string of concatenated expiries.
My function (I pass on ticker='SPY' from the main function):
def get_list_expiries(ticker):
browser = webdriver.Chrome()
options_url = "https://finance.yahoo.com/quote/" + str(ticker) + "/options"
browser.get(options_url)
html_source = browser.page_source
soup = BeautifulSoup(html_source, 'html.parser')
expiries_dt = []
for exp in soup.find_all(class_="Fl(start) Pend(18px) option-contract-control drop-down-selector"):
expiries_dt.append(exp.text)
browser.quit()
return expiries_dt
This produces:
['December 18, 2019December 20, 2019December 23, 2019December 24, 2019December 27, 2019December 30, 2019...']
I understand I need to use selenium for this but I can't figure out how. The result is always a list of a single string. Ideally I would like to return two lists: one with the unix datestamp (option value="1576627200") and another list with the 'normal' dates (ie 18/12/2019).
Any help will be greatly appreciated.
To extract the unix datestamp and Expiry Dates you have to induce WebDriverWait and you can use the following Locator Strategies:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(options=options, executable_path=r'C:\WebDrivers\chromedriver.exe')
driver.get('https://finance.yahoo.com/quote/SPY/options')
select = Select(WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.option-contract-control.drop-down-selector>select"))))
print("Unix datestamp: ")
print([option.get_attribute("value") for option in select.options])
print("Dates: ")
print([option.get_attribute("innerHTML") for option in select.options])
Console Output:
Unix datestamp:
['1576627200', '1576800000', '1577059200', '1577145600', '1577404800', '1577664000', '1577750400', '1578009600', '1578268800', '1578441600', '1578614400', '1578873600', '1579046400', '1579219200', '1579564800', '1579824000', '1580428800', '1582243200', '1584662400', '1585612800', '1587081600', '1589500800', '1592524800', '1593475200', '1594944000', '1600387200', '1601424000', '1602806400', '1605830400', '1606780800', '1608249600', '1610668800', '1616112000', '1623974400', '1631836800', '1639699200', '1642723200']
Dates:
['December 18, 2019', 'December 20, 2019', 'December 23, 2019', 'December 24, 2019', 'December 27, 2019', 'December 30, 2019', 'December 31, 2019', 'January 3, 2020', 'January 6, 2020', 'January 8, 2020', 'January 10, 2020', 'January 13, 2020', 'January 15, 2020', 'January 17, 2020', 'January 21, 2020', 'January 24, 2020', 'January 31, 2020', 'February 21, 2020', 'March 20, 2020', 'March 31, 2020', 'April 17, 2020', 'May 15, 2020', 'June 19, 2020', 'June 30, 2020', 'July 17, 2020', 'September 18, 2020', 'September 30, 2020', 'October 16, 2020', 'November 20, 2020', 'December 1, 2020', 'December 18, 2020', 'January 15, 2021', 'March 19, 2021', 'June 18, 2021', 'September 17, 2021', 'December 17, 2021', 'January 21, 2022']
try use SimplifiedDoc, It's a library for extraction
from simplified_scrapy.simplified_doc import SimplifiedDoc
html='''<div class="Fl(start) Pend(18px) option-contract-control drop-down-selector" data-reactid="4">
<select class="Fz(s)" data-reactid="5">
<option selected="" value="1576627200" data-reactid="6">December 18, 2019</option>
<option value="1576800000" data-reactid="7">December 20, 2019</option>
<option value="1577059200" data-reactid="8">December 23, 2019</option>
...
</select>
</div>
'''
doc = SimplifiedDoc(html)
div = doc.getElementByClass('Fl(start) Pend(18px) option-contract-control drop-down-selector')
options = div.options # get all options
expiries_dt = [option.html for option in options]
print (expiries_dt) # ['December 18, 2019', 'December 20, 2019', 'December 23, 2019']
You don't need selenium for this bit at least (and to be honest for most Yahoo finance info it is overkill). You can regex out timestamps from response text (converting string representation of list returned to actual list with ast) and use datetime module to convert to required date format.
import requests, re, ast
from datetime import datetime
r = requests.get('https://finance.yahoo.com/quote/SPY/options?guccounter=1')
p = re.compile(r'"expirationDates":(\[.*?\])')
timestamps = ast.literal_eval(p.findall(r.text)[0])
dates = [datetime.utcfromtimestamp(ts).strftime("%B %d, %Y") for ts in timestamps]
Regex explanation:
Datetime conversions:
See discussion by #jfs which is where I saw utcfromtimestamp originally
strftime