I am trying to scrape the data table from nasdaq: https://www.nasdaq.com/symbol/msft/interactive-chart?timeframe=5d
What I do is using python and selenium webdriver to click the table button(on top of the chart, with a little table logo) and then scrape.
submit = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, '#dataTableBtn')))
submit.click()
But it does not work.
Button html here:
<div id="dataTableBtn" class="btn hideSmallIR stx-collapsible" onclick="dataTableLoader()"><span>Data Table</span></div>
EC and By
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
The chart and the associated elements are within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("https://www.nasdaq.com/symbol/msft/interactive-chart?timeframe=5d")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[src*='edgar-chartiq']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.hideSmallIR#dataTableBtn>span"))).click()
Using XPATH:
driver.get("https://www.nasdaq.com/symbol/msft/interactive-chart?timeframe=5d")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[contains(#src, 'edgar-chartiq')]")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='btn hideSmallIR stx-collapsible' and #id='dataTableBtn']/span[text()='Data Table']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
Here you can find a relevant discussion on Ways to deal with #document under iframe
The table is in an iframe, so just switch to it!
frame = driver.find_element_by_css_selector('#chartholder > iframe')
driver.switch_to.frame(frame)
You can use WebDriverWait with EC like this:
frame = driver.find_element_by_css_selector('#chartholder > iframe')
wait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it(frame))
The button you're looking for belongs to an iframe so you will have to switch to it prior to attempting to find the button.
Both iframe and the button can be located using XPath contains() function
driver.switch_to.frame(driver.find_element_by_xpath("//iframe[contains(#src,'edgar-chartiq')]"))
Once you're in the iframe you should be able to locate and click the button:
driver.find_element_by_xpath("//*[contains(text(),'Data Table')]").click()
Related
<a id="hlExcel" href="Reports\DC_REG_638123088090882623.xls" target="_blank"><img src="Images/Excel.gif" alt="Click to view a Excel version"></a>
How do I find and click on this excel file that I have to download everyday, given that the file name changes every day. I have tried this:-
link = browser.find_element_by_css_selector('["href="Reports"]')
link.click()
I am using python Selenium
Given the HTML:
<a id="hlExcel" href="Reports\DC_REG_638123088090882623.xls" target="_blank">
<img src="Images/Excel.gif" alt="Click to view a Excel version">
</a>
To click on the clickable element even though the file name changes every day ideally you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a#hlExcel > img[alt='Click to view a Excel version']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#id='hlExcel']/img[#alt='Click to view a Excel version']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You could go for a partial match maybe. Is the DC_REG common in all file names ? If you know a pattern which will be part of the name irrespective of how many times the name changes, you could look for that pattern. Something like
link = browser.find_element_by_css_selector('a[href*="REG"][href$=".xls"]')
link.click()
I'm currently working on a project with Selenium and bs4. So far so good I can navigate through the website using selenium without any problem however, I would need to click on a specific icon that opens a new tab with some information that I would need to scrape using BS4. My issue is when looking at the HTML I have no idea what to locate with Selenium in order to click on that icon.
HTML sample:
<a onmouseover="XYZ.Util.showHoverText(this, 'Print view')" href=
"app/exports/timesheet_print.php?startDate=2020-09-07&endDate=2020-09-11&filter_user=110483" target="_blank">
<img src="images/icons/printer.png" class="icon">
</a>
Usually I'm locating items by name, ID or Class but in that case I don't know what to chose from.
Should I look for the xpath instead ?
To click on the icon you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href^='app/exports/timesheet_print']>img.icon[src^='images/icons/printer']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[starts-with(#href, 'app/exports/timesheet_print')]/img[#class='icon' and starts-with(#src, 'images/icons/printer')]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I'm currently trying to select a radio button on the site linked in the code. I've tried finding it by the xpath, and by the ID, but both routes fail, giving me the 'Unable to locate element error'. So I was wondering if someone where could tell me what I'm doing wrong.
driver = webdriver.Chrome(executable_path="/Users/MrPete/Downloads/chromedriver_win32/chromedriver")
driver.get('https://www.pals.pa.gov/#/page/search')
radio = driver.find_element_by_id('optionsRadios1')
radio.click()
This is the radio button I'm trying to click
Induce WebDriverWait() and visibility_of_element_located() and following xpath.
Use Java scripts executor to click on the element.
code:
driver.get('https://www.pals.pa.gov/#/page/search')
radio=WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//input[#id='optionsRadios1']/following::span[1]")))
driver.execute_script("arguments[0].click();", radio)
Import following libraries.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser snapshot.
I have the following website that i would like to automate:
https://www.phptravels.net/
i would like to click on "My Account" to reach "register" and "login".
currently, i am doing it manually by navigating:
https://www.phptravels.net/demo/login
https://www.phptravels.net/demo/register
I am using python 3.7 for this task
I have tried locate it using Xpath, with no luck. it cannot find the items, nor click them.
[![<i class="icon_set_1_icon-70 go-right"></i> My Account <b class="lightcaret mt-2 go-left"></b>][1]][1]
please see attached image for that
To login in to Costco.com through the url https://www.costcobusinessdelivery.com/LogonForm?URL=%2f as the the desired elements are within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use the following solution:
css_selector:
driver.get("https://www.phptravels.net/")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe.full-screen-preview__frame")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "nav li#li_myaccount>a"))).click()
xpath:
driver.get("https://www.phptravels.net/")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#class='full-screen-preview__frame']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//nav//li[#id='li_myaccount']/a"))).click()
Browser Snapshot:
The link My Account is available inside an iframe name called preview-frame
To access the element you need to switch the iframe first.
Induce WebDriverWait and frame_to_be_available_and_switch_to_it()
Induce WebDriverWait and element_to_be_clickable()
Try below code.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
driver=webdriver.Chrome()
driver.get("https://www.phptravels.net/")
#Switch to Iframe first
WebDriverWait(driver,15).until(EC.frame_to_be_available_and_switch_to_it((By.NAME,"preview-frame")))
#Click on My Account
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//div[#class='container']//li[#id='li_myaccount']/a[contains(.,'My Account')]"))).click()
#To Click on Login
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//div[#class='container']//ul[#class='dropdown-menu']//a[contains(.,'Login')]"))).click()
To click on Sign UP you need to add following code.
#To Click on Sign Up
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//div[#class='container']//ul[#class='dropdown-menu']//a[contains(.,'Sign Up')]"))).click()
Browser Snapshot:
Danny
How to select an option from drop-down menu?
WebDriver provides three ways to select an option from the drop-down menu.
selectByIndex use
dropdown.selectByIndex(5);
selectByValue
dropdown.selectByValue("Database");
selectByVisibleText
dropdown.selectByVisibleText("Database Testing");
Hope to help na kub.
Here you have to click on my account first and then click on either login or signup by finding 2 xpath i.e one for my account, second for login and signup. I have no idea about python but here is the xpath:
1) Myaccount : (.//li[#id='li_myaccount'])[2]/a/b
2) Login : (.//ul[#class='dropdown-menu']/li/a)[3] or
signup : (.//ul[#class='dropdown-menu']/li/a)[4]
If you know how to use List<> you can get the list for login and signup option and click on it using index.
Hope it works for you. Thank you.
Currently I am trying to click the "entry-level" link in an iframe, for this website:https://a127-jobs.nyc.gov/index_new.html?category=CAS
Unfortunately, the url does not change if I click entry level, so therefore I am forced to automate.
I am not sure what to do, if the problem is that I have entered the wrong iframe? there seems to be only one, but I could be wrong.
from selenium import webdriver
import time
#webdriver
driver = webdriver.Firefox(executable_path="/Users/alexandrubordei/Desktop/geckodriver")
#get the website
driver.get ("https://a127-jobs.nyc.gov/index_new.html?category=CAS")
time.sleep(10)
#switch to iframe
iframes = driver.find_elements_by_tag_name("iframe")
driver.switch_to.frame(iframes[0])
time.sleep(10)
#click element "entry level"
driver.find_element_by_xpath('//*[#id="ti_S13"]').click()
When I run my code everything seems to function, except the link is not clicked. My search results are not narrowed.
However I do not get any errors. I get an error as:
Process finished with exit code 0
you have to click the a element inside that ID
driver.find_element_by_xpath('//*[#id="ti_S13"]/a').click()
To click() on the element with text as entry-level as the the desired elements are within an <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("https://a127-jobs.nyc.gov/index_new.html?category=CAS")
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='ptifrmtgtframe' and #name='TargetContent']")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "Entry-Level"))).click()
Browser Snapshot:
Here you can find a relevant discussion on Ways to deal with #document under iframe